Sophistication in GenAI Use: Field Evidence from a Large Firm
cs.AI, econ.GN, q-fin.EC
Submitted: 2026-08-27
Updated: 2026-08-27
Comments: 59 pages, 4 figures, 8 tables. Includes Appendix A (variable definitions), Appendix B (supplementary tables), and Online Appendix C (meta prompts)
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: We study how sophistication in generative AI (genAI) use varies among the back-office workforce of a large firm.
Terminology
Abstract
We study how sophistication in generative AI (genAI) use varies among the back-office workforce of a large firm. Using proprietary data, we observe 713,564 employee prompts and their corresponding large language model responses from nearly 4,000 back-office employees across 15 functional areas over eight months in 2025. We document three main findings. First, senior employees exhibit more sophisticated genAI use, consistent with domain expertise complementing genAI capabilities. Second, sophistication varies considerably across functions and is highest in Strategy, Digital Innovation, and Project Management, three groups that share a focus on firmwide strategic initiatives and organizational change. Third, we observe neither improvements in sophistication over time nor lasting improvements following formal AI training, suggesting that sophisticated use can be difficult to change. Together, our study provides measures of and insights into sophisticated genAI use that managers can use to improve outcomes and that researchers can use in future research.
Sources
- AI in the Enterprise: How People Use M365 Copilot Chat
- Early Impacts of M365 Copilot
- Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations
- From Exposure to Adoption: Generative AI in European Workplaces
- The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
- Working with AI: Measuring the Applicability of Generative AI to Occupations
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection