A good manager is one whose teams keep beating the prediction
Team results mix the manager's contribution with the skills of the people they happened to get. The studies separate the two the way a causal analysis would. They predict what each team should score from its members' individually tested skills, which is the counterfactual of an average manager, and then measure how far each manager's teams land above or below that prediction, across several randomly assigned teams.
The procedure
Test everyone on their own first
Measure task-specific skill with an individual version of the team task (Study 1: numerical, spatial and analytical problems; Study 2: three solo hidden-profile puzzles, plus typing speed). Add fluid intelligence and any other candidate predictors you want to check later.
Rotate managers through random teams
Each manager leads several teams of randomly drawn members (4 in Study 1, 6 in Study 2), with as few repeat pairings as possible. Random assignment makes the manager effect causal: nobody gets to pick their team.
Predict every team's score from skills alone
Standardise team scores within each round, then regress them on the manager's and the members' task skill.
Gg = βM·skillmanager + βW·skillmembers + εgAverage each manager's surplus
A manager's effect is the mean residual of the teams they led. A multilevel model separates real differences between managers (σα) from noise, and randomization inference checks whether the spread exceeds what reshuffled assignments produce.
α̂i = mean(ε̂g for teams led by i); ε̂gi = αi + egi, αi ~ N(0, σα²)Optional: replace the human followers with AI agents
In Study 2, leaders also directed teams of LLM agents through the same kind of puzzle. Their AI-team scores tracked their causal effect on human teams closely, at a fraction of the cost and without coordinating groups of people.
One manager, four random teams
IllustrationEach team's predicted score comes from its members' skills. The manager effect is the average gap between what the team actually scored and that prediction.
How Do You Find a Good Manager?
Weidmann, Vecci, Said, Deming & Bhalotra. 555 participants, 728 three-person teams, Collaborative Production Task, 4 random rotations, pre-registered. Figures below are from the 2024 NBER working paper; the peer-reviewed version, How Do You Identify a Good Manager? (QJE 2026), adds co-authors and further evidence, so some numbers may differ.
Typical manager effect. A manager 1 SD above average raises team performance by 0.23 SD (randomization inference p = .04), about as much as raising both workers' production skill by 1 SD (0.26 SD). A good manager is worth almost twice a good worker.
Self-nominated managers did worse than managers picked by lottery (−0.10 to −0.13 SD, marginally significant). Self-promoters were more overconfident, especially about their social skills.
Economic decision-making and fluid IQ predicted manager performance (lottery arm). Gender, age, ethnicity and personality did not.
How good managers help: they monitor (half the wasted effort), assign people by comparative advantage, and keep teams motivated late in the task.
Measuring Human Leadership Skills with AI Agents
Weidmann, Xu & Deming. 249 leaders, hidden-profile puzzles, 6 human teams and 6 AI teams (GPT-4o followers) per leader, order counterbalanced, pre-registered.
AI-team scores matched human ground truth. Disattenuated correlation 0.81 (raw 0.67); 0.69 after controlling for hard skills.
Leader effect beyond hard skills, nearly identical in the AI and human versions (about 0.65 SD before skill controls).
Good leaders asked more questions and had more back-and-forth. Talking more did not help. Positive tone mattered only with human followers.
Per participant for the AI test, versus $114 and two supervising researchers for the human version.
Read the results with care
- Study 1 has been peer reviewed and published in the Quarterly Journal of Economics (2026). Study 2, the AI-agent study behind the leadership test, is still a working paper (under revise and resubmit at Management Science), so its results have not yet passed peer review. Both rest on lab tasks done mostly by students.
- The causal reading depends on random assignment. In everyday HR data, managers and teams are not matched at random, so the same calculation would be confounded.
- With four or six rotations, the estimate for any single manager is noisy. Study 1 therefore focused on the typical size of manager effects (σα) and on what predicts them. Use the reliability calculator before ranking individuals.
- The leadership test on this site follows Study 2's structure but uses new puzzles, other AI models, two or four team puzzles instead of six and its own scoring. It has not been validated against human teams and has no norms, so use it for practice or as a research tool with a group, not to judge an individual.
- Treat a score as one input among several, and check for adverse impact before using any test in selection.
A short summary of both papers: Luděk Stehlík, Novel way to measure leadership skills via causal inference (and AI)?
Plan the rotations
Assign every manager to a series of randomly drawn teams, keeping repeat pairings to a minimum, and decide how many rotations you need before trusting an individual estimate.
What you need to bring
This tab and the next are a toolkit for running your own experiment with real teams. They plan the rotations and analyse the results; they do not run the sessions or supply the task.
- A team task with an objective score, in one parallel version per round. Study 1 used its own Collaborative Production Task: numerical, spatial and analytical modules, with the team scored on its weakest module. Its materials are not included here.
- An individual version of that task, taken by everyone before the team sessions, to measure the skill the analysis controls for.
- People and sessions for several rounds of random teams. Study 1 had 555 participants, three-person teams and four rounds.
- A way to record results in the CSV format that the rotation generator and step 2 provide.
Without the capacity to run team sessions, the leadership test (first tab) is self-contained, but it is not a validated assessment.
How many rotations do you need?
Reliability of one manager's estimate after n random teams
Generate a random rotation
Paste IDs one per line, or separated by commas
Running it well
- Use a task with an objective score that needs real collaboration and gives the manager real work: assigning people, monitoring progress, keeping the team going. Study 1 scored teams on their weakest module, so nobody could carry the team alone.
- Measure individual skill before the team sessions, with an individual analogue of the team task. That measure is what lets the analysis separate leadership from the luck of the draw.
- Keep roles fixed and assignment random. Use a fresh, parallel version of the task in each round. The analysis standardises scores within rounds, so differences in difficulty between rounds wash out.
- Record the selection route (self-nominated, appointed, lottery) if you want to compare them, and collect candidate predictors such as decision-making tests or 360 ratings for the correlation analysis.
Estimate manager effects
Load one row per team per round, plus an optional file of individual skill scores. The analysis follows the pre-registered models of Study 1: conditioning regression, manager averages, a REML multilevel model with profile-likelihood intervals, randomization inference, leave-one-round-out validation and the correlates of good management.
Data
Columns: round, team_id, manager_id, member_ids, team_score (members separated by ;), plus optional design columns such as arm.
Columns: person_id, the task-skill scores to control for, and any candidate predictors (tests, ratings, demographics).