This page says how Trailhead is built and how it is measured, with the studies each decision rests on. It does not report results. When the first cohort has numbers, they appear below with the sample size, the pre/post on both transfer parts and the agreement coefficient, or they do not appear at all.
Computed from evidence the learner produced, predictive of transfer or transfer itself, and explainable in one sentence to the person who sees it. These findings set the bar.
Task success does not predict transfer.
In the invent-first study the inventing group did slightly worse on the learning task and better on transfer (Schwartz, Chase et al. 2011); invention success and transfer were uncorrelated, r = .01 (Chase et al. 2019). So lessons completed and flaws caught are progress, not outcomes.
Transfer must be measured without the coach.
A guardrailed AI still left students worse on a transfer task once access was removed (Lichand, SAL 2026). Every Try-it-new probe is taken with Scout switched off.
One transfer item is often too weak to detect transfer.
Invention beat tell-and-practice only when a learning resource was embedded in the test (Schwartz & Martin 2004; Bransford & Schwartz 1999). So transfer is measured two ways: a cold item and one with a resource to learn from.
Persistence after failure predicts learning.
Fail-abandon predicts the opposite (Chase 2011). Responding to pushback is an outcome; quitting after a refusal is the at-risk signal.
Teaching produces learning only from a faithful student.
The protégé effect holds when the agent reasons solely from what it was taught (Chase et al. 2009; Chin et al. 2010). Teach-back is scored only under a runner that adds nothing of its own.
Choices are a measure, not a lever.
Choosing critical feedback and revising predicted learning; a yoked design showed the choice itself is the signal (Cutumisu et al. 2016). Feedback is offered, never nudged.
Game features can hurt learning.
Points, streaks and leaderboards changed reactions to failure for the worse (Chase et al. 2021). None are shown as a score.
Products no longer evidence process.
AI detectors fall to 26 to 42 percent accuracy under paraphrase and are biased against non-native writers. The process trace and an oral defense authenticate work; no detector is used anywhere.
Compare, probe, learn, apply, teach, try it new. Scout implements the guidance stance shown to raise transfer, problematizing rather than explaining (Chase, Connolly, Lamnina & Aleven 2019), and structures every turn as attend, interpret, decide, generate, the expert chain that improved human tutoring in a preregistered trial (Wang, Ribeiro, Robinson, Loeb & Demszky 2024).
Preparation for future learning
Invention with contrasting cases and delayed telling (Schwartz & Bransford 1998; Schwartz, Chase, Oppezzo & Chin 2011; Sinha & Kapur 2021). Every module opens with two cases that differ on one deep feature; the lesson unlocks on evidence of generation, computed server-side, never on an attempt count.
Responsive teaching
A developmental map of learner strategies and professional noticing, in the form Cognitively Guided Instruction gave them (Carpenter et al. 1989; Jacobs, Lamb & Philipp 2010). Scout’s counter-case and the facilitator’s check-in are chosen from where each learner sits on the map.
Learning by teaching
A teachable agent that reasons only from what it is taught (Chase, Chin, Oppezzo & Schwartz 2009). The learner writes rules; the runner applies only those rules to a fresh case and reports where they fell silent; the learner fixes them and runs again.
Modules completed, steps met, minutes and attendance render as a rail, never as a number. Points, streaks, badges and an "AI proficiency percent" are not collected at all.
In schools the metrics go to the teacher. On teams they go to the organization that signed up, and to the sponsor for its own briefs. Reviewers see artifact scores and the trace, never learning telemetry. Learners see their own, in plain language, and never a ranking.
The 7.0 bar on the six dimensions (Work Quality, AI Tool Utilization, Problem-Solving, Communication, Responsible AI Practices, Collaboration) carries every control below, and the platform reports the coefficients rather than asserting them.
Blind review
Pseudonym and the submitted snapshot. No identity reaches a rater.
The process trace
The rule invented, the answer to the counter-case, the rules taught and fixed, the transfer attempts. Shown beside every artifact.
Oral defense
Ten minutes on one artifact the reviewer picks. Recorded on the form.
Two raters, adjudication
Weighted kappa per dimension and ICC(2,1) on the overall; a third rater when scores straddle 7.0 or sit more than a level apart. Never percent agreement.
Calibration and re-scoring
Every rater scores an anchor set blind before a panel and is calibrated at kappa of at least 0.6 against the consensus. A sample of past reviews is re-scored blind at the next panel.
Standardized tasks
Two fixed-prompt tasks per competency beside the brief-specific artifacts, because task sampling dominates score variance.
Each names its measure and the direction it predicts. The equity hypothesis predicts no difference, and matters as much as the directional ones. The first study is the pilot: feasibility, instrument validation and building the strategy map. Design tests follow with more cohorts.
Transfer rises from the module-1 baseline to the course-end battery, on both parts.
The embedded-resource part gains more than the cold part.
Invention quality before the lesson predicts transfer; task success on the artifact does not.
Responding to pushback predicts gain; fail-abandon predicts lower gain.
The post-fix lift under the rule-faithful runner predicts gain; under an unconstrained runner it does not.
Choices correlate with transfer and artifact quality independent of time on task.
Gains do not differ by gender, free-lunch status or first-generation status; if anything they are larger at lower baseline.
Results
The first cohort is the district pilot. When it has finished, this block will carry the number of learners, the pre/post change on both transfer parts, the rater agreement coefficient and the sponsor- scored before/after for the first team cohort. Until then the only claims on this site are about how the program is built.