Altamens

Altamens Ricerca · Reproducible methods note

One Session or Five? How Baseline Choice Changes a Progress Score

The same future score can look like an 8-point or 20-point gain. A worked example shows why the baseline rule belongs beside the result.

Altamens10 min di lettura

Sintesi

A progress score is a difference between a later result and a chosen reference. That reference might be the first session, the mean of several early sessions or a rolling baseline. Those rules can produce different changes from the same sequence.

This reproducible methods note uses six constructed scores. A future score of 105 is 20 points above the first session, 10 points above the first-three mean and 8 points above the first-five mean. An exact no-change model then shows how averaging more independent baseline sessions reduces baseline noise without removing uncertainty from the future score. The examples are teaching calculations, not observations of people, Altamens accounts or game telemetry.

Risultati chiave

  • State the baseline rule and the sessions it uses. A change score without its reference is incomplete.
  • A multi-session baseline can reduce sampling noise under explicit assumptions, but it can also mix early task-familiarity effects into the reference.
  • A higher score after an unusually low first result can partly reflect regression to the mean. Repeated comparable sessions help describe the pattern; they do not identify its cause by themselves.

1A baseline is a rule, not a neutral fact

A dashboard often shows change as later score minus baseline score. The subtraction is simple. Choosing the baseline is the analytical decision. A first-session baseline preserves the starting observation but gives that one session complete influence. An average of several early sessions reduces the influence of any one result, while changing the period that the reference represents.

Neither rule is automatically correct. The useful rule depends on the reader's question. 'How did today compare with my first attempt?' and 'How did today compare with my recent level?' are different questions, even when both are labelled progress.

2Worked example: one future score, three change stories

The constructed sequence is 85, 102, 98, 101, 99 and 105 points. Session 6 is the comparison score. Against session 1 it is a 20-point gain. Against the mean of sessions 1 to 3, which is 95, it is a 10-point gain. Against the mean of sessions 1 to 5, which is 97, it is an 8-point gain.

Figura 1The reference changes while session 6 stays fixed
BaselineSession 6

Session 1 only: +20

85.0 points
105.0 points

Mean of sessions 1 to 3: +10

95.0 points
105.0 points

Mean of sessions 1 to 5: +8

97.0 points
105.0 points

All three statements are arithmetically correct. They answer different baseline questions.

Constructed scores. The figure demonstrates arithmetic, not learning, practice or a meaningful-change threshold.

Download the worked examples CSV, standard-library calculator and methods and claim sheet. The files contain no participant or customer data.

3A no-change model separates the two sources of noise

For a transparent calculation, suppose comparable session scores are independent and normally distributed around one unchanged level with a standard deviation of 10 points. If the baseline averages n sessions and the future score is one independent session, the standard deviation of their difference is 10 multiplied by the square root of 1 plus 1 divided by n.

The change-score standard deviation is 14.14 points for a one-session baseline, 11.55 for three sessions and 10.95 for five. Baseline averaging reduces one part of the variation. It cannot remove the 10-point variation assumed for the future session itself.

Figura 2Apparent gains remain possible when the underlying level is unchanged
At least +10 pointsAt least +15 points

1 baseline session

24.0%
14.4%

3 baseline sessions

19.3%
9.7%

5 baseline sessions

18.1%
8.5%

The exact unrounded probabilities are in examples.csv. Lower probability does not mean that a five-session baseline is automatically valid for every task.

Exact upper-tail probabilities under the stated independent normal model. These are model outputs, not observed Altamens rates or decision thresholds.

4Why a very low first score deserves caution

Regression to the mean occurs when an unusually high or low observation is followed by a result closer to the underlying mean partly because the unusual observation contained random variation. Barnett, van der Pols and Dobson show that the effect becomes more visible with greater measurement error and when follow-up is conditioned on an extreme baseline.1

In the worked sequence, 85 is lower than the next four baseline-period scores. The later 105 may still reflect a real shift, task familiarity, different items or many other causes. The arithmetic alone cannot allocate the 20-point difference among those explanations.

This is related to, but different from, the selection effect explained in Why Personal Bests Rise Even When Performance Is Stable. A personal best selects an extreme from repeated attempts; a baseline rule chooses the reference from which change is calculated.

5More baseline sessions can also measure task familiarity

Repeated cognitive tests can change with exposure. Falleti and colleagues observed early practice effects in short-interval computerized testing of healthy adults. Bartels and colleagues reported larger early changes across repeated neuropsychological sessions in a small one-year study. Those task batteries, intervals and samples do not define an Altamens baseline, but they show why early sessions cannot be assumed exchangeable.2,3

A 2026 conditionally randomized follow-up analysis of 14,802 Nurses' Health Study II participants found that both prior-test count and testing frequency affected Cogstate scores. A separate 2026 observational study of a self-enrolled online cognitive prescreening cohort also reported same-day and later practice effects. These findings concern those tests and study populations; they do not establish a training benefit or a correction factor for consumer games.4,5

6A practical baseline record for personal progress

  1. Define the score, direction and eligible attempts before looking for improvement.
  2. Choose the baseline question: first attempt, fixed multi-session starting period or rolling recent level.
  3. Record the exact session IDs or dates and the aggregation rule. Keep the original observations available.
  4. Use the same task version, difficulty policy, time window, device setup and input method when possible.
  5. Plot the sequence, not only the difference. Mark missing sessions, exclusions and task or device changes.
  6. Report accuracy and trial count with speed. A single composite can hide a changed strategy.
  7. Treat a descriptive change as a description. Use a task-specific reliability and uncertainty model before claiming meaningful individual change.

For denominator-aware accuracy reporting, see Same Accuracy, Different Certainty. For a within-run latency summary, see Mean, Median or Fastest?.

7Limitations, provenance and review status

The worked sequence was constructed to expose the baseline calculation. The no-change result is exact for its stated model, not a fitted description of a game or population. It assumes independent normal session scores with a constant 10-point standard deviation. It omits autocorrelation, bounded scores, adaptive difficulty, learning, fatigue, motivation, device effects and item sampling.

No Altamens user records, accounts, analytics, devices or game telemetry were accessed. The literature review was targeted rather than systematic. Literature search, drafting, calculations, source checks and publication preparation used an OpenAI automated assistant. The calculator and expected outputs received automated consistency checks, but the paper has no independent human methods review and is not journal peer reviewed.

Riferimenti

  1. 1.Barnett AG, van der Pols JC, Dobson AJ Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology 34(1), 215–220, 2005. Methods review with worked examples. DOI: 10.1093/ije/dyh299.. https://pubmed.ncbi.nlm.nih.gov/15333621/
  2. 2.Falleti MG, Maruff P, Collie A, Darby DG Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. Journal of Clinical and Experimental Neuropsychology 28(7), 1095–1112, 2006. Computerized repeated-assessment study in healthy adults. DOI: 10.1080/13803390500205718.. https://pubmed.ncbi.nlm.nih.gov/16840238/
  3. 3.Bartels C, Wegrzyn M, Wiedl A, Ackermann V, Ehrenreich H Practice effects in healthy adults: a longitudinal study on frequent repetitive cognitive testing. BMC Neuroscience 11, 118, 2010. Seven-session, one-year study of 36 healthy adults. DOI: 10.1186/1471-2202-11-118.. https://pubmed.ncbi.nlm.nih.gov/20846444/
  4. 4.Murchland AR, Chen R, Blacker D, Kang JH, Glymour MM, Haneuse S Improvements in Cogstate Test Performance Depend on Number and Frequency of Prior Tests: Evidence from a Randomized Follow-Up Design. Alzheimer Disease and Associated Disorders 40(2), 77–84, 2026. Conditionally randomized testing features among 14,802 Nurses' Health Study II participants. DOI: 10.1097/WAD.0000000000000728.. https://pubmed.ncbi.nlm.nih.gov/42224199/
  5. 5.Wischmann HA, Huijbers W, Gillies M, Nasreddine Z Quantification of practice effects in a self-administered online cognitive pre-screening test. Neurological Sciences 47(9), 2026. Retrospective observational online test-retest study. DOI: 10.1007/s10072-026-09333-5.. https://pubmed.ncbi.nlm.nih.gov/42608601/