Altamens Ricerca · Reproducible methods note
Mean, Median or Fastest? What a Reaction-Time Summary Actually Says
One delayed response can move an average without changing the middle trial. A small worked example shows why the summary must match the question.
Sintesi
A reaction-time run produces a distribution of trial times, not a single natural score. The mean, median, trimmed mean and fastest response answer different questions and respond differently to a long delay, a broad shift and a change in trial count.
This reproducible methods note works through three deterministic nine-trial lists. Replacing one 420 ms response with 1,800 ms moves the mean from 354.4 to 507.8 ms while leaving the median, the stated trimmed mean and the fastest response unchanged. Adding 40 ms to every response moves all four summaries. The examples are teaching calculations, not observations of people, Altamens accounts or game telemetry.
Risultati chiave
- Keep the trial-level distribution long enough to inspect it; no single summary reveals whether one delay or most responses changed.
- Predefine eligible trials, exclusions and the summary before comparing runs. Report correct-trial count, errors and omissions beside latency.
- The fastest response is an extreme, not a stable measure of typical speed; it is especially sensitive to how many trials were offered.
1One run, several defensible questions
Reaction times are commonly right-skewed: many responses cluster in a faster region while a smaller number extend into a slower tail. A run therefore needs both a scoring rule and an interpretation rule. The mean describes total latency divided by the number of eligible trials. The median identifies the middle eligible trial. A trimmed mean averages after a stated fraction is removed from both tails. The fastest value is the minimum observed response.
Research comparing reaction-time summaries does not identify a universal winner. Rousselet and Wilcox argue that different distributional questions require different methods, while Miller's later simulations show circumstances in which participant means give more power than medians for condition comparisons.3,4
2Worked example: one long delay
The baseline list contains nine correct-trial latencies in milliseconds: 310, 320, 330, 340, 350, 360, 370, 390 and 420. In the delayed list, only the final value changes, from 420 to 1,800. The nominal 20% trimmed mean removes floor(9 × 0.20) = one value from each tail before averaging the remaining seven.
Mean
Median
20% trimmed mean
Fastest
The unchanged summaries do not prove that the delay was irrelevant. They show that those summaries discard or ignore that part of this small distribution.
Deterministic nine-trial examples. Lower is faster. The delayed value is retained in the mean and removed by this particular trimming rule.
The mean rises by 153.3 ms because every eligible value contributes. The median stays at 350 ms because the fifth ordered value is unchanged. The trimmed mean stays at 351.4 ms because the rule removes both the smallest and largest values. The fastest stays at 310 ms because the minimum is unchanged.
Download the worked examples CSV, standard-library calculator and methods and claim sheet. The files contain no participant or customer data.
3Contrast: a broad 40 ms shift
The third list adds 40 ms to every baseline response: 350, 360, 370, 380, 390, 400, 410, 430 and 460. Because the entire distribution moves, all four summaries move by 40 ms.
Mean
Median
20% trimmed mean
Fastest
The summaries agree because the example applies the same constant to every trial. Real runs can change shape as well as location.
Deterministic examples with the same nine-trial count and ordering. This construction illustrates arithmetic, not a training or fatigue effect.
Figures 1 and 2 have deliberately different stories. Figure 1 contains a local change in the slow tail; Figure 2 contains a uniform location shift. A dashboard that keeps only one headline number cannot distinguish those stories after the trial data are discarded.
4What each number says—and leaves unsaid
- Mean: uses every eligible trial and answers an average-latency question. It is visibly affected by long delays, so eligibility and exclusions matter.
- Median: is resistant to tail values and describes the middle ordered trial. With small samples it is coarse, and unequal trial counts can complicate comparisons.
- Trimmed mean: balances tail resistance with averaging, but the trim fraction and rounding rule are analyst choices. Trimming can remove an accidental delay or a genuine slow response.
- Fastest: describes the minimum observed latency. More trials create more opportunities to observe an unusually fast value, so the minimum is not a fair typical-speed comparison when trial counts differ.
Miller's 1988 warning focused on bias in sample medians under skew and unequal trial counts. Later work challenged a blanket rejection of medians and proposed bias correction, followed by a further simulation-based counterpoint about power. The disagreement is useful: method choice depends on the intended comparison, sample structure and error costs.2,3,4
5Long responses are observations before they are ‘outliers’
Ratcliff evaluated transformations and cutoffs for reaction-time outliers and emphasized that method performance depends on distributional conditions. More recent simulations by Miller found that common outlier-exclusion procedures can introduce bias and concluded that keeping all observations can often be preferable.1,5
That does not mean every recorded duration is valid. A task may prespecify impossible anticipations, abandoned trials, background-tab periods or device interruptions as ineligible. The important distinction is between a task-defined invalid observation and a slow but valid response. Inspect the reason before deleting the number.
6A compact reporting checklist for personal progress
- Use the same task version, difficulty policy, device setup and input method when comparing sessions.
- State which trials are eligible: usually correct responses for latency, with errors and omissions reported separately.
- Choose the primary summary before looking for improvement, and keep the rule stable across sessions.
- Show the eligible-trial count and a small distribution view or quantiles beside the headline number.
- Keep accuracy with speed. A faster correct-trial summary can coexist with more errors or skipped items.
- Treat a one-session change as a description. Repeated comparable sessions and an uncertainty model are needed before stronger interpretation.
For a broader comparison workflow, use the reaction-time test comparison checklist. To examine the joint speed-and-accuracy question, see Faster Is Not Always Better.
7What these examples cannot show
The three lists were constructed to make the arithmetic transparent. They do not estimate a population distribution, validate a threshold, establish a reliable number of trials or identify why a response was slow. They do not show learning, fatigue, ageing, attention, impairment or transfer to everyday activities.
A reaction-time result from a consumer task is not a diagnostic measure. Device latency, display refresh, browser scheduling, input hardware, task rules and strategy can all affect recorded time. Comparisons across platforms or task versions need separate validation.
8Provenance and review status
This is a targeted narrative methods note with deterministic examples, not a systematic review or an empirical Altamens study. No Altamens user records, accounts, analytics or game telemetry were accessed. The downloadable Python script recomputes every displayed value from the published CSV.
Literature search, drafting, calculations, source checks and publication preparation used an OpenAI automated assistant. The script and expected outputs received automated consistency checks. The paper has no independent human methods review and is not journal peer reviewed.
Riferimenti
- 1.Ratcliff R Methods for dealing with reaction time outliers. Psychological Bulletin 114(3), 510–532, 1993. Evaluation of outlier-handling methods. DOI: 10.1037/0033-2909.114.3.510. https://pubmed.ncbi.nlm.nih.gov/8272468/
- 2.Miller J A warning about median reaction time. Journal of Experimental Psychology: Human Perception and Performance 14(3), 539–543, 1988. Simulation and analytical warning concerning sample medians, skew and unequal trial counts. DOI: 10.1037/0096-1523.14.3.539. https://pubmed.ncbi.nlm.nih.gov/2971778/
- 3.Rousselet GA, Wilcox RR Reaction Times and other Skewed Distributions: Problems with the Mean and the Median. Meta-Psychology 4, 2020. Open methods paper with code; argues that different distributional questions need different tools. DOI: 10.15626/MP.2019.1630. https://open.lnu.se/index.php/metapsychology/article/view/1630
- 4.Miller J Another Warning About Median Reaction Time. Meta-Psychology 7, 2023. Simulation study comparing mean, median and bias-corrected-median power. DOI: 10.15626/MP.2020.2472. https://open.lnu.se/index.php/metapsychology/article/view/2472
- 5.Miller J Outlier exclusion procedures for reaction time analysis: The cures are generally worse than the disease. Journal of Experimental Psychology: General 152(11), 3189–3217, 2023. Simulation assessment of common RT exclusion procedures. DOI: 10.1037/xge0001450. https://pubmed.ncbi.nlm.nih.gov/37498697/