Where Working Memory Runs Out of Room: Score Distributions and a Percentile Ceiling Across 146 NeuroRank Combines

Abstract

We analyzed working-memory percentile scores from NeuroRank's grid-recall module across 146 genre-schema-matched combines (general n=118, MOBA n=28), the two genres whose test battery includes a working-memory dimension; NeuroRank's FPS combine substitutes a flicker-detection module for memory entirely and so cannot be included in this comparison, a schema constraint we treat as a finding in its own right rather than force a three-way comparison the platform's own module mapping does not support. General combines show a working-memory distribution with mean 52.1 (SD 29.4, median 50.0); MOBA combines show mean 55.3 (SD 21.8, median 53.5). Both distributions are visibly non-smooth: 20.3% of general players (24 of 118) and 7.1% of MOBA players (2 of 28) land on the exact same value, 91, the maximum percentile the grid-recall norm table can produce. We trace this to a documented design choice in the scoring engine: the norm table's top anchor sits at accuracy 1.0 (perfect recall) mapped to percentile 91, not 99, because live data showed the task easier for NeuroRank's population than the original literature-derived anchors assumed. We report this ceiling effect, quantify how much of each genre's top-decile spread it flattens, and discuss what it means for the working-memory dimension's ability to differentiate high-performing players.

Introduction and Hypothesis

Working memory, the capacity to hold and manipulate a small amount of information over short intervals, is one of the more heavily studied constructs in cognitive psychology, with a research lineage running from Miller's (1956) observation that immediate memory span clusters around seven items, through Baddeley and Hitch's (1974) multi-component working-memory model, to later capacity-limit and individual-differences work (Cowan, 2001; Kane & Engle, 2002). NeuroRank measures working memory with a 3×3 grid-recall task: players watch a sequence of highlighted cells and reproduce the sequence in order, and the raw metric fed into scoring is the proportion of cells recalled correctly across the module (server/services/scoringService.js, scoreMemoryModule). That raw accuracy is converted to a 1-99 percentile via a piecewise-linear norm table (gridRecallAccuracy in server/utils/normData.js).

This month's data-paper queue asked a straightforward descriptive question: how does the working-memory percentile score distribute across NeuroRank's genre cohorts? The literal framing in the queue named all three genres (general, FPS, MOBA), but NeuroRank's own genre-to-module mapping makes that framing partly unanswerable as stated. Per the platform's module design, FPS combines run a flicker-detection module in the module-4 slot instead of grid recall, precisely because flicker detection was judged more relevant to an FPS skill package than short-term recall; FPS profiles accordingly carry a flickSpeed score and no workingMemory score at all. We treat this as a real constraint on the question rather than a data-cleaning inconvenience to route around: there is no such thing as an FPS working-memory percentile in NeuroRank's current schema, so a three-way comparison would have to either fabricate one or silently compare two different constructs under one label. We do neither. This paper compares working memory between the two genres that actually measure it, general and MOBA, and reports FPS's flicker-detection scores separately, as descriptive context for what occupies the analogous test slot, explicitly flagged as a different construct on a different scale rather than a stand-in for working memory.

While assembling the comparison, we found a second and, we think, more substantive result than the queue's original question anticipated: both distributions show a large, exact-value spike at the grid-recall norm table's ceiling anchor. We report and explain that finding as the paper's central result, alongside the baseline genre comparison the queue asked for.

Data and Methodology

Sample construction

We queried NeuroRank's production Supabase profiles table directly on 2026-09-01: 250 total rows, of which 10 carry the flagged marker (anti-cheat/bot exclusion, per issue #231) and are dropped from all aggregates here, matching the platform's own /api/admin/stats methodology. That leaves 240 clean profiles.

As in this platform's August 2026 data paper, we cross-checked every clean profile's stored score-object keys against its declared genre, because a working combine's score object should contain a fixed, genre-specific set of dimension keys (general and MOBA both include workingMemory; only general and FPS include aimPrecision; only MOBA includes prioritization; only FPS includes flickSpeed). This check reproduces, and does not add to, a data-quality issue that paper already disclosed: 15 profiles (10 tagged fps, 5 tagged moba), all created between 2026-02-06 and 2026-03-03, carry a general-genre score schema despite their declared genre tag. We exclude these 15 from the analysis sample below, exactly as the August paper did, and refer readers there for the full disclosure; nothing in this month's data changes that finding, and no new mismatched rows have appeared since (every profile created after 2026-03-03 in our sample carries a schema consistent with its declared genre).

The resulting genre-schema-matched sample is 225 profiles: general n=118, FPS n=79, MOBA n=28. This is consistent with, and a modest extension of, the August paper's n=223 sample (116/79/28): two additional general combines completed between the two reporting periods account for the entire difference, and the FPS and MOBA cohorts, whose sampling windows had already closed by August (FPS 2026-03-13 to 2026-05-17; MOBA 2026-03-19 to 2026-04-19), are unchanged. The general cohort's window is 2026-02-02 to 2026-08-10.

Of the 118 general profiles, 8 (all created between 2026-02-02 and 2026-03-03, i.e. within the same early period implicated in the schema-mismatch finding above, though these 8 do carry a correct general-schema score object) lack a stored overallScore field, consistent with overallScore having been added to the profile schema after these rows were created. We flag this rather than drop these 8 profiles from the working-memory distribution itself, since their workingMemory value is present and schema-correct; they are excluded only from the one analysis in this paper (the workingMemory-to-overallScore correlation) that requires overallScore.

The working-memory metric and its norm table

scoreMemoryModule computes raw grid-recall accuracy as total correct cells divided by total cells presented across the module's rounds, then passes that single number through one call to toPercentile(accuracy, 'gridRecallAccuracy'). This is a direct pass-through: unlike aimPrecision (a 50/30/20 blend of three separately-normed sub-metrics: hit rate, aim reaction time, and aim distance) or prioritization (a 70/30 blend of completion rate and inter-click reaction time), workingMemory has no blending step. Whatever the gridRecallAccuracy norm table's ceiling and floor behavior is, it applies to the working-memory score directly and undiluted, with nothing else averaged in to smooth it.

The gridRecallAccuracy norm table (server/utils/normData.js, lines 90-95) is a 20-anchor piecewise-linear lookup with an explicit code comment: "Ceiling-capped at 91: the task is far easier for this population than the v1 literature anchors assumed (v1 said 0.46 was median; real median is 0.889)." The table's top anchor is [91, 1.0]: any player who recalls every presented cell correctly (raw accuracy exactly 1.0) is assigned percentile 91, and the scoring function's clamping rule (toPercentile, server/utils/normData.js lines 212-236) means no value above 91 is ever produced for this metric, regardless of how much above the historical median a player performs, because 91 is the highest percentile the table contains. This is a deliberate, disclosed recalibration decision already present in the codebase, not a bug we are reporting for the first time; what this paper adds is a quantification of how visible its effect is in the live score distribution.

Results

Working memory score distributions by genre

Genre	n	Mean	SD	Median	25th pct.	75th pct.	Min	Max
General	118	52.1	29.4	50.0	25.0	78.8	1	91
MOBA	28	55.3	21.8	53.5	43.3	71.3	8	91

Both distributions span nearly the full available range (down to the table's floor near 1, up to its ceiling at 91), and their central tendencies are close to each other: general's mean of 52.1 and MOBA's mean of 55.3 differ by about 0.1 SD pooled, well within what a sample of this size can distinguish with confidence. MOBA's distribution is narrower (SD 21.8 vs. 29.4) and its lower tail is shorter (min 8 vs. min 1 for general), though with n=28 that narrower spread should be read cautiously.

Decile breakdown

Percentile band	General n (%)	MOBA n (%)
0-10	9 (7.6%)	1 (3.6%)
10-20	11 (9.3%)	1 (3.6%)
20-30	12 (10.2%)	1 (3.6%)
30-40	11 (9.3%)	4 (14.3%)
40-50	11 (9.3%)	4 (14.3%)
50-60	13 (11.0%)	5 (17.9%)
60-70	3 (2.5%)	0 (0.0%)
70-80	18 (15.3%)	8 (28.6%)
80-90	6 (5.1%)	2 (7.1%)
90-100	24 (20.3%)	2 (7.1%)

The top decile band (90-100) is the single largest band in the general distribution, at 20.3% of the cohort, roughly double what a uniform distribution across ten bands would predict (10%). This is the first visible sign of the ceiling effect discussed below: an ostensibly continuous percentile scale is producing a disproportionate pile-up at its top end.

The ceiling effect: exact-value repetition at 91

Working memory percentile scores are not, in practice, drawn from a smooth continuum. Listing every distinct value observed:

• General (n=118, 29 distinct values): the single most common value is 91, held by 24 profiles (20.3% of the cohort), more than triple the next most common value (70, held by 9 profiles, 7.6%).
• MOBA (n=28, 12 distinct values): the value 91 is again the most common single value, held by 2 profiles (7.1%), tied with 45 and shared with 70 and 75 as the most frequent values in a much smaller sample.

Every one of these 91-valued profiles corresponds to a player who recalled 100% of presented cells correctly, i.e. raw grid-recall accuracy of exactly 1.0, which is the norm table's top anchor and therefore the highest percentile the metric can express under the current calibration. In the general cohort specifically, that means roughly one in five test-takers who reach perfect recall are, by construction, indistinguishable from each other on this dimension: a player who recalls every cell across every round with zero errors receives the same working-memory score, 91, as every other player who does the same, with no further percentile headroom to separate a merely-perfect run from a dominantly-perfect run repeated across a longer or harder task. For comparison, no other value in either distribution accounts for more than 11.0% of its cohort (general) or 17.9% (MOBA), so the 91-value spike is not simply "the mode of a lumpy but otherwise unremarkable distribution": it is specifically and only the ceiling anchor that produces this concentration, and it is roughly 2-3x larger than the next largest single-value cluster in the same distribution.

This pattern is a direct, mechanical consequence of the documented recalibration described in Methodology, not an artifact of our aggregation choices: the raw grid-recall task turned out to be easy enough for NeuroRank's actual population (median observed accuracy 0.889, per the norm table's own comment, against a literature-derived assumption of 0.46) that a meaningful share of players clear the task perfectly, and every one of those players collapses onto the single point the table reserves for "perfect or better."

FPS context: flicker speed as the analogous, non-comparable dimension

FPS combines run a flicker-detection module instead of grid recall, producing a flickSpeed score built from flickerHitRate and flickerRT, both separately normed and blended (unlike workingMemory's single-table pass-through). Across the 79 genre-schema-matched FPS profiles, flick speed has mean 51.0 (SD 20.8, median 50.0, IQR 32.5-69.5), and unlike working memory it shows no comparable ceiling spike: no single value accounts for more than 20.3% of the cohort in raw terms, but more importantly the maximum observed flick-speed value never approaches a hard table ceiling, because it is a blend rather than a direct pass-through. We report this purely as descriptive context for what sits in FPS's module-4 slot; flick speed and working memory measure different constructs (visual-motor detection speed vs. short-term sequence recall) on different underlying norm tables, and we make no claim that they are comparable or substitutable for the purposes of this paper's central question.

Relationship to overall combine score

We computed the Pearson correlation between working-memory percentile and overall combine score within each genre (general n=110, excluding the 8 profiles missing overallScore; MOBA n=28). General: r=0.461. MOBA: r=0.539. Both are moderate positive correlations, and both should be read with an important caveat: overallScore is itself a weighted linear combination of the six dimension scores including workingMemory (computeWeightedOverall, server/services/scoringService.js), with a weight of 1.0 (of six equally-weighted dimensions) in general and 1.3 (the joint-highest weight in MOBA's six-dimension set, tied with prioritization) in MOBA. Part of each correlation is therefore mechanical, a component correlating with a sum it directly contributes to, and MOBA's larger weight on working memory is a plausible full or partial explanation for its larger correlation coefficient, rather than evidence that working memory is a stronger independent predictor of general combine performance in MOBA specifically. We report both coefficients for completeness but do not treat them as evidence of a "true" predictive relationship independent of the scoring formula's own construction, an important distinction this platform's July 2026 data paper also had to draw when discussing which of its three core dimensions "predicted" overall score.

Discussion and Limitations

The genre comparison the queue originally asked for turns out to be a two-genre comparison, not three, because NeuroRank's own module design deliberately does not measure working memory in the FPS combine. We think this is worth stating plainly rather than working around: a platform whose stated obligation is measurement integrity should not paper over a schema constraint by substituting a superficially similar metric (flick speed) under the working-memory label just to complete a three-way table. Within the two genres that do measure it, general and MOBA show broadly similar working-memory distributions in central tendency, with MOBA's narrower spread the only notable difference, and that difference sits on a small sample (n=28) that limits how much weight it can bear.

The more consequential finding is the ceiling effect. A working-memory percentile scale on which roughly one in five general-cohort players (and a smaller but non-trivial share of the MOBA cohort) all receive the identical top score means the scale currently has reduced ability to differentiate among NeuroRank's higher-performing players on this specific dimension: two players who both cleared the grid-recall task perfectly are reported as identical on working memory even if, say, one of them did so on a materially harder or longer task configuration than NeuroRank currently administers, or if the platform later extends the task's difficulty ceiling. This is not a scoring bug in the sense of a formula error; the norm table's ceiling anchor is a deliberate, documented recalibration responding to a real prior mismatch between literature-derived difficulty assumptions and this platform's actual player population. But a deliberate design choice can still have a measurable resolution cost, and this paper's contribution is putting a number on that cost for the first time: roughly 20% of general test-takers currently sit at the exact same working-memory score, with no room on the current scale to separate them further.

We see two natural, non-exclusive follow-ups this finding suggests, and we are adding both to the platform's topic queue rather than resolving them here, since resolving either would require test-design or norm-table changes outside this paper's observational scope: first, whether the grid-recall task's difficulty could be extended (e.g., longer sequences, a harder configuration for players who clear the base task) to recover discriminability at the top end without re-breaking the difficulty calibration that produced the current, better-fitted median; second, whether the other five dimensions have comparable ceiling concentrations worth quantifying the same way, since two of them (rawSpeed and decisionQuality) are, like workingMemory, single-table pass-throughs rather than blends, and one of their underlying tables (goNoGoAccuracy, feeding decisionQuality) caps at percentile 75 rather than 91, a lower ceiling than working memory's, which the descriptive pattern reported here would predict shows an even larger exact-value pile-up. We did not test that prediction in this paper; it is a specific, falsifiable hypothesis for a future one.

Several limitations bound these results. First, our comparison of general and MOBA is observational and cross-sectional; we cannot separate any genre-driven difference in working-memory distribution from genre-driven differences in who self-selects into the general versus MOBA combine, a limitation this platform's August 2026 data paper also flagged for its own genre comparisons. Second, the MOBA sample (n=28) is small enough that its distributional shape, and particularly its ceiling-value share (7.1%, or 2 profiles), should be treated as imprecise; two additional or two fewer perfect-recall players would meaningfully shift that percentage. Third, we did not have access to item-level or round-level grid-recall data in the profiles table (only the aggregated accuracy that feeds the percentile calculation is stored), so we cannot report, for example, whether the ceiling-value players cleared the task on their first attempt or needed multiple rounds to reach perfect accuracy; a genuinely harder-task extension would need that finer-grained data to design well. Fourth, as with every genre comparison on this platform to date, we cannot rule out that some portion of any general-versus-MOBA difference reflects the earlier average tenure of MOBA's narrower sampling window (2026-03-19 to 2026-04-19, versus general's much longer 2026-02-02 to 2026-08-10 window) rather than a stable genre effect, since the underlying player population using the platform may itself have shifted over that longer general-cohort window.

References

• Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81-97.
• Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. H. Bower (Ed.), Psychology of Learning and Motivation (Vol. 8, pp. 47-89). Academic Press.
• Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87-114.
• Kane, M. J., & Engle, R. W. (2002). The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: An individual-differences perspective. Psychonomic Bulletin & Review, 9(4), 637-671.