Argue with the benchmark
Every decision names the evidence that would overturn it, and known failures sit beside the scores they affect. Bring a series, a case or an objection.
Open disputes
0 disputes in the ledger. New submissions await maintainer review.
No objections have been filed yet. Use the Challenge action in the top navigation to file the first one.
Known failures
15 artifacts are open: places where the model produces a number that misdescribes the world. Two have high severity.
| Id | Severity | What it is |
|---|---|---|
| A3 | high | Trust's low income correlation comes from which countries the survey reaches |
| A12 | high | Coordination and Trust are scored on thin evidence |
| A8 | structural | Every correlation here is a hint, not a result |
| A10 | structural | The frame is every measurable country of a million people, and that is not the world |
| A1 | medium | Experimentation is not measured, it is inferred from patents |
| A2 | medium | Per-capita normalisation flattens India |
| A7 | medium | Learning understates Korea and Japan |
| A11 | medium | Building measures industrial output, and reads as delivery capacity |
| A14 | medium | Half of Agency's scored rows are frozen at 2019 |
| A15 | medium | Two survey rows read regime and question format |
| A6 | low | Doing Business indicators are frozen at 2019 |
| A9 | low | Coordination reads Singapore and other executive-led states low |
| A13 | low | Polarization reads calm where there are no camps to polarize |
| A16 | low | Adaptability reads low unemployment unchecked in 16 countries |
| A17 | low | Venezuela's Building has no manufacturing row |
The limits page carries the full entry, the test behind it and a possible fix.
What would overturn each decision
The decision log lists each challenge clause, newest first. New evidence adds a superseding entry.
| Id | The choice | What would overturn it |
|---|---|---|
| D160 | A panel estimate carries across a release of the same major version only where the evidence it read is unchanged | A carried estimate that a rerun of the same panel on the current brief moves by more than the panel's own IQR, in cells whose hash matched, which would mean the brief hash leaves out something the panelists read; or a run on the current dataset, which supersedes the carry. |
| D159 | A value more than 15 years old does not count toward a score | Evidence that a set-aside value describes the country as well as a current one on some row (a slow-moving construct with a stable series), which would set a per-row limit in the registry rather than raise this one; or a reader of a withdrawn score showing the old value told them something the empty dimension does not, which would publish the old value on a separate, unscored line. |
| D158 | Brazil's layer opens on what the claim test supports, speaks Portuguese around its pages, and asks for no money | A Brazilian reader, ideally an economist, finding a term or a sentence wrong, which replaces this review with theirs; a reader of the opening taking it for the strong claim, which would cut it back to the verdict section's link alone; the owner reopening outside funding, which restores an ask without naming windows; or a Portuguese contact form accepted by a decision that supersedes D71's one inbox. |
| D157 | A World Bank zero is dropped where the row cannot be zero | A publisher documenting a true zero on a flagged row, which removes the flag from that row; a published 0 on an unflagged row shown to be a placeholder, which adds it; or a source-side fix (the World Bank printing Venezuela's series as missing), which leaves the rule in place with nothing to drop. |
| D156 | The Spanish copy is reviewed by Claude, the chosen terms stay, and four wordings change | A native Spanish reader, ideally in development economics, finding a term or a sentence wrong, which replaces this review with theirs; or a reader of the map taking a "posición en la escala" sentence for the raw value, which would need the row's direction printed beside it. |
| D155 | An in-session panel may come from one vendor | A multi-vendor run on the same dataset that disagrees with the one-vendor run's median by more than the one-vendor run's own median IQR on more than a quarter of cells, which would show the shared blind spot is large enough that a one-vendor panel should not publish. Or any gateway or multi-vendor run becoming available, which supersedes the run (not the rule). |
| D154 | Open data and model knowledge only: no gated sources, and an in-session multi-vendor panel is a panel | A source that fails the access test publishing open terms, which readmits it under this rule with no further step (Afrobarometer stating redistribution terms that cover derived country shares is the one worth watching). An owner ruling that accepts one gated source by name. For (b): an in-session panel whose stance-by-vendor spread shows that the two stances on one vendor agree more with each other than with the other vendor's on the same stance, by more than the panel's own median IQR, which would mean the panel is measuring vendors rather than countries and needs a third vendor before publication. For (c): a later human or gateway check that overturns a body the codex check confirmed. |
| D153 | The frame takes every country of a million people that is measurable in all nine dimensions | A rescore on a later release whose D137 interval on r or D138 (b) margin is no tighter than this one, which would mean the added countries are noisier than the 50 they joined and the precision argument failed; a source that measures Experimentation for the OAPI members or Shared purpose for the Gulf, which would bring the eight-dimension group under the rule and reopen it; or a guardrail that rises above 0.526 inside the 125 without a country being added, which is the drift the re-baselined figure exists to catch. |
| D152 | Four gaps that read a stock are redefined as behaviour and stay declared gaps | A source that reads the old definition at full frame and an argument that the stock, not the act, is the capability; evidence that a redefined row's candidate series reads income and nothing else once wired, which would make it a condition under D122; or an owner decision that a construct is no longer wanted, which retires the row under D151. |
| D151 | A construct still wanted stays a gap when its only series fails a construct test; national belonging is reopened | Evidence that readers take a declared gap with a failed series beside it as a measurement the project made, which would argue for a third status between gap and retired; or an owner decision that a specific construct is no longer wanted, which retires that row under this rule. For national_belonging: an item on belonging that reads the same in every regime class, or a behaviour of belonging, for 27 or more of the frame, would make it an indicator candidate. |
| D150 | Informal employment is published beside Adaptability as a condition | A defensible direction, for instance evidence that informality reliably slows or speeds reallocation after a shock across countries, which would make it an indicator on its own entry. Or a harmonised series that closes the EU-SILC gap and the ten missing countries, which would make the rank comparable enough to read across income groups. |
| D149 | New export products are scored in Adaptability from the Growth Lab's Atlas trade data | A domestic-exports series (re-exports removed) at frame coverage that reorders the hubs by more than the frame's typical move, which would make the gross rate a hub artefact and call for that series or a gate. Evidence that entry counts track resource finds or a shrinking basket more than reallocation, for instance a window in which most new products are minerals or crossed the line on a falling total. Or a new release of the file that changes the definition of RCA or the reconciliation, which needs a new pin and a revision run. |
| D148 | The foresight register goes to the 53 under agent coding and a human spot-check | A spot-check that overturns two or more of the 11 sampled bodies, or any body decision that changes a country item. Or, on the 53: alpha under 0.80 on any item or on body-by-body decisions in a 20 percent second coding, regime rho at or above 0.80, or 43 or more of the 53 at one score. |
| D147 | Two evidence records leave the corpus with the row they were filed against | business_rd_share returning as a gap or a scored row, which would make both records admissible again. |
| D146 | The foresight register is piloted before it is coded, and the pilot sends the codebook back for revision | A recode under 1.1, with at least one human coder, reaching alpha of at least 0.80 on every item and on body-by-body decisions, with no ceiling and regime rho under 0.80, which supports coding the 53 at tier expert_panel. Regime rho at or above 0.80, or eight of ten at one score after revision, stops the register. |
| D145 | New public repositories are scored in Experimentation from the GitHub Innovation Graph behind an access gate | Evidence that new repositories track platform adoption rather than attempts, for example a within-country series in which the count follows developer sign-ups and not new projects, which would make the row a behavioural check under D60. A second code-hosting source with frame coverage that reorders the frame, which would show the row reads one platform's share of the market. A held country whose count resumes growing past the threshold for a reason that is not access, or an unheld country falling under it for a reason that is about capability, either of which would show the gate removes real values. Or GitHub changing what repositories counts or how it locates one, which needs a new adapter version and a new entry. |
| D144 | Customs clearance time is published beside Coordination as a behavioural check | Reopened as an indicator question when either holds: someone with microdata access reads the per-country item base, and 40 or more countries pass at 30 firms on the real base; or a second round on the 2024 questionnaire lets stability be tested within one design. If it is reopened, prefer the all-agency measure, which asks about every border agency and needs a pinned adapter from the Enterprise Surveys portal. Retired instead if readers treat the check as a score, as D60 provides. |
| D143 | National belonging is retired: the only cross-national item reads pride and fails the regime test | A cross-national item on belonging that is behavioural or that reads the same in every regime class, for example one that sorts liberal democracies and autocracies alike, which would reopen the row as a gap with that item as its candidate; or evidence that volunteering_rate and civic participation cannot carry the dimension without it. |
| D142 | Business share of R&D is retired: the only working source reads the make-up of a spending stock | A publisher that separates state from private enterprise in R&D by sector of performance across at least half the frame, which would remove the ownership confound and reopen the row as a gap; or an argument that the composition of R&D spending observes firms experimenting rather than what a country spends, which would make it a condition beside Anticipation's R&D spending instead. |
| D141 | Vocational share, labour force participation and transmission losses move to the conditions layer | A use measure showing that one of the three tracks a capability rather than a stock, such as grid losses moving with reform episodes the Adaptability rows also see, which would return that row to the score by its own decision; or evidence that the unemployment rate on two rows orders countries by labour-market rigidity rather than reallocation, which would make the two-row Adaptability scores a known artefact and argue for holding them until long-term unemployment covers them. |
| D140 | Trust in strangers is scored in Trust as the share trusting a first-time acquaintance completely or somewhat (G007_34_B) | A published single column or mean for G007_34_B in a later release, which would replace the sum and close the exception; microdata showing the EVS-WVS difference reorders the frame once fieldwork year is controlled; or a fieldwork review showing the Ethiopia sample, or any other outlier, is not national, which would hold that country's value. |
| D139 | A panel figure is compared with the indicators only on the dataset it was scored on | A reviewed gateway panel scored against the current dataset version (#33), which makes the comparison readable again. |
| D138 | What is left after income is tested in aggregate every release | A release where (b) reads alike or noise, which flips the weaker claim to fails or mixed on every surface without an edit; a churning release reading on any dimension; a measurement-noise floor built from indicator uncertainty that puts the shape share inside it; or a wider country set on which the margin in (b) can be estimated within a few hundredths. |
| D137 | The one-factor test is computed and published every release | A country set wide enough that the factor's r with income can be estimated within a few hundredths, which would replace the bands with a confidence interval; a parallel-analysis or permutation baseline built on the actual score distributions, if it moves the chance level by more than 0.02; or an imputation rule adopted elsewhere in the model, which would make complete cases the odd one out. |
| D136 | Every country gets the capability map in the ground layer, and the layers' maps become readings of it | A reader treating the English index as a scorecard (a ranking of a country's capabilities or a count of above and below quoted as a result), which would drop the position column from every index, layers included; a published field carrying the survey and the gate's drops, which would retire COUNTRY_ROW_FACTS in favour of the data; or the fact table growing past one indicator, which would show the facts are a data problem and move them into the adapter's output. |
| D135 | The evidence corpus is complete when every grid cell is closed, and a cell can close without a record | Notes that a later search with the same lists overturns at a high rate, which would show the notes are closing cells on thin searches and need a second researcher; or a comparable series for any column, which promotes the gap under D20 and removes the column. |
| D134 | Mexico, Colombia, Chile and Argentina get Spanish layers with the computed sections | A Spanish reader showing that a sentence on these pages reads as advice or as a claim about the country the data does not carry; the four layers' sections staying identical to each other through the next releases while Brazil's grow, which would show D69 was right that a layer without country work is a translation, and would move the map to the ground layer for every country with the layers as readings of it; or a reader in one of the four countries finding the neutral Spanish wrong enough to need a national lexicon. |
| D133 | Brazil's layer publishes the capability map for all nine dimensions | A reader reading the index as a scorecard (counting above and below as a result), which would drop the position column from the index and leave it on each page; a lexicon rename the redirect table cannot absorb; or a second country layer, which would move the map index to the ground layer as D130's clause already anticipates. |
| D132 | Confidence in the courts is published as a check beside Trust; `institutional_trust` stays a gap | A release that publishes a valid-answer mean or a two-category share for the battery, with the regime pattern gone (autocracies no higher than democracies) when tested the same way; pooled microdata with a list-experiment or anonymity adjustment that removes the deference component in closed and electoral autocracies; or a confidence measure that tracks court_compliance (r at or above 0.5) rather than running against it, which would show it reads the institution and not the regime. |
| D131 | Government compliance with the courts is scored in Trust; the rest of the V-Dem sweep is declined | Evidence that coders score compliance from a country's general reputation rather than from rulings and responses, for example a validation against documented non-compliance events that the code does not track, which would make it the reputation D23 retired and return it to a check or retire it; a closed regime reading high on silence in a later release, the A13 failure written down here; a harmonised court-performance series (clearance or enforcement) across the frame, which answers the institutional family more directly and would make this row a check beside it; or a redundancy reading at or above 0.85 with another Trust row. |
| D126 | Resident industrial design applications are scored in Experimentation | A redundancy reading at or above 0.85 with trademarks, which would make one of the two rows a check; evidence that subsidised or strategic filing, not attempts, sets the cross-country order; or a WIPO series that counts a country's residents at every office it files at (national, EUIPO, Hague), which would replace this one. |
| D125 | The GEM extension is held, because it adds evidence mostly where income already is | GEM surveying enough of the missing lower-income countries that the extension no longer raises the guardrail, or an open GEM data release that covers them; or a decision that a source raising confidence in richer countries is acceptable when its construct is sound, which would supersede this entry and D117's guardrail together. |
| D130 | Brazil's layer publishes a computed map of Adaptability among its income peers | A reader or reviewer showing that a sentence on the page is read as advice despite the rule, which would cut the reading section back to numbers; evidence that the peer set is unstable across releases (more than half the peers swapping on a routine re-ingest), which would move the rule to a wider set or a band; or a second dimension's map showing the layout does not carry it, which would move the page to the ground layer with the layer as one reading of it. |
| D129 | Voter turnout is published as a behavioural check beside Shared Purpose, not scored | A scoring rule conditioned on regime and compulsion that the project adopts on construct, which would move the conditioned subset to indicators.ts as a row and leave the rest as this check; a turnout source that measures participation outside the electoral channel; a V-Dem release that changes v2eltrnout's denominator or drops coverage below half the frame; or evidence that readers take the check for a score, which under D60 would retire it. |
| D128 | Civic participation is scored in Shared purpose from charitable membership (A080_01) | Microdata showing that the EVS-WVS difference holds within countries after fieldwork year is controlled, at a size that reorders the frame, which would hold the row until a format adjustment is documented; a harmonised "any non-religious membership" or volunteering series with frame coverage, which would replace the single item; or evidence that charitable membership in some countries is a condition of employment or state programmes, which would make it the A13 case after all. |
| D127 | Perceived control is scored in Agency from the Joint EVS/WVS A173 mean | Evidence that the cross-country ordering is mostly response style, for example anchoring-vignette or scale-use studies that reorder the frame when applied, which would make the row a measure of how samples answer 10-point questions; a behavioural Agency row with frame coverage that contradicts it, which would make it a check under D60; or a release whose pooled microdata let the three held countries in, which needs a new entry for the pooling rule. |
| D124 | Research citation impact is wired from OpenAlex as a top 10% share | A fractional-count or domestic-only reading, from the OpenAlex snapshot or the API, that reorders the frame enough to show whole counting is reading collaboration networks rather than use of a country's own research, which would swap the construct or make this row a check; affiliation coverage shown to bias the share by country income or region, which would make the row read indexing rather than research; or OpenAlex changing the percentile's pool or definition, which would need a new adapter version and a new entry. |
| D123 | Bribery incidence is scored in Trust, because it observes whether the rules hold | Evidence that reticence, not experience, drives the cross-country ordering: for example the Enterprise Surveys' published reticence indicators placing the lowest-reporting countries among the most reticent. That would make the row a perception of risk rather than an observation, and it would return to a check with that reason. |
| D122 | What a country has is published beside its score as a condition, not scored | Evidence that a condition observes the capability after all, for instance a use measure that tracks the stock so closely that the two cannot be separated, which would return that row to the score by its own decision. Readers or consumers treating the conditions layer as a second score, which would mean the layer does the harm D60 warns of and the rows should move to the indicator rows with status, or out of the published dimension altogether. Or a capability row for Anticipation, Agency or Learning whose arrival shows that the confidence lost here was a cost of the split rather than of thin evidence, which would reopen whether the move should have waited for replacements, as Tier B does. |
| D121 | V-Dem polarization is published as a behavioural check, not scored | A V-Dem reading conditioned on competition existing at all, or a behavioural Shared Purpose row (civic participation, volunteering, voter turnout) that agrees with this item outside the closed regimes, either of which would let the item or a variant score and move it to indicators.ts; a V-Dem release that changes the question or drops coverage below half the frame; or evidence that readers take the check for a score, which under D60 would retire it. For /sources: a check whose source has neither an API request nor a pinned file to print, which would leave the page short of a call again. |
| D120 | Long-term unemployment is wired from ILOSTAT behind a plausibility gate | A national statistics office or the ILO documenting that a held series is a valid measure of long-term unemployment, which would make the floor a rule that removes real values; a second source for the same countries, such as OECD or Eurostat, that disagrees with the gate's survivors by more than the gate's own 15-point tolerance; or a review of unemployment_rate beside this row showing the two readings cannot be separated in practice, which would make the row a check under D60 instead of an indicator. |
| D119 | Export concentration from UNCTADstat fills export diversification | A source that observes reallocation itself, such as entry into new export products or the speed a basket recovers after a price shock, at comparable coverage, which would answer the construct more directly and replace this row; or evidence that the high-value-line artefact moves more countries than the three named, which would make the row read wealth structure rather than exposure and return it to gap. |
| D118 | A row is chosen for what it measures, and its income correlation is reported, not used to choose it | An audit of rows decided after this entry showing that the construct arguments, written before the values, still drift toward rows that lower the income correlation. That would show the pre-commitment is not working. It would also be overturned by rows admitted under this rule that reviewers agree are levels of spending or adoption under another name, which would show the construct question is too loose to stand in for the number. |
| D117 | Research is chosen by two objectives and triaged before values are fetched | A dimension that reaches both targets and still fails the claim, for example by pairing with another dimension in duplicateDimensionCandidates in diagnostics.json, which would show the objectives measure the wrong thing; or a triaged-out candidate that a later preflight shows would have cleared the half-frame screen, which would show the triage questions are too strict. |
| D116 | The sources page prints the check series too | A check fetched from somewhere other than the World Bank, which the request builder cannot print, and which would need the page to print another source's call shape before it could stay complete. |
| D115 | Out of frame is read against the values that built the frame | A reason to treat the fence rather than the observed extreme as the edge of the frame, which would mean flagging every winsorized historical value, and the current outlier's own winsorized value with it, breaking the D47 promise from the other side. |
| D114 | A count is drawn as a count, and a year as a position in time | A capability with enough rows that the ticks stop being countable, which would want a different form for coverage and not a smaller pitch; or evidence that readers take the trailing line for a bar rather than for elapsed time, which would make the rail worse than the year it replaced. |
| D113 | The front page shows a surface rather than naming it | A second institution map, which makes the Brazilian module a choice the front page has no rule for. Or a measured cost on the front page's response time from the added reads, which would mean the matrix should be computed once by bench institutions and read from data/out. |
| D112 | The hero's dot motif breathes, behind the reader's motion setting | A reader finding the motion distracting at these alphas, which would mean atmosphere has become texture and the still clause of D81 should return. Or a measured cost on the front page's interaction readiness that the loop causes, which would mean the canvas has to wait for idle before it starts. |
| D111 | The network sidebar is the facet navigator | Evidence that readers cannot distinguish browsing a facet from filtering the institution set, or that the sidebar becomes unusable at the supported breakpoints; or a supported Envisioning facet-navigation API that provides the same destinations without route configuration. |
| D110 | The network opens on a national scope, with state institutions one switch away | @envisioning/app exposing a supported visualization-level applyCriteria or scope API, which would let the route remove its adapter and use the library's own control end to end; or evidence that the 282-state-node overview is a clearer first reading than the national scope. |
| D109 | The mounted network owns a deterministic, opaque theme boundary | @envisioning/app exposing a supported scoped-theme API and an opaque network-toolbar option, which would remove the route adapter; or evidence that the graph is more legible when its controls are translucent, which would justify restoring alpha for this visualization. |
| D108 | The registry and its diagnostics are drawn as lanes, and the drawing is a viewer component | A reader taking a dot's position inside a lane, or its absence of a line, for a finding the diagnostics do not make, which would mean the encodings need a legend the field does not carry or the drawing needs to go. Or the lane picture of the institution map being built and the two components diverging, which would mean the projection was not general enough and the second drawing should have been a data change. |
| D107 | A global body lives once, in a ledger, and a country map reaches it by id | A body that belongs to no country but acts inside only one, which would make the ledger a place for regional bodies too and need a rule for which file holds them. Or a second country map whose relations to global bodies do not sort into the four families, the same test D56 set. Or readers treating membership counts as a score, which would mean the line should come off the profile. |
| D106 | The panel is four vendors, activation is explicit, and lost calls are counted | Evidence that stance and vendor are not separable, which would make the round-robin deal the wrong unit; or a provider contract that makes a dropped call recoverable, which would let the run retry rather than record. |
| D104 | The drawn network is mounted in this repository, and the ledger stays authoritative | A contributor outside Envisioning who cannot build the project, which is the failure this trades away and the one that should reverse it. Or @envisioning/app published to a public registry, which removes the whole cost. Or the drawing earning its own deployment for another reason, which would return to what D82 described without needing this entry. |
| D103 | Every chart draws from one set of weights and shades | A chart whose natural drawing scale is not one unit to one pixel, which would make an absolute scale wrong rather than merely coarse and would want the weights expressed as a ratio of the field; or evidence that a reader distinguishes fewer than five weights on these surfaces, which would shorten the scale rather than lengthen it. |
| D102 | The confidence-method update keeps Portugal on the same frame | Evidence that the denominator change altered a score or the normalization frame, or that the version boundary hides a published-shape change, would justify revising the release classification in a later decision. |
| D101 | Portugal's first evidence batch stays outside the score | A source correction, failure of the evidence inclusion tests, or proof that a record is an announcement rather than a delivered institutional case would justify removing or rewriting that record in a later evidence change. |
| D100 | A retired row no longer lowers coverage | Evidence that readers take the higher confidence as a claim about evidence rather than about declared measurability, or a decision to retire rows for reasons other than rejecting an inspected dataset, which would make the denominator selective in a way this entry does not defend. |
| D99 | Portugal joins the benchmark frame as a source-backed country | Evidence that Portugal is outside the benchmark's declared comparison purpose, that its source mappings are not reproducible, or that the expanded frame distorts well-evidenced countries enough to invalidate the relative scale would justify removing or revising this case in a subsequent versioned rebase. |
| D98 | Confidence values use one compact display | Evidence that readers mistake the chip's band color or lose the exact value would justify a more explicit single-element treatment, such as a chip that includes the band label, but not a return to a repeated track and number pair. |
| D97 | Challenge entry is centralized in the header | Evidence that readers cannot identify the target from the selectors, or that objections fall because the score is no longer visible at the moment the form opens, would justify adding a target-aware preview to the central form. It would not justify restoring a button to every score cell. |
| D96 | Native tooltips are for exceptions, not metrics | Evidence that a specific dense view loses an important meaning after the change would justify adding a visible or keyboard-reachable detail control for that view. It would not justify restoring a repeated native tooltip to a shared metric component. |
| D95 | The country shape belongs to the field hover, not the Countries menu | Evidence that readers need country context while navigating from a country page, or that the homepage hover card becomes too tall to read, would justify revisiting the placement. A second surface needing the same shape should extend FlagFieldPoint or a shared profile component, not recreate the radar or add another menu-specific data path. |
| D94 | Dated agenda evidence anchors a separate history rail | Evidence that readers infer causation from proximity alone, or that the rail becomes too dense to identify individual records, would justify removing it or limiting it to an explicit event-selection view. A comparable series that directly measures the documented delivery could move it into the capability line under D20, but it would still not make the event marker a score. |
| D93 | The history window reaches 1960, and the fetch must not truncate it | A registered source release with earlier, comparable data would justify moving the window again. Evidence of incomplete responses under the current page size, or a reader test showing that the extra context is mistaken for an absolute development score, would require pagination or a different presentation boundary before extending the range further. |
| D92 | The agenda history reaches for the oldest fetched year, but never invents a trend | A source window with stable, comparable pre-1976 coverage would justify moving the ingest start earlier. Evidence that a longer matched trend systematically misleads readers about a discontinuity, or that its minimum basket hides more than it reveals, would justify removing the oldest default span while retaining the indicator-level histories. |
| D91 | Country context and page links share one subnavigation band | Evidence that readers cannot tell country context from page links in the shared band, or that the wrapped row becomes harder to operate than the previous stacked treatment, would justify restoring separate context and page bands. A new kind of country layer with more than one reading or a page set that cannot fit the shared row would require a new layout decision rather than an exception in a component. |
| D90 | Evidence coverage is a complete country-by-capability matrix | Evidence that researchers choose better-balanced source checks from a smaller queue view, or that readers consistently mistake filled cells for scored capability, would move the matrix off the public agenda index and leave it as a research artifact. A comparable series that closes a gap removes that indicator from the case lane under D20; the matrix must then be rebuilt from the remaining admissible records rather than preserving a stale cell for visual completeness. |
| D89 | The subnational layer is a computable diagnostic registry, not a second score | A reproducible decomposition method showing that the Gini fixture's state values can validly compose the national coefficient; evidence that readers mistake the diagnostic residual for a score; or a source release whose unit definitions cannot be mapped to the registry without silent imputation would require revising the contract. A successful municipal pilot with a clear audience and stable unit identifiers would justify a separate geometry decision, not a relaxation of the national boundary. |
| D88 | The footer is the tree, not a copy of it | A section whose footer column is the wrong shape for what it holds, most plausibly Countries once there is a way to offer more than the three cross-country readings; or a decision that the footer's job is a chosen short list rather than the whole tree, which would want the choice recorded in the tree as a field, never as a second array. --- |
| D87 | The Countries menu draws the country you are standing in | A measurement that readers open the Countries menu to navigate and are slowed by a picture in the way; or a second menu wanting a preview of its own, which would make menuPreview a general mechanism rather than the one exception it is written as. |
| D86 | The front page states the claim under test, not the shape of the data | Evidence that readers arrive already knowing why capability is worth measuring and leave because the front page will not tell them what it holds, which would make the band the wrong place for an argument. |
| D85 | A section opens its pages from the header, and the header travels with the reader | Evidence that readers open sections they are already in more than sections they are not, which would make the menus a duplicate of the tabs rather than a way past them; or a header that has to grow a fourth row, which would make the sticky band too tall to keep. |
| D84 | An indicator is checked for whether it still separates countries | A demonstration that spread on the fixed frame is the wrong statistic, most plausibly because clamping compresses early years enough to invert the sign for a class of indicator; or a fading flag that survives its carried-forward and clamped counts, which would make the block a trigger for reweighting rather than a report, and would need a decision saying what a faded indicator's share becomes. |
| D83 | V-Dem civil-society strength is the first adapter-backed Coordination repair | A release revision that changes the construct or removes country coverage below the half-frame gate; a diagnostic release that pushes the row above the wealth-proxy threshold or into a redundant pair; or direct validation showing that the CSV values cannot be reproduced from the pinned archive. Any of those would return the row to gap or replace it with a more observable series. |
| D60 | A series that cannot be scored is published beside the score, with the reason attached | Evidence that readers treat a check as a score, which would mean publishing it beside the dimension does the harm the exclusion was meant to avoid, and the row should be retired instead. Or a change in the series that breaks its correlation with income, which would make it an indicator and move it to the other registry. |
| D59 | The data publishes a quoting contract for automated readers, and no MCP server | Evidence that agents are a real consumer rather than an assumed one: referrer or user-agent data showing automated fetches of data/out, or a citation of an NCB number in a generated document that drops the confidence. Either would justify the enforcing surface. The other trigger is the Delphi layer becoming interactive, so a caller wants to run a scored what-if against the frame instead of reading a fixed file. That is a tool call, not a document, and it is the point where MCP earns its keep. It belongs as a route handler in apps/web reading @ncb/core and data/out the way the pages do, so that no third copy of the model exists. |
| D58 | The country's wiring is a system matrix, and its ramp is fixed rather than fitted | A country whose map fills so few cells that the matrix reads as empty rather than sparse, which would mean the surface needs a coverage threshold before it renders. Or relation weights in the schema, which would make a count the wrong quantity to put in a cell. |
| D57 | Trust is two families, and it publishes nothing until both are answered | A harmonised social-trust series and a comparable institutional-performance series that together clear the floor, whose combined dimension survives the D42 wealth-attribution test. That closes the decision by satisfying it. The decision is wrong instead if the two families turn out to correlate above the redundancy threshold once both are measured, which would mean Trust asks one question after all and the split encodes a distinction the world does not make. |
| D56 | The institution map reads as a directory and a relation ledger, never as a drawn network | A country map whose relation set does not sort cleanly into the four families, which would mean the families encode Brazil rather than the model. Or evidence from readers that the whole-network shape is the question they arrive with, which would make the matrix the page's first surface rather than its missing one. |
| D55 | Budget execution is the first API-backed Coordination replacement | Cross-country delivery evidence showing that budget execution fidelity has little relationship to the ability of independent actors to complete shared objectives, or a better comparable indicator that measures joint delivery directly and survives the same wealth-attribution test. |
| D54 | An institution map explains capability and never scores it | A cross-government official register that publishes the same organisations and capability relationships with stable identifiers, or a cross-country construction whose network measures survive differences in documentation coverage. The first would replace the curated skeleton. The second would justify considering a network measure as evidence, but it would still need a separate decision before entering any score. |
| D53 | The radar answers a pointer, and it has no resting number | Evidence that readers want the numbers printed on the chart itself. Nine labels on a 260 unit square collide at the sizes this chart is drawn at, which is why they are not there, but a larger chart on the country page could carry them and would make the readout redundant. |
| D52 | A candidate series is tested before it becomes a registry row | A second ingester. The probe speaks World Bank, and the series that would fill Coordination and Trust are the ones the World Bank does not publish, so its long-term job is to be generalised or retired. |
| D51 | Latin America is covered whole, not sampled | Evidence that thin-coverage countries distort an indicator's Tukey fences enough to change well-evidenced countries' readings, or a regional dataset that covers the twelve better than the World Bank does. |
| D50 | The falsification conditions are an index, not a second document | A decision entry that needs more than one falsification condition, or an artefact whose severity is a judgment rather than a word. Both would mean the structure belongs in data rather than in prose, and the entries would move to a file the documents render from instead of the other way round. |
| D49 | The fetch describes itself once, and the viewer prints that description | A second ingester. The World Bank shape is generalised in sources.ts only as far as one publisher needs, and an ILOSTAT or OECD adapter would want a per-publisher description rather than a World Bank one with other publishers listed beside it. |
| D48 | The panel column takes the same wealth test as the indicators | A gateway run whose panel column holds under 0.70 on the dimensions the indicators cannot measure. That is the evidence that would let a panel estimate carry a dimension, and this diagnostic is how it gets checked. |
| D47 | Every country sets the frame it is scored against | Absolute anchoring per indicator, which would remove the country set from the scale entirely and make a score comparable across versions. It needs a defensible floor and ceiling for each of the 33 scored indicators, hand-set and defended one at a time, and it trades a frame that describes this set for one that describes a claim about the world. |
| D46 | A case study is an address, and the list of them is a filter | The corpus growing past the point where shipping it to the browser is reasonable, which makes the index a server-rendered list reading the same query parameters. Or evidence that readers never use the record pages, which would mean the anchor was enough and the cost was the two extra clicks to reach a mechanism. |
| D45 | A dimension with fewer than two observed indicators publishes no score | Replacement indicators landing in Coordination and Trust, which is what A12 asks for and would make the floor moot for them. Or evidence that readers treat an empty axis as a zero anyway, which would mean the withholding needs words on the chart and not only in the table. |
| D44 | The homicide rate is retired, because it was the wealth signal it was hired to remove | A country set wide enough that homicide stops tracking income, which would mean the correlation was an artefact of ten reference countries. Or a behavioural trust measure landing beside it, which would let the dimension carry homicide without homicide carrying the dimension. |
| D43 | The country page opens on the agenda, and the split has one home | A country page that reads better with the shape first and the agenda second, which would move the block under the radar. Or a hold list long enough to deserve a column of its own. |
| D42 | A candidate is judged on what it does to its dimension, not on its own correlation | Evidence that a high delta is routinely the right answer, which would make the diagnostic noise. Or a dimension-level wealth test derived from a partial correlation rather than a leave-one-out, which would be the stronger statistic if the country set ever grows enough to support it. |
| D41 | The pipeline stops discarding its own warnings, and the viewer carries them | A run where the exemplar exclusion empties most dimensions, which would argue for annotating clamped exemplars instead of excluding them. A persistent ingest failure record, which would replace the silent carry-forward. |
| D40 | A date is metadata, and a document a reader is told to open is a link | A viewer page that renders the limits document itself, which would give the viewer a local target for {limits} while the markdown keeps the repository link. Or a versioned documentation site, which would replace the branch in docHref with a release. |
| D38 | The homepage is global, and one country is a layer on top of it | Use showing readers cannot enter the grid without a worked example. A decision to make this viewer a country-specific product, which would put the focal case back and make the global grid the secondary surface. |
| D37 | The output directory describes itself | A consumer that standard tooling cannot serve from the Data Package, which would argue for a real API. Or the schemas drifting from the files in practice, which would mean generation is wired to the wrong place and validation of the emitted output should be added to bench validate. |
| D36 | A record states where the delivery stands, and a claim can carry two numbers | Records that repeatedly need a third number, or statuses that keep landing in the wrong bin — either means the lifecycle deserves a dated series, not two stamps, and the slot design should be replaced rather than extended. |
| D35 | The agenda is computed, and language is an interpretation layer | A reader study or field use showing the raise cutoff misleads at 50, which would justify deriving it from the score distribution instead. Records gaining translated fields, which would remove the mixed-language cost. A lexicon whose translation drifts from the registry meaning, which would justify review rules for lexicon changes rather than plain PRs. |
| D34 | Countries are identified by ISO 3166-1 alpha-3, recorded after the fact | A primary data source that keys on something else and outweighs the World Bank in the registry. That would justify an internal ID with per-source mappings, and this entry should be superseded when it happens. |
| D33 | Evidence records get an inclusion rule before they get more records | A corpus that satisfies every test and still reads as advocacy — that would mean selection bias lives somewhere the rule does not reach, and the rule needs to move from authoring discipline to independent review. Or a demonstrated need to document sub-national or non-state deliveries, which test three currently excludes. |
| D32 | Evidence is drawn as a gradient, and every chart is a control | Nothing foreseeable. If the gradient reads as noise at small sizes, the icon-labelled radars can fall back to the threshold. |
| D31 | A record carries its mechanism, and patterns get their own page | Enough records to compare mechanisms rather than list them, at which point the page becomes a query rather than a list. |
| D30 | Every number opens onto the field it sits in | Nothing foreseeable for the panel itself. A reader who wants the underlying distribution rather than the normalized one needs a raw axis with a log option, which is a further step. |
| D29 | The eighth dimension keeps one name, and the axes carry their marks | Evidence that the grid is unreadable without words, which would mean going back to labels and making the cards larger. |
| D28 | Icons are copied in, one per concept, never alone | A need for many more icons, at which point installing the package beats maintaining a copied set. |
| D27 | Forty countries, and output split one file per country | A frame rebase, which would be a versioned event with its own decision, or a move to absolute anchoring per indicator. |
| D26 | Every term is defined once, in plain language, in the model | Nothing foreseeable. A second surface that needs the same definitions, such as a printed report or an API, reads the same file. |
| D25 | Every point carries its provenance, and every run records what moved | A move to per-country output files, which would change where the series lives but not what it has to carry. |
| D24 | Two spans for a dimension, and a full line for every indicator | Enough indicator history to compute a full-dimension basket at both ends, which would make the matched basket unnecessary and both spans directly comparable to the headline score. |
| D23 | The perception layer is retired, and the cost is visible | Observable replacements: court throughput and case clearance, budget execution rates, cross-agency programme delivery, voter turnout, volunteering rates, civic participation. Each one that lands raises the coverage this decision knocked down. If none land, the honest conclusion is that Coordination and Trust cannot be measured with public data, and they should be reported as unmeasured rather than scored. |
| D22 | Momentum is measured on one ruler and a matched basket | Enough indicator history to score a full basket at both ends, which would let momentum use the whole dimension rather than a subset. A move to absolute anchoring would also change what a fixed ruler means and this decision would need restating. |
| D21 | GEM is wired by hand, venture capital stays a gap | An inspectable venture capital or business R&D series that covers the reference set, or a GEM licensing change that stops the data being usable this way. |
| D20 | Documented deliveries are recorded as evidence and never scored | A comparable delivery series covering the reference set, at which point large_project_delivery leaves gap, the records become source notes on a scored indicator, and this decision is superseded. |
| D19 | Extended countries get no visual marking | A case where the distinction changes how a number should be read, most likely a country clamping at 0 or 100. Flag outOfFrame on that cell rather than reinstating a badge on the country. |
| D18 | One display for every 0 to 100 score | Evidence that readers misread band edges as real differences, or a move away from a frame-relative scale. |
| D17 | Confidence bands are fixed thresholds, and not a red-to-green scale | Evidence about how readers actually act on the bands, or a change to the confidence formula that shifts its range. |
| D16 | The normalization frame is pinned to the ten reference countries | A sustained pattern of outOfFrame cells, or a decision to move to absolute anchoring per indicator. Either way, rebasing is a versioned event: bump a frame version, re-publish, and say plainly that the old numbers are not comparable. |
| D15 | The World Bank is the only wired ingestion source in v0 | Writing the next adapter. Each one is independent work. |
| D14 | Provenance is stored, never inferred | Nothing. |
| D13 | Panel diversity comes from vendors, not from model size | Evidence that stance dominates model, in which case several stances on one model would be as good and simpler to reason about. |
| D12 | Panel disagreement is recorded, not averaged away | Evidence that panel dissent is noise rather than signal — for instance if dissent does not correlate with low coverage across several runs. |
| D11 | Delphi output never enters the capability score | Nothing at v0. Any future blending must be a new, explicit, named field, never a change to score. |
| D10 | Inspectability is a hard filter on sources | A source opening its microdata, or an explicit decision to accept composite indices with a recorded quality penalty via source.tier. |
| D9 | Gap indicators stay in the registry | Nothing. Do not delete gaps to make numbers look better. |
| D8 | Only the most recent observation, no trends | The cross-section holding up. Trend is the obvious v1 extension and the data is already fetched from 2000 onward. |
| D7 | Winsorize with Tukey fences at k = 3 | Evidence that a specific indicator's distribution needs a transform rather than a clip. Prefer adding a transform to the registry over lowering k. |
| D6 | Dimensions are scored at any coverage above zero | A published deliverable. Before anything is published, either introduce a coverage floor below which a dimension reports null, or mark low-coverage cells visually in every output. Open, unresolved. |
| D5 | Missing data is dropped, never imputed | Nothing. But see D6 — this is why a floor may be needed. |
| D4 | Confidence is reported beside the score, never inside it | Nothing we can foresee. This is close to load-bearing. |
| D3 | Equal weights inside a dimension | Delphi construct-validity ratings that are stable across several real panels. Weight by panel-rated validity only once the panel itself has been shown to agree. |
| D2 | Normalisation is relative to the country set, not to an absolute frontier | A move to a large enough country set that absolute anchoring becomes possible, or an explicit decision to anchor against fixed reference values per indicator. |
| D1 | Nine dimensions, no headline ranking | Evidence that the nine dimensions are so correlated that the shape carries no information beyond a single factor. Watch diagnostics.dimensionPairs: if most pairs sit above r ≈ 0.9 on a larger country set, the dimensional structure is not earning its keep. |
The decision log holds 136 entries with each choice, reason and cost.
How to object
Each one keeps its record in public, so the argument stays visible after it is settled.
Argue with a score
MinutesTell us where a number is wrong about the country you know.
- It needs.
- The country, the capability and what the score misses.
- Usually.
- Anyone who knows a country better than a global dataset does.
- What it changes.
- The objection is published beside the score it targets, so the disagreement travels with the number.
Name a dataset for a gap
MinutesPoint us at a published series that covers a capability we cannot measure.
- It needs.
- A comparable series over at least two countries, with an open URL, a publisher, a reference period and a stated method.
- Usually.
- Statisticians, researchers, anyone who works with a national source.
- What it changes.
- A gap becomes a scored indicator, which raises confidence for every country at once.
File an evidence record
A piece of workDocument something a country actually delivered where the benchmark can only record a gap.
- It needs.
- One published number, the period it covers, the publisher, and a statement of what it does not show.
- Usually.
- Public servants, journalists, researchers with a case in front of them.
- What it changes.
- The record is published against the indicator it answers. It never moves a score, and it is how a capability with no dataset can still be read.
CONTRIBUTING.md has the rules for each, docs/EVIDENCE.md has the inclusion test for evidence records, and docs/WHY.md states the claim under test. The open gaps name every indicator with no dataset behind it, and ways to help lists every other way to help.
How to cite it
Quote the dataset version with every score. Adding a country changes the frame and restates all scores.
Envisioning (2026). NCB, the National Capability Benchmark, dataset 9.3.0. https://github.com/envisioning/national-capability-benchmarkThe code is MIT, in LICENSE. Data keeps the terms of its publishers, listed in NOTICE.md. Keep the attribution when you redistribute a number.
This is a prototype. Read the limits and the confidence beside a score; a thin dimension rests on one or two indicators and is not enough on its own to support an argument.