# Data dictionary and integration contract

All JSON is UTF-8, with machine numbers unrounded. IDs use the exact recorded model key or numeric question ID. No field named `null` should be coerced to zero unless the documented failure policy requires it. Tables can be joined by `model` ↔ `modelKey` and `questionId`.

## Files

| File | Unit / purpose |
|---|---|
| experiment.json | Version, status, dates, denominators, score formula |
| models.json | 17 candidate run headers, exact IDs, provider policies, endpoint snapshots and observed reasoning-token summaries |
| stages.json | Three extraction passes, adjudication, two full judges and five calibration passes, with recorded settings |
| dataset.json | Corpus composition and gold fact counts |
| publishers.json | 29 selected publishers, row/question/language/headline-only counts |
| questions.json | 100 questions, selected coverage, track, overlapping themes, source-window rule and prompt hash |
| source-rows.json | 751 row references, article IDs, publisher, language, timestamp and serialized-row hash; article URL is null |
| leaderboard.json | All-question means and headline intervals for 17 models |
| scores.json | 1,700 model-question outcomes; failed cells omit unmeasured quality metrics |
| judge-scores.json | 3,385 completed pointwise judge grades reduced to numerical components |
| slices.json | Each model on each documented source-language/track/density slice |
| paired-comparisons.json | Both directions of all model comparisons, all questions and Nepali-only source questions |
| production-comparison.json | Detailed Gemini-baseline comparison with DeepSeek, Fable and Gemma; paired components, slices and examples |
| annotations.json | Counts grouped by model, judge, claim location and annotation kind |
| failure-events.json | Seven generation failures and their recorded classes |
| fact-kinds.json | Per-model descriptive recall by gold fact kind, across salience levels |
| calibration.json | Repeated and inter-judge agreement, kappa, MAD and rank correlations |
| judge-profile.json | Full-pass judge means and annotation rates with denominators |
| sensitivity.json | Weight scenarios, sanitized/consensus/shared-success scores, rank frequencies and length correlations |
| length-analysis.json | Complete-panel and cross-model length associations, with interpretation |
| costs.json | Retained-call costs for generation, gold, calibration and grading |
| audit.json | Reconstruction checks, malformed grade IDs, missing grade and generic-metadata conflicts |
| claims.json | Editorial claim IDs linked to evidence and caveats |
| figure-catalog.json | Figure specifications, descriptions, accessible alt text and source data |

## Units and denominators

In `leaderboard`, `scores`, `judge-scores` and `slices`:

- `effective` / `quality` / `cleanEffective` / `consensusEffective`: 0–100.
- `core`, `supporting`, `precision`, `headlineNe`, `headlineEn`, `entityPrecision`, `entityRecall`, `entityF1`: proportions 0–1. Multiply by 100 for a percentage axis.
- `ne`, `en`: 1–10 on valid answers. Do not treat failure as prose score zero in a prose-only chart.
- `unsupported`, `contradicted`, `neOnlyFlags`, `neRelevantFlags`: mean counts per available judge for that answer, or means over valid answers. Not percentages.
- `neLow`: valid answers with averaged NE score <7. `neHigh`: ≥9. Display denominator `valid`.
- `neAnyFlag` / `neOnlyAnyFlag`: answer-level boolean in scores; answer counts in leaderboard/slices. The first includes claim locations ne and both, the second only ne.
- `costUsd`: sum of retained recorded USD costs; `costMissing` gives missing values. The all-question model sum covers 100 slots.
- `p50`, `p95`, `latencySeconds`: seconds for retained calls. No normalization by tokens or throughput guarantee.
- `neWords`: whitespace-delimited Nepali word count. `enWords`: deterministic benchmark count, not tokens.
- `entityF1`: per-answer surface-matching F1 averaged across answers; not a global micro-F1.

`production-comparison.json` has a legacy display-oriented convention: `core`, `precision`, `supporting`, headline and entity metrics in `rows` are already 0–100 percentages. Its paired deltas for those fields are percentage points. Its slice `values` core and precision remain proportions, while slice delta objects are percentage points. Its fact-kind recalls are percentages. Do not blindly combine it with the normalized core tables. Prefer normalized `leaderboard`/`scores` for new UI tables and the focused file for its precomputed intervals.

`fact-kinds.json` recall is a proportion. Facts from failed generations are excluded, so `eligibleFacts` can exceed `scoredFacts`. `judgeLabelCounts` counts labels across available judges and is not the denominator of the averaged per-fact recall. Missing/duplicate IDs are sanitized. Facts are not independent statistical observations.

`calibration.fullPass.modelRankSpearman` uses each judge's available valid grades. `fullPassFailureInclusiveRankSpearman` includes zero-scored generation failures as in the primary leaderboard. They differ slightly. This is a denominator choice, not a contradiction.

## Model metadata

- `modelKey`: local stable experiment identifier; use for joins and URLs.
- `modelSlug`: exact requested OpenRouter model ID.
- `label`: recorded display label; this can contain descriptive suffixes.
- `providerPolicy.order`: requested provider slug; fallbacks disabled.
- `endpoint.providerName`: recorded serving-provider name, distinct from the model namespace.
- `endpoint.tag`, `quantization`, `contextLength`, `supportedParameters`, `modelReasoning`, `fetchedAt`: historical endpoint snapshot. “unknown” quantization must render as unknown.
- `reasoning`: normalized run-header policy. The actual candidate request object is also saved under `requestParams.reasoning`; display both in an advanced drawer.
- `maxTokensSent`: actual request cap, including reasoning headroom where present.
- `temperatureHonoured`: derived from endpoint-supported parameters, with the candidate runner sending 0.2. It is a recorded compatibility assertion.
- `responseFormatMode`: actual schema versus JSON-object mode. `invariants` is contextual and may disagree.
- `servedProviders` / `servedModelIds`: distribution across saved call records, including unrecorded failure responses. A failure without a provider is not evidence that it switched hosts.
- `observedReasoningTokens`: missing tokens remain missing; mean is over reporting calls only. Token cap is not a fixed visible-answer length.
- `goldContributor`, `fullPassJudge`: overlap labels; they do not imply measured bias by themselves.

Stage headers are preserved as recorded. Adjudication's 20,000 cap is established in the archived code and may be absent from its raw header. Do not infer a missing temperature from candidate settings: extraction/adjudication/judge calls have separate request builders.

## Filtering and joins

Each source-language label describes inputs. Every candidate output is bilingual. `ne+en` should never be invented as a second label for the stored `en+ne` grouping. Track groups partition the questions; themes and risk slices overlap. Question IDs are stable within v2 and must not be joined to v0.3 cluster indices by position.

For paired prose or auxiliary-metric comparisons, use joint successful answers and show that n. For paired effective-score comparisons, use all selected question slots with failures zero. Means for each model over its own valid answers are descriptive and need not equal the paired delta on joint successes.

## Raw evidence

The audit archive contains exact candidate content, parsed outputs, source rows, gold, full judge comments, support indices and original API response identifiers. Public numerical data deliberately omits source excerpts and raw content. Read only the files needed for an evidence drawer. Do not ship the entire audit ZIP inside the website JavaScript bundle.

Source-row SHA-256 is computed on canonical JSON (`sort_keys=True`, UTF-8, compact separators). File hashes are computed on exact bytes. Those are two different hashes with different purposes.