Versions & evidence
What is frozen, what can change, and what an edition preserves.
Frozen inputs
A challenge snapshot captures player evidence, cap thresholds, roster evidence, expected salaries, the pick graph, the mandate profile, and model identifiers. Snapshot inputs and a displayed version are necessary provenance, but retaining executable code and external model artifacts is also necessary for a full historical replay.
Published editions
A Field Guide edition is a reviewed, immutable publication of a specific completed team backtest. Future runs do not rewrite it. Its roster evidence, model version, and editorial notes must be interpreted together; it is not necessarily a live ranking of today's teams.
Implementation manifest
The manifest lists current model identifiers, mandate weights, calibration status, source coverage, and SHA-256 hashes of the files included here. Source downloads are from this deployment and do not require repository access. This project currently states no license terms for reuse of the bundled code, and it grants no rights to upstream basketball data; source visibility is not an open-source release.
Source coverage and licensing
This edition publishes a partial, inspectable source bundle—not a standalone algorithm release. It contains 55 aggregation and reference modules, but not the orchestration, shared valuation dependencies, frozen benchmark tables, model artifacts, runtime stores, or upstream datasets needed to reproduce a grade independently.
The manifest names the omitted dependency groups and reports current artifact versions and calibration flags. No project license terms for reuse of this bundled code are currently stated. Visibility permits inspection; it does not by itself make the code open source or grant rights to upstream basketball data.
Current sub-model versions
| Model | Version |
|---|---|
| Benchmarks | 1.0.0 |
| Dav | 1.0.0 |
| Expected salary | 2.0.0 |
| Plausibility | 2.0.0 |
| Player impact | 1.0.0 |
| Progression | 0.1.0 |
| Ptv | 2.0.0 |
| Rookie contract | 2.0.0 |
| Skill indexes | 1.0.0 |
Implementation
These notes and source files are captured from this deployment's code. Expand a module for its algorithm and branches, or download the Elixir file for local inspection.
model_version.exElixir
lib/trade_machine/gm_grade_v2/model_version.ex
The single registry of scoring constants and sub-model versions for the AVC
six-pillar GM grader (
TradeMachine.GMGradeV2
).
Every scoring module in later phases reads its versioned constants from here
rather than hardcoding them, so a submitted grade can name the exact model
version — and every constant that produced it — for reproducibility. When a
constant is retuned, bump the relevant entry in
versions/0
(and, for a
behavioural change,
model_version/0
) so old grades stay reproducible against
their frozen version.
All numeric defaults below are AVC's authored values or explicitly-recorded v1
defaults (see
.maestro/playbooks/2026-07-25-GM-Challenge-Grading/DECISIONS.md
).
They are lab-tunable via an optional
override
map (Phase 16) threaded through
each getter; production call sites pass no override and get the frozen default.
SHA-256 2480bff8105f3786bdb828a03be221456af30eb8dee482c1b263307b337a67b0
defmodule TradeMachine.GMGradeV2.ModelVersion do
@moduledoc """
The single registry of scoring constants and sub-model versions for the AVC
six-pillar GM grader (`TradeMachine.GMGradeV2`).
Every scoring module in later phases reads its versioned constants from here
rather than hardcoding them, so a submitted grade can name the exact model
version — and every constant that produced it — for reproducibility. When a
constant is retuned, bump the relevant entry in `versions/0` (and, for a
behavioural change, `model_version/0`) so old grades stay reproducible against
their frozen version.
All numeric defaults below are AVC's authored values or explicitly-recorded v1
defaults (see `.maestro/playbooks/2026-07-25-GM-Challenge-Grading/DECISIONS.md`).
They are lab-tunable via an optional `override` map (Phase 16) threaded through
each getter; production call sites pass no override and get the frozen default.
"""
@model_version "avc-1.5.0"
alias TradeMachine.Valuation.ContractClassification
alias TradeMachine.Valuation.Coefficients
# `:benchmarks` versions the frozen 30-team benchmark set as a whole — the
# `TradeMachine.GMGradeV2.BenchmarkRotation` algorithm that manufactures each
# real team's 240-minute rotation (rank by Season Minutes, cover 240 minutes
# per game, proportional minutes under a 48-minute ceiling, top five by games
# started) together with the raw metrics and percentile tables built from it.
# Bump it whenever any of those rules change: a stored grade names this
# version, and a benchmark built under different rules is not comparable to
# one built under these.
@versions %{
player_impact: "1.0.0",
skill_indexes: "1.0.0",
expected_salary: "2.0.0",
dav: "1.0.0",
plausibility: "2.0.0",
benchmarks: "1.0.0",
ptv: "2.0.0",
progression: "0.1.0",
rookie_contract: "2.0.0"
}
# Proposed Progression v0 lab calibration. Progression is a separately
# versioned post-composite model: these values never alter the canonical model
# version or the six AVC pillar weights.
#
# Sensitivities and point caps are positive magnitudes. The normalized curve
# signal is solely responsible for direction.
@progression_config %{
version: "0.1.0",
mandates: %{
championship_push: %{
curve_key: :championship_push_v0,
target: %{
type: :headroom,
intercept: 1.0,
reference_score: 80.0,
rate: 0.15,
min: 0.0,
max: 4.0
},
softness: 6.0,
upside_sensitivity: 0.03,
downside_sensitivity: 0.04,
upside_point_cap: 3.0,
downside_point_cap: 4.0,
default_policy: :longitudinal
},
run_it_back: %{
curve_key: :run_it_back_v0,
target: %{
type: :headroom,
intercept: 0.0,
reference_score: 75.0,
rate: 0.20,
min: -2.0,
max: 3.0
},
softness: 6.0,
upside_sensitivity: 0.02,
downside_sensitivity: 0.02,
upside_point_cap: 2.0,
downside_point_cap: 2.0,
default_policy: :maintenance
},
ascending: %{
curve_key: :ascending_v0,
target: %{
type: :headroom,
intercept: 2.0,
reference_score: 75.0,
rate: 0.10,
min: 2.0,
max: 5.0
},
softness: 7.0,
upside_sensitivity: 0.05,
downside_sensitivity: 0.06,
upside_point_cap: 4.0,
downside_point_cap: 5.0,
default_policy: :longitudinal
},
rebuilding: %{
curve_key: :rebuilding_v0,
target: %{
type: :headroom,
intercept: 0.0,
reference_score: 75.0,
rate: 0.15,
min: 3.0,
max: 7.0
},
softness: 6.0,
upside_sensitivity: 0.06,
downside_sensitivity: 0.08,
upside_point_cap: 5.0,
downside_point_cap: 6.0,
default_policy: :longitudinal
},
retool: %{
curve_key: :retool_v0,
target: %{type: :fixed, value: 2.0},
softness: 8.0,
upside_sensitivity: 0.02,
downside_sensitivity: 0.02,
upside_point_cap: 2.0,
downside_point_cap: 2.0,
default_policy: :episodic
},
selling: %{
curve_key: nil,
target: nil,
softness: nil,
upside_sensitivity: 0.0,
downside_sensitivity: 0.0,
upside_point_cap: 0.0,
downside_point_cap: 0.0,
default_policy: :episodic
}
}
}
# Competitive Strength reliability factor denominator (season minutes at which
# the factor reaches 1.0). AVC Pillar 1.
@reliability_minutes 1200
# Roster Construction experience factor denominator. AVC Pillar 2.
@experience_minutes 800
# Provisional replacement-level impact when no meaningful NBA sample exists.
# AVC Pillar 1 "Missing EPM Data".
@provisional_epm %{overall: -1.5, offensive: -0.75, defensive: -0.75}
# Minimum values per input before the blend tier will standardize against a
# season's population. Mirrors the regression tier's `--min-sample` default,
# and deliberately equals it: both tiers estimate an impact from a season
# population, so both need the same floor under it. Without it the two guards
# are backwards — the seasons the regression correctly refuses as biased are
# exactly the ones whose players then fall through to the blend.
@min_blend_population 30
# Numeric mapping of the existing categorical valuation confidence. v1 default
# pending calibration from out-of-sample errors (DECISIONS D-07).
@confidence_mapping %{high: 0.9, moderate: 0.6, low: 0.3}
# Minimum games played before a player's season is a usable comparable — and,
# since it is the same reliability question, before he is a priceable subject.
# Equals `PlayerSimilarity`'s documented default; declared here and passed to
# it explicitly so the grader's two uses of the number cannot drift from the
# engine's, without the grader restating a rule the engine owns.
@min_comparable_games 20
# Confidence decays with the projection horizon: a numeric confidence is
# multiplied by this per season ahead (DECISIONS D-18). The interpretation
# notes flag Confidence(i) vs Confidence(i,y) — this is the v1 answer to the
# second: the same model, trusted less the further out it is read.
@confidence_horizon_decay 0.9
# AVC's documented temporary age-curve adjustment for future-season expected
# salary: one flat multiplier per season ahead, chosen by the player's age *in
# that season* and compounded across the horizon (DECISIONS D-19). Ordered
# `{inclusive_max_age, factor}` bands, `nil` meaning "and above" — ordered
# because band resolution takes the first match, and a map would resolve in
# hash order. A player-specific aging curve is deferred; this is the v1
# fallback shape AVC documented, not a model.
@age_curve [{23, 1.03}, {28, 1.00}, {31, 0.97}, {nil, 0.92}]
# Contract Health multi-year discount weights (current season + four ahead).
# AVC Pillar 3.
@contract_season_weights [1.0, 0.9, 0.8, 0.7, 0.6]
# Long-Term Risk future-season weights (next season + three ahead). AVC Pillar 3.
@risk_season_weights [1.0, 0.9, 0.8, 0.7]
# Strong structural value floor for active standard rookie-scale seasons.
# AVC Pillar 3. The policy identifier names the exact classification
# interpretation paired with the baseline; both are exposed together by
# `rookie_contract_policy/1`, while `@versions.rookie_contract` is frozen into
# challenge snapshots.
@rookie_contract_classification_id "active_standard_rookie_scale_v1"
@rookie_market_blend %{minimum_games: 20, full_games: 82}
# Draft Asset Value present-value discount per year. AVC Pillar 5.
@dav_time_discount 0.92
# Universal prime midpoint age (v1 fallback until player-specific curves are
# calibrated). AVC Pillar 6.
@prime_midpoint_age 27.5
@doc "The overall grader model version frozen onto every grade."
@spec model_version() :: String.t()
def model_version, do: @model_version
@doc "Map of every sub-model version, frozen onto a grade's snapshot refs."
@spec versions() :: %{atom() => String.t()}
def versions, do: @versions
@doc "The separately versioned proposed Progression model configuration."
@spec progression_config(map()) :: map()
def progression_config(override \\ %{}),
do: get(override, :progression, @progression_config)
@doc "Reliability-factor denominator (season minutes → factor 1.0). AVC Pillar 1."
@spec reliability_minutes(map()) :: number()
def reliability_minutes(override \\ %{}),
do: get(override, :reliability_minutes, @reliability_minutes)
@doc "Experience-factor denominator. AVC Pillar 2."
@spec experience_minutes(map()) :: number()
def experience_minutes(override \\ %{}),
do: get(override, :experience_minutes, @experience_minutes)
@doc "Provisional replacement-level EPM (overall/offensive/defensive)."
@spec provisional_epm(map()) :: %{overall: number(), offensive: number(), defensive: number()}
def provisional_epm(override \\ %{}), do: get(override, :provisional_epm, @provisional_epm)
@doc "Minimum per-input population the blend tier will standardize against."
@spec min_blend_population(map()) :: number()
def min_blend_population(override \\ %{}),
do: get(override, :min_blend_population, @min_blend_population)
@doc "Categorical→numeric confidence mapping (DECISIONS D-07)."
@spec confidence_mapping(map()) :: %{high: number(), moderate: number(), low: number()}
def confidence_mapping(override \\ %{}),
do: get(override, :confidence_mapping, @confidence_mapping)
@doc "Minimum games played for a season to be a usable comparable *or* a priceable subject."
@spec min_comparable_games(map()) :: number()
def min_comparable_games(override \\ %{}),
do: get(override, :min_comparable_games, @min_comparable_games)
@doc "Per-season-ahead confidence decay for projected salaries (DECISIONS D-18)."
@spec confidence_horizon_decay(map()) :: number()
def confidence_horizon_decay(override \\ %{}),
do: get(override, :confidence_horizon_decay, @confidence_horizon_decay)
@doc """
Age-curve bands for the temporary multi-year projection (DECISIONS D-19).
An ordered list of `{inclusive_max_age, per-season factor}`; the last band's
`nil` max means "and above". Read by
`TradeMachine.GMGradeV2.ExpectedSalary.age_factor/2`.
"""
@spec age_curve(map()) :: [{pos_integer() | nil, number()}]
def age_curve(override \\ %{}), do: get(override, :age_curve, @age_curve)
@doc "Contract Value multi-year discount weights."
@spec contract_season_weights(map()) :: [number()]
def contract_season_weights(override \\ %{}),
do: get(override, :contract_season_weights, @contract_season_weights)
@doc "Long-Term Risk future-season weights."
@spec risk_season_weights(map()) :: [number()]
def risk_season_weights(override \\ %{}),
do: get(override, :risk_season_weights, @risk_season_weights)
@doc "Structural Contract Health baseline for active standard rookie-scale seasons."
@spec rookie_contract_baseline(map()) :: number()
def rookie_contract_baseline(override \\ %{}),
do:
get(
override,
:rookie_contract_baseline,
ContractClassification.rookie_baseline_score() * 1.0
)
@doc "Stable identifier for the rookie-contract classification policy."
@spec rookie_contract_classification_id() :: String.t()
def rookie_contract_classification_id, do: @rookie_contract_classification_id
@doc """
Returns whether a frozen snapshot opted into the current rookie policy
version. Full policy inputs are frozen separately in the snapshot document.
"""
@spec rookie_contract_policy_active?(map()) :: boolean()
def rookie_contract_policy_active?(model_versions) when is_map(model_versions) do
version =
Map.get(model_versions, :rookie_contract) ||
Map.get(model_versions, "rookie_contract")
version == Map.fetch!(@versions, :rookie_contract)
end
def rookie_contract_policy_active?(_model_versions), do: false
@doc """
Frozen metadata for the rookie-contract classification and scoring policy.
The complete returned payload is captured in every new challenge snapshot.
This freezes the draft curve and blend boundaries, rather than relying on the
live coefficient registry when an old challenge is graded later.
"""
@spec rookie_contract_policy(map()) :: %{
version: String.t(),
classification_id: String.t(),
fallback_baseline: number(),
minimum_market_games: pos_integer(),
full_market_games: pos_integer(),
pick_curve: map()
}
def rookie_contract_policy(override \\ %{}) do
%{
version: Map.fetch!(@versions, :rookie_contract),
classification_id: @rookie_contract_classification_id,
fallback_baseline: rookie_contract_baseline(override),
minimum_market_games:
get(override, :rookie_minimum_market_games, @rookie_market_blend.minimum_games),
full_market_games:
get(override, :rookie_full_market_games, @rookie_market_blend.full_games),
pick_curve: get(override, :rookie_pick_curve, Coefficients.for(:pick_curve))
}
end
@doc "Returns the exact rookie policy frozen into a snapshot document."
@spec rookie_contract_policy_for_snapshot(map()) :: map() | nil
def rookie_contract_policy_for_snapshot(snapshot_data) when is_map(snapshot_data) do
case Map.get(snapshot_data, "rookie_contract_policy") ||
Map.get(snapshot_data, :rookie_contract_policy) do
policy when is_map(policy) ->
policy
_missing ->
versions = Map.get(snapshot_data, "model_versions", %{})
case Map.get(versions, "rookie_contract") || Map.get(versions, :rookie_contract) do
"1.0.0" ->
%{
"version" => "1.0.0",
"classification_id" => "active_standard_rookie_scale_v1",
"baseline" => 75.0
}
_unknown ->
nil
end
end
end
def rookie_contract_policy_for_snapshot(_snapshot_data), do: nil
@doc "Returns the structural baseline applicable to one player in a frozen snapshot."
@spec rookie_contract_baseline_for_snapshot(map(), String.t()) :: number() | nil
def rookie_contract_baseline_for_snapshot(snapshot_data, player_id)
when is_map(snapshot_data) and is_binary(player_id) do
policy = rookie_contract_policy_for_snapshot(snapshot_data)
players = Map.get(snapshot_data, "players") || Map.get(snapshot_data, :players) || %{}
subject = Map.get(players, player_id) || %{}
cond do
is_number(policy_value(policy, :baseline)) ->
policy_value(policy, :baseline)
is_number(policy_value(policy, :fallback_baseline)) and
TradeMachine.Valuation.RookieContractPolicy.draft_prior(subject, policy) == :error ->
policy_value(policy, :fallback_baseline)
true ->
nil
end
end
def rookie_contract_baseline_for_snapshot(_snapshot_data, _player_id), do: nil
@doc "DAV present-value discount per year (AVC 0.92)."
@spec dav_time_discount(map()) :: number()
def dav_time_discount(override \\ %{}),
do: get(override, :dav_time_discount, @dav_time_discount)
@doc "Universal prime midpoint age (v1 fallback, AVC 27.5)."
@spec prime_midpoint_age(map()) :: number()
def prime_midpoint_age(override \\ %{}),
do: get(override, :prime_midpoint_age, @prime_midpoint_age)
# Lab override lookup: an override map (Phase 16) can replace any constant by
# key; production passes an empty map and gets the frozen default.
defp get(override, key, default) when is_map(override) do
case Map.fetch(override, key) do
{:ok, value} -> value
:error -> Map.get(override, Atom.to_string(key), default)
end
end
defp policy_value(policy, key) when is_map(policy) do
Map.get(policy, key) || Map.get(policy, Atom.to_string(key))
end
defp policy_value(_policy, _key), do: nil
end
challenge_snapshot.exElixir
lib/trade_machine/gm_grade_v2/challenge_snapshot.ex
The frozen input set for a single GM Challenge: every value a submission is scored against, captured once when the challenge is prepared.
A challenge is a competition, so the requirement is stronger than reproducibility — it is **fairness**. Two users submitting a week apart must be graded against identical evidence. Without this row the first entrant would be scored against one week's player stats and valuations and the second against another, and the leaderboard would rank the calendar as much as the roster.
The complete evidence set
"Identical evidence" is a claim about *every* input, so it is only true if the list is exhaustive — a grade that mixes frozen player stats with a live salary is still calendar-dependent, just harder to notice. Four things decide a challenge grade, and this row now holds all four:
* **how good each player is** — the
"players"
impact evidence, plus the
"expected_salaries"
market estimates that price it
* **what the cap allows** — the
"thresholds"
table, read by the cap engine
and by Plausibility and Contract Health alike
* **what each player costs** —
"rosters"
: contract rows for the league,
plus each team's persisted buyouts and trade exceptions
* **who owns which draft rights** —
"pick_graph"
: every pick, part and
swap group, which every asset-management pillar is valued from
The last two arrived after some snapshots were already written, and an
insert-only row cannot be back-filled. Those documents therefore grade from
live reads rather than not at all — and say so:
ScenarioAssembly
records
"evidence_source" => "live_fallback"
in the grade's
snapshot_refs
, naming
the inputs it had to read live, where a fully-frozen document records
"frozen"
. A challenge still open when a section ships should have its
snapshot rebuilt; the marker is how you find the grades that need it.
Ordinary challenge setup creates one snapshot. An explicit audited regrade may
retain that row and append a rebuilt immutable snapshot; reads select the
newest row while grade revisions permanently retain the exact row they used.
Each snapshot points at the
TradeMachine.GMGradeV2.Benchmark
in force
(
on_delete: :restrict
, so that benchmark can never be deleted out from under
the grades that cite it).
`data` sections
*
"players"
— per-player frozen impact / skill / valuation inputs
*
"thresholds"
— cap, tax and apron thresholds by season
*
"mandate_profile"
— a *copy* of the challenge's mandate profile, not a
reference: the profile is versioned and editable, and a challenge already
being played must not silently change what it was asking for
*
"model_versions"
—
TradeMachine.GMGradeV2.ModelVersion.versions/0
as of
capture, so a grade can name every constant that produced it
*
"rookie_contract_policy"
— the complete draft-slot curve, fallback, and
NBA-evidence blend frozen for new snapshots. Historical v1 snapshots omit
it and replay through the versioned legacy policy.
*
"expected_salaries"
— per-player market estimate and per-season
projection, built by
TradeMachine.GMGradeV2.ExpectedSalary.snapshot_section/1
(Phase 06). Required for the same reason as the rest: Contract Health, PTV
and plausibility read *only* this section, so a snapshot without it grades
every entrant's contracts against nothing.
*
"season"
— season metadata including ISO-8601
"start_date"
, freezing
the reference date for whole-year player ages (D-12)
*
"rosters"
— the league's contract evidence: every rostered player's row
with its contracts and contract seasons, plus each team's persisted
buyouts and trade exceptions. Built and read back by
TradeMachine.GMGradeV2.FrozenRoster
, which documents why the whole row
is frozen. This is the cap-math counterpart to
"players"
: that section
says how good a player is, this one says what he costs. Optional, because
snapshots written before it existed cannot be back-filled — those grade
from live reads and say so (see
optional_section/2
).
*
"pick_graph"
— the league's draft rights: every pick, every part, and
every swap group with its members and outcomes. Built and read back by
TradeMachine.GMGradeV2.FrozenPickGraph
. Every asset-management pillar is
valued from it, and unlike a roster change a pick changing hands fails no
validation — it simply stops appearing in a total. Optional for the same
reason
"rosters"
is.
Later phases may freeze more; the schema requires the first six and permits the rest.
Every section must be non-empty, and that is not an oversight to relax
Unlike a benchmark's
dav
— whose emptiness is a documented state meaning
"Phase 07 has not run" (D-35) — none of these six has a reading in which it is
legitimately empty. Each is evidence the grader reads, so an empty one does not
fail: it scores every entrant in the challenge as though that whole input were
simply absent, producing a uniformly wrong leaderboard that looks like a valid
one. D-52's rule is that emptiness is meaningful only where it is *documented*,
so all six are required non-empty. If a later phase finds a real empty state
for one of them, that is a decision to record, not a constraint to quietly
loosen — the exemptions are where a fail-closed validator leaks.
expected_salaries
is the sharpest case, because it has a state that *looks*
like emptiness and is not: a player the market model cannot price is stored
with
"status" => "unavailable"
, present and explicit. The section being empty
means the builder never ran; a player being unavailable means it ran and found
nothing to say. Only the second is a legitimate answer, which is why the first
is rejected here.
Insert-only
Snapshots are never updated. That is the entire point — a mutable snapshot is
not a snapshot — and it is why
TradeMachine.GMGradeV2.Benchmarks
exposes
insert_snapshot/1
with no update counterpart, and why this module offers only
insert_changeset/2
(D-37). Correcting a bad snapshot means deleting the
challenge's snapshot and inserting a new one, deliberately: an explicit
destroy-and-recreate leaves the grades that cited it visibly orphaned, where an
in-place edit would leave them looking valid while meaning something else.
SHA-256 7c030f58be13163967ad9a7c28718490f2a623dc3d82a9b174f7a31b27ac3532
defmodule TradeMachine.GMGradeV2.ChallengeSnapshot do
@moduledoc """
The frozen input set for a single GM Challenge: every value a submission is
scored against, captured once when the challenge is prepared.
A challenge is a competition, so the requirement is stronger than
reproducibility — it is **fairness**. Two users submitting a week apart must be
graded against identical evidence. Without this row the first entrant would be
scored against one week's player stats and valuations and the second against
another, and the leaderboard would rank the calendar as much as the roster.
## The complete evidence set
"Identical evidence" is a claim about *every* input, so it is only true if the
list is exhaustive — a grade that mixes frozen player stats with a live salary
is still calendar-dependent, just harder to notice. Four things decide a
challenge grade, and this row now holds all four:
* **how good each player is** — the `"players"` impact evidence, plus the
`"expected_salaries"` market estimates that price it
* **what the cap allows** — the `"thresholds"` table, read by the cap engine
and by Plausibility and Contract Health alike
* **what each player costs** — `"rosters"`: contract rows for the league,
plus each team's persisted buyouts and trade exceptions
* **who owns which draft rights** — `"pick_graph"`: every pick, part and
swap group, which every asset-management pillar is valued from
The last two arrived after some snapshots were already written, and an
insert-only row cannot be back-filled. Those documents therefore grade from
live reads rather than not at all — and say so: `ScenarioAssembly` records
`"evidence_source" => "live_fallback"` in the grade's `snapshot_refs`, naming
the inputs it had to read live, where a fully-frozen document records
`"frozen"`. A challenge still open when a section ships should have its
snapshot rebuilt; the marker is how you find the grades that need it.
Ordinary challenge setup creates one snapshot. An explicit audited regrade may
retain that row and append a rebuilt immutable snapshot; reads select the
newest row while grade revisions permanently retain the exact row they used.
Each snapshot points at the `TradeMachine.GMGradeV2.Benchmark` in force
(`on_delete: :restrict`, so that benchmark can never be deleted out from under
the grades that cite it).
## `data` sections
* `"players"` — per-player frozen impact / skill / valuation inputs
* `"thresholds"` — cap, tax and apron thresholds by season
* `"mandate_profile"` — a *copy* of the challenge's mandate profile, not a
reference: the profile is versioned and editable, and a challenge already
being played must not silently change what it was asking for
* `"model_versions"` — `TradeMachine.GMGradeV2.ModelVersion.versions/0` as of
capture, so a grade can name every constant that produced it
* `"rookie_contract_policy"` — the complete draft-slot curve, fallback, and
NBA-evidence blend frozen for new snapshots. Historical v1 snapshots omit
it and replay through the versioned legacy policy.
* `"expected_salaries"` — per-player market estimate and per-season
projection, built by `TradeMachine.GMGradeV2.ExpectedSalary.snapshot_section/1`
(Phase 06). Required for the same reason as the rest: Contract Health, PTV
and plausibility read *only* this section, so a snapshot without it grades
every entrant's contracts against nothing.
* `"season"` — season metadata including ISO-8601 `"start_date"`, freezing
the reference date for whole-year player ages (D-12)
* `"rosters"` — the league's contract evidence: every rostered player's row
with its contracts and contract seasons, plus each team's persisted
buyouts and trade exceptions. Built and read back by
`TradeMachine.GMGradeV2.FrozenRoster`, which documents why the whole row
is frozen. This is the cap-math counterpart to `"players"`: that section
says how good a player is, this one says what he costs. Optional, because
snapshots written before it existed cannot be back-filled — those grade
from live reads and say so (see `optional_section/2`).
* `"pick_graph"` — the league's draft rights: every pick, every part, and
every swap group with its members and outcomes. Built and read back by
`TradeMachine.GMGradeV2.FrozenPickGraph`. Every asset-management pillar is
valued from it, and unlike a roster change a pick changing hands fails no
validation — it simply stops appearing in a total. Optional for the same
reason `"rosters"` is.
Later phases may freeze more; the schema requires the first six and permits
the rest.
## Every section must be non-empty, and that is not an oversight to relax
Unlike a benchmark's `dav` — whose emptiness is a documented state meaning
"Phase 07 has not run" (D-35) — none of these six has a reading in which it is
legitimately empty. Each is evidence the grader reads, so an empty one does not
fail: it scores every entrant in the challenge as though that whole input were
simply absent, producing a uniformly wrong leaderboard that looks like a valid
one. D-52's rule is that emptiness is meaningful only where it is *documented*,
so all six are required non-empty. If a later phase finds a real empty state
for one of them, that is a decision to record, not a constraint to quietly
loosen — the exemptions are where a fail-closed validator leaks.
`expected_salaries` is the sharpest case, because it has a state that *looks*
like emptiness and is not: a player the market model cannot price is stored
with `"status" => "unavailable"`, present and explicit. The section being empty
means the builder never ran; a player being unavailable means it ran and found
nothing to say. Only the second is a legitimate answer, which is why the first
is rejected here.
## Insert-only
Snapshots are never updated. That is the entire point — a mutable snapshot is
not a snapshot — and it is why `TradeMachine.GMGradeV2.Benchmarks` exposes
`insert_snapshot/1` with no update counterpart, and why this module offers only
`insert_changeset/2` (D-37). Correcting a bad snapshot means deleting the
challenge's snapshot and inserting a new one, deliberately: an explicit
destroy-and-recreate leaves the grades that cited it visibly orphaned, where an
in-place edit would leave them looking valid while meaning something else.
"""
use Ecto.Schema
import Ecto.Changeset
alias TradeMachine.GMGradeV2.FrozenDocument
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
@type t :: %__MODULE__{}
schema "challenge_snapshots" do
field :data, :map
belongs_to :challenge, TradeMachine.Challenges.Challenge
belongs_to :benchmark, TradeMachine.GMGradeV2.Benchmark
# Matches the migration — see `TradeMachine.GMGradeV2.Benchmark`.
timestamps(type: :utc_datetime_usec)
end
@castable ~w(challenge_id benchmark_id data)a
@doc """
Builds the one and only changeset a snapshot ever gets.
Named `insert_changeset` rather than `changeset` on purpose: the conventional
name invites `Repo.update/1`, and the name is the cheapest place to say that
this row has no update path (D-37).
`data` is canonicalized and schema-validated by the same fail-closed pipeline
the benchmark uses — one implementation, so the two frozen tables cannot drift
into enforcing different things (D-54).
"""
def insert_changeset(snapshot, attrs) do
snapshot
|> cast(attrs, @castable)
|> validate_required(@castable)
|> FrozenDocument.validate_document(:data, &data_schema/1)
|> foreign_key_constraint(:challenge_id)
|> foreign_key_constraint(:benchmark_id)
end
@doc "The `data` sections every snapshot must carry, in document order."
@spec required_sections() :: [String.t()]
def required_sections do
{:map, entries} = data_schema(%{})
Enum.map(entries, fn {name, _spec} -> name end)
end
@doc """
The required shape of a canonicalized `data` document.
Takes the document for signature parity with
`TradeMachine.GMGradeV2.Benchmark.data_schema/1` — a snapshot has no
document-relative cardinality to resolve, so nothing here reads it. Public so
the D-54 property test enumerates required paths from the schema rather than
from a restated list.
"""
@spec data_schema(term()) :: FrozenDocument.spec()
def data_schema(_canonical) do
non_empty_map = {:map, [], min_size: 1}
{:map,
[
{"players", non_empty_map},
{"thresholds", non_empty_map},
{"mandate_profile", non_empty_map},
{"model_versions", non_empty_map},
{"expected_salaries", non_empty_map},
{"season", non_empty_map}
]}
end
@doc """
Fetches a `data` section by its atom name.
Raises `KeyError` on a miss rather than defaulting — see
`TradeMachine.GMGradeV2.Benchmark.section/2` for why a silent default is the
dangerous choice here. Only string keys are looked up, because that is the only
spelling a snapshot is ever stored in.
"""
@spec section(t(), atom()) :: term()
def section(%__MODULE__{data: data}, section) when is_atom(section) do
Map.fetch!(data, Atom.to_string(section))
end
@doc """
Fetches a section a snapshot may legitimately not carry, or `nil`.
The counterpart to `section/2`, not a softer spelling of it. A section is
optional here for exactly one reason: it was introduced after some snapshots
were already written, and those rows can never be back-filled because
snapshots are insert-only. `nil` therefore means "this document predates the
section" — a real, dateable state — where a miss on a required section means
the builder is broken, which is why that one still raises.
A consumer reading through this owes the caller an accounting of what it did
instead; see the `"evidence_source"` diagnostics on a grade result.
"""
@spec optional_section(t(), atom()) :: term() | nil
def optional_section(%__MODULE__{data: data}, section) when is_atom(section) do
Map.get(data, Atom.to_string(section))
end
end
challenge_snapshot_builder.exElixir
lib/trade_machine/gm_grade_v2/challenge_snapshot_builder.ex
Builds the immutable evidence document used by one authored GM Challenge.
The builder reads the latest completed-season benchmark available at or before the authored season, freezes league-wide player evidence, contract evidence and expected salaries, and pins the challenge's model and mandate-profile versions in the same transaction as the write-once snapshot insert.
SHA-256 1d2a07f557c92c7e2c1118924b34c9049375a860c0793c11f8aa4cdb4d82ddc8
defmodule TradeMachine.GMGradeV2.ChallengeSnapshotBuilder do
@moduledoc """
Builds the immutable evidence document used by one authored GM Challenge.
The builder reads the latest completed-season benchmark available at or before
the authored season, freezes league-wide player evidence, contract evidence
and expected salaries, and pins the challenge's model and mandate-profile
versions in the same transaction as the write-once snapshot insert.
"""
alias TradeMachine.Challenges
alias TradeMachine.Challenges.Challenge
alias TradeMachine.DataRefresh
alias TradeMachine.GMGradeV2.BenchmarkBuilder
alias TradeMachine.GMGradeV2.Benchmarks
alias TradeMachine.GMGradeV2.CapState
alias TradeMachine.GMGradeV2.ExpectedSalary
alias TradeMachine.GMGradeV2.FrozenPickGraph
alias TradeMachine.GMGradeV2.FrozenRoster
alias TradeMachine.GMGradeV2.MandateProfile
alias TradeMachine.GMGradeV2.ModelVersion
alias TradeMachine.GMGradeV2.PlayerImpact
alias TradeMachine.GMGradeV2.Pillars.AssetManagement.JSONSafe
alias TradeMachine.CBA.CapCalculator
alias TradeMachine.NBA
alias TradeMachine.Repo
@doc """
Freezes all grading evidence for a draft challenge.
Returns `{:error, :benchmark_missing}` when no benchmark has been built at or
before the selected season and `{:error, :incomplete_authoring}` when required
grading configuration is absent.
Returns `{:error, :snapshot_conflict}` when the challenge was edited while its
own evidence was being read. The build reads under REPEATABLE READ so the
frozen league is internally consistent, and that is the isolation level's way
of refusing a write it can no longer serialize. Nothing was written; build
again.
"""
@spec build(Challenge.t()) ::
{:ok, TradeMachine.GMGradeV2.ChallengeSnapshot.t()}
| {:error, atom() | Ecto.Changeset.t()}
def build(%Challenge{} = challenge) do
started_at = timestamp()
result =
try do
do_build(challenge)
rescue
error ->
if serialization_conflict?(error) do
# Recorded by the `case` below like any other failed build, so this
# branch only decides that it is an outcome rather than a crash.
{:error, :snapshot_conflict}
else
record_run(started_at, "failed", challenge, {:exception, Exception.message(error)})
reraise(error, __STACKTRACE__)
end
end
case result do
{:ok, _snapshot} ->
record_run(started_at, "ok", challenge, result)
{:error, reason} when reason in [:incomplete_authoring, :snapshot_exists] ->
:ok
{:error, _reason} ->
record_run(started_at, "failed", challenge, result)
end
result
end
defp do_build(%Challenge{mandate_key: nil}), do: {:error, :incomplete_authoring}
defp do_build(%Challenge{season: nil}), do: {:error, :incomplete_authoring}
defp do_build(%Challenge{asset_valuation_date: nil}), do: {:error, :incomplete_authoring}
defp do_build(%Challenge{} = challenge) do
with false <- Benchmarks.snapshot_exists?(challenge.id),
benchmark when not is_nil(benchmark) <-
Benchmarks.latest_benchmark_at_or_before(challenge.season) do
Repo.transaction(fn ->
request_repeatable_read()
data =
case build_data(challenge, benchmark) do
{:ok, data} -> data
{:error, reason} -> Repo.rollback(reason)
end
attrs = %{
challenge_id: challenge.id,
benchmark_id: benchmark.id,
data: data
}
with {:ok, snapshot} <- Benchmarks.insert_snapshot(attrs),
{:ok, _challenge} <-
Challenges.update_challenge(challenge, %{
model_version: ModelVersion.model_version(),
mandate_profile_version: MandateProfile.current_version()
}) do
snapshot
else
{:error, reason} -> Repo.rollback(reason)
end
end)
|> unwrap_transaction()
else
true -> {:error, :snapshot_exists}
nil -> {:error, :benchmark_missing}
end
end
# REPEATABLE READ trades a blocking write for an aborted one: where READ
# COMMITTED would have waited and re-read, Postgres raises 40001 if the
# challenge row this transaction updates at the end was changed by someone
# else while the evidence was being read. Nothing is written when it does —
# the transaction rolls back whole — so the honest report is a failed build
# the operator can simply repeat, not a crashed LiveView. Deliberately not
# retried in a loop: a conflict means an admin edited this challenge during
# its own snapshot build, and silently rebuilding against the row they just
# changed is worse than telling them.
defp serialization_conflict?(%Postgrex.Error{postgres: %{code: :serialization_failure}}),
do: true
defp serialization_conflict?(_error), do: false
# The evidence build is a dozen statements — players, their contract and
# season preloads, buyouts, trade exceptions, picks, parts, swap groups. Under
# READ COMMITTED each takes its own snapshot, so an *atomic* write landing
# between two of them is captured half-applied and then frozen forever:
# `NBA.waive_player/2` creates a buyout and deletes the contract in one
# transaction, so a build that reads contracts before it and buyouts after it
# records the salary *and* the dead money, and double-counts that payroll for
# every entrant in the challenge. REPEATABLE READ pins one league for the
# whole build.
#
# It cannot be requested under the test sandbox: there the connection's
# transaction belongs to the sandbox and has already run queries, so Postgres
# rejects `SET TRANSACTION ISOLATION LEVEL` (25001). That is a property of the
# pool, not of this code, so it is what gets checked — the production path has
# no flag to be misconfigured. `FrozenPickGraph.validate_closed!/3` stays as
# the defence in depth that *is* exercised by tests.
defp request_repeatable_read do
if Application.get_env(:trade_machine, Repo)[:pool] != Ecto.Adapters.SQL.Sandbox do
Repo.query!("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ")
end
:ok
end
@doc """
Builds the canonical frozen document without inserting it.
The calibration lab uses this read-only seam to grade actual teams through
the same evidence builder as authored challenges.
"""
@spec build_data(Challenge.t(), TradeMachine.GMGradeV2.Benchmark.t(), keyword()) ::
{:ok, map()} | {:error, :player_data_missing | :threshold_data_missing}
def build_data(%Challenge{} = challenge, benchmark, opts \\ []) do
rows = BenchmarkBuilder.load_rows(benchmark.season)
if rows == [] do
{:error, :player_data_missing}
else
official_thresholds = NBA.list_salary_cap_thresholds() |> Map.new(&{&1.season, &1})
case frozen_threshold_horizon(challenge.season, official_thresholds) do
{:ok, thresholds} ->
player_ids = Enum.map(rows, & &1.player_id)
data =
%{
"players" => player_evidence(rows),
"thresholds" => JSONSafe.value(thresholds),
"mandate_profile" => frozen_mandate(challenge.mandate_key),
"model_versions" => JSONSafe.value(ModelVersion.versions()),
"rookie_contract_policy" => JSONSafe.value(ModelVersion.rookie_contract_policy()),
"expected_salaries" => expected_salaries(player_ids, thresholds, opts),
# The cap-math half of the league evidence. Populated with the
# impact section's players as well as the rostered ones so the
# two describe the same league — see `FrozenRoster`.
"rosters" => FrozenRoster.build(player_ids),
# The draft half. Every asset pillar is valued from this graph, so
# a backstage pick edit mid-challenge otherwise revalues one
# entrant's inventory and not another's — see `FrozenPickGraph`.
"pick_graph" => FrozenPickGraph.build(),
"season" => %{
"label" => benchmark.season,
"start_date" => Date.to_iso8601(challenge.asset_valuation_date)
}
}
|> JSONSafe.value()
|> Jason.encode!()
|> Jason.decode!()
{:ok, data}
{:error, :threshold_data_missing} = error ->
error
end
end
end
defp expected_salaries(player_ids, thresholds, opts) do
case Keyword.get(opts, :expected_salaries, :calculate) do
:calculate ->
ExpectedSalary.snapshot_section(
player_ids: player_ids,
thresholds_by_season: thresholds
)
:unavailable ->
ExpectedSalary.unavailable_section(player_ids)
end
end
defp frozen_threshold_horizon(challenge_season, official_thresholds) do
case Map.get(official_thresholds, challenge_season) do
nil ->
{:error, :threshold_data_missing}
current_threshold ->
projection = CapCalculator.build_payroll_projection([], current_threshold, %{}, false)
# Deliberately the whole official table and not just the horizon: the
# cap engine tiers a declined option's hold against the season the
# player last earned, which drops behind the horizon at the next
# rollover. `CapState.frozen_thresholds/2` documents the re-tiering.
{:ok, CapState.frozen_thresholds(projection, official_thresholds)}
end
end
defp player_evidence(rows) do
context = %{
regression: PlayerImpact.load_regression_table(),
population: %{hd(rows).season => BenchmarkBuilder.population_moments(rows)}
}
Map.new(rows, fn row ->
impact = PlayerImpact.adjust(row, context)
evidence =
row
|> Map.merge(impact)
|> Map.put(:name, "#{row.first_name} #{row.last_name}")
|> Map.put(:season_minutes, PlayerImpact.season_minutes(row))
|> JSONSafe.value()
{row.player_id, evidence}
end)
end
defp frozen_mandate(key) do
mandate_key =
Enum.find(MandateProfile.keys(), fn known_key ->
Atom.to_string(known_key) == key
end)
case mandate_key do
nil -> raise ArgumentError, "unknown mandate profile #{inspect(key)}"
known_key -> known_key |> MandateProfile.get!() |> JSONSafe.value()
end
end
defp unwrap_transaction({:ok, snapshot}), do: {:ok, snapshot}
defp unwrap_transaction({:error, reason}), do: {:error, reason}
defp record_run(started_at, status, challenge, result) do
DataRefresh.record_run!(%{
type: "gm_grade_snapshot_build",
status: status,
started_at: started_at,
finished_at: timestamp(),
error_count: if(status == "ok", do: 0, else: 1),
stats: %{
challenge_id: challenge.id,
season: challenge.season,
result: inspect_result(result)
}
})
end
defp inspect_result({:ok, snapshot}), do: %{"snapshot_id" => snapshot.id}
defp inspect_result({:error, reason}), do: %{"error" => inspect(reason)}
defp inspect_result({:exception, message}), do: %{"exception" => message}
defp inspect_result(other), do: %{"result" => inspect(other)}
defp timestamp, do: DateTime.utc_now() |> DateTime.truncate(:second)
end
frozen_document.exElixir
lib/trade_machine/gm_grade_v2/frozen_document.ex
Canonicalization and schema validation for the two write-once
data
documents
—
TradeMachine.GMGradeV2.Benchmark
and
TradeMachine.GMGradeV2.ChallengeSnapshot
.
Both columns are jsonb, both rows are frozen at insert and never updated, and neither can be corrected afterwards — only superseded. That makes the *default* answer for an unrecognised document the whole design question, and D-54 answers it: **canonicalize first, then validate a required schema, fail-closed.**
Why this is one walker and not a pile of guards
The previous implementation grew a new guard per review round — the 30-team
invariant (rounds 1–3), then
dav
/
meta
type-checking (round 4), then
skills.player_percentiles
and the dual
:skills
/
"skills"
key spellings
(round 5, which *defeated* the round-4 remediation). Five rounds, the same
finding class at a new line each time, because enumerating bad shapes is
strictly weaker than declaring the good one. Nothing here validates a shape it
was told about; it validates against a declared schema and treats everything
else as invalid.
Two mechanisms, in order:
1.
canonicalize/1
normalizes the document to string keys and **refuses**
what it cannot normalize cleanly. A map may legally hold
:skills
and
"skills"
at once — Elixir treats them as two keys, **jsonb does not**:
encoding emits the name twice and Postgres keeps the last. So a document
whose atom-keyed section is well-formed can validate, get written, and
read back as whatever the string-keyed duplicate held; the more carefully
a validator checks the spelling it resolved, the more confidently it
approves a row that will not contain it. There is no reading of "both
spellings present" more likely to be intended than to be a builder
assembling a document two different ways, so it is refused rather than
resolved by a precedence rule. Callers store the canonical form, which is
what makes the in-memory document and the read-back document the same
object (D-42).
2.
validate/2
walks a declared
t:spec/0
whose final clause is an
**error**, never
[]
. A value that matches no clause is invalid by
default. That single property is what stops the next unanticipated shape
from being the sixth review round.
The grammar is deliberately tiny — three node types, no escape hatch. There is
no "anything goes" node, because that is precisely the fall-through this module
exists to remove; an unconstrained leaf is expressed as the constraint that is
actually meant (
{:map, []}
— an object, contents unconstrained).
Spec grammar
{:map, entries} # a map; every named entry present and valid
{:map, entries, opts} # ... plus `min_size:`
{:list, opts} # a list; `of:` `min_length:` `length:`
entries
is a **list** of
{name, spec}
pairs rather than a map, so error
order is authored rather than hash order (D-50), and extra keys are permitted:
a later phase must be able to freeze additional sections or diagnostics without
this module being the thing that blocks it (Phase 07 fills
dav
; Phase 06
added
expected_salaries
to the snapshot and then declared it required).
What "required" means here
Required is about the keys a *consumer reads*, not about non-emptiness. The
distinction is D-52's, and it is why the two look different in the schemas:
dav => %{}
is a documented state ("Phase 07 has not run"), so
dav
is
{:map, []}
; an empty
metrics
map is not a state — those arrays *are* the
percentile tables every later grade is scored against, so a section carrying a
full
teams
list and no arrays reads back as a well-formed document that can
score nothing. Both are
%{}
; only one may pass.
SHA-256 0c211e77369b09a76ee28284e879d0303dc5bf0bedf64aed1741c653eaa50fde
defmodule TradeMachine.GMGradeV2.FrozenDocument do
@moduledoc """
Canonicalization and schema validation for the two write-once `data` documents
— `TradeMachine.GMGradeV2.Benchmark` and
`TradeMachine.GMGradeV2.ChallengeSnapshot`.
Both columns are jsonb, both rows are frozen at insert and never updated, and
neither can be corrected afterwards — only superseded. That makes the *default*
answer for an unrecognised document the whole design question, and D-54 answers
it: **canonicalize first, then validate a required schema, fail-closed.**
## Why this is one walker and not a pile of guards
The previous implementation grew a new guard per review round — the 30-team
invariant (rounds 1–3), then `dav`/`meta` type-checking (round 4), then
`skills.player_percentiles` and the dual `:skills`/`"skills"` key spellings
(round 5, which *defeated* the round-4 remediation). Five rounds, the same
finding class at a new line each time, because enumerating bad shapes is
strictly weaker than declaring the good one. Nothing here validates a shape it
was told about; it validates against a declared schema and treats everything
else as invalid.
Two mechanisms, in order:
1. `canonicalize/1` normalizes the document to string keys and **refuses**
what it cannot normalize cleanly. A map may legally hold `:skills` and
`"skills"` at once — Elixir treats them as two keys, **jsonb does not**:
encoding emits the name twice and Postgres keeps the last. So a document
whose atom-keyed section is well-formed can validate, get written, and
read back as whatever the string-keyed duplicate held; the more carefully
a validator checks the spelling it resolved, the more confidently it
approves a row that will not contain it. There is no reading of "both
spellings present" more likely to be intended than to be a builder
assembling a document two different ways, so it is refused rather than
resolved by a precedence rule. Callers store the canonical form, which is
what makes the in-memory document and the read-back document the same
object (D-42).
2. `validate/2` walks a declared `t:spec/0` whose final clause is an
**error**, never `[]`. A value that matches no clause is invalid by
default. That single property is what stops the next unanticipated shape
from being the sixth review round.
The grammar is deliberately tiny — three node types, no escape hatch. There is
no "anything goes" node, because that is precisely the fall-through this module
exists to remove; an unconstrained leaf is expressed as the constraint that is
actually meant (`{:map, []}` — an object, contents unconstrained).
## Spec grammar
{:map, entries} # a map; every named entry present and valid
{:map, entries, opts} # ... plus `min_size:`
{:list, opts} # a list; `of:` `min_length:` `length:`
`entries` is a **list** of `{name, spec}` pairs rather than a map, so error
order is authored rather than hash order (D-50), and extra keys are permitted:
a later phase must be able to freeze additional sections or diagnostics without
this module being the thing that blocks it (Phase 07 fills `dav`; Phase 06
added `expected_salaries` to the snapshot and then declared it required).
## What "required" means here
Required is about the keys a *consumer reads*, not about non-emptiness. The
distinction is D-52's, and it is why the two look different in the schemas:
`dav => %{}` is a documented state ("Phase 07 has not run"), so `dav` is
`{:map, []}`; an empty `metrics` map is not a state — those arrays *are* the
percentile tables every later grade is scored against, so a section carrying a
full `teams` list and no arrays reads back as a well-formed document that can
score nothing. Both are `%{}`; only one may pass.
"""
# D-43: five raw arrays, not four — Two-Way Balance is frozen as its two
# percentile inputs (`rotation_oepm`/`rotation_depm`) rather than as a raw
# metric, and combined 70/30 at scoring time. Declared here rather than in the
# builder so `Benchmark` can require exactly what `BenchmarkBuilder` writes
# without either depending on the other, and so the two cannot drift.
@competitive_strength_metrics [
:star_power,
:starting_five,
:rotation_quality,
:rotation_oepm,
:rotation_depm
]
@type name :: String.t()
@type spec ::
{:map, [{name(), spec()}]}
| {:map, [{name(), spec()}], keyword()}
| {:list, keyword()}
@doc """
The Competitive Strength raw metric names every benchmark must carry.
One declaration, read by both the builder and the schema, so a metric cannot be
added to the build and silently left optional at the persistence boundary.
"""
@spec competitive_strength_metrics() :: [atom()]
def competitive_strength_metrics, do: @competitive_strength_metrics
# ── Canonicalization ───────────────────────────────────────────────────
@doc """
Normalizes a document to string keys, or refuses it.
Returns `{:ok, canonical}` for a document whose every key is an atom or a
string with no two keys normalizing to the same name, and `{:error, messages}`
otherwise. Structs are values, not documents — a `DateTime` in `meta` is left
alone, since its keys are fixed at compile time and cannot collide.
Total: it never raises, including on exotic keys, because raising out of a
validator is the failure mode a fail-closed boundary exists to remove (D-52).
iex> alias TradeMachine.GMGradeV2.FrozenDocument
iex> FrozenDocument.canonicalize(%{skills: %{teams: []}})
{:ok, %{"skills" => %{"teams" => []}}}
iex> FrozenDocument.canonicalize(%{:skills => %{}, "skills" => %{}})
{:error, ["skills has both atom and string keys, which collide in jsonb"]}
"""
@spec canonicalize(term()) :: {:ok, term()} | {:error, [String.t()]}
def canonicalize(document) do
case walk(document, []) do
{canonical, []} -> {:ok, canonical}
{_canonical, errors} -> {:error, errors |> Enum.uniq() |> Enum.sort()}
end
end
defp walk(map, path) when is_map(map) and not is_struct(map) do
{named, key_errors} =
Enum.reduce(map, {[], []}, fn {key, value}, {named, errors} ->
case key_name(key) do
{:ok, name} ->
{[{name, value} | named], errors}
:error ->
{named,
[
render(path) <> " has a key that is neither an atom nor a string: #{inspect(key)}"
| errors
]}
end
end)
collision_errors =
named
|> Enum.group_by(fn {name, _value} -> name end)
|> Enum.reject(fn {_name, group} -> match?([_only], group) end)
|> Enum.map(fn {name, _group} ->
render(path ++ [name]) <> " has both atom and string keys, which collide in jsonb"
end)
{canonical, child_errors} =
Enum.reduce(named, {%{}, []}, fn {name, value}, {acc, errors} ->
{child, child_errors} = walk(value, path ++ [name])
{Map.put(acc, name, child), errors ++ child_errors}
end)
{canonical, key_errors ++ collision_errors ++ child_errors}
end
# `is_list/1` is true of `[1 | 2]`, and every `Enum` traversal raises on one.
# An improper list has no JSON representation at all, so it is refused here
# rather than allowed to raise out of a validator whose whole contract is that
# it does not (D-52).
defp walk(list, path) when is_list(list) do
if proper_list?(list) do
{canonical, errors} =
list
|> Enum.with_index()
|> Enum.reduce({[], []}, fn {value, index}, {acc, errors} ->
{child, child_errors} = walk(value, path ++ ["[#{index}]"])
{[child | acc], errors ++ child_errors}
end)
{Enum.reverse(canonical), errors}
else
{list, [render(path) <> " is an improper list, which has no JSON representation"]}
end
end
defp walk(value, _path), do: {value, []}
@doc """
Whether a term is a proper list.
Public because `is_list/1` admits an improper list while `length/1` raises on
one, so any caller deriving a length from an untrusted document needs the same
test this module uses.
"""
@spec proper_list?(term()) :: boolean()
def proper_list?([]), do: true
def proper_list?([_head | tail]), do: proper_list?(tail)
def proper_list?(_other), do: false
defp key_name(key) when is_binary(key), do: {:ok, key}
defp key_name(key) when is_atom(key), do: {:ok, Atom.to_string(key)}
defp key_name(_key), do: :error
# ── Schema validation ──────────────────────────────────────────────────
@doc """
Validates a canonicalized document against a `t:spec/0`.
Returns `[]` for a conforming document and a list of rendered messages
otherwise. Run this on the output of `canonicalize/1`, never on a raw
document: every name in a spec is a string, so an atom-keyed document would
report every required key as missing.
"""
@spec validate(term(), spec()) :: [String.t()]
def validate(document, spec), do: check(document, spec, [])
defp check(value, {:map, entries}, path), do: check(value, {:map, entries, []}, path)
defp check(value, {:map, entries, opts}, path)
when is_map(value) and not is_struct(value) and is_list(opts) do
min_size = Keyword.get(opts, :min_size, 0)
size_errors =
if map_size(value) < min_size do
[render(path) <> " must not be empty"]
else
[]
end
# For a node whose entries may each legitimately be empty, this is the only
# place the "all of them empty" state can be refused. Counted over the
# node's *declared* entries so an extra key cannot satisfy it.
populated_errors =
case Keyword.get(opts, :min_populated, 0) do
min when min > 0 ->
populated =
Enum.count(entries, fn {name, _sub_spec} ->
case Map.fetch(value, name) do
{:ok, sub_value} -> populated?(sub_value)
:error -> false
end
end)
if populated < min do
[
render(path) <>
" must hold at least #{min} non-empty entry, got #{populated} of #{length(entries)}"
]
else
[]
end
_ ->
[]
end
entry_errors =
Enum.flat_map(entries, fn {name, sub_spec} ->
# `Map.fetch/2`, never `Map.get(a) || Map.get(b)`: both `nil || x` and
# `false || x` fall through, which is how a stored `false` gets mapped
# onto the benign "not supplied" branch (D-52).
case Map.fetch(value, name) do
{:ok, sub_value} -> check(sub_value, sub_spec, path ++ [name])
:error -> [render(path ++ [name]) <> " is missing"]
end
end)
size_errors ++ populated_errors ++ entry_errors
end
defp check(value, {:list, opts}, path) when is_list(value) and is_list(opts) do
# `length/1` raises on an improper list, and `is_list/1` admits one. The
# changeset path canonicalizes first, so this is only reachable through the
# public `validate/2` — but a fail-closed walker that raises on one input
# shape is not fail-closed.
if proper_list?(value) do
length_errors(value, opts, path) ++ element_errors(value, opts, path)
else
[render(path) <> " is an improper list, which has no JSON representation"]
end
end
# The fail-closed default. Every value that reached no clause above is invalid,
# including one whose spec this module does not recognise — the walker never
# falls through to `[]`, which is the whole of D-49's intent made total.
defp check(value, spec, path), do: [render(path) <> " " <> mismatch(spec, value)]
# "Empty" is defined only for the containers a table can be; anything else is
# a value, and a value is populated. Improper lists are handled by the list
# clause, so this is never asked to measure one.
defp populated?(value) when is_list(value), do: proper_list?(value) and value != []
defp populated?(value) when is_map(value) and not is_struct(value), do: map_size(value) > 0
defp populated?(_value), do: true
defp length_errors(value, opts, path) do
actual = length(value)
min_length = Keyword.get(opts, :min_length, 0)
cond do
Keyword.has_key?(opts, :length) and actual != Keyword.fetch!(opts, :length) ->
[
render(path) <>
" must hold exactly #{Keyword.fetch!(opts, :length)} #{element_noun(opts)}, got #{actual}"
]
actual < min_length ->
[render(path) <> " must hold at least #{min_length} #{element_noun(opts)}, got #{actual}"]
true ->
[]
end
end
defp element_errors(value, opts, path) do
case Keyword.fetch(opts, :of) do
:error ->
[]
{:ok, of} ->
value
|> Enum.with_index()
|> Enum.reject(fn {element, _index} -> element?(element, of) end)
|> Enum.map(fn {_element, index} -> index end)
|> case do
[] -> []
bad -> [render(path) <> " must hold only #{element_noun(opts)}#{render_indices(bad)}"]
end
end
end
defp element?(element, :number), do: is_number(element)
defp element?(element, :map), do: is_map(element) and not is_struct(element)
# The same fail-closed default `check/3` carries, one level down. Without it an
# `of: :string` added by a later phase raises `FunctionClauseError` on the
# frozen-write path instead of reporting — an unrecognised spec must invalidate,
# never crash and never silently pass.
defp element?(_element, _of), do: false
defp element_noun(opts) do
case Keyword.get(opts, :of) do
:number -> "numeric values"
:map -> "objects"
nil -> "values"
other -> "values of the unrecognised element type #{inspect(other)}"
end
end
defp render_indices(indices) do
shown = Enum.take(indices, 5)
suffix = if length(indices) > 5, do: ", …", else: ""
" (bad at index #{Enum.join(shown, ", ")}#{suffix})"
end
defp mismatch({:map, _entries}, value), do: mismatch({:map, [], []}, value)
defp mismatch({:map, _entries, _opts}, value), do: "must be a map, got: #{inspect(value)}"
defp mismatch({:list, opts}, value),
do: "must be a list of #{element_noun(opts)}, got: #{inspect(value)}"
defp mismatch(spec, value), do: "does not match #{inspect(spec)}, got: #{inspect(value)}"
defp render([]), do: "the document"
defp render(path), do: path |> Enum.join(".") |> String.replace(".[", "[")
# ── Introspection (used by the D-54 property test) ─────────────────────
@doc """
Every path a spec requires, paired with the spec that governs it.
Derived from the spec rather than restated, so the property test that asserts
"dropping or corrupting any required key invalidates the document" widens
automatically when a section is added — the point of D-54's *one* property test
instead of N example tests.
"""
@spec required_paths(spec()) :: [{[name()], spec()}]
def required_paths({:map, entries}), do: required_paths({:map, entries, []})
def required_paths({:map, entries, _opts}) do
Enum.flat_map(entries, fn {name, sub_spec} ->
[{[name], sub_spec}] ++
Enum.map(required_paths(sub_spec), fn {path, spec} -> {[name | path], spec} end)
end)
end
def required_paths(_spec), do: []
@doc """
Values that must never validate against `spec` — the corruption half of the
D-54 property test.
Derived from the node type so that every kind of malformation the grammar can
express is exercised at every required path: the wrong primitive type, the
documented-empty-vs-meaningfully-empty pair (D-52), and — for arrays — right
cardinality with unusable contents, which is the shape that has the right
length and still cannot be ranked against.
"""
@spec invalid_samples(spec()) :: [term()]
def invalid_samples({:map, entries}), do: invalid_samples({:map, entries, []})
def invalid_samples({:map, entries, opts}) do
# `%{}` is invalid for a node that requires *either* a minimum size or named
# entries, and valid for one that requires neither — which is exactly the
# `meta`-vs-`dav` pair. Deriving it from the node rather than listing it
# keeps the property test asserting the distinction instead of assuming it.
empty = if Keyword.get(opts, :min_size, 0) > 0 or entries != [], do: [%{}], else: []
[nil, false, [], "not a map", 3] ++ empty
end
def invalid_samples({:list, opts}) do
empty = if Keyword.get(opts, :min_length, 0) > 0, do: [[]], else: []
wrong_contents =
case {Keyword.get(opts, :of), Keyword.get(opts, :length)} do
{nil, _} -> []
{_of, nil} -> [[nil]]
# Right cardinality, unusable contents — the case cardinality checks miss.
{_of, length} -> [List.duplicate(nil, length), List.duplicate(0.0, length + 1)]
end
[nil, false, %{}, "not a list"] ++ empty ++ wrong_contents
end
def invalid_samples(_spec), do: [nil]
# ── Changeset integration ──────────────────────────────────────────────
@doc """
Canonicalizes and validates a jsonb document field, storing the canonical form.
One implementation for both frozen tables, so the two cannot drift into
enforcing different things — the divergence that let `Benchmark` gain four
rounds of hardening while `ChallengeSnapshot`, the row every entrant in a
single challenge is scored against, kept the round-1 checks.
`schema_fun` receives the *canonicalized* document and returns its
`t:spec/0`, so a schema can depend on the document's own contents (a ranked
array holding one value per team, say) without any caller re-deriving it.
On success the field is replaced by its canonical string-keyed form: what is
written is then exactly what jsonb reads back (D-42). On failure every problem
is reported at once rather than one per insert attempt — a build that has to
be rerun per error is a build nobody reruns.
"""
@spec validate_document(Ecto.Changeset.t(), atom(), (term() -> spec())) :: Ecto.Changeset.t()
def validate_document(changeset, field, schema_fun) do
# `fetch_field/2`, NOT `fetch_change/2`. `cast/3` records no change when the
# cast value equals the struct's current one (`Ecto.Type.equal?/3`), so
# `fetch_change/2` answers `:error` for a document that was supplied, cast
# cleanly, and is about to be written — `Repo.insert` merges the whole struct
# into the changes regardless. Skipping there let a malformed document reach
# jsonb with the schema never consulted, which is D-54's finding class at a
# new line. `fetch_field/2` returns the *effective* value, which is exactly
# the one Postgres will store.
#
# A `nil` is the one thing still worth skipping: the field was omitted or
# explicitly nil, `validate_required/2` already says so, and a second error
# for the same cause is noise.
case Ecto.Changeset.fetch_field(changeset, field) do
:error ->
changeset
{_source, nil} ->
changeset
{_source, document} ->
case canonicalize(document) do
{:ok, canonical} ->
case validate(canonical, schema_fun.(canonical)) do
[] ->
changeset
|> Ecto.Changeset.put_change(field, canonical)
|> validate_encodable(field, canonical)
errors ->
add_errors(changeset, field, errors)
end
{:error, errors} ->
add_errors(changeset, field, errors)
end
end
end
# A document can satisfy the schema and still fail to reach Postgres: a tuple
# or a PID under `meta` raises inside the jsonb encode, turning an insert into
# an exception rather than a changeset error. A fail-closed boundary that
# crashes instead of reporting is still not reporting (D-52), and on a
# write-once path the caller deserves an error it can act on. One encode of a
# document built once per season is not a cost worth optimizing.
# Encoding proves the document can be written; it does not prove that what
# comes back is what went in. Canonicalization normalizes *keys*, so a leaf
# value Jason TRANSFORMS rather than preserves — a `DateTime`, a `Date`, a
# bare atom — passes the schema, encodes cleanly, and is stored as a string
# while the struct returned to the caller still holds the original term. That
# is exactly the in-memory/Postgres divergence D-42 exists to prevent, moved
# from keys to values. Decoding what was just encoded and comparing closes it
# by construction, and the builder's output already satisfies it.
defp validate_encodable(changeset, field, canonical) do
with {:ok, json} <- Jason.encode(canonical),
{:ok, decoded} <- Jason.decode(json) do
if decoded == canonical do
changeset
else
Ecto.Changeset.add_error(
changeset,
field,
"does not survive a JSON round trip unchanged: a value is rewritten by encoding " <>
"(a struct, date, or bare atom stored as a string), so the frozen document would " <>
"differ from the one in memory (D-42)"
)
end
else
{:error, error} ->
Ecto.Changeset.add_error(changeset, field, "cannot be encoded as JSON: #{inspect(error)}")
end
end
defp add_errors(changeset, field, errors) do
Enum.reduce(errors, changeset, fn message, acc ->
Ecto.Changeset.add_error(acc, field, message)
end)
end
end
editions.exElixir
lib/trade_machine/gm_grade_v2/editions.ex
Reviewed public projections of frozen lab outputs. Never exposes internal diagnostics.
SHA-256 c90a15617a286faf2768ad32e8a8a2b7a6a5b1223b422a3b9446df35dc799c46
defmodule TradeMachine.GMGradeV2.Editions do
@moduledoc "Reviewed public projections of frozen lab outputs. Never exposes internal diagnostics."
import Ecto.Query
alias TradeMachine.Accounts.Policy
alias TradeMachine.GMGradeV2.{Edition, Lab, MandateProfile}
alias TradeMachine.Repo
@pillar_keys ~w(competitive_strength roster_construction contract_health cap_flexibility asset_management timeline_alignment)
def list do
Repo.all(
from e in Edition,
order_by: [desc: e.inserted_at, desc: e.id],
select: struct(e, [:id, :title, :inserted_at]),
limit: 100
)
end
def get(nil),
do: Repo.one(from e in Edition, order_by: [desc: e.inserted_at, desc: e.id], limit: 1)
def get(id) do
case Ecto.UUID.cast(id) do
{:ok, id} -> Repo.get(Edition, id)
_ -> nil
end
end
def preview(run_id, actor) do
with true <- Policy.can?(actor, :run_grading_lab),
run when not is_nil(run) <- Lab.get_run(run_id) do
project(run)
else
_ -> {:error, "This preview is available to grading editors only."}
end
end
def publish(run_id, attrs, actor) do
with {:ok, document} <- preview(run_id, actor) do
%Edition{lab_run_id: run_id, published_by_id: actor.id, document: document}
|> Edition.insert_changeset(attrs)
|> Repo.insert()
end
end
def project(run) do
mandates = Enum.map(MandateProfile.keys(), &Atom.to_string/1)
teams = Enum.group_by(Map.values(run.fixtures), &get_in(&1, ["metadata", "team_id"]))
issues = publication_issues(run)
if issues == [] do
run = Repo.preload(run, :benchmark)
rows =
Enum.map(teams, fn {id, fixtures} ->
meta = hd(fixtures)["metadata"]
%{
"id" => id,
"name" => meta["team_name"],
"abbreviation" => meta["team_abbreviation"],
"evidence" => public_evidence(meta["public_evidence"]),
"grades" =>
Map.new(fixtures, fn fixture ->
{fixture["mandate_key"], public_grade(run.results[fixture["id"]])}
end)
}
end)
{:ok,
%{
"schema_version" => 1,
"model_version" => run.model_version,
"stats_season" => run.benchmark.season,
"benchmark_version" => run.benchmark.version,
"benchmark_id" => run.benchmark_id,
"built_at" => DateTime.to_iso8601(run.inserted_at),
"mandates" => mandates,
"teams" => Enum.sort_by(rows, & &1["name"])
}}
else
{:error, publication_error(run, issues)}
end
end
@doc "Exact publication blockers, grouped by team and mandate in the editor report."
def publication_issues(run) do
mandates = Enum.map(MandateProfile.keys(), &Atom.to_string/1)
teams = Enum.group_by(Map.values(run.fixtures), &get_in(&1, ["metadata", "team_id"]))
population =
failed_checks([
{map_size(run.fixtures) == 180,
"Expected 180 fixtures; found #{map_size(run.fixtures)}."},
{map_size(run.results) == 180,
"Expected 180 grade results; found #{map_size(run.results)}."},
{map_size(teams) == 30, "Expected 30 teams; found #{map_size(teams)}."},
{not Map.has_key?(teams, nil), "Some fixtures have no team identity."},
{Enum.all?(teams, fn {_id, rows} ->
Enum.sort(Enum.map(rows, & &1["mandate_key"])) == Enum.sort(mandates)
end),
"Every team must have exactly one result for each of the six mandates. Use a complete team backtest, not progression scenarios."}
])
fixture_issues =
Enum.flat_map(run.fixtures, fn {id, fixture} ->
meta = fixture["metadata"] || %{}
team = meta["team_name"] || meta["team_id"] || id
Enum.map(fixture_reasons(fixture, run.results[id]), fn reason ->
%{team: team, mandate: fixture["mandate_key"], reason: reason}
end)
end)
Enum.map(population, &%{team: "League", mandate: nil, reason: &1}) ++ fixture_issues
end
defp fixture_reasons(fixture, result) do
meta = fixture["metadata"] || %{}
evidence = meta["public_evidence"] || %{}
failed_checks([
{meta["archetype"] == "team_state" and is_nil(meta["fixture_kind"]),
"This is not an unmodified real-team fixture. Generate a team backtest."},
{is_binary(meta["team_name"]), "Team name is missing."},
{is_binary(evidence["captured_at"]),
"Roster capture date is missing. This run must be rebuilt with edition evidence enabled."},
{is_list(evidence["roster"]) and evidence["roster"] != [],
"Captured payroll roster is missing or empty. Check the team's active contracts before rebuilding."}
]) ++ result_reasons(result, fixture["mandate_key"])
end
defp result_reasons(result, mandate) when is_map(result) do
pillars = result["pillar_results"] || []
source = get_in(result, ["snapshot_refs", "evidence_source"])
live = get_in(result, ["snapshot_refs", "live_evidence"])
failed_checks([
{result["mandate_key"] == mandate, "Result mandate does not match its fixture."},
{valid_final_score?(result["final_score"]), "Final score is missing or outside 0–100."},
{is_number(result["base_score"]), "Weighted base score is missing."},
{source == "frozen",
"Evidence is not fully frozen (source: #{inspect(source)}; live inputs: #{inspect(live)}). Inspect those snapshot inputs before rebuilding."},
{Enum.sort(Enum.map(pillars, & &1["key"])) == Enum.sort(@pillar_keys),
"The result does not contain exactly the six expected pillars."}
]) ++ Enum.flat_map(pillars, &pillar_reasons(&1, result["pillar_weights"] || %{}))
end
defp result_reasons(_, _), do: ["Grade result is missing."]
defp pillar_reasons(pillar, weights) do
label = String.replace(pillar["key"] || "Unknown pillar", "_", " ")
failed_checks([
{is_number(pillar["score"]), "#{label}: numeric score is missing."},
{pillar["provisional?"] == false,
"#{label}: provisional or missing final status. #{pillar["explanation"] || "Inspect this pillar's inputs in the scorecard."}"},
{is_number(weights[pillar["key"]]), "#{label}: mandate weight is missing."}
])
end
defp valid_final_score?(score), do: is_number(score) and score >= 0 and score <= 100
defp failed_checks(checks), do: for({false, reason} <- checks, do: reason)
defp publication_error(run, issues) do
groups = issues |> Enum.group_by(&{&1.team, &1.reason}) |> Enum.sort_by(&elem(&1, 0))
lines =
Enum.map(groups, fn {{team, reason}, rows} ->
mandates =
rows |> Enum.map(& &1.mandate) |> Enum.reject(&is_nil/1) |> Enum.uniq() |> Enum.sort()
scope =
case length(mandates) do
0 -> ""
6 -> " (all six mandates)"
_ -> " (#{Enum.join(mandates, ", ")})"
end
"• #{team}#{scope}: #{reason}"
end)
"Publication blocked for run #{run.id} (#{run.model_version}).\n" <> Enum.join(lines, "\n")
end
def rows(document, mandate, sort \\ "total") do
rows =
Enum.map(document["teams"], fn team -> Map.put(team, "grade", team["grades"][mandate]) end)
ranks =
Map.new(rows, fn team ->
rank = 1 + Enum.count(rows, &(&1["grade"]["final_score"] > team["grade"]["final_score"]))
{team["id"], rank}
end)
rows
|> Enum.map(&Map.put(&1, "rank", ranks[&1["id"]]))
|> Enum.sort_by(fn row -> {-sort_score(row["grade"], sort), row["name"]} end)
end
def pillar(grade, key), do: Enum.find(grade["pillars"], &(&1["key"] == key))
defp sort_score(grade, "total"), do: grade["final_score"]
defp sort_score(grade, key), do: (pillar(grade, key) || %{})["score"] || grade["final_score"]
defp public_evidence(evidence) do
evidence
|> Map.take(~w(captured_at roster_season roster))
|> Map.update!("roster", fn roster ->
Enum.map(roster, &Map.take(&1, ~w(id name salary position contract_type)))
end)
end
defp public_grade(result) do
%{
"final_score" => result["final_score"],
"display_score" => result["display_score"],
"base_score" => result["base_score"],
"weights" => result["pillar_weights"],
"evidence_source" => get_in(result, ["snapshot_refs", "evidence_source"]),
"pillars" =>
Enum.map(result["pillar_results"], fn p ->
%{
"key" => p["key"],
"score" => p["score"],
"explanation" => p["explanation"],
"metrics" =>
Enum.map(p["metric_results"], fn m ->
%{
"key" => m["key"],
"score" => m["score"],
"explanation" => m["explanation"],
"weight" => p["metric_weights"][m["key"]],
"facts" => scalar_facts(m["inputs"] || %{}),
"fallback_count" => length(m["fallbacks"] || []),
"assets" => public_assets(m)
}
end)
}
end)
}
end
defp scalar_facts(inputs) do
inputs |> Enum.filter(fn {_k, v} -> is_number(v) or is_binary(v) end) |> Map.new()
end
defp public_assets(%{"key" => "draft_capital_value"} = m) do
Enum.map(get_in(m, ["supporting", "assets"]) || [], fn asset ->
Map.take(
asset,
~w(year round kind original_team_id inventory_dav dav baseline_dav provisional?)
)
end)
end
defp public_assets(_), do: []
end