Track 18

Structured Post-Model Algorithms

This track teaches what happens after the model score exists: keep or reject, assign under constraints, clean duplicates, and make the final answer respect the real task instead of the raw score alone.

Open Decision Topic Run Fast Examples Use The Study Plan

Primary Goal

Turn Scores Into Structured Decisions

The model score is only evidence. The real answer depends on the rule that decides what to keep, what to merge, and how to respect capacity or one-to-one constraints.

Best For

When The Raw Score Is Not Enough

Use this track when the next gain is likely in ranking, assignment, or cleanup, not in a bigger model family.

Exit Rule

One Simple Rule That Beats Naive Top-1

You are done when you can say which post-model rule you kept, which task constraint it respected, and why the naive score-only answer was not enough.

Use This Track When¶

the model score exists but the final answer is still operationally wrong
the task has keep or reject rules, one-to-one or capacity constraints, or duplicate cleanup
the next improvement is more likely to come from structured post-processing than from a larger model

What This Track Is Training¶

This track trains one practical rule:

keep the simplest post-model rule that matches the real task constraint

That means the learner should be able to keep these explicit:

the raw score or score matrix
the keep or reject rule
the capacity or one-to-one constraint
the duplicate or graph cleanup rule
the weakest slice where the post-model logic still fails

First Session¶

Use this order:

Selective Prediction and Review Budgets
from repo root run academy/.venv/bin/python academy/examples/post-model-algorithm-recipes/ranking_assignment_demo.py
write one short note on why top-1 score is not yet a final answer

Full Track Loop¶

For the complete workflow:

review the decision topic and keep the real task constraint explicit before touching the rule
from repo root run academy/.venv/bin/python academy/examples/post-model-algorithm-recipes/ranking_assignment_demo.py
from repo root run academy/.venv/bin/python academy/labs/structured-post-model-algorithms/src/structured_post_model_workflow.py
finish the matching exercises in academy/exercises/structured-post-model-algorithms/
keep one short post-model memo with the selected keep rule, assignment rule, cleanup rule, and weakest slice

What To Inspect¶

By the end of the track, the learner should have inspected:

threshold_grid.csv for the keep or reject rule that replaces naive top-1
retrieval_summary.csv for what the score can already do before post-processing
assignment_summary.csv for the difference between naive and constraint-aware decisions
graph_summary.csv for when cleanup is solving duplicate structure rather than just polishing the score
score_matrix.png and workflow_report.md only after the tabular summaries are already clear

Common Failure Modes¶

trusting top-1 score without a keep rule
ignoring capacity or one-to-one constraints because the raw score looks strong
changing the candidate set and the post-model rule at the same time
calling cleanup a cosmetic step when it is actually changing the answer
claiming a model gain when the real improvement came from the structured rule

Exit Standard¶

Before leaving this track, the learner should be able to:

explain why a model score is not yet a final decision
keep one simple rule that beats naive top-1 or naive argmax
apply a capacity-aware assignment rule when the task needs it
explain when graph cleanup is the real fix
name the slice or constraint that still limits the workflow

That is enough to move into Problem Adaptation and Post-Processing or another workflow where the decision layer has to stay explicit.