Track 15

Detection and Segmentation Workflows

This track turns localization and masks into a route-guided workflow: compare threshold policies, separate box quality from mask quality, and treat cleanup as part of the decision instead of as an afterthought.

Open The Detection Topic Run Fast Examples Use The Study Plan

Primary Goal

Turn Scores Into Spatial Decisions

The point is not just to predict an object. The point is to decide which box or mask survives the threshold, the cleanup rule, and the weak-slice check.

Best For

After Basic Vision Baselines

Use this when image-level classification feels too simple and you need to reason about overlap, thresholding, and spatial failure modes.

Exit Rule

Leave With One Threshold And Cleanup Story

You are done when you can justify the threshold policy, the cleanup rule, and the slice you still do not trust.

Use This Track When

• you need box or mask quality, not only image-level accuracy
• threshold choice is affecting precision, recall, and coverage in visible ways
• post-processing may matter as much as the model score
• small, noisy, or rare-object slices could break an otherwise good aggregate metric

What This Track Is Training

This track trains one route:

1. compare box IoU and mask IoU on the same cases
2. choose thresholds from validation evidence instead of guesswork
3. treat cleanup and mask-to-box conversion as part of the workflow
4. inspect the weakest slice before escalating model complexity
5. keep the simplest policy that still matches the task

First Session

Use this order:

1. Object Detection Basics
2. Detection and Segmentation
3. run academy/.venv/bin/python academy/examples/detection-segmentation-demo.py
4. run academy/.venv/bin/python academy/examples/detection-segmentation-recipes/synthetic_detection_segmentation_demo.py
5. write one short note on whether the first visible error is a threshold problem, a box problem, or a mask problem

Full Track Loop

For the complete workflow:

1. read Object Detection Basics
2. read Detection and Segmentation
3. run academy/.venv/bin/python academy/examples/detection-segmentation-demo.py
4. run academy/.venv/bin/python academy/examples/detection-segmentation-recipes/synthetic_detection_segmentation_demo.py
5. run academy/.venv/bin/python academy/labs/detection-and-segmentation-workflows/src/detection_segmentation_workflow.py
6. finish the matching exercises in academy/exercises/detection-and-segmentation-workflows/
7. keep one note with the threshold table, the cleanup rule, and the weakest slice that still blocks trust

What To Inspect

By the end of the track, the learner should have inspected:

• box IoU and mask IoU on the same scenes
• the precision-recall tradeoff for at least two threshold policies
• whether the learned or validation-chosen gate changed the right error type
• whether cleanup improved overlap by reducing noise, changing shape, or both
• which slice is weakest by size, shape, or rarity
• whether the problem is really model capacity or still a policy and cleanup problem

Common Failure Modes

• trusting one aggregate score without checking spatial overlap quality
• changing the split and the threshold at the same time
• calling a box improvement a full segmentation improvement
• treating cleanup as harmless when it may be reshaping the object the wrong way
• escalating to a heavier detector before the threshold and cleanup story is honest

Exit Standard

Before leaving this track, the learner should be able to:

• explain the difference between box IoU and mask IoU in one concrete case
• choose a threshold from validation evidence instead of from intuition alone
• defend one cleanup rule as part of the final decision path
• name the slice that still limits trust in the workflow

That is enough to move into stronger vision systems without pretending the raw score already solved the spatial problem.