Timed Checkpoint Sheets

Use these sheets when the topic already makes sense and you want short retrieval pressure.

The rule is simple:

1. set the timer first
2. answer cold
3. mark unstable reasoning
4. open the answer key only after the timer ends
5. route back to the weak topic or pack immediately

Sheet A: Validation and Leakage Sprint

Time: 20 minutes

Prompts:

1. You computed target encoding on the full dataset before the split. What is the problem?
2. You tried many hyperparameter settings and kept looking at the test set. Can the final test score still be treated as clean?
3. A random row split beats a group-by-user split by a large margin. What is the first suspicion?
4. A public leaderboard improves sharply but local cross-validation does not. What is the disciplined next move?
5. Average F1 improved, but the weakest slice collapsed. What mistake should you avoid?
6. Review capacity is capped at a fixed number per day. What should you inspect before default F1 at threshold 0.5?

Answer Key

1. Leakage through preprocessing; the held-out data influenced the encoder. 2. No. The test set became part of model selection. 3. Entity leakage or near-duplicate information across splits. 4. Keep the split fixed and demand stronger local evidence before trusting the jump. 5. Do not let the average score erase a slice failure that matters operationally. 6. Precision, recall, or utility at the review-budget cutoff.

Continue with: Validation, Leakage, and Model Choice

Sheet B: Representation and Geometry Sprint

Time: 20 minutes

Prompts:

1. Why should scale differences matter before KMeans?
2. What does the first principal component optimize?
3. Can a t-SNE plot prove the true number of clusters?
4. In a Gaussian mixture, what happens to responsibilities when variances shrink?
5. A frozen pretrained embedding plus linear head beats a scratch model on small data. Is that surprising?
6. A new embedding looks cleaner in 2D but performs worse downstream. Which signal should control deployment?

Answer Key

1. Distance-based clustering is scale-sensitive, so large-scale features can dominate. 2. Variance explained along a direction. 3. No. t-SNE is an inspection tool, not proof of cluster truth. 4. Responsibilities sharpen toward the more compatible component. 5. No. Reuse often wins in low-data settings because it lowers variance. 6. The downstream task metric should control the decision.

Continue with: Unsupervised Learning and Representation

Sheet C: Training and Checkpoint Sprint

Time: 25 minutes

Prompts:

1. Loss oscillates wildly and turns nan. What is the first optimizer diagnosis?
2. Train loss falls while validation loss rises after epoch 8. Which checkpoint should you keep?
3. You have limited labeled data and a strong pretrained backbone. What is the safer first transfer move?
4. The model already fits training very well but misses validation badly. Bigger network or stronger regularization?
5. Why can tiny-batch BatchNorm behavior be risky during fine-tuning?
6. Lower final train loss but worse best validation metric: which run wins?

Answer Key

1. The learning rate is probably too high. 2. Keep the best-validation checkpoint. 3. Start with a frozen backbone or small unfreeze, not full adaptation immediately. 4. Stronger regularization first. 5. Batch statistics can become noisy and destabilize adaptation. 6. The run with the better held-out metric wins.

Continue with: Deep Learning and Checkpoints

Weekly Use

Use the sheets in this order:

1. study the matching topic or pack
2. return later for one timed sheet without notes
3. log the first shaky question
4. revisit the same sheet after rerunning the weak topic

The page is doing its job only if it routes you back to the exact weak concept instead of just producing another score.