Module 4: Model selection and validation

Theme

Model selection and validation

Essential Question

How do we choose models without overfitting our evidence?

Module Components

• Book prose: conceptual framing, domain scenario, methods, and failure modes

• Assignment: evidence-backed production of a specific artifact

• Slides: presentation sequence for seminar or lecture delivery

• Narration: spoken version of the slide flow

• Rubric: criteria for evaluating the module artifact

• Notebook: executable lab aligned with the module theme using synthetic tabular observations with features, labels, train/test split, baseline score, and error slices

Module Artifact

predictive modeling report with baseline comparison, validation evidence, and model card focused on model selection and validation: Run cross-validation and summarize model tradeoffs.

Professional Setting

Students work as if advising an analytics team choosing a predictive model for an operational decision. Their work must be intelligible to analytics lead, domain owner, operations manager, and model risk reviewer.

Use This Module in Order

1. Read the learning chapter.

2. Review the slide deck with the matching narration.

3. In Populi, open the private student-repository link for this course and enter modules/module-4.

4. Clone the repository once or open its Codespace/Colab copy; run lab.ipynb and complete exercise.ipynb there.

5. Self-check with the rubric, commit and push the work, then submit exactly what Populi requests.