Module 4 Overview

Theme

Model selection and validation

Essential Question

How do we choose models without overfitting our evidence?

Module Components

• Book prose: conceptual framing, domain scenario, methods, and failure modes

• Assignment: evidence-backed production of a specific artifact

• Slides: presentation sequence for seminar or lecture delivery

• Narration: spoken version of the slide flow

• Instructor notes: facilitation plan, discussion prompts, and grading cues

• Rubric: criteria for evaluating the module artifact

• Notebook: executable lab aligned with the module theme using synthetic tabular observations with features, labels, train/test split, baseline score, and error slices

Module Artifact

predictive modeling report with baseline comparison, validation evidence, and model card focused on model selection and validation: Run cross-validation and summarize model tradeoffs.

Professional Setting

Students work as if advising an analytics team choosing a predictive model for an operational decision. Their work must be intelligible to analytics lead, domain owner, operations manager, and model risk reviewer.