Priya is building the diagnostic pipeline for MedScan's AI-powered chest X-ray analyzer. She needs to classify X-rays into four categories: Normal, Pneumonia, COVID-19, and Tuberculosis. But classification accuracy alone isn't enough—radiologists need explainability: which regions of the image drove the model's decision? Priya uses Grad-CAM (Gradient-weighted Class Activation Mapping) to generate saliency heatmaps that highlight the important areas.
🏥 The Capstone Challenge: End-to-End Medical Imaging
This capstone brings together everything: data loading, transfer learning, multi-class classification, evaluation metrics, and explainability. Priya's workflow:
1. Data Ingestion
Load the pneumonia-mnist.npz dataset: 8,144 labeled chest X-rays (28×28 grayscale). Split into train/val/test with stratification to preserve class distribution.
2. Transfer Learning
Fine-tune ResNet50 pretrained on ImageNet. Unfreeze layer4 and train with lower learning rates. Handle grayscale input by replicating channels.
3. Multi-Class Evaluation
Beyond accuracy: compute confusion matrices, per-class precision/recall, and ROC-AUC curves for each disease class. In medical imaging, false negatives (missing a disease) are costlier than false positives.
4. Explainability with Grad-CAM
Generate visual explanations: heatmaps that show which pixels contributed most to each prediction. Grad-CAM is crucial for clinician trust and regulatory compliance.
📊 Meet the Data: pneumonia-mnist.npz
The pneumonia-mnist.npz dataset contains 8,144 chest X-rays across four classes. Each image is 28×28 grayscale pixels (0–255). Class distribution is imbalanced:
🔧 Part 1: Data Loading & Preprocessing
Load the compressed dataset, normalize pixel values, and split into train/val/test sets with stratification to preserve class balance:
🔥 Part 2: Fine-Tune ResNet50 for 4-Class Classification
Load a pretrained ResNet50, replace the final layer for 4 classes, unfreeze layer4, and train with differential learning rates:
📈 Part 3: Multi-Class Evaluation
Go beyond accuracy: compute confusion matrices, per-class precision/recall/F1, ROC-AUC curves, and visualizations:
🔍 Part 4: Grad-CAM for Explainability
Grad-CAM (Gradient-weighted Class Activation Mapping) generates saliency heatmaps that show which regions of the X-ray influenced the model's prediction. This is critical for clinical trust:
🚀 Part 5: Production Deployment Considerations
Moving from research to production requires robust error handling, monitoring, and clinical validation:
Model Checkpointing & Versioning
Always save the best model checkpoint and log metadata: training date, validation metrics, dataset version. Use semantic versioning (v1.0.0 = initial production release, v1.1.0 = bugfix).
Uncertainty Quantification
Don't just predict; estimate confidence. Flag low-confidence predictions (e.g., max probability < 0.85) for radiologist review. Use softmax entropy or ensemble methods to measure uncertainty.
Distribution Monitoring
In production, X-rays may come from different scanners, patient populations, or imaging protocols. Monitor input distribution (pixel statistics) and prediction distribution (class predictions over time) for drift.
Regulatory & Clinical Validation
Medical AI systems must undergo clinical trials and FDA clearance (in the US). Grad-CAM is a key tool for interpretability and regulatory compliance. Document the model's limitations (e.g., not for pediatric patients) and intended use.
Task: Extend the Grad-CAM implementation to generate saliency maps for all four class predictions simultaneously (not just the top-1 predicted class). Create a 2×4 grid showing Normal, Pneumonia, COVID, and TB Grad-CAM maps for a single test image.
Hint: You'll need to run forward+backward for each class separately and store the CAM maps. This helps understand which features the model associates with each disease.
💼 Interview Questions
✓ Quick Quiz
📋 Quick Cheatsheet
| Concept | Code / Usage | Notes |
|---|---|---|
| Stratified split (balanced classes) | train_test_split(X, y, stratify=y) |
Essential for imbalanced medical data |
| Load pretrained ResNet50 | models.resnet50(pretrained=True) |
ImageNet weights; modify last layer |
| Replace final FC layer | model.fc = nn.Linear(in_features, 4) |
4 output neurons for 4 classes |
| Modify conv1 for grayscale | model.conv1 = nn.Conv2d(1, 64, ...) |
Copy weights: model.conv1.weight.data = orig[:, :1, :, :] |
| Grad-CAM saliency map | cam = (gradients * activations).sum(dim=0) |
Generate per-class explanations |
| Confusion matrix | confusion_matrix(y_true, y_pred) |
Diagnose per-class errors |
| ROC-AUC (multi-class) | roc_auc_score(y_true, y_probs, average='macro') |
Threshold-independent; handles imbalance |
| Normalize X-rays to [0,1] | X = X.astype(float32) / 255.0 |
Stabilize training; helps with BN |
⚠️ Common Mistakes
stratify=y in sklearn's train_test_split to preserve class distribution.x.repeat(1, 3, 1, 1) or modify conv1. Mismatched shapes cause silent errors or poor performance.✦ Key Takeaways
- This capstone ties together data loading, transfer learning, multi-class classification, evaluation, and explainability—the full pipeline for production medical imaging systems.
- Always use stratified train/test splits on imbalanced medical data to preserve class distribution and ensure valid metrics.
- Fine-tune ResNet50 with differential learning rates: backbone LR ≈ 1e-4, head LR ≈ 1e-3. Unfreeze only layer4 for limited medical data.
- Go beyond accuracy: compute confusion matrices, per-class precision/recall/F1, and ROC-AUC (threshold-independent). Medical decisions depend on false negative rates.
- Grad-CAM is not optional: generate saliency heatmaps for every prediction. Explainability builds radiologist trust and is required for regulatory compliance.
- Production deployment requires clinical validation, uncertainty quantification, distribution monitoring, and continuous retraining. High test accuracy is a starting point, not the destination.
- Priya's capstone achieves 93.2% accuracy, 0.93+ ROC-AUC, and clinician-ready Grad-CAM explanations. She's ready to deploy with proper validation and monitoring.
You've built a production-grade medical imaging system.
Now join teams transforming healthcare.
Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Agentic Engineer →