👩‍⚕️
Meet Priya
Computer Vision Engineer at MedScan Diagnostics

Priya is building the diagnostic pipeline for MedScan's AI-powered chest X-ray analyzer. She needs to classify X-rays into four categories: Normal, Pneumonia, COVID-19, and Tuberculosis. But classification accuracy alone isn't enough—radiologists need explainability: which regions of the image drove the model's decision? Priya uses Grad-CAM (Gradient-weighted Class Activation Mapping) to generate saliency heatmaps that highlight the important areas.

4
Classes to diagnose
8,144
Training images
0.93+
ROC-AUC target

🏥 The Capstone Challenge: End-to-End Medical Imaging

This capstone brings together everything: data loading, transfer learning, multi-class classification, evaluation metrics, and explainability. Priya's workflow:

1. Data Ingestion

Load the pneumonia-mnist.npz dataset: 8,144 labeled chest X-rays (28×28 grayscale). Split into train/val/test with stratification to preserve class distribution.

2. Transfer Learning

Fine-tune ResNet50 pretrained on ImageNet. Unfreeze layer4 and train with lower learning rates. Handle grayscale input by replicating channels.

3. Multi-Class Evaluation

Beyond accuracy: compute confusion matrices, per-class precision/recall, and ROC-AUC curves for each disease class. In medical imaging, false negatives (missing a disease) are costlier than false positives.

4. Explainability with Grad-CAM

Generate visual explanations: heatmaps that show which pixels contributed most to each prediction. Grad-CAM is crucial for clinician trust and regulatory compliance.

📊 Meet the Data: pneumonia-mnist.npz

The pneumonia-mnist.npz dataset contains 8,144 chest X-rays across four classes. Each image is 28×28 grayscale pixels (0–255). Class distribution is imbalanced:

Sample X-ray images from each class: Normal, Pneumonia, COVID-19, and TB
Grid of 28x28 chest X-rays representing Normal, Pneumonia, COVID, and TB cases
Despite the small 28×28 resolution, texture (infiltrates, consolidation) and density patterns are visible. ResNet50 extracts these subtle features, and Grad-CAM lets us see what it learned.

🔧 Part 1: Data Loading & Preprocessing

Load the compressed dataset, normalize pixel values, and split into train/val/test sets with stratification to preserve class balance:

Load & Preprocess pneumonia-mnist.npz
Data loading output
Dataset shape: (8144, 28, 28), Labels: (8144,) Class distribution: [2050 2000 1850 1244] Train: (6515, 28, 28), Val: (817, 28, 28), Test: (812, 28, 28) ✓ Data loaded and split successfully

🔥 Part 2: Fine-Tune ResNet50 for 4-Class Classification

Load a pretrained ResNet50, replace the final layer for 4 classes, unfreeze layer4, and train with differential learning rates:

Build & Fine-Tune ResNet50
Training output
Epoch 1/15 | Loss: 0.892 | Val Acc: 0.813 Epoch 2/15 | Loss: 0.654 | Val Acc: 0.847 Epoch 3/15 | Loss: 0.521 | Val Acc: 0.872 Epoch 5/15 | Loss: 0.389 | Val Acc: 0.891 Epoch 10/15 | Loss: 0.251 | Val Acc: 0.916 Epoch 15/15 | Loss: 0.187 | Val Acc: 0.932 ✓ Training complete. Best val accuracy: 0.932

📈 Part 3: Multi-Class Evaluation

Go beyond accuracy: compute confusion matrices, per-class precision/recall/F1, ROC-AUC curves, and visualizations:

Multi-Class Evaluation Metrics
Confusion matrix: predicted class vs. actual class across test set
Confusion matrix heatmap for 4-class medical X-ray classification
Priya's model achieves 93.2% overall accuracy and strong per-class ROC-AUC (0.96+ for Normal and COVID, 0.93 for Pneumonia, 0.90 for TB). The slight confusion between Pneumonia and COVID is expected—both show similar infiltration patterns.

🔍 Part 4: Grad-CAM for Explainability

Grad-CAM (Gradient-weighted Class Activation Mapping) generates saliency heatmaps that show which regions of the X-ray influenced the model's prediction. This is critical for clinical trust:

Implement Grad-CAM
Grad-CAM heatmaps for test samples: original X-ray, saliency map, and overlay
Grad-CAM visualization showing which regions of chest X-rays influenced predictions
Grad-CAM reveals that the model correctly focuses on infiltration regions (white areas indicating consolidation) for Pneumonia and COVID diagnoses, and characteristic patterns in upper lungs for TB. This builds radiologist confidence in the system.

🚀 Part 5: Production Deployment Considerations

Moving from research to production requires robust error handling, monitoring, and clinical validation:

Model Checkpointing & Versioning

Always save the best model checkpoint and log metadata: training date, validation metrics, dataset version. Use semantic versioning (v1.0.0 = initial production release, v1.1.0 = bugfix).

Uncertainty Quantification

Don't just predict; estimate confidence. Flag low-confidence predictions (e.g., max probability < 0.85) for radiologist review. Use softmax entropy or ensemble methods to measure uncertainty.

Distribution Monitoring

In production, X-rays may come from different scanners, patient populations, or imaging protocols. Monitor input distribution (pixel statistics) and prediction distribution (class predictions over time) for drift.

Regulatory & Clinical Validation

Medical AI systems must undergo clinical trials and FDA clearance (in the US). Grad-CAM is a key tool for interpretability and regulatory compliance. Document the model's limitations (e.g., not for pediatric patients) and intended use.

🏆 Mini Challenge: Grad-CAM for All Classes

Task: Extend the Grad-CAM implementation to generate saliency maps for all four class predictions simultaneously (not just the top-1 predicted class). Create a 2×4 grid showing Normal, Pneumonia, COVID, and TB Grad-CAM maps for a single test image.

Hint: You'll need to run forward+backward for each class separately and store the CAM maps. This helps understand which features the model associates with each disease.

💼 Interview Questions

Grad-CAM (Gradient-weighted Class Activation Mapping) generates visual saliency maps that highlight which regions of an image influenced a model's prediction. It's critical in medical imaging because radiologists need explainability—they won't trust a "black box" that says "pneumonia detected" without showing where. Grad-CAM builds trust, aids debugging, and is often required for regulatory compliance (FDA clearance).
In medical imaging, class imbalance is common and false negatives are costlier than false positives. Accuracy treats both equally; ROC-AUC captures the true positive rate vs. false positive rate trade-off across all decision thresholds. It's also threshold-independent—clinicians can adjust the threshold based on the acceptable false positive rate.
Two approaches: (1) Replicate channels: x.repeat(1, 3, 1, 1) converts (B, 1, H, W) to (B, 3, H, W) by copying the grayscale channel three times. (2) Modify conv1: Replace the first convolution layer to accept 1 channel instead of 3, copying weights from the first RGB channel of the pretrained model. Approach (2) is more theoretically sound but approach (1) is simpler and often works fine.
Most likely: distribution shift. Production X-rays come from different scanners, patient demographics, or imaging protocols than training data. Other causes: (1) data leakage in train/val split, (2) preprocessing differences (normalization, resizing), (3) model degradation over time (batch norm statistics outdated), or (4) class imbalance differences. Solutions: retrain on production data, use domain adaptation, or monitor distribution drift continuously.
Compute spatial statistics of the Grad-CAM map: (1) entropy (higher = more diffuse), (2) peak-to-average ratio (lower = less focused), (3) center-of-mass distance from image center (unusual = potential artifact). Set thresholds: if entropy > 3.5 OR peak ratio < 1.5, flag as unreliable and request manual radiologist review. Log flagged cases for model debugging—diffuse Grad-CAM often indicates the model is uncertain or using spurious correlations. This is your feedback loop to improve the model.

✓ Quick Quiz

Question 1 / 5
You have 8,144 labeled X-rays across 4 classes (imbalanced). Should you use stratified or random split?
Question 2 / 5
What does Grad-CAM measure?
Question 3 / 5
Your multi-class model has 90% accuracy but ROC-AUC of 0.87 (macro-average). Why the discrepancy?
Question 4 / 5
You modify ResNet50's conv1 from 3 channels to 1 channel for grayscale input. How should you initialize the weights?
Question 5 / 5
Before deploying to a hospital, which is the MOST critical validation step?
0

📋 Quick Cheatsheet

Concept Code / Usage Notes
Stratified split (balanced classes) train_test_split(X, y, stratify=y) Essential for imbalanced medical data
Load pretrained ResNet50 models.resnet50(pretrained=True) ImageNet weights; modify last layer
Replace final FC layer model.fc = nn.Linear(in_features, 4) 4 output neurons for 4 classes
Modify conv1 for grayscale model.conv1 = nn.Conv2d(1, 64, ...) Copy weights: model.conv1.weight.data = orig[:, :1, :, :]
Grad-CAM saliency map cam = (gradients * activations).sum(dim=0) Generate per-class explanations
Confusion matrix confusion_matrix(y_true, y_pred) Diagnose per-class errors
ROC-AUC (multi-class) roc_auc_score(y_true, y_probs, average='macro') Threshold-independent; handles imbalance
Normalize X-rays to [0,1] X = X.astype(float32) / 255.0 Stabilize training; helps with BN

⚠️ Common Mistakes

Using random train/test split on imbalanced data
Random splits risk severe class imbalance between train and test, skewing metrics. Always use stratify=y in sklearn's train_test_split to preserve class distribution.
Not converting grayscale to RGB (or replicate channel)
ImageNet models expect 3-channel RGB input. Either replicate the grayscale channel: x.repeat(1, 3, 1, 1) or modify conv1. Mismatched shapes cause silent errors or poor performance.
Trusting accuracy alone in medical imaging
Accuracy ignores false negatives (missed diagnoses), which are clinically catastrophic. Always compute per-class precision, recall, and ROC-AUC. A 95% accurate model that misses all COVID cases is useless.
Skipping Grad-CAM or other explainability
Grad-CAM is not optional in medical AI. Radiologists and regulators demand explainability. Without Grad-CAM, your model is a black box—no hospital will deploy it.
Not monitoring for distribution shift in production
High validation accuracy ≠ high production accuracy. Different scanners, patient populations, and imaging protocols cause distribution shift. Monitor input statistics and prediction entropy continuously; retrain when drift is detected.

✦ Key Takeaways

  • This capstone ties together data loading, transfer learning, multi-class classification, evaluation, and explainability—the full pipeline for production medical imaging systems.
  • Always use stratified train/test splits on imbalanced medical data to preserve class distribution and ensure valid metrics.
  • Fine-tune ResNet50 with differential learning rates: backbone LR ≈ 1e-4, head LR ≈ 1e-3. Unfreeze only layer4 for limited medical data.
  • Go beyond accuracy: compute confusion matrices, per-class precision/recall/F1, and ROC-AUC (threshold-independent). Medical decisions depend on false negative rates.
  • Grad-CAM is not optional: generate saliency heatmaps for every prediction. Explainability builds radiologist trust and is required for regulatory compliance.
  • Production deployment requires clinical validation, uncertainty quantification, distribution monitoring, and continuous retraining. High test accuracy is a starting point, not the destination.
  • Priya's capstone achieves 93.2% accuracy, 0.93+ ROC-AUC, and clinician-ready Grad-CAM explanations. She's ready to deploy with proper validation and monitoring.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

You've built a production-grade medical imaging system.
Now join teams transforming healthcare.

Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Agentic Engineer →