Mini Project: X-Ray Classifier — Production-Grade Pneumonia Detection
Build a complete chest X-ray pneumonia classifier from scratch to deployment. Load PneumoniaMNIST (28×28 grayscale images), design a CNN with batch normalization, apply transfer learning, evaluate with ROC-AUC. Priya at MedScan Diagnostics achieves AUC 0.9535, 98.46% recall. Medical-grade accuracy for clinical deployment.
🏥 Medical Imaging🧠 CNN + Transfer Learning📊 Clinical Evaluation🎯 Production Capstone⏱ ~30 min
Lesson 17 of Deep Learning Track (Arc C — Advanced Applications)
🏥
Real-world scenario
Priya at MedScan Diagnostics (Bangalore): build a production-grade pneumonia classifier for radiologists
Priya is a computer vision engineer at MedScan Diagnostics in Bangalore, India. The hospital is rolling out an AI-assisted diagnostic tool for chest X-rays to detect pneumonia early. Priya's team has collected thousands of annotated X-rays labeled as normal or pneumonia-positive. She needs to build a classifier that: (1) achieves >95% AUC on the test set, (2) maintains high recall (catch pneumonia cases, even if it means more false alarms), and (3) provides interpretable predictions for radiologists to review. She combines CNN architecture design (Arc B) with advanced evaluation metrics (Arc C) to ship a model that won't miss pneumonia cases—in healthcare, missing a disease is far worse than a false alarm.
4,996
Chest X-ray images
28×28
Image pixels (grayscale)
0.9535 AUC
Target metric
What we're building: an end-to-end pneumonia classifier
Explore data: sample images, class distribution, pixel ranges
Preprocess: normalize pixel values, augment with rotations/flips
Build model: CNN with BatchNorm or ResNet18 transfer learning
Train: Adam optimizer, binary cross-entropy loss, early stopping
Evaluate: training curves, confusion matrix, ROC-AUC curve
Clinical analysis: sensitivity vs specificity trade-off, threshold tuning
Deployment considerations: GradCAM visualization, API design
Step 1: Load and explore PneumoniaMNIST
PneumoniaMNIST is a dataset of 4,996 chest X-ray images (28×28 grayscale) with binary labels: normal (0) or pneumonia (1). We'll load from a .npz file containing pre-split train/val/test sets.
load_xray_data.pyLoad PneumoniaMNIST and explore class distribution
Binary classification: normal (class 0, ~40% of data) vs pneumonia (class 1, ~60%). The imbalance means accuracy alone is a poor metric—we'll use ROC-AUC and recall instead.
Step 2: Data preprocessing and augmentation
Medical imaging benefits from augmentation: rotations, flips, and brightness shifts make the model robust to real X-ray variations. We'll use torchvision transforms.
augment_xray.pyApply augmentation to prevent overfitting
Output
Train batches: 120
Val batches: 17
Test batches: 21
Step 3: Build the model — CNN with BatchNorm or transfer learning
We'll start with a custom CNN with batch normalization, then upgrade to ResNet18 transfer learning. Both achieve high AUC; ResNet is faster.
model_architecture.pyCNN with BatchNorm and ResNet18 transfer learning
Output
Model: ResNet18 for X-ray classification
Device: cuda
Total params: 11,189,505
Step 4: Training loop with early stopping
Use binary cross-entropy loss, Adam optimizer, and early stopping on validation AUC to prevent overfitting.
train_xray.pyTraining loop with early stopping on AUC
Plot training/validation loss and accuracy over 15 epochs. The model converges smoothly with early stopping preventing overfitting.
Training and validation curves: loss decreases, accuracy plateaus at 88.62%, no severe overfitting.
The validation curve flattens around epoch 13, confirming that early stopping prevented the model from memorizing training data. Final test accuracy: 88.62%.
Step 6: Confusion matrix analysis — false negatives are critical
In medical imaging, missing a pneumonia case (false negative) is worse than a false alarm (false positive). Our model achieves 98.46% recall—catching nearly all pneumonia cases.
Our model misses only 7 pneumonia cases out of 414 (1.7% false negative rate). This is production-ready: radiologists will catch the rare missed case on second review, but we won't miss many.
Step 7: ROC curve and AUC — Priya's acceptance metric
Priya set a hard constraint: AUC > 0.95 before deployment. Our model achieves AUC 0.9535—passing the requirement by 33 basis points.
ROC curve on test set: AUC = 0.9535. The curve stays close to the top-left (high TPR, low FPR) across all thresholds, indicating excellent class separation.
AUC measures the probability that the model ranks a random positive sample higher than a random negative sample. 0.9535 means 95.35% of the time, the model correctly orders pneumonia > normal. This exceeds Priya's 0.95 threshold.
Step 8: Clinical threshold tuning — sensitivity vs specificity trade-off
By default, we use threshold 0.5 (model output 0 or 1). But in healthcare, we can shift the threshold to prioritize sensitivity (catch all pneumonia) over specificity (avoid false alarms).
threshold_tuning.pyAdjust decision threshold for clinical needs
At threshold 0.35: sensitivity 98.8%, specificity 58.9%. This means we catch 98.8% of pneumonia cases but flag 41% of negatives as false alarms. Radiologists review all flagged cases; missing pneumonia is worse than one false alarm per patient.
Extending the project — GradCAM, multi-class, deployment
GradCAM: Visualize what the model sees
Radiologists need to trust AI. GradCAM highlights which regions of the X-ray influenced the pneumonia prediction, making the model interpretable.
Multi-class extension
Future version: classify pneumonia type (bacterial, viral, fungal) or other lung diseases. Use softmax instead of sigmoid, change loss to CrossEntropyLoss.
Deployment to production
Convert to ONNX for edge deployment, wrap in a Flask/FastAPI server, add HIPAA-compliant logging, integrate with PACS (Picture Archiving System) for real hospital workflows.
Interview Q&A: Medical AI & Evaluation
In medical imaging, classes are often imbalanced (e.g., 40% normal, 60% pneumonia). Accuracy alone is misleading: a model that predicts "always pneumonia" achieves 60% accuracy but is useless. AUC (Area Under the ROC Curve) measures how well the model separates classes across all thresholds, independent of class balance. AUC = 0.5 is random; AUC = 1.0 is perfect. Our 0.9535 AUC means the model is very confident in its predictions.
Sensitivity (recall) = TP / (TP + FN) = fraction of actual pneumonia cases caught. Specificity = TN / (TN + FP) = fraction of normal cases correctly identified. In healthcare, false negatives (missing pneumonia) are deadly; false positives (flagging a normal as pneumonia) just trigger a second review. So radiologists prefer high sensitivity (98%+) even if specificity drops to 60-70%. At threshold 0.35, we achieve 98.8% sensitivity—missing pneumonia becomes rare.
Batch normalization (BatchNorm) standardizes the output of each layer to have mean 0 and variance 1. This stabilizes training (allows higher learning rates), prevents internal covariate shift, and acts as a regularizer (reduces overfitting). In our CNN, we apply BatchNorm after each convolution layer. This is why our model converges smoothly without diverging.
Transfer learning leverages features learned on ImageNet (1.2M images, 1K classes). ResNet18 already knows how to detect edges, textures, and shapes—we only fine-tune the final layers for X-rays. This saves training time, improves generalization on small datasets, and achieves better AUC with less data. From scratch, we'd need 10K+ X-rays; with transfer learning, 4K is enough.
Early stopping monitors validation AUC during training and halts when it stops improving, preventing overfitting. In medical AI, overfitting is dangerous: a model that memorizes training data will fail on new patients. We stopped at epoch 13 when validation AUC peaked at 0.9525, then plateaued. Continuing would degrade generalization. This is non-negotiable for clinical deployment.
Test your understanding
Question 1 of 5
What is the advantage of AUC over accuracy for imbalanced datasets like our X-ray data (40% normal, 60% pneumonia)?
Question 2 of 5
At threshold 0.35, our model achieves 98.8% sensitivity but only 58.9% specificity. In a healthcare setting, is this trade-off acceptable?
Question 3 of 5
What does batch normalization do in our CNN?
Question 4 of 5
Why did we use transfer learning (ResNet18 pretrained on ImageNet) instead of training a CNN from scratch?
Question 5 of 5
At which epoch did we stop training and why?
0
/ 5
Great job!
Cheatsheet: Medical Image Classification Pipeline
Step
Key Code / Concept
Why It Matters
Load Data
np.load('file.npz'), reshape to (N, 1, 28, 28)
Medical images are grayscale; 1 channel only. Normalize to [0, 1].
Augmentation
RandomRotation, Flip, Affine
Rotations/flips mimic real X-ray variations. Train-only; don't augment val/test.
Model
ResNet18(pretrained=True) + fine-tune FC layer
Transfer learning is 10x faster. 11.2M params, pre-trained edge detection.
Loss
nn.BCELoss() for binary classification
Sigmoid output + BCE = well-calibrated probability predictions.
Optimization
optim.Adam(lr=1e-3) + early stopping on val AUC
Adam adapts learning rate. Early stopping prevents overfitting.
Try thresholds 0.3, 0.4, 0.5 for sensitivity vs specificity
Clinical priority: high sensitivity (catch all pneumonia) > specificity.
Deployment
Save model: torch.save(model.state_dict(), 'model.pth')
Convert to ONNX, wrap in FastAPI, log predictions for audit trail.
Common mistakes in medical AI
Mistake #1: Using accuracy as the main metric
Fix: Always use AUC, sensitivity, and specificity. Accuracy is misleading for imbalanced data (e.g., 60% positive class). A model that always predicts "positive" achieves 60% accuracy but is useless.
Mistake #2: Ignoring the false negative cost
Fix: In healthcare, false negatives are deadly. Prioritize sensitivity > specificity. Miss 1% of pneumonia cases? That's unacceptable. Flag 40% of normals as positive? Radiologists will review—acceptable cost.
Mistake #3: Training on imbalanced data without adjustment
Fix: Use class_weight in loss function to penalize false negatives more. Or oversample the minority class. Our data is 40/60 split—manageable, but larger imbalances need adjustment.
Mistake #4: No data augmentation for medical images
Fix: Medical data is scarce. Augment with rotations, flips, brightness shifts. These are realistic (radiologists view images at different angles) and improve generalization 5-10%.
Mistake #5: Forgetting early stopping
Fix: Monitor validation AUC, stop when it plateaus. Training longer than necessary only overfits. We stopped at epoch 13; epoch 20 would degrade generalization.
Key takeaways
Arc C capstone: Mini projects combine data loading, CNN design, advanced evaluation, and clinical thinking. This is what production ML looks like.
AUC > accuracy: For imbalanced medical data, AUC is the gold standard. Our 0.9535 AUC means the model correctly ranks pneumonia > normal 95.35% of the time.
Sensitivity vs specificity: Healthcare prioritizes catching all cases (high sensitivity) over avoiding false alarms (high specificity). Threshold 0.35 achieves 98.8% sensitivity—missing pneumonia is rare.
Transfer learning wins: ResNet18 pretrained on ImageNet costs 10x less to train than a CNN from scratch, with better accuracy. Use it whenever possible for small medical datasets.
Early stopping is non-negotiable: Stop when validation AUC plateaus. We stopped at epoch 13. Overfitting on 4K training images is easy; early stopping prevents it.
Clinical threshold tuning: The default 0.5 threshold isn't always right. Doctors tune thresholds based on their risk tolerance. High sensitivity (catch all disease) is the priority in diagnosis.
Interpretability matters: Next step: GradCAM visualization so radiologists understand why the model flagged an X-ray. Black-box models are not trusted in healthcare.