🏥
Real-world scenario
Arjun at MedScan Diagnostics (Bangalore): process batches of X-ray images as tensors for a deep learning model

Arjun is an AI engineer at MedScan Diagnostics in Bangalore. He's building a CNN to detect lung abnormalities in X-ray scans. Each X-ray is a 224×224 grayscale image. When training, he processes 4 images at once in a batch. In PyTorch, this becomes a 4D tensor: [batch_size=4, channels=1, height=224, width=224]. Before building the neural network, Arjun must master tensor creation, indexing, reshaping, and PyTorch's killer feature—autograd.

Understanding tensors is the foundation. Without it, all the neural network code that follows is just magic. With it, you see exactly how data flows through every layer, how gradients propagate backward, and why GPU acceleration matters.

4
X-ray images per batch
224×224
Pixels per image
200,704
Elements per batch

Creating tensors — from lists, zeros, randn, arange

PyTorch tensors are multi-dimensional arrays—the core data structure for neural networks. You can create them from Python lists, pre-allocate with zeros or random values, or generate sequences.

create_tensors.py Five ways to create PyTorch tensors
Output
From list: tensor([1, 2, 3, 4, 5]) Shape: torch.Size([5]), dtype: torch.int64 2D tensor shape: torch.Size([2, 3]) Zeros shape: torch.Size([3, 4]) tensor([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]]) Random normal shape: torch.Size([5, 5]) Sample values: tensor([[-0.5234, 0.1892, -1.2481], [ 0.8752, -0.3105, 0.9261]]) Arange: tensor([0, 2, 4, 6, 8])

X-ray batch shape — the 4D medical imaging tensor

A batch of medical images in PyTorch follows the format: [batch_size, channels, height, width]. For Arjun's X-rays: 4 images, 1 channel (grayscale), 224×224 pixels = 4 × 1 × 224 × 224 = 200,704 elements total. This shape is crucial—neural networks expect exactly this layout.

xray_batch.py Create and inspect a batch of X-ray tensors
Output
Batch shape: torch.Size([4, 1, 224, 224]) Total elements: 200704 Memory size: 802816 bytes (~0.8 MB) Single image shape: torch.Size([1, 224, 224]) Pixel range: [-3.152, 2.894]
Anatomy of a 4D image tensor: [batch, channels, height, width]. Each dimension has a specific meaning for CNNs.
Diagram showing 4D tensor structure [4,1,224,224] with labeled dimensions

Medical vitals matrix — indexing and selecting rows/columns

2D tensors represent tabular data. For MedScan patients: 3 patients, 4 vital signs (temp, heart rate, systolic BP, respiratory rate). NumPy-like indexing extracts rows, columns, or subsets.

vitals.py Create and index a medical vitals matrix
Output
Vitals matrix shape: torch.Size([3, 4]) Matrix: tensor([[98.6000, 120.0000, 80.0000, 16.0000], [99.1000, 115.0000, 78.0000, 18.0000], [98.9000, 118.0000, 82.0000, 17.0000]]) First patient: tensor([98.6000, 120.0000, 80.0000, 16.0000]) All heart rates: tensor([120., 115., 118.]) Patients 1-2, systolic BP: tensor([78., 82.])

Element-wise operations — add, multiply, broadcast

PyTorch tensors support element-wise arithmetic just like NumPy. Broadcasting automatically aligns shapes when possible—critical for neural network forward passes.

element_wise.py Element-wise operations and broadcasting
Output
Addition: tensor([5., 7., 9.]) Multiplication: tensor([4., 10., 18.]) Division: tensor([4.0000, 2.5000, 2.0000]) Batch shape: torch.Size([4, 3]) Bias shape: torch.Size([3]) Result shape: torch.Size([4, 3]) ← broadcasted!

Dot product and matrix multiply — neural networks depend on these

Neural networks multiply data with weight matrices. The dot product (two vectors) and matrix multiplication (matrix × vector) are the core operations.

matmul.py Dot product and matrix multiplication
Output
Dot product: tensor(32) Weight matrix: tensor([[1., 2.], [3., 4.]]) Input: tensor([5., 6.]) Output: tensor([17., 39.])
In neural networks, W @ x computes the weighted sum for each neuron. Each row of W is one neuron's weights. This operation repeats millions of times during training.

Reshape and view — rearrange dimensions without copying

view() reshapes without copying data (fast); reshape() handles non-contiguous tensors and may copy. For most cases, use reshape().

reshape.py Reshape tensors to different dimensions
Output
Original: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]), shape: torch.Size([12]) Reshaped to (3, 4): tensor([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]]) Flattened: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]), shape: torch.Size([12])

Squeeze and unsqueeze — add or remove dimensions

unsqueeze() adds a dimension of size 1; squeeze() removes them. Critical for matching tensor shapes in networks.

squeeze.py Manipulate tensor dimensions
Output
Original: torch.Size([3]) Unsqueeze(0): torch.Size([1, 3]) Tensor: tensor([[1, 2, 3]]) Unsqueeze(1): torch.Size([3, 1]) Tensor: tensor([[1], [2], [3]]) Squeeze: torch.Size([3])

NumPy bridge — convert between NumPy and PyTorch

Data often starts in NumPy or pandas. PyTorch can convert without copying (shared memory)—modify one, the other updates too.

numpy_bridge.py Convert between NumPy arrays and PyTorch tensors
Output
NumPy: [1 2 3 4 5] PyTorch: tensor([1, 2, 3, 4, 5]) Shared memory: True PyTorch: tensor([10., 20., 30.]) NumPy: [10. 20. 30.] 2D NumPy shape: (2, 2) 2D PyTorch shape: torch.Size([2, 2])

Autograd — automatic gradient computation (PyTorch's killer feature)

PyTorch computes gradients automatically. Set requires_grad=True on inputs, run forward pass, call .backward(), then access gradients via .grad. This is how neural networks learn.

autograd.py Compute gradients with autograd
Output
Input: x = 3.0 Output: y = x² + 2x + 1 = 16.0 Gradient: dy/dx = 8.0 Expected: 2*x + 2 = 2*3 + 2 = 8 ✓ After loss.backward(): W.grad shape: torch.Size([2, 2]) b.grad shape: torch.Size([2])
Computation graph for y = x² + 2x + 1. Autograd traces this graph and computes gradients using the chain rule during backward().
Computation graph showing forward and backward passes for quadratic function
Autograd's magic: PyTorch tracks every operation, builds a computation graph, then automatically differentiates. This is why neural networks can learn—gradients flow backward, updating weights.

How AI systems use this

Every pixel in an X-ray image, every vital sign in a patient record, every token in text—all become tensors. PyTorch operations on tensors are the language of neural networks. Forward passes multiply data with learned weights. Backward passes (via autograd) compute gradients that update those weights. CNNs apply convolution operations across 4D image tensors. RNNs step through sequence tensors. Transformers manipulate 3D token embeddings. Understanding tensors is understanding how AI models work at the lowest level.

Tensor Fundamentals Autograd & Backprop GPU Acceleration Neural Network Input/Output

Also used in...

Domain Spotlight

🏥
Medical Imaging: X-ray & MRI processing
4D tensors [batch, channels, height, width] process scans. Autograd trains CNNs to detect tumors, fractures, lesions.
🚗
Autonomous Vehicles: Camera feeds as tensors
Video frames become 5D tensors [batch, frames, channels, height, width]. Real-time tensor ops power object detection.
💬
NLP: Embeddings & sequences as tensors
Word embeddings are 2D tensors [vocab_size, embedding_dim]. Sequences are 3D [batch, seq_length, embedding_dim].
🎯 Mini Challenge: Build a batch of custom X-rays

Task: Create a 4D tensor representing 8 X-ray images (224×224, 1 channel each). Add random noise to simulate real medical images, then extract the 5th image and reshape it to 1D for analysis.

Hint 1: xray_batch = torch.randn(8, 1, 224, 224)

Hint 2: Add noise: xray_batch += torch.randn_like(xray_batch) * 0.1

Hint 3: Extract 5th image: image_5 = xray_batch[4]

Hint 4: Flatten to 1D: image_1d = image_5.reshape(-1)

Expected shape: 1D tensor of 50,176 elements (224×224)

Interview questions: test your understanding

[4, 1, 224, 224]. Four images, one grayscale channel each, 224×224 pixels. PyTorch convention: [batch, channels, height, width].
4 × 1 × 224 × 224 = 200,704 elements. Use .numel() to compute this programmatically.
.view() requires a contiguous tensor and doesn't copy memory (fast). .reshape() handles non-contiguous tensors and may copy. For most cases, use .reshape() to be safe.
dy/dx = 2x + 2 = 2(3) + 2 = 8. Analytically, y = (x+1)², so dy/dx = 2(x+1). PyTorch's autograd computes this via y.backward().
Broadcasting. Just do tensor + bias. PyTorch automatically stretches [64] to [1000, 64] and adds element-wise. Same logic powers neural network forward passes.

Quiz: 5 questions to test your knowledge

Question 1 of 5
torch.randn((5, 5)) creates what?
Question 2 of 5
A tensor of shape [4, 1, 224, 224] has how many elements?
Question 3 of 5
vitals[1:3, 2] selects what from a 2D tensor?
Question 4 of 5
torch.dot([1, 2, 3], [4, 5, 6]) = ?
Question 5 of 5
Autograd computes gradients via what method?
0
/5
Excellent work! You've mastered PyTorch tensors.

Cheatsheet: PyTorch tensor operations at a glance

Operation Syntax Example
Create from list torch.tensor([...]) torch.tensor([1, 2, 3])
Zeros torch.zeros((m, n)) torch.zeros((3, 4))
Random normal torch.randn((m, n)) torch.randn((5, 5))
Arange torch.arange(start, end, step) torch.arange(0, 10, 2)
Dot product torch.dot(a, b) torch.dot(v1, v2)
Matrix multiply A @ B W @ x
Reshape tensor.reshape(shape) tensor.reshape(3, 4)
Unsqueeze tensor.unsqueeze(dim) tensor.unsqueeze(0)
Squeeze tensor.squeeze() tensor.squeeze()
NumPy bridge torch.from_numpy(np_arr) torch.from_numpy(data)
Autograd y.backward() loss.backward()

Common mistakes: what breaks and how to fix it

Mistake 1: Forgetting requires_grad=True when you need gradients
Tensors created by default don't track gradients. If you need gradients for training, set requires_grad=True. Fix: x = torch.tensor([1.0], requires_grad=True).
Mistake 2: Mixing tensor and NumPy operations
NumPy ops don't work on GPU tensors. Tensors on GPU are incompatible with NumPy. Always convert: np_array = tensor.cpu().numpy() before NumPy ops.
Mistake 3: Modifying tensors in-place during backprop
In-place operations (+=, .fill_) modify the computation graph, breaking gradients. Use: tensor = tensor + x instead of tensor += x during training.
Mistake 4: Shape mismatches in matrix multiplication
W @ x requires W's last dimension to match x's first. [2, 3] @ [3] works. [2, 3] @ [4] fails. Always check shapes before @.

Key takeaways

Tensors are the foundation of deep learning. Master them, and neural networks become clear.

  • Tensors are arrays: 0D scalars, 1D vectors, 2D matrices, 3D+ tensors. Images are 4D: [batch, channels, height, width].
  • Shape matters: Operations depend on tensor dimensions. Always know your shapes. Use .shape and .numel().
  • Broadcasting is powerful: PyTorch stretches dimensions automatically. Adding [64] bias to [1000, 64] data just works.
  • Autograd is magic: Set requires_grad=True, run forward, call .backward(), access .grad. This is how networks learn.
  • View vs reshape: view() is faster but requires contiguous tensors. reshape() is safer for general use.
  • GPU acceleration: Same code, 10-100× speedup on GPU. Critical for large models.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

You've mastered PyTorch tensors.
Now use them to build real neural networks.

Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Agentic Engineer →