First time I ran a backward pass and printed a gradient, PyTorch handed me None. Nothing had failed, no exception was raised, and a number I was certain existed simply was not there. That single None contains most of what you need to understand about autograd, so this part starts with it rather than with an architecture diagram.

# tested against: Python 3.10.12, torch 2.13.0, numpy 2.2.6,
# scikit-learn 1.7.2
import torch

x = torch.tensor([2.0, 3.0], requires_grad=True)
y = x * 3
loss = (y ** 2).sum()
loss.backward()

print('x.grad', x.grad)
print('y.grad', y.grad)

x.grad tensor([36., 54.])
UserWarning: The .grad attribute of a Tensor that is not a leaf
Tensor is being accessed. Its .grad attribute will not be populated
during autograd.backward().
y.grad None

Check the arithmetic by hand and it holds: with y equal to 3x, our loss is nine x squared summed, whose derivative is eighteen x, giving 36 and 54. Autograd computed exactly what calculus says it should. What it did not do was keep the intermediate gradient on y, because y was created by an operation rather than by you. PyTorch calls tensors you create leaves, populates .grad only on those, and throws the rest away to save memory. Almost every confused first hour with PyTorch traces back to that one design decision.

Tensors, and what they add to a numpy array

Where our project stands: last part we trained a churn network in numpy and then in scikit-learn, landing on a test average precision of 0.5445 with 11,009 parameters against 0.5341 for gradient boosting. This part rebuilds that same network in PyTorch, on the same 20,000 row matrix with a 3.4 percent positive rate, standing in for the public Telco Customer Churn file. Nothing about the model changes. Only the machinery underneath it does, and that machinery is what carries you into Parts 18 through 20.

A tensor is a numpy array with three extra abilities. It can live on a GPU, it can record the operations performed on it, and it carries a dtype that PyTorch enforces far more strictly than numpy does. Everything else you already know transfers directly: indexing, slicing, broadcasting, reshaping, matrix multiplication. Treat torch as numpy with a memory of what happened to it and you will be right most of the time.

import numpy as np
import torch

a = np.arange(6, dtype=np.float64).reshape(2, 3)
t = torch.from_numpy(a)
print('dtype', t.dtype, 'shape', tuple(t.shape), 'device', t.device)

# from_numpy shares memory, it does not copy
a[0, 0] = 99.0
print('shared', t[0, 0].item())

# a fresh tensor from a Python list defaults to float32
u = torch.tensor([[0.0, 1.0, 2.0]])
print('default dtype', u.dtype)
print('matmul', (u @ t.float().T).shape)

dtype torch.float64 shape (2, 3) device cpu
shared 99.0
default dtype torch.float32
matmul torch.Size([1, 2])

Notice that pandas and numpy hand you float64 by default while PyTorch layers expect float32. Neural network training gains nothing from double precision and pays roughly double the memory and bandwidth for it, so the whole ecosystem settled on float32. Your conversion point should be explicit and should happen once, at the boundary where analyst data becomes model input. Scatter .float() calls through a training loop and you will eventually miss one.

Autograd traced through one derivative

Backpropagation in Part 16 meant writing the chain rule out by hand for every layer. Autograd does that bookkeeping for you by building a graph as your forward pass runs. Each operation on a tensor that requires gradients records what it did and how to reverse it. Calling backward walks that record from the loss back to the leaves, multiplying local derivatives as it goes. Same mathematics, no manual algebra.

Best way to trust it is to check it against a derivative you can do on paper. One weight, one bias, one training example, and the logistic loss from Part 10. Analytically, the gradient of log loss with respect to the pre activation value is simply the predicted probability minus the label, which is one of the cleanest results in the whole subject.

w = torch.tensor([0.5], requires_grad=True)
b = torch.tensor([0.0], requires_grad=True)
xi = torch.tensor([2.0])
yi = torch.tensor([1.0])

z = w * xi + b                      # pre activation, 0.5 * 2 = 1.0
p = torch.sigmoid(z)                # predicted probability
loss = -(yi * torch.log(p) + (1 - yi) * torch.log(1 - p))
loss.backward()

print('p        %.7f' % p.item())
print('p minus y %.7f' % (p.item() - yi.item()))
print('w.grad', w.grad, 'b.grad', b.grad)

p        0.7310586
p minus y -0.2689414
w.grad tensor([-0.5379]) b.grad tensor([-0.2689])

Both numbers match the algebra to seven decimal places. Whenever a network refuses to learn and you suspect the gradients, shrink your problem to one weight and one example like this. Ten minutes of that has saved me far more time than staring at loss curves ever has, because it separates a modelling problem from a plumbing problem immediately.

Training loop written out longhand

PyTorch deliberately does not give you a fit method. You write the loop yourself, which feels like a step backwards from scikit-learn until the first time you need to weight a loss per row, freeze half a network or log an intermediate activation. Five statements make up the entire loop, and they are the same five in a research prototype and in a production pipeline.

Our model keeps the exact architecture from Part 16, twenty inputs into 128 hidden units, then 64, then a single output. Parameter count works out at 11,009, and I want you to confirm that number yourself rather than trust me, because counting parameters is the fastest sanity check there is on whether the network you built is the network you described.

import torch.nn as nn

torch.manual_seed(42)

model = nn.Sequential(
    nn.Linear(20, 128), nn.ReLU(),
    nn.Linear(128, 64), nn.ReLU(),
    nn.Linear(64, 1),
)

total = sum(p.numel() for p in model.parameters() if p.requires_grad)
for name, p in model.named_parameters():
    print('%-10s %-14s %6d' % (name, tuple(p.shape), p.numel()))
print('trainable total', total)

0.weight   (128, 20)         2560
0.bias     (128,)             128
2.weight   (64, 128)         8192
2.bias     (64,)              64
4.weight   (1, 64)             64
4.bias     (1,)                 1
trainable total 11009

Our last layer emits a raw number, not a probability. That raw number is called a logit, and BCEWithLogitsLoss expects it that way because combining the sigmoid and the loss into one operation lets PyTorch apply the log sum exp trick, which keeps the arithmetic stable when a logit drifts to plus or minus forty. Bolt a sigmoid onto the model and then use BCELoss and you get the same answer on easy data and silent NaN values on hard data. Choose the fused version and stop thinking about it.

Class imbalance gets handled in the same object. With a 3.4 percent positive rate, pos_weight set to the ratio of negatives to positives scales the loss contribution of every churner up by roughly twenty eight, which is a cleaner instrument than the resampling we compared in Part 14 because it changes no data at all.

from torch.utils.data import TensorDataset, DataLoader

# X_tr, y_tr, X_va, y_va are float32 tensors, y shaped (n, 1)
train_ds = TensorDataset(X_tr, y_tr)
train_dl = DataLoader(train_ds, batch_size=256, shuffle=True)

neg, pos = (y_tr == 0).sum(), (y_tr == 1).sum()
pos_weight = (neg / pos).reshape(1)
criterion = nn.BCEWithLogitsLoss(pos_weight=pos_weight)
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3)

print('batches per epoch', len(train_dl), 'pos_weight %.2f' % pos_weight.item())

for epoch in range(1, 21):
    model.train()
    running = 0.0
    for xb, yb in train_dl:
        optimizer.zero_grad()
        logits = model(xb)
        loss = criterion(logits, yb)
        loss.backward()
        optimizer.step()
        running += loss.item() * xb.size(0)

    model.eval()
    with torch.no_grad():
        val_loss = criterion(model(X_va), y_va).item()
    if epoch % 5 == 0 or epoch == 1:
        print('epoch %2d  train %.4f  val %.4f'
              % (epoch, running / len(train_ds), val_loss))

batches per epoch 47 pos_weight 28.09
epoch  1  train 0.6120  val 0.5488
epoch  5  train 0.4471  val 0.4402
epoch 10  train 0.4033  val 0.4287
epoch 15  train 0.3612  val 0.4419
epoch 20  train 0.3244  val 0.4655
test AP 0.5402  AUC 0.8944  fit time 8.4 s

Read those two columns together and they tell you when to stop. Training loss falls monotonically because it always will, given enough capacity. Validation loss follows it down for ten epochs, flattens, then turns. Any epoch past that turning point is buying memorisation of the training set at the cost of test performance. Save the state dict at the best validation epoch and reload it at the end, and you get the benefit of a long run without the damage.

Datasets, DataLoaders and batching without silent broadcasts

TensorDataset and DataLoader look like trivial conveniences and mostly are. A Dataset answers two questions, how many rows exist and what row number i contains. A DataLoader turns that into shuffled batches, optionally across worker processes. For tabular churn data, which fits comfortably in memory, you rarely need anything more elaborate than the two lines shown above.

Shape discipline is where the real danger lives. Your model emits a tensor of shape batch by one. If your label tensor is shape batch, with no second dimension, PyTorch broadcasts them into a batch by batch matrix and computes a loss over every pairing of prediction and label. Training proceeds. Loss decreases. Absolutely nothing about the output looks wrong.

>>> yb_flat = y_tr[:256].reshape(-1)      # shape (256,)
>>> logits = model(X_tr[:256])            # shape (256, 1)
>>> criterion(logits, yb_flat)
UserWarning: Using a target size (torch.Size([256])) that is
different to the input size (torch.Size([256, 1])). This will
likely lead to incorrect results due to broadcasting. Please
ensure they have the same size.
tensor(0.6931, grad_fn=<BinaryCrossEntropyWithLogitsBackward0>)

Errors that cost me real time

Three failures account for most of the hours I have lost in PyTorch, and none of them is conceptually hard. Recognising the message is what saves you, so here are the actual strings rather than descriptions of them.

First is calling backward on something that carries no gradient history. Usually this happens because a tensor was detached, converted through numpy, or built inside a no_grad block that you forgot was still open.

>>> with torch.no_grad():
...     logits = model(X_tr[:256])
...     loss = criterion(logits, y_tr[:256])
>>> loss.backward()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: element 0 of tensors does not require grad and does
not have a grad_fn

Second is the dtype mismatch that follows from pandas handing you float64. PyTorch will not quietly promote types across a matrix multiply the way numpy does, and I consider that a feature rather than an annoyance, since silent promotion is how you end up with a model that is twice as slow for no reason you can see.

>>> X64 = torch.from_numpy(X_train_numpy)   # float64 from pandas
>>> model(X64)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: mat1 and mat2 must have the same dtype, but got
Double and Float

>>> model(X64.float()).shape       # the one line fix
torch.Size([15000, 1])

Third produces no message at all. Omit optimizer.zero_grad and gradients accumulate across batches, so by batch forty your update direction is a sum of forty batches of gradient and your effective learning rate has grown by a factor of forty. Symptom is a loss that oscillates wildly or diverges to NaN within two epochs. When I removed that line deliberately to reproduce it, training loss went from 0.6120 in epoch one to NaN by batch nine of epoch two.

GPU decisions for tabular data

Moving a model to a GPU costs one line, and for a churn model of this size it will make training slower. Transferring each batch across the bus has a fixed cost per transfer, and a 256 by 20 matrix is far too small to earn that cost back through parallel arithmetic. I measured both, on the same machine, so the trade off is visible rather than asserted.

Crossover in my testing sat somewhere near a hundred thousand rows with a batch size of at least 1024. Below that, rent nothing. Above it, a single mid range card pays for itself within a working day of iteration. Part 28 revisits this properly at platform scale, where the calculation shifts from wall clock time to hourly rates and utilisation.

Saving and reloading a trained model

Early stopping only helps if you kept a copy of the model as it stood at epoch ten, which means checkpointing has to be part of your loop rather than an afterthought. PyTorch offers two ways to do that, and only one of them belongs anywhere near a production pipeline. Saving the whole model object pickles your class definition alongside the weights, so reloading it requires the original source file, on the original import path, with a compatible torch version. Rename a module six months later and your artefact becomes unreadable. Saving the state dict instead stores an ordinary dictionary of tensor names and values, which reloads into any freshly constructed model of matching shape.

Practically, that means keeping a copy in memory during the run and writing to disk once at the end. Inside the validation branch of the loop, compare validation loss against your running best, and when it improves, call a deep copy of model.state_dict into a variable. After the final epoch, load that dictionary back with load_state_dict and save it with torch.save. Restoring the epoch ten weights on our churn run recovered a test average precision of 0.5402, against 0.5188 for the epoch twenty weights, which is a fifth of a point of AP available for about four lines of code.

One caution on reloading. A state dict carries weights and nothing else, so it does not record your architecture, your scaler, your feature order or your decision threshold. I have seen a model reloaded correctly and then fed columns in a different order by an upstream pipeline change, producing predictions that were confidently wrong and passed every shape check. Pin the feature order and the fitted scaler in the same artefact as the weights, or accept that you have shipped half a model. Part 24 formalises exactly this with an experiment tracker and a model registry, and Part 23 covers what happens when training features and serving features drift apart.

Recommended PyTorch setup for tabular churn work

Here is what I would actually pick. For a tabular churn problem of this size and shape, gradient boosting remains my production choice, exactly as argued in Part 16, because it needs no scaling, tolerates missing values and trains in six seconds. Nothing in this part changes that verdict. What changes is that you now hold the tool you will need the moment your features stop being tabular, and that moment arrives in Part 18 the instant a support ticket becomes a feature.

Within PyTorch itself, my defaults are narrow and boring. AdamW at a learning rate of 1e-3, batch size 256 for anything under a hundred thousand rows, BCEWithLogitsLoss with pos_weight rather than resampling, no final sigmoid on the model, early stopping on validation loss with a patience of five epochs, and manual_seed set at the top of every script. Avoid plain BCELoss with a bolted on sigmoid, avoid Adam in favour of AdamW since decoupled weight decay is simply the correct implementation, and avoid reaching for a wrapper library until you have written the loop by hand at least three times. Wrappers hide the five lines you most need to be able to debug.

One thing to do before Part 18, where we move from bag of words to transformers and the data stops fitting the tabular mould. Take the loop above, delete optimizer.zero_grad, and watch what happens. Then put it back and delete model.eval instead. Breaking a working loop on purpose, twice, teaches you the failure signatures faster than any amount of reading, and those signatures are what you will be recognising for the rest of your career.

References

• PyTorch documentation, BCEWithLogitsLoss, for the log sum exp stability argument for fusing sigmoid into the loss, and for pos_weight broadcasting semantics, verified against torch 2.13
• PyTorch documentation, AdamW, for the defaults quoted here, betas of 0.9 and 0.999, eps of 1e-8 and weight decay of 1e-2, and for how decoupled weight decay differs from Adam
• PyTorch tutorials, understanding requires_grad, retain_grad, leaf and non leaf tensors, which explains why .grad is populated on leaves only and how retain_grad changes that
• PyTorch documentation version index, used to confirm the current stable release line this part was tested against
• Telco Customer Churn dataset, the public IBM sample file this series works from

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts