Verified sample · Technical question
Why do model.eval() and disabled gradient recording solve different problems?
This curated sample preserves the structure and wording of the founder edition. The full chapter also includes deeper framework behavior, regression code, follow-ups, a rubric, and claim-by-claim sources.
Scenario
A validation job produces different logits for the same image depending on the other images in its batch. GPU memory is lower than during training, but batch-normalization running means also change during validation.
What the interviewer is testing
Can you separate module mode from gradient recording, connect symptoms to affected modules, and protect the fix with an executable regression test?
Your 30-second answer
Two switches, two responsibilities.
torch.no_grad()disables autograd recording in its context; it does not put modules into evaluation mode.model.eval()switches modules such as dropout and batch normalization to their evaluation behavior; it does not disable gradients. Here the model stayed in train mode, so dropout remained active and batch normalization used batch statistics and updated running state. Callmodel.eval()for validation and also disable gradients when they are not needed, then restore train mode before training resumes.
FRAME answer
Make the reasoning easy to inspect.
F — Frame the problem
There are two independent switches: module behavior and autograd recording.
R — Recall the core principle
eval() changes registered module mode; no_grad() changes whether operations are recorded for differentiation.
A — Apply it to the evidence
Train-mode dropout explains stochastic logits, and train-mode batch normalization explains batch dependence and running-state updates.
M — Mention trade-offs
Evaluation mode is not a global determinism switch, and some evaluation procedures intentionally need gradients.
E — Extend the answer
Restore the previous module mode and add tests for logits, running buffers, and gradient state.
Visual intuition
Module behavior and autograd are independent axes.
Reading across a row changes autograd behavior. Reading down a column changes module behavior. Neither axis substitutes for the other.
Validation record
The answer is checked, not merely asserted.
Framework behavior was checked against PyTorch 2.13.0 documentation and an executable four-state reference test.
- Evaluation mode changes dropout behavior.
- Disabling gradients leaves train-mode module behavior intact.
- Batch-normalization running state can change in train mode under
no_grad(). - That running state remains fixed in evaluation mode.