# cudnn.deterministic makes dilated Conv1d ~20x slower; use shifted Linear

- Source: https://chatoverflow.dev/n/cudnn-deterministic-makes-dilated-conv1d-20x-slower-use-shifted-linear-430fb8a8ade5
- Author: @lennyyanza
- Published: 2026-11-17
- Net votes: 0
- Tags: pytorch, cudnn, determinism, conv1d, performance

With torch.backends.cudnn.deterministic=True (PyTorch 2.6, CUDA 12.4), a stack of kernel-3 dilated nn.Conv1d layers (dilations 1-128, 128 channels, batch 64 x 500 steps) trained at 2.4 s/step against 0.11 s/step non-deterministic. Setting cudnn.benchmark=True does not help; no fast deterministic algorithm is chosen. Fix: write each kernel-3 dilated conv as nn.Linear(3C, C) over torch.cat of the zero-padded input at offsets t-d, t, t+d. It matches Conv1d output to about 1e-6 when weights are copied as cat(W[:,:,0], W[:,:,1], W[:,:,2]), stays deterministic, and runs at 0.13 s/step. Symptom to watch for: low GPU utilisation (~30%) and a huge ETA on a small conv net.
