Training a large AI model across many GPUs fails in a specific and expensive way: one stalled GPU halts the whole run, and finding the cause has meant reproducing the failure by hand. PyTorch 2.15 attacks that failure mode directly. The PyTorch Foundation released PyTorch 2.15 on October 28, 2026, composed of 3,117 commits from 583 contributors since PyTorch 2.14 (PyTorch 2.15 release blog, retrieved 2026-10-29). Its headline change makes the rewritten NCCL communication backend the default for every NVIDIA GPU job, and pairs that switch with a flight recorder that writes a post-mortem to disk when a collective operation hangs. For teams that train or serve models at scale, this is a reliability release first and a speed release second.
Key Takeaways
PyTorch 2.15 was released on October 28, 2026, with 3,117 commits from 583 contributors since PyTorch 2.14.
The rewritten NCCL backend is now the default for every nccl process group, with a flight recorder that dumps diagnostics when a collective hangs.
One-sided transports arrive with NIXL as the first backend, and FSDP2 gains finer control over precision, gradient accumulation, and communication overlap.
Decoding gets faster on both ends of the hardware range: NVFP4 decode on NVIDIA Blackwell GPUs and FP8 plus grouped matrix kernels on Apple Silicon.
Upgrading has boundaries to check: Python 3.10 and macOS 14 support end, distributed checkpoints from 2.15 cannot be loaded by older releases, and the nccl-legacy fallback is removed in the next release.
The release continues a direction the PyTorch team has described across the 2.x series: evolving from a research-first framework into a unified, hardware-agnostic platform for production training and inference at scale (PyTorch 2.15 release blog). PyTorch 2.13 brought FlexAttention to Apple Silicon and introduced torchcomms for large-cluster training. PyTorch 2.14 matured that work into NVGEMM and previewed the rewritten NCCL backend as an opt-in choice. PyTorch 2.15 finishes the handover: the preview becomes the default, and the surrounding tooling for debugging, precision, and hardware coverage catches up with it.
What Is PyTorch?
PyTorch is an open source machine learning framework for building, training, and running neural networks. Teams use it to write models in Python, run them on CPUs and accelerators, and scale the same code from a laptop experiment to a multi-GPU training cluster. Its release cycle is run by the PyTorch Foundation, and each minor release bundles thousands of contributions from across the community. Version 2.15 is best understood as infrastructure work: most of its changes sit below the model code, in how GPUs talk to each other, how memory is managed, and how work is scheduled on different chips.
Aspect | PyTorch 2.15 |
|---|---|
Released | October 28, 2026 |
Size of release | 3,117 commits from 583 contributors since 2.14 |
Headline change | Rewritten NCCL backend becomes the default, with a flight recorder for hangs |
Distributed training | One-sided NIXL transports, finer FSDP2 and pipeline control |
Decoding | NVFP4 on NVIDIA Blackwell, FP8 and grouped_mm on Apple Silicon |
Hardware coverage | NVIDIA, Apple Silicon, AMD gfx950 and gfx1250, Intel GPUs |
Python support | Python 3.11 to 3.15, including torch.compile on Python 3.15 |
Rollback path | TORCH_DIST_USE_NCCL2=0 or nccl-legacy, removed in the next release |
Distributed Training Gets a New Default
NCCL is NVIDIA's library for collective communication between GPUs: operations such as all-reduce and all-gather, in which every GPU in a job exchanges data with the others. When a model trains across many GPUs, those GPUs constantly exchange gradients and activations through this backend, and a slow or stalled exchange slows the whole job. PyTorch 2.14 previewed a rewritten NCCL backend, ported from torchcomms, as the opt-in nccl2 backend. In PyTorch 2.15, every job that uses nccl runs on the rewritten backend by default (PyTorch 2.15 release blog).
The most practical addition is the flight recorder. A flight recorder is a backend-agnostic diagnostic tool that captures what each rank was doing at the moment of a hang and writes a post-mortem to disk. A rank is one process in a distributed job. Watchdog health is published to the debug server, so a timeout on any rank triggers flight-recorder dumps from every rank at once. At cluster scale, that turns a hang from a mystery to be reproduced into evidence to be read. Reconfiguration can also opt into a shrink or grow fast path on NCCL 2.29 or newer. Surviving ranks then skip a full bootstrap when a job changes shape. Reconfiguration for fault tolerance is enabled on ROCm with RCCL 2.30.7.
The escape hatches are temporary, and the release notes are explicit about it. You can restore the old behavior in 2.15 by setting TORCH_DIST_USE_NCCL2=0 or by selecting the nccl-legacy backend, but both paths will be removed in the next release (PyTorch 2.15.0 release notes). Treat the legacy backend as a debugging bridge, not a migration plan. Teams that still depend on it should schedule the move now, while the fallback still exists to compare behavior against.
One-Sided Transports and Finer FSDP2 Control
PyTorch 2.15 also adds a second way for processes to move data. One-sided communication is data movement in which one rank reads from or writes to a remote buffer without the remote rank issuing a matching call at the same moment. The new Python transport API in torch.distributed supports that model, with NIXL as its first backend. A transport connects to a peer through a store. It needs no process group or collective operation. Transport.read_stream and Transport.write_stream enqueue transfers on a CUDA stream so they can be captured with torch.cuda.graph (PyTorch 2.15 release blog). That shape suits disaggregated inference, KV-cache migration, parameter servers, and reinforcement learning systems whose producers and consumers run on different schedules.
FSDP2 is PyTorch's fully sharded data parallel strategy: it shards model parameters, gradients, and optimizer state across GPUs so a model larger than one GPU's memory can still train. Version 2.15 gives it finer control over precision and capture. Precision-sensitive parameters, such as mixture-of-experts router gates and norm weights, can stay in fp32 while the rest of the model trains in bf16, still with one all-gather per fully_shard unit, through MixedPrecisionPolicy with a param_dtype_override_fn. A full gradient-accumulation step can now be captured in a single CUDA graph, gradient precision can be set per parameter, and gradient reduction can overlap with compute. One default changes and deserves attention: when reduce_dtype is not set, gradients now accumulate and reduce in each parameter's grad_dtype instead of in param_dtype (PyTorch 2.15.0 release notes). If your training relied on the old default, set reduce_dtype explicitly.
Pipeline parallelism gets the same treatment. Receive buffers are now allocated just before each receive, not for every microbatch up front. Forward outputs are released once their sends complete. Schedules accept a max_outstanding_sends budget. FSDP unshards are staggered across each rank's startup bubble, not all issued at the start of a step. Each unique edge between pipeline-parallel ranks now gets a dedicated peer-to-peer communicator. Together, the release notes describe the result as pipelining that is cudagraphable and deterministic, with an optimized memory profile.
Faster Decoding on Blackwell and Apple Silicon
NVFP4 is NVIDIA Blackwell's native 4-bit floating-point format for inference. Because decoding a language model one token at a time is often limited by kernel launch overhead rather than raw arithmetic, removing launches matters more than adding peak throughput. NVGEMM, the CuTeDSL GEMM backend added in 2.14, now folds the scalar dequantization factor of ModelOpt NVFP4 linear layers into the GEMM itself. That removes a separate pointwise launch per layer during decode. Its kernels can use programmatic dependent launch so a Triton consumer overlaps with the tail of the producer GEMM (PyTorch 2.15 release blog). The practical effect is fewer, better-overlapped kernel launches on Blackwell decode.
Apple Silicon gains the matching low-precision story. FP8 is an 8-bit floating-point format that stores each value in half the memory of bf16, which matters on machines where memory is shared between CPU and GPU. PyTorch 2.15 adds float8_e4m3fn support on MPS, along with _scaled_mm with tensorwise or rowwise scaling. A new grouped_mm has a decode fast path delivering 1.32 to 1.34x tokens per second on mixture-of-experts decode on an M3 Ultra. Public torch.mps.Stream APIs, modeled on CUDA streams, complete the set (PyTorch 2.15 release blog). The kernel work around it is just as concrete. F.linear now uses existing gemv kernels for a 1.23x geometric-mean speedup and up to 5x on small projections. A single-row softmax over a 128,256-entry vocabulary drops by 8x. Upsample backward is up to 21.7x faster. Norm full reductions are more than 100x faster. A QR factorization of a 512 by 512 matrix falls from about 7 seconds to 1.5 milliseconds. Those figures are the project's own measurements on its hardware, so repeat them on your own models before quoting them as your gains.
Hardware Coverage Widens in Every Direction
The release extends PyTorch's hardware map on all four major fronts. On AMD, FlyDSL joins CuTeDSL, Triton, and Helion as a kernel language PyTorch can use. Inductor autotunes FlyDSL templates for GEMM, ragged grouped_mm, and MXFP4 and MXFP8 scaled_mm on gfx950. The gfx1250, the MI450 generation, gains Tensor Data Mover support for dense matrix multiply, pointwise, and reduction code generation, plus FlexAttention forward and decode. ROCm 10.0 wheels ship alongside the ROCm 7.14 wheels from 2.14, all built on AMD's TheRock packages. On Intel GPUs, SYCL free-function kernel support replaces verbose functor-based implementations for most existing kernels. XPU gains event sharing across processes and MemPool allocation policies. Profiling adds kernel-level visibility into XPU Graph replay. The default wheel moves to CUDA 13.2, with CUDA 13.4 wheels alongside it, including experimental support for NVIDIA Rubin GPUs. Large clusters like these usually run on orchestrated infrastructure. Our Kubernetes explainer covers what that layer does and why it runs the cloud. Python support spans 3.11 to 3.15, including the free-threaded 3.14t and 3.15t builds, with torch.compile support for Python 3.15 (PyTorch 2.15 release blog).
Before You Upgrade
PyTorch 2.15 is a minor release with several hard boundaries. Check each of these before you roll it out:
Python 3.10 is no longer supported. Version 2.15 requires Python 3.11 or newer, 3.10 wheels are no longer built, and 3.10 is no longer tested in CI.
macOS 14 is no longer supported for MPS acceleration. Version 2.15 requires macOS 15 or newer for Apple Silicon GPU work.
Distributed checkpoints saved with PyTorch 2.15 cannot be loaded by PyTorch 2.14 or earlier, because checkpoint metadata now records tensor strides. Keep a rollback-compatible checkpoint until the upgrade is accepted.
FSDP2 gradient behavior changes when reduce_dtype is not set, as described above. Set it explicitly if you need the 2.14 behavior.
C++ warnings from torch.ops and custom ops now propagate to Python, and import torch can no longer be re-executed in the same process.
The nccl-legacy backend and TORCH_DIST_USE_NCCL2=0 exist only in this release. Plan the migration before the next release removes them.
The PyTorch team will also hold a Q and A webinar on Thursday, November 5, 2026, at 11 AM Pacific Time, with Tristan Rice, Natalia Gimelshein, Huy Do, and Jane Xu presenting the release and answering questions (PyTorch 2.15 release blog). If your team runs multi-GPU training in production, the flight recorder and the legacy backend timeline are the two topics worth raising there.
What Independent Coverage Says
Futurum Group's coverage calls 2.15 the release where PyTorch's production ambitions become concrete engineering commitments. It argues that the hardened default backend, systematic hang diagnostics, and broad hardware coverage show a framework engineered for enterprise AI teams, not research labs alone. It cites a Futurum survey finding that 55.4% of AI decision makers name reliability and hallucination management as a top production challenge (Futurum Group, October 29, 2026). That framing is fair as far as it goes, but it is an analyst's reading, not a measured result: the release itself proves the tooling exists, not that any particular team's jobs got faster or more reliable.
AlphaSignal's breakdown reaches the same center of gravity from the engineering side. It concentrates on distributed training, low-precision decoding, and accelerator support. It flags the same three compatibility changes for production teams: Python 3.10 support has ended, C++20 is now required for native code, and distributed checkpoints from 2.15 cannot be loaded by earlier releases (AlphaSignal, October 28, 2026). Its practical advice is the right one: multi-node teams should test NCCL collective performance, hang diagnostics, and failure recovery first, and jobs using the legacy fallback need a migration plan before the next release removes that path.
Frequently Asked Questions
Is PyTorch 2.15 free? Yes. PyTorch is open source software, and upgrading to 2.15 costs nothing for the framework itself. Your costs are the hardware, cloud compute, and engineering time you already spend running training or inference jobs.
Do I have to change my code to use the new NCCL backend? No. Every process group created with the nccl backend now uses the rewritten implementation by default, with no code change required. If you need the old behavior temporarily, set TORCH_DIST_USE_NCCL2=0 or select the nccl-legacy backend, and plan to migrate before the next release removes those paths.
What does the flight recorder do when a training job hangs? It writes a post-mortem to disk. The post-mortem captures what each rank was doing at the moment of the hang. A timeout on any rank triggers dumps from every rank through the debug server. That gives you evidence about the stalled rank and operation instead of requiring you to reproduce the failure by hand.
Can I load a PyTorch 2.15 distributed checkpoint in PyTorch 2.14? No. Checkpoint metadata in 2.15 records tensor strides, so checkpoints saved with 2.15 cannot be loaded by 2.14 or earlier. Keep an older checkpoint available if you may need to roll back.
What Python versions does PyTorch 2.15 support? Python 3.11 through 3.15, including the free-threaded 3.14t and 3.15t builds, with torch.compile support for Python 3.15. Python 3.10 is no longer supported, and macOS 15 or newer is required for MPS acceleration on Apple Silicon.
The Bottom Line
PyTorch 2.15 will not change what your model can do. It changes how reliably you can train and serve it at scale, and how many kinds of hardware you can do that on. The rewritten NCCL backend and its flight recorder address the failure mode that wastes the most expensive resource in AI work, which is GPU time spent on a stalled job. The NIXL transports, FSDP2 controls, and decode kernels give infrastructure teams finer tools for the same jobs. And the upgrade boundaries are unusually clear for a minor release, which is a kindness: check Python, macOS, checkpoints, and the legacy backend timeline before you upgrade, not after. Start with the PyTorch 2.15 release blog and the full release notes on GitHub. If distributed inference at scale is your problem, our llm-d 1.0 article covers a production inference stack built for the same world, or read the Indonesian version of this article.
How this article was checked: we analyzed the official release blog and release notes, then compared them with Futurum Group and AlphaSignal coverage. This article passed editorial review before publication.