IT News on November 5th: NVIDIA officially released the latest CUDA Toolkit 13.5 on November 4th local time, providing official support (GA) for the next-generation AI computing platform Vera Rubin. This marks a transition of the relevant architecture from the early developer preview stage to the deployment phase for enterprise production environments.

According to NVIDIA's official introduction, this update not only improves the underlying software support for the Vera Rubin platform but also makes multiple upgrades to the GPU resource scheduling, multi-GPU communication, mathematical computation libraries, and performance analysis tools, with a focus on large-scale AI training and inference scenarios.

In terms of computational performance, CUDA v13.5 has updated the heuristic strategies for matrix multiplication and the kernel tuning mechanisms of cuBLAS. It has made optimizations for low-precision matrix operations involving NVFP4 and MXFP8 on the Vera Rubin architecture, covering training scenarios such as hybrid expert models (MoE). NVIDIA states that these improvements can enhance training and inference throughput without the need for automatic tuning for each individual computational configuration.

In terms of scheduling at GPU, CUDA has upgraded the multi-process service ( MPS ) v3 to include a new weighted time slice mechanism. Administrators can set longer basic calculation time slices for MPS servers running under heavy loads, and independently adjust the switching frequency of GPU between different tasks, thereby allowing for more flexible control over resource allocation in a multi-tasking environment.

MPS v3 also adds a streaming multiprocessor (SM) partition load balancing feature. Tasks can temporarily borrow unused resources from other partitions when resources are sufficient, and when the original partition needs to use these resources, the system will first reclaim the borrowed portion. Regardless of how resources dynamically flow, the basic partition size seen by the application remains stable. In addition, the new version also includes resource routing based on Linux cgroup and namespace-level SM partition configuration.

Local domains ( Locality Domain ) have also adopted a new resource allocation strategy. Applications can now choose from two SM partitioning methods: one is to allocate the same number of SM to each local domain to ensure consistency and predictability of resource scale; the other is to incorporate additional SM that were previously outside the partitions into the computing process, making full use of all the GPU computing resources. Developers can also query the actual amount of SM obtained by each partition through relevant API to allocate computing tasks reasonably.

Another key focus of this update is the enhanced communication capabilities. CUDA Tile now supports direct access to the video memory of other GPU instances within the computing kernel, and this functionality is achieved through NVLink high-speed interconnects. With the help of multicast pointers and dedicated multi-GPU memory operations, developers can also perform cross-GPU data reduction and writing tasks in NVSwitch networks that support NVLink SHARP.

This means that developers can integrate data communication and computational operations between GPU within the same kernel, reducing the additional overhead associated with kernel boundaries and data transfer that occurs during traditional multi-card collaboration. NVIDIA lists applicable scenarios including fused matrix multiplication and cross-GPU data reduction.

On the platform side, NVIDIA has explicitly adjusted the range of driver model support for the Rubin architecture. Starting from the Rubin architecture, CUDA no longer supports the Tesla Compute Cluster ( TCC ) mode, and only supports Windows Display Driver Model ( WDDM ) and Microsoft Compute Driver Model ( MCDM ). Among these, MCDM is a driver model designed for pure computing scenarios, and related applications and deployment tools need to be adapted to the new range of support.

In terms of development tools, CUDA 13.5 has for the first time introduced NVIDIA Nsight Perf SDK for AI in the form of a public preview, providing continuous performance analysis capabilities for long-running AI training and inference tasks. This tool is capable of continuously collecting GPU CPU performance metrics and execution context, helping developers to identify intermittent performance issues that are difficult to reproduce, and it also supports viewing the collected results through Perfetto.

In addition, CUDA 13.5 has also updated components such as CUDA Python, CUDA Graph, and CCCL 3.5. Among them, CUDA Python 1.2 has added features such as batch memory copying, programmable dependency initiation ( PDL ), and more flexible CUDA Graph update capabilities; CCCL 3.5 has incorporated public tuning interfaces for CUB, batch Top-K selection, and repeatable floating-point scanning functions.

In terms of drivers, NVIDIA simultaneously released the R620 series of data center GPU drivers, where the Linux version is 620.70.06 and the Windows version is 621.14, both with a release date of November 4th. It should be noted that in the IT community, it is mentioned that CUDA Toolkit 13.5 and the data center drivers are now released separately, and the drivers are no longer provided as built-in components of Toolkit.

Related Reading:

• NVIDIA Responds to Changes in Vera Rubin Platform Specifications: The New Version Represents Mass Production Specifications, and Third-Party Benchmarks Are Conducted Based on the Mass Production Version
• NVIDIA Releases Game Ready 621.14 Driver: Optimizes World of Warcraft: Unlimited, Fixes Video Black Screen Issues