CUDA and GPU programming
- 4 months ago @jsuarez
PufferLib 4.0 reports multi-GPU scaling results
The post reports 6.6× scaling on 8 GPUs in a worst-case test with a 32× parameter policy, and peak throughput above 100M steps per second on 8 RTX 5090 GPUs.
The reported figures offer a multi-GPU performance point of reference for GPU-based workloads.
- CUDA and GPU programmingArticle
Is Parallel Programming Hard, and What Can You Do About It?
A resource about the challenges of parallel programming and ways to address them.
Parallel programming concepts can help engineers reason about concurrency in GPU kernels.
Fault-Resilient GPU Multi-Process Service
Researchers describe a GPU multi-process service combining driver-level isolation that confines reachable MMU faults to the faulting client with VMM-based active-standby recovery. They report zero runtime overhead for the isolation mechanism.
The approach addresses fault containment and recovery in GPU multi-process workloads.
- CUDA and GPU programmingArticle
Tile Scheduling in CuTeDSL
A blog post introducing how StaticPersistentTileScheduler works in CuTeDSL and why it is useful when writing persistent kernels.
Understanding tile scheduling can help engineers reason about how persistent kernels are organized.
- 5 months ago @vivekgalatage
Georgia Tech series on high-performance computer architecture
The post recommends a six-part Georgia Tech series for learning high-performance computer architecture.
It may help engineers build background in the architecture behind fast GPU computing.
- CUDA and GPU programmingArticle
CuBridge Uses LLMs to Reconstruct Attention Kernels
CuBridge is an LLM-based framework for understanding and reconstructing high-performance attention kernels. The linked preview notes that supporting diverse, evolving attention variants is challenging.
It addresses the challenge of supporting attention variants with efficient CUDA implementations.
Video Explores GPU Architecture
A video titled “How do Graphics Cards Work? Exploring GPU Architecture” offers an overview of graphics card internals.
GPU architecture provides useful background for engineers working with GPU software.
- CUDA and GPU programmingArticle
A CUDA/PTX matmul kernel beats cuBLAS on B200
Paul Chan describes a pure CUDA/PTX matmul kernel on B200 that he says is 6% faster than cuBLAS at M=N=K=8192. The post links to a technical blog and a GitHub repository.
The blog and code offer engineers a concrete B200 matmul optimization to examine.
- CUDA and GPU programmingProject
pyptx provides a Python DSL for NVIDIA PTX kernels
pyptx is a Python DSL for writing NVIDIA PTX kernels, with Hopper and Blackwell support and JAX and PyTorch integration. It includes GEMM, grouped GEMM, RMSNorm, and SwiGLU examples.
It offers a Python interface for writing PTX kernels and integrating them with JAX or PyTorch.
GP-SIMD Processing-in-Memory
The post links to an ACM paper identified as “GP-SIMD Processing-in-Memory.”
It may interest engineers exploring GPU-style SIMD computing and processing-in-memory.
- 7 months ago @levidiamode
Hugging Face’s Ultra-Scale Playbook on LLM Training
The playbook covers data, expert, tensor, pipeline, and context parallelism for training LLMs on GPU clusters. It includes empirical examples from 4,000 scaling experiments across up to 512 GPUs.
It connects parallelism choices to memory use, pipeline overhead, and cluster topology.
- 7 months ago @SebAaltonen
Bindless access with 64-bit pointers
The post says 64-bit pointers avoid fetching a buffer descriptor from a descriptor heap, describing this approach in CUDA and Metal and via Vulkan BDA in GLSL. It says DX12 has no pointer support.
Useful context for comparing buffer access models across GPU APIs.
- CUDA and GPU programmingArticle
A Primer on GPU Architecture and Computing
An introductory article covering GPU architecture and computing.
Useful background for engineers building or optimizing GPU software.
Overlapping Data Transfers in CUDA C/C++
An NVIDIA Technical Blog post by Mark Harris discusses how to overlap data transfers with computation in CUDA C/C++.
Overlapping transfers and computation can help engineers make better use of host and GPU time.
- CUDA and GPU programmingArticle
Comparing CPU and GPU Performance for ONNX Inference
The article compares ONNX Runtime inference on an NVIDIA L4 GPU and an AMD EPYC 9965 CPU. The author says OCaml and libnuma memory placement made CPU inference faster than using scarce GPUs for TESSERA.
Useful for engineers evaluating inference performance and memory placement across CPUs and GPUs.
- CUDA and GPU programmingArticle
FP4 Training for Large-Scale MoE Models on Hopper
The linked page discusses practical FP4 training for large-scale Mixture-of-Experts models on Hopper GPUs. It identifies activation memory and expert-parallel communication as training bottlenecks.
Relevant to engineers working on GPU memory use and communication in large-scale MoE training.
- CUDA and GPU programmingArticle
CUDA Agent uses agentic RL for CUDA kernel generation
The linked item presents CUDA Agent, a system for large-scale agentic reinforcement learning aimed at generating high-performance CUDA kernels. It notes that GPU kernel optimization requires specialized hardware expertise.
It may interest engineers exploring reinforcement learning approaches to CUDA kernel optimization.
- CUDA and GPU programmingArticle
Khronos Tutorial on Vulkan Subgroups
The tutorial explains Vulkan 1.1 subgroup functionality for sharing and manipulating data among parallel GPU tasks.
Subgroups provide a GPU parallelism mechanism engineers can compare with shared-memory approaches.
Using CUDA Warp-Level Primitives
An NVIDIA Technical Blog post about CUDA warp-level primitives. It describes how GPUs execute groups of threads called warps in SIMT fashion and how CUDA programs can take advantage of warp execution.
Understanding warp execution can help engineers write high-performance CUDA programs.
- CUDA and GPU programmingArticle
Vulkan Memory Barriers and Image Layouts Explained
A RasterGrid blog post explaining Vulkan memory barriers and image layouts.
Useful for engineers working with GPU memory synchronization in Vulkan.
hgpu.org
