国际新闻 · 中文译文与原文对照

星期五 最新 1,000 篇 · 滚动浏览

科技

Meta TLX,一种支持高级硬件功能的原生GPU编译器扩展

来源:InfoQ 技术发布 上传时间 (多伦多时间)1 家来源 · 1 篇报道

中文译文

Meta开发了Triton低级扩展(TLX),旨在填补高级GPU编程抽象与充分利用现代GPU架构所需的低级执行控制之间的差距。

据Meta的工程师们称,现代GPU架构具备先进的性能优化功能,包括线程组专用化、异步内存传输、张量内存以及集群级执行等功能,这些都与Triton的传统编程模型有所不同——后者虽然简化了内核开发,但会抽象掉低级执行结构。

例如,一个现代的FlashAttention内核可能会让一个线程组负责执行TMA传输操作,而另一个线程组则负责执行张量核心计算。而Blackwell架构下的GEMM内核则可能需要同时协调多个线程组、TMEM内存分配以及集群资源。在这两种情况下,性能不仅取决于所执行的计算任务,还取决于多个执行单元之间的协作方式。要使用这些高级功能,需要“大量针对特定架构的调度代码”,因为这类代码并非以Triton编译器抽象的形式直接提供。为解决这一难题,Meta开发了Triton低级扩展(TLX),旨在让工程师在保持Triton编程模型的同时,对GPU执行过程进行精确控制。

TLX引入了一种新的多指令多线程组(MIMW)执行模型,该模型结合了CUDA的线程级控制与Triton的块级抽象。它在五个关键领域对Triton进行了扩展:线程组调度、同步、内存调度、集群执行以及可选的布局功能。

综上所述,这些功能使得TLX能够填补Triton的高效性与CUDA的强大控制能力之间的空白。开发者既能保留Python语言、自动调优以及分块张量编程的优势,又能直接访问那些日益决定现代GPU性能的执行结构。Meta已在三种主要工作负载中部署TLX进行生产环境使用数月。在基于数千块NVIDIA Blackwell GPU的GEM模型训练中,TLX实现了20–25%的端到端模型FLOPs利用率(MFU),同时前向传播速度提升至原来的2倍。在推荐系统推理中的内核内广播优化(IKBO)方面,TLX在H100上使线性压缩内核的处理速度提升4倍,在共同设计的模型中还将计算密集型的网络延迟降低了三分之二。最后,在用于大语言模型预训练的分布式分组GEMM任务中,TLX提升了大规模GPU集群在长时间训练过程中的性能、可靠性与可维护性。

AI工程师Kritika Choudary在LinkedIn上对此公告评论道:

当我在M2芯片的Metal平台上从零开始编写Flash Attention时,最耗时的部分就是确定如何将数据块加载到线程组内存中以及应在何处设置屏障,很高兴看到TLX将这一点明确地体现出来。

Amirul Islam也表达了类似观点:将 warp-group 的交接明确化,同时让布局设置保持可选,似乎是一个合理的界限。生产者/消费者流程在源代码中清晰可见,无需强制每个融合阶段都从手动布局管理开始。

Meta 计划通过添加对更多 NVIDIA、AMD 以及自定义加速器架构的支持,利用更高级的抽象机制实现 warp-group 的自动化编排,还将把 TLX 与 torch.compile 相集成,从而进一步扩展 Triton TLX 的功能。

原文

Meta developed Triton Low-Level Extensions (TLX), designed to bridge the gap between high-level GPU programming abstractions and the low-level execution control required to fully exploit modern GPU architectures.

According to Meta engineers, modern GPU architectures offer advanced performance-optimization features, including warp specialization, asynchronous memory movement, tensor memory, and cluster-level execution, which depart from Triton's traditional programming model, which simplifies kernel development but abstracts away low-level execution structures.

For example, a modern FlashAttention kernel may dedicate one warp group to issuing TMA transfers while another executes tensor-core computation. A Blackwell GEMM kernel may coordinate multiple warp groups, TMEM allocations, and cluster resources simultaneously. In both cases, performance depends not only on the computation being performed, but also on how multiple execution agents cooperate.

Using these advanced features required "large amounts of architecture-specific orchestration code", as they were not readily available as Triton compilers abstractions. To address this challenge, Meta developed Triton Low-Level Extensions (TLX), designed to give engineers explicit control over GPU execution while preserving Triton's programming model.

TLX introduces a new Multiple Instructions, Multiple Warps (MIMW) execution model that bridges CUDA’s thread-level control and Triton’s block-level abstraction. It extends Triton across five key areas: warp-group orchestration, synchronization, memory orchestration, cluster execution, and optional layout.

Taken together, these capabilities allow TLX to occupy the space between Triton’s productivity and CUDA’s control. Developers retain the advantages of Python, autotuning, and blocked tensor programming while gaining direct access to the execution structures that increasingly determine performance on modern GPUs.

Meta has deployed TLX in production for several months across three major workloads. For GEM model training on thousands of NVIDIA Blackwell GPUs, TLX achieved 20–25% end-to-end Model FLOPs Utilization (MFU) with up to 2x forward-pass speedup. For In-Kernel Broadcast Optimization (IKBO) in recommendation inference, TLX delivered a 4x speedup on an H100 for a linear compression kernel and reduced compute-intensive net latency up to two-thirds in co-designed models. Finally, in distributed grouped GEMM for LLM pretraining, TLX improved performance, reliability, and maintainability across large GPU fleets for extended training periods.

Commenting on the announcement on LinkedIn, AI engineer Kritika Choudary observed:

When I wrote Flash Attention from scratch in Metal on my M2, the tile loads into threadgroup memory and where to put the barriers were where most of the thinking went, so it's nice to see TLX make that explicit.

On a similar note, Amirul Islam emphasized that:

Making warp-group handoffs explicit while leaving layouts optional feels like a useful boundary. The producer/consumer pipeline is visible in the source, without forcing every fused stage to start with manual layout bookkeeping.

Meta plans to further expand Triton TLX by adding support for additional NVIDIA, AMD, and custom accelerator architectures, automating warp-group orchestration through higher-level abstractions, and integrating TLX with torch.compile.