The Chinese AI Software Stack

In the spring of 2024, a small group of engineers at the Shanghai AI Lab and at DeepSeek-AI released, in open-source form, a distributed-training library that they called DualPipe. The library was, on the surface, a routine piece of engineering: it was a scheduler for overlapping two pipeline-parallel paths of a transformer model in such a way that the forward and backward phases of training could be interleaved across two different device topologies. The library had been used, in production, to train DeepSeek-V3, and the paper that accompanied it made the case that the technique had been the principal reason that DeepSeek-V3 could be trained at roughly one tenth of the dollar cost that comparable American models had required. DualPipe was not, in absolute terms, a large piece of software. It was, however, a representative example of something larger that had been taking shape across the Chinese AI industry for the better part of a decade: a deliberate, well-funded, and increasingly confident effort to build out the full software layer on which a modern AI model is trained, fine-tuned, served, monitored, and integrated — the complete stack from the silicon at the bottom up to the chat-completion interface at the top.

The Chinese AI software stack is one of the most consequential pieces of engineering in the global AI industry, and it is also, in its current state in 2026, one of the most uneven. It is a stack that, at the model layer, has produced at least four frontier-class general-purpose LLMs — DeepSeek, Qwen, GLM, and Hunyuan — that compete credibly with the best American models on a wide range of benchmarks. It is a stack that, at the framework layer, includes several open-source ML frameworks that are competently engineered, that have substantial domestic adoption, and that play a meaningful role in the Chinese AI ecosystem. It is a stack that, at the serving and inference layer, has produced several widely deployed systems — vLLM being the most prominent, with Chinese contributors accounting for a substantial fraction of the upstream commits. It is, however, a stack that, at the compiler and runtime layer, is still substantially behind the American baseline, principally because the Chinese AI accelerators that the stack has to target are themselves several generations behind the American accelerators, and the engineering effort to extract competitive performance from them is large and ongoing.

This essay is an attempt to describe the Chinese AI software stack as it stands in late 2026: where it came from, what its principal components are, who the principal consumers and contributors are, and how it relates to the American AI software stack that it has, in part, been built to substitute for. The essay focuses on five threads. The first is the intellectual and institutional origin of the Chinese stack in the years between 2010 and 2020, when Chinese AI research shifted from being a consumer of the American AI software ecosystem to being a contributor to it and to building parallel domestic alternatives. The second is the principal Chinese frameworks — PaddlePaddle, MindSpore, OneFlow, Jittor, MegEngine, and ColossalAI — what each of them does, who uses each of them, and what each of them does best. The third is the compiler and runtime layer that the Chinese stack has built to make Chinese AI accelerators competitive with American ones. The fourth is the distributed-training, serving, and integration layer, in which the Chinese stack has produced its most internationally visible recent work. The fifth is the strategic picture: how the Chinese AI software stack relates to the export-control regime, what its trajectory is likely to be on a five-year and a ten-year horizon, and whether, on a sufficiently long horizon, it can substitute for the Nvidia-CUDA-ecosystem baseline on which all of contemporary AI software has so far been built.

The argument the essay develops, in summary form, is the following. The Chinese AI software stack is real, technically credible, and substantively distinct from the American AI software stack in several important respects. It is not, in its current state, a complete substitute for the American stack in a global sense: the principal Chinese frameworks have substantial domestic adoption but limited international adoption, and the principal Chinese compilers are tightly coupled to the Chinese AI accelerators. The Chinese stack is, however, a fully capable domestic stack for the Chinese AI industry, and it is sufficiently mature, as of 2026, that the export-control regime’s effectiveness as a constraint on Chinese AI capability is now substantially lower than it was in 2022. The software layer of the Chinese AI capability has, in other words, become one of the principal sites at which the strategic question of Chinese AI self-sufficiency is being settled, and the answer, as of late 2026, is “substantially self-sufficient, with several important caveats.”

The origins of the Chinese AI software stack

The Chinese AI software stack did not begin with a national AI strategy. It began, like most national AI software efforts, with research software written in university and industrial labs between roughly 2010 and 2016 — the period during which deep learning shifted from a niche academic technique to a mainstream commercial technology. The principal Chinese AI research groups in that period — Baidu, Tencent, Alibaba, Microsoft Research Asia, the Chinese Academy of Sciences’ computer-vision and natural-language labs, and the major universities — were consumers of the American AI software ecosystem. They trained their models on Caffe, on Theano, on Torch, on TensorFlow, and on PyTorch in roughly the same way that American research groups did, and they published their code, when they published it at all, in the same forks and repositories that everyone else used. The Chinese contribution to the global AI software stack in that period was principally at the model layer: the publication of architectures and trained weights that were built on top of the American framework layer, but which themselves made original contributions to the field.

The earliest Chinese contribution to the model layer that had a substantial downstream impact on the global AI software stack was the publication of ResNet in 2015 by the Microsoft Research Asia team led by Kaiming He. ResNet, the deep residual network architecture, was the most influential single paper in practical computer vision between 2015 and 2020, and it triggered a wave of follow-on work that shaped the architectures of every subsequent vision system, including the vision transformers of the early 2020s. The Microsoft Research Asia team was a Chinese AI research group, and the paper was published in a Western venue, with Western co-authors, in the standard international style. The code was released as a fork of Caffe and PyTorch and was widely adopted. ResNet is a model-layer contribution, not a framework-layer contribution, but it is a useful reminder that the principal Chinese contribution to the global AI software stack in the 2010–2016 period was at the model layer and not at the framework layer.

The first deliberate Chinese effort to build a framework-layer contribution was PaddlePaddle, the parallel distributed deep learning framework developed at Baidu and first released in open-source form in 2016. PaddlePaddle was, on the published technical record, a competent piece of engineering: it supported the standard set of deep-learning operations, it had a Python front-end that was familiar to PyTorch and TensorFlow users, and it included a number of performance innovations around the use of fluid programming abstractions for heterogeneous compute. PaddlePaddle was not, however, in 2016 or 2017 or 2018, a serious alternative to TensorFlow or PyTorch at the global scale: it had a smaller community, fewer pretrained models, and a less mature ecosystem. What it had, instead, was substantial internal adoption at Baidu, which used PaddlePaddle for the company’s own speech, vision, and NLP production workloads, and a substantial external adoption at a small number of Chinese AI companies and government R&D; groups. PaddlePaddle was, in other words, an early signal that the Chinese AI industry was beginning to build its own framework layer, even if the principal global AI software stack at the time was still American.

The second Chinese framework to matter at the global scale was not, strictly speaking, Chinese in origin. It was the original Facebook PyTorch. The relevant fact for this essay is that a very large fraction of the global PyTorch community is Chinese-speaking, and that the Chinese-speaking PyTorch community was, between 2018 and 2022, responsible for a meaningful fraction of the upstream PyTorch commits, of the downstream paper reimplementations, and of the high-quality reading materials and tutorials on which the rest of the global PyTorch community relied. This is not the same thing as a Chinese AI software stack. It is, however, the principal reason that the Chinese AI software ecosystem was, in 2022 when the export-control regime was first imposed, already substantially integrated into the global AI software ecosystem, even if the principal globally deployed frameworks were still American.

The first Chinese framework to be designed from scratch with the modern transformer era in mind was MindSpore, the Huawei framework first released in 2019 and developed principally by the Huawei 2012 Labs AI Computing Group. MindSpore was, on the published technical record, a substantially more capable framework than PaddlePaddle had been on its first release. It supported the full set of transformer-era operations, it had a Python front-end that was highly compatible with PyTorch, and it was designed from the outset to target the Huawei Ascend AI accelerators as well as Nvidia GPUs. MindSpore was, in other words, a framework designed for an era in which the Chinese AI industry would need to support Chinese AI hardware. The framework was open-sourced in 2020, and it was, by 2022, the principal framework used by Huawei and by a substantial number of Chinese AI-lab customers. MindSpore was, however, less adopted internationally than PaddlePaddle had been, principally because the export-control regime beginning in 2022 made it less attractive for international users to depend on a Chinese-AI-accelerator-targeted framework. The strategic significance of MindSpore is therefore principally at the domestic level: it was the first Chinese framework whose architecture, rather than being an artefact of historical accident, reflected an explicit strategic intent to build the Chinese AI software stack as a parallel to the American one.

The trajectory of the Chinese AI software stack between 2019 and 2022 was, on the public record, shaped by three forces. The first was the maturation of the Chinese AI model layer, in which a wave of Chinese large-language-model projects — PanGu, Ernie 3.0, M6, GLM-130B, Baichuan, and the early DeepSeek models — brought the model layer of the Chinese stack into close parity with the American model layer. The second was the progressive restriction of Chinese AI-lab access to the most advanced American AI accelerators, beginning with the Entity List listing of certain Chinese AI companies in 2019 and continuing with the export-control expansion of October 2022. The third was the maturation of the Chinese AI accelerators themselves — Huawei Ascend 910, Cambricon MLU, and the early Moore Threads chips — to a level of capability at which they could plausibly be used for serious AI training, even if they were substantially less capable than contemporary American accelerators. The combination of these three forces was the trigger that, in 2022 and 2023, prompted the Chinese AI industry to invest seriously in the framework, compiler, and runtime layers that would make the Chinese AI software stack a substitute for the American stack on Chinese hardware.

The principal Chinese AI frameworks

The principal Chinese AI frameworks in 2026 are PaddlePaddle, MindSpore, OneFlow, Jittor, MegEngine, and ColossalAI. They are not all of the same type. PaddlePaddle and MindSpore are full-stack AI frameworks in the PyTorch or TensorFlow mould: they provide a Python front-end, an automatic-differentiation engine, a library of standard neural-network operations, and a runtime that can execute those operations on one of several underlying accelerators. OneFlow is a framework that was designed from the outset around a specific scheduling abstraction called SBP (split, broadcast, partial-accumulate), which makes it possible to express data-parallelism, model-parallelism, and pipeline-parallelism uniformly within a single API. Jittor is a research-oriented framework from Tsinghua University that uses just-in-time compilation as the principal abstraction for tensor operations. MegEngine is a framework from Megvii, the Chinese computer-vision company, that is principally used for production inference workloads. ColossalAI is, properly speaking, not a standalone framework but a distributed-training and model-parallelism library that sits on top of PyTorch.

PaddlePaddle, currently in its third major version under the name Paddle 3.0, is the most internationally visible of the Chinese AI frameworks. It is maintained by a substantial full-time engineering team at Baidu. It supports training and inference on Nvidia GPUs, on AMD GPUs, on Huawei Ascend accelerators, on Cambricon MLU accelerators, on Moore Threads GPUs, and on a substantial set of other Chinese AI accelerators. Its principal scientific contribution has been the use of a fluid-programming abstraction that allows a single Python-level expression to be compiled into heterogeneous sub-executions that target different parts of the compute fabric. The abstraction has, on the published benchmarks, given PaddlePaddle a measurable performance advantage on multi-accelerator workloads. Its principal limitation, on the international scale, is that its English-language documentation, its international community, and its set of internationally adopted pretrained models are, even after almost a decade of open-source development, smaller than those of PyTorch. PaddlePaddle is, however, the principal framework used for the Baidu internal AI workloads and is also widely used by the Chinese government’s digital-infrastructure R&D; programs and by a substantial number of Chinese academic AI labs.

MindSpore, currently in version 2.x, is the most tightly coupled of the Chinese frameworks to its principal target hardware. MindSpore was, from its first release, designed around a unified execution graph that supported the Huawei Ascend series as its principal target, with Nvidia GPUs as a secondary target. The design choice had, on the published record, several engineering virtues. It allowed MindSpore to expose Ascend-specific features, such as the on-chip cube units and the on-chip interconnect, at the framework level in a way that PyTorch did not. It also allowed MindSpore to integrate tightly with the Huawei CANN compiler and runtime stack. The design choice had, however, one corresponding drawback: it made MindSpore less portable to non-Huawei hardware than PaddlePaddle was, and it increased the strategic lock-in of MindSpore users to Huawei hardware. That lock-in is a feature from Huawei’s perspective and a constraint from the perspective of the broader Chinese AI industry. In 2026, MindSpore has substantial adoption inside Huawei and inside the Huawei cloud customer base, moderate adoption at the Chinese Academy of Sciences and at a number of Chinese universities, and limited adoption internationally.

OneFlow occupies a distinct niche in the framework landscape. It is, on the published technical record, a framework whose principal design idea was that the standard data-parallel and model-parallel abstractions of PyTorch were insufficient to express the parallelism of modern transformer training, and that a more uniform representation — the SBP abstraction — could express those parallelisms more cleanly and more efficiently. OneFlow’s design has been carried forward into a substantial set of training and inference workloads inside the OneFlow Inc. company and inside the framework’s adoption at several Chinese AI-cloud companies. The framework’s principal limitation has been that the architectural decision to use SBP from the ground up made it incompatible with the substantial PyTorch codebase that the global AI community had built. OneFlow’s international adoption has, accordingly, been limited, but its domestic adoption at companies that committed to it early — notably the OneFlow customer base in the Yuanfudao education company and at several Chinese AI-cloud startups — has been substantial and durable.

Jittor is a research-oriented framework that emerged from the Tsinghua University computer science department in the late 2010s. Jittor’s principal design idea was the use of a meta-tensor abstraction that was compiled into specialized tensor-operation kernels at execution time. The just-in-time compilation approach had, on the published research record, several interesting performance characteristics, particularly for dynamic-shaped tensors that other frameworks handled less efficiently. Jittor has, however, remained principally an academic tool. It has had a useful life as a research substrate for a substantial set of Tsinghua computer-vision and natural-language papers, and it has been adopted by a small but persistent set of academic labs in China and Southeast Asia. It has not, to date, become a production framework.

MegEngine is a framework that emerged from Megvii, the Chinese computer-vision and face-recognition company. MegEngine was, for several years, primarily an internal tool, used to power Megvii’s computer-vision products. It was released as open-source in 2020, and the published record suggests that it had a useful early adoption inside Chinese CV shops that needed a high-performance inference framework with good Nvidia GPU support and modest support for heterogeneous deployment targets. MegEngine’s international adoption has been limited. Its domestic adoption, particularly at Megvii itself and at a small set of Chinese AI-on-camera customers, has been durable.

ColossalAI is the most internationally visible of the Chinese distributed-training libraries. It was developed by a team originally at the HPC-AI Technology company, with substantial roots in the HPC-AI Lab at the National University of Singapore and in a set of earlier open-source distributed-training projects. ColossalAI’s principal capability is a set of techniques — tensor parallelism, pipeline parallelism, sequence parallelism, ZeRO-style memory offload, and the associated mixed-precision training support — that allows a large transformer to be trained across a flexible set of distributed training topologies. ColossalAI’s library has, on the published record, been used to train several of the Chinese frontier models, including earlier versions of the Yi and InternLM model families, and it has a substantial international user base among researchers who want flexible model-parallelism without committing to a particular framework. ColossalAI’s principal limitation has been that it sits on top of PyTorch and is therefore not a substitute framework in the way that PaddlePaddle or MindSpore is.

The Chinese framework stack, taken together, is therefore best understood as a federated ecosystem rather than as a single stack. Each of the principal frameworks has a distinctive design, a distinctive target community, and a distinctive set of target hardware. The frameworks are not, in general, interchangeable: a MindSpore-trained model is not trivially portable to OneFlow, a PaddlePaddle-trained model is not trivially portable to MegEngine. The Chinese AI model layer, by contrast, has converged substantially on the use of Hugging Face format model weights and the use of PyTorch-style forward passes, and this convergence has made model-layer portability straightforward even though framework-layer portability remains hard. The Chinese framework stack is therefore not, in 2026, a single stack in the way that the American stack of CUDA + PyTorch + Hugging Face is, but it is a real stack, with substantial domestic adoption, and with the substantial engineering investment that the Chinese AI industry has put behind it.

The compiler and runtime layer: CANN, Neuware, MUSAMath

The compiler and runtime layer is, in 2026, the layer at which the Chinese AI software stack is least mature and most strategically significant. The reason is structural. The American AI software stack, anchored on CUDA + PyTorch + Nvidia hardware, has had a fifteen-year head start on the compiler-and-runtime layer that makes the hardware usable, and the resulting codebase is enormous. The Chinese AI software stack is attempting to do, on a substantially shorter timescale, a comparable piece of engineering on top of a set of Chinese AI accelerators that are themselves several generations behind the American baseline. The compiler-and-runtime layer is, accordingly, where the engineering effort of the Chinese AI industry is most concentrated.

The principal Chinese AI software runtime is CANN, the Huawei Compute Architecture for Neural Networks. CANN has been developed since the late 2010s as the principal compiler and runtime stack for Huawei Ascend hardware. Its architectural map to CUDA is reasonably direct. The Ascend hardware is conceptually a set of compute engines organised around a large number of matrix-multiply units (called cube units in the Ascend documentation) plus a set of conventional scalar and vector compute engines; CANN provides a set of operators that mirror the CUDA operator hierarchy, plus a more substantial set of fused operators that combine multiple CUDA-level primitives. The on-chip tile system (the Ascend analogue of the Nvidia SM tiling system) is exposed through a TIKPYTHON interface that allows vendor engineers and customers to write custom high-performance kernels in a Python-level syntax. The Ascend C language provides a lower-level escape hatch for kernel engineers who need very fine control of the underlying silicon.

CANN’s principal engineering challenge, in the 2024–2026 period, has been to extract competitive inference and training performance from Ascend hardware that is, on most American benchmark measurements, several generations behind Nvidia’s H100 in raw floating-point throughput. The engineering trick that CANN has, on the published record, embraced is a substantially more aggressive use of operator fusion than the CUDA equivalent uses. Operator fusion is, in AI compilers, the practice of combining multiple operations (matrix-multiply, normalisation, activation, attention-weighted aggregate) into a single hardware-level kernel that runs without the intermediate spill to off-chip memory that would otherwise be required. CANN’s level of operator fusion, on the basis of the published benchmark comparisons, has reached the point at which inference of GPT-style models on Ascend 910B hardware is, for certain workload sizes, energy-competitive with the equivalent inference on Nvidia A100 hardware, even though the raw floating-point throughput of the Ascend 910B is substantially lower than the A100. The fusion has also, however, made the CANN stack substantially more complex than its CUDA equivalent, and CANN engineering is, in 2026, a discipline in which several thousand Chinese compiler engineers are working.

The second Chinese AI software runtime is the Cambricon Neuware stack, the compiler and runtime for the Cambricon MLU series of AI accelerators. Neuware has, on the published record, a more CPU-centred architectural flavour than CANN. The Cambricon MLU is conceptually a heterogeneous set of MLU cores attached to a substantial general-purpose CPU, and Neuware treats the CPU as a first-class participant in the computation rather than as a mere scheduler for the MLU cores. The architectural difference is, in engineering terms, similar to the difference between Nvidia’s traditional CUDA-centric GPU and AMD’s more CPU-integrated APUs. Neuware is, on most published benchmarks, less mature than CANN, and the Cambricon AI accelerator roadmap has, in 2025 and 2026, been slower than the Ascend roadmap. Neuware remains, however, the most fully developed Chinese alternative to CANN for users whose hardware is Cambricon rather than Huawei.

The third Chinese AI software runtime worth describing is the Moore Threads MUSAMath stack. Moore Threads has, since 2023, positioned itself as a Chinese GPU manufacturer whose pitch is closer to the Nvidia pitch than to the Cambricon pitch: it is a heterogeneous general-purpose compute platform with substantial graphics capability, rather than a pure AI accelerator. MUSAMath is the AI-software-runtime layer of the Moore Threads platform. It has, on the published record, a substantially smaller engineering footprint than CANN or Neuware, and its technical depth at the matrix-multiply level is correspondingly shallower. It does, however, have the substantial advantage of running on hardware that is architecturally closer to Nvidia than Cambricon is, which has allowed it to integrate more easily with the existing CUDA-emulation and CUDA-fork tooling that the Moore Threads team has built. The Moore Threads push to make MUSAMath a drop-in substitute for CUDA is, in 2026, an active engineering programme and is one of the principal factors that has shaped the second-tier Chinese AI accelerator stack.

The fourth Chinese AI software runtime that deserves mention is the Biren SBRun stack. Biren, the Shanghai-based AI accelerator company, has produced hardware for which the SBRun runtime is the principal software stack. SBRun’s architectural ambition has, on the published record, been to compete with CANN directly, and to do so on a broader set of host CPU architectures. The 2024 Biren hardware was, however, less capable than the comparable Ascend 910B on most published benchmarks, and the 2025 Biren hardware roadmap has been slowed by the export-control regime’s effect on the company’s access to advanced packaging. SBRun has therefore been, in 2026, a smaller presence in the Chinese compiler ecosystem than CANN has been.

The fifth Chinese AI software runtime worth mentioning is the MetaX MACA stack. MetaX, founded in 2020 in Shenzhen, has produced several generations of GPU hardware for the Chinese domestic market. The MACA stack is, on the published record, the smallest and least mature of the principal Chinese AI runtimes, but it has the architectural virtue of having a clear PyTorch-front-end compatibility plan that should make MACA-trained models portable to the MetaX hardware.

The aggregate picture of the Chinese compiler-and-runtime layer in 2026 is that CANN is the only one of the five that has reached the level of capability that the Chinese AI industry depends on. CANN’s adoption is, on the public record, a substantial fraction of the deployed Chinese AI accelerator stack, and the Chinese AI industry’s ability to train large frontier models on Huawei hardware is, in 2026, substantially more robust than it was in 2022. The other four runtimes are real but immature, and their engineering progress in the next several years will be a major determinant of whether the Chinese AI software stack is, by the late 2020s, a fully capable multi-accelerator stack or whether it remains substantially dependent on Huawei Ascend.

The distributed training, serving, and integration layer

The distributed-training layer of the Chinese AI software stack is, in 2026, the layer at which the Chinese software effort has been most visible internationally and where the engineering has been most competitive. The principal Chinese distributed-training library is ColossalAI, and the principal Chinese distributed-training papers of the period have appeared in the principal international ML conferences (NeurIPS, ICML, ICLR, MLSys) and the principal American conferences on parallel computing. The Chinese distributed-training effort is also the layer at which the Chinese AI software stack has the most substantial overlap with the American AI software stack, because ColossalAI is itself a library on top of PyTorch and integrates with the same ZeRO, FSDP, and Megatron-LM ecosystem that American training pipelines do.

The principal Chinese distributed-training innovation of the 2024–2026 period has been the DualPipe scheduler, the scheduler that was mentioned at the beginning of this essay and that was the subject of the DeepSeek-V3 technical report in late 2024. DualPipe is a scheduling technique that overlaps the forward and backward phases of a transformer training step across two pipeline-parallel paths of the model in such a way that the two phases are issued in parallel rather than serially. The paper’s published performance numbers show that DualPipe, combined with the dense Mixture-of-Experts routing strategy that DeepSeek-V3 uses, allows the model to be trained across a heterogeneous network topology at substantially higher utilisation than would otherwise be possible. The technique has, on the public record, been replicated by Qwen, by GLM, and by the Hunyuan team at Tencent. It is, in other words, an example of a piece of Chinese distributed-training innovation that has had a direct effect on the global frontier-model training practice.

The broader picture of the Chinese distributed-training layer in 2026 is that the Chinese AI industry has, since 2022, built substantial domestic expertise in all of the major distributed-training techniques — tensor parallelism, pipeline parallelism, sequence parallelism, expert parallelism, ZeRO-style memory offload, and the associated mixed-precision training strategies. The expertise is housed, in 2026, in a mixture of the major Chinese AI companies (Huawei, Alibaba, DeepSeek, Baidu), the CAS labs, the Chinese AI-cloud companies (Volcano Engine, ModelScope), and a substantial set of Chinese frontier-model startups. The collective capability has made the Chinese distributed-training layer, in some respects, the most mature of the Chinese AI software stack layers, and the international visibility of the Chinese distributed-training contribution is substantially higher than the international visibility of the framework or the runtime contribution.

The serving and inference layer of the Chinese AI software stack is also well-developed. The principal international serving-and-inference pieces of the stack — vLLM, TensorRT-LLM, the Hugging Face TGI — are themselves open-source projects with substantial Chinese contributions, and the principal Chinese serving engines — the vLLM forks maintained at Chinese AI-cloud providers, the MindIE server in the Huawei ecosystem, the Neuware-inference in the Cambricon ecosystem — have the capability, on most published benchmarks, to serve Chinese frontier models at competitive throughput and competitive latency. The most visible Chinese serving-engine contribution to the open-source ecosystem has been vLLM’s Chinese ecosystem adaptations: the China-mainland-hosted vLLM mirror, the Chinese-specific serving-engine forks at Volcano Engine and at ModelScope, and the substantial set of Chinese vLLM community contributions that have, since 2023, accounted for a large fraction of the upstream commits. vLLM’s PagedAttention algorithm, the paper that introduced the technique, is, on the citation record, one of the most heavily cited Chinese-origin papers in the operating-systems literature of the past five years.

The model integration and inference-server layer of the Chinese AI stack has its own distinctive profile. The Chinese AI industry is, in 2026, a substantial consumer of generative AI integration tooling — chat front-ends, function-calling frameworks, agent frameworks, retrieval-augmented-generation frameworks — and the Chinese AI-cloud companies have built substantial capabilities in this layer. The ModelScope platform at Alibaba is, on the public record, the principal Chinese Hugging Face equivalent, hosting a substantial fraction of the Chinese frontier model weights. Volcano Engine, the ByteDance cloud, hosts a substantial set of Chinese frontier model deployments. The Tiangong platform at Kunlun Inc. and the KEG AI Lab at Tsinghua provide laboratory infrastructure for academic and small-industry deployments of Chinese frontier models. The Chinese model-integration stack is, in other words, mature and substantially integrated, and the principal Chinese contribution to this layer has been the substantial set of vector-database and retrieval-system pieces — Milvus from Zilliz, DashVector from Alibaba, the various open-source forks of Qdrant — that have been built and deployed at substantial scale.

The data and instruction-tuning layer of the Chinese AI software stack is also distinctively developed. The Chinese instruction-tuning dataset ecosystem — the Magpie and WildChat-style synthetic-data pipelines, the WuDao corpora, the MNBVC corpus, the InternLM training-corpus — is a substantial and arguably underappreciated part of the Chinese AI software contribution. The reason is that the Chinese instruction-tuning datasets are, by virtue of their size, their quality, and their bilingual coverage, among the most useful instruction-tuning resources in the global ecosystem. Several of the principal American AI labs use Chinese-origin instruction-tuning datasets in their training pipelines, and several of the principal open-source instruction-tuning dataset projects have substantial Chinese contributions.

The MLOps and production-deployment layer of the Chinese AI stack is the layer at which the domestic stack has reached the most complete equivalence with the American stack. The principal MLOps platforms used in the Chinese AI industry — the proprietary platforms at Baidu, Alibaba, and Tencent, the Tencent Angel platform, the various open-source MLOps projects hosted on Chinese code-forges (Gitee, GitCode) — are, on most published benchmarks, substantially comparable to the American MLOps platforms in functionality. They are, however, substantially less adopted internationally, which is a limitation that has more to do with language and with ecosystem lock-in than with technical capability.

The aggregate picture of the Chinese distributed-training, serving, and integration layer in 2026 is, accordingly, that the Chinese AI industry has, in this layer, produced a set of software capabilities that is technically competitive with the American equivalent at the global frontier, even if it has not matched the American stack in international adoption. The distributed-training innovation at DeepSeek and at Qwen has been, on the public record, the most internationally visible Chinese AI software contribution of the 2024–2026 period, and the vLLM community has been the principal site of substantial Chinese open-source contributions to the AI serving ecosystem.

The strategic implications of the Chinese AI software stack

The strategic implications of the Chinese AI software stack, in 2026, are arguably larger than the implications of any single piece of Chinese AI hardware or any single Chinese frontier model. The reason is that software is the layer of the stack at which the United States export-control regime has the weakest direct effect, and at which the Chinese AI industry can accumulate capability at a rate that is largely independent of the United States hardware-availability constraint. The Chinese AI software stack has, accordingly, become one of the principal sites at which the strategic question of Chinese AI self-sufficiency is being settled, and the trajectory of the stack over the next five years will substantially shape what is possible in the Chinese AI capability on both sides of the Pacific.

The first strategic question is the substitution question: can the Chinese AI software stack substitute for the American AI software stack, on Chinese hardware, at sufficient scale and at sufficient performance to sustain Chinese frontier-model development? The honest answer, in 2026, is that it can, in the sense that the Chinese AI industry is currently training and serving its frontier models on a substantially domestic software stack running on a substantially domestic hardware stack, and it is doing so at a level of performance and at a level of cost that is competitive with the American baseline at the same point in the technology cycle. It cannot, however, substitute for the American stack in the sense of being a drop-in replacement on American hardware, and it cannot yet match the American stack on the full breadth of frontier-model workloads. The Chinese AI industry is, accordingly, doing its frontier-model training on a hybrid stack in which the Chinese AI software runs on Chinese AI hardware, with periodic falls back to American hardware where the Chinese hardware is unable to support a specific workload. The hybrid stack is workable, and it is what the Chinese AI industry has settled on for the late 2020s.

The second strategic question is the export-control question. The United States export-control regime, in its October 2022 and October 2023 forms, was structured principally around the constraint on advanced-node CMOS and on advanced packaging. It did not, in its initial forms, target the AI software layer. That began to change in 2025 and 2026. Several of the policy discussions in the United States in the 2025–2026 period have considered whether the AI software layer ought to be brought within the scope of the export-control regime, on the logic that access to capable AI software is at least as important as access to capable AI hardware, and on the logic that allowing access to capable AI software to remain unrestricted has substantially weakened the hardware-side export controls. The specific items under consideration have included commercial AI model weights above a certain capability threshold, certain classes of training and inference software with substantial algorithmic innovation, and certain classes of compiler and runtime tooling. No formal rulemaking had, at the time of writing, been announced, but the policy discussion within the United States government has, on the public record, become substantially more serious about the software layer in the past two years. The Chinese AI software community has, in response, increased its investment in self-sufficient toolchains and increased its use of open-source algorithmic implementations whose distribution is harder to constrain.

The third strategic question is the open-source question. The Chinese AI industry is, in 2026, a substantial contributor to the global open-source AI software ecosystem. The Chinese frontier models — DeepSeek-V3, Qwen 3, GLM 4.5, Hunyuan, Baichuan, Yi, InternLM — are, with the exception of the most recent GPT-generation equivalents, all released as open-weights with permissive licenses. The Chinese AI frameworks — PaddlePaddle, MindSpore, OneFlow, Jittor, ColossalAI — are, with the partial exception of MindSpore, all released as open-source. The Chinese distributed-training libraries are, with the partial exception of internally maintained systems, all open-source. The Chinese serving engines, vector databases, and MLOps platforms are, similarly, substantially open-source. The Chinese AI software ecosystem is, accordingly, among the most open-source of the world’s AI software ecosystems in 2026. The strategic significance of this openness is twofold: it gives the Chinese AI software stack substantial international adoption, and it also gives the export-control regime a hard target by making it difficult to draw a line around the AI software that is constrained and the AI software that is freely available.

The fourth strategic question is the talent and research-base question. The Chinese AI software capability is, in 2026, supported by a substantial research base in the Chinese universities — Tsinghua, Peking, Shanghai Jiao Tong, the University of Science and Technology of China, Huazhong, Zhejiang, and the CAS institutes — and by a substantial engineering base in the Chinese AI companies — Baidu, Alibaba, Tencent, ByteDance, DeepSeek, Zhipu, Moonshot, the Hunyuan team, and a substantial set of smaller companies. The Chinese AI software research community is, on the basis of the publication record, among the most productive in the world, and several of the principal innovations of the 2024–2026 period — DualPipe, the dense Mixture-of-Experts architecture, the vLLM PagedAttention, the ZeRO-offload system, several of the principal MoE routing techniques — have had substantial Chinese contributions. The talent base has, however, been constrained by the same export-control regime and by the same immigration-policy environment that has constrained the broader Chinese AI talent flow. The Chinese AI industry has, in response, deepened its domestic AI-talent pipeline, recruited substantial numbers of Chinese AI researchers from the United States and from Europe, and built out a substantial domestic AI-research presence at the Chinese AI companies. The talent-base assessment, in 2026, is that the Chinese AI software research community is large, productive, and broadly self-sustaining.

The fifth strategic question is the long-horizon trajectory. The Chinese AI software stack is, in 2026, at an inflection point. The framework layer has reached domestic maturity but limited international reach. The compiler-and-runtime layer has reached one mature entry (CANN) and several less mature entries (Neuware, MUSAMath, SBRun, MACA), and the next several years will determine whether the less mature entries catch up to CANN or whether CANN consolidates into a strategic monopoly on Chinese AI compute infrastructure. The distributed-training, serving, and integration layer has reached international competitiveness and has already produced internationally visible contributions. The strategic picture, in 2026, is that the Chinese AI software stack is on track to be, by the late 2020s, a fully self-sufficient domestic stack with substantial international contributions but limited international adoption. The substitution question, in this picture, is not whether the Chinese AI software stack can substitute for the American AI software stack on Chinese hardware — it can — but whether the Chinese AI software stack can attract international adoption in a way that makes it competitive, in international terms, with the American AI software stack. That is a different and harder question, and the answer, in 2026, is not yet settled.

The long-term interpretation of the Chinese AI software stack is, like the long-term interpretation of the Chinese photonic chip effort discussed in the earlier essay in this series, going to depend substantially on the trajectory of the United States export-control regime, on the pace of progress of the Chinese AI accelerators and Chinese AI frameworks, and on the willingness of the Chinese AI industry to commit to the long-horizon engineering work that the substitution task requires. The engineering work is substantial: the Chinese AI software stack needs, to fully replace the American AI software stack on a global scale, years of additional compiler and runtime engineering, additional distributed-training innovation, additional open-source community development, and additional international adoption work. That engineering work is feasible, and a substantial portion of it is currently underway. Whether it will be completed on a five-year or a ten-year timeline is, in 2026, genuinely uncertain. What is not in doubt, in 2026, is that the Chinese AI software stack has reached the point of technical credibility and strategic visibility at which it must be taken seriously by anyone who is thinking about the long-term balance of the AI capability in the second half of the 2020s and into the 2030s.

References

1. DeepSeek-V3 Technical Report — DeepSeek-AI, 2024.
2. Qwen 3 Technical Report — Alibaba Qwen Team, 2025.
3. GLM-130B: An Open Bilingual Pre-trained Model — Zeng et al., ICLR 2023.
4. PaddlePaddle Official Site — Baidu, 2024.
5. MindSpore Official Site — Huawei, 2024.
6. OneFlow GitHub Repository — OneFlow Inc., 2024.
7. Jittor: A Just-in-Time Compilation-Based Deep Learning Framework — Hu et al., MLSys 2020.
8. MegEngine Open Source Repository — Megvii, 2024.
9. Colossal-AI: A Unified Deep Learning System For Large-Scale Model Training — Bian et al., 2021.
10. Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) — Kwon et al., SOSP 2023.
11. CANN (Compute Architecture for Neural Networks) — Huawei, 2024.
12. Milvus Vector Database — Zilliz, 2024.
13. WuDao MM: A Large-Scale Multi-Modal Dataset — Xu et al., BAAI, 2021.
14. TensorRT-LLM — Nvidia, 2024.
15. PyTorch FSDP: PyTorch Fully Sharded Data Parallel — PyTorch Team, 2022.
16. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models — Rajbhandari et al., SC 2020.
17. CUDA C++ Programming Guide — Nvidia, 2024.
18. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism — Shoeybi et al., 2020.
19. Magpie: Alignment Data from In-the-Wild Generative Models — Xu et al., 2024.
20. Tsinghua University Natural Language Processing Lab — THUNLP, 2024.
21. Chinese AI Software Stack: A Survey — Industry survey, 2025.