openPangu-2.0-Pro: Huawei's 505B Ascend-Native LLM

TL;DR — Huawei open-sourced openPangu-2.0-Pro on August 4, 2026: a 505-billion-parameter MoE model trained end-to-end on Ascend 910B NPUs — no Nvidia silicon involved. It’s the first open-weight model above 500B to achieve that milestone. Weights, inference code, and a technical report are available on GitCode. Performance claims look strong on paper but remain unverified by independent benchmarks. Here’s what we know, what’s promising, and what you should wait to confirm.

The First Time I Cared About Training Hardware

I’ll be honest: when I evaluate an open-weight model, the hardware it was trained on usually doesn’t matter. I care about quality, speed, cost, and license. Whether the training ran on A100s, H100s, or custom ASICs — that’s the training team’s problem, not mine.

openPangu-2.0-Pro changed that instinct. When a 505B model ships with the explicit claim that no Nvidia GPUs were used anywhere in pretraining, and backs it up with Ascend-specific operator code and a technical report describing the full Ascend 910B cluster setup, the hardware becomes the story. It’s a proof point for an entire parallel compute ecosystem.

Whether that proof point matters to you depends on whether you’re just running inference on existing hardware (in which case — keep reading for the architecture details and community conversion efforts), or whether you care about where the AI supply chain is heading.

Model Specifications

Attribute	Value
Model name	openPangu-2.0-Pro
Total parameters	505 billion
Architecture	Mixture-of-Experts (MoE)
Active parameters per token	~18 billion
Context window	512,000 tokens
Training tokens	~34 trillion
Training hardware	Ascend 910B NPUs (exclusively)
Optimizer	Muon
Attention mechanisms	Multi-head Latent Attention (MLA), Decoupled Sparse Attention, Sliding Window Attention
Post-training	Three-stage pipeline with Online Policy Distillation (OPD)
License	Open weights (check repository for specific terms)
Availability	GitCode Ascend Tribe community, Huawei Cloud ModelArts Studio
Artifacts released	Weights, inference code, technical report

Architecture Deep Dive

MoE with 18B Active Parameters

openPangu-2.0-Pro follows the now-standard pattern of using Mixture-of-Experts to scale total knowledge capacity while keeping per-token compute manageable. With 505B total parameters but only ~18B active on any given token, the compute cost per forward pass is comparable to a dense model in the 18–20B range — though you still need enough memory to hold (or shard) the full 505B parameter set.

This active-to-total ratio is aggressive. For comparison, DeepSeek V4 uses a similar MoE strategy but with different expert counts and routing. Llama 4 Maverick (400B total, ~17B active) sits in a comparable tier of “small active footprint, huge capacity.” openPangu-2.0-Pro pushes the total parameter count further than either while keeping the active slice lean.

Attention Stack: Three Mechanisms Working Together

The attention design is where the technical report gets interesting. Huawei stacks three complementary mechanisms:

Multi-head Latent Attention (MLA) — Compresses the KV cache into a lower-dimensional latent space. This is critical for a 512K-token context window: naive KV caching at 512K would be prohibitively large. MLA trades a small amount of attention fidelity for dramatically reduced memory footprint per token.
Decoupled Sparse Attention — Routes different attention heads to attend over different subsets of the context, avoiding the O(n²) blowup of full dense attention across the entire 512K window for every head.
Sliding Window Attention — A subset of layers uses local sliding windows for capturing fine-grained local patterns, complementing the sparse global attention in other layers.

The combination is architecturally similar to what we’ve seen in DeepSeek V4’s attention design — the industry is clearly converging on hybrid local/sparse/latent approaches for long-context MoE models.

The Muon Optimizer

The technical report confirms training used the Muon optimizer rather than AdamW. Muon (originally proposed by Kosson et al.) uses momentum in the update direction rather than element-wise scaling, which can yield better loss curves on large-scale runs. This is still a less common choice for models at this scale, and it’s notable that Huawei’s team found it worked well on the Ascend stack.

The Hardware Sovereignty Angle

Let’s address the elephant: the entire pretraining run used Ascend 910B NPUs. Zero Nvidia. Zero AMD. That’s the first time an open-weight model above 500B parameters has shipped with that claim.

What This Proves

It demonstrates that the Ascend 910B cluster can sustain the thousands of hours of continuous distributed training needed for a 505B model across 34 trillion tokens. This isn’t a toy experiment or a fine-tuning exercise — it’s a full pretraining run at frontier scale.

What It Doesn’t Prove (Yet)

Training efficiency vs. Nvidia: The technical report doesn’t provide direct MFU (Model FLOPS Utilization) comparisons against an equivalent H100/H200 run. We don’t know if training took longer, cost more, or required more hardware.
Quality parity: Without independent benchmarks, we can’t confirm whether the model matches the quality of similarly-sized models trained on Nvidia hardware.
Scalability beyond this point: 505B is large, but frontier labs are training at the multi-trillion parameter scale. Whether Ascend can scale further remains to be demonstrated.

The Supply Chain Context

It’s worth noting the hardware supply chain story beneath the silicon. Earlier Ascend chips reportedly used TSMC 7nm dies and Samsung HBM. Future iterations are expected to shift to SMIC fabrication and CXMT memory — completing the domestic supply chain. openPangu-2.0-Pro represents the current capability; the next generation will test whether fully domestic fabrication can maintain this level of performance.

Comparison: openPangu-2.0-Pro vs. DeepSeek V4 vs. Llama 4 Maverick

	openPangu-2.0-Pro	DeepSeek V4	Llama 4 Maverick
Total params	505B	685B	400B
Active params	~18B	~37B	~17B
Context window	512K	1M	1M
Training hardware	Ascend 910B	Nvidia (H800)	Nvidia (H100)
Training tokens	~34T	~30T	~22T
License	Open weights	MIT	Llama license
MoE	Yes	Yes	Yes
Inference availability	Ascend-native; community Nvidia ports in progress	Nvidia, AMD, CPU	Nvidia, broad ecosystem
Independent benchmarks	❌ Pending	✅ Widely tested	✅ Widely tested

The most honest takeaway: on paper, openPangu-2.0-Pro is competitive. In practice, until community members finish converting the weights and running standard evals on Nvidia hardware, we can’t confirm quality parity. The 512K context window is shorter than DeepSeek V4’s 1M and Llama 4’s 1M, but 512K is sufficient for the vast majority of agent workloads.

For a broader comparison of open-weight models for agent use cases, see our Open-Weight LLMs for Agents guide.

Post-Training: Three Stages + Online Policy Distillation

The post-training pipeline is structured in three stages, culminating in Online Policy Distillation (OPD). The flow:

Supervised Fine-Tuning (SFT) — Standard instruction tuning on curated datasets.
Reinforcement Learning from Human Feedback (RLHF) — Preference optimization to align with human judgments.
Online Policy Distillation (OPD) — A larger or more capable teacher model provides real-time training signal during the final alignment phase, allowing the student to improve beyond what static preference data can provide.

OPD is the interesting addition. Rather than distilling once and freezing, the model continues to learn from a live teacher during training. This can close gaps that static SFT/RLHF pipelines miss — particularly for complex reasoning and multi-step tasks where reward models have poor calibration.

Running openPangu-2.0-Pro: Current State

On Ascend Hardware (Official)

The model is available through:

GitCode Ascend Tribe community — weights and inference code
Huawei Cloud ModelArts Studio — managed inference endpoint

If you’re already in the Huawei Cloud ecosystem or have access to Ascend hardware, you can run it today.

On Nvidia Hardware (Community Efforts)

Community conversion efforts are underway to port the model weights and inference code to run on standard Nvidia GPUs via frameworks like vLLM or SGLang. This involves:

Converting Ascend-specific tensor formats to PyTorch-native
Re-implementing any custom operators in CUDA
Validating numerical equivalence across the conversion

As of this writing (August 7, 2026), no complete Nvidia-compatible checkpoint has been released by the community. Watch the GitCode repository for updates.

Memory Requirements

With 505B total parameters in FP16, you’re looking at roughly 1TB of raw weight storage. In practice with quantization (INT8 or INT4), this drops to ~500GB or ~250GB respectively. Even quantized, this requires multi-node or high-end multi-GPU setups (8×H100 80GB minimum for INT4, likely 16+ for comfortable inference with KV cache overhead at long contexts).

What This Means for Agent Builders

If you’re building agents on open-weight models, here’s the practical assessment:

Wait and watch. openPangu-2.0-Pro is architecturally interesting and the hardware story is genuinely significant. But until:

Community ports to Nvidia hardware are validated
Independent benchmarks confirm quality claims
Inference tooling matures beyond the Ascend ecosystem

…it’s not a practical choice for most agent deployments today. The active parameter count (18B) suggests it could be fast and cheap at inference time if the ecosystem catches up.

If you’re in China’s Ascend ecosystem, this is immediately relevant. It’s the largest open-weight model natively available on your hardware stack, and it comes with first-party optimization for Ascend inference.

If you care about supply chain diversification, this is a landmark to track. The model proves that Ascend-only training at scale produces a viable large language model. As the ecosystem matures, this could break the current Nvidia monoculture in AI training and inference.

Limitations and Unknowns

Let me be direct about what we don’t know:

Benchmark claims are self-reported. The technical report includes numbers on standard evals, but no third party has reproduced them. History tells us to wait for independent confirmation.
License details need careful reading. “Open weights” doesn’t always mean “MIT.” Check the repository for specific terms around commercial use, modification, and redistribution.
Long-context quality is unconfirmed. Having a 512K window doesn’t guarantee good retrieval quality throughout. We need needle-in-haystack and long-context benchmarks from independent testers.
Inference ecosystem is immature. Outside of Ascend hardware and Huawei Cloud, running this model today requires community tooling that doesn’t yet exist.

FAQ

Is openPangu-2.0-Pro truly open source?

Weights, inference code, and a technical report are released — making it “open-weight” by current standards. Whether the training code, data mix, or intermediate checkpoints are available is not confirmed. Check the GitCode repository for exact terms.

Can I run it on Nvidia GPUs today?

Not yet with official support. Community conversion efforts are in progress. Expect initial ports within weeks, but quality validation will take longer.

How does the 18B active parameter count affect inference speed?

In theory, inference speed should be comparable to a dense 18B model (plus routing overhead). The MoE architecture means you only compute with a fraction of the full 505B on each token. The practical speed depends on how efficiently the routing and expert selection are implemented in your inference framework.

Is it better than DeepSeek V4 for agent tasks?

We genuinely don’t know yet. DeepSeek V4 has been widely tested by the community and has proven strong for tool-use, long-context, and multi-step reasoning. openPangu-2.0-Pro’s claims suggest it’s competitive, but claims aren’t benchmarks. See our DeepSeek V4 deep dive for a proven alternative.

Why does the training hardware matter to me as a user?

If you’re just running inference, it matters less — once community ports are ready, you’ll run on whatever hardware you have. But it matters for the industry: if Ascend can support frontier-scale training, it creates a second viable compute ecosystem. That’s good for competition, pricing, and supply chain resilience regardless of which hardware you personally use.

What’s the context window compared to competitors?

512K tokens. That’s shorter than DeepSeek V4’s 1M and Llama 4’s 1M, but longer than most models from 2024–2025. For typical agent tasks, 512K is more than sufficient — most agent loops rarely exceed 100K tokens of actual context usage.

Last updated: August 7, 2026. We’ll update this article as independent benchmarks and community Nvidia ports become available.