SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment

Abstract

Mixture-of-Experts (MoE) is a scaling architecture for large language models that activates only a small subset of expert modules per token, enabling massive parameter growth with nearly constant computation. Recent Hybrid MoE architecture adds shared experts to capture consistently useful representations, further improving stability and generalization. MoE now powers many flagship open-source and commercial models, yet remains vulnerable to adversarial attacks. Specifically, sparse routing introduces a structural vulnerability: MoE safety depends on which experts are activated, and adversaries can subvert this selection through jailbreak prompts, malicious fine-tuning, and weight-level pruning of safety-critical neurons. Existing defenses primarily focus on hardening the router, but an adversary may still manipulate or bypass the routing trajectory due to the routing process's nondeterministic nature, thereby collapsing the defense. To cope with this problem, we first identify theoretically and empirically that shared expert, an always-activated component containing a small proportion of safety-critical neurons, can overcome the uncertainty of sparsely activated routing path and serve as a router-independent anchor to enhance global safety alignment. Based on this insight, we propose SEAL, a training-time parameter-efficient defense that produces a plug-and-play adapter attached to shared expert, and SEAL++, a variant that adds an orthogonal constraint preserving pre-existing safety subspaces during training. We evaluate SEAL and SEAL++ across six attack scenarios that combine three adversarial inputs (harmful prompting, jailbreak, malicious fine-tuning) with and without neuron pruning. SEAL reduces attack success rate (ASR) by up to 60%, at a capability cost of at most 1.4% on a five-benchmark average. Additionally, SEAL can seamlessly integrate with router-level defenses for stronger defense. Under this setting, we can further decrease the ASR to 4.8%, more than five times lower than router-level defense alone. More broadly, our work demonstrates that shared experts constitute a unique defense surface overlooked by prior work, offering a complementary safety alignment pathway.

Most frontier MoE models already carry this defense surface

As of September 2026, the DeepSeek-style always-on shared expert appears across the flagship open-weight MoE releases of eighteen vendors. Models evaluated in this paper are shown in bold.

The design has also persisted across model generations: Qwen removed the shared expert in Qwen3-MoE and reintroduced it in Qwen3-Next and Qwen3.5; MiniMax released the M2 line without one and reinstated it in M3.

As of September 2026, per each model's released configuration.

Align only the always-on component: SEAL and SEAL++

Here \(\mathbf{C}_{\text{SE}}\) is a frozen diagonal mask selecting the identified safety neurons (typically 1 to 3% of the intermediate dimension), so the constraint restricts little of the space available for task learning while protecting the directions that make safety neurons identifiable and robust to pruning.

Parameter cost across the four evaluated architectures

Up to 60% ASR reduction across six attack scenarios and four architectures

We evaluate six scenarios that cross three adversarial inputs (harmful prompting, jailbreak, malicious fine-tuning) with and without neuron pruning. The single largest drop is DeepSeek's routed-scope pruning ASR, from 74.2% to 14.0%, achieved without a single gradient update to any routed expert.

The findings hold under an independent judge: re-scoring the direct-harmful setting with Qwen3Guard, ASR drops from 21.8% (Original) to 9.6% (SEAL) and 11.2% (SEAL++).

The defense holds when the attacker adapts

The adapter adds safety without erasing what the model knew

The original safety neurons survive training, so the augmented representations complement rather than replace the existing ones: an attacker must prune a larger neuron population for the same effect. The result is also insensitive to the knobs it depends on. Varying the safety-neuron threshold \(\zeta\) across the tested range changes ASR by at most 3%; the constraint weight \(\lambda_{\text{orth}}\) is flat across three orders of magnitude; and repeated runs under greedy decoding with additional seeds keep the standard deviation within 1%.

BibTeX

@inproceedings{meng2026seal,
  title     = {{SEAL}: Reinforcing Global Safety in Mixture-of-Experts
               through Shared Expert ALignment},
  author    = {Meng, Qingyu and Zha, Yiwei and Pei, Jiahuan and
               Hindriks, Koen and Bos, Herbert and Chen, Min},
  booktitle = {Proceedings of the 2026 ACM SIGSAC Conference on
               Computer and Communications Security (CCS)},
  year      = {2026},
  publisher = {ACM}
}