A Chiplet Expert-Reuse Architecture for MoE Inference

Abstract

Mixture-of-experts models expand parameter capacity through sparse activation, but Top-k routing creates expert hotspots, irregular weight loads, inter-chiplet scatter/gather traffic and synchronization tails. CoMoE-Chip addresses the limited ability of hot-expert replication, static placement and single-step prefetching to jointly exploit cross-layer dependencies and interconnect state. It maintains a versioned directory across attention, expert, routing-control and HBM/PIM memory chiplets. An expert-transition graph predicts next-layer candidates and coordinates bounded replication, confidence-based prefetching, congestion-aware mapping, credit control and hierarchical reduction. A synthetic-trace analytical model uses 64 experts, Top-2 routing, eight expert chiplets, Zipf skew 1.1, cross-layer predictability 0.65 and effective inter-chiplet bandwidth of 512 GB/s. Compared with static mapping, mean layer latency falls from 4.742 to 1.621 ms, P95 from 5.186 to 1.711 ms, and remote expert loading from 12.954 to 1.245 GB per batch: reductions of 65.82%, 67.01% and 90.39%. Load balance rises from 0.516 to 0.804 with a replication factor of 1.26. The study suggests optimizing expert transitions, physical residency, network pressure and tail latency as one feedback loop.

A mixture-of-experts (MoE) model uses a gating network to select a small number of feed-forward experts per token, increasing total parameters while keeping per-token computation approximately stable^{[2][3][4][5][6]}. DeepSeekMoE, DeepSeek-V3, DeepSeek-R1 and Qwen3 extend this approach with fine-grained experts, shared experts and large-scale expert parallelism^{[7][8][9][10]}. Sparse computation does not imply low system cost: expert weights still require storage and on-demand reads, tokens move among routers, expert devices and reduction nodes, and each layer waits for its slowest expert. Tutel, MegaBlocks, DeepSpeed-MoE and Lina reduce GPU-cluster overhead through sparse kernels, communication reordering and pipelining^{[11][12][13][14]}, while MoETuner and CRAFT optimize multi-GPU serving through cross-layer dependencies, placement and replication benefits^[15][16]. As UCIe 3.0 advances chiplet links to 48/64 GT/s^[28], multi-chiplet accelerators create opportunities for fine-grained expert migration and on-chip reuse. However, Expert Streaming, HCRMap, MoEntwine, Mozart, ThAME and Sieve show that inter-chiplet bandwidth, memory hierarchy, expert hotspots and execution queues can become joint bottlenecks^{[17][18][19][20][21][22]}. CoMoE-Chip does not claim to originate cross-layer dependency analysis or expert replication. It combines transition confidence, bounded replication, weight prefetching, token-to-replica mapping and NoP credit control in one hardware–runtime loop. Its contributions are an interpretable expert-transition graph with uncertainty measures; hierarchical residency, bounded replication and cancellable prefetching; constrained mapping that accounts for routing scores, compute queues, inter-chiplet paths and weight loads; and synthetic-trace analysis of routing skew, predictability and die-to-die bandwidth.

1. Chiplet bottlenecks in MoE inference

1.1 Sparse activation and dense system state

Let the input representation of a token at layer l be ht^l, with router scores over E experts denoted by gt^l and Top-k set S_t^l. The MoE layer output is:

Equation (1) computes only k experts, but the device must access all E experts' weights. In low-batch decoding, an expert may receive only one or two tokens, reducing GEMM to poorly utilized matrix–vector work. Nonresident weights also incur fixed loading costs from HBM, PIM or remote chiplets. As expert count grows and the active fraction falls, the active-expert union becomes sparse and long-tailed. Sieve describes this as a bimodal expert distribution^[22], while ReXpert notes that routing skew can reduce matrix utilization through low local occupancy even when weights are resident^[23].

Table 1. How MoE pressure propagates through a multi-chiplet system

1.2 Popularity alone does not describe routing

Expert popularity identifies frequently selected experts in a layer, but not the likely next-layer expert after a current selection. MoETuner shows that cross-layer routing dependencies can inform device placement^[15], and Expert Streaming organizes micro-sliced streams around dynamic expert trajectories^[17]. We represent these dependencies as a transition graph: nodes are layer–expert pairs, and directed edge weights are conditional routing probabilities between adjacent layers. Equally popular experts can have different successor distributions. Concentrated successors favor deterministic prefetching; high-entropy successors make aggressive prefetching wasteful.

Here, Nl(i,j) counts transitions from expert i at layer l to expert j at layer l+1 within a sliding window; ε is a smoothing term, and Hl(i) is transition entropy. The runtime deterministically prefetches low-entropy edges, keeps a few candidates for medium-entropy edges and maintains hot replicas without advance loading for high-entropy nodes. The example matrix for 12 experts contains strong off-diagonal edges, showing why numerical proximity or global popularity cannot replace measured dependencies.

Figure 1. Example cross-layer expert-transition probabilities

1.3 Chiplet opportunities and constraints

Chiplets allow attention, expert computation, routing control and high-capacity memory to use different process nodes connected within a package. UCIe 3.0 supports higher data rates and mappings such as Raw Streaming, providing a standards-based foundation for low-overhead transfers^[28]. However, chiplets do not remove data-movement costs: bidirectional D2D bandwidth, paths, buffers, retransmission, multicast and power remain constrained. MoEntwine uses interleaved mapping to exploit changing link demand^[19], Mozart supports expert cooperation through NoP-Tree and heterogeneous chiplets^[20], and ThAME uses 3D heterogeneous memory for weight and token flows^[21]. CoMoE-Chip therefore treats NoP as a primary resource, rather than calculating communication cost only after placement.

2. CoMoE-Chip design

2.1 Hardware and data paths

Figure 2. CoMoE-Chip architecture

As Figure 2 shows, the system includes an attention chiplet, expert chiplets, a routing-control chiplet, HBM/PIM memory chiplets and UCIe/NoP. Attention handles normalization, projections, attention and pre-routing computation. Expert chiplets contain matrix arrays, SRAM, token queues and programmable expert page tables. Routing control maintains the transition graph, versioned directory, link telemetry and credits. Memory chiplets hold complete weights, cold replicas and prefetch buffers, and may execute low-arithmetic-intensity experts near memory. This division relates to ThAME's heterogeneous memory, Sieve's dynamic GPU–PIM allocation and ReXpert's near-memory expert pools^[21][22][23], while directly coupling cross-layer prediction with the physical replica directory.

Table 2. Modules, state and interfaces

2.2 Versioned expert directory

Each directory entry records expert and layer IDs, weight version, location, precision, remaining lease, current queue, recent hits and replica state. Updates use prepare–publish–reclaim: validate the new replica at its destination, publish its version atomically, then reclaim the old copy after in-flight tokens complete reduction. This prevents a token from reading different weight versions during migration. For static inference, version changes mainly arise from model upgrades or quantization changes; online-adapted experts require stricter consistency and tenant isolation.

2.3 Two control timescales

Expert residency and token mapping need different update rates. Copying replicas moves gigabytes and replaces resident data, so it runs over tens or hundreds of batches. Mapping tokens moves kilobytes of activations and selects queues, so it can run per batch or microbatch. HCRMap similarly separates slow residency from fast mapping^[18]. CoMoE-Chip's slow loop uses stable popularity, transitions and resource pressure to select replicas; its fast loop uses current queues and path congestion to choose token destinations. Prefetching sits between them, triggered by routing events and cancelled when confidence falls.

3. Transition-guided expert reuse

3.1 Replica benefit and capacity constraints

Let binary variable xe,c indicate whether expert e resides on chiplet c, with weight size me and available capacity M_c. The expected benefit of a candidate replica is:

Here, λe is request intensity, phit is estimated from popularity and the transition graph, Ccopy is copying cost, Cmem is the opportunity cost of displacing other experts, and Cnop is the NoP pressure caused by copying. Unlike popularity-only replication, a cold expert that consistently follows a frequent predecessor may still have a high phit. Conversely, a globally popular expert already present on nearby chiplets may offer little marginal replication benefit. CRAFT uses fine-grained hierarchical estimates to avoid excess replication^[16]; CoMoE-Chip additionally includes cross-layer transitions and NoP pressure.

3.2 Confidence-tiered prefetching

After routing a layer, the controller looks up each selected expert's next-layer transition distribution. If its maximum probability exceeds θh and entropy is below Hh, it immediately prefetches the successor's pages to an expert chiplet or memory-side cache. Intermediate confidence triggers only a weight prefix, metadata or the two likeliest candidates. High entropy disables weight prefetching while retaining directory and routing preparation. APEX uses lightweight predictive routing to select edge MoE experts early^[24], and Beyond Capacity uses early expert identification to hide high-bandwidth flash reads^[25]. CoMoE-Chip emphasizes cancellation and bandwidth credits: prefetches start only with sufficient credit and remain preemptible on low-priority virtual channels.

3.3 Congestion-aware token-to-replica mapping

Model semantics require each token to execute the router's Top-k experts. The system may choose only among equivalent replicas of the same expert, not substitute another expert. Let z_t,e,c map token t's expert e to chiplet c. The objective is:

Qc is the compute queue, Lpath is path occupancy, Tfetch is the wait for unavailable weights, Bc is local memory-bank pressure, and Rtail is the resulting batch tail-latency risk; ae,c permits on-demand loading or memory-chiplet execution. The fast loop filters out version mismatches and destinations without credits, then ranks remaining replicas by cost. Predicted completion times and staged credit deductions prevent all tokens from chasing the currently least-loaded chiplet.

Figure 3. Transition prediction, reuse and congestion-aware mapping

3.4 Multicast dispatch and hierarchical reduction

The controller groups tokens by layer, expert, version and destination chiplet, then sends activations through multicast or batched DMA. Local switching nodes reduce weighted Top-k outputs before returning aggregate vectors to attention. Replicas across chiplets split tokens without duplicate execution. NoP uses at least three virtual-channel classes: high-priority tokens/results, normal weight loads and preemptible prefetches. Separate credits prevent gigabyte-scale weight flows from blocking latency-sensitive activations. Deadlock, head-of-line blocking and topology hotspots require established virtual-channel and routing principles^[29][34].

3.5 Latency and energy models

Layer latency follows the critical path through attention, routing, expert execution and reduction. Because computation, prefetching and communication partially overlap, we use:

Tdispatch depends on token bytes, NoP paths and queuing; Tload depends on missed expert weights and effective bandwidth; T_expert depends on the longest replica queue and matrix utilization. Energy includes expert computation, SRAM/HBM/PIM reads, D2D transfers and directory control. The model uses effective rather than peak D2D/HBM bandwidth and discounts concurrent weight streams for congestion. Roofline analysis distinguishes compute, local-bandwidth and remote-loading limits^[35].

4. Synthetic-trace analytical model

4.1 Evaluation scope and reproducibility

This is a proof of concept using synthetic routing traces and a parameterized analytical model, not measurements from fabricated silicon, FPGA prototypes or commercial GPUs. Zipf popularity and correlation parameter ρ control expert demand and predictable adjacent-layer successors. Expert unions, queue imbalance, weight hits, D2D bandwidth and policy overhead determine mean/P95 layer latency, token traffic, remote weight loading, load balance and replication factor. Calibration to low-batch MoE magnitudes does not replace real model traces, cycle-level NoP, DRAM timing or physical power measurements. Future validation should connect DeepSeek or Qwen traces to BookSim, Ramulator/DRAMSim and Timeloop/Accelergy^[30][31][32].

Table 3. Baseline synthetic-trace and analytical-model configuration

4.2 Comparison policies

Static mapping assigns each expert to one chiplet and loads nonresident weights on demand. Hot replication copies frequently used experts without cross-layer transitions. Transition prefetching uses Equation (2) to load next-layer experts early but does not remap for compute queues or NoP pressure. CoMoE-Chip enables bounded replication, tiered prefetching, congestion-aware mapping, credits and hierarchical reduction. All four share the same expert-capacity budget. The replication factor accounts for persistent expert pages, excluding temporary prefetch-buffer occupancy.

5. Results and analysis

5.1 Baseline results

Table 4. Analytical architecture results at the baseline configuration

At α=1.1, ρ=0.65 and 512 GB/s, CoMoE-Chip achieves mean layer latency of 1.6207 ms, 65.82% below static mapping, and P95 of 1.7107 ms, 67.01% lower. Remote loading falls from 12.9540 to 1.2448 GB per batch, a 90.39% reduction, with persistent replication factor 1.26 versus 1.44 for hot replication. Hot replication reduces P95 by only about 15.06% without improving balance: queues remain concentrated unless tokens are remapped. Transition prefetching reduces loading but retains balance near 0.51 because predicted streams may converge on one chiplet. CoMoE-Chip reaches 0.8040, demonstrating the importance of fast mapping to tail latency.

Figure 4. Remote expert loading by policy

Figure 4 isolates weight movement. Static mapping cannot amortize page reads across many tokens at low batch sizes. Hot replication removes some hot-expert reads but still loads cold experts and cross-layer successors. Transition prefetching moves much of the loading off the critical path. CoMoE-Chip further reduces total loading through replica hits and suppression of incorrect prefetches. Token traffic remains similar because all policies preserve Top-2 execution. The gain comes from reducing repeated critical-path weight movement, not removing required activation communication.

5.2 Sensitivity to routing skew

Figure 5. Expert skew and layer P95 latency

As Zipf α rises from 0.5 to 2.0, static-mapping P95 grows rapidly as hotspot queues and path traffic increase together. Hot replication relieves some compute pressure but displaces capacity useful to cold experts. Transition prefetching is less sensitive, although colocated successors can still cause congestion. CoMoE-Chip has the shallowest slope because the slow loop spreads stable hotspots and the fast loop chooses replicas using current queues. Replication cannot solve arbitrary skew: insufficient capacity or hot sets changing faster than migration can cause churn that cancels the benefit.

5.3 Sensitivity to D2D bandwidth

Figure 6. Effective die-to-die bandwidth and P95 latency

As effective D2D bandwidth rises from 128 to 2,048 GB/s, all policies improve at different rates. Static mapping depends most on interconnect transfers for both weights and tokens; hot replication still migrates large replicas. Local reuse makes CoMoE-Chip less dependent on peak bandwidth and more stable at low bandwidth. Curves converge at the high end as expert queues, HBM reads and synchronization tails dominate. Additional bandwidth then has diminishing returns. Higher UCIe 3.0 data rates still require load and traffic shaping to deliver system benefits^[28].

5.4 Cross-layer predictability

Figure 7. Transition predictability and P95 latency

Raising ρ from 0.1 to 0.9 barely changes static mapping and only slightly improves hot replication, as neither directly exploits transitions. Pure prefetching depends strongly on ρ: low predictability causes excess candidates and wasted loads; high predictability moves reads earlier. CoMoE-Chip is less sensitive because replicas and congestion-aware mapping provide a fallback at low ρ, while low-entropy successors reduce loading at high ρ. The transition graph should therefore not be a single point of dependence; low confidence requires graceful fallback.

5.5 Ablation study

Table 5 normalizes full CoMoE-Chip to 1.00 at the baseline configuration. The same analytical model recalculates path, loading and queue costs with each mechanism disabled. These values illustrate coupling and direction, not measured silicon speedups.

Table 5. Directional CoMoE-Chip ablation results

Without transitions, hotspot replication and queue mapping remain possible, but normalized loading rises to 2.14. Without congestion-aware mapping, loading barely changes while P95 rises to 1.31: nearby weights do not guarantee timely completion of the slowest expert. Removing credits has a smaller effect in the average model, but long weight packets may cause more severe P99 head-of-line blocking on a real NoP. Single-timescale updates increase replication while reducing balance, showing that more frequent migration is not necessarily better.

6. Hardware overhead, reliability and security

6.1 Controller and directory overhead

The transition graph need not store complete token sequences. With 32 layers, 64 experts per layer, eight successors per node and 16-bit counts and IDs, raw edge state is 32×64×8×4 B, about 64 KB. Double buffering, timestamps and confidence can remain within hundreds of kilobytes. The directory scales with replica count and is far smaller than gigabyte-scale weights. Fast mapping can use parallel comparators or a small vector core; firmware or a management core can run slow replica selection. Prefetch buffers and replica capacity dominate cost, not the transition table.

6.2 Correctness and consistency

CoMoE-Chip preserves gate-selected experts, choosing only equivalent replicas with matching weight versions and numeric formats. Failed prefetches may add delay but must not change Top-k results. The directory must prevent partially updated reads, mismatched quantization scales and weights, and old tokens reaching new model versions. Version IDs, checksums, in-flight reference counts and rollback pointers address these risks. Near-memory experts using different precision require layerwise error budgets and end-to-end validation; they cannot be assumed lossless replicas.

6.3 Tenant isolation and side channels

Expert demand, transitions and queues may reveal tenant topics or request phases. Maintain statistics per tenant or security domain, constrain cross-tenant replicas, expose aggregate pressure rather than expert IDs, and erase released SRAM and buffers. Adversarial requests may create hotspots, consume replica capacity or deny NoP service, requiring tenant-level replica, prefetch and queue limits. Shared model weights may be physically shared, but tokens, activations, KV and routing traces require separate address and access controls.

Table 6. Engineering risks and controls

7. Limits and validation roadmap

Four limitations remain. Synthetic Zipf–Markov traces cannot capture all correlations across layers, positions, languages and phases; anonymous traces from public MoE models are needed. Effective-bandwidth and queue terms cannot reproduce packet blocking, retransmission, clock domains or thermal constraints; BookSim or RTL validation is required. The results omit numeric precision, model quality and changing online workloads, so they do not establish lossless operation. Finally, replication factors have not been converted into package area and cost, which require process, SRAM/HBM and UCIe PHY analysis.

Table 7. From proof of concept to empirical validation

Evaluation should compare at least three groups: MoETuner/CRAFT for placement and replication, Expert Streaming/HCRMap for chiplet trajectories and pressure awareness, and Sieve/ThAME/ReXpert for PIM, 3D memory and near-memory expert pools^{[15][16][17][18][19][20][21][22][23]}. Models, weight capacity, bandwidth and SLOs must match, avoiding apparent speedups from extra replicas or relaxed tails. Serving evaluations should combine request-, engine- and expert-level scheduling; multi-level scheduling research shows that optimizing one layer can leave new head-of-line blocking^[36].

8. Conclusion

MoE replaces dense parameter computation with input-dependent expert selection, shifting bottlenecks toward residency, scatter/gather, queue imbalance and synchronization tails. CoMoE-Chip combines a versioned directory, cross-layer transition predictions, bounded replication, confidence-based prefetching, congestion-aware mapping, credits and hierarchical reduction across two timescales. In the synthetic analytical baseline, mean layer latency, P95 and remote loading fall by 65.82%, 67.01% and 90.39% versus static mapping, while balance reaches 0.804 with replication factor 1.26. These results indicate the potential of coupled mechanisms, not silicon or cycle-level measurements. Future systems should treat expert, successor, version, replica, path and queue as predictable, controllable execution state rather than static weight blocks. Real routing traces, packet-level interconnects, memory timing and physical power evaluation remain necessary.

A chiplet architecture for expert reuse in MoE inference

Abstract Mixture-of-experts (MoE) models scale parameter capacity through sparse activation, but input-dependent top-k routing creates expert hot spots, random weight fetches, scatter-gather traffic, and synchronization tails in chiplet systems. This paper presents CoMoE-Chip, an expert-reuse architecture that couples a versioned expert directory with inter-layer transition graphs, bounded replication, confidence-gated prefetching, congestion-aware token-to-replica mapping, credit control, and hierarchical reduction across attention, expert, routing-control, and HBM/PIM chiplets. The transition graph estimates conditional successor probabilities and entropy, enabling deterministic prefetch only when prediction confidence and path credits are sufficient. A synthetic trace-driven analytical model uses 64 experts, top-2 routing, eight expert chiplets, a Zipf skew of 1.1, inter-layer predictability of 0.65, and 512 GB/s effective die-to-die bandwidth. CoMoE-Chip reduces mean per-layer latency from 4.742 ms to 1.621 ms and P95 latency from 5.186 ms to 1.711 ms relative to static mapping, corresponding to reductions of 65.82% and 67.01%. Remote expert loading decreases from 12.954 GB to 1.245 GB per batch, a 90.39% reduction, while load-balance score increases from 0.516 to 0.804 with a replica factor of 1.26. These values are analytical, not silicon measurements. The results indicate that MoE chiplet efficiency requires a closed loop across routing transitions, physical residency, network pressure, and tail latency rather than replication alone.

References

[1] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. Long Beach: Curran Associates, 2017, 30: 5998-6008.

[2] SHAZEER N, MIRHOSEINI A, MAZIARZ K, et al. Outrageously large neural networks: the sparsely-gated mixture-of-experts layer[C]//International Conference on Learning Representations. Toulon: OpenReview, 2017.

[3] LEPIKHIN D, LEE H, XU Y, et al. GShard: scaling giant models with conditional computation and automatic sharding[C]//International Conference on Learning Representations. Virtual: OpenReview, 2021.

[4] FEDUS W, ZOPH B, SHAZEER N. Switch transformers: scaling to trillion parameter models with simple and efficient sparsity[J]. Journal of Machine Learning Research, 2022, 23(120): 1-39.

[5] DU N, HUANG Y, DAI A M, et al. GLaM: efficient scaling of language models with mixture-of-experts[C]//International Conference on Machine Learning. Baltimore: PMLR, 2022, 162: 5547-5569.

[6] ZOPH B, BELLO I, KUMAR S, et al. ST-MoE: designing stable and transferable sparse expert models[EB/OL]. arXiv:2202.08906, 2022[2026-09-03]. https://arxiv.org/abs/2202.08906.

[7] DAI D, DENG C, ZHAO C, et al. DeepSeekMoE: towards ultimate expert specialization in mixture-of-experts language models[C]//Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Bangkok: Association for Computational Linguistics, 2024: 1280-1297.

[8] DEEPSEEK-AI. DeepSeek-V3 technical report[EB/OL]. arXiv:2412.19437, 2024[2026-09-03]. https://arxiv.org/abs/2412.19437.

[9] DEEPSEEK-AI. DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning[EB/OL]. arXiv:2501.12948, 2025[2026-09-03]. https://arxiv.org/abs/2501.12948.

[10] YANG A, LI A, YANG B, et al. Qwen3 technical report[EB/OL]. arXiv:2505.09388, 2025[2026-09-03]. https://arxiv.org/abs/2505.09388.

[11] HWANG C, CUI W, XIONG Y, et al. Tutel: adaptive mixture-of-experts at scale[C]//Proceedings of Machine Learning and Systems. Miami: MLSys, 2023, 5: 269-287.

[12] GALE T, NARAYANAN D, YOUNG C, et al. MegaBlocks: efficient sparse training with mixture-of-experts[C]//Proceedings of Machine Learning and Systems. Miami: MLSys, 2023, 5: 288-304.

[13] RAJBHANDARI S, RASLEY J, RUWASE O, et al. DeepSpeed-MoE: advancing mixture-of-experts inference and training to power next-generation AI scale[C]//International Conference on Machine Learning. Baltimore: PMLR, 2022, 162: 18332-18346.

[14] LI J, JIANG Y, ZHU Y, et al. Accelerating distributed MoE training and inference with Lina[C]//2023 USENIX Annual Technical Conference. Boston: USENIX Association, 2023: 945-959.

[15] GO S, MAHAJAN D. MoETuner: optimized mixture of expert serving with balanced expert placement and token routing[EB/OL]. arXiv:2502.06643, 2025[2026-09-03]. https://arxiv.org/abs/2502.06643.

[16] ZHAO A, CAI Z, SONG Z, et al. CRAFT: cost-aware expert replica allocation with fine-grained layerwise estimations[EB/OL]. arXiv:2603.28768, 2026[2026-09-03]. https://arxiv.org/abs/2603.28768.

[17] MA S, LI H, ZHANG W, et al. Expert streaming: accelerating low-batch MoE inference via multi-chiplet architecture and dynamic expert trajectory scheduling[EB/OL]. arXiv:2603.27624, 2026[2026-09-03]. https://arxiv.org/abs/2603.27624.

[18] ZHANG Y. HCRMap: pressure-aware hot-expert residency mapping for 3.5D MoE chiplet inference[EB/OL]. arXiv:2607.11586, 2026[2026-09-03]. https://arxiv.org/abs/2607.11586.

[19] TANG X, HOU J, JIANG D, et al. MoEntwine: unleashing the potential of wafer-scale chips for large-scale expert parallel inference[EB/OL]. arXiv:2510.25258, 2025[2026-09-03]. https://arxiv.org/abs/2510.25258.

[20] LUO S, YE H, LI P, et al. Mozart: modularized and efficient MoE training on 3.5D wafer-scale chiplet architectures[EB/OL]. arXiv:2603.07006, 2026[2026-09-03]. https://arxiv.org/abs/2603.07006.

[21] DHINGRA P, PAL P K, DOPPA J R, et al. ThAME: 3D memory-enabled heterogeneous accelerator for LLM mixture of experts[EB/OL]. arXiv:2607.17074, 2026[2026-09-03]. https://arxiv.org/abs/2607.17074.

[22] KIM J, LACOUTURE R, ZHANG G, et al. Sieve: dynamic expert-aware PIM acceleration for evolving mixture-of-experts models[EB/OL]. arXiv:2605.11277, 2026[2026-09-03]. https://arxiv.org/abs/2605.11277.

[23] SHAO K, ZENG M, YUAN X, et al. MoE expert execution in disaggregated LLM serving with a high-bandwidth ReRAM near-memory architecture[EB/OL]. arXiv:2608.13962, 2026[2026-09-03]. https://arxiv.org/abs/2608.13962.

[24] KANANI A, BADAWI L, OGRAS U Y. APEX: adaptive expert prefetching for memory-efficient edge MoE inference[EB/OL]. arXiv:2608.11688, 2026[2026-09-03]. https://arxiv.org/abs/2608.11688.

[25] KIM S, JIN J, KIM J Y. Beyond capacity: scalable MoE LLM inference via high-bandwidth flash with direct GPU and HBM paths[EB/OL]. arXiv:2608.14333, 2026[2026-09-03]. https://arxiv.org/abs/2608.14333.

[26] WU Q, FANG C, CHEN J, et al. A scheduling framework for efficient MoE inference on edge GPU-NDP systems[EB/OL]. arXiv:2601.03992, 2026[2026-09-03]. https://arxiv.org/abs/2601.03992.

[27] FAN Z, LIU L. Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems[EB/OL]. arXiv:2512.04476, 2025[2026-09-03]. https://arxiv.org/abs/2512.04476.

[28] UNIVERSAL CHIPLET INTERCONNECT EXPRESS CONSORTIUM. UCIe specification revision 3.0[S/OL]. Beaverton: UCIe Consortium, 2025[2026-09-03].

[29] DALLY W J, TOWLES B. Principles and practices of interconnection networks[M]. San Francisco: Morgan Kaufmann, 2004.

[30] JIANG N, BECKER D U, MICHELOGIANNAKIS G, et al. A detailed and flexible cycle-accurate network-on-chip simulator[C]//IEEE International Symposium on Performance Analysis of Systems and Software. Austin: IEEE, 2013: 86-96.

[31] PARASHAR A, RAINA P, SHA A, et al. Timeloop: a systematic approach to DNN accelerator evaluation[C]//IEEE International Symposium on Performance Analysis of Systems and Software. Madison: IEEE, 2019: 304-315.

[32] WU Y N, EMER J S, SZE V. Accelergy: an architecture-level energy estimation methodology for accelerator designs[C]//IEEE/ACM International Conference on Computer-Aided Design. Westminster: IEEE, 2019: 1-8.

[33] JOUUPPI N P, YOUNG C, PATIL N, et al. In-datacenter performance analysis of a tensor processing unit[C]//ACM/IEEE 44th Annual International Symposium on Computer Architecture. Toronto: ACM, 2017: 1-12.

[34] KIM J, DALLY W J, SCOTT S, et al. Technology-driven, highly-scalable dragonfly topology[C]//ACM/IEEE 35th Annual International Symposium on Computer Architecture. Beijing: IEEE, 2008: 77-88.

[35] WILLIAMS S, WATERMAN A, PATTERSON D. Roofline: an insightful visual performance model for multicore architectures[J]. Communications of the ACM, 2009, 52(4): 65-76.

[36] SUN Y, HAFFAR G, XU M, et al. Multi-layer scheduling for MoE-based LLM reasoning[EB/OL]. arXiv:2602.21626, 2026[2026-09-03]. https://arxiv.org/abs/2602.21626.