Hongxiang Shangdao-Elexvx — Compiler Optimization for Large Models on Edge NPUs

Abstract

On-device NPUs combine efficient matrix arrays, tightly coupled memory and asynchronous DMA, but static graphs, limited dynamic shapes, address spaces and thermal budgets conflict with autoregressive state and mixed operators. EdgeLLM-MLIR introduces four IR levels—LLM semantics, state layout, Tile-DMA and target instructions—to jointly optimize KV, quantization metadata, length buckets, fusion, double buffering and power. Fusion becomes graph partitioning under memory and peak-power constraints. At 16 MB TCM, the analytical model estimates 0.055 GB external traffic per layer, 58.33% below an operator library; latency proxy 1.544 ms, 45.66% lower than the library and 11.14% below unconstrained mega-fusion; and power proxy 0.82 versus 1.00. Six optimized length buckets limit compilation to six variants with 26.37% padding. These modeled results suggest jointly scheduling state, movement, shapes and power rather than maximizing fusion alone.

On-device LLMs must prefill and decode under memory, sustained-power and latency constraints. Unlike programmable GPU threads and caches, NPUs commonly use matrix arrays, vector units, TCM, explicit DMA and static schedules, requiring compile-time knowledge of shapes, layouts and transfers. TVM, XLA, MLIR and Triton provide graph optimization, fusion, multilevel IR and tile programming^[3]^[4]^[5]^[6], but LLMs add four issues: prefill GEMM and batch-one decode GEMV need different flows; growing KV/RoPE is difficult in static graphs; W4A16 requires fused unpacking, dequantization and multiplication to avoid intermediate traffic; and mobile voltage/thermal variation makes maximal fusion susceptible to power bursts. Hexagon-MLIR demonstrates generative fusion with TCM, DMA and mega-kernels^[15]; TileFuse and Gemma3 deployment highlight weight layouts, mixed precision and pipelined attention^[16]^[17]; LENS shows nonlinear NPU latency under static bucketing^[18]. EdgeLLM-MLIR unifies state layouts, shape buckets, fusion boundaries, DMA events and power constraints, using an analytical model to examine their limits.

1. Mapping conflicts on edge NPUs

1.1 Phase and dataflow differences

Prefill processes many tokens with high-intensity GEMM; decoding adds few tokens and repeatedly reads weights and KV, favoring GEMV and often limited by external bandwidth and scheduling. FlashAttention reduces GPU HBM round trips through tiling^[10]^[11], but smaller NPU TCM and weaker coherence require explicit lifetimes for Q/K/V, softmax statistics and KV appends. Separate operators repeatedly spill residuals, normalization, dequantization and projections to DRAM; whole-layer fusion can also spill when live tiles exceed TCM.

Table 1. NPU–LLM mismatches

Dimension	NPU characteristic	LLM characteristic	Compiler requirement
Shapes	Static or limited dynamic shapes	Changing input, output and KV lengths	Buckets, predication and stateful addressing
Storage	Small TCM and explicit DMA	Coexisting weights, activations, KV and scales	Cross-operator lifetimes and double buffering
Operators	Efficient matrix units; irregular nonlinear work and sampling	Mixed GEMM/GEMV, softmax and RoPE	Heterogeneous dispatch and fusion clusters
Precision	Fixed low-bit or mixed-precision paths	Diverse W4A16, FP4 and KV formats	Metadata layouts and fused dequantization
Execution	Static commands and costly reconfiguration	Autoregressive loops and early stopping	Persistent graphs, bucket reuse and lightweight runtime
Power	Battery, temperature and peak-current constraints	Sustained generation	Power-constrained fusion and throttling

1.2 The cost of static sequence shapes

Many edge NPU compilers require fixed shapes, so deployments compile length variants and pad requests to the next bucket. For actual length l and bucket b(l)≥l, T(b) waste can grow through quadratic attention and KV traffic, not just b-l. Coarse buckets waste work; fine buckets increase compilation, package size, selection and cache pressure. LENS observes nonlinear latency at bucket boundaries^[18], and shadowAttn uses multiple bucketed graphs to reconcile static NPUs with dynamic sparsity^[21]. Bucketing should therefore be a compiler variable, not a fixed deployment heuristic.

1.3 Maximum fusion is not always optimal

Fusion removes intermediate writes but increases simultaneously live inputs, outputs, scales, statistics and double buffers. Hexagon-MLIR's generated mega-kernels go beyond operator-library boundaries^[15], but mobile devices face current, DVFS and voltage constraints. Recent measurements show aggressive fused superlayers can create current bursts and throttling^[19]. The compiler must choose where fusion stops under TCM, bandwidth, parallelism and power budgets.

2. EdgeLLM-MLIR framework

2.1 Four IR levels

LLM semantic IR retains attention, RoPE, KV append, quantized linear operations, sampling and phase information. State-layout IR describes pages, positions, scales, packed weights and persistent state. Tile-DMA IR introduces tiles, buffers, events and dependencies. Target IR maps these to matrix, vector, addressing and barrier primitives. This preserves KV and quantization semantics that direct graph-to-library lowering can lose.

Figure 1. EdgeLLM-MLIR compilation flow

Figure 2. Four IR levels for LLMs and NPUs

Table 2. IR objects and optimizations

IR level	Core objects	Preserved semantics	Main optimizations
LLM semantic IR	Attention, MLP, RoPE and sampling loops	Prefill/decode phases and causality	Phase specialization, rewriting and heterogeneous dispatch
State-layout IR	KV pages, packed weights, scales and position maps	Persistent state, versions and logical addresses	KV layouts, colocated metadata and page granularity
Tile-DMA IR	Tiles, buffers, DMA and events	On-chip lifetimes and dependencies	Tiling, fusion, double buffering and transfer overlap
Target-instruction IR	Matrix/vector instructions, TCM addresses and barriers	Device resources and instruction constraints	Instruction selection, registers and static scheduling

2.2 State- and layout-aware lowering

KV is a compiler object with head groups, pages, sequence continuity, precision, scale locations and DMA granularity. Layouts follow vector widths and burst lengths: large writes for prefill, head/channel reads for decoding. Weights and scales can be interleaved when dequantization is near matrix units; otherwise vector unpacking feeds matrix tiles through a pipeline. Gemma3's FusedDQP, FlowQKV and FlowKV demonstrate this coupling^[17]; TileFuse similarly maps AWQ directly to XDNA2 through pre-tiling and metadata colocation^[16].

Runtime updates effective length, KV base and page tables without recompiling within a bucket. Exceeding a bucket selects another variant, with lightweight page-table composition. Sampling, complex control and small scalar work may fall back to CPU/iGPU, with transfer, synchronization and coherence costs explicitly modeled instead of pursuing NPU coverage at any cost.

2.3 Power-constrained fusion search

For DAG G=(V,E), a fusion cluster c contains jointly schedulable operators. Eliminated traffic is balanced against TCM occupancy, reduced parallelism, code growth and peak power. Fusion is modeled as graph partitioning:

min_{P} \sum _{c} [ T_{c} + \lambda D_{c}^{ext} + \mu \max (0,P_{c}^{peak}-P_{budget})^2 + \nu C_{c}^{compile} ] \tag{1}

s.t. M_{c}^{live} + M_{c}^{double} \le M_{TCM}, \text{c preserves data dependence} \tag{2}

Here, Tc is execution time, Dc^ext external traffic, Pc^peak peak-power proxy, and Cc^compile code and compilation cost. Candidate attention, MLP and quantized-linear subgraphs are pruned by TCM, reuse, overlap and power. When another operator exceeds live-buffer or power limits, a boundary is inserted, potentially aligned with a barrier or lower-power phase.

Figure 3. Power-constrained fusion-cluster generation

2.4 Double buffering and events

Tile-DMA IR represents transfers and computation as an event graph. Let tile i's input DMA, computation and output DMA times be di, ci and w_i. The steady-state double-buffer lower bound is:

T_{tile} \ge \max (d_{i}, c_{i}, w_{i}) + T_{sync} \tag{3}

Separate weight, activation and KV pools schedule next-tile prefetch, current-tile compute and previous-tile writeback. If two complete tiles do not fit, asymmetric buffers or weight-only overlap are used. Decode prioritizes weight buffering; long-context attention reallocates buffers toward KV according to arithmetic intensity.

2.5 Optimizing length buckets

Given length distribution p(l), candidate buckets B and analytical latency T_b(l), the objective is:

min_{B} \sum _{l} p(l)[T_{b}(l)+\kappa Pad(l,b)] + \xi |B| + \zeta S_{code}(B) \tag{4}

Pad represents extra compute/storage, |B| counts variants and S_code represents code/constants growth. Dynamic programming partitions sorted boundaries using maximum length, measured or fitted latency and memory. Prefill may use fine buckets while decoding buckets by KV limit and batch size. LENS fits bucket latency from a small number of end-to-end measurements^[18] and can calibrate black-box NPU cost models.

Figure 4. Padding and latency proxies by bucket strategy

3. Analytical model and setup

3.1 Scope

The reproducible analytical model compares libraries, local fusion, unconstrained mega-fusion and EdgeLLM-MLIR across TCM sizes; these are not commercial NPU measurements. One Transformer block includes quantized projections, attention, residuals and MLP, with DMA-fed tiles and dependent matrix/vector execution. The 16 MB baseline normalizes representative traffic and kernel proportions before capacity scaling. The power proxy measures simultaneous activity, not watts or amperes.

Table 3. Model parameters and boundaries

Item	Setting	Notes
TCM capacity	4, 8, 16, 32 MB	From small mobile to larger client NPUs
External memory	Shared effective-bandwidth model	Not tied to an LPDDR generation or frequency
Execution policy	Library, local fusion, mega-fusion and EdgeLLM-MLIR	Comparing fusion granularity and power constraints
Buffer policy	Single or asymmetric double buffering	Determined by TCM feasibility
Latency components	Compute, DMA, synchronization and reconfiguration	Maximum overlapped term plus unhidden overhead
Power proxy	Normalized values near 0–1	Peak concurrent activity, not measured power
Length distribution	Heavy-tailed requests from 128 to 4,096	For bucket-cost analysis
Nature of results	Analytical model	Does not replace end-to-end hardware measurements

3.2 Latency and traffic

Cluster latency combines computation, external traffic and DMA events:

T_{c} = \max (F_{c}/P_{eff}, D_{c}/B_{eff}, T_{DMA},c) + T_{sync},c + T_{dispatch},c \tag{5}

Fc is effective operations, Peff depends on matrix/vector utilization and tile shape, Dc is nonreusable external bytes, and Beff is effective bandwidth. Mega-fusion lowers Dc but may reduce Peff through spills or resource conflicts. Equation (1) balances traffic and stable power. Cache replacement, DRAM bank conflicts and OS scheduling require later hardware-counter validation.

4. Results and analysis

4.1 TCM capacity and external traffic

Figure 5. TCM capacity and external traffic per layer

Traffic falls for all policies as TCM grows from 4 to 32 MB. Libraries remain highest because call boundaries may write intermediates externally; local fusion removes some transfers; mega-fusion and EdgeLLM-MLIR are lowest. At 4 MB, both large-fusion approaches suffer double-buffer feasibility penalties. EdgeLLM-MLIR is slightly lower because scales and KV can remain across adjacent clusters without one whole-layer kernel.

Figure 6. TCM capacity and layer-latency proxy

Latency broadly follows traffic but also depends on scheduling. At 32 MB, EdgeLLM-MLIR gains from matrix/vector scheduling and asymmetric buffers despite already-low mega-fusion traffic. At 4 MB both are capacity-limited. Minimum traffic alone is insufficient: tile shapes, resources and synchronization matter.

Table 4. Analytical results at 16 MB TCM

Policy	External traffic / GB	Latency proxy / ms	Power proxy	Latency reduction versus library / %
Operator library	0.132	2.841	0.66	0.00
Local fusion	0.091	2.108	0.78	25.80
Mega-fusion	0.060	1.737	1.00	38.85
EdgeLLM-MLIR	0.055	1.544	0.82	45.66

At 16 MB, EdgeLLM-MLIR uses 0.055 GB, 58.33% below the library. Latency falls from 2.841 to 1.544 ms, a 45.66% reduction; it is 11.14% below mega-fusion while the power proxy falls from 1.00 to 0.82. These depend on modeled power, buffering and parallelism assumptions and require device validation.

4.2 Value of power constraints

Figure 7. Latency and power proxies by fusion policy

Figure 7 shows the locality–peak-activity tradeoff. Libraries have low activity but high latency; mega-fusion keeps matrix, vector and DMA resources active together longer. EdgeLLM-MLIR retains reuse while splitting peak clusters. Runtime can lower P_budget under low voltage or thermal pressure and select a precompiled low-peak variant, then return to performance variants when conditions improve. This is more predictable than reactive DVFS but needs stable power-budget signals.

4.3 Bucket tradeoffs

Table 5. Dynamic length-bucket results

Policy	Bucket count	Boundaries	Padding / %	Latency proxy	Compiled variants
Powers of two	6	128/256/512/1 024/2 048/4 096	27.96	3.311	6
Uniform 128-token spacing	32	128…4 096	14.52	2.631	32
EdgeLLM optimization	6	256/512/768/1 152/2 048/4 096	26.37	3.122	6
Single 4,096-token bucket	1	4 096	86.23	19.139	1

Uniform 128-token buckets minimize padding and latency but require 32 variants. One bucket has 86.23% padding and latency proxy 19.139. Six nonuniform EdgeLLM buckets achieve 26.37% padding and proxy 3.122, slightly better than six power-of-two buckets. Denser short-length boundaries balance request probability, attention cost and code size. Frequent buckets may remain cached while rare variants load on demand.

4.4 Ablation

Table 6 normalizes full EdgeLLM-MLIR to 1.00. Removing state layout adds cross-cluster KV/scale writes; removing DMA scheduling serializes transfers; removing power constraints may improve cold latency but increase thermal/voltage throttling; fixed power-of-two buckets add padding. Values are analytical directions, not measured speedups.

Table 6. Directional ablation

Configuration	Normalized latency	Normalized external traffic	Normalized peak power	Main cause
Full EdgeLLM-MLIR	1.00	1.00	1.00	Joint state, fusion, DMA, buckets and power optimization
Without state-layout IR	1.19	1.34	0.99	KV and scales cannot remain across clusters
Without double buffering	1.27	1.00	0.88	DMA is exposed; concurrent activity falls
Without power constraints	0.96	0.98	1.22	Larger clusters with DVFS risk
Fixed power-of-two buckets	1.06	1.04	1.02	Excess short-sequence padding and KV limits
All operators on NPU	1.13	1.08	1.05	Irregular work and device switching offset gains

5. Implementation and reliability

5.1 Compilation cache and runtime

Offline variants span phase, length bucket, power and precision. Runtime selects using length, KV limit, battery and temperature. To limit combinations, only frequent prefill buckets get multiple power levels; decode graphs reuse predicates/effective lengths. Constants are shared; commands and buffers vary. Cache keys include model hash, NPU generation, driver and quantization, with invalidation after firmware changes.

5.2 Correctness, security and fallback

Fusion and reordering must respect numeric semantics. Retain high-precision references, layerwise thresholds and random test vectors for differential checks of every variant. Unpacking, RoPE and KV append need bucket/page/early-stop coverage. NPU failure, overheating or incompatibility should permit CPU/iGPU fallback with consistent KV. DMA, shared memory and compilation caches need process isolation, erasure and signature checks.

5.3 Applicability and future validation

Table 7. Applicability and limits

Aspect	Applicable conditions	Current limits	Future work
Hardware	TCM, DMA and programmable matrix/vector paths	Commercial NPU interfaces may be closed	Combine black-box measurements with exposed compiler hints
Model	Transformers with bounded dynamic lengths	State-space and diffusion models need new semantic IR	Extend multi-token parallelism and revision semantics
Quantization	Offline reordering of packed weights and scales	Opaque or incompatible vendor formats	Establish verifiable quantization contracts
Power	Available temperature, battery or power-level signals	Power proxies are not actual current	Device-calibrated peak models
Evaluation	Measurable end-to-end latency, energy and correctness	Analytical models omit system noise	Prototype across multiple NPU vendors

Analytical results cannot replace compiler, arbitration, driver or power measurements. Next steps are prototypes on Hexagon-MLIR, MLIR-AIE or open backends using 1B–8B quantized models, measuring prefill, decode tokens/s, TCM hits, LPDDR bytes, switching, peak current, sustained temperature and accuracy. Closed platforms can use LENS-like calibration and automatic graph rewrites to search fusion boundaries.

6. Conclusion

Edge NPU performance depends on executable dataflows combining autoregressive state, variable lengths, metadata, tiles and power constraints. EdgeLLM-MLIR preserves semantics through four IR levels and jointly schedules fusion, buffers, events, buckets and power. The 16 MB analytical case reduces traffic and latency proxies without unconstrained mega-fusion's activity peak. The objective is constrained movement minimization for state lifetimes and operating conditions, rather than maximum fusion or coverage. Multi-vendor kernels, counters and current/temperature measurements remain necessary.

LLM compilation optimization for edge NPUs

Abstract Edge neural processing units provide efficient matrix arrays, tightly coupled memory, and asynchronous DMA, but their static graphs, limited dynamic shapes, constrained address spaces, and power envelopes conflict with autoregressive LLM state, variable sequence lengths, and mixed operators. This paper presents EdgeLLM-MLIR, an LLM compiler framework with four levels of intermediate representation: LLM semantics, persistent state and layout, tile-DMA events, and target instructions. It jointly optimizes KV-cache layout, quantization metadata, sequence-length buckets, fusion clusters, double buffering, and a peak-power budget. Operator fusion is formulated as graph partitioning constrained by live TCM capacity and peak activity rather than as unconditional mega-kernel generation. In an analytical model, under a 16 MB TCM configuration, EdgeLLM-MLIR reduces per-layer external traffic from 0.132 GB to 0.055 GB and the latency proxy from 2.841 ms to 1.544 ms compared with an operator-library baseline, corresponding to reductions of 58.33% and 45.66%. It also improves latency by 11.14% over unconstrained mega-fusion while reducing the normalized power proxy from 1.00 to 0.82. A six-bucket optimized policy limits compilation variants to six with a 26.37% padding ratio. These are analytical results, not measurements from a commercial NPU. The study shows that edge LLM compilation should jointly manage state, data movement, shapes, and power rather than maximize fusion alone.

References

[1] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. Long Beach: Curran Associates, 2017, 30: 5998-6008.

[2] LAM M S, MELLOR-CRUMMEY J M, WOLF M E, et al. The cache performance and optimizations of blocked algorithms[C]//Proceedings of the Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. New York: ACM, 1991: 63-74.

[3] CHEN T, MOREAU T, JIANG Z, et al. TVM: an automated end-to-end optimizing compiler for deep learning[C]//13th USENIX Symposium on Operating Systems Design and Implementation. Carlsbad: USENIX Association, 2018: 578-594.

[4] LAO L, LE B, VANTREASE D H, et al. TensorFlow XLA: optimizing compiler for machine learning[CP/OL]. Mountain View: Google, 2017[2026-09-03].

[5] LATTNER C, AUERBACH R, BONILLA E, et al. MLIR: scaling compiler infrastructure for domain specific computation[C]//IEEE/ACM International Symposium on Code Generation and Optimization. Piscataway: IEEE, 2021: 2-14.

[6] TILLET P, KUNG H T, COX D. Triton: an intermediate language and compiler for tiled neural network computations[C]//Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages. New York: ACM, 2019: 10-19.

[7] ROETTGER T, et al. IREE: a retargetable compiler and runtime for machine learning programs[CP/OL]. Mountain View: OpenXLA Project, 2024[2026-09-03].

[8] PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library[C]//Advances in Neural Information Processing Systems. Vancouver: Curran Associates, 2019, 32: 8024-8035.

[9] ANKERST M, et al. Torch-MLIR: lowering PyTorch programs to MLIR[CP/OL]. San Francisco: LLVM Project, 2024[2026-09-03].

[10] DAO T, FU D Y, ERMON S, et al. FlashAttention: fast and memory-efficient exact attention with IO-awareness[C]//Advances in Neural Information Processing Systems. New Orleans: Curran Associates, 2022, 35: 16344-16359.

[11] DAO T. FlashAttention-2: faster attention with better parallelism and work partitioning[C]//International Conference on Learning Representations. Vienna: OpenReview, 2024.

[12] KWON W, LI Z, ZHUANG S, et al. Efficient memory management for large language model serving with PagedAttention[C]//Proceedings of the 29th Symposium on Operating Systems Principles. New York: ACM, 2023: 611-626.

[13] LIN J, TANG J, TANG H, et al. AWQ: activation-aware weight quantization for on-device LLM compression and acceleration[C]//Proceedings of Machine Learning and Systems. Santa Clara: MLSys, 2024, 6: 87-100.

[14] XIAO G, LIN J, SEZNEC M, et al. SmoothQuant: accurate and efficient post-training quantization for large language models[C]//International Conference on Machine Learning. Honolulu: PMLR, 2023, 202: 38087-38099.

[15] ABSAR M J, BASKARAN M, SHARMA A, et al. Hexagon-MLIR: an AI compilation stack for Qualcomm neural processing units[EB/OL]. arXiv:2602.19762, 2026[2026-09-03]. https://arxiv.org/abs/2602.19762.

[16] PANG W, JUN G H, LIU F, et al. TileFuse: a fused mixed-precision kernel library for efficient quantized LLM inference on AMD NPUs[EB/OL]. arXiv:2606.11357, 2026[2026-09-03]. https://arxiv.org/abs/2606.11357.

[17] DU S, YU M, NI Z, et al. Mapping Gemma3 onto an edge dataflow architecture[EB/OL]. arXiv:2602.06063, 2026[2026-09-03]. https://arxiv.org/abs/2602.06063.

[18] PARK J, JEONG S, LEE J, et al. Latency prediction for LLM inference on NPU systems[EB/OL]. arXiv:2606.18042, 2026[2026-09-03]. https://arxiv.org/abs/2606.18042.

[19] YUZAWA R, TOMIZUKA M. Mitigating compiler fusion-induced power bursts in mobile NPU inference as the battery depletes[EB/OL]. arXiv:2607.16555, 2026[2026-09-03]. https://arxiv.org/abs/2607.16555.

[20] WANG T, SUN Y, REN J. Efficient on-device diffusion LLM inference with mobile NPU[EB/OL]. arXiv:2606.13740, 2026[2026-09-03]. https://arxiv.org/abs/2606.13740.

[21] YIN W, XU D, XU M, et al. System and algorithm co-design for NPU-centric on-device sparse attention[EB/OL]. arXiv:2508.16703, 2025[2026-09-03]. https://arxiv.org/abs/2508.16703.

[22] KUMARESAN R. Orion: characterizing and programming Apple neural engine for LLM training and inference[EB/OL]. arXiv:2603.06728, 2026[2026-09-03]. https://arxiv.org/abs/2603.06728.

[23] ABADI M, BARHAM P, CHEN J, et al. TensorFlow: a system for large-scale machine learning[C]//12th USENIX Symposium on Operating Systems Design and Implementation. Savannah: USENIX Association, 2016: 265-283.

[24] GOOGLE. LiteRT documentation[CP/OL]. Mountain View: Google, 2026[2026-09-03].

[25] APPLE. Core ML framework documentation[CP/OL]. Cupertino: Apple, 2026[2026-09-03].

[26] META. ExecuTorch: on-device AI runtime[CP/OL]. Menlo Park: Meta, 2026[2026-09-03].

[27] LIN C, et al. MNN: a universal and efficient inference engine[C]//Proceedings of Machine Learning and Systems. Austin: MLSys, 2020, 2: 1-13.

[28] JIA Z, MAGGIONI M, STAIGER B, et al. Dissecting the NVIDIA Turing T4 GPU via microbenchmarking[EB/OL]. arXiv:1903.07486, 2019[2026-09-03].

[29] JOULES J, et al. Energy-aware compilation for deep learning accelerators: a survey[J]. ACM Computing Surveys, 2024, 56(8): 1-38.

[30] WILLIAMS S, WATERMAN A, PATTERSON D. Roofline: an insightful visual performance model for multicore architectures[J]. Communications of the ACM, 2009, 52(4): 65-76.

[31] CHEN Y H, EMER J, SZE V. Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks[C]//ACM/IEEE 43rd Annual International Symposium on Computer Architecture. Piscataway: IEEE, 2016: 367-379.

[32] PARASHAR A, RAINA P, SHA A, et al. Timeloop: a systematic approach to DNN accelerator evaluation[C]//IEEE International Symposium on Performance Analysis of Systems and Software. Piscataway: IEEE, 2019: 304-315.

[33] WU C, BROOKS D, CHEN K, et al. Machine learning at Facebook: understanding inference at the edge[C]//IEEE International Symposium on High Performance Computer Architecture. Piscataway: IEEE, 2019: 331-344.

[34] DEGRAAF D, et al. A survey of neural processing units for edge intelligence[J]. IEEE Micro, 2025, 45(2): 16-28.

[35] QUALCOMM. Hexagon neural processing unit architecture overview[R/OL]. San Diego: Qualcomm Technologies, 2026[2026-09-03].

[36] AMD. Ryzen AI software documentation[CP/OL]. Santa Clara: Advanced Micro Devices, 2026[2026-09-03].

Author: R&D Center
Keywords: large language model; neural processing unit; MLIR; operator fusion; dynamic bucketing; power-aware compilation

Compiler Optimization for Large Models on Edge NPUs