Tile status and performance

Current conclusion

As of September 14, 2026, development is on next: the architecture runs, but the general MPS/Torch performance goal is not complete. Several bounded FP32 cohorts on Apple M1 Max beat eager Torch; large GEMM and direct XIR/SIMD still have substantial gaps. These results do not establish production LLM, low-precision, all-shape or cross-device parity.

The C++ language, mutable TileIR, native C++ TIRx bridge, bounded Metal MPP lowering and the XIR/SIMD and XIR/Metal4 Runtime routes are implemented. Execution mapping is planned rather than mechanically copied from logical hierarchy. Backend-owned target information and cost policies are extensible; their legal candidates and calibration remain bounded. See coverage and compiler architecture.

The legacy migration / low-precision checkpoint adds reusable examples and FP16/BF16 coverage. Its diagnostic comparisons retain unstable FP16 wins/regressions and large scan regressions; they do not replace the qualified Torch/MPS scoreboard below.

The Top-K/sort extension adds five-route, two-output correctness and timing coverage through width 1025. Its first 60 records pass numerical checks but expose large reference- algorithm gaps; they do not establish ranking parity. Attention remains a priority, especially phase-specific decode distributions. Historical prefill/decode gaps must not be presented as measurements of the unfinished matrix-initializer work in the current working tree. The compiler-generated native MMA candidate improves one batch-GQA case by 25.8% but regresses the other five attention cases by 6.2%–60.4%; it remains opt-in. See the attention experiments for the matched native-entry results and remaining layout-transition costs. Follow-up native profiling identifies K/V snapshot copies as the dominant sampled MHA-on hotspot. The subsequent generic contiguous-copy candidate, with native MMA held on, reduces paired decode time by 70.2%–78.8%, prefill-q4 by 25.8%, and batch GQA by 50.4%. Prefill-q8 loses its full-packet clone and regresses 2.6%; all six pairs preserve capacity and bitwise outputs. This is an opt-in transfer-mapping improvement, not a new Torch/MPS comparison or calibrated automatic policy.

The September 14 short-packet reduction checkpoint adds generic short-domain XIR cooperation and checked per-value layout geometry, not complete attention distribution. Its fixed-block Metal probe finds that width-seven cooperation can be slower; the existing width-65 cooperative path is faster in GPU intervals. Host and GPU metrics disagree in several cases, so this is not an automatic policy or a new MPS/Torch parity claim.

How to read the performance evidence

Keep these objectives separate:

  • Batched E2E throughput: warm host time per invocation, amortized over dispatches and synchronization; JIT/setup excluded.

  • Single-call E2E latency: one dispatch through completion.

  • CPU native entry: actual emitted kernel entry called from C++, with Runtime dispatch/Python/JIT and caller allocations excluded. Required native traversal, launch resets and compiler-emitted internal allocations stay inside; this is not a hardware cycle counter.

  • GPU timing: legacy Metal compute-pass intervals or Metal4 precise dispatch intervals, each with a separate no-counter command-buffer control. Neither instrumented interval is a zero-overhead observation. The control includes GPU work and intra-buffer gaps, not isolated kernel time.

Counters perturb some Torch cases substantially. Every comparison retains fusion, output-allocation and math-policy differences, paired round ratios and negative results. Historical experiments are not one matched leaderboard.

Results by route

Direct SIMD: bounded native wins, with shape-dependent fusion results. The September 10 24-case fixed-candidate comparison uses actual ORC and TorchInductor 2.14.0 entries, FP32 and one CPU worker, at four sizes through approximately 4.2 million elements. With fusion on, RMSNorm and GELU+residual take 0.344–0.457× and 0.553–0.632× Inductor time; all 48 paired rounds win. LayerNorm spans 0.376–1.084×, with its smallest case losing. Masked softmax and SwiGLU remain 1.344–1.638× and 1.032–1.172×; all 48 paired rounds lose. RoPE spans 0.880–2.370×: two larger cases’ medians win, but 17×66 and 129×768 still lose. Full FP64 checks accompany the documented math differences; these are native-entry times, not Runtime E2E.

This measurement uses existing opt-in use-site memory realization and guarded pointwise fusion, not a new compiler optimization or calibrated automatic policy. The separate 440-visit E2E screen improves many large/local row programs but retains small task regressions and non-winning attention. Shared-input/multi-output DAG fusion has a guarded, opt-in implementation with alias and bounds regressions. The complete matrix shows mixed profitability: large RoPE takes 0.695× off time, but small RoPE and LayerNorm regress to 1.702× and 1.457×. Eight RMSNorm/softmax off/on objects are identical controls, not fusion speedups. The separate pilot’s negative results remain. Current LLVM22 off/on entries must not be spliced into the older September 9 LLVM21 timings; desktop coactivity is recorded, and no default or cost policy is promoted. A separate expression/reduction fusion checkpoint now merges two producer traversals in masked softmax and one in LayerNorm, with full output checks at four sizes. It remains opt-in: an interrupted, contended timing cohort is excluded, so the native ratios above are not replaced by a new speedup claim. The subsequent next integration check keeps two contended native replay attempts out of the performance evidence. Two later native codegen probes isolate scalar address demand and packet-loop inlining. Their correctness and code evidence are complete, but both timing cohorts remain diagnostic-only; the experimental overlays are not promoted into the working compiler. Independent task grain and realization-sensitive cost calibration remain necessary; native superiority alone does not establish dispatch latency or default-path parity.

Metal matrix programs: broader legal lowering, incomplete profitability. Bounded K/M/N views and direct output remove unnecessary staging using access/recurrence proofs. The realized-work model improves three changed choices by 3.88–17.75% median GPU time, but all eight GPU medians still lose to Torch. Scale coverage extends through 8192³ and large rectangles; the historical 1024³ MPS near-parity result does not generalize to those shapes. Traversal/participation experiments retain regressions and have not produced a new universal default.

The latest closed scalar epilogue candidate uses ordinary expression DAGs, not activation-name rules. At fixed 256-worker schedules, ragged ReLU/GELU GPU medians improve 15.54%/14.41%, but 4096³ GELU regresses 35.14%. All 288 complete native/Torch outputs pass. The candidate stays off by default; removing storage is not a profitability proof. Native-MPP and CPU/SIMD emission are unchanged.

Metal elementwise and row programs: measured structural wins, with limits. Multi-output pointwise fusion uses common ownership/effect analysis through 4096×4096. Eight GPU medians beat preallocated eager Torch (new/Torch 0.309–0.952), with mixed single-call latency and one small-GELU GPU reversal. Wide-row reductions use SIMD-group collectives and compact worker-private stripes. Softmax, RMSNorm and LayerNorm beat eager Torch in all 18 GPU/E2E throughput cases in two observed orders, through width 16384 and 512 MiB payloads. Some large margins are only 1–4%; latency and returned-output allocation differ. Cooperating-row packing and held-out cost policies still regress on several cases; they remain opt-in.

Partitioned-output fusion now handles proved disjoint regions of one buffer. Four RoPE GPU batch medians are 0.129–0.319× preallocated eager Torch through 4096×4096, with all 24 paired GPU comparisons winning. SwiGLU’s unchanged single-output code measures 0.486–0.593× Torch; tiny single-call E2E still loses. These are bounded FP32 fusion results, not attention or direct-SIMD parity.

CPU: provider wins are not direct-XIR parity. Proved CBLAS and Accelerate realizations improve admitted TIRx families; CBLAS beats eager Torch on seven of eight replayed shapes. Direct XIR/SIMD has a working mapping solver and packet-index codegen repair, but all six measured GEMMs still lose to Torch. The private-vector realization keeps mapping/layout fixed and realizes common-slot private accesses as contiguous vectors. Native RMSNorm drops from 28.623 to 11.511 µs at 64×256 and from 7380.091 to 3336.237 µs at 1024×4096; paired new/Inductor time ratios still remain 1.260/1.503. This is generic backend lowering, not a calibrated new solver. The subsequent first-consumer load/reduction fusion is legal but default-disabled: actual native RMSNorm regressions are 17–23% despite lower estimated memory work. This points to missing full-packet specialization/inlining and phase-cost interactions. The earlier full-packet specialization addresses one such interaction: at fixed packet-local mapping with fusion off, native RMSNorm reaches 5.334 µs / 1117.151 µs for 64×256 / 1024×4096, with paired new/Inductor ratios 0.603 / 0.511. All 12 corresponding paired rounds win. This generic backend candidate remains opt-in, as does joint mapping search; it does not establish default-path or all-operator parity. The accompanying 240-visit Runtime E2E screen also improves broad norm and GELU cases, while retaining narrow-row mapping regressions and weak SwiGLU results. Native and E2E timing remain separate. General packed microkernels and arbitrary Tile redistribution remain missing. Attention, CNN/filter, sort and Top-K PoCs establish correctness, not broad optimized performance.

The Torch CPU inspection has led to indexable snapshots and bounded traversal/private workspace. Those earlier checkpoints improve compilation/code size and complete all 11 large-shape cases, without establishing Torch parity. Local-vector distribution now exists as a bounded opt-in candidate; safe snapshot/phase fusion and independent CPU task grain remain open. The earlier private-vector 120-visit fixed-mapping E2E screen also improves LayerNorm, softmax, SwiGLU and GELU, with non-winning RoPE and narrow-local cases retained. Native-entry and Runtime E2E measurements are reported separately rather than combined into a historical leaderboard.

The LLM negative-result screen measures six direct-SIMD operators at 1.22–16.35× Torch E2E time. The subsequent automatic cooperative mapping removes Metal attention’s whole-program worker fallback: prefill/decode GPU medians are now 80.612/327.875 µs, but still 2.894×/10.132× Torch SDPA in a six-order replay. All 36 outputs pass. Scalar reductions, the second contraction and shared-state traffic remain important structural work; SIMD is unchanged. This broadens a generic mapping family, not a complete calibrated solver or LLM performance parity.

The September 8 composed-reduction follow-up isolates input storage and group width: changing only the two closed collectives gives small descriptive gains. A QK contribution-axis benchmark probe, not a production planner change, regresses at 64 threads but improves at 1024; three 1024-thread decode cases still take 3.80–9.64× Torch GPU time. The study retains its non-interleaved configuration order and one rejected Torch timing row. It motivates phase-specific partition candidates and joint resource/transition costs, not an unconditional subgroup default for mma.

Validation and next milestone

The latest reduction-policy checkpoint completes an independent full build, 35/35 Tile CTests and 110 Python benchmark tests. It preserves strict folds through final machine compilation and admits closed subgroup reductions in composed programs. The original worktree’s separate run remains 34/35 because of an unfinished matrix experiment excluded from this checkpoint. These are correctness results, not new performance measurements. The earlier matrix default-off controls validate 56 complete native/Torch outputs across Metal/CPU, with byte-identical Metal and only bijective TBAA-label changes in CPU LLVM. See validation.

Next: distribute general Tile elements on XIR/SIMD and optimize reductions and accumulator expressions within Metal’s newly admitted composed programs. Preserve scalar-DAG reuse and estimate its live-state/instruction cost, then compare legal fusion/materialization candidates through staged/JIT selection on held-out graphs and shapes. Physical K/reuse, launch resource limits, large-matrix scaling and direct XIR performance remain open. A lower model score or a small cohort win is not completion.

The October 1 Windows CUDA/SIMD checkpoint adds NVIDIA execution and a 22-record strict-FP32 GEMM experiment through the native C++ TIRx/PTX route, experimental CUDA Tile IR, and XIR/SIMD. Native Tile IR has mixed results at 128³ and 512³; large OpenBLAS reference drift and different library submission overheads remain explicit. These bounded host-wall measurements do not establish general backend or library parity. See Runtime integration for route gates and fail-closed behavior.

The execution calculus now separates primitive assumptions, order, reduction laws and typed mapping obligations. Its sibling fusion, joint resource/time solving and distributed plans remain proposals. The literature survey identifies close precedents; neither completeness nor research novelty is established.

Detailed evidence

The benchmark guide defines reproduction and timing modes. Exact commands and binary/source fingerprints identify each experiment, not just its Git revision.