Metal matrix planning and realization

This reference describes the TIRx bridge’s bounded SIMD-group and MPP mapping families, work models and exact finite solver. The independent native Metal emitter has its own capability boundary; it does not automatically share these plans. Measurements belong in compiler-route results.

Automatic composed programs 

PlannerOptions::map_gpu_cooperative_programs defaults to true. After the existing automatic row-reduction attempt, an unbound Metal parallel program containing reassociable MMA may try the existing cooperative group mapper. The admission audit accepts supported Tile effects, compiler temporaries and ordered carries; it does not recognize an attention name or prove the source parallel domain independent again. Each group’s operations still require correct distribution, convergence, storage and barriers.

Explicit execution/resource constraints, unsupported nested parallel scopes, opaque effects and incompatible exact reduction tuning retain their existing behavior. Optional candidate failure does not leak partially constructed plans or erase the valid worker fallback. GroupPlan::automatic_cooperative records this admission route. Disabling planning or this candidate leaves the reference path available for differential tests.

The matrix matcher also projects constant extent-one axes from a supported perfect output loop nest before matching the two nontrivial matrix axes. Buffer rank, address maps and ancestor coordinates remain unchanged. This is an execution reindexing, not a reinterpretation of all rank-four buffers as compact matrices; unsupported nontrivial axes and atom dimensions fall back.

Pipeline planning happens before final group selection. Its capacity estimate therefore reserves space for possible automatic matrix programs, as it does for explicit groups, before allocating extra versions. It sums allocations conservatively; it is not a liveness solver or a calibrated overlap model. Ordered stages remain legal when an additional version would exclude a group.

This broadens candidate coverage, not every operation’s optimized realization. In the measured composed attention program, an expression-initialized second MMA still misses the atom matcher, and reductions remain scalar within each assigned output element. A query extent of one cannot use the current 8×8 matrix atom. General reduction redistribution, shared-state elimination and joint whole-region profitability remain work for the execution calculus and planner. See the paired replay for improvements and remaining losses, rather than assuming group admission establishes parity.

Native MPP experiment: operation scope is not launch size 

The native benchmark_tile_mpp experiment tests a second atom family before adding it to TileIR lowering. It uses Apple’s MPP tensor operations, not an MPS library call. Its tensor_inline arguments are ordinary buffers plus layouts, so the experiment does not justify a new frontend resource type. The result can remain a backend cooperative tensor until its explicit store.

There are two distinct, tunable execution maps:

whole-group collective                 independent subgroup cohort
group                                  group
  all G subgroups -> one BM x BN MMA      (r, c) subgroup -> one TM x TN MMA
                                         BM = Cm * TM, BN = Cn * TN
                                         G = Cm * Cn

logical output coordinate:
  m = (group_m * Cm + r) * TM + i
  n = (group_n * Cn + c) * TN + j

independent memory-view composition:
  A: (m, k) -> base_A + m * stride_A + k
  B: (k, n) -> base_B + k * stride_B + n
  C: (m, n) -> base_C + m * stride_C + n

These maps describe an execution nest followed by three different resource layouts. No buffer determines the hierarchy. In particular, four subgroups computing a collective 64x64 tile are not interchangeable with four independent 32x32 operations, even though both produce 4096 elements with 128 threads. Their synchronization, input reuse, private layout and compiler implementation can differ. Total FLOPs, thread count and accumulator bytes cannot distinguish them in a cost model.

The local macOS 26.5 MPP headers constrain an operation to one SIMD group or all SIMD groups in its threadgroup. For this FP32 family, descriptor M/N must be multiples of eight, with at least one a multiple of sixteen; a static K must be a multiple of sixteen. A dynamic K supports tails. The native probe checks these constraints and the compiled pipeline’s thread limit. Static tensor slices are used only for proved interior tiles; tail groups use bounded dynamic views. An inactive cohort member may skip work only when the operation is subgroup-scoped and its entire subgroup is inactive.

Integration contract:

The backend-local native realization and an optional independent TIRx MPP emitter now exist; see Tile Runtime. TIRx MPP contract v2 has separate typed D=A*B and D=A*B+C operations and fails closed on mixed allocation modes. plan_group now has a separate MPP basis, enumerates legal thread widths and exact rectangular subgroup factorizations, and ranks them with target-specific realization features. The generic Machine TileIR transform and calibrated time/uncertainty model described here are not yet implemented; the native Metal emitter also does not yet consume this TIRx-local planner.

Planner: enumerate the atom implementation, operation participation scope, outer group/cohort factorization, operand-view maps and temporary materializations together. Respect explicit execution and memory bindings; an exec::group constraint is not permission to merge logical groups into subgroups. Broader tile shapes can use ordinary recapture/JIT.
Legality: direct operand-view forwarding requires a load/effect proof. A Tile load is a snapshot; replacing it with an MPP read is illegal if an intervening aliasing store can change the value. Explicit manual stores, user barriers, arithmetic policy and accumulator initialization remain observable. Recognize semantic bodies, never kernel names.
Cost model: key samples by atom family, overwrite/accumulate mode, operation scope, cohort map, dtype/precision, shape, strides, bounds mode, compiler and device. Account for active/inactive groups, full versus edge operations, K extent, buffer reuse and output materialization. For an opaque MPP operation, unknown internal registers, barriers and copy stages stay unknown; do not invent native-8x8 issue counts or port its coefficients to another atom family.
Solver: exhaustive enumeration is sufficient for the small tested family. Apply integer coverage/resource constraints first, shortlist with a model, then JIT and validate. The standalone native probe records GPU and host batch time; the TIRx outer tuner ranks synchronized host-wall samples. Both retain every rejected/failed candidate. Frozen independent replay evaluates a selected plan, not a search minimum.
Emitter: TileIR -> TIRx retains typed operations, regions, effects and views. A supported Metal realization lowers its chosen atom to MPP and constructs inline tensors from buffer/layout expressions. Explicit TIRx MPP compilation now invokes the MPP legality/cost basis in plan_group; automatic cross-family selection among MPP, SIMD-group and scalar atoms is still absent. The independent native emitter has its own capability gate.

The benchmark runner is compare_mpp.py. It separates search from six-order replay and records host and GPU batch times. Inline tensors use a tracked classic Metal queue, as does MPS. The optional tensor handle probe uses a Metal 4 queue with explicit dispatch barriers and commit feedback; Metal 4 does not supply automatic resource hazard tracking. API differences are recorded, not attributed to shader arithmetic.

MPP cost model boundary 

The optional TIRx MPP path now has a separate read-only snapshot-forwarding candidate family. Its order is effect/bounds proof -> snapshot forwarding -> pipeline planning -> group resource/geometry planning -> typed MPP codegen. Full-K global views therefore do not consume imaginary shared A/B allocations. Contract v2 also distinguishes an overwriting MPP multiply from multiply-accumulate. The bridge selects it only for a reassociable standalone positive-zero MMA or a closed positive-zero direct-output recurrence with one static iteration; nonzero/negative-zero C, multiple K steps and observable carry state retain accumulation. Mode is therefore a derived legal-realization feature, not a name-based peephole or a user-visible DSL primitive.

After these proofs, the current bridge selects a separately versioned metal_mpp_memory_v2 bootstrap model instead of scoring MPP candidates with the SIMD-group reference formula. For G participating subgroups, P logical programs and target-profile subgroup capacity Q, it computes:

issue = weighted(MMA issues, MPP operations, fragment reads, A/B footprints,
                 accumulator initialization, output traffic, aspect terms)
program_score = issue * state_pressure / G
              + independent_elements * element_weight / G
              + group_setup
waves        = max(1, P * G / Q)
kernel_score = program_score * waves

The division by G is important: disjoint subgroup tensor operations are a concurrent program critical path, not serial work. The outer wave factor then prices whole-device subgroup demand once. Q=512 is currently an M1-class replaceable prior, not queried occupancy, and fractional waves are deliberately a smooth ranking heuristic. All quantities and the selected score are emitted in the compilation/benchmark report.

Cost-model v2 narrows a legal candidate set; complete-output validation and measured selection remain the authority.

Legality remains independent of this score. Exact coverage, target thread and shared-memory limits, fragment/code-size bounds, and MPP’s requirement that each local matrix have M or N divisible by 16 are hard rejections. A coefficient cannot rescue an illegal descriptor. The benchmark keeps the old TIRx and staged MPP controls and selects/replays forwarding candidates separately. A measured choice belongs to the specialized configuration, not to a hard-coded shape dispatch in the bridge.

Fragment budgets belong to the emitted realization 

For a subgroup owning an rm x rn rectangle of 8x8 atoms, the compiler-explicit logical fragment state per lane is:

Realization	Explicit tensor state	Scalars per lane (32 lanes)
SIMD-group reference	A fragments, B fragments, accumulator	`2 * (rm*rn + rm + rn)`
MPP memory operands	Output cooperative tensor	`2 * rm*rn`

The candidate gate now uses the same realization-specific accounting as the scorer. For example, a 32x64 local output requires 64 logical MPP scalars/lane, but 88 in the reference emitter. Charging MPP for reference-only A/B fragments incorrectly discarded this candidate at the unchanged 64-scalar budget. The minimum explicit budget is also realization-specific: the smallest MPP 8x16 descriptor needs four output scalars/lane, whereas a reference 8x8 atom needs six for A/B/C. A budget below the corresponding minimum still rejects. This applies equally to staged and direct-global MPP inputs: their backing resource remains separately charged against shared-memory limits where needed.

This is a compiler-state/code-size budget, not a physical register count or occupancy guarantee. MPP may internally materialize input data, address state or spills. Hardware capacity, legal descriptor shape, exact coverage and the existing pressure prior remain independent checks. Admitting a larger output does not imply that the default score selects it or that its performance wins; the state-budget experiment keeps those questions separate.

Bounded K views avoid nominal padding storage 

The optional metal-mpp-bounded-k-v1.patch, applied after MPP memory v2, adds a separately checked capability without changing the DSL or TileIR. It distinguishes the logical contraction tile BK from the physical input interval actual_k = min(BK, source_k - origin_k). With this capability alone, M/N have to be fully in bounds; it is not a general masked tensor atom.

logical A[BM, BK] / B[BK, BN]
              |
 immutable + noalias + dominance
              |
 canonical zero fill + unit K map
              |
equal positive actual_k; full M/N
              |
 A[BM, actual_k] / B[actual_k, BN]
              |
     MPP / pipeline / store

missing proof -> strict snapshots

The matcher derives this from native expression identities and enclosing execution domains, not kernel names. It omits only a common zero×zero suffix; a one-sided mask could multiply zero by an unmasked value and is not interchangeable. A/B transposes retain their descriptor flags and physical leading strides. The backend receives a positive signed actual-K extent in both inline tensors; the nominal M/N/K and cooperative accumulator contract remain unchanged.

Guarded forwarding is transactional. Every reassociable MMA in the candidate must still receive a verified atom plan; otherwise the compiler retries the original strict path, including its usual resource errors. Missing TVM capability, M/N tails, additional masks, nonzero fills and unequal A/B K intervals do not create a scalar fallback disguised as a successful MPP view plan. Existing noalias, mutation, escape, manual-memory and recurrence gates remain in force.

For BM=128, BN=32, BK=1024, this can remove the nominal 640 KiB A/B staging requirement even when the final contraction is short. The existing resource solver sees the resulting physical allocations. The MPP v3 work score below also uses a proved mean physical K when available; unknown expressions retain nominal K. Neither fact establishes model accuracy or MPS parity. M/N edges require the separate capability below; automatic physical K retiming remains separate work. The patch ABI and build order are documented in the TVM patch README.

Bounded M/N views compose with subgroup coordinates 

The optional metal-mpp-bounded-mnk-v1.patch extends the same proof to M/N edges, including wholly empty subgroup rectangles. It keeps the logical MMA shape and execution distribution unchanged: ancestor program origins compose with the existing subgroup rectangle, then constrain the physical input view. For example, actual_m = max(0, min(local_m, source_m - program_m - subgroup_m)); N is analogous, and A/B still share one positive actual-K interval.

This is resource/execution separation, not a new nest or a memory-defined worker hierarchy. Canonical positive-zero guards, affine unit projections, immutability, noalias and dominance remain required. A unit projection also determines orientation when physical dimensions of one make numerical strides ambiguous. For an absent operand, address construction selects the valid buffer base instead of forming an out-of-object pointer.

Positive actual M/N use MPP’s bounded inline tensors. Zero extents need a different implementation: the validated SDK probe read a sentinel row from the naive zero-extent encoding. The emitter therefore handles an empty input inside the existing subgroup’s cooperative output fragments. Only valid elements of the other operand are read; FP32 bit classification preserves 0 * Inf, NaN and signed-zero accumulation. Each lane owns a row/column of that operand; shuffle transfers its classification to each public output coordinate. No opaque MPP distribution is assumed, and collectives stay outside divergent validity predicates. Interior rectangles retain static M/N tensor views. The nominal destination and nonzero C remain observable. Empty does not mean permission to skip the output or silently assume finite values. No A/B staging allocation is needed.

The separately versioned capability retains both older ABIs. Without it the bridge keeps the previous M/N-tail fallback. The empty-rectangle scan is a correctness realization, not a calibrated performance model: the solver still scores nominal M/N work, and default-off forwarding remains opt-in. Noncanonical masks, nonzero fill, unequal K intervals, mutable inputs and manual memory retain snapshots. See the performance checkpoint for actual timings rather than inferring speed from removed storage.

Output bounds are independent of input padding 

The optional bounded-store contract extends the existing direct accumulator realization to guarded output prefixes. The key distinction is three independent access maps, one execution distribution: the valid A/B input rectangle does not determine which C elements the program stores. A zero-padded input can still produce an observable initializer, NaN or signed zero.

For logical output-axis extent T, nonnegative program origin p, physical destination extent D, subgroup origin s and local subgroup extent t, the proved output prefix is:

logical valid length = clamp(D - p, 0, T)
subgroup store length = clamp(logical valid length - s, 0, t)

Each physical axis must have a unit projection onto exactly one logical output axis; a transpose permutes these projections. The sink guard must be equivalent to the memory bounds throughout the ancestor execution domain. Bidirectional matching of nontrivial conjunction clauses handles reordered or reassociated bounds without admitting extra masks. Empty prefixes select a valid buffer-base pointer and perform no writes. Full row-major interiors keep MPP’s bulk store; partial/column-major outputs use only the cooperative tensor’s public coordinates and the independently proved memory stride.

Closed recurrence, original sink position, literal initialization and manual resource/observation checks remain unchanged. For example, a snapshot of the old output must survive even when C itself no longer needs shared backing. Missing capability, negative origins, arbitrary masks, observed accumulators and unproved layouts retain the previous realization.

The existing planner consumes this legality fact through has_direct_output: it accounts for released storage and may derive one-shot overwrite mode. There is no new kernel-name rule, geometry table, DSL concept, solver or cost coefficient. This is not yet a general schedule search: the bounded-store work and edge fraction are not separately calibrated, and the independent native Metal emitter/XIR route do not automatically acquire this TIRx extension. See the patch contract and build order.

Scalar epilogues use the same element owner 

The optional MPP element-v1 extension and PlannerOptions::fuse_matrix_epilogues generalize the direct-output proof from a bare accumulator copy to a closed scalar expression DAG. It recognizes typed IR and access identities, not kernel/activation names or dimensions. For example, the same rules admit a clamp, a polynomial, or tanh-based GELU:

                           same logical (row, column)
MMA recurrence ──► scalar producer ──► scalar consumer ──► global sink
      │              compiler-owned shared tiles               │
      └──────────────── proof + planner choice ────────────────┘
                              ↓
cooperative fragment ──► local element bindings ──► cooperative store
                         capacity / validity       original output map

Each producer must be a uniquely defined, compiler-owned materialization with the same rectangular domain. Each read must address the exact same element of the accumulator or a dominating producer. Pure scalar expressions remain native TIRx expressions; shared producer values become Bind definitions. The backend’s optimizer may further simplify them. A whole-group use audit proves the carry and removed temporaries have no other observations/escapes. The global store remains at its original position, after any old-output read.

MPP owns the lane layout. The emitter iterates its public per-thread capacity, checks local-element validity, applies the scalar DAG and stores through the already proved output projection. It does not infer a matrix coordinate from a lane ordinal. This follows the cooperative-tensor element access described in Apple’s MPP programming guide.

The existing solver receives additional legal direct-output candidates and their exact released temporary bytes. Scalar producer/sink work remains in the independent-element cost proxy, even though shared storage disappears. This is a realization/resource-accounting extension, not a fitted cost-model improvement. The proxy does not model transcendental instruction cost or backend register allocation precisely. The option defaults off: measured JIT selection may explore it, but legality and lower storage do not establish a speedup. Early screen results include regressions, so it is not promoted to the default path.

Manual memory, unmarked producers, transposed/neighbor reads, additional memory operands, free variables and extra consumers retain the old path. Pure does not imply same-owner access. This first contract is closed over the accumulator and literals; bias/residual tensors require a future proof of congruent cooperative layouts. CPU/SIMD and native-MPP do not automatically inherit this TIRx-specific emission, and old TVM builds retain the old output path through the optional capability gate.

Physical program traversal remains a candidate 

Program-grid traversal is another execution-layout choice, independent of memory layout. It composes before the already planned subgroup/local map:

physical program ordinal
          |
  bounded grid permutation
          |
logical program coordinate
          |
  subgroup + local coordinate
          |
     operand address map

parallel instances already have independent execution semantics. A candidate permutation needs an in-domain bijection, not a new dependence proof. It must preserve every program exactly once, including partial grid rectangles, and does not promise the hardware scheduler’s actual execution order. Memory views and nested scope/resource ownership remain unchanged.

The standalone MPP benchmark can now enumerate bounded row/column rectangles with no padded programs. It checks uint32 launch arithmetic before allocation; small/tail GPU outputs and host enumeration validate the mapping. The TIRx bridge now also implements the generic group-program traversal as an explicit JIT constraint, composed after local resource/matrix planning. Neither the automatic TIRx planner nor native MPP selects a new default traversal. TIRx’s cooperative mapper already uses a linear physical grid; the hand benchmark’s legacy-2D versus linear comparison is not a new bridge optimization. The new bounded rectangles are not a Morton-order traversal.

The two-order diagnostics reject a generic preference for larger row groups. For a 128×32 group output, a 4×1 program stripe is a 512×32 output region, not a square reuse region. Square-region candidates were also inconclusive under substantial timing variation. A future backend policy should consider program aspect, operand access maps and K partition jointly; the current per-group MPP footprint model does not measure cross-group cache reuse or actual concurrent occupancy. Keep those as candidate/model extensions pending stable held-out evidence, without adding a DSL primitive or assigning a hardware memory level to a nest.

Implemented matrix mapping family 

The current planner targets a proved Metal group-level FP32 MMA with a complete 32-lane subgroup and an 8x8 atom. Let:

U = M / 8, V = N / 8, Q = K / 8
G = threads_per_group / 32
gm * gn = G
gm * rm = U
gn * rn = V

All quantities are positive integers. gm, gn distribute the atom grid between subgroups; rm, rn specify the resident atom rectangle inside each subgroup. The remaining candidate is the one-atom-at-a-time reference, which supports uniform subgroup job tails. An incompatible exact thread constraint uses the checked scalar reference instead of partially participating matrix operations.

For subgroup ordinal s and local fragment ordinal f:

atom_m = floor(s / gn) * rm + floor(f / rn)
atom_n = (s mod gn) * rn + (f mod rn)

s = floor(atom_m / rm) * gn + floor(atom_n / rn)
f = (atom_m mod rm) * rn + (atom_n mod rn)

The factor equalities prove exact coverage and the two directions agree on the finite domain. This is a mixed-radix layout, not a separate distribution algebra. The forward map exports a native C++ TIRx TileLayout with shard extents (gm, rm, gn, rn) and physical contributions (gn*warpid, rn*m, warpid, m). The inverse is a core Tile IndexMap, lowered to native coordinate arithmetic. TIRx regrouping may remove unit-extent shards; zero offsets retain both named output coordinates in degenerate cases.

Here native m names a fragment ordinal, not a byte address. An 8x8 atom’s internal lane/register layout remains its hardware contract. The access path is still composition:

(ancestor program, subgroup, local fragment, atom operand coordinate)
  -> logical tile coordinate -> view coordinate -> resource address

A, B, and the accumulator therefore share the execution plan without sharing a memory layout. No mma_team or new frontend scope is required.

Realization changes 

The contraction loop surrounds the resident fragment rectangle. Per K atom, each subgroup loads rm A fragments and rn B fragments, then performs rm*rn matrix updates. A/B inputs are reused rather than reloaded once per output fragment.

A closed recurrence can additionally retain the accumulator across iterations:

before loop: load initial C into native fragments
loop:        load A/B; update fragments
after loop:  publish final fragments to C

Promotion requires one directly recognized MMA and its full D -> C carry update, one local D allocation, and no other observation of C or D in that loop body. Reading the old accumulator into another carry, an explicit memory effect that observes it, or an unsupported annotated/control-flow body prevents promotion. In particular, C must not also be an A/B multiplicand: the next iteration would otherwise read stale shared C while the updated value exists only in native fragments. A break, continue, or return between the MMA and its carry update also prevents promotion. Literal initialization, zero-iteration behavior, transposes, tails, and the nonresident fallback have regressions. This baseline eliminates D but retains initial/final C storage.

This recognition currently applies to closed flat serial loops, including the window-1 lowering. Window-2 software-pipeline prologue/steady/drain structure generally retains the reference storage behavior. Joint pipeline/residency planning is future work, not a promise made by this transformation.

Direct accumulator output 

The resource plan can also remove C when its complete lifetime is recognized:

source:    fill C; [load A/B -> MMA -> yield C]*; ...; store output <- C
realized:  fill CF; [load A/B -> update CF]*;     ...; matrix-store output <- CF

Here CF is native fragment state, not a new DSL value kind. The global write stays at the original output statement, including when intervening code reads the old output. The transformation does not infer that different external buffers cannot alias, nor does it move a global write across another effect.

Eligibility requires an unannotated compiler-owned C allocation, one complete FP32 literal fill, the closed recurrence, and exactly one complete C-to-global copy. A whole-group use audit rejects any other observation or pointer escape; SeqStmt grouping is not treated as a lifetime boundary. An explicit manual resource annotation prevents allocation removal. The output must have a positive compact affine row/column-major projection. Every output guard and every destination coordinate must be proved valid over the ancestor execution domains and the local matrix domain. The query uses TVMx’s native C++ StmtSimplify pass; unknown bounds keep the shared/guarded reference path.

PlannerOptions::direct_accumulator_store independently disables this choice. MatrixWorkload::has_direct_output is an analysis fact, not a profitability hint. The selected MatrixDistribution reports whether it uses that proof. Both C and D can then be removed from the shared budget. Padded/offset and transposed destinations, ragged output fallback, an extra accumulator consumer, and a read of old output before the sink have full-output numerical tests. MMA-operand aliasing and interrupted carry updates are native-IR counterexamples, so incidental frontend value copies cannot conceal an unsafe residency proof. The pinned TVMx LLVM emitter rejects native Break/Continue; CPU tests check that specific rejection, while Metal executes those two control-flow oracles.

Implemented relative-work models 

ExecutionCostModel contains two separately reported bases. The selected GroupPlan::cost_basis prevents MPP features from being mislabeled as SIMD-group work. Both are inspectable priors, not nanosecond predictors.

SIMD-group reference basis 

For an MMA executed E times, and a proved resident recurrence of length L:

matrix issues I = U * V * Q * E

input transfers = G * (rm + rn) * Q * E       [rectangle]
                = 2 * U * V * Q * E          [reference]

accumulator transfers = 2 * U * V * E        [nonresident]
                      = 2 * U * V * E / L    [proved resident]
                      = 0                    [proved direct output]

direct global stores  = U * V * E / L        [proved direct output]

live fragment scalars/lane R = 2 * (rm*rn + rm + rn)
                            = 6              [reference]

work     = I * matrix_issue
           + (input transfers + accumulator transfers + direct global stores)
             * shared_fragment_transfer
pressure = max(1, R / preferred_fragment_scalars_per_lane)
parallel = max(1, min(preferred_subgroups, U*V) / G)
score    = sum(work * pressure * parallel)
           + independent_elements * independent_element
           + G * subgroup_setup

Transfers count subgroup fragment operations, not unique DRAM bytes. Fragment scalar counts are live logical state, not compiler register allocations. Direct global stores are reported separately and priced as fragment transfers, even when the corresponding scalar sink disappears. This bootstrap does not yet assign calibrated prices to different output protocols. The candidate’s proved-eliminated scalar loops are excluded as described below. In this reference basis, remaining ordinary independent-element work does not predict the effect of thread count on copy throughput. Logical program count uses a coarse preferred-program wave prior. The pressure/parallelism factors are explicit heuristics with replaceable coefficients. General bank-conflict, spill, repartition, launch, and pipeline-overlap models are not implemented here.

For a 32x64 output tile, K tile 32, and 32 temporal iterations, one legal 128-thread plan uses a 1x4 subgroup grid and 4x2 local fragments. It changes the accounting as follows (these are derived work counts, not timing claims). The last column additionally requires the literal-fill/full-output proof above:

Quantity	One-atom reference	4x2 resident, shared output	4x2 resident, direct output
Matrix atom issues	4096	4096	4096
A/B fragment transfers	8192	3072	3072
C/D shared fragment transfers	2048	64	0
Direct global fragment stores	0	0	32
Live fragment scalars/lane	6	28	28
Compact shared allocation	28 KiB	20 KiB	12 KiB

The tradeoff is explicit: fewer repeated transfers and less shared storage, but more live per-subgroup state. A model must eventually price both using target evidence. Counting static matrix call sites in generated source is not counting dynamic instruction work.

Metal MPP memory v3 basis 

MPP memory-input operations do not have the reference realization’s shared A/B fragment copies. The v3 feature vector records:

Feature	Meaning and limitation
`matrix_issues`	Logical 8×8 result atoms × mean physical K / 8 × executions; normalized work, not measured instruction issues or hardware padding
`nominal_matrix_issues`	The corresponding nominal-K work; remains available for diagnosing padding and unknown extents
`metal_mpp_operations`	Participating subgroup tensor operations × executions
`memory_fragment_reads`	Per-subgroup A/B memory fragment requests; separate from unique footprints
`lhs/rhs_footprint_fragments`	Unique logical A/B footprints with asymmetric reuse priors, not claimed DRAM transactions
`accumulator_initializations`	Absent only when the overwrite proof selects `D=A*B`
output/shared transfers	Derived from direct-output and persistent-accumulator proofs
Tile/local aspect terms	Target-versioned rectangle priors; not layout-equivalence claims
`fragment_scalars_per_lane`	Opaque-output logical live state, not measured registers
`independent_elements`	Remaining scalar address/guard/store work outside the matrix atom
`elided_independent_elements`	Disjoint source-domain elements removed by the selected recurrence/direct-output realization

For one group realization, the code computes:

issue = sum(feature[i] * coefficient[i])
state_pressure = max(1, fragment_scalars_per_lane / preferred_fragment_state)
score = issue * state_pressure / subgroups
      + independent_elements * element_weight / subgroups
      + group_setup
waves = max(1, logical_programs * subgroups / concurrent_subgroup_prior)
kernel_score = score * waves

The solver compares kernel_score across thread widths. The subgroup division models disjoint MPP operations on the program critical path; waves prices outer machine demand once. The default concurrent-subgroup prior is 512 for the tested M1-class profile. It is deliberately replaceable and fractional—neither a target query nor a claim about physical residency boundaries.

The historical v1→v2 study shows why both terms matter and retains every invalid candidate. The v2 score reduces in-cohort mean regret from 74.18% to 8.82%, but the 34.37% maximum miss and absence of held-out data keep measured Staged/JIT ranking authoritative.

Realization-derived work, before candidate pruning 

The v3 correction changes work features, not fitted coefficients or numerical permissions. The matrix matcher supplies the same bounded K expression used by the MPP emitter. Over positive, static, unit-step rectangular ancestor domains, the analyzer computes the mean of a constant or a one-varying-axis affine expression, optionally capped by a positive constant. It sums a capped arithmetic progression analytically, including either traversal direction. It preserves the cap before simplifying its operand because equivalent TVM canonical forms can move min into a subtraction of max. Dynamic domains, unsupported nonlinear/multi-axis expressions and unproved endpoints retain nominal K. This is a cost fact, never authority to omit a load or computation.

The recurrence proof also identifies disjoint counted scalar domains: its yield copy, and the literal initializer plus scalar output sink. A candidate removes the first subset only when it retains the accumulator, and the second only when it realizes direct output. A direct-only overwrite proof cannot remove initialization cost when the candidate still loads shared C.

IR semantics + ownership/bounds proofs
                  |
matrix work + optional scalar domains
                  |
candidate realization
(retained / direct / bounded input)
                  |
remaining work + physical resources
                  |
score -> Pareto -> group plan -> emit

Candidate-dependent scalar work is priced before Pareto pruning. Common work is added once afterward; subtracting it only after selecting a frontier can discard the true minimum. The public workload rejects subset totals that exceed the group count without overflowing; the internal proof supplies operation identities to establish actual disjointness. Tests independently enumerate two-operation objectives under different scalar prices and shared-capacity constraints.

For a 32×64 tile with three nominal K=16 iterations spanning physical K=45, the MPP model records 192 nominal versus 180 physical-equivalent matrix atoms. A direct resident candidate removes 10,240 source scalar elements; a resident candidate retaining its scalar sink removes only the 6,144-element yield copy. Global stores remain charged. These are exact lookup examples of the analysis, not latency predictions.

This mechanism is structural within the admitted matrix family, with no operator-name or benchmark-shape dispatch. It does not yet model M/N edge fractions, hardware K rounding, internal MPP registers/spills or cache traffic. It does not automatically improve native Metal, XIR/SIMD, or non-matrix operators. Performance evidence is kept separate from the analysis contract.

Cooperative copy batching 

PlannerOptions::max_copy_batch optionally groups up to 16 independent values per worker: emit their reads/computation into native TIRx bindings, then their stores. One is the default reference sequence. This is an instruction-level parallelism option, not an asynchronous transfer or a vector-alignment claim. It does not change the worker-to-element map, move a stage, or remove a fence.

The emitter only batches independent domains writing a compiler-owned shared temporary, with no destination read/modify/write, conditional store, or opaque effect. Short-circuit bounded loads retain their predicates. Only full worker chunks are batched; the remainder uses the original guarded path. Reports include the requested maximum and the number of actually batched operations. The benchmark/replay driver preserves the option as --copy-batch N.

This choice is currently explicit, not included in the matrix model’s search or calibrated score. Four-round same-binary measurements at 64x64x32 tiles show substantial improvements on two ragged GEMMs, but essentially no improvement on 512-cubed or 1024-cubed. No universal speedup or default change follows from those observations. The copy-plan report keeps all shapes, timing distributions, and correctness checks.

Dependence-aware group synchronization 

After resource/distribution realization, the bridge can coalesce its own group barriers across independent effects. For example, two input copies to distinct shared buffers can publish together before their matrix consumer:

before:  copy A -> As; fence; copy B -> Bs; fence; MMA; fence
after:   copy A -> As;        copy B -> Bs; fence; MMA; fence

No copy, memory effect, or participant mapping moves. The first fence is unnecessary only if its removal leaves every publication/order dependence covered. At a candidate cut, let P be all effects since the last retained fence and S the next segment. Coalescing requires neither side to be opaque, W(P) disjoint from R(S) and W(S), and R(P) disjoint from W(S). The implementation accumulates P after a removal; checking just the adjacent operation would incorrectly accept write A; unrelated B; read A.

Only the mapper’s fresh unplaced shared allocations have distinct alias classes. All external global buffers share one conservative class, even when their parameter identities or constness differ. Unknown storage, calls, explicit synchronization, and control exits remain hard boundaries. The pass recognizes compiler fences by their emission-local IR identity, not an opcode or external symbol name; an explicit identical-looking native barrier stays. In the general effect analysis, the last fence of a sequential region is retained for the loop backedge or enclosing consumer. Barriers never move across loops/branches, and their full shared-plus-device fence semantics are unchanged. Resource reuse added by a later transformation must invalidate/recheck this synchronization plan.

PlannerOptions::coalesce_group_barriers disables this pass independently; disabling the planner also retains the reference fences. GroupPlan reports group_barrier_sites_before/after: static sites, not dynamic executions or a latency estimate. These post-realization facts do not yet contribute calibrated barrier costs to the bootstrap score. CPU/Metal regressions cover nonadjacent dependencies, non-subgroup-multiple worker counts, aliased global parameters, and aliased output read on the next pipeline iteration. A Metal native-IR test also distinguishes an explicit barrier from compiler-owned barriers.

The M1 Max synchronization report holds the tile, worker count, fragment layout, and resource realization fixed. Four counterbalanced rounds show modest, shape-dependent gains, including one 1024-cubed regression. Fewer barrier sites are not a proportional latency model and do not explain the remaining large-square gap to PyTorch.

Independent subgroup programs: legality is not profitability 

There is now a stricter whole-group proof for the opt-in TIRx MPP read-only view realization. The matrix mapper supplies emission-local statement identities for private cooperative-tensor initialization, synchronous MPP steps and a verified partitioned output. The view analysis separately proves the exact A/B parameter identities immutable under the caller’s noalias contract. A scope name, zero shared-memory usage or metal_mpp=true alone is not enough.

The entire group must contain only these operations, uniform constant-bound serial loops, no-ops and compiler-owned fences, followed by exactly one output store outside all loops. A second global effect, a post-store consumer, an output on a loop backedge, shared storage, branches, escapes, explicit fences or unknown statements rejects the proof. Full subgroup participation and the nonoverlapping output rectangles come from the matrix distribution verifier. No instruction or memory access moves when the fences are removed.

realized body + immutable-input / partition / participation facts
                             |
                   whole-group isolation proof
                    /                       \
                 fails                     succeeds
                   |                          |
       general conservative coalescing   legal choices: retain | elide
                                              |
                                  explicit planner/JIT choice
                                  (retain is the default)

GroupPlan::independent_subgroups records the proof independently of the profitability choice. The elide_independent_subgroup_barriers option in PlannerOptions is default-off and also requires the planner and coalesce_group_barriers enabled. Selecting it never supplies missing proof facts. The fixed-geometry A/B replay is a counterexample to pricing each removed fence as a guaranteed benefit: 512³ got slower in all four rounds, despite six interior cases having no cross-subgroup communication. Apple’s MPP programming guide, §2.3.4 also describes periodic group barriers as a cache-working-set tuning tool. That is a possible mechanism, not a measured diagnosis of this M1 result.

Read-only snapshot forwarding now reaches a fixed point. Axis relabeling can produce input -> snapshot -> relabeled snapshot -> MMA; each round rechecks whole-function effects, bounds, dominance and nonescape, then removes at least one unique compiler allocation. This finite iteration handles anonymous shape axes without asking the DSL author to reuse axis objects merely to avoid storage. It does not remove manual Memory or weaken any snapshot contract.

Implemented solver: enumeration plus Pareto dynamic programming 

For the currently small target thread bound, enumerate every complete-subgroup thread count (or honor one exact requested count). For each MMA enumerate integer subgroup factorizations, derive local factors by division, and reject nonexact coverage or excess compiler fragment/code-size budget before scoring. The one-atom reference is always included. Eligible rectangular candidates use accumulator residency unless that optimization is disabled. When the additional direct-output proof is present and enabled, they also eliminate C and its shared transfers. The current model prefers this choice; independent measurements must still validate its real runtime effect.

Several operations share one group’s resource limit, so independent greedy selection is insufficient. At a fixed thread count, dynamic programming keeps a frontier over:

(selected realizations, summed score, released shared bytes)

A partial choice dominates another only if its score is no larger and its released bytes are no smaller. Remaining operation alternatives contribute independently to these quantities, so dominated states cannot be required by an optimal completion. Reject final combinations exceeding shared capacity, then select the lowest-score surviving combination across thread counts.

This is exact for the supplied additive model and enumerated family. It does not prove a global runtime optimum, and the dominance rule must change if later costs include interactions or overlapping live ranges. Frontier state must then carry those interactions instead of discarding them. The current tests include a case where a slower resident realization is the only capacity-feasible choice, both for one operation and for two coupled operations.

The solver has no mutable global state or timing calls. Changing coefficients cannot make an illegal map legal. CompileOptions::planner controls the model, exact thread constraint, residency choice, and fragment search budget; CompilationResult::plans() reports the selected distributions, resource footprint, work counts, and search counters. Benchmark JSON includes these reports under execution_plans.

Thread capacity and search budget are separate: the reference launch width of 256 is not a hardware cap. An automatic search exceeding max_thread_candidates returns a diagnostic instead of silently searching a prefix and claiming exactness. An exact thread request needs only one width; subgroup divisors are enumerated in square-root time, with deterministic order.