Tile architecture decisions

Architecture decision ledger

This table is the compact answer to the design discussion. Detailed syntax and proofs live in the language reference; the compiler index and implementation coverage separate realized behavior from the remaining design.

Architecture decisions

Question	Decision	Consequence
Programming model	Execution structure first, not an algorithm graph with a schedule attached afterwards	Lexical Nest structure exists before target mapping; operations inherit anchor/frontier from it
Core Nest vocabulary	parallel, serial, pipeline, reduce only	Elementwise, convolution, softmax, attention, Top-K and sort remain composable libraries unless irreducible target semantics justify an atom
C++ surface	Luisa-style staged lambda parameters, range-for Nests, ordinary carried assignment, explicit memory effects	No GemmSpec, public builder prefix, symbolic-integer façade, loop.result() or kernel-specific mma_team entity
Tensor and view	Tensor is storage plus a typed layout/view; A[...] loads a Tile, A(...) names a MemoryRef	Subtiles and bounds are explicit without baking execution or a memory hierarchy into Tensor
Layout algebra	One typed mixed-radix/index-map composition algebra for execution binding, value distribution, views, addresses and atom operands	Domain/codomain and proof obligations prevent composing unrelated coordinate spaces; representability is broader than any one emitter
Execution versus memory	Execution hierarchy chooses participants; resources attach independently to an owner prefix and access map	Several differently laid-out memories may serve one Nest; memory kinds are capabilities, not a fake total hierarchy
Pipeline	A temporal producer/consumer Nest with lexical stage cuts and dependence distances	It may organize participant specialization, overlap and versions, but a stage name alone does not promise async hardware
Reduction	One aggregation Nest; unordered_tree by default, explicit ordered-tree or left/right-fold restrictions	Tree permission needs a compatible merge, not a new primitive or a backend-specific numerical switch; strict folds do not require merge laws
Tile SSA versus Memory	Preserve semantic sharing; plan retain/recompute/materialize per target. Manual Memory means stable addressable identity	Compiler stripes/registers/workspace do not leak into ordinary kernels; manual writes always use .store()
TileIR	Thin, typed, mutable SSA/region IR with managed intrusive ownership/use lists and analyses	It is transformable like XIR/LLVM, not a SPIR-V-style serialization schema and not an MLIR dependency
Backend boundary	Public tile::compile(device, TileIR) calls the optional backend DeviceInterface::create_tile_kernel factory, which selects native lowering or tile/bridge/{tirx,xir}	TIRx and XIR remain comparable bootstrap paths while Metal/CUDA/CPU keep target-specific bindings and atoms
Planning	Solve binding B, distribution D, atom A, resources R and schedule Theta under hard proofs, then rank	Enumeration/Pareto DP are implemented for bounded families; MILP, CP-SAT, beam or annealing are optional search engines, never legality oracles
Autotuning	Ordinary concrete host configurations are recaptured and JIT-compiled as a finite product	No capture-once super-kernel is required; every candidate and the fresh winner receive the full correctness oracle
Machine TileIR	Add it only when several backends/passes need a common scheduled atom/resource/protocol form	Current bridge-local plans remain honest stepping stones; no premature backend instruction serialization

Apply Occam’s razor to new core entities, especially execution-nest primitives. A proposal must show a concrete workload whose required semantics cannot be expressed by the existing primitives, or an unavoidable efficiency loss imposed by those semantics after legal scheduling and lowering. A missing optimization or a slow backend realization alone does not establish that need. First reuse the existing primitives through analysis, cost models, scheduling, and target lowering; width, packing, unrolling, and temporary storage are such choices.

“Layout completeness” therefore has three separate meanings. The algebra is closed over the admitted typed finite maps and can embed the CuTe-style mixed-radix constructions used here; proof procedures intentionally return unknown outside their decidable fragments; emitters support smaller target subsets and fail closed. A complete representation never licenses an unsupported lowering.

Design review checkpoint: September 7, 2026

The detailed Chinese research and implementation review informs the following decisions. It does not replace the language reference or constitute a completed proof. Execution-first means a flexible structured semantic model, with the compiler solving its realizable space/time resource mapping; it does not freeze the source tree into hardware levels.

The documentation checkpoint resolves the reduction-policy contradiction and separates implemented IR from extension contracts. The chosen source default is unordered tree; at that checkpoint the implementation still used a backend opt-in flag. Changing that behavior needs an explicit implementation migration, not a new cost coefficient. This checkpoint changes documentation only: it does not complete or certify the existing uncommitted compiler changes, and it adds no kernel performance measurement.

Next implementation milestones and acceptance criteria

Next milestone	Acceptance evidence
Per-operation reduction semantics	Capture resolves the default and explicit restrictions; all bridges preserve them; mixed-policy and nonassociative fold tests
Checked realization interfaces	Contribution coverage, boundary values/distributions, participants and ready/release obligations survive rewriting and emission
Broader candidate families	Equivalent source variants retain supported matrix/reduction candidates; within-Tile SIMD decomposition is represented, not just rescored
Compositional planning	Original/materialized/resident alternatives compete with explicit conversion and peak-live-resource costs; small cases checked against bounded enumeration
Calibrated generalization	Held-out shapes/operators, equal numerical permissions and budgets, device versus dispatch timing, regressions and regret reported separately

The semantic counterexample set must include: left/right subtraction folds; associative but noncommutative merges; FP32-sensitive regrouping; nonidentity seeds; empty/masked domains; NaN/signed-zero/ties; mixed strict/default reductions; replicated layouts; and one producer with two in-flight consumers sharing a ring slot. These are planned acceptance criteria, not a list of tests already passed. General Machine TileIR, sibling fusion and asynchronous protocols remain extensions rather than prerequisites for every initial compiler change.

Reduction implementation follow-up: September 7, 2026

The first milestone now has a typed implementation: ReductionPolicy is resolved on each REDUCE, with an unordered-tree default and explicit ordered tree / left-fold / right-fold policies. TIRx and XIR preserve the restrictions; the current collective and array-provider families require unordered-tree permission. Ordered trees and general folds retain serial realizations.

Final compilation is part of this contract. SIMD and Metal Runtime disable kernel-wide fast math when order matters, and the raw TIRx LLVM target cannot override it with global fast-math flags. DeviceArtifact carries the required arithmetic mode beside its source/ABI. TVM’s own Metal runtime requires the optional native precise-math extension; missing support fails explicitly. This does not disable legal collectives for another unordered operation.

Composed-group analysis now accounts for complete-subgroup participation of an admitted reduction output. This repairs a reference-binding gap; it is not a calibrated mixed-phase cost model. General merge-law checking, ordered-tree emission and the broader semantic counterexample set above remain incomplete. See the Chinese review’s implementation follow-up and coverage for remaining work. No new kernel-performance measurement accompanies this semantic migration.

Original design checklist

The following preserves the revision-17 design rationale and bootstrap order. It is historical design material, not the current task list or a completion checklist. Use implementation coverage and remaining work for current status.

Minimal implementation plan

Phase A: algebra and IR skeleton

• Dim, Space, ExecNest, ParallelMap/TemporalMap, and interned LayoutMap DAG;

• stable ExecLevel/prefix identities and typed ExecRemap transactions;

• IndexSet, LayoutCorr, mixed-radix, composition, product, projection, permutation, translation;

• Region/Block/Operation/Value/Use and rewriter;

• parser/printer only for debugging and tests;

• tri-state proof API, layout normalization, algebra-law property tests, and exhaustive finite verifier.

Phase B: elegant C++ capture

• scoped builder stack;

• tile_kernel, TensorView, Tile, Memory, and MemorySSA state capture;

• one-pass range-for parallel / ExecScope::parallel / serial / pipeline / reduce, stage cursor cuts lowered to pipeline subregions, inferred outer reduction states/contracts, view/load/store, lifted elementwise ops, semantic mma, and expression-reduce shorthand;

• a header-only reference Tile library for matmul/einsum, convolution, softmax, normalization, Top-K, sort, scan, gather/scatter, and copy;

• direct assignment capture with specified copy/move semantics;

• structured control flow and immediate value-to-SSA promotion.

Phase C: scheduling core

• distribution variables as LayoutMaps;

• reduction-domain factorization, reducer-contract verification, and collective realization selection;

• prefix-preserving split/fuse/permute/reshape and execution binding;

• explicit repartition;

• pipeline dependence graph, version analysis, and memory planning;

• target catalog interfaces.

Phase D: TVM bootstrap

• Scheduled TileIR to native TIRx layout and execution export;

• GEMM, convolution, softmax, attention, loss reduction, Top-K, sort, elementwise, stencil, and copy coverage;

• differential layout/address tests against the TileIR interpreter;

• JIT cache and straightforward multi-variant autotuning.

Phase E: native optimization

• target atom selection and precise cost models;

• native pipeline/resource passes;

• direct lowering to Luisa/XIR/LLVM/native paths where it pays off;

• persistent, sparse, and architecture-specific expert features.

Final decisions

• Tile APIs have no backward-compatibility requirement during this redesign. Once a replacement API is selected, migrate implementations, tests, examples, benchmarks, and documentation together and remove the old API. Do not retain compatibility aliases, forwarding wrappers, duplicate entry points, or legacy behavior switches solely to preserve old source spelling. API choices follow the execution/resource model, not migration avoidance.

• The language is execution-structure first: an open logical ExecNest and anchored regions are the semantic skeleton, not a loop nest invented by a late schedule or backend.

• The C++ hierarchy is written as nested one-pass range-for scopes, so parentage, lifetime, and local-to-prefix coordinate derivation are visible in source.

• parallel, serial, pipeline, and algebraic reduce are the complete core structured-region kinds. Only parallel extends the spatial owner hierarchy; a reduction domain may be mapped across space and time by scheduling.

• Every range-for binding is a scope handle. reduce(domain, contract?) infers its outer Tile states and built-in merge contracts from direct updates; custom algebraic states provide only the otherwise-unprovable contract.

• Public convenience does not imply a core IR entity. Neural-network, collective, ordering, and copy APIs are reference libraries over the minimal core; hardware acceleration normally adds a proved target atom, not syntax. mma(a, b, c) is the one admitted tensor arithmetic primitive because decomposition would discard its fused accumulation, precision, and operand- layout legality contract; concrete MMA instructions remain target atoms.

• Halide’s separation of computation and storage placement is retained, but both are expressed against that pre-existing execution structure.

• Execution transforms are typed layout remaps whose prefix-cut preservation is proved before dependent operations or memories are rewritten.

• Execution binding is a layout map; hardware names are late target data.

• A parallel region may carry a concise exec::block/warp/thread/... constraint. Nested bindings are verified against the target’s containment poset and ancestor projections, never enum ordinal values.

• A value’s declaration scope constrains its logical anchor, not whether an inner scope may read or write it. The innermost parallel supplies the default spatial frontier. Noninterfering ancestor partial updates join into a new value. Overlapping Tile updates need an explicit supported combiner; conflicting mutable Memory effects need an appropriate explicit effect contract, such as atomics. The compiler represents assembly and may validate overlap, but does not ask the user to re-prove parallel’s noninterference promise. Current capture/IR still lack the general Tile partial-update/join protocol; see the calculus page.

• Distribution is a typed layout map/correspondence, not a separate algebra.

• Scalar pure operators lift directly to logical Tiles; map is only the custom scalar-region escape hatch, and physical repartition remains explicit in IR.

• Logical dimensions are fresh function-local Dim identities. Labels are diagnostics only; there is no predefined neural-network axis vocabulary.

• A reducer contract, semantic contribution domain, and grouping projection define reduction meaning. Physical replicas never count twice, and common expression reductions are shorthand for the same nest-like region.

• Fixed-size Top-K uses a merge-and-truncate reducer under an explicit total order. Full sort remains a logical Tile permutation and decomposes into visible multi-pass structure when it cannot be realized in one target scope.

• The layout core is CuTe-derived mixed-radix algebra with composition closure, typed dimensions, explicit correspondence fibers, F2-linear import, pure index expressions, and a finite fallback.

• Memory is an explicit sibling resource owned by an execution prefix, never a child execution level. It is an expert escape hatch for stable address identity; ordinary Tile materialization is compiler-planned.

• Explicit Memory uses memory.store(tile) and memory.load() effects. Assignment remains exclusively Tile SSA syntax; Memory = Tile, Memory = View, and implicit view loads are ill-formed.

• Optional mem::shared/private_/tensor/... tags constrain resource class but do not form a memory hierarchy; target legality is a general execution-scope/resource/operation capability relation.

• Ancestor execution plus participant-local access reaches physical memory through the execution-to-memory composition.

• Pipeline is a temporal producer/consumer nest. k.stage(optional_name) marks source cuts inside its local stage namespace; memory versioning is derived only for materialized edges.

• Direct assignment is the C++ surface; region results exist only inside IR.

• Ordinary repeated JIT of ordinary host configurations is the baseline autotuning model.

• TileIR is thin but fully transformable, with SSA use-def, owned regions, rewriters, analyses, and verification.

• TVM is a replaceable lowering backend, not the semantic owner.

• MLIR is not required.

• Backend-owned create_tile_kernel routes: Metal lowers MPP natively or through the TIRx device artifact; CUDA compiles the TIRx cuda/nvptx artifact with the standalone-NVRTC pipeline and launches it directly (no cooperative matrix/vector, no indirect dispatch). On CUDA every Tile tensor operator (MMA/matmul, REDUCE, elementwise lift, tile map/extract) keeps the reference SIMT realization, and explicit Metal-only planning knobs fail closed instead of being silently downgraded. CUDA/NVPTX blocks are warp aligned (partial mapper domains are rounded up to the 32-thread warp and unaligned caps down), successfully patched old-driver PTX is written back to the tile cache, and NVPTX PTX artifacts that cannot load are reported as non-recompilable text with a recommendation to use the cuda TIRx target.

The accompanying GEMM sketch and kernel gallery exercise the proposed syntax.