MLIR-SIMD-Operations¶
Prompt: 1¶
message time: 2026-09-26 00:00:00
mlir simd
—
googlesearch response
MLIR handles Single Instruction, Multiple Data (SIMD) operations primarily through its Vector Dialect and the Builtin Dialect’s vector type, providing portable abstractions across diverse hardware architectures.
Core SIMD Concepts in MLIR
Vector Types: Represented as
vector<vector-dim-list x vector-element-type>, where dimensions can be fixed-length (e.g.,vector<16xf32>) or scalable (e.g.,vector<[8] x f32>) for architectures like ARM SVE.Vector Dialect: Offers hardware-agnostic operations (like
vector.transfer_read,vector.contract, andvector.broadcast) that map high-level loops or tensor operations down to hardware intrinsics.Progressive Lowering: MLIR allows you to write target-independent SIMD code and progressively lower it—moving from high-level abstractions (
linalgoraffineloops) through thevectordialect, and finally down to LLVM IR or target-specific instructions (AVX2, AVX-512, ARM Neon, and RISC-V Vector extension / RVV).If you want to dive deeper, let me know:
Are you trying to auto-vectorize loops or write explicit vector intrinsics?
What target architecture (x86, ARM, RISC-V) are you targeting?
Copied to clipboardFailed to copy to clipboard. Try again later.
More
—
Prompt: 2¶
message time: 2026-09-26 00:00:00
Hot to use MLIR SIMD with tensors? Unroll first?
—
googlesearch response
No, do not unroll first. In the standard MLIR CodeGen philosophy, unrolling happens at a much lower abstraction level. Unrolling loops or expanding expressions before vectorization obscures the high-level math, destroys regular access patterns, and makes the vectorizer’s job significantly harder.
Instead, you use a structured compilation pipeline. In MLIR, tensors are vectorized while still structured (usually inside the
linalgdialect), and unrolling is performed later on the resulting virtual vectors.* * *
The Canonical MLIR SIMD Pipeline for Tensors
To cleanly go from high-level tensor operations to native SIMD instructions, follow this 4-step progressive lowering sequence:
1. Tiling to Static Shapes
High-level tensor operations often have dynamic sizes (e.g.,
tensor<?x?xf32>). Before vectorizing, you uselinalg::tileToForallOporlinalg::tileToScfForOpto break the workload down into static tiles that match your target vector registers (e.g., a tile size of8x16or1x4).2. Vectorization (at the Tensor level)
Instead of generating loops and unrolling them, you vectorize the structured tensor operation directly using
linalg::LinalgVectorizationPattern(often driven via the Transform Dialect).This pass captures the structured layout and translates it into virtual Vector Transfer Operations:
vector.transfer_read: Pulls a static tile out of a continuoustensordirectly into a high-dimensional virtual register.
vector.contract/arith/mathops: Performs the computation directly on the multi-dimensional virtual vector.
vector.transfer_write: Spills the virtual register back into a destinationtensor.3. Bufferization
Up to this point, you are completely inside the purely functional tensor domain (no side effects). Now, you run One-Shot Bufferization (
-one-shot-bufferize) to translate tensors into concrete memory allocations (memref). Your vector transfer operations seamlessly adapt, shifting from readingtensorsto loading frommemrefs.4. Vector Unrolling & Decomposing (The “Unroll” Phase)
This is where unrolling finally happens. At this stage, your IR likely contains high-dimensional virtual vectors (e.g., a
vector<32x64xf32>from a large tile) that do not fit into actual CPU hardware registers (like anxmm/ymmregister).You populate Vector Unroll Patterns (
vector::populateVectorUnrollPatterns). This unrolls the vector dimensions themselves—not loops—down to native target sizes (likevector<8xf32>for AVX2 orvector<16xf32>for AVX-512).* * *
What it Looks Like in MLIR IR
A. Structured Tensor Op (Before Vectorization)
mlir
// A static tile resulting from step 1 (Tiling) %tile_A = tensor.extract_slice %A[%i, %j] [8, 16] [1, 1] : tensor<1024x1024xf32> to tensor<8x16xf32> %tile_B = tensor.extract_slice %B[%j, %k] [16, 4] [1, 1] : tensor<1024x1024xf32> to tensor<16x4xf32> // High-level tensor math %tiled_matmul = linalg.matmul ins(%tile_A, %tile_B : tensor<8x16xf32>, tensor<16x4xf32>) outs(%tile_C : tensor<8x4xf32>) -> tensor<8x4xf32>Use code with caution.
B. Vectorized Tensor Op (After Vectorization)
Notice how loops were completely bypassed. The vectorizer handles the structure natively via transfer operations:
mlir
// Step 2: Read slices directly into virtual vectors %vA = vector.transfer_read %A[%i, %j], %cst : tensor<1024x1024xf32>, vector<8x16xf32> %vB = vector.transfer_read %B[%j, %k], %cst : tensor<1024x1024xf32>, vector<16x4xf32> %vC = vector.transfer_read %C[%i, %k], %cst : tensor<1024x1024xf32>, vector<8x4xf32> // Multi-dimensional vector math %vD = vector.contract { ... } %vA, %vB, %vC : vector<8x16xf32>, vector<16x4xf32> into vector<8x4xf32> // Write back to tensor %res = vector.transfer_write %vD, %C[%i, %k] : vector<8x4xf32>, tensor<1024x1024xf32>Use code with caution.
Summary Recommendation
Keep your tensors structured for as long as possible. Let
linalgmap your tensors to multi-dimensionalvectorops first. Let MLIR handle the unrolling downstream right before it emits LLVM IR.Would you like to see a complete Transform Dialect script or an mlir-opt pass pipeline that executes this tiling-to-vectorization flow on an operation?
Copied to clipboardFailed to copy to clipboard. Try again later.
More
—
Prompt: 3¶
message time: 2026-09-26 00:00:00
Yes
—
googlesearch response
The following mlir-opt command-line pipeline and accompanying Transform Dialect script execute the entire flow: from high-level, dynamically shaped tensor operations down to target-independent vector code.
1. The Compilation Pass Pipeline
To run this compilation pipeline via command-line, pipe your input through mlir-opt using these explicit scheduling passes:
mlir-opt input.mlir \ --transform-dialect-interpreter \ --one-shot-bufferize="bufferize-function-boundaries=true" \ --drop-equivalent-memref-alloc \ --convert-vector-to-scf \ --convert-scf-to-openmp \ --finalize-memref-to-llvm \ --convert-vector-to-llvm \ --convert-func-to-llvm \ -o output.mlirUse code with caution.
* * *
2. Complete MLIR Input & Transform Script
This unified file contains both the source matrix multiplication on dynamic tensors and the embedded Transform Dialect script. It precisely controls tiling sizes, forces direct tensor vectorization, bufferizes to memory, and then unrolls the virtual vector dimensions down to hardware-friendly register sizes.
mlir
// RUN: mlir-opt %s --transform-dialect-interpreter // 1. HIGH-LEVEL FUNCTION (Purely Functional Tensors) func.func @dynamic_matmul_simd(%A: tensor<?x?xf32>, %B: tensor<?x?xf32>, %C: tensor<?x?xf32>) -> tensor<?x?xf32> { // Named linalg operations retain the mathematical structure needed for clean SIMD mapping %0 = linalg.matmul ins(%A, %B : tensor<?x?xf32>, tensor<?x?xf32>) outs(%C : tensor<?x?xf32>) -> tensor<?x?xf32> return %0 : tensor<?x?xf32> } // 2. EMBEDDED TRANSFORM DIALECT SCRIPT module attributes {transform.with_named_sequence} { transform.named_sequence @__transform_main(%root: !transform.any_op) { // Step A: Find the high-level tensor operation %matmul = transform.structured.match ops{["linalg.matmul"]} in %root : (!transform.any_op) -> !transform.any_op // Step B: Tile to static blocks matching target vector lanes (e.g., 8x16x4 registers) // This injects loops for the remainder but exposes a clean static tile inside. %tiled, %loops:3 = transform.structured.tile_using_for %matmul [8, 16, 4] : (!transform.any_op) -> (!transform.any_op, !transform.any_op, !transform.any_op, !transform.any_op) // Step C: Vectorize the static tensor tile directly // Creates high-dimensional virtual vector transfer operations (vector.transfer_read / contract) %func = transform.structured.match ops{["func.func"]} in %root : (!transform.any_op) -> !transform.any_op transform.structured.vectorize %func : (!transform.any_op) // Step D: Perform Vector Unrolling and Decomposition // Breaks down large virtual vectors (e.g. 8x16) into native architecture dimensions (e.g. 1x8) // right before memory allocations and backend emitting. transform.vector.unroll %func [1, 8] : !transform.any_op transform.yield } }Use code with caution.
* * *
What Makes This Recipe Highly Scalable
Isolation of Architecture Layout: If you port this code from an AVX-512 CPU (requiring a
[1, 16]unroll factor) to an AVX2 CPU (requiring a[1, 8]factor), you only modify the parameters inside the Transform block. The core algorithmic function (@dynamic_matmul_simd) remains perfectly untouched.Dynamic to Static Handling: The
tile_using_forpass automatically handles edge-case masking or padding loops for sizes that don’t divide cleanly by8or16. It keeps the inner hot computation completely unrolled and vector-aligned.If you want to take this further, let me know:
Would you like to see how to incorporate vector masking/predication for when dynamic dimensions don’t divide evenly?
Do you want to see the resulting LLVM-IR assembly output or target-specific dialect conversions (like
X86Vector) for this pipeline?Copied to clipboardFailed to copy to clipboard. Try again later.
More
—
Prompt: 4¶
message time: 2026-09-26 00:00:00
So in MLIR only tensors which are unrolled to arrays are mutable?
—
googlesearch response
No, tensors are never unrolled to arrays to become mutable, because tensors are never mutable in MLIR.
In MLIR,
tensoris a purely mathematical, immutable SSA value. Think of a tensor like the number7—you cannot mutate7into8; you can only perform an operation that returns a completely new value.When you see a vectorizer generate a pipeline that updates values, it uses an explicitly non-mutating, functional dataflow pattern.
how “Mutation” Works in Tensor-level SIMD
At the tensor and vector level, changes are tracked using Value SSA dataflow, not memory overrides.
At the Tensor level (
linalg): Operations use an explicitouts()argument. This is not mutated. Instead, the operation returns a new tensor representing the updated state (e.g.,%result = linalg.matmul ... outs(%C)).At the SIMD Vector level (
vector): When you write data back into a tensor, you usevector.transfer_write. This operation takes the old tensor, takes your vector register, and returns a brand new tensor representing the merged result.mlir
// This does NOT mutate %old_tensor. It returns a NEW tensor handle %new_tensor. %new_tensor = vector.transfer_write %my_simd_vector, %old_tensor[%i, %j] : vector<8xf32>, tensor<1024xf32>Use code with caution.
When does true, in-place mutation actually happen?
True mutation only happens during Bufferization (Step 3 of the pipeline mentioned earlier).
Bufferization analyzes the entire dataflow chain of your tensors. If it notices that
%old_tensoris never used again aftervector.transfer_write, it optimizes away the creation of a new tensor. It translates the abstracttensorinto a concretememref(a pointer to raw memory) and converts the operation into a true, in-place SIMD hardware store that directly updates the array in memory.Summary
Tensors and Vectors are always immutable value tokens. They track data dependencies.
Memrefs represent actual mutable memory blocks (arrays).
Unrolling is strictly a structural sizing optimization for hardware registers; it has nothing to do with mutability.
Would you like to see how One-Shot Bufferization decides whether it can mutate a piece of memory in-place, or would you like to see the exact
memrefIR code generated after the tensor layer is stripped away?Copied to clipboardFailed to copy to clipboard. Try again later.
More