Llama-7b Crashes during IREE Compilation #16317
Assignees
Labels
bug 🐞
Something isn't working
Comments
|
guessing this is an upstream bug in linalg::GeneralizeOuterUnitDimsPackOpPattern |
|
I can not reproduce the issue when I specify the CPU to cascadelake. Here is the command I used: What is the target cpu? |
This is the CPU that I'm running this model on: |
|
Can you share the output of |
|
|
@hanhanW, using this flag |
|
Okay, I can reproduce the issue if I don't specify target cpu... There is definitely a bug in the upstream pattern. Your CPU is broadwell, and I can also reproduce it with |
Thank you @hanhanW! |
|
The quick workaround (for functionality) is deleting these https://github.com/openxla/iree/blob/af387d39d2dd553d03943c6a698cc15b6a8fc483/compiler/src/iree/compiler/Codegen/Common/DecomposePackUnPackOps.cpp#L145-L155 (I already shared it with Kumar) I have a smaller repro now: func.func @main_graph_dispatch_17_pack_f32() {
%c0 = arith.constant 0 : index
%c5 = arith.constant 5 : index
%c1 = arith.constant 1 : index
%cst = arith.constant 0.000000e+00 : f32
%c3200 = arith.constant 3200 : index
%c88320 = arith.constant 88320 : index
%c32 = arith.constant 32 : index
%0 = hal.interface.binding.subspan set(0) binding(0) type(storage_buffer) alignment(64) offset(%c3200) flags(ReadOnly) : !flow.dispatch.tensor<readonly:tensor<?x5x5xf32>>{%c32}
%1 = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) alignment(64) offset(%c88320) : !flow.dispatch.tensor<writeonly:tensor<?x1x5x8x1xf32>>{%c32}
%workgroup_id_x = hal.interface.workgroup.id[0] : index
%workgroup_count_x = hal.interface.workgroup.count[0] : index
%workgroup_id_y = hal.interface.workgroup.id[1] : index
%workgroup_count_y = hal.interface.workgroup.count[1] : index
%workgroup_id_z = hal.interface.workgroup.id[2] : index
%workgroup_count_z = hal.interface.workgroup.count[2] : index
%2 = affine.apply affine_map<()[s0] -> (s0 * 64)>()[%workgroup_id_z]
%3 = affine.apply affine_map<()[s0] -> (s0 * 64)>()[%workgroup_count_z]
scf.for %arg0 = %2 to %c32 step %3 {
scf.for %arg1 = %workgroup_id_y to %c1 step %workgroup_count_y {
%4 = affine.apply affine_map<()[s0] -> (s0 * 2)>()[%workgroup_id_x]
%5 = affine.apply affine_map<()[s0] -> (s0 * 2)>()[%workgroup_count_x]
scf.for %arg2 = %4 to %c5 step %5 {
%6 = affine.min affine_map<(d0) -> (-d0 + 5, 2)>(%arg2)
%7 = flow.dispatch.tensor.load %1, offsets = [%arg0, %arg1, %arg2, 0, 0], sizes = [32, 1, %6, 8, 1], strides = [1, 1, 1, 1, 1] : !flow.dispatch.tensor<writeonly:tensor<?x1x5x8x1xf32>>{%c32} -> tensor<32x1x?x8x1xf32>
%8 = affine.min affine_map<(d0) -> (-d0 + 32, 64)>(%arg0)
%9 = affine.apply affine_map<(d0) -> (d0 * 8)>(%arg1)
%10 = flow.dispatch.tensor.load %0, offsets = [%arg0, %9, %arg2], sizes = [%8, 5, %6], strides = [1, 1, 1] : !flow.dispatch.tensor<readonly:tensor<?x5x5xf32>>{%c32} -> tensor<?x5x?xf32>
%11 = scf.for %arg3 = %c0 to %c32 step %c1 iter_args(%arg4 = %7) -> (tensor<32x1x?x8x1xf32>) {
%12 = scf.for %arg5 = %c0 to %6 step %c1 iter_args(%arg6 = %arg4) -> (tensor<32x1x?x8x1xf32>) {
%13 = affine.min affine_map<(d0, d1) -> (1, d0 - d1)>(%8, %arg3)
%extracted_slice = tensor.extract_slice %10[%arg3, 0, %arg5] [%13, 5, 1] [1, 1, 1] : tensor<?x5x?xf32> to tensor<?x5x1xf32>
%extracted_slice_0 = tensor.extract_slice %arg6[%arg3, 0, %arg5, 0, 0] [1, 1, 1, 8, 1] [1, 1, 1, 1, 1] : tensor<32x1x?x8x1xf32> to tensor<1x1x1x8x1xf32>
%pack = tensor.pack %extracted_slice padding_value(%cst : f32) outer_dims_perm = [0, 1, 2] inner_dims_pos = [1, 2] inner_tiles = [8, 1] into %extracted_slice_0 {lowering_config = #iree_codegen.lowering_config<tile_sizes = [[64, 1, 2], [1, 1, 1]]>} : tensor<?x5x1xf32> -> tensor<1x1x1x8x1xf32>
%inserted_slice = tensor.insert_slice %pack into %arg6[%arg3, 0, %arg5, 0, 0] [1, 1, 1, 8, 1] [1, 1, 1, 1, 1] : tensor<1x1x1x8x1xf32> into tensor<32x1x?x8x1xf32>
scf.yield %inserted_slice : tensor<32x1x?x8x1xf32>
}
scf.yield %12 : tensor<32x1x?x8x1xf32>
}
flow.dispatch.tensor.store %11, %1, offsets = [%arg0, %arg1, %arg2, 0, 0], sizes = [32, 1, %6, 8, 1], strides = [1, 1, 1, 1, 1] : tensor<32x1x?x8x1xf32> -> !flow.dispatch.tensor<writeonly:tensor<?x1x5x8x1xf32>>{%c32}
}
}
}
return
} |
|
Summarize the discussion from Discord: We hit an assertion in runtime because the frontend generates bad code. The below is the example. All the inputs are constants (which are generated by frontend), and then it hits the assertion. I will fix the compilation issue in upstream repo. %cst_6 = arith.constant dense<-1> : tensor<i64>
%cst_7 = arith.constant dense<1> : tensor<4xi64>
%cst_8 = arith.constant dense<[1, 1, 5, 5]> : tensor<4xi64>
%6 = linalg.generic {indexing_maps = [#map2, #map3, #map2], iterator_types = ["parallel"]} ins(%cst_7, %cst_6 : tensor<4xi64>, tensor<i64>) outs(%5 : tensor<4xi64>) {
^bb0(%in: i64, %in_413: i64, %out: i64):
%1654 = arith.muli %in, %in_413 : i64
linalg.yield %1654 : i64
} -> tensor<4xi64>
%7 = tensor.empty() : tensor<4xi1>
%8 = linalg.generic {indexing_maps = [#map2, #map2, #map2], iterator_types = ["parallel"]} ins(%cst_8, %6 : tensor<4xi64>, tensor<4xi64>) outs(%7 : tensor<4xi1>) {
^bb0(%in: i64, %in_413: i64, %out: i1):
%1654 = arith.cmpi eq, %in, %in_413 : i64
linalg.yield %1654 : i1
} -> tensor<4xi1>
%9 = linalg.generic {indexing_maps = [#map2, #map2, #map2, #map2], iterator_types = ["parallel"]} ins(%8, %cst_7, %cst_8 : tensor<4xi1>, tensor<4xi64>, tensor<4xi64>)
^bb0(%in: i1, %in_413: i64, %in_414: i64, %out: i64):
%1654 = arith.select %in, %in_413, %in_414 : i64
linalg.yield %1654 : i64
} -> tensor<4xi64>
%extracted_slice = tensor.extract_slice %9[0] [1] [1] : tensor<4xi64> to tensor<1xi64>
%extracted = tensor.extract %extracted_slice[%c0] : tensor<1xi64>
%extracted_slice_23 = tensor.extract_slice %9[1] [1] [1] : tensor<4xi64> to tensor<1xi64>
%extracted_24 = tensor.extract %extracted_slice_23[%c0] : tensor<1xi64>
%13 = arith.cmpi slt, %extracted_24, %c0_i64 : i64
%14 = arith.index_cast %extracted_24 : i64 to index
%15 = arith.select %13, %c1, %14 : index
%71 = arith.cmpi eq, %15, %c32 : index
cf.assert %71, "mismatched size for broadcast" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What happened?
While compilation, I'm getting the following crash:
Steps to reproduce your issue
IR is available at: https://gist.github.com/vivekkhandelwal1/185ed5eb3f76c7d88fd37a489185d0fb
Command to reproduce the crash:
What component(s) does this issue relate to?
MLIR, Compiler
Version information
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: