Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mlir] fix lit tests #356

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

makslevental
Copy link

@makslevental makslevental commented Dec 20, 2024

Building against LLVM @ 60325abeb3226b17c28429dfa6e175f25c171ec0 and z3==4.13.4 I do not see compile issues with z3 (everything builds fine) but I do see lit tests failing due to drifting MLIR IR syntax and TIMEOUTs (not sure which ones are expected and which not). ctest outputs follow:

ctest -R Litmus

Test project /Users/mlevental/dev_projects/mlir-tv/cmake-build-debug
      Start 18: Litmus-abstraction
 1/19 Test #18: Litmus-abstraction ...............   Passed   16.48 sec
      Start 19: Litmus-affine-ops
 2/19 Test #19: Litmus-affine-ops ................   Passed    0.34 sec
      Start 20: Litmus-arith-ops
 3/19 Test #20: Litmus-arith-ops .................   Passed    0.84 sec
      Start 21: Litmus-bufferization-ops
 4/19 Test #21: Litmus-bufferization-ops .........***Failed    0.88 sec
-- Testing: 15 tests, 10 workers --
PASS: MLIR :: bufferization-ops/buffer_cast_zero (1 of 15)
PASS: MLIR :: bufferization-ops/buffer_cast_i8 (2 of 15)
PASS: MLIR :: bufferization-ops/buffer_cast (3 of 15)
XFAIL: MLIR :: bufferization-ops/clone-no-introduce (4 of 15)
PASS: MLIR :: bufferization-ops/clone-removal (5 of 15)
PASS: MLIR :: bufferization-ops/bufferize-select (6 of 15)
PASS: MLIR :: bufferization-ops/bufferize-select2 (7 of 15)
PASS: MLIR :: bufferization-ops/clone (8 of 15)
TIMEOUT: MLIR :: bufferization-ops/convolution (9 of 15)
******************** TEST 'MLIR :: bufferization-ops/convolution' FAILED ********************

********************
PASS: MLIR :: bufferization-ops/var_buffer_cast (10 of 15)
XFAIL: MLIR :: bufferization-ops/fill-memref-bad (11 of 15)
PASS: MLIR :: bufferization-ops/fill-memref (12 of 15)
PASS: MLIR :: bufferization-ops/tensor_load (13 of 15)
PASS: MLIR :: bufferization-ops/copy (14 of 15)
PASS: MLIR :: bufferization-ops/copy_i32 (15 of 15)
********************
Timed Out Tests (1):
  MLIR :: bufferization-ops/convolution


Testing Time: 0.81s
  Passed           : 12
  Expectedly Failed:  2
  Timed Out        :  1

      Start 22: Litmus-cexprint
 5/19 Test #22: Litmus-cexprint ..................   Passed    0.33 sec
      Start 23: Litmus-diagnostics
 6/19 Test #23: Litmus-diagnostics ...............   Passed    0.33 sec
      Start 24: Litmus-fp-ops
 7/19 Test #24: Litmus-fp-ops ....................   Passed    2.01 sec
      Start 25: Litmus-func-ops
 8/19 Test #25: Litmus-func-ops ..................   Passed    0.52 sec
      Start 26: Litmus-linalg-loops
 9/19 Test #26: Litmus-linalg-loops ..............   Passed    1.67 sec
      Start 27: Litmus-linalg-ops
10/19 Test #27: Litmus-linalg-ops ................***Failed    1.37 sec
-- Testing: 22 tests, 10 workers --
PASS: MLIR :: linalg-ops/convolution_constfold (1 of 22)
PASS: MLIR :: linalg-ops/dot_assoc_varlen (2 of 22)
PASS: MLIR :: linalg-ops/convolution_int (3 of 22)
PASS: MLIR :: linalg-ops/dot (4 of 22)
PASS: MLIR :: linalg-ops/dot2 (5 of 22)
PASS: MLIR :: linalg-ops/dot_associativity (6 of 22)
PASS: MLIR :: linalg-ops/dot_associativity2_multiset (7 of 22)
PASS: MLIR :: linalg-ops/dot_associativity2 (8 of 22)
PASS: MLIR :: linalg-ops/dot_associativity3_multiset (9 of 22)
PASS: MLIR :: linalg-ops/dot_commutative (10 of 22)
PASS: MLIR :: linalg-ops/dot_constfold (11 of 22)
PASS: MLIR :: linalg-ops/fill (12 of 22)
PASS: MLIR :: linalg-ops/init_tensor (13 of 22)
UNSUPPORTED: MLIR :: linalg-ops/pooling_unsupported1 (14 of 22)
PASS: MLIR :: linalg-ops/init_tensor_cast (15 of 22)
UNSUPPORTED: MLIR :: linalg-ops/pooling_unsupported2 (16 of 22)
PASS: MLIR :: linalg-ops/memref_matmul_bad (17 of 22)
TIMEOUT: MLIR :: linalg-ops/dot_rewrite_manually_assoc (18 of 22)
******************** TEST 'MLIR :: linalg-ops/dot_rewrite_manually_assoc' FAILED ********************

********************
XFAIL: MLIR :: linalg-ops/matmul-bad (19 of 22)
PASS: MLIR :: linalg-ops/conv-uninit (20 of 22)
PASS: MLIR :: linalg-ops/memref_matmul (21 of 22)
PASS: MLIR :: linalg-ops/dot_rewrite_manually (22 of 22)
********************
Timed Out Tests (1):
  MLIR :: linalg-ops/dot_rewrite_manually_assoc


Testing Time: 1.30s
  Unsupported      :  2
  Passed           : 18
  Expectedly Failed:  1
  Timed Out        :  1

      Start 28: Litmus-math-ops
11/19 Test #28: Litmus-math-ops ..................   Passed    0.33 sec
      Start 29: Litmus-memref-ops
12/19 Test #29: Litmus-memref-ops ................***Failed   32.52 sec
-- Testing: 31 tests, 10 workers --
PASS: MLIR :: memref-ops/alloc-varsize (1 of 31)
PASS: MLIR :: memref-ops/alloc (2 of 31)
PASS: MLIR :: memref-ops/alloca (3 of 31)
XFAIL: MLIR :: memref-ops/alloc-bad (4 of 31)
PASS: MLIR :: memref-ops/alloc-alloc-noalias (5 of 31)
PASS: MLIR :: memref-ops/alloc-arg-noalias (6 of 31)
XFAIL: MLIR :: memref-ops/alloc-arg-noalias-bad (7 of 31)
PASS: MLIR :: memref-ops/collapse-shape (8 of 31)
PASS: MLIR :: memref-ops/dealloc-alloca (9 of 31)
PASS: MLIR :: memref-ops/dealloc (10 of 31)
XFAIL: MLIR :: memref-ops/dealloc-bad2 (11 of 31)
XFAIL: MLIR :: memref-ops/dealloc-view-bad (12 of 31)
XFAIL: MLIR :: memref-ops/dealloc-bad (13 of 31)
PASS: MLIR :: memref-ops/dim (14 of 31)
PASS: MLIR :: memref-ops/copy-fill (15 of 31)
PASS: MLIR :: memref-ops/expand-collapse-shape (16 of 31)
PASS: MLIR :: memref-ops/global (17 of 31)
XFAIL: MLIR :: memref-ops/global-arg-alias-bad (18 of 31)
PASS: MLIR :: memref-ops/global-const (19 of 31)
XFAIL: MLIR :: memref-ops/global-const-bad (20 of 31)
XFAIL: MLIR :: memref-ops/load-intro-bad (21 of 31)
PASS: MLIR :: memref-ops/global-const-intro (22 of 31)
PASS: MLIR :: memref-ops/expand-shape (23 of 31)
XFAIL: MLIR :: memref-ops/load-intro-bad2 (24 of 31)
PASS: MLIR :: memref-ops/memref-inputs-simple (25 of 31)
PASS: MLIR :: memref-ops/nonidentity-layout-store (26 of 31)
XFAIL: MLIR :: memref-ops/subview_inbounds-noub (27 of 31)
PASS: MLIR :: memref-ops/subview_out_of_bounds (28 of 31)
PASS: MLIR :: memref-ops/subview_reduce_rank (29 of 31)
TIMEOUT: MLIR :: memref-ops/nonidentity-layout-store-bad (30 of 31)
******************** TEST 'MLIR :: memref-ops/nonidentity-layout-store-bad' FAILED ********************

********************
TIMEOUT: MLIR :: memref-ops/copy-fill-bad (31 of 31)
******************** TEST 'MLIR :: memref-ops/copy-fill-bad' FAILED ********************

********************
********************
Timed Out Tests (2):
  MLIR :: memref-ops/copy-fill-bad
  MLIR :: memref-ops/nonidentity-layout-store-bad


Testing Time: 32.45s
  Passed           : 19
  Expectedly Failed: 10
  Timed Out        :  2

      Start 30: Litmus-refinement
13/19 Test #30: Litmus-refinement ................   Passed    0.41 sec
      Start 31: Litmus-sparsetensor-ops
14/19 Test #31: Litmus-sparsetensor-ops ..........   Passed    0.35 sec
      Start 32: Litmus-tensor-constant
15/19 Test #32: Litmus-tensor-constant ...........   Passed    1.85 sec
      Start 33: Litmus-tensor-ops
16/19 Test #33: Litmus-tensor-ops ................   Passed    1.64 sec
      Start 34: Litmus-tosa-constant
17/19 Test #34: Litmus-tosa-constant .............   Passed    0.36 sec
      Start 35: Litmus-tosa-ops
18/19 Test #35: Litmus-tosa-ops ..................***Failed    3.61 sec
-- Testing: 57 tests, 10 workers --
UNSUPPORTED: MLIR :: tosa-ops/add_broadcast_unknownsz (1 of 57)
PASS: MLIR :: tosa-ops/add_broadcast1 (2 of 57)
PASS: MLIR :: tosa-ops/add_broadcast4 (3 of 57)
PASS: MLIR :: tosa-ops/add_broadcast5 (4 of 57)
PASS: MLIR :: tosa-ops/add_broadcast2 (5 of 57)
PASS: MLIR :: tosa-ops/bitwise_and (6 of 57)
PASS: MLIR :: tosa-ops/bitwise_and2 (7 of 57)
PASS: MLIR :: tosa-ops/add (8 of 57)
PASS: MLIR :: tosa-ops/add_broadcast3 (9 of 57)
PASS: MLIR :: tosa-ops/bitwise_not (10 of 57)
PASS: MLIR :: tosa-ops/avgpool2d (11 of 57)
TIMEOUT: MLIR :: tosa-ops/avgpool2d_memref (12 of 57)
******************** TEST 'MLIR :: tosa-ops/avgpool2d_memref' FAILED ********************

********************
PASS: MLIR :: tosa-ops/bitwise_broadcast (13 of 57)
PASS: MLIR :: tosa-ops/bitwise_or (14 of 57)
PASS: MLIR :: tosa-ops/bitwise_xor (15 of 57)
PASS: MLIR :: tosa-ops/bitwise_or2 (16 of 57)
PASS: MLIR :: tosa-ops/bitwise_xor2 (17 of 57)
PASS: MLIR :: tosa-ops/concat (18 of 57)
PASS: MLIR :: tosa-ops/concat2 (19 of 57)
XFAIL: MLIR :: tosa-ops/concat-bad (20 of 57)
PASS: MLIR :: tosa-ops/exp-inf (21 of 57)
PASS: MLIR :: tosa-ops/conv2d3 (22 of 57)
PASS: MLIR :: tosa-ops/exp-nan (23 of 57)
PASS: MLIR :: tosa-ops/conv2d2 (24 of 57)
PASS: MLIR :: tosa-ops/exp-zero (25 of 57)
PASS: MLIR :: tosa-ops/exp-neginf (26 of 57)
PASS: MLIR :: tosa-ops/depthwise3 (27 of 57)
PASS: MLIR :: tosa-ops/gather (28 of 57)
PASS: MLIR :: tosa-ops/gather-const (29 of 57)
PASS: MLIR :: tosa-ops/depthwise1 (30 of 57)
PASS: MLIR :: tosa-ops/depthwise4 (31 of 57)
XFAIL: MLIR :: tosa-ops/gather-const-bad (32 of 57)
PASS: MLIR :: tosa-ops/gather-uninit-index (33 of 57)
PASS: MLIR :: tosa-ops/gather-uninit (34 of 57)
PASS: MLIR :: tosa-ops/gather-oob (35 of 57)
PASS: MLIR :: tosa-ops/conv2d1 (36 of 57)
PASS: MLIR :: tosa-ops/gather_unknown_size (37 of 57)
UNSUPPORTED: MLIR :: tosa-ops/mul_shift (38 of 57)
PASS: MLIR :: tosa-ops/depthwise2 (39 of 57)
PASS: MLIR :: tosa-ops/maxpool2d (40 of 57)
TIMEOUT: MLIR :: tosa-ops/maxpool2d_memref (41 of 57)
******************** TEST 'MLIR :: tosa-ops/maxpool2d_memref' FAILED ********************

********************
PASS: MLIR :: tosa-ops/maxpool_noop (42 of 57)
PASS: MLIR :: tosa-ops/negate (43 of 57)
PASS: MLIR :: tosa-ops/negate_int (44 of 57)
PASS: MLIR :: tosa-ops/mul_comm (45 of 57)
PASS: MLIR :: tosa-ops/reshape (46 of 57)
PASS: MLIR :: tosa-ops/reverse (47 of 57)
XFAIL: MLIR :: tosa-ops/reduce_sum_int-bad (48 of 57)
XFAIL: MLIR :: tosa-ops/reduce_sum-bad (49 of 57)
PASS: MLIR :: tosa-ops/sub (50 of 57)
PASS: MLIR :: tosa-ops/tile (51 of 57)
XFAIL: MLIR :: tosa-ops/tile-bad (52 of 57)
PASS: MLIR :: tosa-ops/transpose2 (53 of 57)
PASS: MLIR :: tosa-ops/transpose3 (54 of 57)
PASS: MLIR :: tosa-ops/transpose1 (55 of 57)
PASS: MLIR :: tosa-ops/depthwise5 (56 of 57)
PASS: MLIR :: tosa-ops/reverse-fold (57 of 57)
********************
Timed Out Tests (2):
  MLIR :: tosa-ops/avgpool2d_memref
  MLIR :: tosa-ops/maxpool2d_memref


Testing Time: 3.54s
  Unsupported      :  2
  Passed           : 48
  Expectedly Failed:  5
  Timed Out        :  2

      Start 36: Litmus-verbose
19/19 Test #36: Litmus-verbose ...................   Passed    3.05 sec

79% tests passed, 4 tests failed out of 19

Total Test time (real) =  68.91 sec

The following tests FAILED:
	 21 - Litmus-bufferization-ops (Failed)
	 27 - Litmus-linalg-ops (Failed)
	 29 - Litmus-memref-ops (Failed)
	 35 - Litmus-tosa-ops (Failed)
Errors while running CTest
ctest -R Opts

Test project /Users/mlevental/dev_projects/mlir-tv/cmake-build-debug
      Start  1: Opts-conv2d-to-img2col
 1/12 Test  #1: Opts-conv2d-to-img2col ...............   Passed    5.83 sec
      Start  2: Opts-convert-elementwise-to-linalg
 2/12 Test  #2: Opts-convert-elementwise-to-linalg ...   Passed    0.34 sec
      Start  3: Opts-fold-memref-subview-op
 3/12 Test  #3: Opts-fold-memref-subview-op ..........   Passed    0.73 sec
      Start  4: Opts-fold-tensor-extract-op
 4/12 Test  #4: Opts-fold-tensor-extract-op ..........   Passed    0.40 sec
      Start  5: Opts-fusion-tensor
 5/12 Test  #5: Opts-fusion-tensor ...................   Passed    4.88 sec
      Start  6: Opts-linalg-bufferize
 6/12 Test  #6: Opts-linalg-bufferize ................   Passed   19.97 sec
      Start  7: Opts-linalg-canonicalize
 7/12 Test  #7: Opts-linalg-canonicalize .............   Passed    1.40 sec
      Start  8: Opts-linalg-fold-unit-extent-dims
 8/12 Test  #8: Opts-linalg-fold-unit-extent-dims ....   Passed    0.36 sec
      Start  9: Opts-linalg-generalize-named-ops
 9/12 Test  #9: Opts-linalg-generalize-named-ops .....***Failed   31.03 sec
-- Testing: 3 tests, 3 workers --
PASS: MLIR :: linalg-generalize-named-ops/fill (1 of 3)
PASS: MLIR :: linalg-generalize-named-ops/matmul (2 of 3)
TIMEOUT: MLIR :: linalg-generalize-named-ops/copy (3 of 3)
******************** TEST 'MLIR :: linalg-generalize-named-ops/copy' FAILED ********************

********************
********************
Timed Out Tests (1):
  MLIR :: linalg-generalize-named-ops/copy


Testing Time: 30.96s
  Passed   : 2
  Timed Out: 1

      Start 10: Opts-tensor-constant-bufferize
10/12 Test #10: Opts-tensor-constant-bufferize .......   Passed    0.38 sec
      Start 11: Opts-tosa-make-broadcastable
11/12 Test #11: Opts-tosa-make-broadcastable .........   Passed    0.32 sec
      Start 12: Opts-tosa-to-linalg
12/12 Test #12: Opts-tosa-to-linalg ..................   Passed    4.26 sec

92% tests passed, 1 tests failed out of 12

Total Test time (real) =  69.91 sec

The following tests FAILED:
	  9 - Opts-linalg-generalize-named-ops (Failed)
Errors while running CTest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant