Fuzzer seg faulting on certain seeds

[kgpai]

Description

Expression fuzzer seems to be seg faulting in some prs.
I presume this bug is inside velox (might be inside fuzzer too ?).

Error Reproduction

See here : https://app.circleci.com/pipelines/github/facebookincubator/velox/35528/workflows/7fc31ec9-1a6f-4a3d-ae40-8e1e23a51c2e/jobs/234352

https://app.circleci.com/pipelines/github/facebookincubator/velox/35470/workflows/dd76500f-40d5-4a54-a1aa-4cfba56c63e6/jobs/233879/artifacts

Relevant logs

I1011 19:30:01.577975  7316 ExpressionFuzzer.cpp:1313] ==============================> Started iteration 6385 (seed: 3871643008)
I1011 19:30:01.641149  7316 ExpressionVerifier.cpp:86] Executing expression 0 : 12 elements starting at 134 {null, [135->52] [52->30] {0.5190461694728583, 1 elements starting at 292 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.0860636904835701}, null, 0.5854220390319824, [30->91] [91->154] [154->200] {null, 1978-04-20T20:58:14.726106380, 1804152232457195298, 0.8317597508430481, [200->217] 7123, null, false}, 0.13037705142050982}, [136->197] null, [137->199] [199->216] {0.723726817406714, 1 elements starting at 1999 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.754521369934082}, 74, 0.586267352104187, [216->188] [188->168] [168->10] {[10->230] 16492, 2028-01-04T14:43:35.411203011, 4005586656405382395, 0.9147925972938538, [10->49] 572, 4msgfUP4j$l"2U&r')e/, false}, 0.10480337566696107}, [138->124] [124->53] {0.755961311981082, 1 elements starting at 511 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.34427136182785034}, 82, 0.557868242263794, null, 0.6331583296414465}, ...}
I1011 19:30:01.641242  7316 ExpressionVerifier.cpp:86] Executing expression 1 : substr("c0",switch("c1","c2",eq(greatest(2116667230,1691520997,1146621674,1562683503,subscript("c3",from_big_endian_32("c4"))),"c5"),"c6","c7",year_of_week(coalesce(array_max("c8"),subscript(12 elements starting at 581 {2010-03-02T13:26:57.621140804, 1975-12-27T23:08:31.365754329, 2007-09-19T13:39:32.997635826, 1989-09-01T17:53:02.765301608, 2008-01-08T16:30:55.003614383, ...},580100088028589200),array_max("c8"),try_cast 1899049195 as TIMESTAMP,1989-03-16T10:26:06.716578520,from_unixtime(round(sqrt(log10(clamp(to_unixtime(1987-02-18T13:39:46.473469779),to_unixtime(1987-02-18T13:39:46.473469779),null))),"c9")))),is_nan(round(to_unixtime(1987-02-18T13:39:46.473469779))),"c10"))
I1011 19:30:01.650269  7316 FuzzerToolkit.cpp:103] Exceptions match.
I1011 19:30:01.650352  7316 ExpressionFuzzer.cpp:1353] Both paths failed with compatible exceptions. Retrying expression using try().
I1011 19:30:01.651218  7316 ExpressionVerifier.cpp:86] Executing expression 0 : try(12 elements starting at 134 {null, [135->52] [52->30] {0.5190461694728583, 1 elements starting at 292 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.0860636904835701}, null, 0.5854220390319824, [30->91] [91->154] [154->200] {null, 1978-04-20T20:58:14.726106380, 1804152232457195298, 0.8317597508430481, [200->217] 7123, null, false}, 0.13037705142050982}, [136->197] null, [137->199] [199->216] {0.723726817406714, 1 elements starting at 1999 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.754521369934082}, 74, 0.586267352104187, [216->188] [188->168] [168->10] {[10->230] 16492, 2028-01-04T14:43:35.411203011, 4005586656405382395, 0.9147925972938538, [10->49] 572, 4msgfUP4j$l"2U&r')e/, false}, 0.10480337566696107}, [138->124] [124->53] {0.755961311981082, 1 elements starting at 511 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.34427136182785034}, 82, 0.557868242263794, null, 0.6331583296414465}, ...})
I1011 19:30:01.651290  7316 ExpressionVerifier.cpp:86] Executing expression 1 : try(substr("c0",switch("c1","c2",eq(greatest(2116667230,1691520997,1146621674,1562683503,subscript("c3",from_big_endian_32("c4"))),"c5"),"c6","c7",year_of_week(coalesce(array_max("c8"),subscript(12 elements starting at 581 {2010-03-02T13:26:57.621140804, 1975-12-27T23:08:31.365754329, 2007-09-19T13:39:32.997635826, 1989-09-01T17:53:02.765301608, 2008-01-08T16:30:55.003614383, ...},580100088028589200),array_max("c8"),try_cast 1899049195 as TIMESTAMP,1989-03-16T10:26:06.716578520,from_unixtime(round(sqrt(log10(clamp(to_unixtime(1987-02-18T13:39:46.473469779),to_unixtime(1987-02-18T13:39:46.473469779),null))),"c9")))),is_nan(round(to_unixtime(1987-02-18T13:39:46.473469779))),"c10")))
I1011 19:30:01.660547  7316 ExpressionVerifier.cpp:74] All results match.
I1011 19:30:01.662393  7316 ExpressionVerifier.cpp:74] All results match.
I1011 19:30:01.662415  7316 ExpressionVerifier.cpp:74] All results match.
I1011 19:30:01.662564  7316 ExpressionFuzzer.cpp:1274] Retrying original expression on 33 rows without errors
I1011 19:30:01.663427  7316 ExpressionVerifier.cpp:86] Executing expression 0 : 12 elements starting at 134 {null, [135->52] [52->30] {0.5190461694728583, 1 elements starting at 292 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.0860636904835701}, null, 0.5854220390319824, [30->91] [91->154] [154->200] {null, 1978-04-20T20:58:14.726106380, 1804152232457195298, 0.8317597508430481, [200->217] 7123, null, false}, 0.13037705142050982}, [136->197] null, [137->199] [199->216] {0.723726817406714, 1 elements starting at 1999 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.754521369934082}, 74, 0.586267352104187, [216->188] [188->168] [168->10] {[10->230] 16492, 2028-01-04T14:43:35.411203011, 4005586656405382395, 0.9147925972938538, [10->49] 572, 4msgfUP4j$l"2U&r')e/, false}, 0.10480337566696107}, [138->124] [124->53] {0.755961311981082, 1 elements starting at 511 {h:pat,=gA/@\g?Vb9]yn]Z`y3k"7lqEs/T => 0.34427136182785034}, 82, 0.557868242263794, null, 0.6331583296414465}, ...}
I1011 19:30:01.663497  7316 ExpressionVerifier.cpp:86] Executing expression 1 : substr("c0",switch("c1","c2",eq(greatest(2116667230,1691520997,1146621674,1562683503,subscript("c3",from_big_endian_32("c4"))),"c5"),"c6","c7",year_of_week(coalesce(array_max("c8"),subscript(12 elements starting at 581 {2010-03-02T13:26:57.621140804, 1975-12-27T23:08:31.365754329, 2007-09-19T13:39:32.997635826, 1989-09-01T17:53:02.765301608, 2008-01-08T16:30:55.003614383, ...},580100088028589200),array_max("c8"),try_cast 1899049195 as TIMESTAMP,1989-03-16T10:26:06.716578520,from_unixtime(round(sqrt(log10(clamp(to_unixtime(1987-02-18T13:39:46.473469779),to_unixtime(1987-02-18T13:39:46.473469779),null))),"c9")))),is_nan(round(to_unixtime(1987-02-18T13:39:46.473469779))),"c10"))
*** Aborted at 1697052601 (Unix time, try 'date -d @1697052601') ***
*** Signal 11 (SIGSEGV) (0x90) received by PID 7316 (pthread TID 0x7f641b3605c0) (linux TID 7316) (code: address not mapped to object), stack trace: ***
(error retrieving stack trace)
/bin/bash: line 1:  7316 Segmentation fault      (core dumped) _build/debug/velox/expression/tests/velox_expression_fuzzer_test --seed ${RANDOM} --lazy_vector_generation_ratio 0.2 --duration_sec 1800 --enable_variadic_signatures --velox_fuzzer_enable_complex_types --velox_fuzzer_enable_column_reuse --velox_fuzzer_enable_expression_reuse --max_expression_trees_per_step 2 --retry_with_try --enable_dereference --logtostderr=1 --minloglevel=0 --repro_persist_path=/tmp/fuzzer_repro
FAIL: Expression run failed

[laithsakka]

ran fuzzer locally for hour no repo , also most of meta fuzzer tests that run continuously have not been failing

[laithsakka]

will keep an eye on it

[laithsakka]

the update allowed for repo
internal only link:
https://www.internalfb.com/intern/test/562950057189274?ref_report_id=0

[liujiayi771]

@laithsakka My pull request encountered this error, and from the logs, it seems to be the issue you were trying to reproduce. Indeed, a null VARCHAR was generated.
https://app.circleci.com/pipelines/github/facebookincubator/velox/38051/workflows/e85f88c3-fd09-433d-a815-a9b906ccc3f6/jobs/255934