[CI] Failures in build-and-test / unittest-matrix don't cause the job to fail

[dmitryax]

Look at the failing TestLoadInvalidConfig_NoScrapers test in https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/3700493629/jobs/6269016994 :

=== RUN   TestLoadInvalidConfig_NoScrapers
    config_test.go:112: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/config_test.go:112
        	Error:      	"receivers::hostmetrics: must specify at least one scraper when using hostmetrics receiver" does not contain "receiver \"hostmetrics\" has invalid configuration: must specify at least one scraper when using hostmetrics receiver"
        	Test:       	TestLoadInvalidConfig_NoScrapers
--- FAIL: TestLoadInvalidConfig_NoScrapers (0.00s)

[github-actions]

Pinging code owners for exporter/loadbalancing: @jpkrohling. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dmitryax]

Looks like we released 0.67.0 with the following failing unit tests:

=== RUN   TestLoadInvalidConfig_NoScrapers
    config_test.go:112: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/config_test.go:112
        	Error:      	"receivers::hostmetrics: must specify at least one scraper when using hostmetrics receiver" does not contain "receiver \"hostmetrics\" has invalid configuration: must specify at least one scraper when using hostmetrics receiver"
        	Test:       	TestLoadInvalidConfig_NoScrapers
--- FAIL: TestLoadInvalidConfig_NoScrapers (0.00s)

=== RUN   TestConsumeLogsUnexpectedExporterType
    log_exporter_test.go:198: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/loadbalancingexporter/log_exporter_test.go:198
        	Error:      	Error message not equal:
        	            	expected: "unable to export logs, unexpected exporter type: expected *exporter.Logs but got loadbalancingexporter.mockComponent"
        	            	actual  : "unable to export logs, unexpected exporter type: expected *component.LogsExporter but got loadbalancingexporter.mockComponent"
        	Test:       	TestConsumeLogsUnexpectedExporterType
--- FAIL: TestConsumeLogsUnexpectedExporterType (0.00s)

=== RUN   TestConsumeTracesUnexpectedExporterType
    trace_exporter_test.go:300: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/loadbalancingexporter/trace_exporter_test.go:300
        	Error:      	Error message not equal:
        	            	expected: "expected *exporter.Traces but got loadbalancingexporter.mockComponent"
        	            	actual  : "expected *component.TracesExporter but got loadbalancingexporter.mockComponent"
        	Test:       	TestConsumeTracesUnexpectedExporterType
--- FAIL: TestConsumeTracesUnexpectedExporterType (0.00s)

See https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/3659745132/jobs/6186112298

Failing hostmetrics test is not a problem. But I need help from the routing processor owner to assess the impact

[dmitryax]

My first guess is that the issue is caused by #16731

[MovieStoreGuy]

Ooooh, I think I have an idea what has happened here.

Due to the fact that:

go test -race -timeout 300s --tags="" -v ./... 2>&1 | tee -a ./foresight-test-report.txt; \

go test pipes the result to tee, make doesn't capture the exit failure from go test but from tee

[dmitryax]

I think the tests broken in 0.67.0 are safe to not worry about 0.67.0 release

[dmitryax]

We are done. CI is back :)