How can CodeQL extractor parse and extract *_test.go files in Go?

[Lslightly]

As far as I know, CodeQL will instrument the compiler to extract AST and other information. But when I query information in testing code, I can't get related information.

I suspect that CodeQL ignores the testing files becuase when go build the project, it will simply ignore testing files. How can I use CodeQL to extract information from testing files since they are just special go files?

Renaming *_test.go to other files like *_tes.go does work but I think it's not elegant and it destroys the project structure and semantics.

[smowton]

If you're using Code Scanning, you can use an advanced setup and replace the autobuild step with a go build invocation that does build the test files.

Or, if you're using the CLI locally, you can pass the --command flag to the database create operation in order to achieve the same thing.

[Lslightly]

I tried go test -c . which compiles the test files to binary. But when I passed it as the argument of --command, codeql failed to create a database with the following log.

 pkgtest git:(main) ✗  24-07-30 16:19 ls                                                                                                              
go.mod  lib.go  lib_test.go

 pkgtest git:(main) ✗  24-07-30 16:23 codeql database create -l=go -s . ../../codeql-db/pkgtest --command="go test -c ." --overwrite          
Initializing database at ../codeql-queries/test/codeql-db/pkgtest.
Running build command: [go, test, -c, .]
Finalizing database at ../codeql-queries/test/codeql-db/pkgtest.
CodeQL detected code written in Go but could not process any of it. This can occur if the specified build commands failed to compile or process any code.
 - Confirm that there is some source code for the specified language in the project.
 - For codebases written in Go, JavaScript, TypeScript, and Python, do not specify 
   an explicit --command.
 - For other languages, the --command must specify a "clean" build which compiles 
   all the source code files without reusing existing build artefacts.

pkgtest git:(main) ✗  24-07-30 16:23 ls
go.mod  lib.go  lib_test.go  pkgtest.test

The binary pkgtest.test is generated but not analyzed by codeql.

I also tried go test -run=NOMATCH_XXX ./... as pointed out in stackoverflow answer, but it comes out the same.

I guess that codeql does not analyze Go testing code because testing code is not important in security alert. But I think it's necessary for a code scanning tool to scan testing code because it's also part of a project. This is my opinion.

[mbg]

@Lslightly make sure that you are in fact building the tests by performing a clean build. E.g. run go clean first. CodeQL does not analyse binaries, but source code. In order for CodeQL to find the source code, it must see it being built.

If performing a clean build still doesn't work, would you be able to share the resulting database with us?

[Lslightly]

pkgtest.zip

Unfortunately, go clean and go clean -testcache does not take effect. The zip above includes source code, testing code, codeql database and a bash script to generate the database.

[mbg]

Thanks for providing all of this. It seems that we are deliberately ignoring go test commands after all. We will discuss internally whether this filtering is necessary or we can ship something that would allow your use-case in a future release. I'll see if we can come up with a workaround that you can use in the meantime.

[smowton]

As of CodeQL 2.19.1, you will be able to set environment variable CODEQL_EXTRACTOR_GO_OPTION_EXTRACT_TESTS to true, or if you're using the CLI use -O extract_tests=true, to extract tests.

Alternatively if you are using a manual build command / the CLI -c option, you can use a go test command (I recommend go test -c to build the tests without executing them) in order to extract them.

[thaJeztah]

Also running into this; CodeQL is complaining to be mis-configured because it skipped ~50% of the files. Looking at the downloaded CSV with files included/skipped, that indeed turned out to be;

• _test.go files
• Some platform-specific (_windows.go) files

Scary error

The error

The reason

[smowton]

That error suggests actually there is one or more CodeQL job finding zero Go files to scan -- but evidently, also at least one finding some to populate the "Scanned files" box. Can you share any more details about the setup here? Are you using Code Scanning default setup, or an advanced (in-repo configuration file) setup? If the latter, can you share the config file? Is there more than one Action running CodeQL, and/or a matrix action that uses manual build tagging in order to analyse multiple different codebases stored in the same repo?