Running queries on Android application #5039

mzfr · 2021-01-28T04:45:09Z

mzfr
Jan 28, 2021

Hey,

So I am trying to learn codeql and write my own queries. I was trying to the testing of my android project but I am having some difficulties.

I made the database from my application by running:

codeql database create testing-app --language=java

I ran this inside the directory in which my source code is present.

Then in that directory, I made a new file called test-queries.ql and tried to write some code.

I am doing this in vscode and have already installed the codeql extension for it.

Problem

When I am writing any queries errors like could not resolve module java or could not resolve module DataFlow.

I am not sure what I might be missing.

Answered by mzfr

Jan 28, 2021

Running all those commands eventually gave me the same errors. So I deleted the directory and re-did the whole process again and now things seem to be working.

I didn't do anything differently I just followed the instruction mentioned here(https://github.com/github/vscode-codeql-starter#instructions) basically the same thing that I did before but now this is working. 🙂

View full answer

mzfr · 2021-01-28T09:03:32Z

mzfr
Jan 28, 2021
Author

I restarted by following this documentation and I've done everything that is mentioned there but now in the output section of VS code I get the following error:

[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving query at normalized path /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/parsing-check.ql.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Found enclosing pack 'codeql-custom-queries-java' at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Adding compilation cache at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/.cache.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving library dependencies from /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/qlpack.yml.
[2021-01-28 14:25:51] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.scanForPacks(QLPackLibraryPathResolver.java:107)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.resolveSinglePack(QLPackLibraryPathResolver.java:84)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.lambda$executeJSON$1(ResolveLibraryPathCommand.java:214)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:231)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:44)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSON(SimpleJsonSubcommand.java:101)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:77)
                      com.semmle.cli2.picocli.SubcommandCommon.executeWithParent(SubcommandCommon.java:414)
                      com.semmle.cli2.execute.CliServerCommand.lambda$executeSubcommand$0(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.execute.CliServerCommand.executeSubcommand(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)
A fatal error occurred: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits

Resolving library paths failed: [2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving query at normalized path /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/parsing-check.ql.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Found enclosing pack 'codeql-custom-queries-java' at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Adding compilation cache at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/.cache.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving library dependencies from /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/qlpack.yml.
[2021-01-28 14:25:51] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.scanForPacks(QLPackLibraryPathResolver.java:107)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.resolveSinglePack(QLPackLibraryPathResolver.java:84)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.lambda$executeJSON$1(ResolveLibraryPathCommand.java:214)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:231)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:44)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSON(SimpleJsonSubcommand.java:101)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:77)
                      com.semmle.cli2.picocli.SubcommandCommon.executeWithParent(SubcommandCommon.java:414)
                      com.semmle.cli2.execute.CliServerCommand.lambda$executeSubcommand$0(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.execute.CliServerCommand.executeSubcommand(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)
A fatal error occurred: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
 (codeQL.runQuery)
Error: Resolving library paths failed: [2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving query at normalized path /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/parsing-check.ql.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Found enclosing pack 'codeql-custom-queries-java' at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Adding compilation cache at /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/.cache.
[2021-01-28 14:25:50] [DETAILS] resolve library-path> Resolving library dependencies from /run/media/mzfr/Data/CodeQL/codeql-custom-queries-java/qlpack.yml.
[2021-01-28 14:25:51] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.scanForPacks(QLPackLibraryPathResolver.java:107)
                      com.semmle.frontend.packs.QLPackLibraryPathResolver.resolveSinglePack(QLPackLibraryPathResolver.java:84)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.lambda$executeJSON$1(ResolveLibraryPathCommand.java:214)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:231)
                      com.semmle.cli2.resolve.ResolveLibraryPathCommand.executeJSON(ResolveLibraryPathCommand.java:44)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSON(SimpleJsonSubcommand.java:101)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:77)
                      com.semmle.cli2.picocli.SubcommandCommon.executeWithParent(SubcommandCommon.java:414)
                      com.semmle.cli2.execute.CliServerCommand.lambda$executeSubcommand$0(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.execute.CliServerCommand.executeSubcommand(CliServerCommand.java:67)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)
A fatal error occurred: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits

	at CodeQLCliServer.runCodeQlCliInternal (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/cli.js:190:23)
	at processTicksAndRejections (internal/process/task_queues.js:94:5)
	at async CodeQLCliServer.runJsonCodeQlCliCommand (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/cli.js:350:24)
	at async CodeQLCliServer.resolveLibraryPath (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/cli.js:369:16)
	at async Object.compileAndRunQueryAgainstDatabase (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/run-queries.js:370:24)
	at async compileAndRunQuery (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/extension.js:306:26)
	at async ctx.subscriptions.push.commandRunner_1.commandRunnerWithProgress.title (/home/mzfr/.vscode-insiders/extensions/github.vscode-codeql-1.3.10/out/extension.js:338:121)

0 replies

hmakholm · 2021-01-28T12:00:05Z

hmakholm
Jan 28, 2021
Maintainer

First let's see if we can reproduce the problem from the command line -- if we can, that would be easier to troubleshoot.

Can you try codeql resolve qlpacks, please? If this reproduces a failure, please provide

the output of codeql version --format json
the log file that codeql resolve qlpacks --logdir . will drop in the current directory

0 replies

hmakholm · 2021-01-28T12:16:55Z

hmakholm
Jan 28, 2021
Maintainer

Also, do you by any chance have an ancestor directory of /opt/nessus/var/nessus/audits set up to be a VS Code workspace? I'm struggling to understand why CodeQL would even try to scan that directory.

0 replies

alexet · 2021-01-28T12:40:18Z

alexet
Jan 28, 2021
Maintainer

Also if there is a line just above those logs that contains using CodeQL CLI: near the start followed by a command line it could be helpful to have that line.

0 replies

mzfr · 2021-01-28T12:55:48Z

mzfr
Jan 28, 2021
Author

Running all those commands eventually gave me the same errors. So I deleted the directory and re-did the whole process again and now things seem to be working.

I didn't do anything differently I just followed the instruction mentioned here(https://github.com/github/vscode-codeql-starter#instructions) basically the same thing that I did before but now this is working. 🙂

6 replies

mzfr Jan 28, 2021
Author

I did before deleting I ran: codeql resolve qlpacks --logdir . in the vscode-codeql-starter/codeql-custom-queries-cpp directory and the log file named resolve-qlpacks-20210128.175431.071.log was generated with the following content:

[2021-01-28 17:54:31] This is codeql resolve qlpacks --logdir .
[2021-01-28 17:54:31] [DETAILS] resolve qlpacks> Scanning for qlpack.yml from /opt/codeql/.codeqlmanifest.json
[2021-01-28 17:54:31] [DETAILS] resolve qlpacks> Scanning for qlpack.yml from /opt/
[2021-01-28 17:54:32] Exception caught at top level: Could not list directory /opt/nessus/var/nessus/audits: java.nio.file.AccessDeniedException: /opt/nessus/var/nessus/audits
                      com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:98)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:344)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:202)
                      com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:123)
                      com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:263)
                      com.semmle.frontend.packs.ScanByManifests.scanFromRoot(ScanByManifests.java:412)
                      com.semmle.frontend.packs.ScanByManifests.scanUnseenInMany(ScanByManifests.java:294)
                      com.semmle.frontend.packs.ScanByManifests.scanForNamedPacks(ScanByManifests.java:227)
                      com.semmle.frontend.packs.QlPackResolver.lambda$scanForNamedPacks$1(QlPackResolver.java:97)
                      java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
                      com.semmle.frontend.packs.QlPackResolver.scanForNamedPacks(QlPackResolver.java:95)
                      com.semmle.cli2.resolve.ResolveQlpacksCommand.executeJSON(ResolveQlpacksCommand.java:41)
                      com.semmle.cli2.resolve.ResolveQlpacksCommand.executeJSON(ResolveQlpacksCommand.java:20)
                      com.semmle.cli2.picocli.SimpleJsonSubcommand.executeSubcommand(SimpleJsonSubcommand.java:80)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:430)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:201)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:209)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:91)

mzfr Jan 28, 2021
Author

oh btw if it's possible would you be able to look into this(github/securitylab#253)

hmakholm Jan 28, 2021
Maintainer

Thanks for that log -- I can now reproduce the problem you ran into. I expect we'll be able to release a CLI version that fixes it somewhen next week.

As for the other question, I've poked our Java analysis people about it. (At least I assume it's Java -- in general it will be easier to answer questions if you state explicitly which language they're about; the code snippets in the question could also look like C# at first sight).

mzfr Jan 28, 2021
Author

Yes that's java. Thanks for the help
I'll mention it on that discussion

mzfr Jan 29, 2021
Author

Hi @hmakholm, I am sorry to tag you again but I haven't got any reply on my other question(github/securitylab#253). If it's possible can you look at it?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Running queries on Android application #5039

{{title}}

Replies: 5 comments 6 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Running queries on Android application #5039

mzfr Jan 28, 2021

Replies: 5 comments · 6 replies

mzfr Jan 28, 2021 Author

hmakholm Jan 28, 2021 Maintainer

hmakholm Jan 28, 2021 Maintainer

alexet Jan 28, 2021 Maintainer

mzfr Jan 28, 2021 Author

mzfr Jan 28, 2021 Author

mzfr Jan 28, 2021 Author

hmakholm Jan 28, 2021 Maintainer

mzfr Jan 28, 2021 Author

mzfr Jan 29, 2021 Author

mzfr
Jan 28, 2021

Replies: 5 comments 6 replies

mzfr
Jan 28, 2021
Author

hmakholm
Jan 28, 2021
Maintainer

hmakholm
Jan 28, 2021
Maintainer

alexet
Jan 28, 2021
Maintainer

mzfr
Jan 28, 2021
Author

mzfr Jan 28, 2021
Author

mzfr Jan 28, 2021
Author

hmakholm Jan 28, 2021
Maintainer

mzfr Jan 28, 2021
Author

mzfr Jan 29, 2021
Author