sqllogictest: parallelize test

[skyzh]

depends on create database #2529

Just as what @lmatz has proposed, we can use database to isolate different test cases, and run them in parallel.

[lmatz]

Actually, it's @wangrunji0408 who proposed to use create database. I thought it is a very neat idea.

Now on second thought, I find this may have some potential problems:

1. if we are testing crate database switch to these types of statements, will the test produce unexpected results?
2. the database is created for each .slt file or each .slt.part file? Do we allow a .slt.part file to include other slt.part files?

[skyzh]

For 1, maybe yes. I'm thinking of users should explicitly write create database and use database at the beginning of the sqllogictest files as statement ok. This solution might also solve 2. So the only thing sqllogictest itself needs to support if to run tests in parallel.

[lmatz]

users should explicitly write create database and use database

We may need to hack the name a bit to avoid different test files using the same database name when running tests in parallel.

[skyzh]

We may need to hack the name a bit to avoid different test files using the same database name when running tests in parallel.

Yes, that's true.

[wangrunji0408]

We may introduce a new command to make the tester create an anonymous database for the current file. As an alias for create database XXX; use database XXX and implicitly drop database XXX on exit.

[skyzh]

implemented in risinglightdb/sqllogictest-rs#45, we can integrate it with our CI soon 🥰

[xxchan]

resolved by #3046

[skyzh]

Still not resolved, parallel test can only be running in in-memory mode. There're bugs with concurrent query in production code.

[lmatz]

Is it solved now?
ci/scripts/e2e-test-parallel.sh is currently being enabled for each PR