Consider using integration test instead of unit test

[xxchan]

          how about splitting `src/stream/src/` and `src/stream/src/test`? In other words, the test in this PR will be in `src/stream/src/test/executor/over_window`. This arrangement of test may not be able to access some private field, but currently most of our test does not acquire access them at all.

          Btw, I remember we have mentioned that we can move the unit test into a separate because our streaming executor's tests do not need access private field and methods. And after that we can depend on more utils such as frontend crate. But I can not find that comments or issue.

Originally posted by @st1page in #9787 (comment)

[xxchan]

I've tried this change in fae044a (#9787)

BTW, I didn't changed any visibility, so I think this claim is true:

but currently most of our test does not acquire access them at all.

About compilation speed

Changing one line in /src (even under mod tests) takes 10s to re-compile, while changing one line in /tests takes only 2s to re-compile.

The former (UTs) are slower to compile because you need to compile the whole lib with --cfg tests. (btw, another fact is that the library is recompiled twice: once with, and once without --test). The latter can reuse the compiled lib, so it's much faster and offers much better DX when developing unit tests (without changing the code).

Another factor is that if you 1. risedev p, 2. modified an IT, 3. risedev p, you won't need to re-compile at all! (Although I'm not sure how frequent this case occurs... If you modified an UT, that can take some time (some data here #9553 (comment))

[xxchan]

after that we can depend on more utils such as frontend crate

We might be able to simplify executor creation which is also painful (not sure). (Note that #9787 simplified deriving outputs in the tests, without changing executor creation).

[xxchan]

This also applies to other packages especially those compile slowly. 😇 Although I'm not sure about the frequency of adding new tests. Anyway if you feel painful about that you may consider writing new tests in /tests

❯ cargo test -p risingwave_meta --no-run  
   Compiling risingwave_test_runner v1.0.0-alpha (/Users/xxchan/Projects/risingwave/src/test_runner)
   Compiling risingwave_meta v1.0.0-alpha (/Users/xxchan/Projects/risingwave/src/meta)
    Finished test [unoptimized + debuginfo] target(s) in 36.69s

[xxchan]

Close as no futher action planned. This issue can be used as a reference later.

[xxchan]

A crate will be compiled twice, once for lib and once for UTs (even if it doesn't have any UTs). Splitting out integration tests (and then set test = false) can also reduce quite some compilation time. 🤣

---

Note: the bottleneck is risingwave_stream. So it should be a good idea to use integration tests for stream