Chart test framework as package

[ad-m]

Description

In #18127 Apache Superset noticed the potential of using a test framework created on the basis of Pytest to verify the Helm Chart. I believe that a synergistic effect is possible in this respect.

I am aware that Apache Airflow is going to introduce some optimizations to its framework. However, they - for the Apache Superset - are not blocking to the naturally low number of tests at the beginning of the adaptation and the planned strategy to make them fast enough.

A more important issue is how to share this code between projects, so as not to have duplicate it (which is necessary for a synergistic effect) and copyright requirements (despite the Apache license, it is necessary to exercise moral rights).

What actions are required on the Apache Airflow side to be able to share this code and make it available e.g. as a shared library?

Use case/motivation

Apache Superset would like perform off-line Helm Charts tests to verify that generated manifests have valid format.

Apache Airflow had already a few iterations of Helm Chart testing, including generated manifest format. Currently, the solution based on the pytest is satisfying Apache Airflow developers, hence I am looking to adapt it to Apache Superset.

Related issues

apache/superset#18127 (discussed by @kaxil, @mik-laj, @potiuk )

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

My opinion: I think it's far too early (and maybe even never will be a good time) to share that code.

Sharing code is not always 100% good. While it removes duplication it also introduces coupling between the parties that use the same code - and often the coupling is hidden. And when you have completely unrelated release schedule and needs and when you will need to fix something quickly, such coupling (and necessary need to introduce another release for the shared library) might have very bad impact on both projects.

Sharing code is always a trade-off of "duplication" and "coupling". We saw it for example in Flask App Builder - also shared between Airflow and Superset. It causes us a lot of headeaches (for Airflow mainly because FAB is necessarily much closer to Superset due to Daniel working there) and eventually we are about to (in the future) vendor-in only parts of Flask Appbuilder - mainly becaue of different and unpredictable release cycle between Airflow, FAB and Superset.

I think the code to copy for that is very little and benefits of non-dupllcating the code are fare ouweighted by the coupling introduced. Maybe - after we introduce some optimisations it will be worthwile to make a shared library, but for now I think just copying the code responsible for those tests is better. Even if it will mean that some future changes in Airlfow will be several times copy&pasted to Superset or the other way round. At least unti we find few other 3rd-parties that would also like to use it.

Maybe - at some point in time - there will be enough code and reusability to merge those approaches but I would be against introducing it now. I have the saying "the code must be usable first and you can make it reusable later" so I tend to think about having a shared "reusable" library when there are at least 3 users of very similar "usable" code.

This is just my opinion though so I wonder what others think.

[mik-laj]

@potiuk I agree with it. Simply copying the code will be sufficient. Especially since Apache Superset and Apache Airflow are under the same license, so we can keep the same license header.

[ad-m]

Thanks for the explanation. I am not fully aware of the Apache Foundation project approach for this scenario. How do you resolve the author tagging of this code (oznaczenie autorstwa w ramach autorskich praw osobistych)? Normally, the commit history does this.

[potiuk]

Thanks for the explanation. I am not fully aware of the Apache Foundation project approach for this scenario. How do you resolve the author tagging of this code (oznaczenie autorstwa w ramach autorskich praw osobistych)? Normally, the commit history does this.

We do not tag authors, we tag the projects we copied the code from.

• ASF policy for 3rd-party licences is here: https://www.apache.org/legal/resolved.html

Particularly the last FAQ entry states:

WHAT ARE REQUIRED THIRD-PARTY NOTICES?
When a release contains third party works, the licenses covering those works may ask that you inform consumers in certain specific fashions. These third party notices vary from license to license. Apache releases should contain a copy of each license, usually contained in the LICENSE document. For many licenses this is a sufficient notice. Some licenses require some additional notice. In many cases, you can include this notice within the dependent artifact.

However those regulations are only necessary for code that gets released in "official sources" using release policy https://www.apache.org/legal/release-policy.html

For the "build/dev/test" code that is not used to actually build release package and that is not included in those source releases just mentioning where the code was copied from in Commit should be enough I think (but I would anyhow follow the 3rd-party code policy for all code used)

[potiuk]

There are few more links in the ASF-policies that I encourage you to follow @ad-m - this might be very interesting reading for you :D

[ad-m]

Thank you very much for presenting your opinion. In this case - depending on the discussion on the Apache Superset side - I think the code should be copied, and then we will wonder if it can and should be done better. Since two PMC Apache Airflow expressed the same position, I see no more for discussion and close the issue.

@potiuk, thank you very much for the links. I am aware that the Apache Foundation has strict licensing requirements, and I do not know them fully yet (after all, the role of PMC is to guard them), so I try to be conservative and ask questions.