TFX usage instrumentation

[casassg]

As a platform maintainer, I would like to track usage of TFX pipelines. Currently I can do so by exporting MLMD metadata, but I lack context on who is using it (aka number of users) which is an imporant metric for my success.

[zhitaoli]

who is using it (aka number of users) which is an imporant metric for my success.

Can you share a bit more on this? Is this number of events (pipeline create/run), attribute, or something else?

In general, if you can derive these metrics from MLMD queries that'd be much easier to build, but if it's too impossible, we are open for certain discussions.

We also want to make sure the metrics created can be properly scrubbed.

[casassg]

We would like to get something like weekly active users for our code. Aka how many people have used TFX per week.

We can track active pipelines and number of components per pipelines, but we are lacking context on who is actually using it. It may happen though that 2 users are working on the same pipeline and developing on it.

I think this problem is also related to how to namespace pipelines. Currently in TFX you need to manually namespace through Pipeline name.