Error: 'No module named tfx' - DirectRunner vs DataflowRunner

[luischinchillagarcia]

Given a TFX Pipeline that works perfectly with DirectRunner, it fails with DataflowRunner during the TFT component with the error:

No module named tfx.

However, it's unclear how or why this error occurs. Here is a Colab notebook that shows the Pipeline working under DirectRunner while failing under DataflowRunner.

Would this a Beam-side issue? And if not, what are the possible solutions and sources this error is originating from?

[ntakouris]

Is this a fork of tensorflow/workshops/tfx_colabs/TFX_Workshop_Colab.ipynb ?

[singhniraj08]

@luischinchillagarcia,

I have found a similar closed issue 1 and 2 in which resolution is provided for dependency and cloudpickle conflicts. Please let us know if this issue can be closed.

[singhniraj08]

Closing this due to inactivity. Please take a look into the answers provided above, feel free to reopen and post your comments(if you still have queries on this). Thank you!

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No