Importing theta-python as dependency

[dmvianna]

I am using poetry to manage my Python dependencies. Given that, and as theta-python is not available via pypi, I depend on using the git facilities in poetry to download and install it.

Turns out that as of today, poetry has no support for Python packages that don't have their configuration (pyproject.toml, setup.cfg, etc) at the root of the git repository.

Would it be possible to move the Python configuration files to the root? I am not proficient with setup.cfg and setup.py, but with pyproject.toml you would have something like:

[tool.poetry]
name = "theta-python"
version = "0.0.1"
description = "Support libraries for using Theta-IDL types in Python"
authors = ["Target"]
packages = [
    { include = "python/**/*.py" },
]

[tool.poetry.dependencies]
python = "^3.6.5"
hypothesis = "^6.36.1"

[TikhonJelvis]

Oh, that's a weird limitation from Poetry. I guess some people do not like monorepos very much :P.

I've been using Poetry and pyproject.toml for my internal Python projects at work for a while, this project just preceded that—I'd be happy to switch it over. Putting the project file in the root of the repo isn't ideal from an organizational point of view, but seems like a reasonable workaround for now.

Down the road, I should be able to upload the support Python package to PyPI, but I guess that would still be a bit limited if you needed to use a version from a specific commit or something...

[TikhonJelvis]

Okay, I created a top-level pyproject.toml file in #22. Eventually I'd like to publish the library to PyPI (as part of CI, ideally), but this should be a workaround for now.

Can you test it out? If it works for you, I'll merge the PR.

[dmvianna]

Well, this is embarrassing. It does work, but it is an ugly workaround. The prefix now is

import python.theta

That's due to the name of your directories, which is how Python namespacing works. It... works.

We ended up creating a local fork for ourselves, so that we could use

theta-python = { path = "../../theta-idl/python", develop = true }

and then import theta as

import theta

Which is what we want.

This is the depth of my Python skills. Given your directory structure and naming, I don't know how to fix that. Unless, of course, you rework the directory structure, which I wouldn't. Or poetry implements this feature request, which I despair about, as their backlog has thousands of similar ones.

[TikhonJelvis]

Oh man, that's pretty annoying :/. I guess it also breaks the generated Python code.

In that case I won't merge that change. Sounds like you have a workaround for now, and the "right" solution is going to be publishing the theta-python library to PyPI.

[dmvianna]

Oh, there's hope!

[TikhonJelvis]

Nice. Looks like it isn't in an official Poetry release yet, but it can be used by pulling poetry-core itself from Git.

In the meantime, do you need any changes to how the Python library is organized in the current state of the repo?

[dmvianna]

No, thanks. You can close it.