[refactor] Better import order by using __all__

[archibate]

Related issue = #

[Click here for the format server]

---

@rexwangcc As you said, from xxx import * can be a bad anti-pattern unless we use __all__, so here I am.
This PR is stage 1, I will dig further into taichi.lang only after @yuanming-hu merge all other PRs about lang to pvc.

[k-ye]

Thanks this LGTM. But please wait on @rexwangcc or other Py experts to take a pass on this..

[k-ye]

IIUC, this is a package, and __all__ of a package should define the modules it loads upon a wildcard import? (https://docs.python.org/3/tutorial/modules.html#importing-from-a-package). But I'm just voicing my wondering,
maybe this is the only option without breaking tons of things in the current situation...

[archibate]

I thought __all__ here is only work for from taichi import * and our user should never use that..? Anyway, let's cover that usage too. I think in the future we can use this for user shortcuts like:

Suggested change

	__all__ = [s for s in dir() if not s.startswith('_')]
	ti = __import__('taichi')
	__all__ = ['Vector', 'Matrix', ..., 'ti']

So that users can directly use Vector([2, 3]) if they use from taichi import *.

[rexwangcc]

Thanks for making the PR!

I'm not a 🐍 expert, just trying to give my 2 cents here:

When I was advocating "getting rid of import *" at #1478 (comment) , I was thinking about:

1. namespace pollution problems (e.g. os got shadowed by ti.os and forced us to use __import__('os')) and the
2. redundancy of importing unnecessary names
3. how to make interpreter + developers' life easier when tracking the provenance of a foreign name

While there are various good and practical ways to address 1. and 2.: e.g. __all__, from util import os as _os, import util.foo/import util and stick with namespace.foo across the module, I personally don't have any perferences among them as long as the change is smooth and doesn't break a ton of stuff as @k-ye mentioned.

My only concern is that using __all__ v.s. explicilty enumerate each imported name can add fuel to further abuse import * patterns, let alone it makes a bit harder to keep track of the provenance of an imported name (3.). (But we may don't have to worry about this if it isn't a problem to others as it's not wise to solve a problem that doesn't exist yet)

[archibate]

let alone it makes a bit harder to keep track of the provenance of an imported name

Both:

# __init__.py
from .util import a, b, c, d

or

# __init__.py
from .util import *

# util.py
__all__ = ['a', 'b', 'c', 'd']

solve the problem. Why I choose 2? Because I think it's convinent to manage exported-name inplace in util.py, instead of in another file __init__.py.

[codecov]

Codecov Report

Merging #1510 into master will decrease coverage by 0.04%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##           master    #1510      +/-   ##
==========================================
- Coverage   66.20%   66.15%   -0.05%     
==========================================
  Files          38       40       +2     
  Lines        5403     5404       +1     
  Branches      975      976       +1     
==========================================
- Hits         3577     3575       -2     
- Misses       1658     1661       +3     
  Partials      168      168              

Impacted Files	Coverage Δ
python/taichi/tools/np2ply.py	13.37% <ø> (ø)
python/taichi/core/settings.py	48.14% <45.45%> (ø)
python/taichi/__init__.py	75.00% <75.00%> (-17.00%)	⬇️
python/taichi/core/__init__.py	100.00% <100.00%> (ø)
python/taichi/core/util.py	21.25% <100.00%> (ø)
python/taichi/misc/__init__.py	100.00% <100.00%> (ø)
python/taichi/misc/gui.py	23.98% <100.00%> (+0.28%)	⬆️
python/taichi/misc/image.py	56.86% <100.00%> (+0.86%)	⬆️
python/taichi/misc/util.py	44.89% <100.00%> (+0.66%)	⬆️
python/taichi/tools/__init__.py	100.00% <100.00%> (ø)
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e518981...7ab621a. Read the comment docs.

[rexwangcc]

Your latest argument makes sense to me. (I still think using less * can help track the provenance better w/o an IDE, but let's see how this goes) This looks the most smooth approach to the current code organization among all potential solutions, LGTM! Thanks for doing this