Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decide about cythonizing #166

Closed
ceball opened this issue Jul 4, 2017 · 2 comments
Closed

Decide about cythonizing #166

ceball opened this issue Jul 4, 2017 · 2 comments

Comments

@ceball
Copy link
Contributor

ceball commented Jul 4, 2017

Currently, pip install/python setup.py attempts to cythonize. This may bring performance improvements (e.g. when creating many parameterized objects), although we have no performance tests running to back that up (#165). However, there are a number of potential downsides of cythonizing:

(1) source code is obscured/people might think it's not purely python e.g. if they try to open the file they've imported;

(2) possible errors/incompatibilities introduced by cython we're not aware of (although you could also argue cythonizing helps catch a class of problems, and also cythonizing by default means we're all testing cython's effects all the time);

(3) possible installation failures (e.g. #129) or confusion (e.g. #161);

(4) just more stuff to think about overall.

Incidentally, conda packages have never included cythonized param. People would only have the cythonized version by pip install/python setup.py from pypi/github on a machine with cython and a working compiler.

@ceball
Copy link
Contributor Author

ceball commented Aug 27, 2017

pip-installed (i.e. cythonized) param causes the following error somewhere in topographica's tests:

$ TOPO_SKIP_RUNTESTS_GUI_IF_NO_DISPLAY=1 DISPLAY= ./topographica -t all
[...]
Traceback (most recent call last):
  File "/Users/cball/code/ioam/topographica/topo/command/__init__.py", line 867, in __call__
    save_script_repr()
  File "/Users/cball/code/ioam/topographica/topo/command/__init__.py", line 396, in save_script_repr
    script = header+topo.sim.script_repr()
  File "/Users/cball/code/ioam/topographica/topo/base/simulation.py", line 1537, in script_repr
    sorted(self.objects().values(), cmp=lambda x, y: cmp(x.name,y.name))]
  File "/Users/cball/code/ioam/topographica/topo/base/simulation.py", line 240, in script_repr
    super(EventProcessor,self).script_repr(imports=imports,prefix=prefix)
  File "param/parameterized.py", line 1236, in param.parameterized.Parameterized.script_repr
    return self.pprint(imports,prefix, unknown_value=None, qualify=True,
  File "param/parameterized.py", line 1285, in param.parameterized.Parameterized.pprint
    value = pprint(values[k], imports, prefix=prefix,settings=[],
  File "param/parameterized.py", line 891, in param.parameterized.pprint
    rep=val.script_repr(imports, prefix+"    ")
  File "param/parameterized.py", line 1236, in param.parameterized.Parameterized.script_repr
    return self.pprint(imports,prefix, unknown_value=None, qualify=True,
  File "param/parameterized.py", line 1285, in param.parameterized.Parameterized.pprint
    value = pprint(values[k], imports, prefix=prefix,settings=[],
  File "param/parameterized.py", line 887, in param.parameterized.pprint
    rep = script_repr_reg[type(val)](val,imports,prefix,settings)
  File "param/parameterized.py", line 912, in param.parameterized.container_script_repr
    result.append(pprint(i,imports,prefix,settings))
  File "param/parameterized.py", line 891, in param.parameterized.pprint
    rep=val.script_repr(imports, prefix+"    ")
  File "param/parameterized.py", line 1236, in param.parameterized.Parameterized.script_repr
    return self.pprint(imports,prefix, unknown_value=None, qualify=True,
  File "param/parameterized.py", line 1285, in param.parameterized.Parameterized.pprint
    value = pprint(values[k], imports, prefix=prefix,settings=[],
  File "param/parameterized.py", line 891, in param.parameterized.pprint
    rep=val.script_repr(imports, prefix+"    ")
  File "param/parameterized.py", line 1236, in param.parameterized.Parameterized.script_repr
    return self.pprint(imports,prefix, unknown_value=None, qualify=True,
  File "param/parameterized.py", line 1261, in param.parameterized.Parameterized.pprint
    spec = inspect.getargspec(self.__init__)
  File "/Users/cball/Eunectes/mc3/envs/topographica/lib/python2.7/inspect.py", line 815, in getargspec
    raise TypeError('{!r} is not a Python function'.format(func))
TypeError: <cyfunction Time.__init__ at 0x113cd0c50> is not a Python function
[...]
runtests: targets with errors: 2 [scriptrepr,batch]
runtests: ran 7 targets [training,unopt,snapshots,pickle,scriptrepr,batch,maps]
WARNING:root:Time: 000000.00 CommandLine: Errors encountered; exiting with return code 2

Need to investigate.

@ceball
Copy link
Contributor Author

ceball commented Aug 27, 2017

Note: the above probably goes away with python >= 3.4 (http://bugs.python.org/issue17159), although that doesn't make it less of a problem.

@ceball ceball added this to the 1.6.0 milestone Aug 28, 2017
@ceball ceball mentioned this issue Sep 10, 2017
@ceball ceball closed this as completed Sep 25, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant