Fix pickling for instances of classes created using make_class() #282

trendels · 2017-11-04T09:50:58Z

Currently, trying to pickle an instance of a class created using attrs.make_class fails (tested on Python 2.7 and 3.5):

>>> import attr
>>> import pickle
>>> C = attr.make_class('C', ['x'])
>>> obj = C(1)
>>> pickle.dumps(obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_pickle.PicklingError: Can't pickle <class 'attr._make.C'>: attribute lookup C on attr._make failed

It has something to do with pickle looking for the class/type in the wrong module. I was curious how collections.namedtuple solves this and found this (also present in Python 2.7):

https://github.com/python/cpython/blob/v3.5.4/Lib/collections/__init__.py#L436-L445

This PR adds the same code (with comment) verbatim to attrs, to make pickling work. As a side effect, the repr of classes created this way now also contain name of the module where make_class() was called instead of "attr._make". I'm not sure if this needs a Changelog/Documentation update?

I also noticed that Python 3.6 added an explicit module parameter to collections.namedtuple, to allow manually overriding the module name to fix issues with pickling on other Python implementations such as IronPython (https://bugs.python.org/issue17941). Maybe this would also be useful to add to attrs in the future.

codecov · 2017-11-04T10:08:41Z

Codecov Report

Merging #282 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #282   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           9      9           
  Lines         660    666    +6     
  Branches      138    138           
=====================================
+ Hits          660    666    +6

Impacted Files	Coverage Δ
src/attr/_make.py	`100% <100%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1bb8fbd...93cd3f8. Read the comment docs.

hynek · 2017-11-04T11:14:01Z

That looks like a good last-minute change for 17.3 (famous last words).

Would you mind completing coverage and add a changelog entry? There’s a bunch of cosmetic problems but I can deal with them myself once it’s merged.

Thanks!

trendels · 2017-11-04T12:58:29Z

That would be awesome.

I've added a changelog entry. I'm not sure what do do about the coverage though, other than using # pragma: no cover. The line that is missing coverage is only there to silence exceptions on more "esoteric" Python interpreters (IronPython, Jython) where sys._getframes is not available or works differently than in CPython, according to the comment from collections.namedtuple.

hynek · 2017-11-04T13:02:23Z

Maybe try monkeypatch.delattr(sys, "_getframe)"?

If that doesn’t work because it’s too invasive, just add a no cover and call it a Saturday.

trendels · 2017-11-04T13:25:50Z

Ah yes, that should work. TIL about pytest monkeypatch :)

hynek · 2017-11-04T13:38:32Z

Wonderful, so you got something out of it too. ;) Thank you for your contribution!

167: Scheduled weekly dependency update for week 46 r=mithrandi ## Updates Here's a list of all the updates bundled in this pull request. I've added some links to make it easier for you to find all the information you need. <table align="center"> <tr> <td><b>attrs</b></td> <td align="center">17.2.0</td> <td align="center">»</td> <td align="center">17.3.0</td> <td> <a href="https://pypi.python.org/pypi/attrs">PyPI</a> | <a href="https://pyup.io/changelogs/attrs/">Changelog</a> | <a href="http://www.attrs.org/">Homepage</a> </td> <tr> <td><b>cryptography</b></td> <td align="center">2.1.2</td> <td align="center">»</td> <td align="center">2.1.3</td> <td> <a href="https://pypi.python.org/pypi/cryptography">PyPI</a> | <a href="https://pyup.io/changelogs/cryptography/">Changelog</a> | <a href="https://github.com/pyca/cryptography">Repo</a> </td> <tr> <td><b>hypothesis</b></td> <td align="center">3.33.0</td> <td align="center">»</td> <td align="center">3.37.0</td> <td> <a href="https://pypi.python.org/pypi/hypothesis">PyPI</a> | <a href="https://pyup.io/changelogs/hypothesis/">Changelog</a> | <a href="https://github.com/HypothesisWorks/hypothesis/issues">Repo</a> </td> </tr> </table> ## Changelogs ### attrs 17.2.0 -> 17.3.0 >### 17.3.0 >------------------- >Backward-incompatible Changes >^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >- Attributes are not defined on the class body anymore. > This means that if you define a class ``C`` with an attribute ``x``, the class will *not* have an attribute ``x`` for introspection anymore. > Instead of ``C.x``, use ``attr.fields(C).x`` or look at ``C.__attrs_attrs__``. > The old behavior has been deprecated since version 16.1. > (`253 <https://github.com/python-attrs/attrs/issues/253>`_) >Changes >^^^^^^^ >- ``super()`` and ``__class__`` now work on Python 3 when ``slots=True``. > (`102 <https://github.com/python-attrs/attrs/issues/102>`_, `226 <https://github.com/python-attrs/attrs/issues/226>`_, `269 <https://github.com/python-attrs/attrs/issues/269>`_, `270 <https://github.com/python-attrs/attrs/issues/270>`_, `272 <https://github.com/python-attrs/attrs/issues/272>`_) >- Added ``type`` argument to ``attr.ib()`` and corresponding ``type`` attribute to ``attr.Attribute``. > This change paves the way for automatic type checking and serialization (though as of this release ``attrs`` does not make use of it). > In Python 3.6 or higher, the value of ``attr.Attribute.type`` can alternately be set using variable type annotations > (see `PEP 526 <https://www.python.org/dev/peps/pep-0526/>`_). (`151 <https://github.com/python-attrs/attrs/issues/151>`_, `214 <https://github.com/python-attrs/attrs/issues/214>`_, `215 <https://github.com/python-attrs/attrs/issues/215>`_, `239 <https://github.com/python-attrs/attrs/issues/239>`_) >- The combination of ``str=True`` and ``slots=True`` now works on Python 2. > (`198 <https://github.com/python-attrs/attrs/issues/198>`_) >- ``attr.Factory`` is hashable again. (`204 > <https://github.com/python-attrs/attrs/issues/204>`_) >- Subclasses now can overwrite attribute definitions of their superclass. > That means that you can -- for example -- change the default value for an attribute by redefining it. > (`221 <https://github.com/python-attrs/attrs/issues/221>`_, `229 <https://github.com/python-attrs/attrs/issues/229>`_) >- Added new option ``auto_attribs`` to ``attr.s`` that allows to collect annotated fields without setting them to ``attr.ib()``. > Setting a field to an ``attr.ib()`` is still possible to supply options like validators. > Setting it to any other value is treated like it was passed as ``attr.ib(default=value)`` -- passing an instance of ``attr.Factory`` also works as expected. > (`262 <https://github.com/python-attrs/attrs/issues/262>`_, `277 <https://github.com/python-attrs/attrs/issues/277>`_) >- Instances of classes created using ``attr.make_class()`` can now be pickled. > (`282 <https://github.com/python-attrs/attrs/issues/282>`_) >---- ### hypothesis 3.33.0 -> 3.37.0 >### 3.37.0 >------------------- >This is a deprecation release for some health check related features. >The following are now deprecated: >* Passing :attr:`~hypothesis.HealthCheck.exception_in_generation` to > :attr:`~hypothesis.settings.suppress_health_check`. This no longer does > anything even when passed - All errors that occur during data generation > will now be immediately reraised rather than going through the health check > mechanism. >* Passing :attr:`~hypothesis.HealthCheck.random_module` to > :attr:`~hypothesis.settings.suppress_health_check`. This hasn't done anything > for a long time, but was never explicitly deprecated. Hypothesis always seeds > the random module when running given tests, so this is no longer an error > and suppressing it doesn't do anything. >* Passing non-:class:`~hypothesis.HealthCheck` values in > :attr:`~hypothesis.settings.suppress_health_check`. This was previously > allowed but never did anything useful. >In addition, passing a non-iterable value as :attr:`~hypothesis.settings.suppress_health_check` >will now raise an error immediately (it would never have worked correctly, but >it would previously have failed later). Some validation error messages have >also been updated. >This work was funded by `Smarkets <https://smarkets.com/>`_. >------------------- >### 3.36.1 >------------------- >This is a yak shaving release, mostly concerned with our own tests. >While :func:`~python:inspect.getfullargspec` was documented as deprecated >in Python 3.5, it never actually emitted a warning. Our code to silence >this (nonexistent) warning has therefore been removed. >We now run our tests with ``DeprecationWarning`` as an error, and made some >minor changes to our own tests as a result. This required similar upstream >updates to :pypi:`coverage` and :pypi:`execnet` (a test-time dependency via >:pypi:`pytest-xdist`). >There is no user-visible change in Hypothesis itself, but we encourage you >to consider enabling deprecations as errors in your own tests. >------------------- >### 3.36.0 >------------------- >This release adds a setting to the public API, and does some internal cleanup: >- The :attr:`~hypothesis.settings.derandomize` setting is now documented (:issue:`890`) >- Removed - and disallowed - all 'bare excepts' in Hypothesis (:issue:`953`) >- Documented the :attr:`~hypothesis.settings.strict` setting as deprecated, and > updated the build so our docs always match deprecations in the code. >------------------- >### 3.35.0 >------------------- >This minor release supports constraining :func:`~hypothesis.strategies.uuids` >to generate :class:`~python:uuid.UUID`s of a particular version. >(:issue:`721`) >Thanks to Dion Misic for this feature. >------------------- >### 3.34.1 >------------------- >This patch updates the documentation to suggest >:func:`builds(callable) <hypothesis.strategies.builds>` instead of >:func:`just(callable()) <hypothesis.strategies.just>`. >------------------- >### 3.34.0 >------------------- >Hypothesis now emits deprecation warnings if you apply >:func:`given <hypothesis.given>` more than once to a target. >Applying :func:`given <hypothesis.given>` repeatedly wraps the target multiple >times. Each wrapper will search the space of of possible parameters separately. >This is equivalent but will be much more inefficient than doing it with a >single call to :func:`given <hypothesis.given>`. >For example, instead of >``given(booleans()) given(integers())``, you could write >``given(booleans(), integers())`` >------------------- >### 3.33.1 >------------------- >This is a bugfix release: >- :func:`~hypothesis.strategies.builds` would try to infer a strategy for > required positional arguments of the target from type hints, even if they > had been given to :func:`~hypothesis.strategies.builds` as positional > arguments (:issue:`946`). Now it only infers missing required arguments. >- An internal introspection function wrongly reported ``self`` as a required > argument for bound methods, which might also have affected > :func:`~hypothesis.strategies.builds`. Now it knows better. >------------------- That's it for now! Happy merging! 🤖

161: Scheduled weekly dependency update for week 46 r=mithrandi ## Updates Here's a list of all the updates bundled in this pull request. I've added some links to make it easier for you to find all the information you need. <table align="center"> <tr> <td><b>attrs</b></td> <td align="center">17.2.0</td> <td align="center">»</td> <td align="center">17.3.0</td> <td> <a href="https://pypi.python.org/pypi/attrs">PyPI</a> | <a href="https://pyup.io/changelogs/attrs/">Changelog</a> | <a href="http://www.attrs.org/">Homepage</a> </td> <tr> <td><b>cryptography</b></td> <td align="center">2.1.2</td> <td align="center">»</td> <td align="center">2.1.3</td> <td> <a href="https://pypi.python.org/pypi/cryptography">PyPI</a> | <a href="https://pyup.io/changelogs/cryptography/">Changelog</a> | <a href="https://github.com/pyca/cryptography">Repo</a> </td> <tr> <td><b>lxml</b></td> <td align="center">4.1.0</td> <td align="center">»</td> <td align="center">4.1.1</td> <td> <a href="https://pypi.python.org/pypi/lxml">PyPI</a> | <a href="https://pyup.io/changelogs/lxml/">Changelog</a> | <a href="http://lxml.de/">Homepage</a> | <a href="https://bugs.launchpad.net/lxml">Bugtracker</a> </td> <tr> <td><b>pytz</b></td> <td align="center">2017.2</td> <td align="center">»</td> <td align="center">2017.3</td> <td> <a href="https://pypi.python.org/pypi/pytz">PyPI</a> | <a href="http://pythonhosted.org/pytz">Homepage</a> | <a href="http://pythonhosted.org/pytz/">Docs</a> </td> </tr> </table> ## Changelogs ### attrs 17.2.0 -> 17.3.0 >### 17.3.0 >------------------- >Backward-incompatible Changes >^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >- Attributes are not defined on the class body anymore. > This means that if you define a class ``C`` with an attribute ``x``, the class will *not* have an attribute ``x`` for introspection anymore. > Instead of ``C.x``, use ``attr.fields(C).x`` or look at ``C.__attrs_attrs__``. > The old behavior has been deprecated since version 16.1. > (`253 <https://github.com/python-attrs/attrs/issues/253>`_) >Changes >^^^^^^^ >- ``super()`` and ``__class__`` now work on Python 3 when ``slots=True``. > (`102 <https://github.com/python-attrs/attrs/issues/102>`_, `226 <https://github.com/python-attrs/attrs/issues/226>`_, `269 <https://github.com/python-attrs/attrs/issues/269>`_, `270 <https://github.com/python-attrs/attrs/issues/270>`_, `272 <https://github.com/python-attrs/attrs/issues/272>`_) >- Added ``type`` argument to ``attr.ib()`` and corresponding ``type`` attribute to ``attr.Attribute``. > This change paves the way for automatic type checking and serialization (though as of this release ``attrs`` does not make use of it). > In Python 3.6 or higher, the value of ``attr.Attribute.type`` can alternately be set using variable type annotations > (see `PEP 526 <https://www.python.org/dev/peps/pep-0526/>`_). (`151 <https://github.com/python-attrs/attrs/issues/151>`_, `214 <https://github.com/python-attrs/attrs/issues/214>`_, `215 <https://github.com/python-attrs/attrs/issues/215>`_, `239 <https://github.com/python-attrs/attrs/issues/239>`_) >- The combination of ``str=True`` and ``slots=True`` now works on Python 2. > (`198 <https://github.com/python-attrs/attrs/issues/198>`_) >- ``attr.Factory`` is hashable again. (`204 > <https://github.com/python-attrs/attrs/issues/204>`_) >- Subclasses now can overwrite attribute definitions of their superclass. > That means that you can -- for example -- change the default value for an attribute by redefining it. > (`221 <https://github.com/python-attrs/attrs/issues/221>`_, `229 <https://github.com/python-attrs/attrs/issues/229>`_) >- Added new option ``auto_attribs`` to ``attr.s`` that allows to collect annotated fields without setting them to ``attr.ib()``. > Setting a field to an ``attr.ib()`` is still possible to supply options like validators. > Setting it to any other value is treated like it was passed as ``attr.ib(default=value)`` -- passing an instance of ``attr.Factory`` also works as expected. > (`262 <https://github.com/python-attrs/attrs/issues/262>`_, `277 <https://github.com/python-attrs/attrs/issues/277>`_) >- Instances of classes created using ``attr.make_class()`` can now be pickled. > (`282 <https://github.com/python-attrs/attrs/issues/282>`_) >---- ### lxml 4.1.0 -> 4.1.1 >### 4.1.1 >================== >* Rebuild with Cython 0.27.3 to improve support for Py3.7. That's it for now! Happy merging! 🤖

118: Update attrs to 17.3.0 r=mithrandi There's a new version of [attrs](https://pypi.python.org/pypi/attrs) available. You are currently using **17.2.0**. I have updated it to **17.3.0** These links might come in handy: <a href="https://pypi.python.org/pypi/attrs">PyPI</a> | <a href="https://pyup.io/changelogs/attrs/">Changelog</a> | <a href="http://www.attrs.org/">Homepage</a> ### Changelog > >### 17.3.0 >------------------- >Backward-incompatible Changes >^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >- Attributes are not defined on the class body anymore. > This means that if you define a class ``C`` with an attribute ``x``, the class will *not* have an attribute ``x`` for introspection anymore. > Instead of ``C.x``, use ``attr.fields(C).x`` or look at ``C.__attrs_attrs__``. > The old behavior has been deprecated since version 16.1. > (`253 <https://github.com/python-attrs/attrs/issues/253>`_) >Changes >^^^^^^^ >- ``super()`` and ``__class__`` now work on Python 3 when ``slots=True``. > (`102 <https://github.com/python-attrs/attrs/issues/102>`_, `226 <https://github.com/python-attrs/attrs/issues/226>`_, `269 <https://github.com/python-attrs/attrs/issues/269>`_, `270 <https://github.com/python-attrs/attrs/issues/270>`_, `272 <https://github.com/python-attrs/attrs/issues/272>`_) >- Added ``type`` argument to ``attr.ib()`` and corresponding ``type`` attribute to ``attr.Attribute``. > This change paves the way for automatic type checking and serialization (though as of this release ``attrs`` does not make use of it). > In Python 3.6 or higher, the value of ``attr.Attribute.type`` can alternately be set using variable type annotations > (see `PEP 526 <https://www.python.org/dev/peps/pep-0526/>`_). (`151 <https://github.com/python-attrs/attrs/issues/151>`_, `214 <https://github.com/python-attrs/attrs/issues/214>`_, `215 <https://github.com/python-attrs/attrs/issues/215>`_, `239 <https://github.com/python-attrs/attrs/issues/239>`_) >- The combination of ``str=True`` and ``slots=True`` now works on Python 2. > (`198 <https://github.com/python-attrs/attrs/issues/198>`_) >- ``attr.Factory`` is hashable again. (`204 > <https://github.com/python-attrs/attrs/issues/204>`_) >- Subclasses now can overwrite attribute definitions of their superclass. > That means that you can -- for example -- change the default value for an attribute by redefining it. > (`221 <https://github.com/python-attrs/attrs/issues/221>`_, `229 <https://github.com/python-attrs/attrs/issues/229>`_) >- Added new option ``auto_attribs`` to ``attr.s`` that allows to collect annotated fields without setting them to ``attr.ib()``. > Setting a field to an ``attr.ib()`` is still possible to supply options like validators. > Setting it to any other value is treated like it was passed as ``attr.ib(default=value)`` -- passing an instance of ``attr.Factory`` also works as expected. > (`262 <https://github.com/python-attrs/attrs/issues/262>`_, `277 <https://github.com/python-attrs/attrs/issues/277>`_) >- Instances of classes created using ``attr.make_class()`` can now be pickled. > (`282 <https://github.com/python-attrs/attrs/issues/282>`_) >---- *Got merge conflicts? Close this PR and delete the branch. I'll create a new PR for you.* Happy merging! 🤖

trendels added 4 commits November 4, 2017 10:13

Add failing test for make_class & pickle

9895a66

Fix pickling of classes created using make_class

86ed904

Fix failing flake8 test

dac0b45

Fix example in docs

6d96e05

hynek added this to the 17.3 milestone Nov 4, 2017

Add changelog entry.

29d92ab

python-attrs deleted a comment from codecov bot Nov 4, 2017

Add test for missing sys._getframe()

ac5324c

Make flake8 happy

93cd3f8

hynek approved these changes Nov 4, 2017

View reviewed changes

hynek merged commit c1adf5c into python-attrs:master Nov 4, 2017

trendels deleted the fix-pickle-make_class branch November 4, 2017 13:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix pickling for instances of classes created using make_class() #282

Fix pickling for instances of classes created using make_class() #282

trendels commented Nov 4, 2017 •

edited

Loading

codecov bot commented Nov 4, 2017 •

edited

Loading

hynek commented Nov 4, 2017

trendels commented Nov 4, 2017

hynek commented Nov 4, 2017

trendels commented Nov 4, 2017

hynek commented Nov 4, 2017

Fix pickling for instances of classes created using make_class() #282

Fix pickling for instances of classes created using make_class() #282

Conversation

trendels commented Nov 4, 2017 • edited Loading

codecov bot commented Nov 4, 2017 • edited Loading

Codecov Report

hynek commented Nov 4, 2017

trendels commented Nov 4, 2017

hynek commented Nov 4, 2017

trendels commented Nov 4, 2017

hynek commented Nov 4, 2017

trendels commented Nov 4, 2017 •

edited

Loading

codecov bot commented Nov 4, 2017 •

edited

Loading