Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop python 2 #3566

Merged
merged 8 commits into from
Dec 2, 2019
Merged

Drop python 2 #3566

merged 8 commits into from
Dec 2, 2019

Conversation

sphuber
Copy link
Contributor

@sphuber sphuber commented Nov 21, 2019

Fixes #3562

Copy link
Member

@greschd greschd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, that's the good stuff. Some minor comments below.

setup.json Show resolved Hide resolved
aiida/sphinxext/__init__.py Show resolved Hide resolved
aiida/transports/transport.py Show resolved Hide resolved
aiida/backends/general/abstractqueries.py Outdated Show resolved Hide resolved
aiida/backends/testimplbase.py Show resolved Hide resolved
aiida/manage/backup/backup_base.py Outdated Show resolved Hide resolved
aiida/orm/implementation/backends.py Outdated Show resolved Hide resolved
aiida/orm/implementation/backends.py Outdated Show resolved Hide resolved
aiida/parsers/parser.py Show resolved Hide resolved
aiida/tools/dbimporters/plugins/materialsproject.py Outdated Show resolved Hide resolved
@sphuber sphuber force-pushed the fix_3562_drop_python_2 branch 3 times, most recently from 6974652 to 66550b8 Compare November 22, 2019 13:44
@greschd greschd self-requested a review November 25, 2019 10:54
greschd
greschd previously approved these changes Nov 25, 2019
@sphuber sphuber changed the title Drop python 2 [BLOCKED] Drop python 2 Nov 25, 2019
@greschd
Copy link
Member

greschd commented Nov 25, 2019

I think the only thing left to do now is telling Jenkins to use py3 as well.

@sphuber
Copy link
Contributor Author

sphuber commented Nov 25, 2019

Yes, but I also wanted to merge some final things that should go into v1.0.1 that I want to release soon. Then we can merge this in develop which is then py3 only and this will become v1.1.0. We are about to accept the AEP that defines that version as the one where we drop py2 support.

@greschd
Copy link
Member

greschd commented Nov 25, 2019

@sphuber I've just stumbled across a very important little detail: In order for the "graceful fallback" on pip install to work for older versions, we need to first release another version that also specifies python_requires.

So that means, for the 1.0.1 release we should add python_requires=">=2.7" in setup.json.

No idea why that's the case, but just tested this with a different package. It's described here (last paragraph), although you could easily miss it.

@sphuber
Copy link
Contributor Author

sphuber commented Nov 26, 2019

Good catch! I will open a PR now

@sphuber sphuber mentioned this pull request Nov 26, 2019
@sphuber sphuber force-pushed the fix_3562_drop_python_2 branch 5 times, most recently from a4d7d7f to e351001 Compare November 29, 2019 18:15
Remove python 2 from supported version in `setup.json` and remove all
dependencies that were only necessary for `python<3.5`. Additionally
python 2 is removed from the build matrix on Travis.

Add explicit `python_requires` keyword to `setup.json` which will ensure
that `pip>=9` and other clients that support the metadata 1.2 spec
only install a compatible version for the current Python runtime when
installing the package.
@sphuber sphuber force-pushed the fix_3562_drop_python_2 branch from e351001 to 3517f4b Compare December 2, 2019 07:49
@sphuber sphuber changed the title [BLOCKED] Drop python 2 Drop python 2 Dec 2, 2019
@sphuber
Copy link
Contributor Author

sphuber commented Dec 2, 2019

@greschd if you'd like, I added three more commits on top since last time you reviewed. If you have the time you can just look at those commits.

@sphuber sphuber requested a review from greschd December 2, 2019 07:55
@sphuber sphuber force-pushed the fix_3562_drop_python_2 branch from 3517f4b to 8126314 Compare December 2, 2019 08:32
The removal of all future statements was performed with the following:

    find . -type f -not -path './.git*' -exec sed -i '/from __future__.*$/d' {} +

The `modernizer` hook is also removed from the pre-commit config.
The majority of adaptations was done with the following commands:

find . -type f -not -path './.git*' -exec sed -i '/\(import six\|from six\).*$/d' {} +
find . -type f -not -path './.git*' -exec sed -i '/\(six\.add_metaclass\).*$/d' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.integer_types/int/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.string_types/str/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.text_type/str/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.binary_type/bytes/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.moves.StringIO/io.StringIO/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.StringIO/io.StringIO/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.moves.BytesIO/io.BytesIO/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.BytesIO/io.BytesIO/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's/six.moves.range/range/g' {} +
find . -type f -not -path './.git*' -exec sed -i 's|six.iterkeys(\([^)]*\))|\1.keys()|' {} +
find . -type f -not -path './.git*' -exec sed -i 's|six.iteritems(\([^)]*\))|\1.items()|' {} +
find . -type f -not -path './.git*' -exec sed -i 's|six.itervalues(\([^)]*\))|\1.values()|' {} +
This is no longer needed for python 3.
In python 3 strings are stored as unicode by default so now that we
dropped pyhon 2 support there is no need to keep prefixing them with the
string encoding declaration `u`.
This is no longer necesary in python 3. To remove all the occurrences
of this usage, the following command was used:

    find . -type f -not -path './.git*' -exec \
    sed -i 's/super([^)]*,[^)]*)/super()/g' {} +

The regex tries to find all instances of `super()` where there is
content between the parentheses. The search for `[^)]*` means to look
for all characters except a closing parens, which essentially makes the
search non-greedy. Having just that would not be enough and we have to
explicitly add another such clause separated by a comma. This is the
exact format required by python 2 where the `super` call expects two
arguments, first being the class itself and the second the reference,
typically `self` or `cls. By making the regex more specific we avoid
matching cases like:

    def test_without_super(self):

which would be replaced to

    def test_without_super():

if we don't include the explicit comma in the regex between the parens.
@sphuber sphuber force-pushed the fix_3562_drop_python_2 branch from 8126314 to b8c458b Compare December 2, 2019 08:49
Hash arbitrary binary strings (str in Python 2, bytes in Python 3).
For compat reason between Python 2 and 3, this gets the same hash-type
as for unicode in Python 2, resp. str in Python 3."""
"""Hash arbitrary byte strings."""
return [_single_digest('str', bytes_obj)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not sure if we might want to change the "type specifier" here to be bytes - specific, now that py2 / py3 compatibility is no longer a concern.

Copy link
Member

@greschd greschd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! The bytes hashing prefix can also be discussed separately -- since that would invalidate current hashes, it probably shouldn't be buried in this merge.

@sphuber sphuber merged commit 0644e7d into aiidateam:develop Dec 2, 2019
@sphuber sphuber deleted the fix_3562_drop_python_2 branch December 2, 2019 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Drop python 2 support
2 participants