Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added tests for newer versions of ipython and python #44

Closed
22 changes: 16 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,20 @@
language: python
sudo: false
python:
- "2.7"
- "3.4"
env:
- IPYTHON=3
- IPYTHON=4

matrix:
include:
- python: 2.7
env: IPYTHON=3
- python: 2.7
env: IPYTHON=4

- python: 3.4
env: IPYTHON=3
- python: 3.4
env: IPYTHON=4

- python: 3.6
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does IPython 6 require 3.6 or higher? My inclination would be to test on the oldest supported version of Python, since that's the most likely to surface version compatibility bugs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not, though it's probably a good idea to test a newer version of python too.

env: IPYTHON=6

addons:
postgresql: "9.3"
Expand All @@ -16,6 +25,7 @@ install:
script:
- if [[ $TRAVIS_PYTHON_VERSION = '2.7' ]]; then tox -e py27-ipython$IPYTHON,flake8; fi
- if [[ $TRAVIS_PYTHON_VERSION = '3.4' ]]; then tox -e py34-ipython$IPYTHON,flake8; fi
- if [[ $TRAVIS_PYTHON_VERSION = '3.6' ]]; then tox -e py36-ipython$IPYTHON; fi

branches:
only:
Expand Down
6 changes: 5 additions & 1 deletion pgcontents/pgmanager.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,11 @@ def _checkpoints_class_default(self):
return PostgresCheckpoints

def _checkpoints_kwargs_default(self):
kw = super(PostgresContentsManager, self)._checkpoints_kwargs_default()
try:
klass = PostgresContentsManager
kw = super(klass, self)._checkpoints_kwargs_default()
except AttributeError:
kw = {'parent': self, 'log': self.log}
kw.update({
'create_user_on_startup': self.create_user_on_startup,
'crypto': self.crypto,
Expand Down
40 changes: 36 additions & 4 deletions pgcontents/tests/test_pgcontents_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
)
from ..utils.ipycompat import (
APITest, Config, FileContentsManager, GenericFileCheckpoints, to_os_path,
)
IPY3)
from ..utils.sync import walk, walk_dirs


Expand Down Expand Up @@ -163,6 +163,38 @@ def test_list_checkpoints_sorting(self):
)
)

# skip test as it is not satisfyable across supported notebook versions
def test_delete_non_empty_dir(self):
# ContentsManager has different behaviour in notebook 5.5+
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, from reading the discussion on jupyter/notebook#3108, it seems like the reason notebook now allows deleting non-empty directories is that they switched to using send2trash for deletions by default, which means it's possible for a user to recover a lost directory that they accidentally deleted. Since deletions in pgcontents are still irreversible, my inclination would be to continue to test that we don't support deletion of nonempty directories when using PostgresContentsManager.

I think the easiest way to test that ensure that behavior is to make this method conditional on the value of self.config.NotebookApp.contents_manager_class. If it's a subclass of PostgresContentsManager, we should keep the test for the old behavior. For FileContentsManager we can just call super() to test that we haven't somehow broken the default behavior from upstream. For HybridContentsManager, the expected behavior would depend on how the sub-managers are configured, so the right test would probably just be to assert that we prevent deletions in the subdirectory owned by the postgres manager.

Apologies for the somewhat complex feedback here: this suite is a bit convoluted, but since this is testing the code that controls whether/how we delete user data on Quantopian, I want to err on the side of caution. If you prefer, I can also open a PR against this branch with the changes I have in mind for the suite.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's probably for the best

# https://github.com/jupyter/notebook/pull/3108

# Old behaviour
# from notebook.tests.launchnotebook import assert_http_error

if isinstance(self.notebook.contents_manager, PostgresContentsManager):
_test_delete_non_empty_dir_fail(self)
else:
_test_delete_non_empty_dir_pass(self)


def _test_delete_non_empty_dir_fail(self):
if IPY3:
return
from notebook.tests.launchnotebook import assert_http_error
with assert_http_error(400):
self.api.delete(u'å b')


def _test_delete_non_empty_dir_pass(self):
if IPY3:
return
from notebook.tests.launchnotebook import assert_http_error
# Test that non empty directory can be deleted
self.api.delete(u'å b')
# Check if directory has actually been deleted
with assert_http_error(404):
self.api.list(u'å b')


def postgres_contents_config():
"""
Expand Down Expand Up @@ -441,12 +473,12 @@ def _method(self, api_path, *args):
'isfile',
'isdir',
]
l = locals()
locs = locals()
for method_name in __methods_to_multiplex:
l[method_name] = __api_path_dispatch(method_name)
locs[method_name] = __api_path_dispatch(method_name)
del __methods_to_multiplex
del __api_path_dispatch
del l
del locs

# Override to not delete the root of the file subsystem.
def test_delete_dirs(self):
Expand Down
6 changes: 3 additions & 3 deletions pgcontents/utils/ipycompat.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"""
import IPython

SUPPORTED_VERSIONS = {3, 4, 5}
SUPPORTED_VERSIONS = {3, 4, 5, 6}
IPY_MAJOR = IPython.version_info[0]
if IPY_MAJOR not in SUPPORTED_VERSIONS:
raise ImportError("IPython version %d is not supported." % IPY_MAJOR)
Expand Down Expand Up @@ -47,8 +47,8 @@
)
else:
import notebook
if notebook.version_info[0] >= 5:
raise ImportError("Notebook versions 5 and up are not supported.")
if notebook.version_info[0] >= 6:
raise ImportError("Notebook versions 6 and up are not supported.")

from traitlets.config import Config
from notebook.services.contents.checkpoints import (
Expand Down
5 changes: 3 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@ def main():
install_requires=reqs,
extras_require={
'test': test_reqs,
'ipy3': ['ipython[test,notebook]<4.0'],
'ipy4': ['ipython<6.0', 'notebook[test]>=4.0,<5.0'],
'ipy3': ['ipython[test,notebook]<4.0', 'tornado<5.0'],
'ipy4': ['ipython<6.0', 'notebook[test]>=4.0,<5.0', 'tornado<5.0'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the tornado change? I wouldn't expect pgcontents to care much about the version of tornado.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pgcontents doesn't care, but notebook very much does and older versions of notebook don't pin tornado to an upper bound. tornado 5 did enough backward incompatible changes to basically make it not work unless you are running a substantially closer to master version of notebook.

'ipy6': ['ipython>=6.0', 'notebook[test]>=5.0']
},
scripts=[
'bin/pgcontents',
Expand Down
5 changes: 4 additions & 1 deletion tox.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[tox]
envlist=py{27,34}-ipython{3,4},flake8
envlist=py{27,34}-ipython{3,4},py36-ipython6,flake8
skip_missing_interpreters=True

[testenv]
Expand All @@ -9,11 +9,14 @@ whitelist_externals =
deps =
py{27,34}-ipython3: .[test,ipy3]
py{27,34}-ipython4: .[test,ipy4]
py36-ipython6: .[test,ipy6]
flake8: flake8
notest: .[ipy4]

commands =
py{27,34}-ipython{3,4}: -createdb pgcontents_testing
py{27,34}-ipython{3,4}: nosetests pgcontents/tests
py36-ipython6: -createdb pgcontents_testing
py36-ipython6: nosetests pgcontents/tests
flake8: flake8 pgcontents
notest: python -c 'from pgcontents.utils.ipycompat import *'