Enable ServerApp to discover ExtensionApps (and their config). (#180)

* enable extensionapp config to be discoverable from serveapp

* add some comments for future devs

* allow None in non-ExtensionApp extensions

* adjust tests to capture changes

* minor bug fixes

* renamed config pytest fixture

* standardize extension loading mechanism

* pass serverapp and extesnionapp to extensionapp handlers

* use static url prefix for static paths

* iniitalize all enabled extension, then load later

* split extension initialization and loading

* Upgrade examples to align on discovery branch

* Polish examples

* Launch example via python module

* Avoid to run initialisation methods twice

* Add main for simple_ext2 and simple_ext11

* minor changes to extension toggler

* adding some comments throughout the code

* move all CLI handling to the ServerApp

* remove old traits from extensionapp

* update tests

* update tests with changes to extensionapp

* fix examples entrypoint

* add test dependency: pytest-lazy-fixture

* unpin pytest

* import lazyfixture directly due to changes in pytest

* drop pytest-lazy-fixture

* cleaner error handling in init_server_extension

* minor clean up

* minor fixes after review

* add underscore as prefix to extension function

* remove load_jupyter_server_extension from examples

* minor typo in example comment

Co-authored-by: Eric Charles <eric@datalayer.io>

examples/simple/README.md

@@ -67,6 +67,12 @@ open http://localhost:8888/simple_ext1/redirect
 open http://localhost:8888/static/simple_ext1/favicon.ico
 ```
 
+You can also start the server extension with python modules.
+
+```bash
+python -m simple_ext1
+```
+
 ## Extension 1 and Extension 2
 
 The following command starts both the `simple_ext1` and `simple_ext2` extensions.
@@ -90,9 +96,12 @@ open http://localhost:8888/simple_ext2/params/test?var1=foo
 Optionally, you can copy `simple_ext1.json` and `simple_ext2.json` configuration to your env `etc` folder and start only Extension 1, which will also start Extension 2.
 
 ```bash
-pip uninstall -y jupyter_simple_ext && \
+pip uninstall -y jupyter_server_example && \
   python setup.py install && \
   cp -r ./etc $(dirname $(which jupyter))/..
+```
+
+```bash
 # Start the jupyter server extension simple_ext1, it will also load simple_ext2 because of load_other_extensions = True..
 # When you invoke with the entrypoint, the default url will be opened in your browser.
 jupyter simple-ext1
@@ -102,18 +111,20 @@ jupyter simple-ext1
 
 Stop any running server (with `CTRL+C`) and start with additional configuration on the command line.
 
-The provided settings via CLI will override the configuration that reside in the files (`jupyter_simple_ext1_config.py`...)
+The provided settings via CLI will override the configuration that reside in the files (`jupyter_server_example1_config.py`...)
 
 ```bash
 jupyter simple-ext1 --SimpleApp1.configA="ConfigA from command line"
 ```
 
-Check the log, it should return on startup something like the following base on the trait you have defined in the CLI and in the `jupyter_simple_ext1_config.py`.
+Check the log, it should return on startup print the Config object.
+
+The content of the Config is based on the trait you have defined via the `CLI` and in the `jupyter_server_example1_config.py`.
 
 ```
 [SimpleApp1] Config {'SimpleApp1': {'configA': 'ConfigA from file', 'configB': 'ConfigB from file', 'configC': 'ConfigC from file'}}
 [SimpleApp1] Config {'SimpleApp1': {'configA': 'ConfigA from file', 'configB': 'ConfigB from file', 'configC': 'ConfigC from file'}}
-[SimpleApp2] WARNING | Config option `configD` not recognized by `SimpleApp2`.  Did you mean `config_file`?
+[SimpleApp2] WARNING | Config option `configD` not recognized by `SimpleApp2`.  Did you mean one of: `configA, configB, configC`?
 [SimpleApp2] Config {'SimpleApp2': {'configD': 'ConfigD from file'}}
 [SimpleApp1] Config {'SimpleApp1': {'configA': 'ConfigA from command line', 'configB': 'ConfigB from file', 'configC': 'ConfigC from file'}}
 ```
@@ -133,27 +144,31 @@ Try with the above links to check that only Extension 2 is responding (Extension
 
 `Extension 11` extends `Extension 1` and brings a few more configs.
 
-Run `jupyter simple-ext11 --generate-config && vi ~/.jupyter/jupyter_config.py`.
-
-> TODO `--generate-config` returns an exception `"The ExtensionApp has not ServerApp "`
+```bash
+# TODO `--generate-config` returns an exception `"The ExtensionApp has not ServerApp "`
+jupyter simple-ext11 --generate-config && vi ~/.jupyter/jupyter_config.py`.
+```
 
 The generated configuration should contains the following.
 
 ```bash
-TBD
+# TODO
 ```
 
 The `hello`, `ignore_js` and `simple11_dir` are traits defined on the SimpleApp11 class.
 
 It also implements additional flags and aliases for these traits.
 
-+ The `--hello` flag will log on startup `Hello Simple11 - You have provided the --hello flag or defined a c.SimpleApp1.hello == True`.
-+ The `--simple11-dir` alias will set `SimpleExt11.simple11_dir` settings.
+- The `--hello` flag will log on startup `Hello Simple11 - You have provided the --hello flag or defined a c.SimpleApp1.hello == True`
+- The `ignore_js` flag
+- The `--simple11-dir` alias will set `SimpleExt11.simple11_dir` settings
 
 Stop any running server and then start the simple-ext11.
 
 ```bash
 jupyter simple-ext11 --hello --simple11-dir any_folder
+# You can also launch with a module
+python -m simple_ext11 --hello
 # TODO FIX the following command, simple11 does not work launching with jpserver_extensions parameter.
 jupyter server --ServerApp.jpserver_extensions="{'simple_ext11': True}" --hello --simple11-dir any_folder
 ```

examples/simple/jupyter_simple_ext1_config.py

@@ -1,3 +1,4 @@
 c.SimpleApp1.configA = 'ConfigA from file'
 c.SimpleApp1.configB = 'ConfigB from file'
 c.SimpleApp1.configC = 'ConfigC from file'
+c.SimpleApp1.configD = 'ConfigD from file'

examples/simple/package.json

@@ -1,5 +1,5 @@
 {
-  "name": "jupyter-simple-ext",
+  "name": "jupyter-server-example",
   "version": "0.0.1",
   "private": true,
   "scripts": {

examples/simple/setup.py

@@ -23,9 +23,9 @@ def add_data_files(path):
     return data_files
 
 setuptools.setup(
-    name = 'jupyter_simple_ext',
+    name = 'jupyter_server_example',
     version = VERSION,
-    description = 'Jupyter Simple Extension',
+    description = 'Jupyter Server Example',
     long_description = open('README.md').read(),
     packages = find_packages(),
     python_requires = '>=3.5',

examples/simple/simple_ext1/__init__.py

@@ -1,8 +1,8 @@
 from .application import SimpleApp1
 
-def _jupyter_server_extension_paths():
-    return [
-        {'module': 'simple_ext1'}
-    ]
 
-load_jupyter_server_extension = SimpleApp1.load_jupyter_server_extension
+def _jupyter_server_extension_paths():
+    return [{
+        'module': 'simple_ext1.application',
+        'app': SimpleApp1
+    }]

examples/simple/simple_ext1/__main__.py

@@ -0,0 +1,4 @@
+from .application import main
+
+if __name__ == "__main__":
+    main()

examples/simple/simple_ext1/application.py

@@ -1,19 +1,19 @@
 import os, jinja2
 from traitlets import Unicode
 from jupyter_server.extension.application import ExtensionApp, ExtensionAppJinjaMixin
-from .handlers import (DefaultHandler, RedirectHandler, 
+from .handlers import (DefaultHandler, RedirectHandler,
   ParameterHandler, TemplateHandler, TypescriptHandler, ErrorHandler)
 
 DEFAULT_STATIC_FILES_PATH = os.path.join(os.path.dirname(__file__), "static")
 DEFAULT_TEMPLATE_FILES_PATH = os.path.join(os.path.dirname(__file__), "templates")
 
 class SimpleApp1(ExtensionAppJinjaMixin, ExtensionApp):
-    
+
     # The name of the extension.
     extension_name = "simple_ext1"
 
-    # Te url that your extension will serve its homepage.
-    default_url = '/simple_ext1/default'
+    # The url that your extension will serve its homepage.
+    extension_url = '/simple_ext1/default'
 
     # Should your extension expose other server extensions when launched directly?
     load_other_extensions = True

examples/simple/simple_ext11/__init__.py

@@ -1,8 +1,10 @@
-from .application import SimpleApp1
+from .application import SimpleApp11
+
 
 def _jupyter_server_extension_paths():
     return [
-        {'module': 'simple_ext1'}
-    ]
-
-load_jupyter_server_extension = SimpleApp1.load_jupyter_server_extension
+        {
+            'module': 'simple_ext11.application',
+            'app': SimpleApp11
+        }
+    ]

examples/simple/simple_ext11/__main__.py

@@ -0,0 +1,4 @@
+from .application import main
+
+if __name__ == "__main__":
+    main()

examples/simple/simple_ext11/application.py

@@ -13,12 +13,12 @@ class SimpleApp11(SimpleApp1):
     aliases.update({
         'simple11-dir': 'SimpleApp11.simple11_dir',
     })
-    
+
     # The name of the extension.
     extension_name = "simple_ext11"
 
     # Te url that your extension will serve its homepage.
-    default_url = '/simple_ext11/default'
+    extension_url = '/simple_ext11/default'
 
     # Local path to static files directory.
     static_paths = [
@@ -37,12 +37,12 @@ class SimpleApp11(SimpleApp1):
 
     hello = Bool(False,
         config=True,
-        help='Say hello', 
+        help='Say hello',
     )
 
     ignore_js = Bool(False,
         config=True,
-        help='Ignore Javascript', 
+        help='Ignore Javascript',
     )
 
     @observe('ignore_js')
@@ -56,8 +56,8 @@ def simple11_dir_formatted(self):
 
     def initialize_settings(self):
         self.log.info('hello: {}'.format(self.hello))
-        if self.config['hello'] == True:
-            self.log.info("Hello Simple11 - You have provided the --hello flag or defined 'c.SimpleApp1.hello == True' in jupyter_server_config.py")
+        if self.hello == True:
+            self.log.info("Hello Simple11: You have launched with --hello flag or defined 'c.SimpleApp1.hello == True' in your config file")
         self.log.info('ignore_js: {}'.format(self.ignore_js))
         super().initialize_settings()
 

examples/simple/simple_ext2/__init__.py

@@ -1,8 +1,10 @@
 from .application import SimpleApp2
 
+
 def _jupyter_server_extension_paths():
     return [
-        {'module': 'simple_ext2'},
-    ]
-
-load_jupyter_server_extension = SimpleApp2.load_jupyter_server_extension
+        {
+            'module': 'simple_ext2.application',
+            'app': SimpleApp2
+        },
+    ]

examples/simple/simple_ext2/__main__.py

@@ -0,0 +1,4 @@
+from .application import main
+
+if __name__ == "__main__":
+    main()

examples/simple/simple_ext2/application.py

@@ -7,15 +7,15 @@
 DEFAULT_TEMPLATE_FILES_PATH = os.path.join(os.path.dirname(__file__), "templates")
 
 class SimpleApp2(ExtensionAppJinjaMixin, ExtensionApp):
-    
+
     # The name of the extension.
     extension_name = "simple_ext2"
 
     # Te url that your extension will serve its homepage.
-    default_url = '/simple_ext2'
+    extension_url = '/simple_ext2'
 
     # Should your extension expose other server extensions when launched directly?
-    load_other_extensions = False
+    load_other_extensions = True
 
     # Local path to static files directory.
     static_paths = [

jupyter_server/base/handlers.py

@@ -154,7 +154,7 @@ def cookie_name(self):
             self.request.host
         ))
         return self.settings.get('cookie_name', default_cookie_name)
-    
+
     @property
     def logged_in(self):
         """Is a user currently logged in?"""
@@ -203,23 +203,23 @@ def log(self):
     def jinja_template_vars(self):
         """User-supplied values to supply to jinja templates."""
         return self.settings.get('jinja_template_vars', {})
-    
+
     #---------------------------------------------------------------
     # URLs
     #---------------------------------------------------------------
-    
+
     @property
     def version_hash(self):
         """The version hash to use for cache hints for static files"""
         return self.settings.get('version_hash', '')
-    
+
     @property
     def mathjax_url(self):
         url = self.settings.get('mathjax_url', '')
         if not url or url_is_absolute(url):
             return url
         return url_path_join(self.base_url, url)
-    
+
     @property
     def mathjax_config(self):
         return self.settings.get('mathjax_config', 'TeX-AMS-MML_HTMLorMML-full,Safe')
@@ -241,27 +241,27 @@ def contents_js_source(self):
         self.log.debug("Using contents: %s", self.settings.get('contents_js_source',
             'services/contents'))
         return self.settings.get('contents_js_source', 'services/contents')
-    
+
     #---------------------------------------------------------------
     # Manager objects
     #---------------------------------------------------------------
-    
+
     @property
     def kernel_manager(self):
         return self.settings['kernel_manager']
 
     @property
     def contents_manager(self):
         return self.settings['contents_manager']
-    
+
     @property
     def session_manager(self):
         return self.settings['session_manager']
-    
+
     @property
     def terminal_manager(self):
         return self.settings['terminal_manager']
-    
+
     @property
     def kernel_spec_manager(self):
         return self.settings['kernel_spec_manager']
@@ -273,7 +273,7 @@ def config_manager(self):
     #---------------------------------------------------------------
     # CORS
     #---------------------------------------------------------------
-    
+
     @property
     def allow_origin(self):
         """Normal Access-Control-Allow-Origin"""
@@ -310,7 +310,7 @@ def set_default_headers(self):
 
         if self.allow_credentials:
             self.set_header("Access-Control-Allow-Credentials", 'true')
-    
+
     def set_attachment_header(self, filename):
         """Set Content-Disposition: attachment header