Merge pull request #334 from moremoban/dev

Release 0.6.2

.moban.cd/changelog.yml

@@ -3,6 +3,14 @@ organisation: moremoban
 releases:
   - changes:
       - action: Added
+        details:
+          - "`#322`: Implicit targets with template extensions default to copy"
+          - "`#257`: '-e' to load extensions for template engines, i.e. jinja2"
+          - "`#333`: command line template fails with version 0.6.1"
+    date: 15.09.2019
+    version: 0.6.2
+  - changes:
+      - action: Fixed
         details:
           - "`#328`: update backward compatibility"
     date: 10.09.2019
@@ -14,7 +22,7 @@ releases:
           - "`#185`: -v will enable moban application logging for development. And -V is for version."
           - "`#325`: -vv show debug trace"
           - "`#126`: Allow mobanfile to include data from arbitrary config files"
-          - "`#256`: jinja2-cli parity: '-D hello=world' to define custom variable on cli"
+          - "`#256`: jinja2-cli parity: '-d hello=world' to define custom variable on cli"
       - action: Updated
         details:
           - "`#275`: fix moban 0.4.5 test failures on openSUSE Tumbleweed"

.moban.cd/moban.yml

@@ -4,9 +4,9 @@ organisation: moremoban
 author: C. W.
 contact: wangc_2011@hotmail.com
 license: MIT
-version: 0.6.1
-current_version: 0.6.1
-release: 0.6.1
+version: 0.6.2
+current_version: 0.6.2
+release: 0.6.2
 branch: master
 master: index
 command_line_interface: "moban"

CHANGELOG.rst

@@ -1,12 +1,25 @@
 Change log
 ================================================================================
 
-0.6.1 - 10.09.2019
+0.6.2 - 15.09.2019
 --------------------------------------------------------------------------------
 
 Added
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+#. `#322 <https://github.com/moremoban/moban/issues/322>`_: Implicit targets
+   with template extensions default to copy
+#. `#257 <https://github.com/moremoban/moban/issues/257>`_: '-e' to load
+   extensions for template engines, i.e. jinja2
+#. `#333 <https://github.com/moremoban/moban/issues/333>`_: command line
+   template fails with version 0.6.1
+
+0.6.1 - 10.09.2019
+--------------------------------------------------------------------------------
+
+Fixed
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 #. `#328 <https://github.com/moremoban/moban/issues/328>`_: update backward
    compatibility
 
@@ -24,7 +37,7 @@ Added
 #. `#126 <https://github.com/moremoban/moban/issues/126>`_: Allow mobanfile to
    include data from arbitrary config files
 #. `#256 <https://github.com/moremoban/moban/issues/256>`_: jinja2-cli parity:
-   '-D hello=world' to define custom variable on cli
+   '-d hello=world' to define custom variable on cli
 
 Updated
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

README.rst

@@ -1,5 +1,5 @@
 ================================================================================
-moban - 模板 Yet another jinja2 cli command for static text generation
+moban - 模板 Any template, any data in any location
 ================================================================================
 
 .. image:: https://api.travis-ci.org/moremoban/moban.svg?branch=master
@@ -17,26 +17,25 @@ moban - 模板 Yet another jinja2 cli command for static text generation
 .. image:: https://img.shields.io/gitter/room/gitterHQ/gitter.svg
    :target: https://gitter.im/chfw_moban/Lobby
 
-:Author: C.W.
+:Author: C.W. and its contributors (See contributors.rst)
 :Issues: http://github.com/moremoban/moban/issues
 :License: MIT
 :Version: |version|
 :Generated: |today|
 
 
-**moban** brings the high performance template engine (JINJA2) for web into
-static text generation. It is used in `pyexcel` and `coala` project to keep
-documentation consistent across the documentations of individual libraries.
+**moban** started with bringing the high performance template engine (JINJA2) for web
+into static text generation. It has been used in `pyexcel` and `coala` project to keep
+documentation consistent across the documentations of individual libraries in the same
+organisation.
 
-Our vision is: any template, any data in any location. Our current architecture
-enables moban to plugin any python template engine: mako, handlebars, velocity,
-haml, slim and tornado, to plugin any data format: json and yaml, and in
-any location: zip, git, pypi package, s3, etc. Please
-look at our issues. We have many more template engines and data format on the
-road map.
+**moban** can use other python template engine: mako, handlebars, velocity,
+haml, slim and tornado, can read other data format: json and yaml, and can access both
+template file and configuration file in
+any location: zip, git, pypi package, s3, etc.
 
-Documentation
-=================================================================================
+Please look at our issues. We have many more template engines and data format on the
+road map.
 
 All use cases are documented `here <http://moban.readthedocs.org/en/latest/#tutorial>`_
 
@@ -104,12 +103,10 @@ Please note that data.yml will take precedence over environment variables.
 Work with files in a git repo
 ================================================================================
 
-Please install `gitfs2 <https://github.com/moremoban/gitfs2>`_::
+`gitfs2 <https://github.com/moremoban/gitfs2>`_ is installed by default since v0.6.1
 
-    $ pip install gitfs2
 
-
-And then you can do the following:
+You can do the following with moban:
 
 .. code-block:: bash
 
@@ -127,12 +124,9 @@ And then you can do the following:
 Work with files in a python package
 ================================================================================
 
-Please install `pypifs <https://github.com/moremoban/pypifs>`_::
-
-    $ pip install pypifs
+`pypifs <https://github.com/moremoban/pypifs>`_ is installed by default since v0.6.1
 
-
-And then you can do the following:
+You can do the following with moban:
 
 .. code-block:: bash
 
@@ -156,6 +150,9 @@ Please install `fs-s3fs <https://github.com/PyFilesystem/s3fs>`_::
 
     $ pip install fs-s3fs
 
+
+Then you can access your files in s3 bucket:
+
 .. code-block:: bash
 
     $ moban -c s3://${client_id}:${client_secrect}@moremoban/s3data.yml \

docs/conf.py

@@ -25,9 +25,9 @@
 copyright = '2017-2019 Onni Software Ltd.'
 author = 'C. W.'
 # The short X.Y version
-version = '0.6.1'
+version = '0.6.2'
 # The full version, including alpha/beta/rc tags
-release = '0.6.1'
+release = '0.6.2'
 
 # -- General configuration ---------------------------------------------------
 

docs/level-15-copy-templates-as-target/.moban.yml

@@ -8,6 +8,7 @@ targets:
   - output: target_without_template_type
     template: file_extension_will_trigger.copy
   - target_in_short_form: as_long_as_this_one_has.copy
+  - output_is_copied.same_file_extension: when_source_have.same_file_extension
   - output: "misc-1-copying/can-create-folder/if-not-exists.txt"
     template: file-in-template-sources-folder.txt
     template_type: copy

docs/level-15-copy-templates-as-target/README.rst

@@ -23,7 +23,7 @@ Shorthand syntax::
 
     targets:
        - explicit: template_file.copy
-
+       - output_is_copied.same_file_extension: when_source_have.same_file_extension
 
 No implicit nor short hand syntax for the following directory copying unless
 you take a look at `force-template-type`. When you read
@@ -64,6 +64,7 @@ Here is example moban file for copying::
       - output: target_without_template_type
         template: file_extension_will_trigger.copy
       - target_in_short_form: as_long_as_this_one_has.copy
+      - output_is_copied.same_file_extension: when_source_have.same_file_extension
       - output: "misc-1-copying/can-create-folder/if-not-exists.txt"
         template: file-in-template-sources-folder.txt
         template_type: copy

docs/level-15-copy-templates-as-target/template-sources/when_source_have.same_file_extension

@@ -0,0 +1 @@
+it is implicit copy as well

docs/level-3-data-override/README.rst

@@ -34,12 +34,12 @@ command to launch it:
 
 
 New development
-================================================================================
+--------------------------------------------------------------------------------
 
 Since verison 0.6.0, `overrides` syntax support two more use cases:
 
 1 override more than one configuration file
----------------------------------------------
+*********************************************
 
 For example::
 
@@ -48,7 +48,7 @@ For example::
       - config-file-b.yaml
 
 2 override more than one configuration file
----------------------------------------------
+********************************************
 
 For example::
 

moban/_version.py

@@ -1,2 +1,2 @@
-__version__ = "0.6.1"
+__version__ = "0.6.2"
 __author__ = "C. W."

moban/constants.py

@@ -37,7 +37,9 @@
 LABEL_VERSION = "version"
 LABEL_GROUP = "group"
 LABEL_DEFINE = "define"
+LABEL_EXTENSION = "extension"
 CLI_DICT = "user_dict"
+EXTENSION_DICT = "user_extensions"
 
 DEFAULT_CONFIGURATION_DIRNAME = ".%s.cd" % PROGRAM_NAME
 DEFAULT_TEMPLATE_DIRNAME = ".%s.td" % PROGRAM_NAME

moban/core/moban_factory.py

@@ -1,6 +1,7 @@
 import os
 import sys
 import logging
+from collections import defaultdict
 
 from moban import utils, reporter, constants, exceptions, file_system
 from fs.errors import ResourceNotFound
@@ -18,11 +19,24 @@
 class MobanFactory(PluginManager):
     def __init__(self):
         super(MobanFactory, self).__init__(constants.TEMPLATE_ENGINE_EXTENSION)
-        self.extensions = {}
+        self.extensions = defaultdict(set)
         self.options_registry = {}
 
     def register_extensions(self, extensions):
-        self.extensions.update(extensions)
+        for user_template_type in extensions.keys():
+            template_type = self.get_primary_key(user_template_type)
+
+            log.debug(
+                "Registering extensions: {0}={1}".format(
+                    user_template_type, extensions[user_template_type]
+                )
+            )
+            if template_type in self.extensions:
+                self.extensions[template_type] = self.extensions[
+                    user_template_type
+                ].union(extensions[user_template_type])
+            else:
+                self.extensions[template_type] = extensions[user_template_type]
 
     def register_options(self, template_types):
         # need the value of 'template_types'
@@ -34,6 +48,7 @@ def get_engine(self, template_type, template_dirs, context_dirs):
         template_dirs = utils.verify_the_existence_of_directories(
             template_dirs
         )
+
         if template_type in self.options_registry:
             custom_engine_spec = self.options_registry[template_type]
             engine_cls = self.load_me_now(
@@ -43,7 +58,10 @@ def get_engine(self, template_type, template_dirs, context_dirs):
         else:
             engine_cls = self.load_me_now(template_type)
             engine_extensions = self.extensions.get(template_type)
-            options = dict(extensions=engine_extensions)
+            if engine_extensions:
+                options = dict(extensions=list(engine_extensions))
+            else:
+                options = dict()
         template_fs = file_system.get_multi_fs(template_dirs)
         engine = engine_cls(template_fs, options)
         return MobanEngine(template_fs, context_dirs, engine)

moban/definitions.py

@@ -1,5 +1,9 @@
+import logging
+
 from moban import constants
 
+LOG = logging.getLogger(__name__)
+
 
 class TemplateTarget(object):
     def __init__(
@@ -16,6 +20,7 @@ def __init__(
         self.output = self.original_output
 
         self.set_template_type(template_type)
+        LOG.info("create a target {}".format(self))
 
     def set_template_type(self, new_template_type):
         self.template_type = new_template_type

moban/jinja2/engine.py

@@ -2,6 +2,7 @@
 import logging
 from importlib import import_module
 
+import fs.errors
 from moban import constants, file_system
 from jinja2 import Template, Environment
 from lml.loader import scan_plugins_regex
@@ -11,7 +12,7 @@
 from jinja2_fsloader import FSLoader
 
 JINJA2_LIBRARIES = "^moban_jinja2_.+$"
-JINJA2_EXENSIONS = [
+JINJA2_EXTENSIONS = [
     "moban.jinja2.filters.repr",
     "moban.jinja2.filters.github",
     "moban.jinja2.filters.text",
@@ -87,7 +88,6 @@ def __init__(self, template_fs, options=None):
                     # because it is modified here
                     env_params["extensions"] += extensions
                     import_module_of_extension(extensions)
-
             env_params.update(options)
         self.jj2_environment = Environment(**env_params)
         for filter_name, filter_function in FILTERS.get_all():
@@ -117,10 +117,13 @@ def get_template(self, template_file):
         """
         try:
             template = self.jj2_environment.get_template(template_file)
+            return template
         except TemplateNotFound:
-            content = file_system.read_unicode(template_file)
-            return Template(content)
-        return template
+            try:
+                content = file_system.read_unicode(template_file)
+                return self.jj2_environment.from_string(content)
+            except fs.errors.ResourceNotFound:
+                return self.jj2_environment.from_string(template_file)
 
     def get_template_from_string(self, string):
         return Template(string)
@@ -143,7 +146,7 @@ def apply_template(self, template, data, output):
 
 
 def load_jinja2_extensions():
-    scan_plugins_regex(JINJA2_LIBRARIES, "moban", None, JINJA2_EXENSIONS)
+    scan_plugins_regex(JINJA2_LIBRARIES, "moban", None, JINJA2_EXTENSIONS)
 
 
 def is_extension_list_valid(extensions):