In celebration of becoming an official OSGeo Community Project,
a version 1.0.0
release is now out!
Support for Python 2.7 has now been dropped. See #166 - Drop Python 2.7 support. The codebase has had all Python2 specific code removed - see #188.
Breaking Change - in the
MapfileToDict
class the parametertransformerClass
has been renamedtransformer_class
. An example of how to fix this is shown below:from mappyfile.transformer import MapfileToDict from mappyfile_colors import ColorsTransformer m = MapfileToDict( include_position=True, include_comments=True, # replace the following parameter # transformerClass=ColorsTransformer, transformer_class=ColorsTransformer, conversion_type=None, include_color_names=True, )
Breaking Change -
LAYER
DATA
has been changed in the schema from a list to a simple string.layer = { '__type__': 'layer', // pre v1 the data clause had to be in a list // 'data': ['/path/to/data'] // in v1 this should now be a string 'data': '/path/to/data' } mappyfile.dumps(layer)
Support added for lark_cython - see #178 - thanks @erezsh. To use
lark_cython
is as simple as installing the option withpip
:pip install mappyfile[lark_cython]
All mappyfile dicts now have human readable output when displayed as a string:
mf = mappyfile.open("./docs/examples/before.map") print(mf) # previous output # DefaultOrderedDict(<class 'mappyfile.ordereddict.CaseInsensitiveOrderedDict'>, CaseInsensitiveOrderedDict([('__type__', 'map'),.. # new output { "__type__": "map", "layers": [ { "__type__": "layer", "name": "Layer1", "type": "POLYGON" },
Approach to resolving JSON references updated due to the deprecated
jsonschema.RefResolver
- see this link, the associated JSONSchema pull request and the migration approach.
Other improvements and fixes in the v1.0.0 release:
- #196 - Code base fixes for
Prospector
warnings - #195 - Update test suite from latest msautotests
- #194 - Docs overhaul in preparation for v1 release
- #193 - Update to
jsonschema
v4 and replace deprecatedRefResolver
- #191 - Simplify processing of comments
- #189 - Add type hints to the code base
- #153 - Support querying items without the given key in
utils.findunique()
- thanks @DonQueso89 for fix - Schema fixes for
grid
,label
,style
,leader
, addflatgeobuf
- Code reformatted using black
Resolution of long-standing parsing issues, and all msautotest examples now pass successfully:
- Fix ""ResourceWarning: unclosed"" when reading mapfile.lark in Python 3.10
- #151 - Updates for COMPOSITE blocks
- #150 - Unknown COMPOP "SOFT-LIGHT" and error with several
- lines with COMPFILTER with validate
- Schema fixes for GRID LABELFORMAT and set max versions for MAP DATAPATTERN and TEMPLATEPATTERN
- Allow TRUE/FALSE values for OUTPUTFORMAT TRANSPARENT
- #147 - Create list objects for containers when modifying dicts
- #146 - Add COMPOSITE validation
- #145 - layers.insert fails with dict error
- #144 - Invalid value in COMPOSITE - 'compfilter'
- #140 - New feature: group complex types at the end
- #137 - Checking mapfile dict properties creates invalid empty dictionaries
- #119 - STYLE GEOMTRANSFORM 'labelcenter'
- #143 - Automate schema building
- #142 - Allow newer versions of jsonschema for py3
- #141 - Update and fix Continuous Integration
- #139 - Feature: align values in column
- #138 - Update schema based on new Mapfile validation rules
- Adds a new
mappyfile.create
function to allow creation of Mapfile objects with default values - Update the Mapfile schema to include
default
values for keywords
- Add the "idw" to
LAYER
CONNECIONTYPE
- Correct "minVersion" of
LABEL
EXPRESSION
- Add validation to
LEGEND
LABELS
- Add correct validation for
MAP
LEGEND
andOUTPUTFORMAT
- Add "byte" to
OUTPUTFORMAT
IMAGEMODE
- Add "maxVersion" to
WEB
LOG
- #120 - Expression list element with apostrophe throws error
- #118 - LABEL -> FONT and LABEL -> POSITION gives errors in validate when attributes are used
- Allow any version of lark-parser > 0.9 to be used
- Fixes for requirements for Python 2.7
- #115 - Fix for issue #109 (OFFSET numeric and attribute pairs)
- #114 - Style OFFSET: mixed attribute and numerical value fail to parse
- Schemas updated to include
minVersion
andmaxVersion
metadata to define which Mapfile keywords are valid for different versions of MapServer - A new
schema
command line tool to export Mapfile schemas for different versions of MapServer - Allow Mapfile validation based on a specific version of MapServer
- Add better error message when incorrect dicts are passed to printer
- Add py38 to continuous integration testing
- Add command line scripts to continuous integration testing
- Fix
CONNECTIONOPTIONS
formatted output - Update to lark-parser 0.9.0
- #109 - Add validation based on MapServer version
- #96 - Unquoted Unicode strings cause parsing errors
- #102 - Added support for accented-latin in unquoted strings (Issue #96) - thanks @erezsh
- #97 - Allow for negative expressions
- #101 - Fix for issue #97 (unary negation) - thanks @erezsh
- #85 - Coding of NOT logical expression
- #100 - Allowing non-bracketed NOT expression (Issue #85) - thanks @erezsh
- Update to lark-parser 0.7.8
- #95 - Allow Mapfile input from
io.StringIO
as well as from a file - thanks @ianturton for pull request - #93 - fix to ensure Mapfiles are closed after reading
- #89 - List expressions with spaces in the attributes fail to parse - thanks @ianturton for fix
- Update to lark-parser 0.7.7
- Update to jsonref 0.2
- Add automated releases to GitHub using Appveyor
- Add automated releases to PyPI using Appveyor
- Add missing CLASS properties to JSON schema
- Additional tests for CaseInsensitiveOrderedDict and EXPRESSIONs
- #37 - LIKE not recognised in FILTER - thanks @ianturton for fix
- #87 - JSON schema add join tag- thanks @hugbe8 for fix
- #74 - Map files containing Unicode can fail in mappyfile.load with python2.7 thanks @ianturton
- #73 - Deepcopy not working (Python3 >=3.5) - thanks @guardeivid
- Add support for CLUSTER keyword along with schema changes and tests
- Fix comments on root objects in a MapFile
- Fix issues with duplicated METADATA keys and comments
- Fix ReadTheDocs build
- Add more sample MapFiles for testing to the project
- Update code to work with Lark 0.6.6 (see #71)
- New end_comment option for pprint - Add a comment with the block type at each closing END statement e.g. END # MAP (see request #69)
- Add
**kwargs
to main API to allow greater flexibility with plugins - Fix DeprecationWarnings relating to Python 3.7.2 (thanks @tigerfoot for the report)
- Tested use with new jsonschema 3.0.0 release
- Deprecated
write
function removed from the API and codebase - Update OFFSET validation to allow attribute bindings - see MapServer/MapServer-documentation#256
- #68 - Support pickling of DefaultOrderedDict in Python3
- #67 - Fix deprecation warnings for grammar regular expressions in Python 3.6
- #65 - Handle hexadecimal color translucence
- Save tokens for value lists
- Update README and fix example code
- Support for modulus operator
- Allow custom transformers to be used with kwargs
- Two new CLI programs -
format
andvalidate
- Update of Lark parser to 0.6.4 (fixes some validation line number issues)
- Improvements to validation log messages
- Normalise include paths
- Update of Lark parser to 0.6.2 and associated changes - thanks @erezsh
mappyfile.findall
returns a list rather than a generatorSYMBOLSET
files now supported (both parsing and transforming)- #63 - Set the PROJECTION value correctly for single strings
- #61 - Remove quotes in mappyfile.findall()
- Breaking Change
utils.dictfind
renamedutils.findkey
- new dictionary update function - allowing for easier creation of Mapfiles using YAML
- allow any custom hidden metadata tags of the form
__property__
to be used in dicts for custom processing - Schema validation updates including RANGEITEM and CLUSTER
- Appveyor builds added
- #56 Can't parse expressions with a : in them
- #54 fix windows cwd name issue in includes - thanks @ianturton
- Finalise validation API
- Finalised Mapfile comments API
- New
dictfind
function - Allow non-string function parameters in expressions
- Use of CaseInsensitiveOrderedDict throughout transformer
- UTF comments
- JSONSchema updates and fixes
- Breaking Change - the
mappyfile.load
method now accepts a file-like object rather than a filename to match the usage in other Python libraries. A newmappyfile.open
method allows opening directly with a filename. - New preserve comments feature - experimental
- Add basic plugin system
- Updates to schema docs (fixes for POSITION, AUTO, and added new default values)
- Fix issue with comments on INCLUDE lines
- #50 Allow END keyword for GEOTRANSFORM parameter
- #49 Allow non-ASCII characters in parser
- #47 Add in missing expression operators - divide, multiply, and power.
- Fixes to setup.py
- Extensive refactoring of grammar and transformer
- Removal of Earley grammar
- Whitespace ignored when parsing
- JSON schema fixes
- #45 Set fixed dependency ranges
- Experimental - inclusion of token positions
- Experimental - inclusion of validation comments
- #45 Remove unnecessary parser keyword
- Add in jsonschema and validation class
- #44 Includes should be relative to Mapfile
- #36 Create a unique logger for mappyfile logger
- #35 Add support for missing arithmetic expressions and run flake8 within tox - thanks @loicgrasser
- #33 Fix max recursion limit count - thanks @loicgrasser
- Add a LALR grammar and parser, now a 8k line Mapfile is now parsed 12x faster
- Add a experimental validator module using jsonschema
- #30 Flake8 support - thanks @loicgrasser
- #28 Add support for relative path for nested include - thanks @loicgrasser
- #25 Expression grammar not allowing
!
- Revert back to a single grammar, but add linebreaks before all
END
keywords to keep acceptable performance
- Add in alternative grammar that allows for no line breaks between composites, and fall back to this if parsing fails (otherwise most use cases suffer a 3x performance hit)
- Allow multiple composites to be parsed directly (e.g.
CLASS..END CLASS..END
) - Allow direct parsing of the
METADATA
andVALIDATION
blocks - UTF-8 checks when opening a Mapfile
- #23 Alternative NE and EQ comparisons not defined
- #22 Handle AUTO Projection setting
- #21 INCLUDES throw error when no cwd set
- #20 Only the first FORMATOPTION is kept after transform
- #19 IMAGEMODE FEATURE throws parsing error
- #18 CONFIG keyword not capitalised
- 0.2.2 - various fixes to grammar, and allow for alternate comparison operators
- 0.2.1 - new
findall
function, see #12 - thanks @Jenselme - 0.2.0 - switch to Lark parser
- 0.1.0 - initial release