Merge remote-tracking branch 'origin/master' into feature/redrawhook

* origin/master: (297 commits)
  driver: Follow-up to grandparent: Have all test suite entry points declare the mock $region_highlight.
  Use the new, unreleased zsh 'memo=' feature to remove only our own entries from $region_highlight.
  driver: Stop re-declaring $region_highlight.  It's unneeded.
  docs: regexp highlighter: Fix a wrong associative array name in the example.
  docs: Fix obs-repository link
  tests: Fix a wrong value of $PREBUFFER in a test, and add checks to prevent this from recurring.
  test harness: Fix use of an undefined variable in an error message.
  'main': Don't progress the $in_redirection staller while $in_param.
  tests: Add an XFail test for issue #712.
  'main': Highlight the parentheses of array assignments as reserved words.
  CI += zsh-5.8
  main: Add tests for arithmetic expansion
  main: Add arithmetic substitution highlighting
  changelog.md: Restore vertical whitespace before section headers.
  'main': Fix issue #677, concerning multiline aliases.
  changelog: Update through HEAD.
  'main': Further optimize argument parsing.
  'main': Optimize a hot path.
  tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
  changelog: Update through HEAD.
  test harness: Print the expected-v.-actual on every failure, not just upon cardinality failures.
  Document ZSH_HIGHLIGHT_MAXLENGTH.
  'main': Fix the last commit's bug concerning parameter elision not happening in redirects in command position.
  'main': Add a test for parameter elision not happening in redirects in command position.
  'main': Fix regression in zsh 5.3.1 and older: all precmd hooks later than z-sy-h would be aborted.
  changelog += WARN_NESTED_VAR fixes (#727, #731)
  'main': Fix a regression caused by the great-grandparent commit's WARN_NESTED_VAR fix.
  'main': Don't run `_zsh_highlight_main__type` on every non-command word.
  'make perf': Show only a cumulative datum per highligher, rather than per test file.
  'main': Don't trip WARN_NESTED_VAR.
  'main': Follow-up to previous: Document the version number, and deduplicate some option letters.
  'main': precommands += strace
  editorconfig: Fix Makefile settings
  Fix typo
  Bump copyright years.
  driver: Fix "_zsh_highlight:3: read-only variable: ret" warnings when POSIX_BUILTINS is set.
  tests: Add a test for the infinite loop fixed by each of the last two commits.
  'main': Fix expansion of positional parameters in `_zsh_highlight_main_highlighter__try_expand_parameter`.
  'main': Fix an infinite loop.
  'main': precommands += ionice(1) (from util-linux)
  driver: Simplify initialization of $zsyh_user_options in the fallback codepath.
  driver: Make sure we don't change the return value in a called function.
  'main': Make logic more robust.  No functional change.
  'main': Break out an anonymous function into a named function.
  Fix typos in comments.
  main: Add test for issue #713
  'main': Support the 'env' precommand.
  test harness: Fix the pretty-printer's padding implementation.
  Revert "test harness: Rewrite the columnar pretty-printer without external tools." and "travis: Remove bsdmainutils since column(1) has been removed, three commits ago."
  changelog: Update through HEAD.
  'main': Correctly highlight '&&' and '||' inside '[[ … ]]' conditions.
  'main': Highlight reserved words following assignments as errors.
  tests: Add tests for issue #461.
  test harness: Output the time information to the same place the test name was printed to.
  test harness: Stringify values in a more readable manner.
  tests: Add a unit test for a path specified with mixed quoting.
  tests: Add a test for issue #498, which has already been fixed.
  tests: Test that global qualifiers and command substitutions aren't evaluated.
  'main': Don't consider path_prefix in alias expansions.
  'main': Add a test for aliases to AUTO_CD directories.
  'main': Let AUTO_CD directories be highlighted with their own style.
  'main': Add an auxiliary variable for readability.
  'main': In command position, do not highlight directories (unless AUTO_CD is set) and non-executable files.
  'main': Extend tests to capture the current behaviour.
  'main': Add an XFail test for issue #202.
  'main': Highlight errors from the EQUALS option.
  'main': Let the type determination ignore global aliases when it ignores regular ones.
  'main': Add a regression test for parameters that expand to global aliases.
  'main': Enable the zsh/parameter codepath of global aliases highlighting.
  changelog: Update through HEAD.
  travis: Remove bsdmainutils since column(1) has been removed, three commits ago.
  'main': Highlight global aliases
  tests: Record current behaviour on global aliases.
  test harness: Rewrite the columnar pretty-printer without external tools.
  test harness: Fix an issue with the pretty-printed $expected_region_highlight/$region_highlight diffing.
  'main': Support the "close file descriptor" and "coproc" redirection syntaxes
  tests: Add a test for the "close file descriptor" and "coproc" redirection syntaxes
  tests: Fix the test added in the last commit.
  tests: Add a test for issue #705, concerning continuation lines.
  test harness: Let tests fail early by exiting non-zero or by setting a flag.
  test harness: Print the test name when $skip_test is set.
  test harness: Remove a bogus check.
  test harness: Fix $skip_test support, broken yesterday.
  travis: Install bsdmainutils to provide column(1).
  test harness: When the cardinality check fails, pretty-print \$expected_region_highlight and \$region_highlight.
  test harness: Don't leak options from test files to the test harness.
  test harness: Fix test failures under zsh 5.0.8 and older.
  'main': Fix a bug manifesting under zsh 5.2 and older.
  'main': Don't highlight arithmetic expansions as command substitutions.
  tests: Add a test documenting the current state, prior to introducing #704.
  test harness: Change cardinality check semantics
  test harness: No-op change to minimize the next diff.
  'main': Document additional meanings of the 'S' $braces_stack flag.
  'main': When the redirection operator '>&' or '<&' is followed by a positive integer, do not consider that as a filename; it's always a file descriptor.
  'main': Add $last_arg for "lookbehind".
  noop: Clarify comment.
  'main': Honour the MULTIOS option when applying the 'globbing' style.
  'main': Document what $in_redirection is currently used for.
  'main': The optimized cmdsubst input syntax doesn't glob.
  changelog: Fix markup.
  ...

.editorconfig

@@ -0,0 +1,15 @@
+# Top-most editorconfig file
+
+root = true
+
+[*]
+end_of_line = lf
+tab_width   = 2
+indent_size = 2
+indent_style = space
+
+[Makefile]
+tab_width   = 8
+indent_size = 8
+indent_style = tab
+

.travis.yml

@@ -3,6 +3,9 @@ sudo: required
 
 env:
   - ZSH=master
+  - ZSH=5.8
+  - ZSH=5.7.1
+  - ZSH=5.7
   - ZSH=5.6.2
   - ZSH=5.6.1
   - ZSH=5.6
@@ -33,7 +36,7 @@ env:
   - ZSH=4.3.12
   - ZSH=4.3.11
 
-script: docker run -v $PWD:/work -w /work zshusers/zsh-${ZSH} /bin/sh -c 'install_packages make procps && make test'
+script: docker run -v $PWD:/work -w /work zshusers/zsh:${ZSH} /bin/sh -c 'install_packages make procps bsdmainutils && make test'
 
 notifications:
   webhooks:

.version

@@ -1 +1 @@
-0.7.0-dev
+0.7.2-dev

COPYING.md

@@ -1,4 +1,4 @@
-Copyright (c) 2010-2018 zsh-syntax-highlighting contributors
+Copyright (c) 2010-2020 zsh-syntax-highlighting contributors
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification, are permitted

HACKING.md

@@ -11,16 +11,18 @@ The `main` highlighter
 
 The following function `pz` is useful when working on the `main` highlighting:
 
-    pq() {
-      (( $#argv )) || return 0
-      print -r -l -- ${(qqqq)argv}
-    }
-    pz() {
-      local arg
-      for arg; do
-        pq ${(z)arg}
-      done
-    }
+```zsh
+pq() {
+  (( $#argv )) || return 0
+  print -r -l -- ${(qqqq)argv}
+}
+pz() {
+  local arg
+  for arg; do
+    pq ${(z)arg}
+  done
+}
+```
 
 It prints, for each argument, its token breakdown, similar to how the main
 loop of the `main` highlighter sees it.
@@ -32,17 +34,19 @@ Since the test harness empties `ZSH_HIGHLIGHT_STYLES` and the `brackets`
 highlighter interrogates `ZSH_HIGHLIGHT_STYLES` to determine how to highlight,
 tests must set the `bracket-level-#` keys themselves.  For example:
 
-    ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
-    ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+```zsh
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
 
-    BUFFER='echo ({x})'
+BUFFER='echo ({x})'
 
-    expected_region_highlight=(
-      "6  6  bracket-level-1" # (
-      "7  7  bracket-level-2" # {
-      "9  9  bracket-level-2" # }
-      "10 10 bracket-level-1" # )
-    )
+expected_region_highlight=(
+  "6  6  bracket-level-1" # (
+  "7  7  bracket-level-2" # {
+  "9  9  bracket-level-2" # }
+  "10 10 bracket-level-1" # )
+)
+```
 
 Testing the `pattern` and `regexp` highlighters
 -----------------------------------------------
@@ -53,20 +57,40 @@ cannot get the `ZSH_HIGHLIGHT_STYLES` keys.  Therefore, when writing tests, use
 the style itself as third word (cf. the
 [documentation for `expected_region_highlight`](docs/highlighters.md)).  For example:
 
-    ZSH_HIGHLIGHT_PATTERNS+=('rm -rf *' 'fg=white,bold,bg=red')
+```zsh
+ZSH_HIGHLIGHT_PATTERNS+=('rm -rf *' 'fg=white,bold,bg=red')
 
-    BUFFER='rm -rf /'
+BUFFER='rm -rf /'
+
+expected_region_highlight=(
+  "1 8 fg=white,bold,bg=red" # rm -rf /
+)
+```
+
+Memos and commas
+----------------
+
+We append to `region_highlight` as follows:
+
+
+```zsh
+region_highlight+=("$start $end $spec, memo=zsh-syntax-highlighting")
+```
+
+That comma is required to cause zsh 5.8 and older to ignore the memo without
+ignoring the `$spec`.  It's a hack, but given that no further 5.8.x patch
+releases are planned, it's been deemed acceptable.  See issue #418 and the
+cross-referenced issues.
 
-    expected_region_highlight=(
-      "1 8 fg=white,bold,bg=red" # rm -rf /
-    )
 
 Miscellany
 ----------
 
 If you work on the driver (`zsh-syntax-highlighting.zsh`), you may find the following zstyle useful:
 
-    zstyle ':completion:*:*:*:*:globbed-files' ignored-patterns {'*/',}zsh-syntax-highlighting.plugin.zsh
+```zsh
+zstyle ':completion:*:*:*:*:globbed-files' ignored-patterns {'*/',}zsh-syntax-highlighting.plugin.zsh
+```
 
 IRC channel
 -----------

INSTALL.md

@@ -6,7 +6,7 @@ How to install
 * Arch Linux: [community/zsh-syntax-highlighting][arch-package] / [AUR/zsh-syntax-highlighting-git][AUR-package]
 * Debian: `zsh-syntax-highlighting` package [in `stretch`][debian-package] (or in [OBS repository][obs-repository])
 * Fedora: [zsh-syntax-highlighting package][fedora-package-alt] in Fedora 24+ (or in [OBS repository][obs-repository])
-* FreeBSD: `pkg install zsh-syntax-highlighting` (port name: [`textproc/zsh-syntax-highlighting`][freebsd-port])
+* FreeBSD: `pkg install zsh-syntax-highlighting` (port name: [`shells/zsh-syntax-highlighting`][freebsd-port])
 * Gentoo: [mv overlay][gentoo-overlay]
 * Mac OS X / Homebrew: [brew install zsh-syntax-highlighting][brew-package]
 * Ubuntu: `zsh-syntax-highlighting` package [in Xenial][ubuntu-package] (or in [OBS repository][obs-repository])
@@ -23,8 +23,8 @@ How to install
 [ubuntu-package]: https://launchpad.net/ubuntu/+source/zsh-syntax-highlighting
 [fedora-package]: https://apps.fedoraproject.org/packages/zsh-syntax-highlighting
 [fedora-package-alt]: https://bodhi.fedoraproject.org/updates/?packages=zsh-syntax-highlighting
-[obs-repository]: https://software.opensuse.org//download.html?project=shells%3Azsh-users%3Azsh-syntax-highlighting&package=zsh-syntax-highlighting
-[void-package]: https://github.com/voidlinux/void-packages/tree/master/srcpkgs/zsh-syntax-highlighting
+[obs-repository]: https://software.opensuse.org/download.html?project=shells%3Azsh-users%3Azsh-syntax-highlighting&package=zsh-syntax-highlighting
+[void-package]: https://github.com/void-linux/void-packages/tree/master/srcpkgs/zsh-syntax-highlighting
 
 See also [repology's cross-distro index](https://repology.org/metapackage/zsh-syntax-highlighting/versions)
 
@@ -33,17 +33,23 @@ See also [repology's cross-distro index](https://repology.org/metapackage/zsh-sy
 
 Simply clone this repository and source the script:
 
-    git clone https://github.com/zsh-users/zsh-syntax-highlighting.git
-    echo "source ${(q-)PWD}/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh" >> ${ZDOTDIR:-$HOME}/.zshrc
+```zsh
+git clone https://github.com/zsh-users/zsh-syntax-highlighting.git
+echo "source ${(q-)PWD}/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh" >> ${ZDOTDIR:-$HOME}/.zshrc
+```
 
   Then, enable syntax highlighting in the current interactive shell:
 
-    source ./zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
+```zsh
+source ./zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
+```
 
   If `git` is not installed, download and extract a snapshot of the latest
   development tree from:
 
-    https://github.com/zsh-users/zsh-syntax-highlighting/archive/master.tar.gz
+```
+https://github.com/zsh-users/zsh-syntax-highlighting/archive/master.tar.gz
+```
 
   Note the `source` command must be **at the end** of `~/.zshrc`.
 
@@ -69,15 +75,17 @@ your `.zshrc`.
 
 1. Clone this repository in oh-my-zsh's plugins directory:
 
-       git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
+    ```zsh
+    git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
+    ```
 
 2. Activate the plugin in `~/.zshrc`:
 
-       plugins=( [plugins...] zsh-syntax-highlighting)
+    ```zsh
+    plugins=( [plugins...] zsh-syntax-highlighting)
+    ```
 
-3. Source `~/.zshrc`  to take changes into account:
-
-       source ~/.zshrc
+3. Restart zsh (such as by opening a new instance of your terminal emulator).
 
 #### [Prezto](https://github.com/sorin-ionescu/prezto)
 
@@ -106,10 +114,14 @@ Any of the above methods is suitable for a single-user installation,
 which requires no special privileges.  If, however, you desire to install
 zsh-syntax-highlighting system-wide, you may do so by running
 
-    make install
+```zsh
+make install
+```
 
 and directing your users to add
 
-    source /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
+```zsh
+source /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
+```
 
 to their `.zshrc`s.

Makefile

@@ -41,7 +41,7 @@ test:
 	for test in highlighters/*; do \
 		if [ -d $$test/test-data ]; then \
 			echo "Running test $${test##*/}"; \
-			$(ZSH) -f tests/test-highlighting.zsh "$${test##*/}"; \
+			env -i QUIET=$$QUIET $${TERM:+"TERM=$$TERM"} $(ZSH) -f tests/test-highlighting.zsh "$${test##*/}"; \
 			: $$(( result |= $$? )); \
 		fi \
 	done; \

README.md

@@ -27,6 +27,11 @@ Before: [![Screenshot #3.1](images/before3-smaller.png)](images/before3.png)
 <br/>
 After:&nbsp; [![Screenshot #3.2](images/after3-smaller.png)](images/after3.png)
 
+Before: [![Screenshot #4.1](images/before4-smaller.png)](images/before4-smaller.png)
+<br/>
+After:&nbsp; [![Screenshot #4.2](images/after4-smaller.png)](images/after4-smaller.png)
+
+
 
 How to install
 --------------