port from resource.open to resource.watch (Talon v0.4)

[lunixbochs]

This change for Talon v0.4 should substantially improve the stability of knausj_talon around first install and CSV loading.

[knausj85]

This looks solid (though I haven't tested yet), but was planning to convert most of these to .talon-list(s) for 0.4. most of the work is done in this branch
https://github.com/knausj85/knausj_talon/tree/rust

The special cases where csvs may still be necessary

• abbreviate
• emacs
• symbol_key_words (doesn't exist yet, would be ~necessary for create spoken form support)
• file_extensions
• homophones
• app_name_overrides

However, it seems to me if we wire up something like track_csv_list for .talon-list, we could deprecate the vast majority of these. Obviously, it may not be quite as robust as users could still create more specific context-specific lists that the mechanism wouldn't know about. Thoughts?

[lunixbochs]

you can test this on beta 0.3.1-630

[knausj85]

I've put this through some paces on both mac and windows, and it works great.

I don't use emacs, so I can't entirely vouch for that bit. It looks correct based on the contents of the talon debug window.

[phillco]

I couldn't tell from the description -- does this API require v0.4? If so we can't really merge this until beta ships to public, right?

[rntz]

We might want to make get_list_from_csv raise its own DeprecationWarning (and suppress the resource.open one) - users of knausj may be using it directly rather than resource.open (I was, for instance), so a more detailed deprecation might be helpful.

[rntz]

I ran into a problem using this, triggered by the emacs stuff, but more generally it has to do with using actions.user.create_spoken_forms_from_list from a resource.watch callback. Currently emacs_commands.py does this here.

The issue is that resource.watch may call the function it wraps before Talon is ready and all actions have been defined. The workaround is to call resource.watch from inside an on_ready callback:

def on_ready():
    resource.watch("emacs_commands.csv")(load_commands)

This is an OK workaround, but maybe it would be better if resource.watch didn't call the callback until talon was ready?

[rntz]

Because this PR is on your own repo I can't push changes to it. So here's a diff that brings this PR up to date with knausj main and applies the workaround I mention:

diff --git a/BREAKING_CHANGES.txt b/BREAKING_CHANGES.txt
index 695212f3..32ea7700 100644
--- a/BREAKING_CHANGES.txt
+++ b/BREAKING_CHANGES.txt
@@ -6,7 +6,7 @@ applied given the delay between changes being submitted and the time they were r
 and merged.
 
 ---
-
+* 2023-06-06 Deprecate `go marks` command for VSCode. Use 'bar marks' instead.
 * 2023-02-04 Deprecate `murder` command for i3wm. Use 'win kill' instead.
 * 2022-12-11 Deprecate user.insert_with_history. Just use `user.add_phrase_to_history(text);
 insert(text)` instead. See #939.
diff --git a/apps/emacs/emacs.py b/apps/emacs/emacs.py
index 853e49ea..c69709e3 100644
--- a/apps/emacs/emacs.py
+++ b/apps/emacs/emacs.py
@@ -1,4 +1,5 @@
 import logging
+from typing import Optional
 
 from talon import Context, Module, actions
 
@@ -62,18 +63,34 @@ class Actions:
         # TODO: handle corner-cases like key(" ") and key("ctrl- "), etc.
         actions.key(" ".join(meta_fixup(k) for k in keys.split()))
 
-    def emacs_meta_x():
-        "Prompts user to enter a command name via execute-extended-command (M-x)."
-        actions.user.emacs_meta("x")
-
-    def emacs_prefix(n: int):
-        "Inputs a numeric prefix argument."
-        # Applying meta to each key can use fewer keypresses and 'works' in ansi-term
-        # mode.
-        actions.user.emacs_meta(" ".join(str(n)))
-        # # Alternative implementation using universal-argument (ctrl-u):
-        # actions.user.emacs("universal-argument")
-        # actions.key(" ".join(str(n)))
+    def emacs_prefix(n: Optional[int] = None):
+        "Inputs a prefix argument."
+        if n is None:
+            # `M-x universal-argument` doesn't have the same effect as pressing the key.
+            prefix_key = actions.user.emacs_command_keybinding("universal-argument")
+            actions.key(prefix_key or "ctrl-u")  # default to ctrl-u
+        else:
+            # Applying meta to each key can use fewer keypresses and 'works' in ansi-term
+            # mode.
+            actions.user.emacs_meta(" ".join(str(n)))
+
+    def emacs(command_name: str, prefix: Optional[int] = None):
+        """
+        Runs the emacs command `command_name`. Defaults to using M-x, but may use
+        a key binding if known or rpc if available. Provides numeric prefix argument
+        `prefix` if specified.
+        """
+        meta_x = actions.user.emacs_command_keybinding("execute-extended-command")
+        keys = actions.user.emacs_command_keybinding(command_name)
+        short_form = actions.user.emacs_command_short_form(command_name)
+        if prefix is not None:
+            actions.user.emacs_prefix(prefix)
+        if keys is not None:
+            actions.user.emacs_key(keys)
+        else:
+            actions.user.emacs_key(meta_x or "meta-x")
+            actions.insert(short_form or command_name)
+            actions.key("enter")
 
     def emacs_help(key: str = None):
         "Runs the emacs help command prefix, optionally followed by some keys."
diff --git a/apps/emacs/emacs.talon b/apps/emacs/emacs.talon
index 65b74707..c2b0fbd1 100644
--- a/apps/emacs/emacs.talon
+++ b/apps/emacs/emacs.talon
@@ -8,13 +8,13 @@ tag(): user.line_commands
 #suplex: key(ctrl-x)
 cancel: user.emacs("keyboard-quit")
 exchange: user.emacs("exchange-point-and-mark")
-execute: user.emacs_meta_x()
+execute: user.emacs("execute-extended-command")
 execute {user.emacs_command}$: user.emacs(emacs_command)
 execute <user.text>$:
-    user.emacs_meta_x()
+    user.emacs("execute-extended-command")
     user.insert_formatted(text, "DASH_SEPARATED")
 evaluate | (evaluate | eval) (exper | expression): user.emacs("eval-expression")
-prefix: user.emacs("universal-argument")
+prefix: user.emacs_prefix()
 prefix <user.number_signed_small>: user.emacs_prefix(number_signed_small)
 
 abort recursive [edit]: user.emacs("abort-recursive-edit")
@@ -53,11 +53,11 @@ display: user.emacs("display-buffer")
 # SHELL COMMANDS #
 shell command: user.emacs("shell-command")
 shell command inserting:
-    user.emacs("universal-argument")
+    user.emacs_prefix()
     user.emacs("shell-command")
 shell command on region: user.emacs("shell-command-on-region")
 shell command on region replacing:
-    user.emacs("universal-argument")
+    user.emacs_prefix()
     user.emacs("shell-command-on-region")
 
 # CUSTOMIZE #
@@ -207,11 +207,11 @@ highlight lines matching [regex]: user.emacs("highlight-lines-matching-regexp")
 highlight regex: user.emacs("highlight-regexp")
 unhighlight regex: user.emacs("unhighlight-regexp")
 unhighlight all:
-    user.emacs("universal-argument")
+    user.emacs_prefix()
     user.emacs("unhighlight-regexp")
 
 recenter:
-    user.emacs("universal-argument")
+    user.emacs_prefix()
     user.emacs("recenter-top-bottom")
 (center | [center] <number_small> from) top:
     user.emacs("recenter-top-bottom", number_small or 0)
diff --git a/apps/emacs/emacs_commands.csv b/apps/emacs/emacs_commands.csv
index 7984fb46..722e2425 100644
--- a/apps/emacs/emacs_commands.csv
+++ b/apps/emacs/emacs_commands.csv
@@ -46,6 +46,7 @@ eval-expression, meta-:
 eval-print-last-sexp,, ev-p
 eval-region,, ev-r
 exchange-point-and-mark, ctrl-x ctrl-x
+execute-extended-command, meta-x
 fileloop-continue,, filel
 fill-paragraph, meta-q
 find-file, ctrl-x ctrl-f
diff --git a/apps/emacs/emacs_commands.py b/apps/emacs/emacs_commands.py
index 3e9cbeb7..61f3c187 100644
--- a/apps/emacs/emacs_commands.py
+++ b/apps/emacs/emacs_commands.py
@@ -2,6 +2,7 @@ import csv
 from dataclasses import dataclass
 from pathlib import Path
 from typing import NamedTuple, Optional
+import logging
 
 from talon import Context, Module, actions, app, resource
 
@@ -9,8 +10,6 @@ mod = Module()
 mod.list("emacs_command", desc="Emacs commands")
 
 ctx = Context()
-emacs_ctx = Context()
-emacs_ctx.matches = "app: emacs"
 
 
 class Command(NamedTuple):
@@ -20,22 +19,21 @@ class Command(NamedTuple):
     spoken: Optional[str] = None
 
 
-@dataclass
-class CommandInfo:
-    by_name: dict  # maps name to Commands.
-    by_spoken: dict  # maps spoken forms to Commands.
+# Maps command name to Command.
+emacs_commands = {}
 
-    def __init__(self):
-        self.by_name = {}
-        self.by_spoken = {}
 
+@mod.action_class
+class Actions:
+    def emacs_command_keybinding(command_name: str) -> Optional[str]:
+        "Looks up the keybinding for command_name in emacs_commands.csv."
+        return emacs_commands.get(command_name, Command(command_name)).keys
 
-emacs_commands = CommandInfo()
-
+    def emacs_command_short_form(command_name: str) -> Optional[str]:
+        "Looks up the short form for command_name in emacs_commands.csv."
+        return emacs_commands.get(command_name, Command(command_name)).short
 
-@resource.watch("emacs_commands.csv")
 def load_commands(f):
-    global emacs_commands
     rows = list(csv.reader(f))
     # Check headers
     assert rows[0] == ["Command", " Key binding", " Short form", " Spoken form"]
@@ -45,6 +43,7 @@ def load_commands(f):
         if 0 == len(row):
             continue
         if len(row) > 4:
+            filepath = Path(__file__).parents[0] / "emacs_commands.csv"
             print(
                 f'"{filepath}": More than four values in row: {row}. '
                 + " Ignoring the extras"
@@ -54,46 +53,27 @@ def load_commands(f):
         )[:4]
         commands.append(Command(name=name, keys=keys, short=short, spoken=spoken))
 
-    info = CommandInfo()
-    info.by_name = {c.name: c for c in commands}
+    # Update global command info.
+    global emacs_commands
+    emacs_commands = {c.name: c for c in commands}
 
     # Generate spoken forms and apply overrides.
     try:
         command_list = actions.user.create_spoken_forms_from_list(
             [c.name for c in commands], generate_subsequences=False
         )
-    except:
-        pass
+    except Exception as e:
+        logging.warning(f"In emacs_commands.py: load_commands(): {e}")
+        command_list = {}
     else:
         for c in commands:
             if c.spoken:
                 command_list[c.spoken] = c.name
-        info.by_spoken = command_list
-
-    global emacs_commands
-    emacs_commands = info
-    ctx.lists["self.emacs_command"] = info.by_spoken
+    ctx.lists["self.emacs_command"] = command_list
 
+# We need this because until talon is ready, actions.user.create_spoken_forms_from_list
+# won't exist, so we shouldn't call load_commands().
+def on_ready():
+    resource.watch("emacs_commands.csv")(load_commands)
 
-@mod.action_class
-class Actions:
-    def emacs(command_name: str, prefix: Optional[int] = None):
-        """
-        Runs the emacs command `command_name`. Defaults to using M-x, but may use
-        a key binding if known or rpc if available. Provides numeric prefix argument
-        `prefix` if specified.
-        """
-
-
-@emacs_ctx.action_class("user")
-class UserActions:
-    def emacs(command_name, prefix=None):
-        if prefix is not None:
-            actions.user.emacs_prefix(prefix)
-        command = emacs_commands.by_name.get(command_name, Command(command_name))
-        if command.keys is not None:
-            actions.user.emacs_key(command.keys)
-        else:
-            actions.user.emacs_meta_x()
-            actions.insert(command.short or command.name)
-            actions.key("enter")
+app.register("ready", on_ready)
diff --git a/apps/git/git_arguments.csv b/apps/git/git_arguments.csv
index 6277e665..f0abf0c8 100644
--- a/apps/git/git_arguments.csv
+++ b/apps/git/git_arguments.csv
@@ -33,6 +33,7 @@ Option, Spoken form
 --move, move
 --no-edit, no edit
 --no-keep-index, no keep index
+--no-rebase, no rebase
 --no-track, no track
 --no-verify, no verify
 --orphan, orphan
diff --git a/apps/tmux/tmux.py b/apps/tmux/tmux.py
index aed2f012..53af8ed2 100644
--- a/apps/tmux/tmux.py
+++ b/apps/tmux/tmux.py
@@ -10,7 +10,7 @@ tag: user.tmux
 setting_tmux_prefix_key = mod.setting(
     "tmux_prefix_key",
     type=str,
-    default="b",
+    default="ctrl-b",
     desc="The key used to prefix all tmux commands",
 )
 
@@ -19,7 +19,7 @@ setting_tmux_prefix_key = mod.setting(
 class TmuxActions:
     def tmux_prefix():
         """press control and the configured tmux prefix key"""
-        actions.key(f"ctrl-{setting_tmux_prefix_key.get()}")
+        actions.key(f"{setting_tmux_prefix_key.get()}")
 
     def tmux_keybind(key: str):
         """press tmux prefix followed by a key bind"""
@@ -60,7 +60,10 @@ class AppActions:
 @ctx.action_class("user")
 class UserActions:
     def tab_jump(number: int):
-        actions.user.tmux_execute_command(f"select-window -t {number}")
+        if number < 10:
+            actions.user.tmux_keybind(f"{number}")
+        else:
+            actions.user.tmux_execute_command(f"select-window -t {number}")
 
     def tab_close_wrapper():
         actions.user.tmux_execute_command_with_confirmation(
diff --git a/apps/vscode/vscode.talon b/apps/vscode/vscode.talon
index 6d6782c8..5d6bff18 100644
--- a/apps/vscode/vscode.talon
+++ b/apps/vscode/vscode.talon
@@ -131,7 +131,10 @@ go recent [<user.text>]:
 go edit: user.vscode("workbench.action.navigateToLastEditLocation")
 
 # Bookmarks. Requires Bookmarks plugin
-go marks: user.vscode("workbench.view.extension.bookmarks")
+bar marks: user.vscode("workbench.view.extension.bookmarks")
+go marks:
+    user.deprecate_command("2023-06-06", "go marks", "bar marks")
+    user.vscode("workbench.view.extension.bookmarks")
 toggle mark: user.vscode("bookmarks.toggle")
 go next mark: user.vscode("bookmarks.jumpToNext")
 go last mark: user.vscode("bookmarks.jumpToPrevious")
diff --git a/core/app_switcher/app_switcher.py b/core/app_switcher/app_switcher.py
index 7b9c9d8b..ab85e9eb 100644
--- a/core/app_switcher/app_switcher.py
+++ b/core/app_switcher/app_switcher.py
@@ -1,4 +1,5 @@
 import os
+import shlex
 import subprocess
 import time
 from pathlib import Path
@@ -33,6 +34,14 @@ mac_application_directories = [
     "/System/Applications/Utilities",
 ]
 
+linux_application_directories = [
+    "/usr/share/applications",
+    "/usr/local/share/applications",
+    os.path.expandvars("/home/$USER/.local/share/applications"),
+    "/var/lib/flatpak/exports/share/applications",
+    "/var/lib/snapd/desktop/applications",
+]
+
 words_to_exclude = [
     "zero",
     "one",
@@ -162,6 +171,39 @@ if app.platform == "windows":
         return items
 
 
+if app.platform == "linux":
+    import configparser
+    import re
+
+    def get_linux_apps():
+        # app shortcuts in program menu are contained in .desktop files. This function parses those files for the app name and command
+        items = {}
+        # find field codes in exec key with regex
+        # https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html#exec-variables
+        args_pattern = re.compile(r" \%[UufFcik]")
+        for base in linux_application_directories:
+            if os.path.isdir(base):
+                for entry in os.scandir(base):
+                    if entry.name.endswith(".desktop"):
+                        config = configparser.ConfigParser(interpolation=None)
+                        config.read(entry.path)
+                        # only parse shortcuts that are not hidden
+                        if config.has_option("Desktop Entry", "NoDisplay") == False:
+                            name_key = config["Desktop Entry"]["Name"]
+                            exec_key = config["Desktop Entry"]["Exec"]
+                            # remove extra quotes from exec
+                            if exec_key[0] == '"' and exec_key[-1] == '"':
+                                exec_key = re.sub('"', "", exec_key)
+                            # remove field codes and add full path if necessary
+                            if exec_key[0] == "/":
+                                items[name_key] = re.sub(args_pattern, "", exec_key)
+                            else:
+                                items[name_key] = "/usr/bin/" + re.sub(
+                                    args_pattern, "", exec_key
+                                )
+        return items
+
+
 @mod.capture(rule="{self.running}")  # | <user.text>)")
 def running_applications(m) -> str:
     "Returns a single application name"
@@ -280,7 +322,10 @@ class Actions:
     def switcher_launch(path: str):
         """Launch a new application by path (all OSes), or AppUserModel_ID path on Windows"""
         if app.platform != "windows":
-            ui.launch(path=path)
+            # separate command and arguments
+            cmd = shlex.split(path)[0]
+            args = shlex.split(path)[1:]
+            ui.launch(path=cmd, args=args)
         else:
             is_valid_path = False
             try:
@@ -337,6 +382,10 @@ def update_launch_list():
 
     elif app.platform == "windows":
         launch = get_windows_apps()
+
+    elif app.platform == "linux":
+        launch = get_linux_apps()
+
         # actions.user.talon_pretty_print(launch)
 
     ctx.lists["self.launch"] = actions.user.create_spoken_forms_from_map(
diff --git a/core/file_extension/file_extension.py b/core/file_extension/file_extension.py
index 11d53d8b..944ea89e 100644
--- a/core/file_extension/file_extension.py
+++ b/core/file_extension/file_extension.py
@@ -50,6 +50,9 @@ _file_extensions_defaults = {
     "dot g zip": ".gzip",
     "dot zip": ".zip",
     "dot toml": ".toml",
+    "dot java": ".java",
+    "dot class": ".class",
+    "dot log": ".log",
 }
 
 ctx = Context()
diff --git a/core/keys/keys.py b/core/keys/keys.py
index 2fac0a78..9381c134 100644
--- a/core/keys/keys.py
+++ b/core/keys/keys.py
@@ -18,7 +18,7 @@ def setup_default_alphabet():
 
 # used for number keys & function keys respectively
 digits = "zero one two three four five six seven eight nine".split()
-f_digits = "one two three four five six seven eight nine ten eleven twelve".split()
+f_digits = "one two three four five six seven eight nine ten eleven twelve thirteen fourteen fifteen sixteen seventeen eighteen nineteen twenty".split()
 
 mod = Module()
 mod.list("letter", desc="The spoken phonetic alphabet")
diff --git a/core/text/formatters.py b/core/text/formatters.py
index 59bff3c9..c50f4ebd 100644
--- a/core/text/formatters.py
+++ b/core/text/formatters.py
@@ -125,10 +125,10 @@ formatters_dict = {
     "DOT_SEPARATED": words_with_joiner("."),
     "DOT_SNAKE": (NOSEP, lambda i, word, _: "." + word if i == 0 else "_" + word),
     "SLASH_SEPARATED": (NOSEP, every_word(lambda w: "/" + w)),
-    "CAPITALIZE_FIRST_WORD": (SEP, first_vs_rest(lambda w: w.capitalize())),
+    "CAPITALIZE_FIRST_WORD": (SEP, first_vs_rest(lambda w: w[:1].upper() + w[1:])),
     "CAPITALIZE_ALL_WORDS": (
         SEP,
-        lambda i, word, _: word.capitalize()
+        lambda i, word, _: word[:1].upper() + word[1:]
         if i == 0 or word not in words_to_keep_lowercase
         else word,
     ),
diff --git a/core/websites_and_search_engines/websites_and_search_engines.py b/core/websites_and_search_engines/websites_and_search_engines.py
index 5d36c06e..a787a078 100644
--- a/core/websites_and_search_engines/websites_and_search_engines.py
+++ b/core/websites_and_search_engines/websites_and_search_engines.py
@@ -12,6 +12,8 @@ mod.list(
     desc="A search engine.  Any instance of %s will be replaced by query text",
 )
 
+# Please do not edit these defaults.  Instead, add / edit your own entries in
+# settings/websites.csv in your user directory.
 website_defaults = {
     "talon home page": "http://talonvoice.com",
     "talon slack": "http://talonvoice.slack.com/messages/help",
@@ -31,6 +33,8 @@ website_defaults = {
     "youtube": "https://www.youtube.com/",
 }
 
+# Please do not edit these defaults.  Instead, add / edit your own entries in
+# settings/search_engines.csv in your user directory.
 _search_engine_defaults = {
     "amazon": "https://www.amazon.com/s/?field-keywords=%s",
     "google": "https://www.google.com/search?q=%s",
diff --git a/core/websites_and_search_engines/websites_and_search_engines.talon b/core/websites_and_search_engines/websites_and_search_engines.talon
index d30d3a63..eab5887d 100644
--- a/core/websites_and_search_engines/websites_and_search_engines.talon
+++ b/core/websites_and_search_engines/websites_and_search_engines.talon
@@ -1,6 +1,10 @@
 open {user.website}: user.open_url(website)
+open that: user.open_url(edit.selected_text())
+open paste: user.open_url(clip.text())
+
 {user.search_engine} hunt <user.text>$:
     user.search_with_search_engine(search_engine, user.text)
 {user.search_engine} (that | this):
     text = edit.selected_text()
     user.search_with_search_engine(search_engine, text)
+{user.search_engine} paste: user.search_with_search_engine(search_engine, clip.text())
diff --git a/lang/lua/lua.py b/lang/lua/lua.py
index 9f14cbc6..9fbad5f3 100644
--- a/lang/lua/lua.py
+++ b/lang/lua/lua.py
@@ -120,7 +120,7 @@ class UserActions:
         actions.insert("break ")
 
     # Assumes a ::continue:: label
-    def code_state_continue():
+    def code_next():
         actions.insert("goto continue")
 
     def code_try_catch():
diff --git a/lang/tags/imperative.talon b/lang/tags/imperative.talon
index 6ce2b070..64640228 100644
--- a/lang/tags/imperative.talon
+++ b/lang/tags/imperative.talon
@@ -14,5 +14,4 @@ state do: user.code_state_do()
 state goto: user.code_state_go_to()
 state return: user.code_state_return()
 state break: user.code_break()
-state continue: user.code_state_continue()
-state next: user.code_next()
+state (continue | next): user.code_next()
diff --git a/plugin/repeater/repeater.talon b/plugin/repeater/repeater.talon
index 4e182978..f97f128f 100644
--- a/plugin/repeater/repeater.talon
+++ b/plugin/repeater/repeater.talon
@@ -3,3 +3,6 @@
 <number_small> times: core.repeat_command(number_small - 1)
 (repeat that | twice): core.repeat_command(1)
 repeat that <number_small> [times]: core.repeat_command(number_small)
+
+(repeat phrase | again) [<number_small> times]:
+    core.repeat_partial_phrase(number_small or 1)
diff --git a/tags/line_commands/line_commands.talon b/tags/line_commands/line_commands.talon
index e6427986..1ca61139 100644
--- a/tags/line_commands/line_commands.talon
+++ b/tags/line_commands/line_commands.talon
@@ -14,7 +14,6 @@ comment <number> until <number>:
     user.select_range(number_1, number_2)
     code.toggle_comment()
 clear [line] <number>:
-    edit.jump_line(number)
     user.select_range(number, number)
     edit.delete()
 clear <number> until <number>:

[lunixbochs]

Talon 704 defers resource.watch to app ready

[phillco]

From slack:

rntz
28 days ago
Ok, I tried it out and am now running your PR locally. It mostly worked except there was an issue with the emacs commands at startup. There's a workaround I've applied; I think the underlying problem could be fixed in most cases by not running resource.watch callbacks until after on_ready happens. I've left a comment on the PR. (edited)

aegis 3 days ago
@rntz can you try emacs again with talon 704 which defers resource.watch to app ready?

[phillco]

Prefer earlier return for simplicity

Suggested change

	if not path.is_file() and default is not None:
	with open(path, "w", encoding="utf-8", newline="") as file:
	if default is None or path.is_file():
	return

[knausj85]

After discussing with Ryan, closing this pull request since it conflicted with some initial .talon-list conversions. Merged the existing changes into #1239