Standard escape character (\) doesn't seems to be handled correctly.

[rcauble]

Hi,

I'm using a modified version of the picocli.shell.jline3.example.Example class (see https://github.com/remkop/picocli/blob/main/picocli-shell-jline3/README.md) in which I've added an Echo command to test string arguments.

My modification is to add an echo method to the Nested class:

        @Command(mixinStandardHelpOptions = true, subcommands = {CommandLine.HelpCommand.class},
                description = "echos a value")
        public void echo(@Option(names = {"--value"}, required = true) String value) {
            System.out.println(value);
        }

As long as there's no space is WAI:

prompt> cmd nested echo --value foobar
foobar

But if I try to add an escaped space character it gets an error:

cmd nested echo --value foo\ bar
Unmatched argument at index 5: 'bar'
Usage:  cmd nested echo [-hV] --value=<arg0> [COMMAND]
echos a value
  -h, --help           Show this help message and exit.
  -V, --version        Print version information and exit.
      --value=<arg0>
Commands:
  help  Displays help information about the specified command

Upon further digging, I added a println between the reader.readLine and the execute:

                        line = reader.readLine(prompt, rightPrompt, (MaskingCallback) null, null);
                        System.out.println(line);
                        systemRegistry.execute(line);

When I do that I see

cmd nested echo --value foo\ bar
cmd nested echo --value foo bar
Unmatched argument at index 5: 'bar'
Usage:  cmd nested echo [-hV] --value=<arg0> [COMMAND]
echos a value
  -h, --help           Show this help message and exit.
  -V, --version        Print version information and exit.
      --value=<arg0>
Commands:
  help  Displays help information about the specified command

So it seems that readLine has removed the escape character.

So I try double-escaping it:

cmd nested echo --value foo\\ bar
cmd nested echo --value foo\ bar
foo\ bar

But now echo prints "foo\ bar".

I see similar issues when trying to escape quotes:

prompt> cmd nested echo --value \"foobar\"
line: cmd nested echo --value "foobar"
foobar
prompt> cmd nested echo --value \\"foobar\\"
line: cmd nested echo --value \"foobar\"
\"foobar\"
prompt> 

[rcauble]

I dug into this a bit more and found a few things:

1. If I set LineReader.Option.DISABLE_EVENT_EXPANSION=true then the lines returned from line = reader.readLine match the actual line
2. However, now CommandRegistry.invoke gets called with an array of args each of which are still escaped.
3. I was able to partially workaround problem#2 by wrapping the parser in an UnescapingParser (see below). However SystemRegistryImpl.ArgsParser.unquote does unquoting of quoted strings, so that combined with the previous means for the case where we have input \"foo\", we end up with just foo.

    private static class UnescapedParsedLine implements ParsedLine {
        private final ParsedLine base;
        private final String word;
        private final List<String> words;

        private UnescapedParsedLine(ParsedLine base) {
            this.base = base;
            this.word = unescape(base.word());
            this.words = unescape(base.words());
        }

        private static List<String> unescape(List<String> words) {
            List<String> rv = new ArrayList<>();
            for (String word : words) {
                rv.add(unescape(word));
            }
            return rv;
        }

        private static String unescape(String word) {
            return StringEscapeUtils.unescapeXSI(word);
        }

        @Override
        public String word() {
            return this.word;
        }

        @Override
        public int wordCursor() {
            return base.wordCursor();
        }

        @Override
        public int wordIndex() {
            return base.wordIndex();
        }

        @Override
        public List<String> words() {
            return this.words;
        }

        @Override
        public String line() {
            return base.line();
        }

        @Override
        public int cursor() {
            return base.cursor();
        }
    }
    private static class UnescapingParser extends DefaultParser {
        public ParsedLine parse(String line, int cursor, Parser.ParseContext context) {
            return new UnescapedParsedLine(super.parse(line, cursor, context));
        }
    }

[rcauble]

Ok, I think I found a workaround. It consists of 2 pieces

1. When you construct the LineNumberReader it needs to specify DISABLE_EVENT_EXPANSION

LineReaderBuilder.builder().terminal(terminal)
                                .option(LineReader.Option.DISABLE_EVENT_EXPANSION,
                                        true)
...

2. Extends picocli.shell.jline3.PicocliCommands, overriding the invoke method to perform the unescaping of individual args

private static class MyCommands
            extends picocli.shell.jline3.PicocliCommands {
        ...
       @Override
        public Object invoke(CommandSession session, String command,
                             Object... args) throws Exception {
            String[] unesc = new String[args.length];
            for (int i = 0; i < args.length; ++i) {
                unesc[i] = StringEscapeUtils.unescapeXSI(args[i].toString());
            }
            return super.invoke(session, command, unesc);
        }
}