quoting in config files is not being handled as expected

[mattsb42-aws]

Problem

Because the config files are read as raw data, not interpreted by a shell, quoted contents are not being handled correctly.

ex:

• quotes read as characters
• quoted strings with whitespace being interpreted as multiple arguments

Possible Solution

I think we should be able to solve this by just using shlex.split rather than str.split in the custom config file handler.

[juneb]

I can't detect any effect from this change in the PowerShell console.

(py) PS C:\ps-test> aws-crypto --version
aws-encryption-sdk-cli/1.0.2.2 aws-encryption-sdk/1.3.2

(py) PS C:\ps-test> [string]$psversiontable.psversion
5.1.14393.1770

As I noted before the change, the value in an --encryption-context pair can be a single or double-quoted string, but the key cannot be quoted.

These commands fail:

(py) PS C:\ps-test> aws-crypto -e -m key=$keyid -i .\TestCLI\Hello.txt -o .\TestEnc --encryption-context "purpose"=testing
(py) PS C:\ps-test> aws-crypto -e -m key=$keyid -i .\TestCLI\Hello.txt -o .\TestEnc --encryption-context 'purpose'=testing

These commands work as expected:

(py) PS C:\ps-test> aws-crypto -e -m key=$keyid -i .\TestCLI\Hello.txt -o .\TestEnc --encryption-context purpose="whatever you want here"

(py) PS C:\ps-test> aws-crypto -e -m key=$keyid -i .\TestCLI\Hello.txt -o .\TestEnc --encryption-context purpose='whatever you want here'

(py) PS C:\ps-test> aws-crypto -e -m key=$keyid -i .\TestCLI\Hello.txt -o .\TestEnc --encryption-context 'purpose=whatever you want here'

(py) PS C:\ps-test> aws-crypto -e -m key=$keyid -i .\TestCLI\Hello.txt -o .\TestEnc --encryption-context "purpose=whatever you want here"

[mattsb42-aws]

I did some further testing and found that this is a quirk with PowerShell. I'm guessing that in PS the quotes override whitespace in separating values, but only if they come first?

Test code

import argparse
import sys


def main():
    # Print the raw arguments that we get from the shell
    print(sys.argv)
    parser = argparse.ArgumentParser(fromfile_prefix_chars='@')
    parser.add_argument('-f', nargs='+', action='append')
    args = parser.parse_args()
    # Print the processed arguments from argparse
    print(args)


if __name__ == '__main__':
    main()

Results

Linux (bash)

$ python testme -f suhefis jeofisj=seofijhsoi "osijef=sioejfs" seijfs="oisjeof" "osijefos"=soiejhfosi
['testme', '-f', 'suhefis', 'jeofisj=seofijhsoi', 'osijef=sioejfs', 'seijfs=oisjeof', 'osijefos=soiejhfosi']
Namespace(f=[['suhefis', 'jeofisj=seofijhsoi', 'osijef=sioejfs', 'seijfs=oisjeof', 'osijefos=soiejhfosi']])

MacOS (bash)

$ python testme  -f suhefis jeofisj=seofijhsoi "osijef=sioejfs" seijfs="oisjeof" "osijefos"=soiejhfosi
['testme', '-f', 'suhefis', 'jeofisj=seofijhsoi', 'osijef=sioejfs', 'seijfs=oisjeof', 'osijefos=soiejhfosi']
Namespace(f=[['suhefis', 'jeofisj=seofijhsoi', 'osijef=sioejfs', 'seijfs=oisjeof', 'osijefos=soiejhfosi']])

PowerShell

(testme) PS > python.exe testme -f suhefis jeofisj=seofijhsoi "osijef=sioejfs" seijfs="oisjeof" "osijefos"=soiejhfosi
['testme', '-f', 'suhefis', 'jeofisj=seofijhsoi', 'osijef=sioejfs', 'seijfs=oisjeof', 'osijefos', '=soiejhfosi']
Namespace(f=[['suhefis', 'jeofisj=seofijhsoi', 'osijef=sioejfs', 'seijfs=oisjeof', 'osijefos', '=soiejhfosi']])

Given that this usage is not something we wanted to encourage anyway, I'm inclined to just say "don't do this". Maybe we should include a note explicitly saying note to do this in PowerShell?

The only way to fix this would be to modify the user input, but I really don't think we should do that. I think that would be much more likely to cause surprising behavior than breaking.

[juneb]

I agree. Right now, the docs say that if you are working in a PowerShell console and you want to include spaces or special characters in the name or value of a name=value pair, enclose the entire pair in quotation marks.

[mattsb42-aws]

Sounds good!