Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storing ciphertext in variable OSX/Linux results in failed decryption #11

Closed
mattsb42-aws opened this issue Sep 28, 2017 · 2 comments
Closed

storing ciphertext in variable OSX/Linux results in failed decryption #11

mattsb42-aws opened this issue Sep 28, 2017 · 2 comments
Assignees
@mattsb42-aws
Labels
Platform::Linux Platform::MacOS

Comments

@mattsb42-aws
Copy link
Member

From an initial overview, it appears that this is caused by encoding issues. Apparently PS is not the only one that wraps encoding.

(py36) 186590df9307:aws-encryption-sdk-cli bullocm$ aa=`echo 'asdf' | aws-crypto -e -i - -o - @test/integration/integration_tests.conf`
(py36) 186590df9307:aws-encryption-sdk-cli bullocm$ echo $aa
?x+??Z??6?nypj(?_aws-crypto-public-keyDAnCD9EN1ToOTTS//IzbmxAC8H99wcJzPWELU2WJjXhIOKnG7fbRpugw/IcGMdLWCFA==aws-kmsKarn:aws:kms:us-east-1:248168362296:key/ce78d3b3-f800-4785-a3b9-63e30bb4b183?x???????+Sn?P?n0 `?He.0?H??0| *?H??
        w??z?&???@??;?????dڃ?^??`<i??lZ?Q?5?????=?!??=?6?ow????<???
                                                                   $?m?`lz??=???W??????-?J??+????L)?Z??|g0e0?/#t2g???l7jfm=F??Z)??(?C?va??-??cQw?-,EKI?1?
                                                                                                                                                         =?S??_U?μP3P????ұʹ??0̤??-?????ujbZ?
(py36) 186590df9307:aws-encryption-sdk-cli bullocm$ echo $aa | aws-crypto -d -i - -o - @test/integration/integration_tests.conf
ERROR:aws_encryption_sdk.streaming_client:Error on closing
Traceback (most recent call last):
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/encryption_context.py", line 132, in read_string
    codecs.decode(source[offset:end], aws_encryption_sdk.internal.defaults.ENCODING),
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/bin/../lib/python3.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa7 in position 172: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk_cli/internal/io_handling.py", line 90, in _single_io_write
    for chunk in handler:
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 262, in next
    return self.readline()
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 240, in readline
    line = self.read(self.line_length)
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 201, in read
    self._prep_message()
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 687, in _prep_message
    self._header, self.header_auth = self._read_header()
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 700, in _read_header
    header, raw_header = aws_encryption_sdk.internal.formatting.deserialize.deserialize_header(self.source_stream)
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/deserialize.py", line 103, in deserialize_header
    tee_stream.read(ser_encryption_context_length)
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/encryption_context.py", line 172, in deserialize_encryption_context
    length=key_size
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/encryption_context.py", line 136, in read_string
    raise SerializationError('Bad format of serialized context.')
aws_encryption_sdk.exceptions.SerializationError: Bad format of serialized context.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 184, in __exit__
    self.close()
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 890, in close
    raise SerializationError('Footer not read')
aws_encryption_sdk.exceptions.SerializationError: Footer not read
Traceback (most recent call last):
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/encryption_context.py", line 132, in read_string
    codecs.decode(source[offset:end], aws_encryption_sdk.internal.defaults.ENCODING),
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/bin/../lib/python3.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa7 in position 172: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/bin/aws-crypto", line 11, in <module>
    sys.exit(cli())
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk_cli/__init__.py", line 177, in cli
    no_overwrite=args.no_overwrite
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk_cli/__init__.py", line 78, in process_cli_request
    no_overwrite=no_overwrite
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk_cli/internal/io_handling.py", line 118, in process_single_operation
    destination_writer=destination_writer
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk_cli/internal/io_handling.py", line 90, in _single_io_write
    for chunk in handler:
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 262, in next
    return self.readline()
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 240, in readline
    line = self.read(self.line_length)
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 201, in read
    self._prep_message()
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 687, in _prep_message
    self._header, self.header_auth = self._read_header()
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/streaming_client.py", line 700, in _read_header
    header, raw_header = aws_encryption_sdk.internal.formatting.deserialize.deserialize_header(self.source_stream)
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/deserialize.py", line 103, in deserialize_header
    tee_stream.read(ser_encryption_context_length)
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/encryption_context.py", line 172, in deserialize_encryption_context
    length=key_size
  File "/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/lib/python3.6/site-packages/aws_encryption_sdk/internal/formatting/encryption_context.py", line 136, in read_string
    raise SerializationError('Bad format of serialized context.')
aws_encryption_sdk.exceptions.SerializationError: Bad format of serialized context.
@mattsb42-aws mattsb42-aws self-assigned this Sep 28, 2017
@mattsb42-aws
Copy link
Member Author

For the record, base64-encoding solves this. It looks like this is definitely caused by encoding weirdness.

(py36) 186590df9307:aws-encryption-sdk-cli bullocm$ aa=`echo 'asdf' | aws-crypto -e -i - -o - @test/integration/integration_tests.conf | base64`
(py36) 186590df9307:aws-encryption-sdk-cli bullocm$ echo $aa
AYADeF6YMeeZTCME9fdWnnIwgHEAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREE2OE51dk9vUmNCSSt4bDBZNXI1Ykw5bGNRaktnbldUZmJXRE8xNUc2b1RXZzkrclNtdldWbmNHQm50UWh2bkFnUT09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMToyNDgxNjgzNjIyOTY6a2V5L2NlNzhkM2IzLWY4MDAtNDc4NS1hM2I5LTYzZTMwYmI0YjE4MwCnAQEBAHiLxv2Rx9XcK1MVbtlQmW4dst0V6lfDE2sy9gLQD4XsngAAAH4wfAYJKoZIhvcNAQcGoG8wbQIBADBoBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDLNR85u5g+V9NIT9CgIBEIA7PhKTmRu1NwVwj8sZE4ksIppkhbUUQ5xnMsFE3uW+ueZQo1W+yvCkiZ6zcAHy2swp6MTR69VBE1fHeXECAAAAAAwAABAAAAAAAAAAAAAAAAAA3+M3Z5vAWBlbsYtipKKEfv////8AAAABAAAAAAAAAAAAAAABAAAABZ2MWK6ZPi1FwYC8VdFrYDlYEMTy/ABnMGUCMHom6Yb+IGFG8cVa4/LzwBfvDd++eBq5sb6viTIl6klF1FPjpi/hF1heFtzlyxCo9AIxANTO0CkAwqJP6RCd8W6KYsVeIou+7Afo7po8zoleobL4murWx5vgxeBzjgEVftXs1g==
(py36) 186590df9307:aws-encryption-sdk-cli bullocm$ echo $aa | base64 -D | aws-crypto -d -i - -o - @test/integration/integration_tests.conf
asdf

@mattsb42-aws mattsb42-aws mentioned this issue Oct 12, 2017
Closed
@mattsb42-aws
Copy link
Member Author

This will be addressed by #29.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattsb42-aws mattsb42-aws

Labels
Platform::Linux Platform::MacOS
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mattsb42-aws