file.check_file_meta does not handle contents that are binary.

[whytewolf]

Description of Issue

Original problem started out as file.managed with content_pillar throwing an error.

Unable to manage file: endswith first arg must be bytes or a tuple of bytes, not str

with a trace in the debug logs of

[DEBUG   ] Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt\states\file.py", line 2747, in managed
    **kwargs)
  File "c:\salt\bin\lib\site-packages\salt\modules\file.py", line 5276, in manage_file
    _splitlines_preserving_trailing_newline(contents))
  File "c:\salt\bin\lib\site-packages\salt\modules\file.py", line 157, in _splitlines_preserving_trailing_newline
    if str.endswith('\n') or str.endswith('\r'):
TypeError: endswith first arg must be bytes or a tuple of bytes, not str

Which pointed to file.check_managed_changes when it was checking the meta of the file. for pchanges

This only happens when any kind of binary contents is used instead of a source.

Setup

Quickest setup i found.

setup pillar.file_tree

ext_pillar:
  - file_tree:
      root_dir: /srv/file_tree
      follow_dir_links: False
      keep_newline: False

add a windows minion to the master.
Add a binary file to the file_tree pillar for that windows minion that is a binary file named random.bin

Use the following state twice. Once to create the binary file. and again to fail.

dump_secret:
  file.managed:
    - name: C:/random.bin
    - contents_pillar: 'random.bin'

Steps to Reproduce Issue

See above

Versions Report

PS C:\salt\conf> c:\salt\salt-call --local test.versions_report
local:
    Salt Version:
               Salt: 2018.3.4

    Dependency Versions:
               cffi: 1.10.0
           cherrypy: 10.2.1
           dateutil: 2.6.1
          docker-py: Not Installed
              gitdb: 2.0.5
          gitpython: 2.1.3
              ioflo: Not Installed
             Jinja2: 2.9.6
            libgit2: Not Installed
            libnacl: 1.6.1
           M2Crypto: Not Installed
               Mako: 1.0.6
       msgpack-pure: Not Installed
     msgpack-python: 0.4.8
       mysql-python: Not Installed
          pycparser: 2.17
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)]
       python-gnupg: 0.4.1
             PyYAML: 3.12
              PyZMQ: 16.0.3
               RAET: Not Installed
              smmap: 2.0.5
            timelib: 0.2.4
            Tornado: 4.5.1
                ZMQ: 4.1.6

    System Versions:
               dist:
             locale: cp1252
            machine: AMD64
            release: 10
             system: Windows
            version: 10 10.0.18362 SP0 Multiprocessor Free

Note under py2 the error is different but looks to be caused by the same inability to handle binary data coming from contents

[whytewolf]

zd-3868

[xeacott]

This looks to be a windows specific issue - after testing this out with a linux minion and master, the failure doesn't occur. This only occurs with a windows minion and a linux master. The solution seems to point towards making sure the win_file module handles the file appropriately, ensuring line endings are windows-based, otherwise the state reports a failure on the second apply.

DESKTOP-SQIU2I1:
----------
          ID: dump_secret
    Function: file.managed
        Name: C:\newfile
      Result: False
     Comment: Unable to manage file: 'ascii' codec can't decode byte 0xc0 in position 24: ordinal not in range(128)
     Started: 13:15:45.005000
    Duration: 22.0 ms
     Changes:

Summary for DESKTOP-SQIU2I1
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:  22.000 ms

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.