coercing to Unicode: need string or buffer, NoneType found - for mysql_user #52723

jbontech · 2019-04-27T03:41:02Z

Description of Issue/Question

We are using mysql_user.present saltstate to maintain the db user via salstack
After the last release the we are not able to use the saltstate any more it is showing the error

Setup

 mysql_user.present:
    - name: {{ single_user['name'] }}
    - host: '%'
    - password_hash: '{{ single_user['password'] }}'
    - connection_user: '{{ mysql_user }}'
    - connection_pass: '{{ mysql_pass }}'
    - connection_host: '{{ mysql_host }}'`

Steps to Reproduce Issue

          ID: create admintest
    Function: mysql_user.present
        Name: admintest
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1939, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/mysql_user.py", line 174, in present
                  **connection_args):
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1475, in user_chpass
                  " IDENTIFIED BY '" + password + "';")
              TypeError: coercing to Unicode: need string or buffer, NoneType found
     Started: 03:24:56.335720
    Duration: 58.34 ms

Versions Report

Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: 1.10.6
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 12 2018, 14:36:49)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-1079-aws
         system: Linux
        version: Ubuntu 16.04 xenial

The text was updated successfully, but these errors were encountered:

jbontech · 2019-04-29T16:58:05Z

anyone facing the same issue

waynew · 2019-04-30T22:40:27Z

@oliyavan Oh yeah, apparently that state is super broken in 2019.0 😞 . It was fixed with ed62a2f which should be in the (coming soon) 2019.2.1 release

tcristina · 2019-08-30T10:02:24Z

I'm facing exactly the same error with Salt version 2019.2.0 (Fluorine)

----------
          ID: repl
    Function: mysql_user.present
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1939, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/mysql_user.py", line 174, in present
                  **connection_args):
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1475, in user_chpass
                  " IDENTIFIED BY '" + password + "';")
              TypeError: coercing to Unicode: need string or buffer, NoneType found
     Started: 09:56:22.456732
    Duration: 12.912 ms
     Changes:

waynew · 2019-09-06T17:00:05Z

@tcristina if you can test it, does the aforementioned patch fix the problem for you?

poofyteddy · 2019-09-11T20:15:22Z

i'm having the same issue and i would like to test the patch. but i have no idea where the file is supposed to be. doing a locate only pointed me to python2.7 lib, and i remember salt now needing python3.
Is there a doc on how to test patch somewhere ?
(running in a lab on salt 2019.2.0)

waynew · 2019-09-11T20:33:07Z

@poofyteddy you can do python -c 'import salt; print(salt.__file__)' and that should tell you where your salt is installed - from your description it's probably something like /usr/lib/python2.7/dist-packages/salt - you're going to want the salt/modules/mysql.py file. You should just be able to copy the relevant parts of the file to your local one (or possibly just copy the entire file). If you're not familiar with Python code, do make sure that your editor isn't set to indent with tabs or you might end out with some issues.

HTH!

poofyteddy · 2019-09-12T16:10:37Z

Thank's @waynew, I did this on the salt master last time, and it didn't work. i just swap the file on the minion, run a state.apply of my sls, and the issue is still there. Is there something i'm not doing right ?

The error changed for this

 ID: pkgs.mysql.install.server_root_access
    Function: mysql_user.present
        Name: root
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1939, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/mysql_user.py", line 174, in present
                  **connection_args):
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1556, in user_chpass
                  result = _execute(cur, qry, args)
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 609, in _execute
                  return cur.execute(qry, args)
                File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 210, in execute
                  query = query % args
              KeyError: 'password'
     Started: 18:05:18.567825
    Duration: 16.289 ms
     Changes:

waynew · 2019-09-12T21:02:36Z

You would need to change this file on your salt minion, yeah. I believe that you can also drop it in as a custom module, which should send it to the minion on a highstate/saltutil.sync_all.

One way that you can test to make sure that the code you expect is being executed is by adding raise Exception('hello!') somewhere in the code. You should then get to see the message pop up somewhere, either in the state return, or the master or minion log (/var/log/salt/master or ../minion, probably)

poofyteddy · 2019-09-13T17:29:07Z

i can confirm i used the file from the "fix" and get the error i write about. I'm taking a look at it but i'm not a dev :(
It seam to be trigger by allow_passwordless

user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass frank localhost ''
hostname.domain
    False
user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass root localhost toto
hostname.domain:
    True
user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass root localhost fifi
hostname.domain:
    True
user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass root localhost allow_passwordless=True
hostname.domain:
    The minion function caused an exception: Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 1660, in _thread_return
        return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
      File "/usr/lib/python2.7/dist-packages/salt/executors/direct_call.py", line 12, in execute
        return func(*args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1556, in user_chpass
        result = _execute(cur, qry, args)
      File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 609, in _execute
        return cur.execute(qry, args)
      File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 210, in execute
        query = query % args
    KeyError: 'password'
ERROR: Minions returned with non-zero exit code

poofyteddy · 2019-09-13T17:54:23Z

so yea the fix break because of this

-                 qry = ("ALTER USER '" + user + "'@'" + host + "'"
-                        " IDENTIFIED BY '" + password + "';")
-                 args = {}
+                qry = "ALTER USER %(user)s@%(host)s IDENTIFIED BY %(password)s;"

Before when password wasn't defined i didn't force it, now it have to be there.
EDIT: if you take the actual file from developement it look like it work. i'm getting mysql error, but that might be on me

stale · 2020-01-07T22:05:33Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale · 2020-01-08T13:39:37Z

Thank you for updating this issue. It is no longer marked as stale.

waynew · 2020-03-26T15:17:43Z

@oliyavan / @tcristina / @poofyteddy

Can you verify whether or not this problem exists on Salt 3000?

poofyteddy · 2020-03-26T19:59:46Z

I'll need to test it, last time i try the mysql module i had issue with changing the root login from password to unix_socket, so i didn't check further
But i don't think i've seen this one since
I'll try this weekend. I happily do still work during the week

poofyteddy · 2020-03-29T15:00:51Z

yea, sadly i'm encountering other issue with the actual mysql module which stop me from testing this specific bug
i can't connect to the db from any salt state, but i can just sudo mysql without issue.

Sorry, but i can't help checking the validity of this issue while the mysql module is still broken on my side :(

ID: pkgs.mysql.install.server_root_access
    Function: mysql_user.present
        Name: root
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/salt/state.py", line 1981, in call
                  **cdata['kwargs'])
                File "/usr/lib/python3/dist-packages/salt/loader.py", line 1977, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/states/mysql_user.py", line 131, in present
                  **connection_args):
                File "/usr/lib/python3/dist-packages/salt/modules/mysql.py", line 1266, in user_exists
                  server_version = salt.utils.data.decode(version(**connection_args))
                File "/usr/lib/python3/dist-packages/salt/modules/mysql.py", line 858, in version
                  dbc = _connect(**connection_args)
                File "/usr/lib/python3/dist-packages/salt/modules/mysql.py", line 392, in _connect
                  dbc = MySQLdb.connect(**connargs)
                File "/usr/lib/python3/dist-packages/MySQLdb/__init__.py", line 86, in Connect
                  return Connection(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 172, in __init__
                  for k, v in conv.items():
              AttributeError: 'str' object has no attribute 'items'
     Started: 16:48:56.392379
    Duration: 17.315 ms
     Changes:

{# Ensure the root user is present and can only connect to  mysql from the unix socket #}
{{ sls }}_root_access:
  mysql_user.present:
    - name: root
    - host: localhost
    - allow_passwordless: True
    - unix_socket: True

Server version: 10.4.12-MariaDB-1:10.4.12+maria~buster mariadb.org binary distribution

local:
    Salt Version:
               Salt: 3000
     
    Dependency Versions:
               cffi: Not Installed
           cherrypy: Not Installed
           dateutil: 2.7.3
          docker-py: Not Installed
              gitdb: Not Installed
          gitpython: Not Installed
             Jinja2: 2.10
            libgit2: Not Installed
           M2Crypto: Not Installed
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.5.6
       mysql-python: 1.3.10
          pycparser: Not Installed
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 3.7.3 (default, Dec 20 2019, 18:57:59)
       python-gnupg: Not Installed
             PyYAML: 3.13
              PyZMQ: 17.1.2
              smmap: Not Installed
            timelib: Not Installed
            Tornado: 4.5.3
                ZMQ: 4.3.1
     
    System Versions:
               dist: debian 10.3 
             locale: UTF-8
            machine: x86_64
            release: 4.19.0-8-amd64
             system: Linux
            version: debian 10.3

#56124 should have fixed it, but i can't tell if salt 3000.1 is out yet or not ... all i know is that i'm up to date with the latest salt-server and minion on salt-stack official debian repository

sagetherage · 2020-04-01T22:49:24Z

the packages will be available soon on repo.saltstack.com

poofyteddy · 2020-04-02T04:54:29Z

i'll try again this weekend with 3000.1

sagetherage · 2020-05-29T21:53:26Z

fixes were released in Neon, please open a new ticket if issues persist

waynew added v2019.2.0 unsupported version Bug broken, incorrect, or confusing behavior fixed-pls-verify fix is linked, bug author to confirm fix labels Apr 30, 2019

waynew added this to the Approved milestone Apr 30, 2019

poofyteddy mentioned this issue Sep 13, 2019

module mysql user_chpass force the use of auth_socket #54479

Closed

stale bot added the stale label Jan 7, 2020

waynew added the Confirmed Salt engineer has confirmed bug/feature - often including a MCVE label Jan 8, 2020

stale bot removed the stale label Jan 8, 2020

sagetherage assigned waynew Mar 26, 2020

sagetherage added ZRelease-Sodium retired label and removed fixed-pls-verify fix is linked, bug author to confirm fix labels Mar 26, 2020

sagetherage added fixed-pls-verify fix is linked, bug author to confirm fix and removed v2019.2.0 unsupported version ZRelease-Sodium retired label labels Apr 1, 2020

sagetherage closed this as completed May 29, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

coercing to Unicode: need string or buffer, NoneType found - for mysql_user #52723

coercing to Unicode: need string or buffer, NoneType found - for mysql_user #52723

jbontech commented Apr 27, 2019 •

edited

Loading

jbontech commented Apr 29, 2019

waynew commented Apr 30, 2019

tcristina commented Aug 30, 2019

waynew commented Sep 6, 2019

poofyteddy commented Sep 11, 2019

waynew commented Sep 11, 2019

poofyteddy commented Sep 12, 2019

waynew commented Sep 12, 2019 •

edited

Loading

poofyteddy commented Sep 13, 2019 •

edited

Loading

poofyteddy commented Sep 13, 2019 •

edited

Loading

stale bot commented Jan 7, 2020

stale bot commented Jan 8, 2020

waynew commented Mar 26, 2020

poofyteddy commented Mar 26, 2020

poofyteddy commented Mar 29, 2020 •

edited

Loading

sagetherage commented Apr 1, 2020

poofyteddy commented Apr 2, 2020

sagetherage commented May 29, 2020

coercing to Unicode: need string or buffer, NoneType found - for mysql_user #52723

coercing to Unicode: need string or buffer, NoneType found - for mysql_user #52723

Comments

jbontech commented Apr 27, 2019 • edited Loading

Description of Issue/Question

Setup

Steps to Reproduce Issue

Versions Report

jbontech commented Apr 29, 2019

waynew commented Apr 30, 2019

tcristina commented Aug 30, 2019

waynew commented Sep 6, 2019

poofyteddy commented Sep 11, 2019

waynew commented Sep 11, 2019

poofyteddy commented Sep 12, 2019

waynew commented Sep 12, 2019 • edited Loading

poofyteddy commented Sep 13, 2019 • edited Loading

poofyteddy commented Sep 13, 2019 • edited Loading

stale bot commented Jan 7, 2020

stale bot commented Jan 8, 2020

waynew commented Mar 26, 2020

poofyteddy commented Mar 26, 2020

poofyteddy commented Mar 29, 2020 • edited Loading

sagetherage commented Apr 1, 2020

poofyteddy commented Apr 2, 2020

sagetherage commented May 29, 2020

jbontech commented Apr 27, 2019 •

edited

Loading

waynew commented Sep 12, 2019 •

edited

Loading

poofyteddy commented Sep 13, 2019 •

edited

Loading

poofyteddy commented Sep 13, 2019 •

edited

Loading

poofyteddy commented Mar 29, 2020 •

edited

Loading