MySQL-python-1.2.5 will not compile against MariaDB 10.2 (libmariadb-dev)

[ghost]

Is there any way we can get defectdojo to work with MariaDB
MySQL-python-1.2.5 will not compile against MariaDB 10.2.10 and libmariadb-dev 10.2.10 installed

Building wheels for collected packages: MySQL-python
Running setup.py bdist_wheel for MySQL-python ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-3oXEzH/MySQL-python/setup.py';exec(compile(getattr( tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" bdist_wheel -d /tmp/tmpXv05tnpip-wheel- --python-tag cp27:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying mysql_exceptions.py -> build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/init.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/converters.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/connections.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/cursors.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/release.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/times.py -> build/lib.linux-x86_64-2.7/MySQLdb
creating build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/init.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/CR.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/ER.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/FLAG.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/REFRESH.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/CLIENT.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
running build_ext
building 'mysql' extension
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Dversion_info=(1,2,5,'final',1) -D__version=1.2.5 -I/usr/include/mysql -I/usr/include/ python2.7 -c _mysql.c -o build/temp.linux-x86_64-2.7/_mysql.o
In file included from _mysql.c:44:0:
/usr/include/mysql/my_config.h:3:2: warning: #warning This file should not be included by clients, include only <mysql.h> [-Wcpp]
#warning This file should not be included by clients, include only <mysql.h>
^
In file included from _mysql.c:46:0:
/usr/include/mysql/mysql.h:440:3: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
MYSQL_CLIENT_PLUGIN_HEADER
^
/usr/include/mysql/mysql.h:585:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
my_bool STDCALL mysql_embedded();
^
_mysql.c: In function ‘_mysql_ConnectionObject_ping’:
_mysql.c:2005:41: error: ‘MYSQL {aka struct st_mysql}’ has no member named ‘reconnect’
if ( reconnect != -1 ) self->connection.reconnect = reconnect;
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Failed building wheel for MySQL-python
Running setup.py clean for MySQL-python
Failed to build MySQL-python

[morissette]

typedef struct st_mysql {
    NET         net;                    /* Communication parameters */
    void  *unused_0;
    char *host,*user,*passwd,*unix_socket,*server_version,*host_info;
    char *info,*db;
    const struct ma_charset_info_st *charset;      /* character set */
    MYSQL_FIELD *fields;
    MA_MEM_ROOT field_alloc;
    unsigned long long affected_rows;
    unsigned long long insert_id;               /* id if insert on table with NEXTNR */
    unsigned long long extra_info;              /* Used by mysqlshow */
    unsigned long thread_id;            /* Id for connection in server */
    unsigned long packet_length;
    unsigned int port;
    unsigned long client_flag;
    unsigned long server_capabilities;
    unsigned int protocol_version;
    unsigned int field_count;
    unsigned int server_status;
    unsigned int server_language;
    unsigned int warning_count;          /* warning count, added in 4.1 protocol */
    unsigned int reconnect; /* matt hacking mysql */
    struct st_mysql_options options;
    enum mysql_status status;
    my_bool     free_me;                /* If free in mysql_close */
    my_bool     unused_1;
    char                scramble_buff[20+ 1];
    /* madded after 3.23.58 */
    my_bool       unused_2;
    void          *unused_3, *unused_4, *unused_5, *unused_6;
    LIST          *stmts;
    const struct  st_mariadb_methods *methods;
    void          *thd;
    my_bool       *unbuffered_fetch_owner;
    char          *info_buffer;
    struct st_mariadb_extension *extension;
} MYSQL;

See line 22 (matt hacking line) allow's me to compile

[Behinder]

This not work on MacoS and homebrew, there is no libmariadb-dev package :/

[mwasilew2]

in case someone comes across this comment, this is caused by a change in mariadb, in particular how reconnect is being used, for more info see: https://lists.launchpad.net/maria-developers/msg10744.html

as stated in the email linked above, it might be fixed by submitting patches to python/perl connectors in the future. In my case, I couldn't downgrade mysql, was using pip 9.0.1, python 2.7 and had to install MySQL-python. morissette's workaround worked for me (adding that line to /usr/include/mysql/mysql.h ) it's far from ideal, but allowed the installation to finish

[devGregA]

Hi all, we recently switched mysql libraries. Is anyone able to easily retest?

[Warfront1]

@morissette Thank you. This fixed it for me.

In alpine you easily execute his work around with
sed '/st_mysql_options options;/a unsigned int reconnect;' /usr/include/mysql/mysql.h -i.bkp

or in your docker file obviously
RUN sed '/st_mysql_options options;/a unsigned int reconnect;' /usr/include/mysql/mysql.h -i.bkp

[bguruprasad]

@Warfront1 solution solved my problem! Thanks and Cheers!

[gitaarik]

For documentation sake:

The file I needed to edit in my MariaDB installation was: /usr/include/mariadb/mysql.h. Adding a line with unsigned int reconnect; after the line unsigned int warning_count; like morissette suggested.

[mohammadasim]

@Warfront1 your solution solved my problem. Thanks

[alexsazo]

@Warfront1 thanks man!!! Your solution solved my problem. Its magic!

[yuharsenergi]

For documentation sake:

The file I needed to edit in my MariaDB installation was: /usr/include/mariadb/mysql.h. Adding a line with unsigned int reconnect; after the line unsigned int warning_count; like morissette suggested.

Thanks..

[samchorn2022]

**Warfront1 ** commented on Aug 25, 2018

Hello all dear
Could help me i face this problem

Thanks

[root@rancids ~]# pip2 install mysql-connector-python MySQL-Python
WARNING: Running pip install with root privileges is generally not a good idea. Try pip2 install --user instead.
Collecting mysql-connector-python
Downloading https://files.pythonhosted.org/packages/94/42/5f1c4974d346ff015af67a5b1b1818487caaee60659aefd7263af3e2e52a/mysql_connector_python-8.0.30-py2.py3-none-any.whl (351kB)
100% |████████████████████████████████| 358kB 1.6MB/s
Collecting MySQL-Python
Downloading https://files.pythonhosted.org/packages/a5/e9/51b544da85a36a68debe7a7091f068d802fc515a3a202652828c73453cad/MySQL-python-1.2.5.zip (108kB)
100% |████████████████████████████████| 112kB 4.1MB/s
Collecting protobuf<=3.20.1,>=3.11.0 (from mysql-connector-python)
Downloading https://files.pythonhosted.org/packages/74/4e/9f3cb458266ef5cdeaa1e72a90b9eda100e3d1803cbd7ec02f0846da83c3/protobuf-3.18.0-py2.py3-none-any.whl (174kB)
100% |████████████████████████████████| 184kB 2.6MB/s
Installing collected packages: protobuf, mysql-connector-python, MySQL-Python
Running setup.py install for MySQL-Python ... error
Complete output from command /usr/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-build-zMIYIU/MySQL-Python/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-QdpCRW-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying mysql_exceptions.py -> build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/init.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/converters.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/connections.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/cursors.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/release.py -> build/lib.linux-x86_64-2.7/MySQLdb
copying MySQLdb/times.py -> build/lib.linux-x86_64-2.7/MySQLdb
creating build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/init.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/CR.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/ER.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/FLAG.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/REFRESH.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
copying MySQLdb/constants/CLIENT.py -> build/lib.linux-x86_64-2.7/MySQLdb/constants
running build_ext
building 'mysql' extension
creating build/temp.linux-x86_64-2.7
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Dversion_info=(1,2,5,'final',1) -D__version=1.2.5 -I/usr/include/mysql -I/usr/include/mysql/mysql -I/usr/include/python2.7 -c _mysql.c -o build/temp.linux-x86_64-2.7/_mysql.o
In file included from _mysql.c:44:
/usr/include/mysql/my_config.h:3:2: warning: #warning This file should not be included by clients, include only <mysql.h> [-Wcpp]
#warning This file should not be included by clients, include only <mysql.h>
^~~~~~~
_mysql.c: In function ‘_mysql_ConnectionObject_ping’:
_mysql.c:2005:41: error: ‘MYSQL’ {aka ‘struct st_mysql’} has no member named ‘reconnect’
if ( reconnect != -1 ) self->connection.reconnect = reconnect;
^
error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-build-zMIYIU/MySQL-Python/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-QdpCRW-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-zMIYIU/MySQL-Python/

[Kungergely]

Check out the MySQL-python codebase manually and apply the following patch:

--- _mysql.c.orig       2024-04-18 14:27:04.014299866 +0000
+++ _mysql.c    2024-04-18 21:43:41.923189457 +0000
@@ -2002,7 +2002,7 @@
        int r, reconnect = -1;
        if (!PyArg_ParseTuple(args, "|I", &reconnect)) return NULL;
        check_connection(self);
-       if ( reconnect != -1 ) self->connection.reconnect = reconnect;
+       if ( reconnect != -1 ) mysql_options(&self->connection, MYSQL_OPT_RECONNECT, (void *)&reconnect);
        Py_BEGIN_ALLOW_THREADS
        r = mysql_ping(&(self->connection));
        Py_END_ALLOW_THREADS

Then run pip install . to install the package:

[mtesauro]

@Kungergely Heads up that MySQL will shortly no longer be a supported RDMS for DefectDojo see #9690