local_infile not enabled on a big endian environment

[junaruga]

The client option local_infile is not enabled on a big endian environment.

There is a RPM package for this mysql2 on Fedora Project [1]
I try to build mysql2 on several platforms for that.

Ruby: 2.4.2
DB Client: mariadb-connector-c: 3.0.2 ([2])

On little endian environment such as x86_64, it is fine.
But in big endian such as ppc64, s390x, this issue happens.

The unit test was failed as below result on the environment.
The behavior The used command is not allowed with this MariaDB version usually happens when local_infile is off.

Failures:
  1) Mysql2::Client :local_infile should raise an error when a non-existent file is loaded
     Failure/Error:
       expect {
         client.query "LOAD DATA LOCAL INFILE 'this/file/is/not/here' INTO TABLE infileTest"
       }.to raise_error(Mysql2::Error, 'No such file or directory: this/file/is/not/here')
       expected Mysql2::Error with "No such file or directory: this/file/is/not/here", got #<Mysql2::Error: The used command is not allowed with this MariaDB version> with backtrace:
         # ./lib/mysql2/client.rb:120:in `_query'
         # ./lib/mysql2/client.rb:120:in `block in query'
         # ./lib/mysql2/client.rb:119:in `handle_interrupt'
         # ./lib/mysql2/client.rb:119:in `query'
         # ./spec/mysql2/client_spec.rb:399:in `block (4 levels) in <top (required)>'
         # ./spec/mysql2/client_spec.rb:398:in `block (3 levels) in <top (required)>'
     # ./spec/mysql2/client_spec.rb:398:in `block (3 levels) in <top (required)>'

The reason is this code.
https://github.com/MariaDB/mariadb-connector-c/blob/v3.0.2/libmariadb/mariadb_lib.c#L2652

When usigned int intval is used for MYSQL_OPT_LOCAL_INFILE in _mysql_client_options, the value is set as a cast to my_bool (= char)in test(*(my_bool*) arg1).

little endian: unsigned intval (0x01 0x00 0x00 0x00) => cast to char (0x01) -> test result is true => mysql->options.client_flag|= CLIENT_LOCAL_FILES; is run => ok
big endian: unsigned intval(0x00 0x00 0x00 0x01) -> cast to char (0x00) -> test result is false => mysql->options.client_flag&= ~CLIENT_LOCAL_FILES; is run => local_infile is not set to mysql->options.client_flag.

[1] https://src.fedoraproject.org/rpms/rubygem-mysql2
[2] https://github.com/MariaDB/mariadb-connector-c

[sodabrew]

This looks sensible, but intermittently fails now on just the same test that you're trying to fix for big endian. Try passing this in charval instead of boolval?

[junaruga]

@sodabrew ok let me try it!

[junaruga]

charval is a pointer to char. Did you mean using char charval = 0?

Checking mysql and mariadb's source code, they are setting uint as the cast type, though mariadb-connector-c is using my_bool(= char).

https://github.com/mysql/mysql-server/blob/mysql-5.7.20/sql-common/client.c#L5443
https://github.com/MariaDB/server/blob/10.3/sql-common/client.c#L4275

if (!arg || MY_TEST(*(uint*) arg))

[junaruga]

@sodabrew

This looks sensible, but intermittently fails now on just the same test that you're trying to fix for big endian.

Seeing the test result, it is not same test.

The test that is failed on big endian is this test: should raise an error when a non-existent file is loaded.
But the test that is failed on Travis is this test: should raise an error when local_infile is disabled (FAILED - 1).
That means disabling local_infile is failed because maybe by referring char area + some data by uint cast, MY_TEST(0x00 + some data) becomes true

include/my_global.h:#define MY_TEST(a)		((a) ? 1 : 0)

[sodabrew]

We may have been too clever by switching from my_bool to native C bool types, @tamird, given the need to pass a my_bool through the C API.

[tamird]

Did you mean to mention me? I don't think I was aware/involved with that change #840

[sodabrew]

Oops! I thought it was from a linting pass :)

[sodabrew]

Can you check if the big endian errors begin to appear at version 0.4.6 and higher, which is the first version to have the bool change?

[junaruga]

@sodabrew yes I can do it on next Monday or Tuesday.

[sodabrew]

See #919 to see if it resolves this issue for you.

[sodabrew]

By the way, I checked the docs, and even though the behavior is boolean on/off for this feature, it does require an int argument https://dev.mysql.com/doc/refman/5.7/en/mysql-options.html

MYSQL_OPT_LOCAL_INFILE (argument type: optional pointer to unsigned int)

This option affects client-side LOCAL capability for LOAD DATA operations. By default, LOCAL capability is determined by the default compiled into the MySQL client library (see Section 13.2.6, “LOAD DATA INFILE Syntax”). To control this capability explicitly, invoke mysql_options() to set the MYSQL_OPT_LOCAL_INFILE option:

LOCAL is disabled if the pointer points to an unsigned int that has a zero value.

LOCAL is enabled if no pointer is given or if the pointer points to an unsigned int that has a nonzero value.

Successful use of a LOCAL load operation by a client also requires that the server permits it.

[junaruga]

See #919 to see if it resolves this issue for you.

This does not solve on mariadb-c-connecter 3.0.2 ppc64 big endian environment.

Failures:
  1) Mysql2::Client :local_infile should raise an error when a non-existent file is loaded
     Failure/Error:
       expect {
         client.query "LOAD DATA LOCAL INFILE 'this/file/is/not/here' INTO TABLE infileTest"
       }.to raise_error(Mysql2::Error, 'No such file or directory: this/file/is/not/here')
       expected Mysql2::Error with "No such file or directory: this/file/is/not/here", got #<Mysql2::Error: The used command is not allowed with this MariaDB version> with backtrace:
         # ./lib/mysql2/client.rb:120:in `_query'
         # ./lib/mysql2/client.rb:120:in `block in query'
         # ./lib/mysql2/client.rb:119:in `handle_interrupt'
         # ./lib/mysql2/client.rb:119:in `query'
         # ./spec/mysql2/client_spec.rb:399:in `block (4 levels) in <top (required)>'
         # ./spec/mysql2/client_spec.rb:398:in `block (3 levels) in <top (required)>'
     # ./spec/mysql2/client_spec.rb:398:in `block (3 levels) in <top (required)>'
  2) Mysql2::Client :local_infile should LOAD DATA LOCAL INFILE
     Failure/Error: _query(sql, @query_options.merge(options))
     Mysql2::Error:
       The used command is not allowed with this MariaDB version
     # ./lib/mysql2/client.rb:120:in `_query'
     # ./lib/mysql2/client.rb:120:in `block in query'
     # ./lib/mysql2/client.rb:119:in `handle_interrupt'
     # ./lib/mysql2/client.rb:119:in `query'
     # ./spec/mysql2/client_spec.rb:405:in `block (3 levels) in <top (required)>'
  3) Mysql2::Statement row data type mapping should return Fixnum for a YEAR value
     Failure/Error: expect(@test_result['year_test']).to eql(2009)
       expected: 2009
            got: 131661824
       (compared using eql?)
     # ./spec/mysql2/statement_spec.rb:444:in `block (3 levels) in <top (required)>'
Finished in 17.83 seconds (files took 0.27056 seconds to load)
313 examples, 3 failures

I guess the reason is unsigned int intval is used in below part in your patch.

    case MYSQL_OPT_LOCAL_INFILE:
      intval = (value == Qfalse ? 0 : 1);
      retval = &intval;

and it is casted with as char in mariadb-c-connector.

By the way, I checked the docs, and even though the behavior is boolean on/off for this feature, it does require an int argument https://dev.mysql.com/doc/refman/5.7/en/mysql-options.html

OK, mariadb-c-connector's definition of MYSQL_OPT_LOCAL_INFILE is different.
I will ask mariadb-c-connector project about it.

[sodabrew]

Thanks for your help running this down!

[junaruga]

@sodabrew you are welcome. Reported here. https://jira.mariadb.org/browse/CONC-297
I am going to discuss there.

[sodabrew]

Thanks for getting this fixed upstream!

[junaruga]

Yeah, that's good news.

[sodabrew]

For those who find this PR in the future: Fixed in mariadb-connector-c 3.0.3 or higher.

[junaruga]

@sodabrew I needed this patch to run mysql2 0.5.2 on mariadb-connector-c 3.0.5 on the big endian environment.
I do not know why.

Though mariadb-connector-c fixed something on mariadb-connector-c >= 3.0.3.
mariadb-corporation/mariadb-connector-c@434b67e