String object whose length is incorrect is created

[skhrshin]

Summary

I found a problem where pg creates String object whose length is incorrect.

Steps to reproduce

pg version 1.3.1
PostgreSQL version 12.6

1. Prepare a table named "test_records" with a VARCHAR column "name".
2. Insert a record with command INSERT INTO test_records VALUES ('日本語');
3. Run the following code (substitute ActiveRecord::Base.connection.raw_connection with your appropriate connection object)

to_sql = 'SELECT name FROM test_records'
full_query = "COPY (#{to_sql}) TO STDOUT"
conn = ActiveRecord::Base.connection.raw_connection

deco = PG::TextDecoder::CopyRow.new
rows = []

conn.copy_data(full_query, deco) do
  while (row = conn.get_copy_data) do
    rows << row
  end
end

top = rows[0]
p [top[0], top[0].length]

Expected Result

["日本語", 3]

is printed.

Actual Result

["日本語", 9]

is printed. 9 is actually size in bytes in UTF-8 encoding and not a number of characters.

[larskanis]

I'm able to reproduce the issue. It was present in ruby versions 2.4 to 3.1 and since pg-0.18.0.