gettext: fix build on Sonoma

[fxcoudert]

Apple's iconv has a regression in macOS Sonoma. This unfortunately happens to be triggered in gettext's configure test. This patch bypasses the detection.

[chenrui333]

#142161

[ychin]

Hi, is there more information on what the regression actually is (since Apple doesn't let you view other people's feedbacks)? I'm looking into some Sonoma text related issues (basically, Vim tests are failing in Sonoma when handling encoding conversions) and I'm guessing they are related. Would probably be easiest if I know what they are though.

[fxcoudert]

@ychin the issue I reported is the following:

The following C code:

#include <iconv.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>

#ifndef ICONV_CONST
# define ICONV_CONST const
#endif

int
main (void)
{
  iconv_t cd_utf8_to_88591 = iconv_open ("ISO8859-1", "UTF-8");
  if (cd_utf8_to_88591 != (iconv_t)(-1))
    {
      static ICONV_CONST char input[] = "\342\202\254"; /* EURO SIGN */
      char buf[10];
      ICONV_CONST char *inptr = input;
      size_t inbytesleft = strlen (input);
      char *outptr = buf;
      size_t outbytesleft = sizeof (buf);
      size_t res = iconv (cd_utf8_to_88591,
                          &inptr, &inbytesleft,
                          &outptr, &outbytesleft);
      printf("%ld\n", res);
      printf("%ld\n", errno);
      iconv_close (cd_utf8_to_88591);
    }
  return 0;
}

when compiled and linked against iconv, with clang a.c -liconv -w && ./a.out should output:

-1
92

That means iconv() has failed, and set the error to EILSEQ (invalid sequence). This was the case on macOS Ventura and earlier versions.

In macOS Sonoma, the iconv() call unexpectedly succeeds, i.e., it returns value 0, and does not set errno. I believe this is invalid behaviour, as an error is expected.

For the record, this code is used to test for iconv() functionality in gettext and other GNU projects. This is how we came up with the issue.

[ychin]

I took a closer look into it and I think it's a little more complicated. Apple has been removing GPL code from their code base for years now. The most famous case is moving from Bash to ZSH, but they have been removing others as well. E.g. in macOS 13 Ventura they silently replaced GNU diff with "Apple diff (based on FreeBSD diff)" (in which I also found bugs 😅), and seems like in macOS 14 Sonoma, they have replaced iconv with a custom version based on BSD. If you go to https://opensource.apple.com/releases/, you can see that the macOS 13 iconv was still GNU based whereas the macOS 14 iconv is now under BSD license.

If you run the code above, seems like they are converting a Euro sign (€) with the literal "EUR" string, and therefore not failing conversions. Seems like for other characters like ellipsis (…) it will also get converted to "..." instead. Under GNU iconv (which macOS 13 used), both would fail to convert to basic ISO8859-1 instead.

I'm not sure what the "correct" behavior here is but it seems somewhat intentional and not necessarily a "bug" per se, so I am curious if Apple will actually fix this or not. I'm not sure what gettext's policy here is, since gettext is a GNU project, so they probably would argue that only the GNU iconv is supported? Not sure.

I did find other issues with Apple's iconv though. E.g. if you convert UTF-8 to CP932 (Shift-JIS) and input "…", it seems to miss the proper conversion (which should be 0x8163) and output a geta mark instead. So maybe I'm giving it too much credit in the above paragraph.

Note: Btw I tested this under FreeBSD 13 as well and seems like FreeBSD's iconv behaves a little differently from both Apple iconv and GNU iconv. Instead of returning 0 or -1, it will return 1, meaning that it still considers the character to have failed conversion, but instead it decided to move forward and replaced the euro sign with a "?" instead.

[fredowski]

The regression for pspp fails on Sonoma. The test tries to convert an invalid UTF-8 character and the iconv library on Sonoma gives a new error code translating to "Operation not supported on socket" which was not existing on MacOS 12. (13 not checked).

fredowski/homebrew-pspp#5 (comment)

The code that triggers this different behaviour is here:

https://git.savannah.gnu.org/cgit/pspp.git/tree/src/libpspp/i18n.c#n235

[fredowski]

Would it be possible to move to gnu libiconv?