Unable to build with macOS SDK 13.0

[stanhu]

XCode 14.1, which was released yesterday, now ships with macOS SDK 13.0.

pg_query does not build at the moment. This is the output of a bundle exec rake compile:

In file included from gram.y:46:
In file included from /Users/stanhu/github/pg_query/ext/pg_query/include/postgres.h:46:
In file included from /Users/stanhu/github/pg_query/ext/pg_query/include/c.h:1330:
In file included from /Users/stanhu/github/pg_query/ext/pg_query/include/port.h:17:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/netdb.h:91:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/netinet/in.h:81:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/socket.h:425:1: error: expected ';' after top level declarator
__CCT_DECLARE_CONSTRAINED_PTR_TYPES(struct sockaddr, sockaddr);
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/constrained_ctypes.h:587:101: note: expanded from macro '__CCT_DECLARE_CONSTRAINED_PTR_TYPES'
#define __CCT_DECLARE_CONSTRAINED_PTR_TYPES(basetype, basetag)                                      \
                                                                                                    ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/constrained_ctypes.h:580:2: note: expanded from macro '\
__CCT_DECLARE_CONSTRAINED_PTR_TYPE'
        __CCT_DISPATCH(__CCT_DECLARE_CONSTRAINED_PTR_TYPE, basetype, basetag, __VA_ARGS__)
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/constrained_ctypes.h:548:2: note: expanded from macro '__CCT_DISPATCH'
        __CCT_DISPATCH1(base, __CCT_COUNT_ARGS(__VA_ARGS__), __VA_ARGS__)
        ^
note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:113:26: note: expanded from macro '__CONCAT'
#define __CONCAT(x, y)   x ## y
                         ^
:415:1: note: expanded from here
__CCT_DECLARE_CONSTRAINED_PTR_TYPE_3
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/constrained_ctypes.h:563:14: note: expanded from macro '__CCT_DECLARE_CONSTRAINED_PTR_TYPE_3'
        __CCT_DEFER(__CONCAT, basetag,  __CCT_CONTRACT_LIST_TO_TAGGED_SUFFIX_1(kind))
                    ^

pg_query-rake-compile.txt

Downgrading to macOS SDK 12.3 appears to solve the problem:

sudo mkdir /Library/Developer/CommandLineTools/SDKs/backup
sudo mv /Library/Developer/CommandLineTools/SDKs/MacOSX13* /Library/Developer/CommandLineTools/SDKs/backup

I'm not yet sure what's going on here because this test program works fine:

#include <ctype.h>
#include <netdb.h>
#include <pwd.h>

int main()
{
}

[stanhu]

For some reason, commenting out that #include <netdb.h> from port.h seems to make the build work:

diff --git a/ext/pg_query/include/port.h b/ext/pg_query/include/port.h
index 271ff0d..5960bac 100644
--- a/ext/pg_query/include/port.h
+++ b/ext/pg_query/include/port.h
@@ -14,7 +14,7 @@
 #define PG_PORT_H
 
 #include <ctype.h>
-#include <netdb.h>
+// #include <netdb.h>
 #include <pwd.h>
 
 /*

[lfittl]

Strange - possibly something that gets defined before the file gets included, which then causes errors?

Happy to fast track a release if we can find out a bit better why this is failing. It might be worth trying a regular Postgres build on the new XCode to see if this is an upstream problem, or related to what pg_query is doing.

[antmachine]

Same issue here

[stanhu]

Good suggestion. It appears to be failing in the same way in the PostgreSQL REL_14_1 tag but not in master.

postgres/postgres@98e93a1fc dropped netdb.h, so I wonder if that fixed it.

[lfittl]

@stanhu Excellent, thanks for finding that - let me see if I can make the necessary libpg_query change and then do the release dance.

[stanhu]

Oh, wow, it's failing because on line 937 of https://raw.githubusercontent.com/pganalyze/pg_query/main/ext/pg_query/src_backend_parser_gram.c, there is:

#define REF 582

This REF gets used in this macro:

In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/netinet/in.h:81:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/socket.h:425:1: error: expected ';' after top level declarator
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/constrained_ctypes.h:588:101: note: expanded from macro '__CCT_DECLARE_CONSTRAINED_PTR_TYPES'
__CCT_DECLARE_CONSTRAINED_PTR_TYPE(basetype, basetag, REF);                                         \
                                                                                                    ^

[lfittl]

A-ha! Thanks for finding that (again!) -- here is the upstream commit that fixes it:

postgres/postgres@bc7a40b

I'll work on getting that backported into pg_query today.

[stanhu]

Ah, good find. A number of my colleagues reported this compilation issue with macOS 13.0 SDK beta, but I thought it was a beta issue. Glad it's already fixed upstream, though the next PostgreSQL releases are in about a week (November 10 according to https://www.postgresql.org/developer/roadmap/). Thanks for the quick turnaround on this.

[lfittl]

FYI, fix in libpg_query is merged - now working on the releases 🚂

[lfittl]

This has been released as 2.2.0!

Thanks @stanhu & others for reporting this :)

[stanhu]

Thanks @lfittl! Confirmed 2.2.0 installs fine.

[directionless]

Any chance at backporting this to the 1.x line?

[lfittl]

@directionless Is there a reason you are using an old version of libpg_query? (and are you using an old version of the Ruby binding, or a different language binding / the original C library?)

We generally don't backport changes, but happy to consider it if there is a good case for why an upgrade is hard to do.

[directionless]

@lfittl Thanks so much for replying!

In this case, I'm trying to understand what expectations are, so that I can evaluate whether to try to update my project's parsing tools from v1 to v2. As an OSS maintainer, I understand something of the cost to maintaining old versions, I imagine there's interest deprecating the older tooling.

@directionless Is there a reason you are using an old version of libpg_query? (and are you using an old version of the Ruby binding, or a different language binding / the original C library?)

Mostly just history. The the change from json to protobufs means working with the resulting tree is different. About 12 hours later, and I think I have a working tool. This promoted #267

If your curious though, we're using ruby-3.1.2. Nothing special about the bindings.

[directionless]

(Just to close the loop, I did upgrade our parsing to use the v2 gem)