node 22.x fails to build with pointer compression

[TerrorJack]

Version

No response

Platform

Linux 824a726bf62a 6.10.5 #1-NixOS SMP PREEMPT_DYNAMIC Wed Aug 14 13:34:38 UTC 2024 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Checkout the v22.x branch, configure with --experimental-enable-pointer-compression and build it.

How often does it reproduce? Is there a required condition?

The build fails consistently.

What is the expected behavior? Why is that the expected behavior?

The build should succeed.

What do you see instead?

FAILED: gen/node_snapshot.cc 
cd ../../; /workspace/node/out/Release/node_mksnapshot /workspace/node/out/Release/gen/node_snapshot.cc
[4187/4192] ccache c++ -MMD -MF obj/test/cctest/cctest.node_test_fixture.o.d -D_GLIBCXX_USE_CXX11_ABI=1 -DNODE_OPENSSL_CONF_NAME=nodejs_conf -DNODE_OPENSSL_HAS_QUIC -DICU_NO_USER_DATA_OVERRIDE -DV8_COMPRESS_POINTERS -DV8_COMPRESS_POINTERS_IN_ISOLATE_CAGE -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_ENABLE_SANDBOX -D__STDC_FORMAT_MACROS -DOPENSSL_NO_PINSHARED -DOPENSSL_THREADS '-DNODE_ARCH="x64"' '-DNODE_PLATFORM="linux"' -DNODE_WANT_INTERNALS=1 -DHAVE_OPENSSL=1 -DHAVE_INSPECTOR=1 -D__POSIX__ -DNODE_USE_V8_PLATFORM=1 -DNODE_HAVE_I18N_SUPPORT=1 -DNODE_BUNDLED_ZLIB -DOPENSSL_API_COMPAT=0x10100000L -DGTEST_HAS_POSIX_RE=0 -DGTEST_LANG_CXX11=1 -DUNIT_TEST -DUCONFIG_NO_SERVICE=1 -DU_ENABLE_DYLOAD=0 -DU_STATIC_IMPLEMENTATION=1 -DU_HAVE_STD_STRING=1 -DUCONFIG_NO_BREAK_ITERATION=0 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_C_SOURCE=200112 -DNGHTTP2_STATICLIB -DNDEBUG -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_BUILDING_OPENSSL -DAES_ASM -DBSAES_ASM -DCMLL_ASM -DECP_NISTZ256_ASM -DGHASH_ASM -DKECCAK1600_ASM -DMD5_ASM -DOPENSSL_BN_ASM_GF2m -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DPADLOCK_ASM -DPOLY1305_ASM -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DX25519_ASM -DOPENSSL_PIC -DNGTCP2_STATICLIB -DNGHTTP3_STATICLIB -I../../src -I../../tools/msvs/genfiles -I../../deps/v8/include -I../../deps/cares/include -I../../deps/uv/include -I../../deps/sqlite -I../../test/cctest -I../../deps/googletest/include -I../../deps/histogram/src -I../../deps/histogram/include -I../../deps/simdjson -I../../deps/simdutf -I../../deps/ada -I../../deps/nbytes/include -I../../deps/ncrypto -I../../deps/icu-small/source/i18n -I../../deps/icu-small/source/common -I../../deps/zlib -I../../deps/llhttp/include -I../../deps/uvwasi/include -I../../deps/nghttp2/lib/includes -I../../deps/brotli/c/include -I../../deps/openssl/openssl/include -I../../deps/openssl/openssl/crypto/include -I../../deps/openssl/config/archs/linux-x86_64/asm/include -I../../deps/openssl/config/archs/linux-x86_64/asm -I../../deps/ngtcp2 -I../../deps/ngtcp2/ngtcp2/lib/includes -I../../deps/ngtcp2/ngtcp2/crypto/includes -I../../deps/ngtcp2/ngtcp2/crypto -I../../deps/ngtcp2/nghttp3/lib/includes -Wall -Wextra -Wno-unused-parameter -pthread -Wall -Wextra -Wno-unused-parameter -Wno-error=deprecated-declarations -m64 -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++20  -c ../../test/cctest/node_test_fixture.cc -o obj/test/cctest/cctest.node_test_fixture.o
FAILED: obj/test/cctest/cctest.node_test_fixture.o 
ccache c++ -MMD -MF obj/test/cctest/cctest.node_test_fixture.o.d -D_GLIBCXX_USE_CXX11_ABI=1 -DNODE_OPENSSL_CONF_NAME=nodejs_conf -DNODE_OPENSSL_HAS_QUIC -DICU_NO_USER_DATA_OVERRIDE -DV8_COMPRESS_POINTERS -DV8_COMPRESS_POINTERS_IN_ISOLATE_CAGE -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_ENABLE_SANDBOX -D__STDC_FORMAT_MACROS -DOPENSSL_NO_PINSHARED -DOPENSSL_THREADS '-DNODE_ARCH="x64"' '-DNODE_PLATFORM="linux"' -DNODE_WANT_INTERNALS=1 -DHAVE_OPENSSL=1 -DHAVE_INSPECTOR=1 -D__POSIX__ -DNODE_USE_V8_PLATFORM=1 -DNODE_HAVE_I18N_SUPPORT=1 -DNODE_BUNDLED_ZLIB -DOPENSSL_API_COMPAT=0x10100000L -DGTEST_HAS_POSIX_RE=0 -DGTEST_LANG_CXX11=1 -DUNIT_TEST -DUCONFIG_NO_SERVICE=1 -DU_ENABLE_DYLOAD=0 -DU_STATIC_IMPLEMENTATION=1 -DU_HAVE_STD_STRING=1 -DUCONFIG_NO_BREAK_ITERATION=0 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_C_SOURCE=200112 -DNGHTTP2_STATICLIB -DNDEBUG -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_BUILDING_OPENSSL -DAES_ASM -DBSAES_ASM -DCMLL_ASM -DECP_NISTZ256_ASM -DGHASH_ASM -DKECCAK1600_ASM -DMD5_ASM -DOPENSSL_BN_ASM_GF2m -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DPADLOCK_ASM -DPOLY1305_ASM -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DX25519_ASM -DOPENSSL_PIC -DNGTCP2_STATICLIB -DNGHTTP3_STATICLIB -I../../src -I../../tools/msvs/genfiles -I../../deps/v8/include -I../../deps/cares/include -I../../deps/uv/include -I../../deps/sqlite -I../../test/cctest -I../../deps/googletest/include -I../../deps/histogram/src -I../../deps/histogram/include -I../../deps/simdjson -I../../deps/simdutf -I../../deps/ada -I../../deps/nbytes/include -I../../deps/ncrypto -I../../deps/icu-small/source/i18n -I../../deps/icu-small/source/common -I../../deps/zlib -I../../deps/llhttp/include -I../../deps/uvwasi/include -I../../deps/nghttp2/lib/includes -I../../deps/brotli/c/include -I../../deps/openssl/openssl/include -I../../deps/openssl/openssl/crypto/include -I../../deps/openssl/config/archs/linux-x86_64/asm/include -I../../deps/openssl/config/archs/linux-x86_64/asm -I../../deps/ngtcp2 -I../../deps/ngtcp2/ngtcp2/lib/includes -I../../deps/ngtcp2/ngtcp2/crypto/includes -I../../deps/ngtcp2/ngtcp2/crypto -I../../deps/ngtcp2/nghttp3/lib/includes -Wall -Wextra -Wno-unused-parameter -pthread -Wall -Wextra -Wno-unused-parameter -Wno-error=deprecated-declarations -m64 -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++20  -c ../../test/cctest/node_test_fixture.cc -o obj/test/cctest/cctest.node_test_fixture.o
In file included from ../../deps/googletest/include/gtest/gtest-printers.h:122,
                 from ../../deps/googletest/include/gtest/gtest-matchers.h:49,
                 from ../../deps/googletest/include/gtest/internal/gtest-death-test-internal.h:47,
                 from ../../deps/googletest/include/gtest/gtest-death-test.h:43,
                 from ../../deps/googletest/include/gtest/gtest.h:64,
                 from ../../test/cctest/node_test_fixture.h:6,
                 from ../../test/cctest/node_test_fixture.cc:1:
../../test/cctest/node_test_fixture.cc: In member function 'virtual void NodeTestEnvironment::SetUp()':
../../test/cctest/node_test_fixture.cc:24:23: error: 'InitializeSandbox' is not a member of 'v8::V8'
   24 |   ASSERT_TRUE(v8::V8::InitializeSandbox());
      |                       ^~~~~~~~~~~~~~~~~
../../deps/googletest/include/gtest/internal/gtest-internal.h:1453:38: note: in definition of macro 'GTEST_TEST_BOOLEAN_'
 1453 |           ::testing::AssertionResult(expression))                     \
      |                                      ^~~~~~~~~~
../../deps/googletest/include/gtest/gtest.h:1831:32: note: in expansion of macro 'GTEST_ASSERT_TRUE'
 1831 | #define ASSERT_TRUE(condition) GTEST_ASSERT_TRUE(condition)
      |                                ^~~~~~~~~~~~~~~~~
../../test/cctest/node_test_fixture.cc:24:3: note: in expansion of macro 'ASSERT_TRUE'
   24 |   ASSERT_TRUE(v8::V8::InitializeSandbox());
      |   ^~~~~~~~~~~
ninja: build stopped: subcommand failed.

Additional information

The last known good version to build with pointer compression is v22.5.1. I've bisected on the v22.x branch and the first bad commit is ee97c04.

[avivkeller]

CC @codebytere

[avivkeller]

(I've removed the v22 label after adding it because this probably affects main as well, if it's reproducible)

[avivkeller]

@nodejs/build I'm also experiencing an error against the main branch, but it's different from @TerrorJack's:

ninja -C out/Release -j8
ninja: Entering directory `out/Release'
[1/1623] CXX obj/deps/v8/src/ast/v8_base_without_compiler.ast-function-literal-id-reindexer.o
FAILED: obj/deps/v8/src/ast/v8_base_without_compiler.ast-function-literal-id-reindexer.o 
c++ -MMD -MF obj/deps/v8/src/ast/v8_base_without_compiler.ast-function-literal-id-reindexer.o.d -D_GLIBCXX_USE_CXX11_ABI=1 -DNODE_OPENSSL_CONF_NAME=nodejs_conf -DNODE_OPENSSL_HAS_QUIC -DICU_NO_USER_DATA_OVERRIDE -DV8_GYP_BUILD -DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64 -DV8_COMPRESS_POINTERS -DV8_COMPRESS_POINTERS_IN_ISOLATE_CAGE -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_ENABLE_SANDBOX -D__STDC_FORMAT_MACROS -DOPENSSL_NO_PINSHARED -DOPENSSL_THREADS -DV8_TARGET_ARCH_X64 -DV8_HAVE_TARGET_OS -DV8_TARGET_OS_LINUX '-DV8_EMBEDDER_STRING="-node.13"' -DENABLE_DISASSEMBLER -DV8_PROMISE_INTERNAL_FIELD_COUNT=1 -DV8_ENABLE_PRIVATE_MAPPING_FORK_OPTIMIZATION -DV8_SHORT_BUILTIN_CALLS -DOBJECT_PRINT -DV8_INTL_SUPPORT -DV8_ATOMIC_OBJECT_FIELD_WRITES -DV8_ENABLE_LAZY_SOURCE_POSITIONS -DV8_USE_SIPHASH -DNDEBUG -DV8_WIN64_UNWINDING_INFO -DV8_ENABLE_REGEXP_INTERPRETER_THREADED_DISPATCH -DV8_USE_ZLIB -DV8_ENABLE_SPARKPLUG -DV8_ENABLE_MAGLEV -DV8_ENABLE_TURBOFAN -DV8_ENABLE_WEBASSEMBLY -DV8_ENABLE_JAVASCRIPT_PROMISE_HOOKS -DV8_ENABLE_CONTINUATION_PRESERVED_EMBEDDER_DATA -DV8_ALLOCATION_FOLDING -DV8_ALLOCATION_SITE_TRACKING -DV8_ADVANCED_BIGINT_ALGORITHMS -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC -DUCONFIG_NO_SERVICE=1 -DU_ENABLE_DYLOAD=0 -DU_STATIC_IMPLEMENTATION=1 -DU_HAVE_STD_STRING=1 -DUCONFIG_NO_BREAK_ITERATION=0 -I../../deps/v8 -I../../deps/v8/include -Igen/inspector-generated-output-root -I../../deps/v8/third_party/inspector_protocol -Igen -Igen/generate-bytecode-output-root -I../../deps/icu-small/source/i18n -I../../deps/icu-small/source/common -I../../deps/v8/third_party/zlib -I../../deps/v8/third_party/zlib/google -I../../deps/v8/third_party/abseil-cpp -I../../deps/v8/third_party/fp16/src/include -pthread -Wno-unused-parameter -Wno-strict-overflow -Wno-return-type -Wno-int-in-bool-context -Wno-deprecated -Wno-stringop-overflow -Wno-stringop-overread -Wno-restrict -Wno-array-bounds -Wno-nonnull -Wno-dangling-pointer -flax-vector-conversions -m64 -m64 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -fno-strict-aliasing -std=gnu++20 -Wno-invalid-offsetof  -c ../../deps/v8/src/ast/ast-function-literal-id-reindexer.cc -o obj/deps/v8/src/ast/v8_base_without_compiler.ast-function-literal-id-reindexer.o
In file included from ../../deps/v8/src/heap/memory-chunk-metadata.h:19,
                 from ../../deps/v8/src/heap/memory-allocator.h:24,
                 from ../../deps/v8/src/heap/sweeper.h:20,
                 from ../../deps/v8/src/heap/heap.h:38,
                 from ../../deps/v8/src/heap/factory.h:19,
                 from ../../deps/v8/src/objects/script.h:13,
                 from ../../deps/v8/src/parsing/parse-info.h:18,
                 from ../../deps/v8/src/parsing/scanner.h:18,
                 from ../../deps/v8/src/parsing/import-assertions.h:8,
                 from ../../deps/v8/src/ast/modules.h:8,
                 from ../../deps/v8/src/ast/ast.h:11,
                 from ../../deps/v8/src/ast/ast-traversal-visitor.h:8,
                 from ../../deps/v8/src/ast/ast-function-literal-id-reindexer.h:8,
                 from ../../deps/v8/src/ast/ast-function-literal-id-reindexer.cc:5:
../../deps/v8/src/heap/memory-chunk.h:361:2: error: #error The global metadata pointer table requires a single external code space.
  361 | #error The global metadata pointer table requires a single external code space.
      |  ^~~~~

[laurisvan]

I can confirm error: 'InitializeSandbox' is not a member of 'v8::V8' also appears on 20.17.0. We previously used 20.15.1 where it worked, but planned to move into a more recent version due to fetch() changes. Need to try 20.16.0 still...

[laurisvan]

I am a total newbie of V8 internals, but it appears that V8::InitializeSandbox was deprecated already 2022 because it became part of V8:Initialize (https://chromium.googlesource.com/v8/v8/+/f3737bbb12351042cf81198202ba07288b99f597), and no longer appears in V8 main https://chromium.googlesource.com/v8/v8/+/refs/heads/main/src/init/v8.cc).

Perhaps the initialize call at

node/test/cctest/node_test_fixture.cc

Line 24 in 18acff0

ASSERT_TRUE(v8::V8::InitializeSandbox());

is not needed after all?

[sapics]

The built is succeeded, when I revert the PR #53884 for file "common.gypi" as

- o['variables']['v8_enable_sandbox'] = 1 if options.enable_pointer_compression else 0

I guess that #53884 causes the problem, but I could not understand why #53884 was needed, thus, it is difficult for me to create a PR about this.
If anyone can provide an accurate description for maintainers, it would be helpful if you could make a PR.