Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[pg15] merge: master branch commit '6d1729d44a00e55f4bd68dc1540850582…
…9ef6beb' into pg15 Summary: Merge YB master commit 6d1729d titled [#17993] docdb: Fix build failure due to a51a66d and committed 2024-05-06T17:37:59-07:00 into YB pg15. YB pg15 initial merge refers to 55782d5. - postgres_fdw.out: - As in YB pg15 9432924, ignore the incoming YB master 9309788 change. - postgres_fdw.c: - postgresExplainForeignModify: YB master 88b4977 reverts "Storage SQL" back to "Remote SQL". Adjacent lines conflict with upstream PG b663a4136331de6c7364226e3dbf7c88bfee7145. - yb_scan.c: - YBCLockTuple: - signature: YB master 54ceff9 changes the style of this function's signature and also adds comments. Adjacent lines conflict with YB pg15 initial commit changing the return type from HTSU_Result to TM_Result. - body: YB master 54ceff9 adds usage of HeapTupleMayBeUpdated. Upstream PG 5db6df0c0117ff2a4e0cd87594d2db408cd5022f changes that to TM_Ok. - ybginutil.c: - includes: YB master 3429e32 adds includes. Adjacent lines conflict with YB pg15 initial merge renaming/moving nodes/relation.h include to access/relation.h (perhaps based on upstream PG fa2cf164aaf91e074be653c28e035f65d80eb666). - bootstrap.c: - BootstrapModeMain: YB master ec6e419 adds argument to YBCCreateDatabase function call. Adjacent lines conflict with YB pg15 initial merge which renames FirstBootstrapObjectId to YbFirstBootstrapObjectId. - yb_catalog_version.c: - YbIncrementMasterCatalogVersionTableEntry: YB master b5d95a9 adds argument to YbIncrementMasterDBCatalogVersionTableEntryImpl function call. Adjacent lines conflict with YB pg15 initial merge which renames TemplateDbOid to Template1DbOid. - yb_system_views.sql: - pg_replication_slots: YB master 30b1982 changes the pg_replication_slots view definition. YB pg15 initial merge moves the view definition to system_views.sql, so apply the change there. - system_views.sql: - pg_replication_slots: (receive change from yb_system_views.sql) - dbcommands.c: - createdb: - variable declarations: YB master ec6e419 adds variable declarations dclonetime and dbclonetime. YB pg15 initial merge moves variables such as dcolocated to new "yb variables" section. Do the same for these. - foreach: YB master ec6e419 adds else if "clone_time" case in same area upstream PG aa01051418f10afbdfa781b8dc109615ca785ff9 and 9c08aea6a3090a396be334cc58c511edab05776a add other else if cases. Adjacent lines conflict. - if (dclonetime && dclonetime->arg): YB master ec6e419 adds this condition. Upstream PG 37851a8b83d3d57ca48736093b10aa5f3bc0c177 adds an if (dbcollversion) in the same area. YB pg15 initial merge brings this PG commit in incorrectly, adding an extra newline above. The YB master commit also deletes an empty line owned by PG when it shouldn't. Fix the newline issues when merging this. - if (OidIsValid(dboid)): complicated merge between old YB master 1c781b7, YB master ec6e419, YB pg15 84fe79f, upstream PG aa01051418f10afbdfa781b8dc109615ca785ff9. Upstream PG aa01051418f10afbdfa781b8dc109615ca785ff9 moves existing code into an else case of a new if-else. YB pg15 84fe79f adds YBCCreateDatabase call into the if case. YB master ec6e419 adds dbtemplate and dbclonetime arguments to the call in the else case. Add the same arguments to the if case's call. - if (strcmp(dbtemplate, "template0") != 0 &&: YB master ec6e419 adds this code with heap_close. Change heap_close to table_close for YB pg15. - indexcmds.c: - DefineIndex: YB master 3429e32 moves accessMethodForm definition up coincidentally the same way as YB pg15 initial merge. But master also moves the amRoutine definition up whereas YB pg15 initial merge moves/changes the accessMethodId definition down to that place. Apply all moves. - nodeLockRows.c: - ExecLockRows: YB master 54ceff9 changes style on code that was previously merged by YB pg15 initial merge between YB master c9fcd93 and many upstream PG commits between 5db6df0c0117ff2a4e0cd87594d2db408cd5022f and a90641eac24dfc8889122d88eb7f482cd3db8b39. Complicated merge: just apply the style changes to the existing YB pg15 code. - createplan.c: - create_bitmap_subplan: YB master d588f79 changes indexpushdownquals if condition. YB pg15 merge cd1df46 moves that if condition lower. Apply the changes to the new location. - reorderbuffer.c: - ReorderBufferQueueChange: YB master f2ce232 adds YbWalSndTotalTimeInReorderBufferMicros calculation at the end of the function. Upstream PG makes multiple changes in the same area. Merge by making sure the YB code is last since it tracks the time spent in the function. - ReorderBufferCommit: YB master f2ce232 defines yb_start_time and uses it for ReorderBufferStreamCommit calculation at the start and end of ReorderBufferCommit. Upstream PG changes the code a lot. For now, position those same pieces of code to the start and end of ReorderBufferCommit, but there appear to be a lot more code paths that may be lacking timing coverage. - slot.c: - function/variable declarations: YB master 8fd1786 deletes PG_OUTPUT_PLUGIN declaration/definition. Upstream PG 2f6501fa3c54bbe4568e3bcccd9a60d26a46b5ee adds function declaration ReplicationSlotShmemExit in the same area. Adjacent lines conflict. - ReplicationSlotCreate: YB master 8fd1786 adds yb_plugin_name parameter. YB pg15 merge f87edee deals with upstream PG 19890a064ebf53dedcefed0d8339ed3d449b06e6 adding extra parameter two_phase. Resolve in a similar way here. - ReplicationSlotAcquire: YB master 8fd1786 changes "PG_OUTPUT_PLUGIN" to "yb_replication_slot->output_plugin". YB pg15 merge 417e9b3 changes "slot" to "s". Apply both. - slotfuncs.c: - create_physical_replication_slot: (see pg_create_physical_replication_slot). Merge with the addition of the two_phase parameter by upstream PG. - pg_create_physical_replication_slot: YB master 8fd1786 modifies ReplicationSlotCreate function call in this function, but upstream PG 9f06d79ef831ffa333f908f6d3debdb654292414 moves this to new create_physical_replication_slot. Apply the change to the new location. - create_logical_replication_slot: (see pg_create_logical_replication_slot). Also, upstream PG 9f06d79ef831ffa333f908f6d3debdb654292414 changes type of local variable plugin from Name to char *, so avoid the NameStr call. - pg_create_logical_replication_slot: (similar explanation as pg_create_physical_replication_slot) - pg_get_replication_slots: - slot_contents.data.plugin definition: conflict between YB pg15 merge 417e9b3 and YB master 8fd1786. - yb_restart_commit_ht definition: this was added by YB master 30b1982. Adjacent lines conflict. - walsender.c: - CreateReplicationSlot: for three ReplicationSlotCreate calls, merge YB master 8fd1786 adding yb_plugin_name parameter with upstream PG adding two_phase parameter (previously merged by YB pg15 f87edee). - catcache.c: - SearchCatCacheMiss: YB master f2046c1 adds more text to a comment that YB pg15 initial commit fixes indentation for. - guc.c: - function declarations: YB master 54ceff9 adds check_yb_explicit_row_locking_batch_size function declaration. Adjacent lines conflict with upstream PG 249d64999615802752940e017ee5166e726bc7cd adding show_tcp_user_timeout declaration. - ReportGUCOption: YB master 8d64af8 adds YSQL Connection Manager code in the same area upstream PG adds last_reported code. Adjacent lines conflict. - yb_scan.h: - function declarations: YB master 54ceff9 adds YBCFlushTupleLocks function declaration where YB pg15 initial merge makes lots of modifications. Adjacent lines merge. - pg_proc.dat: - pg_get_replication_slots: YB master 30b1982 adds parameter yb_restart_commit_ht to this function, and YB pg15 merge ed96733 also has extra parameters. Apply both. - slot.h: - ReplicationSlotCreate: YB master 8fd1786 adds yb_plugin_name parameter. Conflicts with YB pg15 merge f87edee. - guc.h: - variable declarations: YB master 54ceff9 adds yb_explicit_row_locking_batch_size variable declaration. YB pg15 initial merge moves YB declarations down to a separate section, so do the same for this one. - pg_hint_plan/expected/init.out: - YB master 5877187 updates enable_bitmapscan default from off to on. Adjacent lines conflict with upstream pg_hint_plan 454f72a adding enable_async_append. - yb_xcluster_ddl_replication.c: - HandleSourceDDLEnd: YB master 6ddff2e moves code that YB pg15 merge c177183 touched into source_ddl_end_handler.c ProcessSourceEventTriggerDDLCommands. Move the YB pg15 modifications there. - IsInIgnoreList: YB master 6ddff2e changes strcmp to strncmp. YB pg15 merge c177183 replaces strcmp with enum equality. Drop the incoming master changes. - DisallowMultiStatementQueries: YB master 2f7cd0e adds this code, but command tag changes from string to enum by upstream PG, so modify accordingly. - source_ddl_end_handler.c: - ProcessSourceEventTriggerDDLCommands: (see also yb_xcluster_ddl_replication.c HandleSourceDDLEnd). Change some strncmp to command tag comparisons by using GetCommandTagEnum to get the command tag from command tag name. - gram.y: - createdb_opt_item: YB master ec6e419 adds rule for option value equals FCONST. Upstream PG 2c6d43650d16d91a3e731d236315beffd98db729 already adds that support through NumericOnly (though for different reasons). Drop the incoming YB master change. - pg_yb_utils.c: - YbGetDdlMode: YB master b5d95a9 adds CreateCommandTag call expecting char* but gets CommandTag because of upstream PG 2f9661311b83dc481fc19f6e3bda015392010a40. Wrap in GetCommandTagName to get a char*. It may be worth replacing all char* with CommandTag enum all the way down, but this causes more differences with YB master, so consider it only for the future. - build_postgres.py: - make_postgres: build contrib/test_decoding because Java test TestPgRegressReplicationSlot relies on it after YB master 8fd1786 and it appears to work out of the box. - pg15_tests/passing_tests.tsv: - PgWaitQueueRF1Test.TestResumingWaitersDoesntBlockTabletShutdown: YB master eccd4a8 turns this test into a parameterized test, so adjust the gtest filter accordingly. - ysql_ddl_verification_task.cc: - PgSchemaCheckerWithReadTime: grab changes from future YB master a558f71 because otherwise we get check failures. - pg15_tests/test_yb_bitmap_scans.sh: - Update diff numbers by +131 because YB master 5877187 adds 131 lines above the first diff area. - pg15_tests/test_D31365.sh: - Add yb_explicit_row_lock_batching to the list of failed tests to deal with it later. This test was added by YB master 54ceff9. Test Plan: On Almalinux 8: #!/usr/bin/env bash set -eu ./yb_build.sh fastdebug --gcc11 pg15_tests/run_tests.sh On Almalinux 8, fastdebug, gcc11, pg15_tests/run_tests.sh the following input: pgwrapper_pg_mini-test PgMiniTest.SkipTableTombstoneCheckMetadata integration-tests_cdcsdk_tablet_split-test CDCSDKTabletSplitTest.TestCleanUpCDCStreamsMetadataDuringTabletSplitImplicit integration-tests_cdcsdk_tablet_split-test CDCSDKTabletSplitTest.TestCleanUpCDCStreamsMetadataDuringTabletSplitExplicit integration-tests_cdcsdk_consumption_consistent_changes-test CDCSDKConsumptionConsistentChangesTest.TestDynamicTablesAdditionForTableCreatedAfterStream integration-tests_cdcsdk_consumption_consistent_changes-test CDCSDKConsumptionConsistentChangesTest.TestRetentionBarrierRaceWithUpdatePeersAndMetrics integration-tests_cdcsdk_consumption_consistent_changes-test CDCSDKConsumptionConsistentChangesTest.TestFailureSettingRetentionBarrierOnDynamicTable integration-tests_cdcsdk_consumption_consistent_changes-test CDCSDKConsumptionConsistentChangesTest.TestRetentionBarrierMovementForTablesNotInPublication integration-tests_cdcsdk_consumption_consistent_changes-test CDCSDKConsumptionConsistentChangesTest.TestConsumptionAfterDroppingTableNotInPublication JAVA org.yb.pgsql.TestPgRegressMisc#testPgRegressMiscIndependent client_client-test ClientTest.TestKeyRangeUpperBoundFiltering integration-tests_xcluster_ddl_replication-test XClusterDDLReplicationTest.CreateTable integration-tests_xcluster_ddl_replication-test XClusterDDLReplicationTest.DisableSplitting integration-tests_xcluster_outbound_replication_group-itest XClusterOutboundReplicationGroupTest.Repair integration-tests_xcluster_outbound_replication_group-itest XClusterOutboundReplicationGroupTest.RepairWithYbAdmin integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.DropTableOnProducerThenConsumer integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.DropAllTables integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.DisableAutoTableProcessing client_snapshot-schedule-test CloneFromScheduleTest.CloneAfterDrop pgwrapper_pg_packed_row-test */PgPackedRowTest.* integration-tests_xcluster-test XClusterTest.DeleteWhenSourceIsDown integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.AddRemoveNamespace integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.RemoveNamespaceWhenTargetIsDown integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.RemoveNamespaceWhenSourceIsDown integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.Delete integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.DeleteWhenTargetIsDown integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.DeleteWhenSourceIsDown integration-tests_xcluster_db_scoped-test XClusterDBScopedTest.TestYbAdmin JAVA org.yb.pgsql.TestPgTransparentRestarts#testUpdateLong integration-tests_cdcsdk_consumption_consistent_changes-test CDCSDKConsumptionConsistentChangesTest.TestChangingPublicationRefreshInterval integration-tests_cdcsdk_consumption_consistent_changes-test CDCSDKConsumptionConsistentChangesTest.TestLSNDeterminismWithChangingPubRefreshInterval integration-tests_master_heartbeat-itest MasterHeartbeatITest.IgnoreEarlierHeartbeatFromSameTSProcess integration-tests_master_heartbeat-itest MasterHeartbeatITest.ProcessHeartbeatAfterTSRestart client_ql-tablet-test GetTabletKeyRangesTest.Distribution/* client_ql-tablet-test GetTabletKeyRangesTest.Boundaries/* pgwrapper_pg_table_size-test PgTableSizeTest.PartitionedTableSize pgwrapper_pg_tablet_split-test PgTabletSplitTest.SplitAmidstRunningTransaction integration-tests_master_path_handlers-itest MasterPathHandlersItest.TestTestFlag integration-tests_minicluster-snapshot-test PgCloneTest.CloneYsqlSyntax integration-tests_cdcsdk_ysql-test CDCSDKYsqlTest.TestCDCSDKMetricsTwoTablesSingleStream JAVA org.yb.pgsql.TestPgDepend#testSequenceDeletionWithCascade xcluster_ddl_replication-test XClusterDDLReplicationTest.CreateIndex JAVA org.yb.pgsql.TestPgRegressYbExtensionsYbXclusterDdlReplication integration-tests_xcluster_ddl_replication-test XClusterDDLReplicationTest.BlockMultistatementQuery master_clone_state_manager-test CloneStateManagerTest.AbortInStartTabletsCloning master_clone_state_manager-test CloneStateManagerTest.AbortIfFailToSchedulePgCloneSchema master_clone_state_manager-test CloneStateManagerTest.AbortInPgSchemaClone master_clone_state_manager-test CloneStateManagerTest.AbortInStartTabletsCloningPg master_clone_state_manager-test CloneStateManagerTest.AbortInCreatingState master_clone_state_manager-test CloneStateManagerTest.AbortInRestoringState master_clone_state_manager-test CloneStateManagerTest.AbortIncompleteCloneOnLoad Only org.yb.pgsql.TestPgRegressMisc#testPgRegressMiscIndependent should fail. Also, #./yb_build.sh fastdebug --gcc11 --enable-ysql-conn-mgr-test --java-test 'org.yb.pgsql.TestYsqlUpgrade#creatingSystemRelsAfterFailure' YB_ENABLE_YSQL_CONN_MGR_IN_TESTS=true ./yb_build.sh fastdebug --gcc11 --java-test 'org.yb.pgsql.TestPgUniqueConstraint' ./yb_build.sh fastdebug --gcc11 --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestUserLoginLimit #./yb_build.sh fastdebug --gcc11 --enable-ysql-conn-mgr-test --java-test 'org.yb.pgsql.TestPgAuthorization#testRoleRenamingMidSession' Jenkins: rebase: pg15 Reviewers: fizaa, tfoucher, stiwary Reviewed By: tfoucher, stiwary Subscribers: stiwary, yql Differential Revision: https://phorge.dev.yugabyte.com/D35394
- Loading branch information
