Skip to content

Commit

Permalink
ENG-5274 #1291 [YSQL] initdb fails in debug build on CentOS
Browse files Browse the repository at this point in the history
Summary:
`initdb` process on the tablet server reads data from postgres system table with thousands rows hosted in
the master.
All these rows are fetched in context of single RPC with `DocRowwiseIterator`.
Timeout of single RPC call is set to 2500msec and in `debug` build this timeout is not enough.

```
W0517 20:01:22.840813 19574 yb_rpc.cc:361] Call yb.tserver.TabletServerService.Read 127.0.0.1:41018 => 127.0.0.1:7100 (request call id 5035) took 2514ms (client timeout 2499ms)
```

After several RPC retries the whole read operations fails with timeout

```
W0501 14:40:41.087746  1236 tablet_rpc.cc:320] Timed out (yb/rpc/outbound_call.cc:436): Failed Read(tablet: 00000000000000000000000000000000, num_ops: 1, num_attempts: 47, txn: 00000000-0000-0000-0000-000000000000) to tablet 00000000000000000000000000000000 on tablet server { uuid: dd0d97df2897493fb8c454947e446efa private: [host: "127.0.0.1" port: 7100] cloud_info: placement_cloud: "cloud1" placement_region: "datacenter1" placement_zone: "rack1" after 47 attempt(s): Read RPC to 127.0.0.1:7100 timed out after 1.162s
```

Debug build uses no optimization, but in case of using `-O1` optimization
`DocRowwiseIterator::HasNext` method works ~9 times faster and RPC takes no more than 250msecs

Test Plan:
- rebuild YB in debug mode `/yb_build.sh --sj`
- check cluster starts `./bin/yb-ctl start`

Reviewers: timur, mikhail

Reviewed By: mikhail

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D6660
  • Loading branch information
d-uspenskiy committed May 28, 2019
1 parent 3de20a3 commit f710367
Showing 1 changed file with 12 additions and 3 deletions.
15 changes: 12 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -732,8 +732,11 @@ ADD_CXX_FLAGS(-DYB_HEADERS_NO_STUBS=1)
# For all builds:
# For CMAKE_BUILD_TYPE=Debug
# -ggdb: Enable gdb debugging
# -O1: Enable minimum optimization
# -fno-omit-frame-pointer
# use frame pointers to allow simple stack frame walking for backtraces.
# For CMAKE_BUILD_TYPE=FastDebug
# Same as DEBUG, except with some optimizations on.
# Same as DEBUG (TODO: remove fastdebug build type, use debug instead)
# For CMAKE_BUILD_TYPE=Release
# -O3: Enable all compiler optimizations
# -g: Enable symbols for profiler tools (TODO: remove for shipping)
Expand All @@ -747,10 +750,16 @@ ADD_CXX_FLAGS(-DYB_HEADERS_NO_STUBS=1)
# 2. Run the benchmarks (generates *.gcda profiling data).
# 3. Build again with CMAKE_BUILD_TYPE_PROFILE_BUILD
# -fprofile-use: Compiler will use the profile outputs for optimizations
set(CXX_FLAGS_DEBUG "-ggdb")
set(CXX_FLAGS_FASTDEBUG "-ggdb -O1 -fno-omit-frame-pointer")
set(CXX_FLAGS_DEBUG "-ggdb -O1 -fno-omit-frame-pointer")
set(CXX_FLAGS_FASTDEBUG ${CXX_FLAGS_DEBUG})
set(CXX_FLAGS_RELEASE "-O3 -g -DNDEBUG -fno-omit-frame-pointer")

# Nullify CMake's predefined flags since we are handling different build types on our own.
# Without this change CMake will add flags from these variables and
# optimization level will be changed because in case of multiple -O flags gcc uses the last one.
set(CMAKE_CXX_FLAGS_DEBUG "")
set(CMAKE_CXX_FLAGS_RELEASE "")

if (NOT "${YB_USE_LTO}" STREQUAL "")
set(CXX_FLAGS_RELEASE "${CXX_FLAGS_RELEASE} flto -fno-use-linker-plugin")
endif()
Expand Down

0 comments on commit f710367

Please sign in to comment.