Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash when selecting with FINAL and PREWHERE from table with ReplacingMergeTree engine #2525

Closed
Schmak opened this issue Jun 18, 2018 · 2 comments
Closed

Crash when selecting with FINAL and PREWHERE from table with ReplacingMergeTree engine #2525

Schmak opened this issue Jun 18, 2018 · 2 comments

Comments

@Schmak
Copy link

Schmak commented Jun 18, 2018

Environment:
Version: v1.1.54385-testing
Git hash: ba48e31
Docker image: sha256:2f64d4c08bbb10b415dac1ba78db82b0ef05b20b2b60dd0dd273a678964ae5ce
Can be reproduced on earlier builds too, but works well in:
Version: v1.1.54342-testing
Docker image: sha256:e4af84ec8e1a610e6b9aaef7c51135cbe017fdaca80170ee4b96d0d14086196f_

Create table and fill it with some data:

CREATE TABLE test (id String, version Date) ENGINE = ReplacingMergeTree(version, id, 8192)

INSERT INTO test (id, version) VALUES ('2018-01-01', '2018-01-01')

Request data with FINAL modifier and PREWHERE condition, which should return empty result set:

SELECT * FROM test FINAL 
PREWHERE id=='2018-01-02'

ClickHouse server crashes with the following stacktrace:

2018.06.18 10:48:53.691142 [ 1 ] <Error> Application: Cannot resolve listen_host (::), error -9: Address family for hostname not supported: -9. If it is an IPv6 address and your host has disabled IPv6, then consider to specify IPv4 address to listen in <listen_host> element of configuration file. Example: <listen_host>0.0.0.0</listen_host>
2018.06.18 10:48:53.691229 [ 1 ] <Error> Application: Listen [::]: -9: DNS error: Address family for hostname not supported: -9  If it is an IPv6 or IPv4 address and your host has disabled IPv6 or IPv4, then consider to specify not disabled IPv4 or IPv6 address to listen in <listen_host> element of configuration file. Example for disabled IPv6: <listen_host>0.0.0.0</listen_host> . Example for disabled IPv4: <listen_host>::</listen_host>
2018.06.18 10:49:31.080614 [ 22 ] <Error> BaseDaemon: ########################################
2018.06.18 10:49:31.080644 [ 22 ] <Error> BaseDaemon: (from thread 21) Received signal Segmentation fault (11).
2018.06.18 10:49:31.080655 [ 22 ] <Error> BaseDaemon: Address: NULL pointer.
2018.06.18 10:49:31.080665 [ 22 ] <Error> BaseDaemon: Access: read.
2018.06.18 10:49:31.080675 [ 22 ] <Error> BaseDaemon: Address not mapped to object.
2018.06.18 10:49:31.140609 [ 22 ] <Error> BaseDaemon: 0. /usr/bin/clickhouse-server(DB::ReplacingSortedBlockInputStream::insertRow(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long&)+0x185) [0x803c775]
2018.06.18 10:49:31.140660 [ 22 ] <Error> BaseDaemon: 1. /usr/bin/clickhouse-server(DB::ReplacingSortedBlockInputStream::merge(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, std::priority_queue<DB::SortCursor, std::vector<DB::SortCursor, std::allocator<DB::SortCursor> >, std::less<DB::SortCursor> >&)+0x407) [0x803cca7]
2018.06.18 10:49:31.140681 [ 22 ] <Error> BaseDaemon: 2. /usr/bin/clickhouse-server(DB::ReplacingSortedBlockInputStream::readImpl()+0xf3) [0x803d763]
2018.06.18 10:49:31.140696 [ 22 ] <Error> BaseDaemon: 3. /usr/bin/clickhouse-server(DB::IProfilingBlockInputStream::read()+0x25a) [0x75f5d4a]
2018.06.18 10:49:31.140711 [ 22 ] <Error> BaseDaemon: 4. /usr/bin/clickhouse-server(DB::ExpressionBlockInputStream::readImpl()+0x2e) [0x7fd553e]
2018.06.18 10:49:31.140727 [ 22 ] <Error> BaseDaemon: 5. /usr/bin/clickhouse-server(DB::IProfilingBlockInputStream::read()+0x25a) [0x75f5d4a]
2018.06.18 10:49:31.140742 [ 22 ] <Error> BaseDaemon: 6. /usr/bin/clickhouse-server(DB::ExpressionBlockInputStream::readImpl()+0x2e) [0x7fd553e]
2018.06.18 10:49:31.140757 [ 22 ] <Error> BaseDaemon: 7. /usr/bin/clickhouse-server(DB::IProfilingBlockInputStream::read()+0x25a) [0x75f5d4a]
2018.06.18 10:49:31.140777 [ 22 ] <Error> BaseDaemon: 8. /usr/bin/clickhouse-server(DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::atomic<bool>*)+0x81) [0x760b691]
2018.06.18 10:49:31.140799 [ 22 ] <Error> BaseDaemon: 9. /usr/bin/clickhouse-server(DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, DB::Context&, std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)+0x563) [0x7d6f763]
2018.06.18 10:49:31.140817 [ 22 ] <Error> BaseDaemon: 10. /usr/bin/clickhouse-server(DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&)+0x2d67) [0x2ce8787]
2018.06.18 10:49:31.140835 [ 22 ] <Error> BaseDaemon: 11. /usr/bin/clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x24c) [0x2ceab0c]
2018.06.18 10:49:31.140850 [ 22 ] <Error> BaseDaemon: 12. /usr/bin/clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x332) [0x86dcbf2]
2018.06.18 10:49:31.140865 [ 22 ] <Error> BaseDaemon: 13. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x86d549f]
2018.06.18 10:49:31.140880 [ 22 ] <Error> BaseDaemon: 14. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x16a) [0x86d587a]
2018.06.18 10:49:31.140906 [ 22 ] <Error> BaseDaemon: 15. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x77) [0x8776a37]
2018.06.18 10:49:31.140920 [ 22 ] <Error> BaseDaemon: 16. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x38) [0x8772c48]
2018.06.18 10:49:31.140934 [ 22 ] <Error> BaseDaemon: 17. /usr/bin/clickhouse-server() [0x8dd3a8f]
2018.06.18 10:49:31.140948 [ 22 ] <Error> BaseDaemon: 18. /lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc) [0x7f03fbaf57fc]

If you omit FINAL modifier or use WHERE instead of PREWHERE or use condition which will return non-empty result set, then server will not crash.

See attached files for full log:

clickhouse-server.log
clickhouse-server.err.log
@proller
Copy link
Contributor

proller commented Jun 20, 2018

ready test:

DROP TABLE IF EXISTS test.final_test;                                                                 
CREATE TABLE test.final_test (id String, version Date) ENGINE = ReplacingMergeTree(version, id, 8192);
INSERT INTO test.final_test (id, version) VALUES ('2018-01-01', '2018-01-01');                        
SELECT * FROM test.final_test FINAL PREWHERE id == '2018-01-02';                                      
DROP TABLE test.final_test;

broken in 66b94d4

@adubovikov
Copy link

confirm, same issue, if the query contains FINAL and PREWHERE:

2018.06.21 09:20:28.015993 [ 12 ] <Debug> data.test_table (SelectExecutor): Key condition: unknown, unknown, and, (column 0 in [1529536154257000, +inf)), (column 0 in (-inf, 1529540354257000]), and, and, unknown, and
2018.06.21 09:20:28.016047 [ 12 ] <Debug> data.test_table (SelectExecutor): MinMax index condition: (column 0 in [1529536154, +inf)), (column 0 in (-inf, 1529540354]), and, unknown, unknown, and, and, unknown, and
2018.06.21 09:20:28.017461 [ 12 ] <Debug> data.test_table (SelectExecutor): Selected 19 parts by date, 19 parts by key, 927 marks to read from 19 ranges
2018.06.21 09:20:28.017554 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_15770_19149_7, approx. 1925120 rows starting from 204800
2018.06.21 09:20:28.017710 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_19154_21879_7, approx. 1957888 rows starting from 0
2018.06.21 09:20:28.017841 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_21880_23278_6, approx. 1064960 rows starting from 0
2018.06.21 09:20:28.018010 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_23279_23652_5, approx. 303104 rows starting from 0
2018.06.21 09:20:28.018153 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_23653_23801_5, approx. 131072 rows starting from 0
2018.06.21 09:20:28.018267 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_23802_24073_6, approx. 212992 rows starting from 0
2018.06.21 09:20:28.018393 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_24076_24493_6, approx. 303104 rows starting from 0
2018.06.21 09:20:28.018508 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_24494_24583_4, approx. 24576 rows starting from 0
2018.06.21 09:20:28.018628 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_24585_57526_1013, approx. 16384 rows starting from 0
2018.06.21 09:20:28.018751 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_58416_59117_1, approx. 8192 rows starting from 0
2018.06.21 09:20:28.018871 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424871_61331_61331_0, approx. 8192 rows starting from 0
2018.06.21 09:20:28.018993 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_24506_27048_8, approx. 851968 rows starting from 0
2018.06.21 09:20:28.019118 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_27049_29760_7, approx. 737280 rows starting from 0
2018.06.21 09:20:28.019227 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_29761_30902_8, approx. 8192 rows starting from 0
2018.06.21 09:20:28.019337 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_30903_33462_8, approx. 8192 rows starting from 0
2018.06.21 09:20:28.019462 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_33463_33938_7, approx. 8192 rows starting from 0
2018.06.21 09:20:28.019573 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_33939_34848_8, approx. 8192 rows starting from 0
2018.06.21 09:20:28.019696 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_34854_40466_68, approx. 8192 rows starting from 0
2018.06.21 09:20:28.019803 [ 12 ] <Trace> MergeTreeBlockInputStream: Reading 1 ranges from part 424872_40468_71651_140, approx. 8192 rows starting from 0
2018.06.21 09:20:28.020029 [ 12 ] <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2018.06.21 09:20:28.021310 [ 12 ] <Debug> executeQuery: Query pipeline:
Limit
 Expression
  Expression
   ReplacingSorted
    Expression × 19
     MergeTree

2018.06.21 09:20:28.399744 [ 23 ] <Error> BaseDaemon: ########################################
2018.06.21 09:20:28.399830 [ 23 ] <Error> BaseDaemon: (from thread 22) Received signal Segmentation fault (11).
2018.06.21 09:20:28.399861 [ 23 ] <Error> BaseDaemon: Address: NULL pointer.
2018.06.21 09:20:28.399903 [ 23 ] <Error> BaseDaemon: Access: read.
2018.06.21 09:20:28.399911 [ 23 ] <Error> BaseDaemon: Address not mapped to object.
2018.06.21 09:20:28.429556 [ 23 ] <Error> BaseDaemon: 0. clickhouse-server(DB::ReplacingSortedBlockInputStream::insertRow(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long&)+0x1c5) [0x46c7585]
2018.06.21 09:20:28.429615 [ 23 ] <Error> BaseDaemon: 1. clickhouse-server(DB::ReplacingSortedBlockInputStream::merge(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, std::priority_queue<DB::SortCursor, std::vector<DB::SortCursor, std::allocator<DB::SortCursor> >, std::less<DB::SortCursor> >&)+0x3ee) [0x46c7ace]
2018.06.21 09:20:28.429633 [ 23 ] <Error> BaseDaemon: 2. clickhouse-server(DB::ReplacingSortedBlockInputStream::readImpl()+0xba) [0x46c84ba]
2018.06.21 09:20:28.429647 [ 23 ] <Error> BaseDaemon: 3. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x3d30abd]
2018.06.21 09:20:28.429661 [ 23 ] <Error> BaseDaemon: 4. clickhouse-server(DB::ExpressionBlockInputStream::readImpl()+0x1b) [0x466b54b]
2018.06.21 09:20:28.429674 [ 23 ] <Error> BaseDaemon: 5. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x3d30abd]
2018.06.21 09:20:28.429688 [ 23 ] <Error> BaseDaemon: 6. clickhouse-server(DB::ExpressionBlockInputStream::readImpl()+0x1b) [0x466b54b]
2018.06.21 09:20:28.429701 [ 23 ] <Error> BaseDaemon: 7. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x3d30abd]
2018.06.21 09:20:28.429714 [ 23 ] <Error> BaseDaemon: 8. clickhouse-server(DB::LimitBlockInputStream::readImpl()+0x25b) [0x3d317db]
2018.06.21 09:20:28.429727 [ 23 ] <Error> BaseDaemon: 9. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x3d30abd]
2018.06.21 09:20:28.429741 [ 23 ] <Error> BaseDaemon: 10. clickhouse-server(DB::AsynchronousBlockInputStream::calculate(MemoryTracker*)+0x47) [0x26f0217]
2018.06.21 09:20:28.429755 [ 23 ] <Error> BaseDaemon: 11. clickhouse-server(ThreadPool::worker()+0x167) [0x4bc0667]
2018.06.21 09:20:28.429767 [ 23 ] <Error> BaseDaemon: 12. clickhouse-server() [0x4f7345f]
2018.06.21 09:20:28.429779 [ 23 ] <Error> BaseDaemon: 13. /usr/lib64/libpthread.so.0(+0x7e25) [0x7f51712ede25]

KochetovNicolai added a commit that referenced this issue Jun 25, 2018
@KochetovNicolai
Fixed reading from ReplacingMergeTree from emprty row set after prewh…
91c9906 
…ere. #2525
KochetovNicolai added a commit that referenced this issue Jun 25, 2018
@KochetovNicolai
Added test. #2525
7c95d15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@proller @KochetovNicolai @adubovikov @Schmak