Commit Graph

371 Commits

Author SHA1 Message Date
Nikolai Kochetov
75ac87c241 Fixing build. 2020-12-14 17:42:12 +03:00
Nikolai Kochetov
e295dfe6e3 Use ucontext for asan 2020-12-14 17:42:08 +03:00
Azat Khuzhin
77a1d00dee Add new remote context as interpreter context to the Pipe 2020-12-12 17:43:10 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
Nikolai Kochetov
32b38f389e Merge branch 'master' into async-read-from-socket 2020-12-09 17:15:36 +03:00
Nikolai Kochetov
e8667bad45 Fix build and tests. 2020-12-09 17:12:27 +03:00
Nikolai Kochetov
156f44808f Fixing crash. 2020-12-09 17:11:45 +03:00
Nikolai Kochetov
088c128832 Fixing build and tests. 2020-12-09 17:11:20 +03:00
Anton Popov
d7aad3bf79 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-07 19:13:01 +03:00
Pavel Kruglov
4b58528b9e Rename getPos to getRow, change mergeBlock, pass setting instead of context 2020-12-04 19:25:30 +03:00
Pavel Kruglov
905ba78adc Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-12-04 18:56:46 +03:00
Nikolai Kochetov
9ca837d535 Add async status to RemoteSource. 2020-12-04 13:52:57 +03:00
Nikolai Kochetov
92d60a7f38 Add AsyncTaskQueue. 2020-12-03 20:03:13 +03:00
Nikolai Kochetov
082a496364 Add async read to RemoteQueryExecutor 2020-12-03 15:21:10 +03:00
Nikolai Kochetov
e3946bc2b5 Add async read to RemoteQueryExecutor. 2020-12-02 20:02:14 +03:00
Alexey Milovidov
7e06037e59 Fix typo 2020-11-28 08:38:05 +03:00
alexey-milovidov
fb3a69b298
Merge pull request #17254 from azat/fix-dist-query-cancelation
Fix "Unexpected packet Data received from client"  for Distributed queries
2020-11-26 03:46:27 +03:00
Anton Popov
b384beb564 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-23 17:46:51 +03:00
Azat Khuzhin
9689b293ca Fix "Unexpected packet Data received from client" error
Fix query cancelation in case of Distributed queries with LIMIT (when
the initator does not required to read all the data), since this cannot
be done until the query was sent (from the Query packet up to the empty
data Block), otherwise you will get:

    2020.11.21 21:47:23.297161 [ 184 ] {} <Error> TCPHandler: Code: 101, e.displayText() = DB::Exception: Unexpected packet Data received from client, Stack trace:

    0. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:129: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x244f5bc9 in /usr/bin/clickhouse
    1. /build/obj-x86_64-linux-gnu/../src/Common/Exception.cpp:40: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0xa14a421 in /usr/bin/clickhouse
    2. /build/obj-x86_64-linux-gnu/../src/Common/NetException.h:0: DB::TCPHandler::receiveUnexpectedData() @ 0x1e032a74 in /usr/bin/clickhouse
    3. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:824: DB::TCPHandler::receivePacket() @ 0x1e024685 in /usr/bin/clickhouse
    4. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:178: DB::TCPHandler::runImpl() @ 0x1e01736b in /usr/bin/clickhouse
    5. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x1e035c1b in /usr/bin/clickhouse
    6. /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x243559cf in /usr/bin/clickhouse
    7. /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:0: Poco::Net::TCPServerDispatcher::run() @ 0x24356521 in /usr/bin/clickhouse
    8. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x24609175 in /usr/bin/clickhouse
    9. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x24603cb7 in /usr/bin/clickhouse
    10. start_thread @ 0x9669 in /usr/lib/x86_64-linux-gnu/libpthread-2.30.so
    11. __clone @ 0x1222b3 in /usr/lib/x86_64-linux-gnu/libc-2.30.so
2020-11-22 12:42:15 +03:00
Anton Popov
1f65478b49 implement nested with multiple nesting 2020-11-21 20:27:42 +03:00
Alexander Kuzmenkov
7958b739e8 compile 2020-11-18 20:44:33 +03:00
Alexander Kuzmenkov
1570320e20 fixes for context hierarchy 2020-11-18 20:43:18 +03:00
Pavel Kruglov
547ec19fb3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-18 12:01:59 +03:00
Nikita Mikhaylov
68b6f08e36
Merge pull request #16721 from nikitamikhaylov/parsing-constraints
Turn off parallel parsing when memory limit is small.
2020-11-17 17:37:14 +03:00
alexey-milovidov
b494b18748
Merge pull request #17108 from ClickHouse/ya-make-owners
Add "OWNER" to ya.make files.
2020-11-17 05:19:46 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
tavplubix
88de1b052c
Merge pull request #16048 from ClickHouse/materialized-view-check-stack-size
Fix possible stack overflow in materialized views.
2020-11-16 12:02:27 +03:00
Pavel Kruglov
8d5e0784d3 Add setting optimize_on_insert 2020-11-12 23:37:23 +03:00
Nikita Mikhaylov
33bada767c
Merge branch 'master' into parsing-constraints 2020-11-12 23:25:39 +03:00
Nikolai Kochetov
46f70dd0de Merge branch 'master' into actions-dag-f14 2020-11-12 11:54:44 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Nikolai Kochetov
6717c7a0af Merge branch 'master' into actions-dag-f14 2020-11-09 14:57:48 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream
Fix "server failed to start" error
2020-11-08 14:27:08 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
nikitamikhaylov
dabb23b668 done 2020-11-06 21:55:13 +03:00
Nikolai Kochetov
9aeb757da4 Merge branch 'master' into actions-dag-f14 2020-11-06 15:04:20 +03:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
Nikolai Kochetov
9f7de6a1a4 Try fix header for TTLBlockInputStream. 2020-11-03 17:26:47 +03:00
Nikolai Kochetov
6767a226fc Merge branch 'master' into actions-dag-f14 2020-11-03 15:21:06 +03:00
Nikolai Kochetov
07a7c46b89 Refactor ExpressionActions [Part 3] 2020-11-03 14:28:28 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
Alexey Milovidov
32ed8c9681 Fix trailing whitespace 2020-10-26 22:12:40 +03:00
Alexey Milovidov
cc3feb36a6 Avoid stack overflow in materialized views, part 3: more checks 2020-10-16 03:21:50 +03:00
Amos Bird
47fcd8bffb
Fix some problems introduced by refactoring. 2020-10-12 17:58:09 +08:00
Amos Bird
d2dcfc3f0d
Refactor processors. 2020-10-12 17:30:05 +08:00
alexey-milovidov
10e9d14466
Merge pull request #15818 from ClickHouse/style-pragma-once
Check for #pragma once in headers
2020-10-11 13:14:09 +03:00
alexey-milovidov
e16c1c781e
Merge pull request #15816 from ClickHouse/change-level-of-some-log-messages
Change level of some log messages
2020-10-11 13:13:13 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Alexey Milovidov
7502fad77c Change level of some log messages 2020-10-10 20:47:34 +03:00
Alexey Milovidov
5b482f4191 Cleanups 2020-10-10 19:31:10 +03:00
Azat Khuzhin
b838214a35 Pass non-const SelectQueryInfo (and drop mutable qualifiers) 2020-10-02 22:42:35 +03:00
Azat Khuzhin
587cde853e Avoid skipping unused shards twice (for query processing stage and read itself) 2020-10-02 22:42:09 +03:00
alesapin
17b86dec96 Recursive defaults 2020-10-02 15:38:50 +03:00
Anton Popov
0d79474acc Fix TTL with GROUP BY and fix test according to new logic 2020-09-30 19:10:15 +03:00
Anton Popov
8ea4c2e26f Fix TTL in cases, when its expression is a function and is the same as ORDER BY key 2020-09-29 19:31:19 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Nikolai Kochetov
e29f27e333
Merge pull request #14815 from ClickHouse/storage-read-query-plan
Return query plan from storage.
2020-09-15 18:34:11 +03:00
Nikolai Kochetov
6eac433c55 Review fixes. 2020-09-15 13:40:39 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Nikolai Kochetov
1b618fd130 Fix build 2020-09-14 23:56:56 +03:00
Nikolai Kochetov
d09422e60f Fix build. 2020-09-14 22:49:46 +03:00
Nikolai Kochetov
bcfc83022a Return query plan from storage. 2020-09-14 17:13:58 +03:00
alesapin
bd14c79b0b
Merge pull request #14494 from ClickHouse/recompression_in_background
Recompression TTL
2020-09-09 17:46:01 +03:00
Vasily Nemkov
3973a17530
MySql datatypes dateTime64 and decimal (#11512) 2020-09-09 15:18:02 +03:00
alesapin
a696cf12f6 Merge branch 'master' into recompression_in_background 2020-09-09 10:12:03 +03:00
alexey-milovidov
f2e9a09b11
Merge pull request #14585 from ClickHouse/join-processor
Remove Join from ExpressionActions
2020-09-08 23:56:09 +03:00
alesapin
f0f7f2b5d2 Merge branch 'master' into recompression_in_background 2020-09-08 15:43:37 +03:00
Nikolai Kochetov
39aad9979a Remove Join from ExpressionActions. 2020-09-08 13:40:53 +03:00
Ilya Yatsishin
11a247d2f4
Merge pull request #14556 from ClickHouse/fix-arcadia-2
Lower binary size in "Arcadia" build
2020-09-08 12:08:35 +03:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
alesapin
f274ffc9d1 Add comments 2020-09-07 10:59:14 +03:00
alesapin
fecb2f1311 Less copypaste 2020-09-04 17:08:43 +03:00
Nikolai Kochetov
92c937db8b Remove CreatingSetsBlockInputStream 2020-09-02 16:13:13 +03:00
alesapin
f0dc5a3085 First working test 2020-09-01 13:49:53 +03:00
alesapin
10c7a6c45e
Add ability to specify Default codec for columns (#14049)
* Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml

* Fix style

* Rename DefaultCompression to simple Default

* Fix compression codec

* Better codec description representation

* Less strange code and one method

* Fix delta
2020-08-28 20:40:45 +03:00
alesapin
7428c46737 Merge branch 'master' into recompressing_in_background 2020-08-26 10:28:03 +03:00
alesapin
1180bad3a7 Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml 2020-08-25 18:02:32 +03:00
Alexey Milovidov
612382b74e Minor modification 2020-08-24 17:45:00 +03:00
Alexey Milovidov
e1ccebe8f3 Minor modification 2020-08-24 17:42:12 +03:00
Azat Khuzhin
2e933a94a6 Handle Protocol::Server::Log in the RemoteQueryExecutor::finish()
Even when finish() is called, there can be no EndOfStream sent, so it is
100% correct to handle Log packages there.

W/o fix in RemoteQueryExecutor:

    Code: 100, e.displayText() = DB::Exception: Unknown packet 10 from one of the following replicas: : While executing Remote (version 20.8.1.1) (from [::1]:56960) (in query: select * from remote('127.{2,3}', system.numbers) where number = 10 limit 1; ), Stack trace (when copying this message, always include the lines below):

    0. Common/StackTrace.cpp:291: StackTrace::tryCapture() @ 0x28b80a
    1. Common/StackTrace.cpp:256: StackTrace::StackTrace() @ 0x28b6ab
    2. Common/Exception.cpp:42: DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) @ 0x2481ba
    3. Common/Exception.h:28: DB::Exception::Exception<>() @ 0x334d48
    4. DataStreams/RemoteQueryExecutor.cpp:288: DB::RemoteQueryExecutor::finish() @ 0x330314
    5. Processors/Sources/RemoteSource.cpp:46: DB::RemoteSource::generate() @ 0x6fce5
2020-08-17 21:55:09 +03:00
Azat Khuzhin
3c6a8ed7be Improve error message on unknown packet for distributed queries
Although replica information is likely already empty (since it was
reseted due to error or disconnect), but let's keep it for now.
2020-08-17 21:38:24 +03:00
Nikolai Kochetov
9b67cd9faf Merge branch 'master' into refactor-pipes-3 2020-08-10 10:50:17 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
Nikolai Kochetov
20e63d2271 Refactor Pipe [part 6] 2020-08-06 15:24:05 +03:00
Nikolai Kochetov
9385f3de0e Refactor Pipe [part 4]. 2020-08-04 16:06:59 +03:00
Nikolai Kochetov
09fbce1b1e Merge branch 'master' into refactor-pipes-3 2020-08-04 11:32:34 +03:00
Nikolai Kochetov
e411916bde Refactor Pipe [part 1]. 2020-08-03 14:33:11 +03:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Vitaly Baranov
18b21511a9
Merge pull request #13013 from vitlibar/implement-custom-settings
Implement custom settings
2020-08-02 05:01:14 +03:00
Nikolai Kochetov
2ae94f4570 Refactor Pipes part 1. 2020-07-31 19:54:54 +03:00
Vitaly Baranov
cd372de417 Use SettingUInt64 only in Settings, use UInt64 in other places. 2020-07-31 19:11:27 +03:00
Nikolai Kochetov
39530f837e Remove TreeExecutorBlockInputStream. 2020-07-31 16:23:19 +03:00
alexey-milovidov
065db05665
Merge pull request #12638 from ianton-ru/selected_rows_event
Add SelectedRows and SelectedBytes events
2020-07-31 12:48:46 +03:00
Alexander Kuzmenkov
f6f2812f73 Block structure mismatch is a program error 2020-07-28 21:35:18 +03:00
Anton Ivashkin
89a59d43f2 Merge 2020-07-27 17:59:12 +03:00
Azat Khuzhin
417dc43353 Fix thread group for ParallelParsingBlockInputStream after removing std::bind 2020-07-23 19:52:09 +03:00
Azat Khuzhin
a5885b476f Fix modernize-avoid-bind in ParallelParsingBlockInputStream 2020-07-23 11:31:13 +03:00
Azat Khuzhin
303635a59e Fix memory tracking for input_format_parallel_parsing by attaching thread to group 2020-07-23 11:31:13 +03:00
Azat Khuzhin
77eab4ac91 Move ParallelParsingBlockInputStream code into module part 2020-07-23 11:31:12 +03:00
Artem Zuikov
2afd123eda
Refactoring: extract TreeOptimizer from SyntaxAnalyzer (#12645) 2020-07-22 20:13:05 +03:00
Anton Ivashkin
742793be54 Add SelectedRows and SelectedBytes events 2020-07-21 17:50:02 +03:00
Nikolai Kochetov
9a414a090c Fix style. 2020-07-21 11:52:02 +03:00
Nikolai Kochetov
cab9146ff3 Remove SortDescription from IBlockInputStream. 2020-07-21 11:05:52 +03:00
alesapin
5978c9720d Better exception message in CheckingSortedBlockInputStream 2020-07-20 17:57:01 +03:00
alexey-milovidov
0f1bfa5b19
Merge pull request #12042 from ClickHouse/fix-constraints
Fix constraints check for constant columns, step 1.
2020-07-05 19:01:33 +03:00
Nikolai Kochetov
57d727d078 Fix result_rows and result_bytes metrics for selects. 2020-07-02 17:51:10 +03:00
Alexey Milovidov
a23df81dd5 Fix error 2020-06-30 03:53:30 +03:00
Alexey Milovidov
d97cb1e7b4 Fix constraints check for constant columns 2020-06-30 01:54:20 +03:00
alesapin
e72bc39654
Merge pull request #11983 from ClickHouse/storage_mongodb
Merging #10931 (Storage MongoDB)
2020-06-29 11:49:18 +03:00
alesapin
6b3092290d Trying to fix ya.make 2020-06-26 19:33:51 +03:00
alesapin
6fc6422c4d Remove empty lines 2020-06-26 18:19:00 +03:00
alesapin
252706e956 Add comment 2020-06-26 17:34:37 +03:00
alesapin
4969da85d9 Fixes 2020-06-26 17:28:00 +03:00
Artem Zuikov
bfe30a9723 remove a trick with expression continuation 2020-06-25 22:26:03 +03:00
alesapin
f262c4f652 Better comments 2020-06-22 12:49:21 +03:00
alesapin
b9e74f4e82 Merge branch 'master' into atomic_metadata5 2020-06-22 12:03:53 +03:00
Alexey Milovidov
4dcc0a0f86 Better diagnostics for logical error triggered by test 00992 2020-06-22 00:07:06 +03:00
alesapin
d79982f497 Better locks in Storages 2020-06-18 19:10:47 +03:00
alesapin
d4c49816ab Really atomic metadata 2020-06-18 14:02:31 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
31abbe5dbd Select query in metadata 2020-06-17 17:06:22 +03:00
alesapin
ed8f3b2fc4 TTL in storage in memory metadata 2020-06-17 16:39:26 +03:00
alesapin
2fac3290f0 Merge branch 'master' into atomic_metadata5 2020-06-17 11:14:03 +03:00
Alexey Milovidov
a655765040 Merge branch 'master' into return-not-nullable-from-count-distinct-2 2020-06-17 01:15:57 +03:00
Artem Zuikov
857d84bf7c
Buffer left blocks optimisation for MergeJoin (#10601) 2020-06-16 23:13:18 +03:00
Alexey Milovidov
0a6054eece Merge branch 'master' into return-not-nullable-from-count-distinct-2 2020-06-16 22:31:33 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
alesapin
08b9aa6b2e getSampleBlockWithVirtuals in StorageInMemoryMetadata 2020-06-16 15:58:05 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
Alexey Milovidov
394fb64a9c Better way of implementation 2020-06-14 20:42:11 +03:00
alesapin
3427da1c43 Fix incorrect usage of rows TTL 2020-06-13 13:39:25 +03:00
alesapin
4f398b5dea Merge remote-tracking branch 'origin/master' into consistent_metadata4 2020-06-09 23:53:38 +03:00
Nikolai Kochetov
595fc9eff6
Merge pull request #11440 from ClickHouse/add-remote-source
Add remote source
2020-06-09 19:27:28 +03:00
alesapin
ac67d0485c Merge branch 'master' into consistent_metadata4 2020-06-08 15:06:51 +03:00
Alexey Milovidov
f2cfb7d132 Whitespaces 2020-06-08 00:05:36 +03:00
Alexey Milovidov
d3d02a67f4 Speed up build by removing old example programs 2020-06-06 19:19:16 +03:00
alesapin
5bc7f67e61 Better metadata for select query and renames 2020-06-05 14:54:54 +03:00
Nikolai Kochetov
b419d73880 Fix build. 2020-06-04 16:16:58 +03:00
Nikolai Kochetov
83b6467308 Added RemoteSource. 2020-06-03 22:50:11 +03:00
Nikolai Kochetov
cba05a0dd7 Fix style. 2020-06-02 21:10:42 +03:00
Nikolai Kochetov
9b4db3edb0 Fix totals for remote. 2020-06-02 19:37:30 +03:00
Nikolai Kochetov
5f8c2c4dd5 Fix build. 2020-06-02 19:30:50 +03:00
Nikolai Kochetov
f1cccf31b2 Fix build. 2020-06-02 19:29:29 +03:00
Nikolai Kochetov
b1d1034111 Refactor RemoteBlockInputStream. 2020-06-02 19:27:05 +03:00
Nikolai Kochetov
13c5ec5b54 Refactor RemoteBlockInputStream. 2020-06-02 18:59:57 +03:00
Nikolai Kochetov
9a283de670
Merge pull request #11339 from ClickHouse/remove-experimantal-use-processors-flag-5
Remove some streams.
2020-06-02 14:46:02 +03:00
Nikolai Kochetov
45a1db7a80 Try fix tets. 2020-06-02 08:16:45 +03:00
alexey-milovidov
92ac608447
Merge pull request #11330 from ClickHouse/fix-10241
More parallel MV processing.
2020-06-02 04:06:39 +03:00