thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
101,964 Commits 9,338 Branches 1,783 Tags 2.1 GiB
c2e92fd274
Commit Graph

262 Commits

Author SHA1 Message Date
Amos Bird
2c4bc43014
Backward compatible 2021-02-18 20:20:29 +08:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
alexey-milovidov
dc3ffd3fe2
Merge pull request #19451 from azat/safe-writes 
Do not silently ignore write errors
 2021-02-11 21:19:11 +03:00
Alexey Milovidov
dbb41ce4e2 Remove debug output 2021-02-11 02:55:52 +03:00
Alexey Milovidov
f442b30f30 Fix test 2021-02-11 01:23:27 +03:00
Alexey Milovidov
30b2554fa3 Fix error 2021-02-10 20:48:39 +03:00
Alexey Milovidov
869bca74a7 Fix some tests 2021-02-07 23:37:55 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
alesapin
449e8e3fd9 More checks for setting 2021-02-05 13:15:02 +03:00
Azat Khuzhin
64c0bf9829 TCPHandler: catch exceptions from the WriteBuffer in destructor 
For TCPHandler it is safe thing todo.

Otherwise *San will report [1]:

    2021.01.24 15:33:40.103996 [ 270 ] {} <Trace> BaseDaemon: Received signal -1
    2021.01.24 15:33:40.110693 [ 270 ] {} <Fatal> BaseDaemon: (version 21.2.1.5789, build id: FF421B087D1E2EAA19FA17B5AB3AE413832744E0) (from thread 48318) Terminate called for uncaught exception:
    2021.01.24 15:33:40.114845 [ 270 ] {} <Trace> BaseDaemon: Received signal 6
    2021.01.24 15:33:40.138738 [ 218027 ] {} <Fatal> BaseDaemon: ########################################
    2021.01.24 15:33:40.138838 [ 218027 ] {} <Fatal> BaseDaemon: (version 21.2.1.5789, build id: FF421B087D1E2EAA19FA17B5AB3AE413832744E0) (from thread 48318) (no query) Received signal Aborted (6)
    2021.01.24 15:33:40.138912 [ 218027 ] {} <Fatal> BaseDaemon:
    2021.01.24 15:33:40.139277 [ 218027 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f185474118b 0x7f1854720859 0xaddc0cc 0x2af9fab8 0x2af9fa04 0xa91758b 0x1e418bb5 0x20725b4f 0x20725d9e 0x266b47a3 0x269772f5 0x26971847 0x7f18548f6609 0x7f185481d293
    2021.01.24 15:33:40.139637 [ 218027 ] {} <Fatal> BaseDaemon: 3. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.01.24 15:33:40.140113 [ 218027 ] {} <Fatal> BaseDaemon: 4. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.01.24 15:33:40.144121 [ 218027 ] {} <Fatal> BaseDaemon: 5. ./obj-x86_64-linux-gnu/../base/daemon/BaseDaemon.cpp:0: terminate_handler() @ 0xaddc0cc in /usr/bin/clickhouse
    2021.01.24 15:33:40.151208 [ 218027 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:61: std::__terminate(void (*)()) @ 0x2af9fab8 in /usr/bin/clickhouse
    2021.01.24 15:33:40.153085 [ 218027 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:0: std::terminate() @ 0x2af9fa04 in /usr/bin/clickhouse
    2021.01.24 15:33:40.155209 [ 218027 ] {} <Fatal> BaseDaemon: 8. ? @ 0xa91758b in /usr/bin/clickhouse
    2021.01.24 15:33:40.156621 [ 218027 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::~WriteBufferFromPocoSocket() @ 0x1e418bb5 in /usr/bin/clickhouse
    2021.01.24 15:33:40.161041 [ 218027 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518: DB::TCPHandler::~TCPHandler() @ 0x20725b4f in /usr/bin/clickhouse
    2021.01.24 15:33:40.164557 [ 218027 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:101: DB::TCPHandler::~TCPHandler() @ 0x20725d9e in /usr/bin/clickhouse
    2021.01.24 15:33:40.165921 [ 218027 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AtomicCounter.h:314: Poco::Net::TCPServerDispatcher::run() @ 0x266b47a3 in /usr/bin/clickhouse
    2021.01.24 15:33:40.167347 [ 218027 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x269772f5 in /usr/bin/clickhouse
    2021.01.24 15:33:40.169401 [ 218027 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x26971847 in /usr/bin/clickhouse
    2021.01.24 15:33:40.169498 [ 218027 ] {} <Fatal> BaseDaemon: 15. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    2021.01.24 15:33:40.169566 [ 218027 ] {} <Fatal> BaseDaemon: 16. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.01.24 15:33:41.027601 [ 218027 ] {} <Fatal> BaseDaemon: Calculated checksum of the binary: 63D7491B39260494BA0D785E1860B427. There is no information about the reference checksum.

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/1e16bd6f337985a82fbdf4eded695dc6e663af58/stress_test_(address).html#fail1

v2: Fix catching errors in WriteBufferFromPocoSocket destructor
 2021-02-05 01:32:36 +03:00
Azat Khuzhin
4beb5c1b8a TCPHandler: Move constructor into the module and add missing headers 2021-02-05 01:31:42 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
Alexey Milovidov
438f0f971b Fix the issue with async Distributed INSERTs and network_compression_method #18741 2021-01-06 03:24:42 +03:00
filimonov
8d9ad60afa
Decrease verbosity of disconnecting clients 
Right now they go to error log.
 2020-12-11 11:51:48 +01:00
Alexey Milovidov
3e2447391b Support for PROXY protocol 2020-12-03 00:05:51 +03:00
Alexander Kuzmenkov
6cb378e072 cleanup 2020-11-19 18:52:11 +03:00
Alexander Kuzmenkov
b16c5a1748 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-17 22:54:54 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexander Kuzmenkov
26229ed231 tmp spans for threads 
(doesn't compile because of json metadata changes)
 2020-11-09 18:07:38 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Alexander Kuzmenkov
5a70e26c70 Merge remote-tracking branch 'origin/master' into HEAD 2020-10-26 19:21:13 +03:00
Alexey Milovidov
74558a4e27 Better diagnostics when client has dropped connection 2020-10-24 06:41:47 +03:00
Alexander Kuzmenkov
41ac15a71c fix initial query id 2020-10-20 00:26:10 +03:00
Alexander Kuzmenkov
6200433e12 Merge origin/master into tmp (using imerge) 2020-10-12 21:15:31 +03:00
Nikolai Kochetov
190fd88af9
Merge pull request #15785 from amosbird/e1 
Code refactor.
 2020-10-12 00:36:36 +03:00
Alexey Milovidov
7502fad77c Change level of some log messages 2020-10-10 20:47:34 +03:00
Amos Bird
8e0862a0c3
BlockIO can be empty. 2020-10-10 23:25:50 +08:00
Amos Bird
dbc14655fc
Code refactoring. 2020-10-10 18:22:33 +08:00
Alexander Kuzmenkov
dde4cf70e1 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 14:03:59 +03:00
Alexander Kuzmenkov
7b64ca33b1 Merge remote-tracking branch 'origin/master' into tmp 2020-09-22 14:02:20 +03:00
Alexander Kuzmenkov
bfc3be9e43 Merge remote-tracking branch 'origin/aku/tcp-versio' into tmp 2020-09-22 14:02:15 +03:00
Vitaly Baranov
02acedd6a3 Use initial_address for quota calculation when the interserver-secret mode enabled. 2020-09-19 01:04:25 +03:00
Alexander Kuzmenkov
fd33fa7f39 fixup 2020-09-17 17:55:41 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Alexander Kuzmenkov
a374541214 straighten the protocol version 2020-09-17 15:15:05 +03:00
Azat Khuzhin
785d1b2a75 OpenSSLHelpers cleanup 
Add few more specializations for encodeSHA256():
- std::string encodeSHA256(const std::string_view &);
- std::string encodeSHA256(const void *, size_t);
- void encodeSHA256(const void *, size_t, unsigned char *);
 2020-09-15 01:36:28 +03:00
Azat Khuzhin
0159c74f21 Secure inter-cluster query execution (with initial_user as current query user) [v3] 
Add inter-server cluster secret, it is used for Distributed queries
inside cluster, you can configure in the configuration file:

  <remote_servers>
      <logs>
          <shard>
              <secret>foobar</secret> <!-- empty -- works as before -->
              ...
          </shard>
      </logs>
  </remote_servers>

And this will allow clickhouse to make sure that the query was not
faked, and was issued from the node that knows the secret. And since
trust appeared it can use initial_user for query execution, this will
apply correct *_for_user (since with inter-server secret enabled, the
query will be executed from the same user on the shards as on initator,
unlike "default" user w/o it).

v2: Change user to the initial_user for Distributed queries if secret match
v3: Add Protocol::Cluster package
v4: Drop Protocol::Cluster and use plain Protocol::Hello + user marker
v5: Do not use user from Hello for cluster-secure (superfluous)
 2020-09-15 01:36:28 +03:00
Azat Khuzhin
ec9d7745f3 Break the query loop on UNEXPECTED_PACKET_FROM_CLIENT 2020-09-15 01:36:28 +03:00
Azat Khuzhin
d53bbd3394 Not only send the exception to the client but also log it 
Otherwise there will be only tricky "Unknown packet" exception after
exception during handling INSERT queries.
 2020-09-15 01:36:28 +03:00
Alexander Kuzmenkov
c62bf555d2 Merge origin/master into tmp (using imerge) 2020-09-08 16:26:38 +03:00
Alexander Kuzmenkov
d0a9926e7d fixes and some docs 2020-09-08 16:19:27 +03:00
alesapin
10c7a6c45e
Add ability to specify Default codec for columns (#14049) 
* Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml

* Fix style

* Rename DefaultCompression to simple Default

* Fix compression codec

* Better codec description representation

* Less strange code and one method

* Fix delta
 2020-08-28 20:40:45 +03:00
Alexander Kuzmenkov
4612794678 opentelemetry context propagation 2020-08-28 04:21:08 +03:00
Vitaly Baranov
56665a15f7 Rework and rename the template class SettingsCollection => BaseSettings. 2020-07-31 20:54:18 +03:00
alexey-milovidov
b5e87f5b25
Revert "Abort on std::out_of_range in debug builds" 2020-07-24 18:12:03 +03:00
Azat Khuzhin
eaaa4543f4 Abort on std::out_of_range in debug builds 
This helps notice them in tests, same as:
- std::logic_error
- LOGICAL_ERROR
 2020-07-23 11:36:12 +03:00
Alexander Kuzmenkov
a4a23673cc fixup 2020-07-17 17:25:13 +03:00
Alexander Kuzmenkov
4089f761e1 fix one out_of_range 2020-07-17 17:20:33 +03:00
Alexander Kuzmenkov
9d728d60d0 Abort on std::logic_error in debug builds 
This helps notice them in tests, same as assert() and LOGICAL_ERROR.
 2020-07-15 15:08:01 +03:00
alesapin
4185fe9975 Merge branch 'master' into atomic_metadata5 2020-06-23 11:13:05 +03:00
Alexey Milovidov
7ba25bd799 Make it work 2020-06-20 14:17:15 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
Alexander Kuzmenkov
435f53e05a Improve error messages 
Also support fmt for exception messages, the same way we use it in
logging macros.
 2020-06-04 20:50:29 +03:00
tavplubix
a4e40fb5f2
Merge pull request #11269 from ClickHouse/add_context_to_idatabase_methods 
Add context to IDatabase methods
 2020-05-29 21:15:35 +03:00
Alexey Milovidov
23d7947549 Fix constraints for DEFAULT fields; fix constraints for temporary tables; improve performance of INSERT for small blocks 2020-05-29 05:08:48 +03:00
Alexander Tokmakov
c071e0a473 add context to DatabaseCatalog methods 2020-05-29 02:01:18 +03:00
Nikolai Kochetov
2dcee3db5c Merge branch 'master' into remove-experimental-use-processors-flag-3 2020-05-27 21:25:34 +03:00
Ivan
5b858de374
Split programs/server into actual program and library (#11186) 
* Split programs/server into actual program and library
 2020-05-27 20:52:52 +03:00