thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
63,062 Commits 9,328 Branches 1,783 Tags 2.1 GiB
e51cd90c6f
Commit Graph

1651 Commits

Author SHA1 Message Date
kssenii
f6610ceaad Try fix build 2021-03-10 08:09:05 +00:00
alexey-milovidov
f4db00bdb2
Merge pull request #21546 from kitaisreal/shell-command-wait-pid-signal-eintr-fix 
ShellCommand waitpid eintr signal fix
 2021-03-10 09:12:53 +03:00
tavplubix
5a6cc876e4
Merge pull request #20626 from ClickHouse/fix_force_drop_materialized_view 
Make force_drop_table work with MaterializedView
 2021-03-09 20:05:51 +03:00
Maksim Kita
43ba554191
Merge pull request #20595 from kitaisreal/cache-dictionaries-lru-cache 
Cache dictionaries lru cache
 2021-03-09 19:14:30 +03:00
alesapin
bcd073278d
Merge pull request #21425 from ClickHouse/persistent_nukeeper_snapshot_storage 
Persistent nukeeper snapshot storage
 2021-03-09 18:37:23 +03:00
Maksim Kita
289712f19d
Merge pull request #21532 from kitaisreal/pod-array-left-pad-not-multiple-of-element-crash-fix 
Pod array left pad not multiple of element crash fix
 2021-03-09 14:14:52 +03:00
Maksim Kita
8dd52f087e Fixed PODArray 2021-03-09 14:14:31 +03:00
Maksim Kita
882dde56ea ShellCommand waitpid eintr signal fix 2021-03-09 13:13:19 +03:00
alesapin
c00c8c36b7 Merge branch 'master' into ianton-ru-s3_zero_copy_replication 2021-03-09 10:59:11 +03:00
Kruglov Pavel
c464e3c57b
Merge pull request #21311 from ucasFL/fix-format 
improve clickhouse-format
 2021-03-09 02:16:05 +03:00
Alexey Milovidov
1b3043a583 Fix error (found by @kitaisreal) 2021-03-09 02:15:33 +03:00
Alexey Milovidov
d3aeb30cc0 Remove unused method 2021-03-09 02:10:26 +03:00
Alexey Milovidov
9e23d16739 Cleanup PODArray 2021-03-09 02:03:46 +03:00
Maksim Kita
9ed791fd30 Updated style check 2021-03-09 01:33:16 +03:00
Maksim Kita
642c45d769 PODArray left pad not multiple of element crash fix 2021-03-09 01:32:31 +03:00
kssenii
5ee2925123 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-08 19:53:48 +00:00
alesapin
f555e65cc8 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-08 22:30:17 +03:00
Alexey Milovidov
355411dfa8 Merge branch 'master' into DateTime64_extended_range 2021-03-08 02:55:14 +03:00
kssenii
8a9fa561c7 Fix style check, build, ya check 2021-03-07 20:07:42 +00:00
kssenii
b7b15fe920 Remove unused 2021-03-07 15:23:20 +00:00
kssenii
9e9bf2bb75 Common base for bridges part 3 2021-03-07 13:55:40 +00:00
Maksim Kita
92107a6ded Fixed documentation message 2021-03-07 14:43:08 +03:00
kssenii
1c4d4c8e54 Better handlers 2021-03-07 11:31:55 +00:00
kssenii
61d8e27ea7 Common base for bridges part 2 2021-03-07 11:21:49 +00:00
alesapin
0ee7425b16 Remove redundant methods 2021-03-07 10:30:50 +03:00
Alexey Milovidov
1bb62f578b Unit tests passing 2021-03-07 04:10:38 +03:00
Alexey Milovidov
8271cec093 Fix error 2021-03-07 02:36:24 +03:00
kssenii
25fc6a29e5 Make common BridgeHelper base class for all bridges 2021-03-06 23:22:22 +00:00
Alexey Milovidov
e33fae7625 Improve test 2021-03-07 01:35:49 +03:00
Maksim Kita
6414da73bf Fixed code review issues 2021-03-06 21:45:51 +03:00
kssenii
f83a5d83a2 Better 2021-03-06 18:44:40 +00:00
Maksim Kita
c314de7388 Fixed documentation 2021-03-06 14:37:56 +03:00
Maksim Kita
717a9b048a Fixed tests 2021-03-06 14:37:36 +03:00
Maksim Kita
7602c6c6cb Fixed sanitizers errors 2021-03-06 14:36:37 +03:00
Maksim Kita
d04e707567 Fixed special build 2021-03-06 14:36:37 +03:00
Maksim Kita
a22ec65e8a Fixed minor issues 2021-03-06 14:36:37 +03:00
Maksim Kita
b6e72334ef Fixed DefaultHash for StringRef 2021-03-06 14:36:37 +03:00
Pavel Kruglov
dbae58f30b Add slowdown_count and show actual information in system.clusters 2021-03-06 02:45:17 +03:00
kssenii
2c080da51b Better 2021-03-05 15:37:43 +00:00
feng lv
1438fd20cb Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-format 2021-03-05 14:33:26 +00:00
kssenii
e0cda1033a More methods 2021-03-05 10:43:47 +00:00
Maksim Kita
cdee265378 Fix typos 2021-03-05 13:27:24 +03:00
kssenii
dd4a7b6e3d First version 2021-03-05 10:19:01 +00:00
Maksim Kita
175c485fcb Fix style check 2021-03-05 13:17:46 +03:00
Maksim Kita
9173544003 Added documentation 2021-03-05 13:17:46 +03:00
Maksim Kita
aa0778933c Added BorrowedObjectPool 2021-03-05 13:17:46 +03:00
Maksim Kita
91e43a6b3b Updated ExecutablePoolDictionarySource 2021-03-05 13:17:46 +03:00
Maksim Kita
6b69d859c1 Added ExecutablePool dictionary source 2021-03-05 13:17:45 +03:00
alesapin
c9c301e10c Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-05 10:12:09 +03:00
Alexey Milovidov
3e77c9b1c5 Merge branch 'master' into DateTime64_extended_range 2021-03-05 03:07:19 +03:00
Nikita Mikhaylov
37d4c69d3a
Merge pull request #21433 from azat/parsing-exception-message-fix 
Fix ParsingException::displayText()
 2021-03-05 01:59:06 +03:00
alesapin
aa1312ac79 Merge with master 2021-03-04 14:43:12 +03:00
alesapin
ff67860770 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-04 14:30:51 +03:00
Anton Ivashkin
bec9de7161 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-04 13:29:45 +03:00
Anton Ivashkin
e69124a0a6 Merge master 2021-03-04 13:26:40 +03:00
Alexey Milovidov
9416f59014 Some progress on tests 2021-03-04 06:48:24 +03:00
Alexey Milovidov
b9a8509f79 Adjustments 2021-03-04 05:18:17 +03:00
Alexey Milovidov
6dc21cd098 Remove discrepances 2021-03-04 03:48:37 +03:00
Alexey Milovidov
9f3760e1e3 Bad filename 2021-03-04 03:18:48 +03:00
Alexey Milovidov
3eb3830dd8 Remove trash 2021-03-04 02:13:49 +03:00
Alexey Milovidov
1096fc328a Make some tests automated 2021-03-04 02:11:27 +03:00
Alexey Milovidov
329074bd02 Fix test 2021-03-04 01:55:56 +03:00
Alexey Milovidov
2632b568ae Move tests to appropriate place 2021-03-04 01:34:30 +03:00
Azat Khuzhin
e7948819f9 Fix ParsingException::displayText() 
Before it was silently try-catched for messages with additional {}, and
it is very easy to trigger, i.e.:

    SELECT toDateTime(format('{}-{}-01 00:00:00', '2021', '1'))

Will print:

    Code: 41. DB::Exception: Received from localhost:9000. DB::Exception: Cannot parse datetime 2021-1-01 00:00:00{}: Cannot parse DateTime from String: while executing 'FUNCTION toDateTime(format('{}-{}-01 00:00:00', '2021', '1') :: 3) -> toDateTime(format('{}-{}-01 00:00:00', '2021', '1')) DateTime : 2'.
 2021-03-03 23:44:51 +03:00
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced 
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
 2021-03-03 23:30:23 +03:00
Azat Khuzhin
70049db143 CurrentMetrics/Increment: Introduce add() 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT 
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
 2021-03-03 23:30:00 +03:00
alesapin
e6083c4dc4 Fix style 2021-03-03 19:25:47 +03:00
alesapin
eb5e416fd5 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-03 19:19:02 +03:00
alesapin
e615299ae6 Fix some bugs and add snapshots validation 2021-03-03 14:10:24 +03:00
feng lv
742e5a05ff fix conflict 2021-03-03 09:38:30 +00:00
alesapin
8e6252b25f Use snapshot object in serialization 2021-03-02 17:30:56 +03:00
alesapin
e7399bf66f Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-02 14:03:42 +03:00
Pavel Kruglov
153bfbfc28 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-03-02 11:59:32 +03:00
alesapin
03960b1eed Some compileable code 2021-03-01 16:33:34 +03:00
Anton Ivashkin
98065ec56e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-01 13:33:40 +03:00
alesapin
91bc4478d7 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-01 13:18:35 +03:00
Nikolai Kochetov
976dbe8077
Merge pull request #20341 from ClickHouse/filter-push-down 
Filter push down
 2021-03-01 12:35:06 +03:00
feng lv
cbeb2853fb improve clickhouse-format 
fix

fix error word
 2021-02-28 11:34:25 +00:00
alexey-milovidov
b8fba768e5
Merge pull request #21264 from ClickHouse/fix_zookeeper_update 
Fix several bugs with ZooKeeper client
 2021-02-28 01:57:04 +03:00
alesapin
9dc3952ff3 Fix typo 2021-02-27 16:30:43 +03:00
alesapin
9e93d7f507 Fix tidy and add comments 2021-02-27 11:07:14 +03:00
Pavel Kruglov
197058dc12 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-27 00:36:33 +03:00
alesapin
a48bb7e3a0 Fix redundant zookeeper reconnect and possibility of multiple zookeeper session for a single ClickHouse server 2021-02-26 22:05:25 +03:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
Nikolai Kochetov
545528917f Fix tests. 2021-02-26 13:17:00 +03:00
Anton Ivashkin
5b267b7eec Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-02-26 13:15:18 +03:00
Anton Ivashkin
c891cf4557 Fixes by review response 2021-02-26 12:48:57 +03:00
Nikolai Kochetov
64ea1f8298 Save packet keys. 2021-02-25 18:51:01 +03:00
Maksim Kita
63173eebdb
Merge pull request #21088 from azat/__has_feature-gcc-fix 
Fix __has_feature macros under gcc
 2021-02-25 18:34:14 +03:00
alesapin
cf42824756 Experimenting 2021-02-25 17:22:51 +03:00
Azat Khuzhin
810023a65c Do not undefine __has_feature (may break c++ headers) 
If the order of <common/defines.h> and c++ header will be wrong the
compilation will be broken.

v2: rename __ch_has_feature to ch_has_feature to fix -Wreserved-id-macro
v3: do not fallback to 0
 2021-02-23 19:46:47 +03:00
Azat Khuzhin
2ced21c837 Decrease number of stored stack frame pointers to fit into 512 PIPE_BUF 2021-02-23 07:52:01 +03:00
Azat Khuzhin
6b39e61d09 Decrease QUERY_ID_MAX_LEN for freebsd/osx (PIPE_BUF=512) 2021-02-23 07:47:58 +03:00
Azat Khuzhin
89c311981d Fix buffer size for trace collection 
[1]: https://clickhouse-test-reports.s3.yandex.net/0/a860e3e7582f489a640c42db3bd4bd320f97a5bc/stress_test_(debug).html#fail1
 2021-02-22 22:45:04 +03:00
Kruglov Pavel
815b4ff0d9
Merge branch 'master' into hedged-requests 2021-02-21 17:10:24 +03:00
Pavel Kruglov
dd5185c779 Refactor 4.0 2021-02-21 17:03:24 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
Alexey Milovidov
00e6b6232f Revert "optimize aggfunc column data copy (#19407)" 
This reverts commit 7e3186c826.
 2021-02-21 12:49:50 +03:00
tavplubix
103d569885
Merge branch 'master' into fix_force_drop_materialized_view 2021-02-20 20:41:03 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated 
DatabaseReplicated: continuation of #10485
 2021-02-20 19:39:34 +03:00
Alexander Tokmakov
2a36d6cb55 review suggestions 2021-02-20 02:41:58 +03:00
Alexander Tokmakov
5cfd687dfb fix 2021-02-19 19:22:47 +03:00
Ivan
414f470c79
Make Poco HTTP Server zero-copy again (#19516) 
* Refactoring: part 1

* Refactoring: part 2

* Handle request using ReadBuffer interface

* Struggles with ReadBuffer's

* Fix URI parsing

* Implement parsing of multipart/form-data

* Check HTTP_LENGTH_REQUIRED before eof() or will hang

* Fix HTTPChunkedReadBuffer

* Fix build and style

* Fix test

* Resist double-eof

* Fix arcadian build
 2021-02-19 15:51:26 +03:00
Alexander Tokmakov
8097532cb7 Merge branch 'master' into database_replicated 2021-02-18 17:42:06 +03:00
Amos Bird
77fd060665
Normalize function names 2021-02-18 20:20:28 +08:00
Kruglov Pavel
142a541361
Merge branch 'master' into hedged-requests 2021-02-18 14:23:00 +03:00
Vitaly Baranov
3cbb325913
Merge pull request #20506 from vitlibar/refactor-protobuf-format-io 
Improved serialization in Protobuf format.
 2021-02-18 11:31:37 +03:00
Vitaly Baranov
18e036d19b Improved serialization for data types combined of Arrays and Tuples. 
Improved matching enum data types to protobuf enum type.
Fixed serialization of the Map data type.
Omitted values are now set by default.
 2021-02-17 20:50:09 +03:00
Kruglov Pavel
794c3449ca
Merge branch 'master' into hedged-requests 2021-02-17 20:36:36 +03:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
Alexander Tokmakov
1aac7b3471 Merge branch 'master' into database_replicated 2021-02-17 00:39:56 +03:00
tavplubix
68b427a99d
Merge pull request #20448 from ClickHouse/better_ddl_queue_cleanup 
Better distributed DDL queue cleanup
 2021-02-16 19:29:50 +03:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
Pavel Kruglov
937a3192eb Fix data race 2021-02-16 00:06:34 +03:00
Alexander Tokmakov
cf57c3b4a2 update comments 2021-02-15 23:01:21 +03:00
Maksim Kita
e3003add57 HashTable fix bug during resize with nonstandard grower 2021-02-15 22:23:10 +03:00
Pavel Kruglov
454b1e0d60 Merge branch 'hedged-requests' of github.com:Avogar/ClickHouse into hedged-requests 2021-02-15 17:44:29 +03:00
Pavel Kruglov
eb0847e2b6 Style 2021-02-15 17:44:05 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
Pavel Kruglov
df09a5cac9 Refactor 3.0 and fix tests 2021-02-15 16:21:36 +03:00
Alexey Milovidov
8b7063acf4 Merge branch 'master' into improve-performance-group-by-multiple-fixed-keys 2021-02-15 04:11:01 +03:00
Alexey Milovidov
5bdc570046 Improve performance of GROUP BY multiple fixed size keys 2021-02-14 01:56:04 +03:00
Alexander Tokmakov
96cc01926c Merge branch 'master' into better_ddl_queue_cleanup 2021-02-14 00:22:09 +03:00
Azat Khuzhin
06e8065ee6 Add missing sync of underlying files 2021-02-13 13:19:51 +03:00
alesapin
f80137626a
Merge pull request #19580 from ClickHouse/in_memory_raft 
In memory coordination inside ClickHouse
 2021-02-13 10:19:23 +03:00
Alexander Tokmakov
184ec67dac better ddl queue cleanup 2021-02-12 22:23:50 +03:00
alexey-milovidov
f669a9f97a
Merge pull request #20348 from kitaisreal/common-types-template-instantiations 
Common types template instantiations
 2021-02-12 15:53:00 +03:00
Maksim Kita
f9527738c9 Added comments 2021-02-12 12:12:04 +03:00
alexey-milovidov
4e32dbd024
Merge pull request #19317 from abyss7/dwarf-folly 
Print inline frames for fatal stacktraces
 2021-02-12 02:38:50 +03:00
Maksim Kita
b276eac197 Common types template instantiations 2021-02-12 00:54:50 +03:00
alesapin
5f32f65211 Merge branch 'master' into in_memory_raft 2021-02-11 23:09:29 +03:00
alexey-milovidov
dc3ffd3fe2
Merge pull request #19451 from azat/safe-writes 
Do not silently ignore write errors
 2021-02-11 21:19:11 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Anton Popov
ece7e87cfb
Merge pull request #20079 from ClickHouse/vdimir/fix-join-used-flag 
Split use flags and RowRef in HashJoin
 2021-02-10 20:34:22 +03:00
alesapin
b4d53886a3 Add sync request/response 2021-02-10 16:01:05 +03:00
alesapin
0d179e021b Add sync cmd 2021-02-10 13:15:42 +03:00
alesapin
9667bdcbd2 Merge branch 'master' into in_memory_raft 2021-02-09 16:06:31 +03:00
Alexander Tokmakov
5e0cbfe972 Merge branch 'master' into database_replicated 2021-02-08 23:01:12 +03:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
tavplubix
ac477d9850
Merge pull request #19771 from ClickHouse/thread_state_improvements 
Minor code improvements around ThreadStatus
 2021-02-08 22:34:55 +03:00
Maksim Kita
ebf5868652
Merge pull request #20164 from kitaisreal/cache-dictionaries-lru-cache 
Added LRUHashMap
 2021-02-08 21:00:27 +03:00
Ivan Lezhankin
a287443438 Merge remote-tracking branch 'upstream/master' into dwarf-folly 2021-02-08 18:36:41 +03:00
templarzq
7e3186c826
optimize aggfunc column data copy (#19407) 
* optimize agg column data copy

* add aggfunc col data copy test

* fix null warning

* fix style

* fix style

* fix style

* fix style

* fix copy same dat to same pos bug

* fix style

* fix style

* Update ColumnAggregateFunction.cpp

fix type check

* control record buf size

* fix performance

* fix performance

* fix destroy data speed

* decrease run time.

* fix warning

* decrease test run time

* Update aggfunc_col_data_copy.xml

* replace unordered_map with HashMap

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix hash map compile err

* fix compile err

* fix compile err

* fix format

* faster test

Co-authored-by: centos7 <templarzq@localhost.localdomain>
 2021-02-08 16:18:48 +03:00
vdimir
776b682f28
Add comment for used_flags in hash join, fix build 2021-02-08 14:38:31 +03:00
Kruglov Pavel
b256bd455c
Merge pull request #19961 from ucasFL/shard-id 
add setting insert_shard_id
 2021-02-08 14:28:13 +03:00
Maksim Kita
7ee1ad6833 Fixed unit tests 2021-02-08 11:44:32 +03:00
alesapin
d5ee7d33e9 Merge branch 'master' into in_memory_raft 2021-02-08 10:30:31 +03:00
Maksim Kita
285d46bc56 Fix gcc build issues 2021-02-07 23:54:14 +03:00
Maksim Kita
1201c4c55f Fixed build issues 2021-02-07 16:45:29 +03:00
Maksim Kita
eadd23aa38 Added LRUHashMap 2021-02-07 00:06:27 +03:00
Pavel Kruglov
9048dc43d4 Fix style and build 2021-02-06 22:13:50 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
Pavel Kruglov
794f185442 Fix 2021-02-06 17:23:48 +03:00
Kruglov Pavel
d94e8624d7
Merge branch 'master' into shard-id 2021-02-06 16:48:17 +03:00
Alexey Milovidov
4092916db6 Useless changes 2021-02-06 14:56:10 +03:00
Pavel Kruglov
0704d3cf27 Refactor 2021-02-06 03:54:27 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID 
CLICKHOUSE-606: query deduplication based on parts' UUID
 2021-02-05 22:47:34 +03:00
vdimir
01a6e01ad7
Store usage info flags separate from row refs in hash join 2021-02-05 16:20:13 +03:00
Azat Khuzhin
98e3a99a88 Do not catch exceptions during final flush in writers destructors 
Since this hides real problems, since destructor does final flush and if
it fails, then data will be lost.

One of such examples if MEMORY_LIMIT_EXCEEDED exception, so lock
exceptions from destructors, by using
MemoryTracker::LockExceptionInThread to block these exception, and allow
others (so std::terminate will be called, since this is c++11 with
noexcept for destructors by default).

Here is an example, that leads to empty block in the distributed batch:

    2021.01.21 12:43:18.619739 [ 46468 ] {7bd60d75-ebcb-45d2-874d-260df9a4ddac} <Error> virtual DB::CompressedWriteBuffer::~CompressedWriteBuffer(): Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 332.07 GiB (attempt to allocate chunk of 4355342 bytes), maximum: 256.00 GiB, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception<>() @ 0x86f7b88 in /usr/bin/clickhouse
    ...
    4. void DB::PODArrayBase<>::resize<>(unsigned long) @ 0xe9e878d in /usr/bin/clickhouse
    5. DB::CompressedWriteBuffer::nextImpl() @ 0xe9f0296 in /usr/bin/clickhouse
    6. DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0xe9f0415 in /usr/bin/clickhouse
    7. DB::DistributedBlockOutputStream::writeToShard() @ 0xf6bed4a in /usr/bin/clickhouse
 2021-02-05 01:31:45 +03:00
Azat Khuzhin
a46d65f99d Fix typo in comment for memoryTrackerCanThrow() 2021-02-05 01:29:46 +03:00
alesapin
da51ea1794 Simplify shutdown and requests processing 2021-02-04 15:07:41 +03:00
Vladimir Chebotarev
a597ed0ff0 Fixed open behavior of remote host filter in case when there is remote_url_allow_hosts section in configuration but no entries there. 2021-02-04 09:34:31 +03:00
alesapin
1ff87ac6f9 Add background session lifetime control 2021-02-03 23:32:15 +03:00
Alexander Tokmakov
87502d0220 Merge branch 'thread_state_improvements' into database_replicated 2021-02-03 20:19:35 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Alexander Kuzmenkov
bb45c5a8ff Merge remote-tracking branch 'origin/master' into HEAD 2021-02-03 12:13:23 +03:00
Alexander Kuzmenkov
7e945bab03 fix the calculation for moving frame start 2021-02-03 08:53:21 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Kruglov Pavel
0b4a9ed87a
Fix gcc-9 build 2021-02-02 17:38:42 +03:00
alesapin
9979181c02 Merge branch 'master' into in_memory_raft 2021-02-02 17:05:20 +03:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
Alexey Milovidov
656cd583f7 Add MSan annotation for system.stack_trace 2021-02-02 06:46:54 +03:00
Pavel Kruglov
7d9eb966f0 Fix 2021-02-01 20:09:55 +03:00
alesapin
0aca40d8cb Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
alexey-milovidov
342baa4223
Merge pull request #19720 from ClickHouse/unicode-asan 
Fix unsufficient args check (trash code) in StringSearcher
 2021-01-31 15:43:05 +03:00
alexey-milovidov
842c2c840c
Merge pull request #19773 from ClickHouse/tavplubix-patch-3 
Fix stack overflow in FieldVisitorAccurateEquals
 2021-01-31 11:09:41 +03:00
Alexey Milovidov
40a356db74 Fix warning 2021-01-31 07:40:51 +03:00
Alexey Milovidov
0fa5142715 Remove tons of garbage 2021-01-31 05:36:52 +03:00
Alexey Milovidov
5c718c8754 Merge branch 'master' into unicode-asan 2021-01-31 02:40:24 +03:00
Azat Khuzhin
89c4055202 Lock MEMORY_LIMIT_EXCEEDED error from tryLogCurrentException() 
This will avoid hiding some exceptions in logs, when the server is under
high memory pressure (i.e. when any new allocation will lead to
MEMORY_LIMIT_EXCEEDED error).

This became more relevent after all memory allocations was tracked with
MemoryTracker, by falling back to total_memory_tracking, in #16121
 2021-01-29 21:38:14 +03:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
Pavel Kruglov
01a0cb649a Fix build, style, tests 2021-01-29 18:46:28 +03:00
Alexander Tokmakov
004517009e fix 2021-01-29 18:13:27 +03:00
Anton Popov
031132038b fix filtering by uint8 greater than 127 2021-01-29 10:39:18 +03:00
Alexey Milovidov
1e2669fd3c Fix error 2021-01-29 07:54:46 +03:00
tavplubix
37455304e7
Update FieldVisitorsAccurateComparison.h 2021-01-28 18:51:01 +03:00
tavplubix
817df99d11
Update FieldVisitorsAccurateComparison.h 2021-01-28 18:29:21 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
Alexander Tokmakov
52e5c0aad7 fix thread status 2021-01-28 16:48:17 +03:00
alesapin
c3db529543 Merge branch 'master' into in_memory_raft 2021-01-28 12:01:54 +03:00
Alexey Milovidov
355c99568e Fix error 2021-01-28 10:16:36 +03:00
Alexey Milovidov
95e15131a8 Fix unsufficient args check (trash code) in StringSearcher 2021-01-27 20:32:59 +03:00
alesapin
7af28e758a Process read requests without raft 2021-01-27 16:37:58 +03:00
alesapin
241d3ec8c2 Merge with master 2021-01-27 15:40:16 +03:00
alesapin
dd54e5afe8 Merge branch 'master' into in_memory_raft 2021-01-27 15:32:16 +03:00
alexey-milovidov
c3828a1fb2
Merge pull request #19660 from CurtizJ/sync-directory 
Better abstractions in disk interface
 2021-01-27 13:47:53 +03:00
Maksim Kita
bd81f43ecb
Merge pull request #19672 from ClickHouse/bigint-remove-old-code 
Remove old code for BigInt.
 2021-01-27 11:28:45 +03:00
Alexey Milovidov
f3929b151a Remove even more code 2021-01-26 22:04:03 +03:00
Alexey Milovidov
5f13fdddc0 Remove useless code from BigInt 2021-01-26 21:22:40 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
alesapin
0ff05d2403 Fix deadlock in testkeeper 2021-01-26 17:30:29 +03:00
alesapin
dea4b5009b Some server initialization 2021-01-25 15:29:12 +03:00
Alexander Tokmakov
3bd4d97353 Merge branch 'master' into database_replicated 2021-01-25 14:19:04 +03:00
alesapin
173b6fefdf Merge branch 'master' into in_memory_raft 2021-01-25 12:03:28 +03:00
alexey-milovidov
ba3e064a63
Merge pull request #19371 from kssenii/test-coverage-with-factories 
test coverage with factories
 2021-01-25 07:11:09 +03:00
alexey-milovidov
636d4c41bb
Merge pull request #19520 from amosbird/fixbuildid 
Support new .debug location
 2021-01-24 20:06:13 +03:00
Amos Bird
366c75ed29
Ensure buildid length 2021-01-24 10:28:43 +08:00
Amos Bird
554663a533
Support new .debug location 2021-01-24 03:03:54 +08:00
alexey-milovidov
9818bd319a
Merge pull request #19381 from azat/parts-types-metrics 
Add metrics for MergeTree parts (Wide/Compact/InMemory) types
 2021-01-22 17:24:50 +03:00
alesapin
4a06bf1d79 Merge branch 'fix_one_more_tiny_race' into in_memory_raft 2021-01-22 15:56:43 +03:00
Alexey Milovidov
dcd03eb155 Remove useless file 2021-01-22 00:57:00 +03:00
alesapin
5037d3befd
Merge pull request #19355 from ClickHouse/fix_race_test_keeper 
Fix race condition in TestKeeperHandler on session finish
 2021-01-21 23:53:18 +03:00
Azat Khuzhin
cb951c2116 Add metrics for MergeTree parts types 
- PartsWide
- PartsCompact
- PartsInMemory
 2021-01-21 21:17:00 +03:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
alexey-milovidov
062f00aa5d
Merge pull request #19290 from azat/dist-broken-on-EOF-fix 
Do not mark file for distributed send as broken on EOF
 2021-01-21 17:00:36 +03:00
alesapin
7f32926a21 Fix race condition in TestKeeperHandler on session finish 2021-01-21 14:37:20 +03:00
alesapin
f7175819d5 Add storage simpliest serialization 2021-01-21 14:07:55 +03:00
Ivan
0cbbb84f24
Add missing header 2021-01-21 02:20:11 +03:00
Ivan Lezhankin
2129dc13f6 Fix style and build 2021-01-20 20:44:18 +03:00
alesapin
d5a3adffbd Replicate something in test keeper storage with raft 2021-01-20 19:25:30 +03:00
Ivan Lezhankin
2bb28fbc14 Print inline frames augmenting usual ones 2021-01-20 17:05:25 +03:00
alesapin
a241793a14 Merge branch 'decompose_test_keeper_storage' into in_memory_raft 2021-01-20 15:51:54 +03:00
Azat Khuzhin
e4350e078c Add ability to distinguish remote exceptions from local 2021-01-20 01:10:17 +03:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
alesapin
8cdfbd996b Fix header 2021-01-19 18:51:52 +03:00
alesapin
7e71a5b1b0 Fix new lines 2021-01-19 18:25:08 +03:00
alesapin
0ee5629527 Fix style 2021-01-19 18:10:49 +03:00
alesapin
ace6d906b0 Fix normal close scenario 2021-01-19 17:45:45 +03:00
alesapin
ac64a13392 Split storage and requests processing 2021-01-19 17:22:28 +03:00
Ivan Lezhankin
3fb50dfa1b Initial implementation of inline frames 2021-01-19 15:34:27 +03:00
alesapin
6883143994 Merge branch 'master' into in_memory_raft 2021-01-19 10:27:21 +03:00
alesapin
6896558424 Some code movements 2021-01-18 19:03:26 +03:00
Alexander Tokmakov
7f97a11c84 Merge branch 'master' into database_replicated 2021-01-18 17:09:39 +03:00
Alexey Milovidov
e482c5ea15 Fix TSan 2021-01-17 15:40:14 +03:00
Alexey Milovidov
950bfb3ec6 Allow change max_server_memory_usage without restart 2021-01-16 18:33:46 +03:00
alexey-milovidov
bf86d3bab8
Merge pull request #19153 from azat/async_socket_for_remote-fix-fd-leak 
Fix leaking of pipe fd for async_socket_for_remote
 2021-01-16 11:23:55 +03:00
alexey-milovidov
2e2988e5d8
Merge pull request #19146 from azat/server-memory-limit-blocking 
MemoryTracker: Do not ignore server memory limits during blocking by default
 2021-01-16 11:09:19 +03:00
Azat Khuzhin
cf085b0687 Split RemoteQueryExecutorReadContext into module part 2021-01-16 01:57:36 +03:00
alexey-milovidov
5f189c5756
Merge pull request #19122 from ClickHouse/data-part-better-code 
Add metrics for part number in MergeTree in ClickHouse
 2021-01-16 00:20:15 +03:00
Azat Khuzhin
61b2d0ce42 MemoryTracker: Do not ignore server memory limits during blocking by default 2021-01-15 22:46:58 +03:00
alexey-milovidov
f8830e0f11
Merge pull request #19107 from ClickHouse/bar-nan 
Fix fuzz test for function `bar`.
 2021-01-15 20:38:02 +03:00
Alexey Milovidov
e238fd64ac Add part metrics 2021-01-15 15:28:53 +03:00
Alexey Milovidov
33d045b72e Fix issue in function "bar" 2021-01-15 13:17:09 +03:00
Alexey Milovidov
8276a1c8d2 Faster parts removal, more safe and efficient interface of IDisk 2021-01-14 19:24:13 +03:00
Alexander Tokmakov
73e96250a8 add comments 2021-01-13 15:19:41 +03:00
Alexander Tokmakov
ff6c3c75c2 add protection from unsafe allocations 2021-01-12 18:41:24 +03:00
tavplubix
6022311a37
Merge pull request #18285 from weeds085490/hotfix/abort_when_failed_to_init_ddlworker 
exit clickhouse when failing to initialize zookeeper with logical err…
 2021-01-11 17:04:33 +03:00
Alexey Milovidov
6eb5a5f4d9 Remove useless code 2021-01-10 03:28:59 +03:00