thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,476 Commits 9,328 Branches 1,783 Tags 2.1 GiB
e7948819f9
Commit Graph

1367 Commits

Author SHA1 Message Date
alexey-milovidov
dc3ffd3fe2
Merge pull request #19451 from azat/safe-writes 
Do not silently ignore write errors
 2021-02-11 21:19:11 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Anton Popov
ece7e87cfb
Merge pull request #20079 from ClickHouse/vdimir/fix-join-used-flag 
Split use flags and RowRef in HashJoin
 2021-02-10 20:34:22 +03:00
alesapin
b4d53886a3 Add sync request/response 2021-02-10 16:01:05 +03:00
alesapin
0d179e021b Add sync cmd 2021-02-10 13:15:42 +03:00
alesapin
9667bdcbd2 Merge branch 'master' into in_memory_raft 2021-02-09 16:06:31 +03:00
Alexander Tokmakov
5e0cbfe972 Merge branch 'master' into database_replicated 2021-02-08 23:01:12 +03:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
tavplubix
ac477d9850
Merge pull request #19771 from ClickHouse/thread_state_improvements 
Minor code improvements around ThreadStatus
 2021-02-08 22:34:55 +03:00
Maksim Kita
ebf5868652
Merge pull request #20164 from kitaisreal/cache-dictionaries-lru-cache 
Added LRUHashMap
 2021-02-08 21:00:27 +03:00
Ivan Lezhankin
a287443438 Merge remote-tracking branch 'upstream/master' into dwarf-folly 2021-02-08 18:36:41 +03:00
templarzq
7e3186c826
optimize aggfunc column data copy (#19407) 
* optimize agg column data copy

* add aggfunc col data copy test

* fix null warning

* fix style

* fix style

* fix style

* fix style

* fix copy same dat to same pos bug

* fix style

* fix style

* Update ColumnAggregateFunction.cpp

fix type check

* control record buf size

* fix performance

* fix performance

* fix destroy data speed

* decrease run time.

* fix warning

* decrease test run time

* Update aggfunc_col_data_copy.xml

* replace unordered_map with HashMap

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix hash map compile err

* fix compile err

* fix compile err

* fix format

* faster test

Co-authored-by: centos7 <templarzq@localhost.localdomain>
 2021-02-08 16:18:48 +03:00
vdimir
776b682f28
Add comment for used_flags in hash join, fix build 2021-02-08 14:38:31 +03:00
Kruglov Pavel
b256bd455c
Merge pull request #19961 from ucasFL/shard-id 
add setting insert_shard_id
 2021-02-08 14:28:13 +03:00
Maksim Kita
7ee1ad6833 Fixed unit tests 2021-02-08 11:44:32 +03:00
alesapin
d5ee7d33e9 Merge branch 'master' into in_memory_raft 2021-02-08 10:30:31 +03:00
Maksim Kita
285d46bc56 Fix gcc build issues 2021-02-07 23:54:14 +03:00
Maksim Kita
1201c4c55f Fixed build issues 2021-02-07 16:45:29 +03:00
Maksim Kita
eadd23aa38 Added LRUHashMap 2021-02-07 00:06:27 +03:00
Pavel Kruglov
9048dc43d4 Fix style and build 2021-02-06 22:13:50 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
Pavel Kruglov
794f185442 Fix 2021-02-06 17:23:48 +03:00
Kruglov Pavel
d94e8624d7
Merge branch 'master' into shard-id 2021-02-06 16:48:17 +03:00
Alexey Milovidov
4092916db6 Useless changes 2021-02-06 14:56:10 +03:00
Pavel Kruglov
0704d3cf27 Refactor 2021-02-06 03:54:27 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID 
CLICKHOUSE-606: query deduplication based on parts' UUID
 2021-02-05 22:47:34 +03:00
vdimir
01a6e01ad7
Store usage info flags separate from row refs in hash join 2021-02-05 16:20:13 +03:00
Azat Khuzhin
98e3a99a88 Do not catch exceptions during final flush in writers destructors 
Since this hides real problems, since destructor does final flush and if
it fails, then data will be lost.

One of such examples if MEMORY_LIMIT_EXCEEDED exception, so lock
exceptions from destructors, by using
MemoryTracker::LockExceptionInThread to block these exception, and allow
others (so std::terminate will be called, since this is c++11 with
noexcept for destructors by default).

Here is an example, that leads to empty block in the distributed batch:

    2021.01.21 12:43:18.619739 [ 46468 ] {7bd60d75-ebcb-45d2-874d-260df9a4ddac} <Error> virtual DB::CompressedWriteBuffer::~CompressedWriteBuffer(): Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 332.07 GiB (attempt to allocate chunk of 4355342 bytes), maximum: 256.00 GiB, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception<>() @ 0x86f7b88 in /usr/bin/clickhouse
    ...
    4. void DB::PODArrayBase<>::resize<>(unsigned long) @ 0xe9e878d in /usr/bin/clickhouse
    5. DB::CompressedWriteBuffer::nextImpl() @ 0xe9f0296 in /usr/bin/clickhouse
    6. DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0xe9f0415 in /usr/bin/clickhouse
    7. DB::DistributedBlockOutputStream::writeToShard() @ 0xf6bed4a in /usr/bin/clickhouse
 2021-02-05 01:31:45 +03:00
Azat Khuzhin
a46d65f99d Fix typo in comment for memoryTrackerCanThrow() 2021-02-05 01:29:46 +03:00
alesapin
da51ea1794 Simplify shutdown and requests processing 2021-02-04 15:07:41 +03:00
alesapin
1ff87ac6f9 Add background session lifetime control 2021-02-03 23:32:15 +03:00
Alexander Tokmakov
87502d0220 Merge branch 'thread_state_improvements' into database_replicated 2021-02-03 20:19:35 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Alexander Kuzmenkov
bb45c5a8ff Merge remote-tracking branch 'origin/master' into HEAD 2021-02-03 12:13:23 +03:00
Alexander Kuzmenkov
7e945bab03 fix the calculation for moving frame start 2021-02-03 08:53:21 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Kruglov Pavel
0b4a9ed87a
Fix gcc-9 build 2021-02-02 17:38:42 +03:00
alesapin
9979181c02 Merge branch 'master' into in_memory_raft 2021-02-02 17:05:20 +03:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
Alexey Milovidov
656cd583f7 Add MSan annotation for system.stack_trace 2021-02-02 06:46:54 +03:00
Pavel Kruglov
7d9eb966f0 Fix 2021-02-01 20:09:55 +03:00
alesapin
0aca40d8cb Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
alexey-milovidov
342baa4223
Merge pull request #19720 from ClickHouse/unicode-asan 
Fix unsufficient args check (trash code) in StringSearcher
 2021-01-31 15:43:05 +03:00
alexey-milovidov
842c2c840c
Merge pull request #19773 from ClickHouse/tavplubix-patch-3 
Fix stack overflow in FieldVisitorAccurateEquals
 2021-01-31 11:09:41 +03:00
Alexey Milovidov
40a356db74 Fix warning 2021-01-31 07:40:51 +03:00
Alexey Milovidov
0fa5142715 Remove tons of garbage 2021-01-31 05:36:52 +03:00
Alexey Milovidov
5c718c8754 Merge branch 'master' into unicode-asan 2021-01-31 02:40:24 +03:00
Azat Khuzhin
89c4055202 Lock MEMORY_LIMIT_EXCEEDED error from tryLogCurrentException() 
This will avoid hiding some exceptions in logs, when the server is under
high memory pressure (i.e. when any new allocation will lead to
MEMORY_LIMIT_EXCEEDED error).

This became more relevent after all memory allocations was tracked with
MemoryTracker, by falling back to total_memory_tracking, in #16121
 2021-01-29 21:38:14 +03:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
Pavel Kruglov
01a0cb649a Fix build, style, tests 2021-01-29 18:46:28 +03:00
Alexander Tokmakov
004517009e fix 2021-01-29 18:13:27 +03:00
Anton Popov
031132038b fix filtering by uint8 greater than 127 2021-01-29 10:39:18 +03:00
Alexey Milovidov
1e2669fd3c Fix error 2021-01-29 07:54:46 +03:00
tavplubix
37455304e7
Update FieldVisitorsAccurateComparison.h 2021-01-28 18:51:01 +03:00
tavplubix
817df99d11
Update FieldVisitorsAccurateComparison.h 2021-01-28 18:29:21 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
Alexander Tokmakov
52e5c0aad7 fix thread status 2021-01-28 16:48:17 +03:00
alesapin
c3db529543 Merge branch 'master' into in_memory_raft 2021-01-28 12:01:54 +03:00
Alexey Milovidov
355c99568e Fix error 2021-01-28 10:16:36 +03:00
Alexey Milovidov
95e15131a8 Fix unsufficient args check (trash code) in StringSearcher 2021-01-27 20:32:59 +03:00
alesapin
7af28e758a Process read requests without raft 2021-01-27 16:37:58 +03:00
alesapin
241d3ec8c2 Merge with master 2021-01-27 15:40:16 +03:00
alesapin
dd54e5afe8 Merge branch 'master' into in_memory_raft 2021-01-27 15:32:16 +03:00
alexey-milovidov
c3828a1fb2
Merge pull request #19660 from CurtizJ/sync-directory 
Better abstractions in disk interface
 2021-01-27 13:47:53 +03:00
Maksim Kita
bd81f43ecb
Merge pull request #19672 from ClickHouse/bigint-remove-old-code 
Remove old code for BigInt.
 2021-01-27 11:28:45 +03:00
Alexey Milovidov
f3929b151a Remove even more code 2021-01-26 22:04:03 +03:00
Alexey Milovidov
5f13fdddc0 Remove useless code from BigInt 2021-01-26 21:22:40 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
alesapin
0ff05d2403 Fix deadlock in testkeeper 2021-01-26 17:30:29 +03:00
alesapin
dea4b5009b Some server initialization 2021-01-25 15:29:12 +03:00
Alexander Tokmakov
3bd4d97353 Merge branch 'master' into database_replicated 2021-01-25 14:19:04 +03:00
alesapin
173b6fefdf Merge branch 'master' into in_memory_raft 2021-01-25 12:03:28 +03:00
alexey-milovidov
ba3e064a63
Merge pull request #19371 from kssenii/test-coverage-with-factories 
test coverage with factories
 2021-01-25 07:11:09 +03:00
alexey-milovidov
636d4c41bb
Merge pull request #19520 from amosbird/fixbuildid 
Support new .debug location
 2021-01-24 20:06:13 +03:00
Amos Bird
366c75ed29
Ensure buildid length 2021-01-24 10:28:43 +08:00
Amos Bird
554663a533
Support new .debug location 2021-01-24 03:03:54 +08:00
alexey-milovidov
9818bd319a
Merge pull request #19381 from azat/parts-types-metrics 
Add metrics for MergeTree parts (Wide/Compact/InMemory) types
 2021-01-22 17:24:50 +03:00
alesapin
4a06bf1d79 Merge branch 'fix_one_more_tiny_race' into in_memory_raft 2021-01-22 15:56:43 +03:00
Alexey Milovidov
dcd03eb155 Remove useless file 2021-01-22 00:57:00 +03:00
alesapin
5037d3befd
Merge pull request #19355 from ClickHouse/fix_race_test_keeper 
Fix race condition in TestKeeperHandler on session finish
 2021-01-21 23:53:18 +03:00
Azat Khuzhin
cb951c2116 Add metrics for MergeTree parts types 
- PartsWide
- PartsCompact
- PartsInMemory
 2021-01-21 21:17:00 +03:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
alexey-milovidov
062f00aa5d
Merge pull request #19290 from azat/dist-broken-on-EOF-fix 
Do not mark file for distributed send as broken on EOF
 2021-01-21 17:00:36 +03:00
alesapin
7f32926a21 Fix race condition in TestKeeperHandler on session finish 2021-01-21 14:37:20 +03:00
alesapin
f7175819d5 Add storage simpliest serialization 2021-01-21 14:07:55 +03:00
Ivan
0cbbb84f24
Add missing header 2021-01-21 02:20:11 +03:00
Ivan Lezhankin
2129dc13f6 Fix style and build 2021-01-20 20:44:18 +03:00
alesapin
d5a3adffbd Replicate something in test keeper storage with raft 2021-01-20 19:25:30 +03:00
Ivan Lezhankin
2bb28fbc14 Print inline frames augmenting usual ones 2021-01-20 17:05:25 +03:00
alesapin
a241793a14 Merge branch 'decompose_test_keeper_storage' into in_memory_raft 2021-01-20 15:51:54 +03:00
Azat Khuzhin
e4350e078c Add ability to distinguish remote exceptions from local 2021-01-20 01:10:17 +03:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
alesapin
8cdfbd996b Fix header 2021-01-19 18:51:52 +03:00
alesapin
7e71a5b1b0 Fix new lines 2021-01-19 18:25:08 +03:00
alesapin
0ee5629527 Fix style 2021-01-19 18:10:49 +03:00
alesapin
ace6d906b0 Fix normal close scenario 2021-01-19 17:45:45 +03:00
alesapin
ac64a13392 Split storage and requests processing 2021-01-19 17:22:28 +03:00
Ivan Lezhankin
3fb50dfa1b Initial implementation of inline frames 2021-01-19 15:34:27 +03:00
alesapin
6883143994 Merge branch 'master' into in_memory_raft 2021-01-19 10:27:21 +03:00
alesapin
6896558424 Some code movements 2021-01-18 19:03:26 +03:00