thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-05 08:00:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
42,427 Commits 8,978 Branches 1,768 Tags 1.9 GiB
c8a58299ac
Commit Graph

755 Commits

Author SHA1 Message Date
alesapin
a429760641 Fix debug build 2020-05-29 17:10:09 +03:00
alesapin
748e8fbfc3 Merge with master 2020-05-29 14:03:11 +03:00
alesapin
2ff7bbb511 Better names 2020-05-29 13:56:04 +03:00
alesapin
0170250c5e Merge branch 'master' into consistent_metadata2 2020-05-29 13:48:09 +03:00
Mikhail Filimonov
b885337454 Virtual columns for accessing kafka message headers 2020-05-29 10:44:10 +02:00
Alexander Tokmakov
c071e0a473 add context to DatabaseCatalog methods 2020-05-29 02:01:18 +03:00
Alexander Tokmakov
3cff070f89 add context to IDatabase methods 2020-05-28 23:10:45 +03:00
alesapin
026244b07f Better 2020-05-28 21:57:09 +03:00
alesapin
93801a1a9d Fix unit tests 2020-05-28 21:52:08 +03:00
Mikhail Filimonov
7211af9404 Fixing style 2020-05-28 20:19:10 +02:00
Mikhail Filimonov
b97ff8136b Make function static 2020-05-28 20:10:54 +02:00
Mikhail Filimonov
16de65a6f0 Add prefix & facility to librdkafka logs 2020-05-28 19:52:59 +02:00
Mikhail Filimonov
e39c4cdf71 Add _timestamp_ms virtual columns to Kafka engine 2020-05-28 19:41:57 +02:00
alesapin
eb5a8dccfa Fix ya.make 2020-05-28 20:32:22 +03:00
Vitaly Baranov
d7cc703233
Merge pull request #11080 from vitlibar/add-authentication-type-to-system-users 
Show authentication type in system.users table
 2020-05-28 20:20:41 +03:00
alesapin
66ccb30299 Fix style check 2020-05-28 20:17:05 +03:00
alesapin
860dc2c814 Merge with consistent_metadata2 2020-05-28 19:45:51 +03:00
alesapin
c5fe87e123 Less includes 2020-05-28 19:32:35 +03:00
alesapin
acc3e6b15a Better structure 2020-05-28 19:31:27 +03:00
alesapin
c6f82590ad Add missed files 2020-05-28 18:34:33 +03:00
alesapin
61aff798d3 Compileable code 2020-05-28 18:33:44 +03:00
Nikolai Kochetov
ea3f638b30 Try fix tests. 2020-05-28 17:11:36 +03:00
alesapin
099bfc4657 Fix bug with empty column 2020-05-28 17:03:21 +03:00
Mikhail Filimonov
4de1d6d421 Fixing style 2020-05-28 15:53:27 +02:00
alesapin
5f8b69547b More readable code 2020-05-28 16:45:08 +03:00
Mikhail Filimonov
6b694d54b4 Better kafka client.id 2020-05-28 15:39:55 +02:00
alesapin
f5e5290c0a Better naming 2020-05-28 16:09:03 +03:00
alesapin
5b5f749592 Split files 2020-05-28 16:03:18 +03:00
alesapin
26a9829e04 Indices description as vector 2020-05-28 15:47:17 +03:00
alesapin
52ca6b2051 I'm able to build it 2020-05-28 15:37:05 +03:00
Nikolai Kochetov
235fdc831b Merged with master. 2020-05-28 15:23:23 +03:00
tavplubix
801c1c1240
Merge pull request #11205 from ClickHouse/fix-view-totals 
Fix totals and extremes header for view.
 2020-05-28 15:05:34 +03:00
Nikolai Kochetov
da0052858d Fix build. 2020-05-28 13:57:04 +03:00
Nikolai Kochetov
1fa795988f Remove some code. 2020-05-28 11:24:59 +03:00
madianjun
e3c7ab2ca7 Add Kafka client.id 2020-05-28 08:32:51 +02:00
Vitaly Baranov
98172deffc Show authentication type in table system.users and while executing SHOW CREATE USER query. 2020-05-28 08:34:10 +03:00
alexey-milovidov
69d5fd0edb
Merge pull request #11220 from ClickHouse/keep-metric-distributed-queue-on-exception 
Keep the value of DistributedFilesToInsert metric on exceptions
 2020-05-28 01:23:11 +03:00
Pavel Kovalenko
677cb56a06 S3 Poco HTTP Client WIP. 2020-05-27 22:41:49 +03:00
alesapin
380c60a33f Intermediate stage (doesn't compile) 2020-05-27 21:38:34 +03:00
alexey-milovidov
dffeec8637
Merge pull request #10537 from expl0si0nn/ttl_expr_data_rollup 
Support of WHERE and GROUP BY in TTL expressions
 2020-05-27 21:35:55 +03:00
alesapin
18656841a3 Implement methods for index struct 2020-05-27 16:34:23 +03:00
alesapin
c4b2f72fe6 Add common struct for skip indices 2020-05-27 14:25:02 +03:00
Nikolai Sorokin
57555dbabf Fix after rebase 2020-05-27 14:00:17 +03:00
alexey-milovidov
992f0dfec5
Merge pull request #11149 from filimonov/kafka_reschedule 
Fixed reschedule issue in Kafka
 2020-05-27 13:26:55 +03:00
Alexey Milovidov
146370934a Keep the value of DistributedFilesToInsert metric on exceptions 2020-05-27 13:07:38 +03:00
Anton Popov
f62adfd588 Fix Arcadia build 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
141ed88751 Allow functions in group by keys; Add default aggregate function; Add more tests 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
66496dc7e3 Fix trailing spaces 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
eeaf608aa9 Refactor code; support prefix of PK as GROUP BY key 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
61974e0047 TTL Expressions WHERE and GROUP BY draft 2020-05-27 12:53:15 +03:00
alesapin
c7cda399c7 Merge branch 'master' into consistent_metadata2 2020-05-27 12:51:56 +03:00
alesapin
3399e573d1
Merge pull request #11199 from ClickHouse/fix_primary_key_method_call 
Fix primary key check
 2020-05-27 11:55:12 +03:00
Mikhail Filimonov
efc0b97707 Fix Kafka performance issue related to reschedules based on limits, which were always applied. 2020-05-27 07:39:06 +02:00
Dmitry
41d1cd1c9b fix bad merge 2020-05-27 01:17:32 +03:00
Dmitry
38c585f867 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-26 21:27:50 +03:00
alesapin
9d0c28d96c Add missing methods 2020-05-26 17:14:08 +03:00
alesapin
8614487a5a Fix primary key check 2020-05-26 16:46:19 +03:00
Nikolai Kochetov
12384551da Fix totals and extremes header for views. 2020-05-26 16:19:24 +03:00
alesapin
6626ef08e8 Fix PVS studio errors 2020-05-26 15:14:50 +03:00
Vitaly Baranov
e27ded8798
Merge pull request #11172 from vitlibar/grpc-improve-build-scripts 
Improve build scripts for protobuf & gRPC
 2020-05-26 13:27:54 +03:00
alesapin
5780240e88 Fix primry key check 2020-05-26 12:46:04 +03:00
alesapin
7bbb064b34 Fix metadata update 2020-05-26 11:58:32 +03:00
Alexey Milovidov
c858f4d89c Clarifications 2020-05-26 02:47:21 +03:00
alexey-milovidov
d4acd1af3a
Merge pull request #11179 from excitoon-favorites/s3globfix 
Fixed S3 globbing which could fail in case of more than 1000 keys and some backends
 2020-05-26 02:45:42 +03:00
Alexey Milovidov
4bb3381bc9 Fix build after merge 2020-05-26 00:53:29 +03:00
filimonov
73a0ef6c0e
Fix for the hang during deletion of engine=Kafka (one more time) (#11145) 
* Added flag for safer rdkafka destruction, but more testing detected another hang case, which has no straigt solutions and can be workarounded currenly only by draining the consumer queue, so destructor is back

* After review fixes

* After review fixes2

* After review fixes3
 2020-05-26 00:13:10 +03:00
alesapin
355fbadb31 Fix style 2020-05-25 22:18:01 +03:00
alesapin
211d7e229f Better using 2020-05-25 21:03:04 +03:00
alesapin
adc7439a4c Comments and refactoring 2020-05-25 20:57:08 +03:00
alesapin
c3a6571036 Compilable code 2020-05-25 20:22:20 +03:00
alesapin
9b3cc9e525 Almost working ttl in IStorage 2020-05-25 20:07:14 +03:00
Vitaly Baranov
58a2172154 Improve build scripts related to protobuf and gRPC. 2020-05-25 18:46:57 +03:00
Vladimir Chebotarev
7f9a257c31 Fixed S3 globbing which could fail in case of more than 1000 keys and some backends. 2020-05-25 15:57:32 +03:00
alesapin
6281dd6893
Merge pull request #11115 from ClickHouse/consistent_metadata 
Refactoring in storage metadata (more consistent keys)
 2020-05-25 13:09:56 +03:00
Alexey Milovidov
4ebf63fb23 Fixed error 2020-05-24 22:17:15 +03:00
Azat Khuzhin
c36708caf9 Split data skipping indexes from the order by to evaluate order by before merge 2020-05-24 16:12:08 +03:00
Azat Khuzhin
71731b2e99 Fix data skipping indexes for columns with additional actions during merge 
For example for AggregatingMergeTree if the column in the data skipping
index is under some aggregate function, then data skipping index will be
incorrect.

Fix this by calculating data skipping indexes after aggregation has been
done.
 2020-05-24 16:12:08 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
af2a59e49b Progress on task 2020-05-24 00:50:34 +03:00
Alexey Milovidov
c87fb91532 Progress on task 2020-05-24 00:41:35 +03:00
Alexey Milovidov
ce0619dabf Progress on task 2020-05-24 00:26:45 +03:00
Alexey Milovidov
2d7d5a1547 Apply all transformations again 2020-05-24 00:16:27 +03:00
Alexey Milovidov
bab24879e9 Progress on task 2020-05-24 00:16:05 +03:00
Alexey Milovidov
e1695feb7f Apply all transformations again 2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba Progress on task 2020-05-23 23:37:37 +03:00
Alexey Milovidov
f75587e562 Progress on task 2020-05-23 23:13:34 +03:00
Alexey Milovidov
18febd7b97 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | while read file; do perl -pne 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/${1}_FORMATTED(${2}, "${3}{}${5}{}${7}{}${9}{}${11}", ${4}, ${6}, ${8}, ${10});/' $file > ${file}.tmp; mv ${file}.tmp $file; done 2020-05-23 22:56:05 +03:00
Alexey Milovidov
cd6d3c7509 Progress on task 2020-05-23 22:55:06 +03:00
Alexey Milovidov
d5b7883451 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/' 2020-05-23 22:41:08 +03:00
Alexey Milovidov
7e2fb9ad65 Apply all transformations again 2020-05-23 22:38:30 +03:00
Alexey Milovidov
eacff92d0e Progress on task 2020-05-23 22:35:08 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
7fed65cbe2 Remove duplicate whitespaces (preparation) 2020-05-23 22:14:58 +03:00
Alexey Milovidov
86b4d5a86f Remove duplicate whitespaces (preparation) 2020-05-23 22:09:06 +03:00
Alexey Milovidov
ab0562a574 Make all LOG in single line (preparation) 2020-05-23 22:05:41 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
3384f3b77e Progress on task 2020-05-23 21:07:38 +03:00
Alexey Milovidov
fc412f2d9d Progress on task 2020-05-23 20:59:03 +03:00
Alexey Milovidov
9d24908e53 Progress on task 2020-05-23 20:52:11 +03:00
Alexey Milovidov
3a09f9ca1c find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\W_]+\([^,]+, [^<+)"]+\);' | xargs sed -i -r -e 's/(LOG_[^\W_]+)\(([^,]+), ([^<+)"]+)\);/\1_FORMATTED(\2, \3);/' 2020-05-23 20:29:56 +03:00
Alexey Milovidov
b3f4fe12dd Progress on task 2020-05-23 20:25:19 +03:00
Alexey Milovidov
d6bc4d2bd0 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "{}\4{}", \3, \5);/' 2020-05-23 20:14:57 +03:00
Alexey Milovidov
241f8c5431 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+ \+ "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+) \+ "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 20:10:21 +03:00
Alexey Milovidov
f69cbdcbfc find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 20:09:37 +03:00
Alexey Milovidov
f68d1ceb4f find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}\7{}", \4, \6, \8);/' 2020-05-23 20:02:09 +03:00
Alexey Milovidov
533f86278a find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7", \4, \6);/' 2020-05-23 20:00:41 +03:00
Alexey Milovidov
8042e5febe find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/' 2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/' 2020-05-23 19:42:39 +03:00
Alexey Milovidov
35d79e1252 fmt-style logging, part 1 2020-05-23 19:33:07 +03:00
alexey-milovidov
ac6deaf8ec
Merge pull request #11036 from excitoon-favorites/s3fixes 
Fixed parsing of S3 URL
 2020-05-23 17:06:59 +03:00
Azat Khuzhin
bc4b75dead Add table name into logs for StorageDistributed 2020-05-23 11:57:14 +03:00
alesapin
cd83ebc599
Merge pull request #11106 from ClickHouse/fix_deadlock_system_logs_startup 
Fix deadlock in system tables during server startup
 2020-05-22 20:53:07 +03:00
alesapin
e812fa15d1 Merge branch 'consistent_metadata' into consistent_metadata2 2020-05-22 17:41:53 +03:00
alesapin
e971ff627e Fix gcc-9 build 2020-05-22 17:39:49 +03:00
alesapin
f026fa7de1 Rename PartDestinationType to DataDestionationType, because there is no part-specific information 2020-05-22 16:29:33 +03:00
alesapin
a935db4169
boop CI 2020-05-22 15:34:26 +03:00
alexey-milovidov
b82c633716
Merge pull request #11048 from filimonov/kafka_missed_data_during_drop 
Fixes the potential missed data during termination of Kafka engine table
 2020-05-22 14:11:10 +03:00
alesapin
f4b0d475af Fix system.tables output 2020-05-22 14:04:15 +03:00
alesapin
98ffefd90c Fix bug with return type 2020-05-22 13:09:24 +03:00
Vladimir Chebotarev
98f31374d5 Added improved S3 URL parsing to DiskS3. 2020-05-22 11:18:34 +03:00
alesapin
cbcdb6219e Slighty more verbose comment 2020-05-22 11:17:57 +03:00
Vladimir Chebotarev
02cb08ea2f Fixed parsing of S3 URL. 2020-05-22 11:17:10 +03:00
alesapin
b9dfa61774
Merge pull request #11042 from CurtizJ/modify-ttl 
Apply TTL for old data, after ALTER MODIFY TTL query
 2020-05-22 11:12:14 +03:00
Dmitry
47778c0259 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-21 23:45:12 +03:00
alesapin
59b3bc0c05
Merge branch 'master' into fix_deadlock_system_logs_startup 2020-05-21 22:57:52 +03:00
alesapin
09034157cd Remove comments 2020-05-21 22:53:46 +03:00
alesapin
327d17ac6a Better 2020-05-21 22:46:03 +03:00
alesapin
41914588fd More clear 2020-05-21 22:07:18 +03:00
alesapin
1906762119
Merge pull request #11109 from ClickHouse/less_verbose_logging 
Less verbose logging in mutation finalization task
 2020-05-21 21:53:08 +03:00
Alexander Kuzmenkov
f98ffdbc4c
Merge pull request #11087 from azat/context-fwd-decl 
[RFC] Forward declaration for Context as much as possible.
 2020-05-21 19:43:29 +03:00
alesapin
87eab514fb Merge branch 'master' into consistent_metadata 2020-05-21 18:54:52 +03:00
alesapin
5676340f6b Less verbose logging in mutation finalization task 2020-05-21 18:52:11 +03:00
alesapin
c036af0261 Fix deadlock after clickhouse-server update (with changes in one of system log tables structure) during startup between concurrent merge and table rename. 2020-05-21 17:11:56 +03:00
alexey-milovidov
765510d5c3
Merge pull request #10991 from ClickHouse/enable-clang-tidy-for-programs-and-utils 
Enable clang-tidy for programs and utils
 2020-05-21 15:12:07 +03:00
Alexey Milovidov
2fa264d83e Fix style #10591 2020-05-21 12:46:00 +03:00
alexey-milovidov
b6b62dbdbc
Merge pull request #10591 from excitoon-favorites/ttl_introspection 
Added `move_ttl_info` to `system.parts`
 2020-05-21 12:43:49 +03:00
BohuTANG
c9d315654c Fix Storages/MergeTree typo 2020-05-21 17:00:44 +08:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible. 
Now after changing Context.h 488 modules will be recompiled instead of 582.
 2020-05-21 01:53:18 +03:00
alesapin
616902a995 Sorting and primary key (broken) 2020-05-20 21:11:38 +03:00
alesapin
9fb28f5ac0 Add sampling key 2020-05-20 18:16:39 +03:00
Anton Popov
d62fe980ef Merge remote-tracking branch 'upstream/master' into modify-ttl 2020-05-20 16:52:30 +03:00
alesapin
8bc527eecd Compile partition key 2020-05-20 15:16:55 +03:00
alexey-milovidov
da33858d39 Update StorageSystemParts.cpp 2020-05-20 14:07:31 +03:00
Vladimir Chebotarev
02b52df86d Added delete_ttl_info_min and delete_ttl_info_max to system.parts. 2020-05-20 14:07:31 +03:00
Vladimir Chebotarev
6f68979662 Switched system.parts.move_ttl_info to separate arrays. 2020-05-20 14:07:31 +03:00
Vladimir Chebotarev
3ed5d9434a Added move_ttl_info to system.parts. 2020-05-20 14:07:31 +03:00
Mikhail Filimonov
e8a13842af Better fix, previous fix was wrong (was leaving the cycle by eof condition) 2020-05-20 11:02:02 +02:00
alesapin
dc330ca7cb
Merge pull request #11037 from ClickHouse/alter_rename_and_modify 
Fix alter rename with different types of alter
 2020-05-20 11:28:13 +03:00
Mikhail Filimonov
34198336ed Style 2020-05-19 19:02:37 +02:00
Mikhail Filimonov
b50480cbde Fixes the potential missed data during termination of Kafka engine table 2020-05-19 18:55:37 +02:00
alesapin
7d382cc7b5 Return missed command 2020-05-19 19:03:10 +03:00
alesapin
4906dbef41 Remove redundant logs 2020-05-19 13:46:56 +03:00
alesapin
2b37a418f2 More clear logic 2020-05-19 13:44:53 +03:00
alesapin
111c576579 Slightly less garbage 2020-05-19 13:03:05 +03:00
alesapin
70e5553204 At least something works 2020-05-19 12:54:56 +03:00
Anton Popov
58ed04dc24 materialize TTL after its modifying 2020-05-19 04:53:01 +03:00
alesapin
07cb21ccb7
Merge branch 'master' into refactor-reservations 2020-05-18 11:43:48 +03:00
Alexey Milovidov
f54435e7fd Fix clang-tidy 2020-05-18 11:08:55 +03:00
alexey-milovidov
7efd2a825f
Merge pull request #10463 from ClickHouse/parallel-final 
Parallel final
 2020-05-18 05:20:18 +03:00
alexey-milovidov
073dc2e1e3
Merge pull request #10986 from ClickHouse/try-fix-use-after-free-mergetree 
Try to fix use-after-free error in MergeTree
 2020-05-18 03:41:53 +03:00
alexey-milovidov
3820075813
Merge pull request #10983 from azat/configure-librdkafka 
Configure librdkafka (logging and thread names)
 2020-05-17 23:25:18 +03:00
Alexey Milovidov
70c31cbdc4 Fix error 2020-05-17 22:44:31 +03:00
Alexey Milovidov
81a7930c3e Merge branch 'master' of github.com:yandex/ClickHouse into try-fix-use-after-free-mergetree 2020-05-17 22:42:34 +03:00
Alexey Milovidov
20dbd6c3ef Fix error 2020-05-17 22:41:49 +03:00
alexey-milovidov
ca1a9b890d
Revert "Try to fix use-after-free error in MergeTree" 2020-05-17 22:36:27 +03:00
alexey-milovidov
8d3c37992a
Merge pull request #10533 from ClickHouse/alter-key-column-better-exception-message 
Better exception message and error code for ALTER of key column #9078
 2020-05-17 22:21:14 +03:00
Azat Khuzhin
e3d69eea88 Configure thread names for librdkafka 
First of all it is nice to have part of the table name in it.

And secondly, librdkafka uses pthread_setname_np(), but due to
glibc-compatibility it is a noop in clickhouse sources:

  # librdkafka uses rdk: prefix for thread names by default
  $ grep rdk /proc/$(pgrep clickhouse-serv)/task/*/comm
  # just in case:
  $ grep rdk /proc/*/task/*/comm
  $ grep rdk /proc/*/comm

  (gdb) disas pthread_setname_np
  Dump of assembler code for function pthread_setname_np:
     0x000000000c603250 <+0>:     xor    %eax,%eax
     0x000000000c603252 <+2>:     retq
  End of assembler dump.
 2020-05-17 20:40:18 +03:00
Azat Khuzhin
dbe781451a Redirect librdkafka logs from stderr to clickhouse logger 2020-05-17 20:27:36 +03:00
alexey-milovidov
7cf3538840
Merge pull request #10270 from ClickHouse/quota-key-in-client 
Support quota_key for Native client
 2020-05-17 14:09:40 +03:00
alexey-milovidov
7ee35f102d
Merge pull request #10867 from azat/dist-INSERT-load-balancing 
Respect prefer_localhost_replica/load_balancing on INSERT into Distributed
 2020-05-17 11:11:35 +03:00
Alexey Milovidov
7971f62776 Fix test 2020-05-17 10:47:53 +03:00
alexey-milovidov
421eb6df8f
Merge pull request #10970 from ClickHouse/try-fix-use-after-free-mergetree 
Try to fix use-after-free error in MergeTree
 2020-05-17 09:09:45 +03:00
alexey-milovidov
a3346e9331
Merge pull request #10912 from ClickHouse/static-mutate 
Static mutate
 2020-05-17 09:05:35 +03:00
Alexey Milovidov
397859ccb8 Fix error 2020-05-17 08:45:20 +03:00
Alexey Milovidov
9fcfe5fb14 Better exception message and error code for ALTER of key column #9078 2020-05-17 07:46:25 +03:00
alexey-milovidov
0fd0711b51
Merge pull request #10910 from filimonov/kafka_drop_hang_fix 
Fix for the hang during deletion of engine=Kafka
 2020-05-17 07:13:54 +03:00
Alexey Milovidov
88ee76186b Try to fix use-after-free error in MergeTree 2020-05-17 07:12:33 +03:00
alexey-milovidov
0d76091fd9
Merge pull request #10940 from azat/dist-send-partially-written-read-fix 
Avoid sending partially written files by the DistributedBlockOutputStream
 2020-05-17 06:53:51 +03:00
Gleb Novikov
1a25ac6e1f Merge branch 'master' into refactor-reservations 2020-05-16 23:34:45 +03:00
Gleb Novikov
45b84d491e small fixes of review remarks 2020-05-16 23:31:17 +03:00
alesapin
d0b56a4c7d
Merge pull request #10887 from ClickHouse/vzakaznikov-add_alter_rename_column_to_distributed 
Vzakaznikov add alter rename column to distributed
 2020-05-16 12:54:32 +03:00
alesapin
be4037abb5
Merge pull request #10895 from ClickHouse/fix_multiple_renames 
Fix error in multiple rename commands in a single query.
 2020-05-16 12:54:04 +03:00
Azat Khuzhin
2498041cc1 Avoid sending partially written files by the DistributedBlockOutputStream 2020-05-16 01:00:42 +03:00
tavplubix
cc4ddfd89e
Merge pull request #10901 from ClickHouse/merging_10554 
Merging #10554
 2020-05-15 23:23:02 +03:00
Mikhail Filimonov
405cce2c53 Another variant of fix. It looks like we do not need destructor at all 2020-05-15 22:19:21 +02:00
alesapin
a504db3451 Fix typo 2020-05-15 18:11:13 +03:00
alesapin
97c2e549be
Merge pull request #10865 from azat/dist-dir-mon-fix 
Fix ENOENT exception on current_batch.txt in DirectoryMonitor
 2020-05-15 18:05:48 +03:00
Vitaly Baranov
d92ac9ea47
Merge branch 'master' into RBAC-10 2020-05-15 14:12:43 +03:00
alesapin
26606dd640 Fix polymorphic parts 2020-05-15 13:36:35 +03:00
alesapin
8eb27f6f13 Fix rename with compact parts 2020-05-14 19:56:13 +03:00
tavplubix
a47bf09337
Merge pull request #10859 from ClickHouse/fix_watch_livelock_with_database_atomic 
Fix livelock with WATCH queries and DatabaseAtomic
 2020-05-14 15:11:41 +03:00
Vitaly Baranov
c30587196a Add system tables for users, roles and grants. 2020-05-14 14:12:20 +03:00
Vitaly Baranov
b1f362b2dc Add system table for access types. 2020-05-14 14:12:20 +03:00
Vitaly Baranov
a14f322723 Add system tables for settings profiles. 2020-05-14 14:12:15 +03:00
alesapin
316773d853 Better comment 2020-05-14 13:30:53 +03:00
Mikhail Filimonov
6499d702f9 After review fixes 2020-05-14 12:15:38 +02:00
Nikolai Kochetov
4d22374f24 Merged with master. 2020-05-14 12:06:15 +03:00
Mikhail Filimonov
a1013f1e71 Fix for the hang during deletion of engine=Kafka 2020-05-14 10:38:02 +02:00
Nikolai Kochetov
4d0503f021 Make IColumn::mutate() static. 2020-05-14 11:30:18 +03:00
Azat Khuzhin
52d73c7f45 Fix prefer_localhost_replica=0 and load_balancing for Distributed INSERT 2020-05-14 03:29:03 +03:00
Azat Khuzhin
e1d4837753 Fix list of possible nodes for Distributed INSERT for internal_replication=0 2020-05-14 03:28:08 +03:00
Azat Khuzhin
cdf3845e43 Respect load_balancing in DirectoryMonitor, to fix w/o internal_replication 2020-05-14 01:33:25 +03:00
alexey-milovidov
623b2e5a43
Merge pull request #10849 from ClickHouse/fix_optimize_and_alter_hangs 
Fix mutations and OPTIMIZE hangs when replica becomes inactive
 2020-05-13 23:41:13 +03:00
Nikolai Kochetov
496d90b613
Merge pull request #10860 from CurtizJ/pk-function 
Fix sample with final (addition to #10715)
 2020-05-13 23:04:25 +03:00
Alexander Tokmakov
ef52acdbde Merge branch 'master' into merging_10554 2020-05-13 21:30:26 +03:00
alesapin
ab760846c2 Fix some rename cases 2020-05-13 20:43:30 +03:00
Vitaly Baranov
5b84121d81 Improve system tables for quotas. Remove function currentQuota(). 2020-05-13 19:40:48 +03:00
Vitaly Baranov
e64e2ebdf6 Improve system table for row policies. Remove function currentRowPolicies(). 2020-05-13 19:40:48 +03:00
Vitaly Baranov
b6fe726777 Rename row policy's 'name' to 'short_name', 'full_name' to 'name'. 
This change simplifies the interface of IAccesEntity.
 2020-05-13 19:40:48 +03:00
Vitaly Baranov
6f15a0d443 Improve the function range() to allow iterating through enum values. 2020-05-13 19:40:48 +03:00
Anton Popov
84d4ad4315 add some comments 2020-05-13 19:00:06 +03:00
Dmitry
bbe0245b9d changes after review #1 2020-05-13 16:49:10 +03:00
alesapin
2316851be6
Merge pull request #10855 from ClickHouse/get_rid_of_strange_fields 
Better comments in IStorage interface
 2020-05-13 15:59:50 +03:00
alesapin
352c402cf2 Merge branch 'add_alter_rename_column_to_distributed' of https://github.com/vzakaznikov/ClickHouse into vzakaznikov-add_alter_rename_column_to_distributed 2020-05-13 14:14:23 +03:00
alesapin
f209052519 Better comments 2020-05-13 14:04:27 +03:00
alesapin
377ef6535d
Merge pull request #10847 from ClickHouse/fix_alter_rename_and_constraints 
Fix constraints after column rename
 2020-05-13 11:09:55 +03:00
Alexander Tokmakov
032197b015 fix alter and optimize hangs when replica becomes inactive 2020-05-13 10:06:57 +03:00
alesapin
13c8ffc0c9
Fix exception message after server shutdown (#10841) 
* After shutdown cancel background logs pulling after task were deleted from pool

* Fix comment

* Cancel with lock

* Remove whitspace
 2020-05-13 06:35:43 +03:00
Dmitry
152a636c23 fix bad merge 2020-05-13 03:13:01 +03:00
Azat Khuzhin
085bafad05 Handle prefer_localhost_replica on INSERT into Distributed 
Right now it will issue remote send even if finally the local replica
will be selected - not good I guess.

This should also fix load_balancing.
 2020-05-13 01:38:03 +03:00
Azat Khuzhin
889f54b549 Fix ENOENT exception on current_batch.txt in DirectoryMonitor 
current_batch.txt will not exist if there was no send, this is the case
when all batches that was pending has been marked as pending.
 2020-05-13 01:23:18 +03:00
Anton Popov
67213b8ad4 fix sample with final 2020-05-12 21:23:40 +03:00
Alexander Tokmakov
5ffb7372af fix livelock with watch queries and database atomic 2020-05-12 21:10:10 +03:00
alesapin
262b6287ab Fix clang-tidy warnings 2020-05-12 19:38:11 +03:00
Dmitry
c4295d97f9 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-12 17:57:32 +03:00
Nikolai Kochetov
0ef9b20823 Try fix sample final. 2020-05-12 17:37:19 +03:00
alesapin
2b51b7aa2a Remove redundant using 2020-05-12 14:47:59 +03:00
alesapin
979779e06f Fix constraints after column rename 2020-05-12 14:26:44 +03:00
Nikolai Kochetov
5d13327ad8 Try fix merge with master. 2020-05-12 12:22:01 +03:00
Anton Popov
a1333895eb
Merge pull request #10791 from oandrew/key-condition-source-type 
Use src_type for conversion in KeyCondition
 2020-05-12 03:29:43 +03:00
Alexey Milovidov
25cf842491 Fix the issue with ODBC bridge and identifier_quoting_style = None #7984 2020-05-11 23:05:08 +03:00
Gleb Novikov
d255bb998a VolumePtr instead of DiskPtr in IMergedBlockOutputStream and inherited classes 2020-05-11 20:20:26 +03:00
alexey-milovidov
33d491edf3
Merge pull request #10516 from azat/dist-GROUP_BY-sharding_key-fixes 
Disable GROUP BY sharding_key optimization by default (and fix for WITH ROLLUP/CUBE/TOTALS)
 2020-05-11 12:03:27 +03:00
alexey-milovidov
06ba85fdf2
Merge pull request #10795 from ClickHouse/list-licenses 
Added "system.licenses" table
 2020-05-11 09:00:55 +03:00
Alexey Milovidov
3fcbb06cab Fix MSan report in MergeTreeIndexFullText 2020-05-11 08:04:59 +03:00
Alexey Milovidov
141218d44a Fix Arcadia 2020-05-11 07:36:32 +03:00
Alexey Milovidov
c5e78d12e7 Merge branch 'master' into list-licenses 2020-05-11 07:35:35 +03:00
alexey-milovidov
40ef113503
Merge branch 'master' into parallel-final 2020-05-11 06:47:44 +03:00
alexey-milovidov
ddc84163a7
Merge pull request #10486 from azat/dist-send-on-INSERT 
Fix distributed send that are scheduled by INSERT query
 2020-05-11 06:28:35 +03:00
Alexey Milovidov
da583760c4 Added "system.licenses" table 2020-05-11 05:39:01 +03:00
Andrew Onyshchuk
330f06328f Use src_type for convertion in KeyCondition 2020-05-10 18:17:54 -05:00
Gleb Novikov
1235c36f3e data_part instead of volume in IMergeTreeDataPartWriter (as it done in IMergeTreeReader) 2020-05-10 16:33:27 +03:00
Gleb Novikov
57b6fb6d80 Merge branch 'master' into refactor-reservations 2020-05-10 13:01:45 +03:00
Azat Khuzhin
5c89cdbe61 Fix distributed send retries on distributed_directory_monitor_{max_,}sleep_time_ms > 5min 
In this case error_count can be decreased before checking it for
rescheduling send.

And actually this can be a problem not only when
distributed_directory_monitor_{max_,}sleep_time_ms > 5min, because all
threads can be occupated and the real timeout between sends will be > 5min.
 2020-05-10 12:37:38 +03:00
Alexey Milovidov
36d9d646a9 Fix race condition in ReplicatedMergeTreeQueue 2020-05-10 05:18:10 +03:00
alexey-milovidov
992e58923a
Merge pull request #10588 from excitoon-favorites/fixmutations 
Fixed handling condition variable for synchronous mutations
 2020-05-10 03:20:41 +03:00
alexey-milovidov
63c4a7661f
Merge pull request #10715 from CurtizJ/pk-function 
Fix usage of primary key wrapped into a function.
 2020-05-10 02:08:59 +03:00
Gleb Novikov
390b39b272 VolumePtr instead of DiskPtr in MergeTreeData* 2020-05-10 00:24:15 +03:00
Dmitry
0286b60ed6 return multiple blocks 2020-05-08 22:46:52 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396) 
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
 2020-05-08 17:11:19 +03:00
Dmitry
e7b747b0b7 limit fixes + func(primary_key) group by works 2020-05-08 16:13:50 +03:00
Vladimir Chebotarev
28631f461e Fixed handling of condition variable for synchronous mutations. 2020-05-08 12:01:06 +03:00
Nikolai Kochetov
c22f7a0864 Try fix header for GenerateRandom. 2020-05-07 17:14:56 +03:00
Vitaliy Zakaznikov
90e52e7fea Adding support for ALTER RENAME COLUMN query to Distributed table engine. 2020-05-07 14:54:35 +02:00
Anton Popov
cbbd8e521c fix usage of primary key wrapped into a function with 'FINAL' modifier and 'ORDER BY' optimization 2020-05-07 01:27:35 +03:00
alexey-milovidov
4df7dd8c2b
Merge pull request #10656 from azat/kafka-consumer-hang 
Kafka cleanup
 2020-05-07 00:20:31 +03:00
bobrovskij artemij
a2916c23a5 fixes/style 2020-05-06 02:42:44 +03:00
Dmitry
e98c23a8cf read in order enabled 2020-05-05 17:35:23 +03:00
alexey-milovidov
114f09cfca
Merge pull request #10666 from NanoBjorn/refactor-volume 
Volumes and storages refactoring
 2020-05-05 16:25:26 +03:00
alexey-milovidov
c54f6d5a66
Merge pull request #10546 from kyprizel/master 
Lexer, ast, checksum and column specs fuzzers added
 2020-05-05 14:50:55 +03:00
alexey-milovidov
09756a036a
Merge pull request #10645 from ClickHouse/check-suspicious-codecs 
Check for suspicious codecs #4966
 2020-05-04 23:33:59 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring: 
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
 2020-05-04 23:15:38 +03:00
alesapin
d47d28d51f Fix mutations assignment 2020-05-04 19:30:39 +03:00
Azat Khuzhin
61c5df5662 Drop claimed from the KafkaBlockInputStream 2020-05-04 17:23:09 +03:00
Azat Khuzhin
a10cfc3ae5 Pass stream_cancelled to the copyData as stub in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
c69217e9f3 Force destrying of kafka buffers from the shutdown in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
9e14db2091 Add missing kafka settings into docs 2020-05-04 17:23:05 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
alexey-milovidov
884c2aa631
Merge pull request #10641 from ClickHouse/storage-buffer-nullptr-dereference 
Fix nullptr dereference in StorageBuffer
 2020-05-04 02:01:57 +03:00
Alexey Milovidov
a5bdc375ca Fix Arcadia #10569 2020-05-04 00:36:11 +03:00
Alexey Milovidov
f8866af0c1 Fix nullptr dereference in StorageBuffer 2020-05-03 23:53:54 +03:00
Azat Khuzhin
6ffdd53b6a Share auto-increment for first batch and tmp file in DistributedBlockOutputStream 2020-05-03 14:47:59 +03:00
Azat Khuzhin
53c470cab4 Fix directory monitor initialization from INSERT into Distributed 
This also fixes hardlink code (when one file should be sent to multiple
servers, i.e. internal_replication == false) of writeToShard() with
distributed_storage_policy (i.e. when StorageDistributed::getPath() will
path to different filesystems).

Plus also cleanup DistributedBlockOutputStream::writeToShard() a little.
 2020-05-03 14:47:51 +03:00
Azat Khuzhin
e97e1f06db Do not schedule distributed send if there were no error 
Since in this case it will be scheduled from the
DistributedBlockOutputStream with the
distributed_directory_monitor_max_sleep_time_ms, and this will overwrite
timer that was set by the DistributedBlockOutputStream, not good.
 2020-05-03 14:46:44 +03:00
Azat Khuzhin
947b3942dd Schedule distributed sends after the file has been written 2020-05-03 14:46:43 +03:00
Azat Khuzhin
0157fd5d93 Fix distributed send that are scheduled by INSERT query 
Before this patch each INSERT query re-schedule distributed send, thus
each time it resets the timer, while this is not the expected behaviour,
since in on frequent INSERT distributed sends will not be triggered at
all.

Fix this by not resetting the timer.
 2020-05-03 14:46:42 +03:00
Azat Khuzhin
6bb39dafc3 Drop decreated code (cond var and note for thread) in DirectoryMonitor 2020-05-03 14:46:41 +03:00
alexey-milovidov
db4c2352e1
Merge pull request #10569 from zhang2014/fix/ISSUES-10551 
ISSUES-10551 add backward compatibility for create bloom filter index
 2020-05-02 22:25:29 +03:00
alexey-milovidov
229f666dea
Merge pull request #10611 from azat/optimize_skip_unused_shards-LowCardinality 
Fix optimize_skip_unused_shards with LowCardinality
 2020-05-02 22:14:33 +03:00
alexey-milovidov
4f1b115e76
Merge pull request #10610 from azat/optimize_skip_unused_shards-fixes 
Fix logging for optimize_skip_unused_shards
 2020-05-02 22:13:01 +03:00
tavplubix
a073cd6c79
Fix segfault in StorageBuffer when exception on server startup (#10609) 
* fix segfault in StorageBuffer when exception on server startup

* Trigger CI

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-05-02 19:54:20 +03:00
Alexey Milovidov
81adcc9058 Remove obsolete records from system.build_options 2020-05-02 17:00:03 +03:00
alexey-milovidov
404452bad3
Update MergeTreeIndices.cpp 2020-05-01 21:40:56 +03:00
alesapin
4d8660f270 Remove obsolete comment 2020-05-01 13:05:37 +03:00
Azat Khuzhin
63d8ab8f03 Make createSelector() static (in storage) and const (in stream) 2020-05-01 11:31:05 +03:00
Azat Khuzhin
f22ba15b4a Reduce copy-paste of DistributedBlockOutputStream::createSelector 
This will make it less error prone.
 2020-05-01 02:59:40 +03:00
Azat Khuzhin
cdd7013438 Drop superfluous "Skipping irrelevant shards" messages 
Before this patch it printed 3 times:
- from StorageDistributed::getProcessingStageImpl()
- from StorageDistributed::read()
- from StorageDistributed::getProcessingStageImpl() (from StorageDistributed::read() -> getSampleBlock())
(But this should be optimized)
 2020-05-01 02:56:13 +03:00
Azat Khuzhin
c648c300bf Fix optimize_skip_unused_shards with LowCardinality 2020-05-01 02:39:58 +03:00
Azat Khuzhin
4cbe625567 Fix shard numbers output in logs (full cluster had been printed over optimized) 2020-05-01 02:13:07 +03:00
Nikolai Kochetov
19dadb8c2d Add parallel final. 2020-04-30 12:59:08 +03:00
alesapin
15e8f37839
Merge pull request #10531 from CurtizJ/polymorphic-parts-2 
Fix index corruption in merges with compact parts.
 2020-04-30 12:15:25 +03:00
alesapin
5e2afad549
Merge pull request #10477 from ClickHouse/refactor_istorage 
Remove getColumn and hasColumn from IStorage, better virtuals.
 2020-04-30 10:42:52 +03:00
Anton Popov
6ed3092c8a make fillIndexGranularity less complicated 2020-04-30 00:57:58 +03:00
alesapin
69fdf2a6a3 Review fixes and better test 2020-04-29 15:15:23 +03:00
zhang2014
8ac709b5d3 ISSUES-10551 add backward compatibility for create bloom filter index 2020-04-29 11:57:28 +08:00
Azat Khuzhin
038235684d Add optimize_distributed_group_by_sharding_key and disable it by default 
I know at least one way to fool that optimization, by using as sharding
key something like `if(col1>0, col1, col2)` (although this is not common
sharding key I would say, but can be useful if this will work
correctly), so let's disable it by default.
 2020-04-29 00:09:25 +03:00