thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 09:52:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
104,053 Commits 9,324 Branches 1,783 Tags 2.1 GiB
dfe93b5d82
Commit Graph

3746 Commits

Author SHA1 Message Date
xiedeyantu
304b6ebf3a s3 table function can support select nested column using {column_name}.{subcolumn_name} 2022-11-23 23:36:12 +08:00
Alexey Milovidov
c23b8c69db
Merge pull request #42791 from azat/fix-pe-overflow 
Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting
 2022-11-22 19:22:51 +01:00
Raúl Marín
4aa29b6a63 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-22 19:09:00 +01:00
Raúl Marín
e63ba06048 Better cache management 2022-11-22 19:03:17 +01:00
Kruglov Pavel
98d6b96c82
Merge pull request #42033 from mark-polokhov/BSONEachRow 
Add BSONEachRow input/output format
 2022-11-22 14:45:21 +01:00
Nikolai Kochetov
9e84a351c7 Fixing special build. Disable setting by default. 2022-11-22 11:24:24 +00:00
Nikolai Kochetov
aeb1de5b59 Aggregating in order [in progress]. 2022-11-21 18:58:44 +00:00
avogar
ecdeff622b Add small improvements in CapnProto format 2022-11-18 20:13:00 +00:00
Nikolai Kochetov
5da1d893fd
Merge branch 'master' into read-from-mt-in-io-pool 2022-11-18 21:10:45 +01:00
Azat Khuzhin
ccde92f855 Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting 
After #40732 it became possible that getrusage() (from detachQuery(),
from buildPushingToViewsChain()) will be called for incorrect thread,
and so when the difference will be calculated it will be simply garbage.

But actually the root of this problem is #25714, after which it became
possible to have multiple ThreadStatus for one thread, and this is very
tricky (sigh).

Here are some other thoughts about it:
- Make ThreadStatus nested - decided that complexity does not worth it,
  at least only for this case
- Move some members into ThreadGroupStatus - will break per-thread
  statistics (and hence query_thread_log, BTW does somebody uses it?)
- Move some members into a separate structure

But decided to fix the issue w/o any refactoring, to make easy for
backport.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-18 21:10:43 +01:00
Nikolai Kochetov
03c35e146b Refactor a bit. 2022-11-18 20:09:20 +00:00
Han Fei
b80a2d6c89 address comments 2022-11-18 17:22:05 +01:00
Alfonso Martinez
4b4486e139 Improved stability of test 2022-11-18 12:18:01 +01:00
Arthur Passos
414fd07bba add docs 2022-11-17 17:28:51 -03:00
Arthur Passos
dd37ca7767 add docs 2022-11-17 17:25:27 -03:00
Arthur Passos
12d3f799a5 small change 2022-11-17 17:18:54 -03:00
Arthur Passos
fcc032a31e handle both zero based and non zero based arrow offsets 2022-11-17 17:15:24 -03:00
avogar
fcfdd73d17 Improve reading CSV field in CustomSeparated/Template format 2022-11-17 15:36:56 +00:00
Raúl Marín
80403015e7 Fix assert reached with lines without data 2022-11-17 16:19:53 +01:00
Alfonso Martinez
6a8101b8b6 Test redone on bash 2022-11-17 10:14:59 +01:00
Han Fei
14701670b5 Merge branch 'master' into hanfei/aysnc-insert 2022-11-16 17:24:36 +01:00
Arthur Passos
ed080b8ba5 fix style 2022-11-16 13:22:23 -03:00
Arthur Passos
e1236340b5 Flatten list type arrow chunks on parsing 2022-11-16 12:27:01 -03:00
Nikolai Kochetov
29098045d8 Try read from MergeTree in background pool. 2022-11-15 17:38:37 +00:00
avogar
2af60f34eb Restrict document size in parallel parsing, allow to read ObjectId/JS code into String column 2022-11-15 13:35:17 +00:00
avogar
842d25c358 Minor improvements, better docs 2022-11-14 20:05:01 +00:00
Alfonso Martinez
e62ecf1a59 Merge remote-tracking branch 'upstream/master' into fix-race-condition-between-inserts-and-dropping-mvs 2022-11-14 18:23:31 +01:00
avogar
098dfcff56 Fix tests 2022-11-14 15:48:23 +00:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Alexey Milovidov
127631ee47
Merge branch 'master' into perf_experiment 2022-11-12 18:58:25 +01:00
Nikolai Kochetov
9106910de8 Try read from MergeTree in background pool. 2022-11-11 20:47:11 +00:00
Alfonso Martinez
9bce4b8178 Added warning to fixed case 2022-11-11 21:02:37 +01:00
Alfonso Martinez
ed3a576e7d Added warning to fixed case 2022-11-11 21:02:14 +01:00
avogar
564d83bbc7 Better handle uint64 2022-11-11 13:24:12 +00:00
avogar
94c6dc42eb Use better types 2022-11-11 13:17:48 +00:00
Alfonso Martinez
b4769811fc Fixed race condition between inserts and dropping MVs 2022-11-11 12:07:50 +01:00
avogar
cd36caf013 Fix style 2022-11-10 20:37:24 +00:00
avogar
e0b3b9efae Remove old test, clean up a bit 2022-11-10 20:21:29 +00:00
avogar
4d787f3953 Remove unneded method 2022-11-10 20:18:52 +00:00
avogar
9e89af28c6 Refactor BSONEachRow format, fix bugs, support more data types, support parallel parsing and schema inference 2022-11-10 20:15:14 +00:00
Nikolai Kochetov
77c0728e77 Fix aarch build. 2022-11-10 11:59:25 +00:00
Nikolai Kochetov
f2f5c170e4
Merge branch 'master' into read-in-order-from-query-plan 2022-11-09 22:36:36 +01:00
Nikolai Kochetov
ff65ca42ad Fix limit. 2022-11-09 18:12:01 +00:00
Nikolai Kochetov
3c3771a8d7 Fix typo. 2022-11-09 16:17:36 +00:00
Nikolai Kochetov
997881c7f7 Review fixes. 2022-11-09 16:07:38 +00:00
Nikolai Kochetov
5a3d4cd72e Fix typos. Add comments. 2022-11-08 19:57:29 +00:00
Raúl Marín
0c5159d26c Merge remote-tracking branch 'blessed/master' into agg_memory_rails 2022-11-08 10:59:07 +01:00
Vladimir C
88033562cd
Merge pull request #42874 from azat/query-hung-on-cancel-fix 2022-11-08 10:52:58 +01:00
Raúl Marín
a04b9fa947 Add some guard rails around aggregation memory management 2022-11-07 17:25:04 +01:00
Nikolai Kochetov
9043df56a8 Add comments, fix tests. 2022-11-04 17:31:20 +00:00
Nikolai Kochetov
2db1638b41 Disable optimize_in_window_order in case if read-in-order for query plan is enabled. 2022-11-04 16:18:35 +00:00
Nikolai Kochetov
51ec95eb82 Try to fix #39157 in a different way. 2022-11-03 20:33:48 +00:00
Nikolai Kochetov
d01ef8ca59
Merge branch 'master' into read-in-order-from-query-plan 2022-11-03 18:59:28 +01:00
Nikolai Kochetov
280e609744 More fixes. 2022-11-03 16:42:26 +00:00
Nikolay Degterinsky
30ad1a6826
Merge branch 'master' into perf_experiment 2022-11-03 02:18:21 +03:00
avogar
7cc87679e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BSONEachRow 2022-11-02 19:47:42 +00:00
Vladimir C
512abfe511 Fix style, remove commented code 2022-11-02 19:42:57 +00:00
vdimir
ef3dbf8192 clang-format BSONUtils 2022-11-02 19:42:49 +00:00
vdimir
223614ee1d Fix typos 2022-11-02 19:42:32 +00:00
vdimir
ab61932223 Apply clang-format for BSONEachRow 2022-11-02 19:42:13 +00:00
Mark Polokhov
2fff4887ac Add BSON input/output format 2022-11-02 19:39:14 +00:00
Nikolai Kochetov
1f11c73dab Another try. 2022-11-02 18:46:56 +00:00
Nikolai Kochetov
4641f12964 Another try. 2022-11-02 18:33:48 +00:00
Nikolai Kochetov
d551161a29 Another one try. 2022-11-02 17:11:17 +00:00
Kruglov Pavel
38124b6533
Merge pull request #42780 from Avogar/parallel-parsing 
Support parallel parsing for LineAsString input format
 2022-11-02 13:21:53 +01:00
Nikolai Kochetov
2766c550f6 Fix typos. 2022-11-02 10:38:32 +00:00
Nikolai Kochetov
478d3079c8 Merge branch 'master' into read-in-order-from-query-plan 2022-11-01 21:02:22 +01:00
Nikolai Kochetov
30f7c04896 Remove some debug output. 2022-11-01 19:58:20 +00:00
Nikolai Kochetov
9ffebf4a0c Fixing read-in-order for special storages. 2022-11-01 19:51:52 +00:00
Azat Khuzhin
e58008b711 Fix rare possible hung on query cancellation 
Some queries can hung after cancelling (because enough rows had been
read for Distributed queries) or after KILL. Example of such query is a
Distributed query that uses DelayedSource that uses ExpandPipeline to
add new processors, and in this case it is possible that all already
existing processes was cancelled but new had been added and now
PipelineExecutor will wait for them undefinitelly since nobody will read
from them and nobody will cancel them either.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-01 18:17:47 +01:00
Nikolai Kochetov
52204233e4 Fix some more tests. 2022-11-01 15:11:26 +00:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14 
Support `Object` type inside other types
 2022-11-01 15:20:02 +01:00
Maksim Kita
fe48a1ce7e
Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag 
MergeTree indexes use ActionsDAG
 2022-11-01 13:55:17 +03:00
Nikolai Kochetov
e99fd4ed25 Add test 2022-10-31 20:50:10 +00:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Nikolai Kochetov
068ae90448 Comment debug code. 2022-10-31 14:01:51 +00:00
Nikolai Kochetov
375db5bd15 Use read-in-order from query plan by default. 2022-10-31 13:54:03 +00:00
vdimir
6a4247ca32
Merge branch 'master' into grace_hash_join 2022-10-31 09:54:37 +00:00
Gabriel
94866979c5
Merge branch 'master' into comment_typo 2022-10-31 13:29:53 +08:00
Gabriel
620caeb07c Fix typo in comments 2022-10-31 13:20:58 +08:00
avogar
fe0aea2e3a Support parallel parsing for LineAsString input format 2022-10-28 21:56:09 +00:00
Nikolai Kochetov
5d41e7a6d5 Read-in-order over query plan (continuation) 2022-10-28 20:03:23 +00:00
Nikolai Kochetov
5106c24e88 Merge branch 'master' into read-in-order-from-query-plan 2022-10-27 18:12:20 +00:00
Raúl Marín
e77fcb0a99 More style 2022-10-27 13:22:44 +02:00
Raúl Marín
891484b462 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-10-27 13:17:07 +02:00
Robert Schulze
31ef6c66cb
Merge pull request #42513 from ClickHouse/update-libcxx-to-15 
Build with libcxx(abi) 15
 2022-10-27 10:30:36 +02:00
Raúl Marín
56a802188b Fix bugs introduced when changing the logic 2022-10-26 18:05:05 +02:00
Maksim Kita
1c17e9d454 Fixed tests 2022-10-26 15:54:37 +02:00
vdimir
506bf2d225
Merge branch 'master' into grace_hash_join 2022-10-26 12:25:50 +00:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows 
fix behaviour of max_rows_to_read for trival limit queries
 2022-10-26 14:09:59 +02:00
Maksim Kita
1b6293f6db MergeTree indexes use ActionsDAG 2022-10-26 12:44:37 +02:00
Raúl Marín
9395f77421 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-10-26 11:46:17 +02:00
Robert Schulze
728342ec7c
Un-inline stuff 2022-10-25 21:41:08 +00:00
Raúl Marín
6e0a9452e7 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-10-25 15:25:06 +02:00
Raúl Marín
2fa3c54caa ValuesBlockInputFormat: Adapt to the full tokenizer 2022-10-25 15:22:22 +02:00
Maksim Kita
e6bfff3de2 Analyzer change setting into allow_experimental_analyzer 2022-10-25 12:35:25 +02:00
Han Fei
8f00d0d1cc fix tidy 2022-10-24 12:56:36 +02:00
Maksim Kita
8cadb1b318 Added WINDOW functions support 2022-10-24 10:22:20 +02:00
Maksim Kita
4f68305dbe Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
ca93ee7479 Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
fed146e198 Added SAMPLE BY support. Added SAMPLE BY, FINAL support for JOINS. 2022-10-24 10:22:20 +02:00
Maksim Kita
85193ede42 Added UNION support 2022-10-24 10:22:19 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Han Fei
2fc91fd338 fix behaviour of max_rows_to_read for trival limit queries 2022-10-22 18:27:24 +02:00
Azat Khuzhin
56bc85746f Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 
Conflicts:
- src/Interpreters/ProcessList.cpp
 2022-10-22 16:49:08 +02:00
Azat Khuzhin
5094c0dd6d Fix clang-tidy performance-inefficient-vector-operation 
By some reason it appears only after static_cast<> was added [1]:

    /build/src/Processors/Formats/Impl/AvroRowInputFormat.cpp
    Oct 18 01:03:56 /build/src/Processors/Formats/Impl/AvroRowInputFormat.cpp:351:21: error: 'push_back' is called inside a loop; consider pre-allocating the container capacity before the loop [performance-inefficient-vector-operation,-warnings-as-errors]
    Oct 18 01:03:56                     symbols.push_back(root_node->nameAt(i));
    Oct 18 01:03:56                     ^
    Oct 18 01:03:56 /build/src/Processors/Formats/Impl/AvroRowInputFormat.cpp:511:17: error: 'push_back' is called inside a loop; consider pre-allocating the container capacity before the loop [performance-inefficient-vector-operation,-warnings-as-errors]
    Oct 18 01:03:56                 union_skip_fns.push_back(createSkipFn(root_node->leafAt(i)));
    Oct 18 01:03:56                 ^
    Oct 18 01:03:56 /build/src/Processors/Formats/Impl/AvroRowInputFormat.cpp:552:17: error: 'push_back' is called inside a loop; consider pre-allocating the container capacity before the loop [performance-inefficient-vector-operation,-warnings-as-errors]
    Oct 18 01:03:56                 field_skip_fns.push_back(createSkipFn(root_node->leafAt(i)));
    Oct 18 01:03:56                 ^
    Oct 18 01:03:56 197965 warnings generated.

  [1]: https://s3.amazonaws.com/clickhouse-builds/42190/453d91fa3539882dcef1d5ecd5097747499572d8/clickhouse_special_build_check/report.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Alexey Milovidov
ee5f5a4cb4 Unfathomable amount of trash 2022-10-21 05:33:17 +02:00
Alexey Milovidov
ff26251477 Merge branch 'master' into fix-race-condition-finish-cancel 2022-10-21 04:14:21 +02:00
Alexander Tokmakov
68c18abfbb
Merge pull request #42406 from ClickHouse/template_format_better_error 
Better error message for unsupported delimiters in custom formats
 2022-10-20 15:52:08 +03:00
Robert Schulze
820e6b4276
Build with libcxx(abi) 15 2022-10-20 10:52:43 +00:00
vdimir
5118c5d74f
wip grace hash 2022-10-19 10:20:41 +00:00
Alexey Milovidov
dfa202a15d Merge branch 'master' into fix-race-condition-finish-cancel 2022-10-19 02:35:42 +02:00
Kruglov Pavel
29513f6a1f
Merge pull request #41885 from Avogar/with-names-error-message 
Better exception message for duplicate column names in schema inference
 2022-10-18 15:26:46 +02:00
Kruglov Pavel
25e13bdd2f
Merge pull request #41107 from Avogar/improve-combinators 
Support all combinators combination in WindowTransform/arratReduce*/initializeAggregation/aggregate functions versioning
 2022-10-18 15:24:49 +02:00
vdimir
bf3f66b5b6
fix delayed blocks for grace hash join 2022-10-18 11:43:01 +00:00
Vladimir C
a298a5238b
Merge pull request #42217 from ClickHouse/vdimir/style-check-code-used-in-comment 2022-10-18 13:26:39 +02:00
Alexander Tokmakov
fffecbb9ad better error message for unsupported delimiters in custom formats 2022-10-17 18:08:52 +02:00
vdimir
adb63a5583
Merge branch 'master' into grace_hash_join 2022-10-17 12:32:56 +00:00
Alexey Milovidov
f88ed8195b Fix trash 2022-10-17 04:21:08 +02:00
Alexey Milovidov
c6b2ee47df Fix data race in query finish/cancel 2022-10-17 02:10:36 +02:00
Kruglov Pavel
6fc12dd922
Merge pull request #41703 from Avogar/json-object-each-row 
Add setting to obtain object name as column value in JSONObjectEachRow format
 2022-10-14 20:11:04 +02:00
Kruglov Pavel
ff11904850
Merge branch 'master' into improve-combinators 2022-10-14 17:19:31 +02:00
vdimir
ad255206f4
Exclude comments from style-check defined extern 2022-10-14 11:42:18 +00:00
vdimir
0178307c27 Followup for TemporaryDataOnDisk 2022-10-12 15:25:23 +02:00
Nikolai Kochetov
dac71f445b
Merge pull request #42186 from ClickHouse/igor/distinct_in_order_fix 
Fix: DISTINCT in order fails with LOGICAL_ERROR
 2022-10-10 09:51:29 +02:00
Igor Nikonov
f4792c37ef Fix #42185: DISTINCT in order fails with LOGICAL_ERROR 
... if first column in sorting key contains function
 2022-10-08 10:13:12 +00:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
vdimir
5a7c493559
Merge branch 'master' into pr/BigRedEye/38191 2022-10-06 14:43:35 +00:00
Robert Schulze
78be400ac0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 2022-10-06 12:22:43 +00:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
vdimir
ff55c369bc
Merge branch 'tmp-data-followup' 2022-10-05 18:10:05 +00:00
vdimir
ccd29e172f
Followup for TemporaryDataOnDisk 2022-10-05 16:42:27 +00:00
vdimir
a0349cf8c5
wip grace hash 2022-10-05 12:40:32 +00:00
Anton Popov
fe90919f10 fix crash in SummingMergeTree with LowCardinality 2022-10-04 15:33:38 +00:00
vdimir
e5678afc2d
wip grace hash 2022-10-04 08:21:02 +00:00
vdimir
f3781be762
wip grace hash 2022-10-04 08:20:13 +00:00
Anton Popov
6e61cf92f5 Merge remote-tracking branch 'upstream/master' into HEAD 2022-10-03 13:16:57 +00:00
Nikolai Kochetov
dcfb8a1131
Merge pull request #41949 from ClickHouse/improvement-from-38083 
Earlier throw exception in PullingAsyncPipelineExecutor.
 2022-10-03 13:05:58 +02:00
Robert Schulze
db5ef7b3cb
Merge branch 'master' into generated-file-cleanup 2022-10-02 23:13:18 +02:00
Vitaly Baranov
f65d3ff95a Fix parallel parsing: segmentator now checks max_block_size. 2022-09-30 22:34:03 +02:00
vdimir
7ebc297f4c
Merge branch 'master' into pr/BigRedEye/38191 2022-09-30 09:40:47 +00:00
Vladimir C
895afdec45
Merge pull request #40893 from ClickHouse/vdimir/track-tmp-disk 2022-09-30 11:27:24 +02:00
Robert Schulze
cc92a2d174
Merge branch 'master' into generated-file-cleanup 2022-09-30 09:56:31 +02:00
Nikolai Kochetov
8f9e6dd03a Earlier throw exception in PullingAsyncPipelineExecutor. 2022-09-29 18:07:43 +00:00
Nikolai Kochetov
8e23bee03c
Merge pull request #41650 from ClickHouse/keep-destruction-order-of-view-thread-statuses 
Fix a destruction order for views ThreadStatus
 2022-09-29 14:28:14 +02:00
vdimir
f495361e28
fixes for TemporaryDataOnDisk 2022-09-29 10:09:29 +00:00
vdimir
14cd2b632c
fix Context::shutdown 2022-09-29 09:51:49 +00:00
vdimir
0f1a7c252d
better TemporaryDataOnDisk 2022-09-29 09:51:46 +00:00
vdimir
9f3f34548c
Allow to create temporaty streams on leaf TemporaryDataOnDisk 2022-09-29 09:51:45 +00:00