avogar
4ab3e90382
Validate types in table function arguments/CAST function arguments/JSONAsObject schema inference
2022-12-21 21:21:30 +00:00
Igor Nikonov
c7650850fe
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-21 15:56:08 +00:00
Igor Nikonov
919fa9b84e
If aggregation step is in order, - do not remove underneath sorting step
2022-12-21 12:55:42 +00:00
Igor Nikonov
fdb277245c
Remove current sorting only if next is full sorting
2022-12-21 12:21:35 +00:00
Kruglov Pavel
5e01a3d74e
Merge branch 'master' into improve-streaming-engines
2022-12-21 10:51:50 +01:00
Kruglov Pavel
09ab5832b1
Merge pull request #44382 from Avogar/fix-bson-object-id
...
Fix reading ObjectId in BSON schema inference
2022-12-21 10:48:50 +01:00
Igor Nikonov
1624647f07
Fix: distributed query with cluster() table function
2022-12-20 23:27:01 +00:00
Alexey Milovidov
a6f50edde3
Merge branch 'master' into add-test-29883
2022-12-21 01:45:18 +03:00
Igor Nikonov
5cd4767838
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-20 17:56:59 +00:00
avogar
c49638e3a9
Respect setting settings.schema_inference_make_columns_nullable in Parquet/ORC/Arrow formats
2022-12-20 17:46:42 +00:00
Dmitry Novik
4793412887
Merge branch 'master' into refector-function-node
2022-12-20 18:26:19 +01:00
Kruglov Pavel
643a35bed1
Merge pull request #44019 from Avogar/refactor-schema-inference
...
Refactor and improve schema inference for text formats
2022-12-20 17:29:03 +01:00
Kruglov Pavel
c0b17ca0af
Merge branch 'master' into fix-bson-object-id
2022-12-20 17:18:10 +01:00
Kruglov Pavel
a0ee1c1227
Merge branch 'master' into fix-sum-if
2022-12-20 15:20:23 +01:00
Kruglov Pavel
fe28faa32d
Fix style
2022-12-20 14:49:39 +01:00
Kruglov Pavel
3f1e40aacd
Merge branch 'master' into fix-orc
2022-12-20 13:32:46 +01:00
Igor Nikonov
15bb4cb65c
Fix distributed query check
2022-12-20 10:21:32 +00:00
Raúl Marín
45d27f461b
Merge branch 'master' into perf_experiment
2022-12-20 09:07:48 +00:00
Igor Nikonov
2164b18485
Disable the optimization for distributed queries
...
+ based on ReadFromRemote
- improve by disabling optimization only between Union and ReadFromRemote
2022-12-19 23:24:35 +00:00
avogar
e262e375dc
Fix reading columns that are not presented in input data in Parquet/ORC formats
2022-12-19 20:30:54 +00:00
avogar
0c406adce2
Fix reading Map type in ORC format
2022-12-19 18:23:07 +00:00
Igor Nikonov
1e53b3be38
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-19 16:44:22 +00:00
Nikolai Kochetov
860afd1832
Add a very bad fix.
2022-12-19 16:33:10 +00:00
avogar
21cdf6e6ae
Fix reading ObjectId in BSON schema inference
2022-12-19 14:13:42 +00:00
Alexey Milovidov
3e8dcdc155
Add earlier check
2022-12-18 08:01:43 +01:00
avogar
22e70fa71e
Try fix tests
2022-12-16 18:09:42 +00:00
Dmitry Novik
875a24a650
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-16 16:07:30 +00:00
avogar
291e51c533
Merge branch 'better-capn-proto' of github.com:Avogar/ClickHouse into better-capn-proto
2022-12-16 14:43:06 +00:00
avogar
4a51bdce86
Fix comments
2022-12-16 13:58:54 +00:00
Kruglov Pavel
3fad5c7f1f
Merge branch 'master' into refactor-schema-inference
2022-12-16 14:24:51 +01:00
avogar
34274be8df
Fix use after free
2022-12-16 13:20:36 +00:00
Igor Nikonov
ea0bd38ab1
Avoid to remove sorting expression for now
2022-12-15 21:11:02 +00:00
avogar
cfcb444699
Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-capn-proto
2022-12-15 20:04:43 +00:00
Kruglov Pavel
25f199dd89
Merge pull request #43332 from Avogar/csv-custom-delimiter
...
Improve reading CSV field in CustomSeparated/Template format
2022-12-15 21:03:29 +01:00
Igor Nikonov
629f317d7d
Fix for full sorting merge join
2022-12-15 20:01:50 +00:00
avogar
f26b5b57ad
Better comment
2022-12-15 19:52:57 +00:00
avogar
5b4ea53fed
Fix fields order
2022-12-15 19:52:25 +00:00
avogar
755b08a49e
Fix comments
2022-12-15 19:47:10 +00:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines
2022-12-15 18:44:35 +01:00
Igor Nikonov
6dd2849ae6
Fix for distributed queries
...
+ 01952_optimize_distributed_group_by_sharding_key
2022-12-14 21:54:20 +00:00
avogar
a94a0d9c85
Fix tests, fix bugs
2022-12-14 21:17:00 +00:00
Igor Nikonov
b5de352861
Move checks to functions
2022-12-14 17:51:02 +00:00
Igor Nikonov
6ca0773516
Fix for distributed query
...
- do not apply optimization in case of UnionStep on top of Sorting
01104_distributed_one_test.sql
2022-12-14 16:10:52 +00:00
Dmitry Novik
1df038e39c
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-14 14:55:30 +00:00
Igor Nikonov
e25053dec0
Query plan visitor with debug logs
2022-12-13 22:46:39 +00:00
Igor Nikonov
1cbdce8eea
Fix window function case
2022-12-13 16:21:14 +00:00
Nikolay Degterinsky
9b6d31b95d
Merge branch 'master' into perf_experiment
2022-12-13 17:15:07 +01:00
Igor Nikonov
1bed5064e7
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-13 10:55:50 +00:00
Igor Nikonov
6f5fc6645b
Fix: remove ORDER BY before "mergeExpressions" optimization
...
When removing ORDER BY we remove corresponding expression,
so to remove only ORDER BY expression, do it before "mergeExpressions"
optimization is applied
2022-12-13 10:52:18 +00:00
Robert Schulze
8fc23262c5
Merge pull request #44125 from ClickHouse/minor-renaming
...
Minor file renaming
2022-12-13 08:35:26 +01:00
avogar
739ad23b1f
Make better, fix bugs, improve error messages
2022-12-12 22:00:45 +00:00
Dmitry Novik
3d2fccab87
Merge branch 'master' into refector-function-node
2022-12-12 21:36:39 +01:00
Igor Nikonov
69d1b39ef8
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-12 13:57:48 +00:00
Kruglov Pavel
d8d7385e68
Merge pull request #43161 from AlfVII/fix-race-condition-between-inserts-and-dropping-mvs
...
Fixed race condition between inserts and dropping MVs
2022-12-12 13:53:36 +01:00
Robert Schulze
8bcafe4544
Renaming PipelineResourcesHolder.h/cpp to QueryPlanResourceHolder.h/cpp
2022-12-10 18:08:10 +00:00
avogar
cd4fa00d2c
Merge branch 'master' of github.com:ClickHouse/ClickHouse into refactor-schema-inference
2022-12-09 14:45:10 +00:00
Maksim Kita
3a3c6eb458
Merge pull request #43341 from kitaisreal/analyzer-support-indexes
...
Analyzer added indexes support
2022-12-09 16:05:40 +03:00
Igor Nikonov
55da7a0d33
Polishing
2022-12-09 12:06:40 +00:00
Igor Nikonov
1658276c41
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-09 11:59:14 +00:00
Igor Nikonov
c2b10e8dba
Respect stateful functions in the optimization
2022-12-09 11:57:58 +00:00
Han Fei
6740ae9fd7
Merge pull request #43304 from hanfei1991/hanfei/aysnc-insert
...
Support dedup for async insert.
2022-12-09 12:50:48 +01:00
Alfonso Martinez
9654e06ec8
Fixed merge issue
2022-12-09 09:30:20 +01:00
avogar
c224e397ac
Check if delimiters are empty, add comments
2022-12-08 20:00:10 +00:00
avogar
1ec5f8451b
Merge branch 'master' of github.com:ClickHouse/ClickHouse into csv-custom-delimiter
2022-12-08 19:17:42 +00:00
Kruglov Pavel
de5ffc96e9
Fix style
2022-12-08 19:02:36 +01:00
avogar
556746692b
Fix build
2022-12-08 17:20:43 +00:00
Maksim Kita
4d39a7d103
Fixed code review issues
2022-12-08 17:30:32 +01:00
Maksim Kita
99de697ea2
Analyzer added indexes support
2022-12-08 17:30:32 +01:00
Yakov Olkhovskiy
0641066183
Merge branch 'master' into refactoring-ip-types
2022-12-08 11:12:05 -05:00
avogar
7375a7d429
Refactor and improve schema inference for text formats
2022-12-07 21:19:27 +00:00
Igor Nikonov
49694578ba
GROUP BY with check if aggr func depends on order
...
+ tests
2022-12-07 20:16:58 +00:00
Dmitry Novik
25ecb75ca8
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-07 18:36:50 +00:00
Igor Nikonov
5684453f09
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-07 11:44:43 +00:00
Alfonso Martinez
a6a37f21a4
Merge remote-tracking branch 'upstream/master' into fix-race-condition-between-inserts-and-dropping-mvs
2022-12-07 09:36:47 +01:00
Nikolai Kochetov
0ed82f3cc0
Merge branch 'master' into aggregating-in-order-from-query-plan
2022-12-06 16:36:49 +01:00
Nikolai Kochetov
9de53389ae
Review fixes.
2022-12-06 12:34:07 +00:00
Igor Nikonov
611de4d520
Fix + CROSS JOIN tests
2022-12-05 12:31:12 +00:00
Alfonso Martinez
da965c5705
Merge remote-tracking branch 'upstream/master' into fix-race-condition-between-inserts-and-dropping-mvs
2022-12-05 09:30:52 +01:00
Alfonso Martinez
e9e971ffad
Fixed style
2022-12-05 09:27:31 +01:00
Igor Nikonov
63df878e88
Fix type
2022-12-02 21:46:27 +00:00
Igor Nikonov
5a3abaad2a
Add GROUP BY w/o aggrecation functions analyze + test
2022-12-02 21:21:15 +00:00
Igor Nikonov
e59703abc6
Some tests + setting
2022-12-02 20:53:22 +00:00
Igor Nikonov
07f8ef1030
Something working
2022-12-02 19:27:45 +00:00
Igor Nikonov
47ca855816
save
2022-12-02 19:27:45 +00:00
Dmitry Novik
48e8aeac01
Clean up code
2022-12-02 19:15:26 +00:00
Dmitry Novik
2c70dbc76a
Refactor FunctionNode
2022-12-02 19:15:26 +00:00
Vitaly Baranov
e1f7f04752
Referential dependencies for RESTORE ( #43834 )
...
* Rename DDLDependencyVisitor -> DDLLoadingDependencyVisitor.
* Move building a loading graph to TablesLoader.
* Implement referential dependencies for tables and use them
when restoring tables from a backup.
* Remove StorageID::operator < (because of its inconsistency with ==).
* Add new tests.
* Fix test.
* Fix memory leak.
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2022-12-02 15:05:46 +01:00
Kruglov Pavel
c35b2a6495
Add a limit for string size in RowBinary format ( #43842 )
2022-12-02 13:57:11 +01:00
Alfonso Martinez
795a8b00fb
Added assert and comment for materialized_view->is_dropped
2022-12-02 09:52:10 +01:00
Alfonso Martinez
d6bd3ad691
Merge remote-tracking branch 'upstream/master' into fix-race-condition-between-inserts-and-dropping-mvs
2022-12-01 15:05:01 +01:00
Nikolai Kochetov
5edc9953b0
Fixing tests.
2022-12-01 12:21:21 +00:00
Alexander Tokmakov
431f6551cb
Merge branch 'master' into fix_assertion_in_thread_status
2022-11-30 23:05:15 +03:00
Nikolai Kochetov
7db145bece
Fix style.
2022-11-30 19:19:46 +00:00
Vladimir C
53dc70a2d0
Merge pull request #38191 from BigRedEye/grace_hash_join
...
Closes https://github.com/ClickHouse/ClickHouse/issues/11596
2022-11-30 17:01:00 +01:00
Nikolai Kochetov
43b3c10e9a
Add a separate optimisation to enable memory bound aggregation.
2022-11-30 14:31:39 +00:00
Anton Popov
fe5fff0347
Merge pull request #43329 from xiedeyantu/support_nested_column
...
s3 table function can support select nested column using {column_name}.{subcolumn_name}
2022-11-29 22:27:19 +01:00
vdimir
21f93e9f4c
Add IJoin::initialize
2022-11-29 17:11:00 +00:00
Nikolai Kochetov
51439e2c19
Merge pull request #43260 from ClickHouse/read-from-mt-in-io-pool
...
Read from MergeTree in I/O pool
2022-11-29 12:09:03 +01:00
Alexander Tokmakov
e45105bf44
detach threads from thread group
2022-11-28 21:31:55 +01:00
Nikolai Kochetov
500d54847e
Fixing style.
2022-11-28 19:11:05 +00:00
Han Fei
f23ee45d84
Merge branch 'master' into hanfei/aysnc-insert
2022-11-28 16:58:45 +01:00
Nikolai Kochetov
6d0646ed8f
Merge branch 'master' into aggregating-in-order-from-query-plan
2022-11-28 16:53:29 +01:00
Nikolai Kochetov
1dfa188c7a
Add order info for aggregating step in plan. Added test.
2022-11-28 15:15:36 +00:00
Yakov Olkhovskiy
770b520ded
Merge branch 'master' into refactoring-ip-types
2022-11-28 08:50:19 -05:00
Kruglov Pavel
dd7ac8bb96
Update src/Processors/Formats/Impl/CapnProtoRowOutputFormat.cpp
...
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2022-11-28 14:17:52 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order ( #40879 )
...
* impl
* fix style
* make executeQueryWithParallelReplicas similar to executeQuery
* impl for parallel replicas
* cleaner code for remote sorting properties
* update test
* fix
* handle when nodes of old versions participate
* small fixes
* temporary enable for testing
* fix after merge
* Revert "temporary enable for testing"
This reverts commit cce7f8884c
2022-11-28 00:41:31 +01:00
Nikolai Kochetov
4632e7c644
Add max_streams_for_merge_tree_reading setting.
2022-11-25 17:14:22 +00:00
Kruglov Pavel
2818ecf7f0
Merge pull request #43297 from arthurpassos/fix_arrow_list_column_parsing
...
Flatten list type arrow chunks on parsing
2022-11-25 18:13:27 +01:00
Han Fei
ee5be80d9f
Merge branch 'master' into hanfei/aysnc-insert
2022-11-25 17:27:56 +01:00
Anton Popov
c811f34a41
Simplify AsynchronousInsertQueue and fix race ( #43233 )
2022-11-25 15:02:22 +01:00
Nikolai Kochetov
660c942258
Merge branch 'master' into read-from-mt-in-io-pool
2022-11-25 13:56:48 +00:00
Igor Nikonov
236e7e3989
Small fixes
2022-11-25 12:04:12 +00:00
Igor Nikonov
20e67b7140
Merge remote-tracking branch 'origin/master' into HEAD
2022-11-24 13:10:37 +00:00
Raúl Marín
4d3217cb18
Merge branch 'master' into perf_experiment
2022-11-24 10:32:12 +01:00
Han Fei
28564df6f1
Merge branch 'master' into hanfei/aysnc-insert
2022-11-24 09:12:43 +01:00
Nikolai Kochetov
62debf42b4
Aggregating in order [in progress].
2022-11-23 18:38:12 +00:00
xiedeyantu
304b6ebf3a
s3 table function can support select nested column using {column_name}.{subcolumn_name}
2022-11-23 23:36:12 +08:00
Alexey Milovidov
c23b8c69db
Merge pull request #42791 from azat/fix-pe-overflow
...
Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting
2022-11-22 19:22:51 +01:00
Raúl Marín
4aa29b6a63
Merge remote-tracking branch 'blessed/master' into perf_experiment
2022-11-22 19:09:00 +01:00
Raúl Marín
e63ba06048
Better cache management
2022-11-22 19:03:17 +01:00
Kruglov Pavel
98d6b96c82
Merge pull request #42033 from mark-polokhov/BSONEachRow
...
Add BSONEachRow input/output format
2022-11-22 14:45:21 +01:00
Nikolai Kochetov
9e84a351c7
Fixing special build. Disable setting by default.
2022-11-22 11:24:24 +00:00
Nikolai Kochetov
aeb1de5b59
Aggregating in order [in progress].
2022-11-21 18:58:44 +00:00
avogar
ecdeff622b
Add small improvements in CapnProto format
2022-11-18 20:13:00 +00:00
Nikolai Kochetov
5da1d893fd
Merge branch 'master' into read-from-mt-in-io-pool
2022-11-18 21:10:45 +01:00
Azat Khuzhin
ccde92f855
Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting
...
After #40732 it became possible that getrusage() (from detachQuery(),
from buildPushingToViewsChain()) will be called for incorrect thread,
and so when the difference will be calculated it will be simply garbage.
But actually the root of this problem is #25714 , after which it became
possible to have multiple ThreadStatus for one thread, and this is very
tricky (sigh).
Here are some other thoughts about it:
- Make ThreadStatus nested - decided that complexity does not worth it,
at least only for this case
- Move some members into ThreadGroupStatus - will break per-thread
statistics (and hence query_thread_log, BTW does somebody uses it?)
- Move some members into a separate structure
But decided to fix the issue w/o any refactoring, to make easy for
backport.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-18 21:10:43 +01:00
Yakov Olkhovskiy
dbaeabcf38
fixed some bugs, some functions corrected, some tests corrected
2022-11-18 20:10:27 +00:00
Nikolai Kochetov
03c35e146b
Refactor a bit.
2022-11-18 20:09:20 +00:00
Han Fei
b80a2d6c89
address comments
2022-11-18 17:22:05 +01:00
Alfonso Martinez
4b4486e139
Improved stability of test
2022-11-18 12:18:01 +01:00
Arthur Passos
414fd07bba
add docs
2022-11-17 17:28:51 -03:00
Arthur Passos
dd37ca7767
add docs
2022-11-17 17:25:27 -03:00
Arthur Passos
12d3f799a5
small change
2022-11-17 17:18:54 -03:00
Arthur Passos
fcc032a31e
handle both zero based and non zero based arrow offsets
2022-11-17 17:15:24 -03:00
avogar
fcfdd73d17
Improve reading CSV field in CustomSeparated/Template format
2022-11-17 15:36:56 +00:00
Raúl Marín
80403015e7
Fix assert reached with lines without data
2022-11-17 16:19:53 +01:00
Alfonso Martinez
6a8101b8b6
Test redone on bash
2022-11-17 10:14:59 +01:00
Han Fei
14701670b5
Merge branch 'master' into hanfei/aysnc-insert
2022-11-16 17:24:36 +01:00
Arthur Passos
ed080b8ba5
fix style
2022-11-16 13:22:23 -03:00
Arthur Passos
e1236340b5
Flatten list type arrow chunks on parsing
2022-11-16 12:27:01 -03:00
Nikolai Kochetov
29098045d8
Try read from MergeTree in background pool.
2022-11-15 17:38:37 +00:00
avogar
2af60f34eb
Restrict document size in parallel parsing, allow to read ObjectId/JS code into String column
2022-11-15 13:35:17 +00:00
avogar
842d25c358
Minor improvements, better docs
2022-11-14 20:05:01 +00:00
Alfonso Martinez
e62ecf1a59
Merge remote-tracking branch 'upstream/master' into fix-race-condition-between-inserts-and-dropping-mvs
2022-11-14 18:23:31 +01:00
avogar
098dfcff56
Fix tests
2022-11-14 15:48:23 +00:00
Han Fei
11f93dec3f
tmp commit
2022-11-14 15:53:45 +01:00
Yakov Olkhovskiy
9aeebf3bdf
Merge branch 'master' into refactoring-ip-types
2022-11-14 09:21:54 -05:00
Yakov Olkhovskiy
4d144be39c
replace domain IP types (IPv4, IPv6) with native
2022-11-14 14:17:17 +00:00
Alexey Milovidov
127631ee47
Merge branch 'master' into perf_experiment
2022-11-12 18:58:25 +01:00
Nikolai Kochetov
9106910de8
Try read from MergeTree in background pool.
2022-11-11 20:47:11 +00:00
