Commit Graph

2650 Commits

Author SHA1 Message Date
avogar
04fdd75c56 Make JSONColumns frormats mono block by default 2022-05-09 11:13:44 +00:00
avogar
3a13c3e372 Fix comments 2022-05-06 16:50:34 +00:00
avogar
62a7ba3f26 Add columnar JSON formats 2022-05-06 16:48:48 +00:00
Vladimir C
bd5fab97d9
Merge pull request #36415 from bigo-sg/concurrent_join 2022-05-06 17:11:10 +02:00
Anton Popov
13e8db6299
Merge pull request #36762 from CurtizJ/dynamic-columns-12
Fix insertion to columns of type `Object` from multiple files
2022-05-06 14:14:32 +02:00
Alexander Tokmakov
e30365a531
Merge pull request #36872 from ClickHouse/fix_exception_message
Add extra info when sending exception
2022-05-04 23:24:47 +03:00
Kruglov Pavel
77e55c344c
Merge pull request #36667 from Avogar/mysqldump-format
Add MySQLDump input format
2022-05-04 19:49:48 +02:00
Robert Schulze
c48d9a4174
Merge pull request #36844 from rschu1ze/shared_ptr_helper3
Remove inherited create() method + disallow copying of storages
2022-05-04 19:16:30 +02:00
mergify[bot]
e0fa563092
Merge branch 'master' into fix_exception_message 2022-05-04 15:56:03 +00:00
Kruglov Pavel
ffec3655fe
Fix special build 2022-05-04 17:14:15 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
mergify[bot]
57fcca6dda
Merge branch 'master' into nth_value 2022-05-03 16:37:09 +00:00
Alexander Tokmakov
6dfaffdb7a fix test 2022-05-03 17:53:15 +02:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing
Parallel parsing of hive text format
2022-05-03 15:56:28 +02:00
Dmitry Novik
9be17ef50c
Merge pull request #35111 from azat/optimize_aggregation_in_order-prefix
Implement partial GROUP BY key for optimize_aggregation_in_order
2022-05-02 17:49:48 +02:00
Kruglov Pavel
d613f7eab0
Merge branch 'master' into mysqldump-format 2022-05-02 13:31:57 +02:00
Antonio Andelic
a1a22b0007
Merge pull request #35149 from ContentSquare/nullables_with_proto3
Nullables with proto3 using Google wrappers
2022-05-02 09:49:37 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Alexey Milovidov
b034146ba4
Merge pull request #36799 from azat/cleanup
Tiny cleanup
2022-05-01 14:23:13 +03:00
Robert Schulze
89aa9ae00f
Fixed clang-tidy check "bugprone-branch-clone"
The check is currently *not* part of .clang-tidy. It complains about:
(1) "switch has multiple consecutive identical branches"
(2) "repeated branch in conditional chain"

About (1): Lots of findings in switches were about redundant
"[[fallthrough]]" in places where the compiler would not warn anyways. I
have cleaned these up.

About (2): In if-else_if-else chains, fixing the warning would usually
mean concatenating multiple if-conditions. As this would reduce
readability in most cases, I did not fix these places.

Because of (2), I also refrained from adding "bugprone-branch-clone" to
.clang-tidy.
2022-04-30 19:40:28 +02:00
Azat Khuzhin
7a092e2a8c Remove unused AggregatedArenasChunkInfo
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 17:04:56 +03:00
Azat Khuzhin
b7b7d91bd1 Remove memory reservation for SquashingTransform
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 17:04:56 +03:00
Azat Khuzhin
8845fb0883 Fix outdated comment in buildPushingToViewsChain
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 17:04:56 +03:00
Jakub Kuklis
a1f2dd6d34 Adding two settings in place of one, improvements to the test clarity 2022-04-29 10:01:51 +02:00
Jakub Kuklis
507ba1042c Adding a setting to enable Google wrappers special treatment 2022-04-29 10:01:51 +02:00
Jakub Kuklis
6d5c1e2fc0 Adding a setting to enable special treatment of google wrappers 2022-04-29 10:01:50 +02:00
Azat Khuzhin
0ce44f3021 Optimize optimize_aggregation_in_order with a prefix key
Before it does lots of extra work, now, it will be significantly more
optimal (thousands of rows -> 1-2 million of rows).

v2: s/executeOnBlockSimple/executeOnBlockSmall/
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 06:58:28 +03:00
Azat Khuzhin
190ce217bb Disable GROUP BY statistics for optimize_aggregation_in_order
This statistics significantly decrease performance of
optimize_aggregation_in_order with a prefix key.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 06:58:27 +03:00
Azat Khuzhin
3931dbd848 Implement partial GROUP BY key for optimize_aggregation_in_order
Suppose you have a table with lots of rows, like:

    create table data_02233 (parent_key Int, child_key Int, value Int) engine=MergeTree() order by parent_key

And you want to do GROUP BY (parent_key, child_key) with optimize_aggregation_in_order:

    select parent_key, child_key, count() from data_02233 group by parent_key, child_key with totals order by parent_key, child_key

Right now, it is not possible, because optimize_aggregation_in_order
supports only w/o key aggregation, i.e. GROUP BY cannot be done inside
unique parent_key region.

v2: rebase on top SortDescriptionWithPositions
v3: disable two-level aggregation
v4: fix merging of aggregates
v5: improve tests coverage (add a test with multiple parts, to add merge processor)
v6: add a test for compiled aggregate functions (sum()) explicitly
v7: add missing sortBlock()
v8: remove group_by_description_optimized
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 06:58:07 +03:00
Azat Khuzhin
767acd53fb Add ability to pass range of rows to Aggregator
v2: fix compiled aggregate functions (seek result to row_start)
v3: fix compiled aggregate functions (seek args to row_start)
v4: change signatures for JIT
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 06:57:55 +03:00
Azat Khuzhin
599a255741 AggregatingInOrderTransform: correctly invalidate variants
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-29 06:56:32 +03:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Anton Popov
1fc51e09ff fix insertion to column of type Object from multiple files via table function 2022-04-28 18:51:13 +00:00
avogar
d295de1689 Fix comments and test 2022-04-28 14:59:35 +00:00
Kruglov Pavel
4d08587559
Merge branch 'master' into mysqldump-format 2022-04-28 15:58:18 +02:00
Kseniia Sumarokova
4c371f710e
Merge pull request #36676 from kssenii/refactor-with-size-buffer
Better version of SeekableReadBufferWithSize
2022-04-28 13:44:25 +02:00
Vladimir C
1cbdc1ef3a
Merge pull request #36206 from vdimir/output-format-prometheus 2022-04-28 12:09:53 +02:00
fenglv
1b84d59047 fix typo
modify comment
2022-04-27 12:24:49 +00:00
taiyang-li
0341880250 Merge remote-tracking branch 'origin/master' into hive_text_parallel_parsing 2022-04-27 11:15:16 +08:00
taiyang-li
99aa5fdc81 remove useless code 2022-04-27 11:15:04 +08:00
lgbo-ustc
5738871a8b update QueryPipelineBuilder::joinPipelines 2022-04-27 10:24:19 +08:00
lgbo-ustc
520b05b9f1 update test case tests/queries/0_stateless/02236_explain_pipeline_join.sql 2022-04-27 10:08:22 +08:00
vdimir
81b86799e7
Fixup PrometheusTextOutputFormat 2022-04-26 14:57:37 +00:00
vdimir
d5d98ed951
PrometheusTextOutputFormat: support lables, histograms and summaries 2022-04-26 14:57:36 +00:00
vdimir
be0aa06958
Add output format Prometheus 2022-04-26 14:57:35 +00:00
Alexander Gololobov
3c000b098a
Merge pull request #36638 from nickitat/fix_sorting_step
Fix SortingStep::updateOutputStream()
2022-04-26 15:49:49 +02:00
kssenii
9d364cdce2 Refactor 2022-04-26 15:33:53 +02:00
Kruglov Pavel
a462d94157
Fix error codes 2022-04-26 13:25:07 +02:00
Kruglov Pavel
e3b222b519
Fix typo 2022-04-26 13:24:10 +02:00
lgbo-ustc
6cb7b7888f update test case 02236_explain_pipeline_join 2022-04-26 19:07:07 +08:00