thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-04 21:42:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
127,312 Commits 9,185 Branches 1,782 Tags 2.3 GiB
f748f12426
Commit Graph

232 Commits

Author SHA1 Message Date
Anton Popov
cc3ed12ed6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-02 02:56:00 +03:00
Alexey Milovidov
8b4c49d931 Fix test 2021-07-25 22:09:42 +03:00
Alexey Milovidov
650cbd2484 GraphiteMergeTree: respect timezone for rounding #5098 2021-07-25 04:27:40 +03:00
Anton Popov
cb042afa9d pushdown limit while reading in order of primary key 2021-07-13 17:24:45 +03:00
Anton Popov
fa8ad1a7c6 reduce memory usage in queries with 'ORDER BY primary_key' 2021-06-25 19:23:41 +03:00
Anton Popov
567043113c Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-21 01:36:06 +03:00
alexey-milovidov
496aff2118
Merge pull request #25300 from ClickHouse/simpla-agg-func-for-summing-mt 
Support simple aggregate functions for SummingMergeTree.
 2021-06-17 20:18:51 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Nikolai Kochetov
192fe1fc5b Add test. 2021-06-16 15:05:31 +03:00
Nikolai Kochetov
c8cbb44455 Support simple aggregate functions for SummingMergeTree. 2021-06-15 21:33:46 +03:00
Nikolai Kochetov
03b92dc1b7 Support simple aggregate functions for SummingMergeTree. 2021-06-15 21:32:27 +03:00
Alexey Milovidov
447d7bb8cd Minor changes 2021-06-14 07:13:35 +03:00
Alexey Milovidov
885ce194e0 Making fundamentals correct 2021-06-07 00:49:55 +03:00
Anton Popov
3e92c7f61a Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-25 21:45:19 +03:00
Alexey Milovidov
ea30bb7abf Improve implementation 2021-05-21 04:17:18 +03:00
Alexey Milovidov
1006a970f7 Remove AutoArray 2021-05-20 09:30:13 +03:00
Anton Popov
d8df0903b9 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-14 23:38:16 +03:00
Anton Popov
3548e31afd fix corner cases in vertical merges with ReplacingMergeTree 2021-04-22 01:27:18 +03:00
Anton Popov
298251e55d fix merges with sparse columns and disable sparse for some data types 2021-04-12 02:33:53 +03:00
Anton Popov
4b8e90338d ColumnSparse: fixes 2021-04-08 05:08:37 +03:00
Anton Popov
88cd775f6a Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-02 00:14:03 +03:00
Nikolai Kochetov
ed19864e5b
Update CollapsingSortedAlgorithm.cpp 
Add comment.
 2021-03-23 11:25:45 +03:00
Nikolai Kochetov
d7fc5e69f4
Update CollapsingSortedAlgorithm.h 
Added comment.
 2021-03-23 11:18:49 +03:00
Nikolai Kochetov
35ff8925df Fix crash. 2021-03-22 17:43:45 +03:00
Nikolai Kochetov
22444045f0 CollapsingSortedAlgorithm should not return more then index_granularity rows. 2021-03-22 16:18:14 +03:00
Anton Popov
6f7c8894e9 fix bugs in aggregation by primary key 2021-03-18 23:57:42 +03:00
Alexander Kuzmenkov
4b0cbb6ed7
Update FinishAggregatingInOrderAlgorithm.h 2021-03-09 13:43:06 +03:00
Anton Popov
87da7f0589
Update FinishAggregatingInOrderAlgorithm.h 2021-03-05 12:50:28 +03:00
Anton Popov
2e8b45afc1 fix ubsan report 2021-02-01 16:35:08 +03:00
Anton Popov
38e8bab6b1 fix tests 2021-01-27 03:44:36 +03:00
Anton Popov
666aab676e add comments to algorithm 2021-01-26 21:45:22 +03:00
Anton Popov
573edbcd11 improve performance of aggregation in order of sorting key 2021-01-22 05:34:08 +03:00
Alexey Milovidov
dab4719aac Remove some headers 2021-01-05 06:22:06 +03:00
Amos Bird
6a644b2af1
Fix SimpleAggregateFunction in SummingMergeTree 2 2021-01-01 12:42:22 +08:00
alexey-milovidov
a2e1f21ef2
Merge pull request #18637 from amosbird/summingsimple 
Fix SimpleAggregateFunction in SummingMergeTree
 2020-12-31 15:23:26 +03:00
Amos Bird
ae72f96111
Fix SimpleAggregateFunction in SummingMergeTree 2020-12-30 22:38:11 +08:00
Alexey Milovidov
be884a89f8 Minor fixes for min/sim hash 2020-12-29 13:16:43 +03:00
Pavel Kruglov
4b58528b9e Rename getPos to getRow, change mergeBlock, pass setting instead of context 2020-12-04 19:25:30 +03:00
Pavel Kruglov
905ba78adc Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-12-04 18:56:46 +03:00
Azat Khuzhin
9c801239cf
Better description of costly ArenaAllocChunks in AggregatingSortedAlgorithm 
Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
 2020-12-01 00:52:12 +03:00
Azat Khuzhin
35231662b3 Improve performance of AggregatingMergeTree w/ SimpleAggregateFunction(String) 
While reading from AggregatingMergeTree with
SimpleAggregateFunction(String) in primary key and
optimize_aggregation_in_order perf top shows:

    Samples: 1M of event 'cycles', 4000 Hz, Event count (approx.): 287759760270 lost: 0/0 drop: 0/0
      Children      Self  Shared Object         Symbol
    +   12.64%    11.39%  clickhouse            [.] memcpy
    +    9.08%     0.23%  [unknown]             [.] 0000000000000000
    +    8.45%     8.40%  clickhouse            [.] ProfileEvents::increment    # <-- this, and in debug it has not 0.08x overhead, but 5.8x overhead
    +    7.68%     7.67%  clickhouse            [.] LZ4_compress_fast_extState
    +    5.29%     5.22%  clickhouse            [.] DB::IAggregateFunctionHelper<DB::AggregateFunctionNullUnary<true, true> >::addFree

The reason is obvious, ProfileEvents is atomic counters (and also they
are nested):

<details>

```
    Samples: 7M of event 'cycles', 4000 Hz, Event count (approx.): 450726149337
    ProfileEvents::increment  /usr/bin/clickhouse [Percent: local period]
    Percent│
           │
           │
           │    Disassembly of section .text:
           │
           │    00000000078d8900 <ProfileEvents::increment(unsigned long, unsigned long)@@Base>:
           │    ProfileEvents::increment(unsigned long, unsigned long):
      0.17 │      push  %rbp
      0.00 │      mov   %rsi,%rbp
      0.04 │      push  %rbx
      0.20 │      mov   %rdi,%rbx
      0.17 │      sub   $0x8,%rsp
      0.26 │    → callq DB::CurrentThread::getProfileEvents
           │    ProfileEvents::Counters::increment(unsigned long, unsigned long):
      0.00 │      lea   0x0(,%rbx,8),%rdi
      0.05 │      nop
           │    unsigned long std::__1::__cxx_atomic_fetch_add<unsigned long, unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order):
      1.02 │      mov   (%rax),%rdx
     97.04 │      lock  add   %rbp,(%rdx,%rdi,1)
           │    ProfileEvents::Counters::increment(unsigned long, unsigned long):
      0.21 │      mov   0x10(%rax),%rax
      0.04 │      test  %rax,%rax
      0.00 │    → jne   78d8920 <ProfileEvents::increment(unsigned long, unsigned long)@@Base+0x20>
           │    ProfileEvents::increment(unsigned long, unsigned long):
      0.38 │      add   $0x8,%rsp
      0.00 │      pop   %rbx
      0.04 │      pop   %rbp
      0.38 │    ← retq
```

</details>

These ProfileEvents was ArenaAllocChunks (it shows ~1.5M events per
second), and the reason is that the table has
SimpleAggregateFunction(String) in PK, which requires Arena.
But most of the time there Arena wasn't even used, so avoid this cost by
re-creating Arena only if it was "used" (i.e. has new chunks).

Another possibility is to avoid populating Arena::head in ctor, but this
will make the Arena code more complex, so for now this was preferred.

Also as a long-term solution it worth looking at implementing them via
RCU (to move the extra overhead out from the write code path into read
side).
 2020-11-19 23:06:12 +03:00
Pavel Kruglov
547ec19fb3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-18 12:01:59 +03:00
Pavel Kruglov
6a57c0a8cf Move merge in MergeTreeDataWriter 2020-11-13 10:55:56 +03:00
Pavel Kruglov
8d5e0784d3 Add setting optimize_on_insert 2020-11-12 23:37:23 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Alexey Milovidov
17b3dff0c2 Whitespaces 2020-11-06 20:58:04 +03:00
alexey-milovidov
10e9d14466
Merge pull request #15818 from ClickHouse/style-pragma-once 
Check for #pragma once in headers
 2020-10-11 13:14:09 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Alexey Milovidov
5b482f4191 Cleanups 2020-10-10 19:31:10 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
Nikita Mikhaylov
d31ed58f01 done 2020-07-06 17:33:31 +03:00
Alexey Milovidov
1462a66d1e Fix typos 2020-06-27 22:05:00 +03:00
Anton Popov
14e09e5650 Merge remote-tracking branch 'upstream/master' into HEAD 2020-06-25 14:59:15 +03:00
Nikolai Kochetov
35ab9ad051 Merge branch 'master' into fix-parallel-final-stuck 2020-06-18 16:04:14 +03:00
Anton Popov
8ba5bd8530 Merge remote-tracking branch 'upstream/master' into distinct-combinator 2020-06-18 01:44:36 +03:00
Anton Popov
88b325dcdc rework distinct combinator 2020-06-17 22:36:27 +03:00
Nikolai Kochetov
b456a3cc77 Fix tests. 2020-06-15 20:48:04 +03:00
Nikolai Kochetov
5436ef38bf Fix MergingSortedAlgorithm. 2020-06-15 18:21:10 +03:00
Nikolai Kochetov
b5ecef6adf Fix tests. 2020-06-15 16:56:38 +03:00
Nikolai Kochetov
ccf2ceb876 Fix pipeline stuck for parallel final. 2020-06-15 14:02:47 +03:00
Alexey Milovidov
394fb64a9c Better way of implementation 2020-06-14 20:42:11 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956 Preparation for structured logging 2020-05-31 00:35:52 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
ce0619dabf Progress on task 2020-05-24 00:26:45 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
3384f3b77e Progress on task 2020-05-23 21:07:38 +03:00
alexey-milovidov
7efd2a825f
Merge pull request #10463 from ClickHouse/parallel-final 
Parallel final
 2020-05-18 05:20:18 +03:00
alexey-milovidov
d3c9f5cb65
Update RowRef.h 2020-05-17 13:31:06 +03:00
Nikolai Kochetov
4d0503f021 Make IColumn::mutate() static. 2020-05-14 11:30:18 +03:00
alexey-milovidov
40ef113503
Merge branch 'master' into parallel-final 2020-05-11 06:47:44 +03:00
Nikolai Kochetov
97dde2bcdf Fix limit for MergingSorted with fast-forward optimization. 2020-05-04 20:16:16 +03:00
Nikolai Kochetov
10afbc22b5 Fix build. 2020-04-30 16:36:20 +03:00
Nikolai Kochetov
62e9be7ba8 Fix build. 2020-04-30 14:33:27 +03:00
Nikolai Kochetov
19dadb8c2d Add parallel final. 2020-04-30 12:59:08 +03:00
Alexey Milovidov
03e17405cd Checkpoint 2020-04-22 10:03:43 +03:00
Nikolai Kochetov
15a8ac3245 Fix tests. 2020-04-14 22:31:49 +03:00
Nikolai Kochetov
7def7b021e Fix test with vesrioned collapsing. 2020-04-14 21:58:22 +03:00
Nikolai Kochetov
80c27689ff Add debug output. 2020-04-14 21:22:14 +03:00
Nikolai Kochetov
92692aff5e Fix build. 2020-04-14 21:03:29 +03:00
Nikolai Kochetov
2da9f889cd Fix unit test. 2020-04-14 14:09:23 +03:00
Nikolai Kochetov
1ce2e1b93b Move comments. 2020-04-14 12:43:53 +03:00
Nikolai Kochetov
fc42851186 Move comments. 2020-04-14 12:43:17 +03:00
Nikolai Kochetov
434a13a8bd Move algorithm to separate files. 2020-04-14 12:21:24 +03:00
Nikolai Kochetov
71feedea6f Remove some code. 2020-04-14 12:05:29 +03:00
Nikolai Kochetov
848026ac4c Fix style. 2020-04-14 11:48:38 +03:00
Nikolai Kochetov
2543741a9a Fix build. 2020-04-14 00:03:11 +03:00
Nikolai Kochetov
1ef6e8f7c7 Fix build. 2020-04-13 23:58:29 +03:00
Nikolai Kochetov
801320983b Refactor SummingSortedAlgorithm. 2020-04-13 23:50:13 +03:00
Nikolai Kochetov
84b1dd4c1e Fix tests. 2020-04-13 22:20:26 +03:00
Nikolai Kochetov
d7d29d8250 Fix build. 2020-04-13 22:02:57 +03:00
Nikolai Kochetov
586c295b94 Added SummingSortedAlgorithm 2020-04-13 21:51:17 +03:00
Nikolai Kochetov
89aef7aaf9 Add comments. 2020-04-13 18:04:54 +03:00
Nikolai Kochetov
2b81d68b56 Fix build. 2020-04-13 17:58:23 +03:00
Nikolai Kochetov
9acea16b7f Fix build. 2020-04-13 17:57:21 +03:00
Nikolai Kochetov
d3adcc1cb0 Fix build. 2020-04-13 17:53:04 +03:00
Nikolai Kochetov
377e16c00c Refactor AggregatingSortedAlgorithm. 2020-04-13 17:42:58 +03:00
Nikolai Kochetov
9ce0607de7 Remove header from AggregatingSortedAlgorithm. 2020-04-13 16:40:53 +03:00
Nikolai Kochetov
0544c1f25b Fix build. 2020-04-13 16:26:50 +03:00
Nikolai Kochetov
5efdf1203e Fix build. 2020-04-13 16:22:32 +03:00
Nikolai Kochetov
a2b5fd4b3d Fix build. 2020-04-13 16:18:46 +03:00
Nikolai Kochetov
a6bf7a14c8 Fix build. 2020-04-13 15:02:59 +03:00
Nikolai Kochetov
03dad5ccf5 Added AggregatingSortedAlgorithm. 2020-04-13 14:45:59 +03:00
Nikolai Kochetov
198f02d4cf Try simplify Replacing. 2020-04-10 20:22:15 +03:00
Nikolai Kochetov
ca8c78bfa2 Try simplify Replacing. 2020-04-10 20:19:44 +03:00
Nikolai Kochetov
44a3e216d1 Try simplify Replacing. 2020-04-10 20:16:38 +03:00
Nikolai Kochetov
ba840b5a59 Added VersionedCollapsingAlgorithm. 2020-04-10 19:44:47 +03:00
Nikolai Kochetov
2fa40ad554 Fix build. 2020-04-10 19:18:53 +03:00
Nikolai Kochetov
864e0a407f Added GraphiteRollupSortedAlgorithm. 2020-04-10 19:15:51 +03:00
Nikolai Kochetov
f35a8bf351 Fix style. 2020-04-10 18:41:22 +03:00
Nikolai Kochetov
994f95ddb0 Added ReplacingSortedAlgorithm. 2020-04-10 17:03:36 +03:00
Nikolai Kochetov
123c654db3 Fix build. 2020-04-10 16:41:00 +03:00
Nikolai Kochetov
51e5603044 Fix build. 2020-04-10 16:03:37 +03:00
Nikolai Kochetov
e9587b8f4c Fix build. 2020-04-10 15:53:51 +03:00
Nikolai Kochetov
d46c228b9c Fix build. 2020-04-10 15:47:30 +03:00
Nikolai Kochetov
7b85a36975 Added CollapsingSortedAlgorithm. 2020-04-10 15:40:16 +03:00
Nikolai Kochetov
2aba662d2c Fix build. 2020-04-10 12:32:34 +03:00
Nikolai Kochetov
97158f7656 Fix build. 2020-04-10 12:30:53 +03:00
Nikolai Kochetov
27f3d2711a Move Graphite params to separate file. 2020-04-10 12:20:43 +03:00
Nikolai Kochetov
868f7e9aea Move Graphite params to separate file. 2020-04-10 12:16:23 +03:00
Nikolai Kochetov
f7fdfe4ed2 Fix build. 2020-04-10 12:04:56 +03:00
Nikolai Kochetov
fc605d411d Added IMergingAlgorithm. 2020-04-10 12:04:28 +03:00
Nikolai Kochetov
2ce753a316 Fix tests. 2020-04-10 12:04:28 +03:00
Nikolai Kochetov
cbd1f6a524 Added comments. 2020-04-10 12:04:28 +03:00
Nikolai Kochetov
f96a5fe923 Fix build, 2020-04-10 12:04:50 +03:00
Nikolai Kochetov
71fab516f2 Fix AggregatingSorted for simple aggregate functions. 2020-04-10 12:04:34 +03:00
Nikolai Kochetov
fe442d8c9a Remove special case for simple input in IMergingTransform. 2020-04-10 12:04:28 +03:00
Nikolai Kochetov
8726ad9e3b Try fix collapsing. 2020-04-10 12:04:28 +03:00
Nikolai Kochetov
e23fda80aa Merge with master 2020-04-04 19:02:17 +03:00