thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
81,343 Commits 9,326 Branches 1,783 Tags 2.1 GiB
f7e315bb0a
Commit Graph

6662 Commits

Author SHA1 Message Date
kssenii
26f6a697b5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql-fix 2021-11-04 13:25:48 +03:00
Vasily Nemkov
67852e9134 Implemented CREATE [MATERIALIZED|LIVE] VIEW ... COMMENT 2021-11-03 22:32:29 +02:00
Anton Popov
f37d12d16e keep serialization infos after drops and renames 2021-11-03 23:29:48 +03:00
Alexander Tokmakov
d4677dc6d8 fix intersecting parts in parts_to_do 2 2021-11-03 21:48:17 +03:00
avogar
2dd4393ca1 Small refactoring in formats 2021-11-03 20:07:05 +03:00
kssenii
35a4be601a Select query commit only with settings 2021-11-03 15:43:58 +03:00
kssenii
ec11179f91 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-03 14:33:31 +03:00
kssenii
45ea820297 Reduce memory usage for some formats 2021-11-03 14:30:03 +03:00
kssenii
38466cb76f Storage merge fix aliases with where 2021-11-03 14:17:11 +03:00
Kruglov Pavel
77f5ef476a
Merge branch 'master' into hdfs-bad-url 2021-11-03 14:08:55 +03:00
avogar
3839d521a0 Add additional hdfs url check 2021-11-03 14:01:50 +03:00
Nikita Mikhaylov
59c8ed9b0c
Fixed is_cancelled predicate inside MergeTask (#30996) 2021-11-03 13:25:45 +03:00
Vitaly Baranov
1f217aeb5a
Merge pull request #30998 from vitlibar/rename-access-control-manager 
Rename AccessControlManager
 2021-11-03 12:20:17 +03:00
kssenii
285742dcba Fix tests 2021-11-03 11:57:33 +03:00
Amos Bird
710f7d64e4
Add retry to check alter_partition_version 2021-11-03 15:48:16 +08:00
Vitaliy Zakaznikov
db4a914b07 Reverting change that breaks live view tests. 2021-11-02 20:20:48 -04:00
Anton Popov
46fa062a81 fix tests 2021-11-02 23:30:28 +03:00
Kseniia Sumarokova
065106c299
Merge pull request #30690 from kssenii/partitioned-write 
Partitioned write for file, hdfs, url storages and table functions
 2021-11-02 22:58:16 +03:00
kssenii
b899dd9af5 Better 2021-11-02 22:47:27 +03:00
Anton Popov
e1ab0ad383 fix tests 2021-11-02 20:46:03 +03:00
Vitaliy Zakaznikov
239432009b Fixing query performance issue in StorageLiveView.cpp by moving to PullingAsyncPipelineExecutor. 2021-11-02 11:36:52 -04:00
kssenii
f662c1829d Fix 2021-11-02 18:08:10 +03:00
Alexander Tokmakov
56aeff734a Merge branch 'master' into check_dependencies_on_drop 2021-11-02 16:15:30 +03:00
Vitaly Baranov
afe2c9c040 Rename AccessControlManager -> AccessControl. 2021-11-02 14:06:20 +03:00
Kruglov Pavel
901ebcede6
Merge pull request #30351 from arenadata/ADQM-335 
output_format_avro_rows_in_file
 2021-11-02 12:25:27 +03:00
Vitaly Baranov
a0c38a3dfd Rename AuthenticationType::MAX_TYPE. 2021-11-02 12:02:22 +03:00
kssenii
9080d2ea52 Fix tests 2021-11-02 07:27:13 +00:00
alesapin
085cb275de
Merge pull request #30826 from azat/replicated-mt-fix-sanity-on-failed-drop 
Fix possible "The local set of parts of X doesn't look like the set of parts in ZooKeeper" error
 2021-11-02 10:15:43 +03:00
Vitaly Baranov
acf4e66771
Merge pull request #30973 from vitlibar/move-access-code 
Move access-rights source code
 2021-11-02 09:57:15 +03:00
Anton Popov
1628f50e51
Merge branch 'master' into sparse-serialization 2021-11-02 06:26:18 +03:00
Anton Popov
9823f28855 fix nested 2021-11-02 06:03:52 +03:00
robot-clickhouse
4289336c27 Auto version update to [21.12.1.1] [54457] 2021-11-02 00:56:45 +03:00
alexey-milovidov
af4a0a9ed2
Merge pull request #29528 from DimasKovas/allow_nulls_first_in_key_condition 
Support NULLS FIRST in KeyCondition
 2021-11-01 23:02:22 +03:00
alexey-milovidov
38f7d1ebc2
Update KeyCondition.cpp 2021-11-01 22:59:57 +03:00
Alexander Tokmakov
2b566c5bdf basic check for dependencies on drop table 2021-11-01 22:20:18 +03:00
Vitaly Baranov
3bc9bf28f4
Merge pull request #30688 from vitlibar/backup-and-restore-for-log-family 
Implement BACKUP & RESTORE for Log family
 2021-11-01 19:57:39 +03:00
Vitaly Baranov
ab01b9afc8 Split Authentication.h to common and main parts. 2021-11-01 19:13:49 +03:00
Vitaly Baranov
3ed7f8f0b3 Move access-rights' source files needed for parser to a separate target. 2021-11-01 19:13:49 +03:00
Anton Popov
d50137013c Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-01 16:55:53 +03:00
Anton Popov
a2b98f253e fix nested 2021-11-01 16:52:13 +03:00
Vitaly Baranov
a1f29d31ea Support BACKUP & RESTORE for log family. 2021-11-01 12:07:17 +03:00
Vladimir C
2e58a3bf62
Merge branch 'master' into MDB-15168 2021-11-01 11:59:13 +03:00
Vitaly Baranov
8fbd6e3d46
Merge pull request #30941 from vitlibar/fix-flaky-read-tiny-log 
Fix reading from TinyLog
 2021-11-01 11:44:06 +03:00
Azat Khuzhin
60a411581f Fix possible "The local set of parts of X doesn't look like the set of parts in ZooKeeper" error 
If during removing replica_path from zookeeper, some error occurred
(zookeeper goes away), then it may not remove everything from zookeeper.

And on DETACH/ATTACH (or server restart, like stress tests does in the
analysis from this comment [1]), it will trigger an error:

    The local set of parts of table test_1.alter_table_4 doesn't look like the set of parts in ZooKeeper:

  [1]: https://github.com/ClickHouse/ClickHouse/pull/28296#issuecomment-915829943

Fix this, by removing "metadata" at first, and only after this
everything else, this will avoid this error, since on ATTACH such table
will be marked as read-only.

v2: forget to remove remote_replica_path itself
v3: fix test_drop_replica by adding a check for remote_replica_path existence
 2021-11-01 08:59:28 +03:00
Azat Khuzhin
ca22e7acfb Use existing local remote_replica_path var in StorageReplicatedMergeTree::dropReplica() 2021-11-01 08:53:31 +03:00
Anton Popov
8fe3de16c6 fix nested 2021-11-01 05:40:43 +03:00
Anton Popov
c66207f606 fix nested 2021-11-01 05:13:07 +03:00
Vitaly Baranov
d2363d625c Fix reading from TinyLog. 2021-11-01 04:07:43 +03:00
alexey-milovidov
f652c8ce24
Merge pull request #30852 from ucasfl/modify-table-comment 
enable modify table comment of rest table engine
 2021-10-31 22:12:07 +03:00
alexey-milovidov
ac38f8f459
Merge pull request #30903 from ClickHouse/reimplement-28639 
Reimplement #28639
 2021-10-31 22:10:58 +03:00
Kseniia Sumarokova
ca48c5f302
Merge pull request #30832 from ucasfl/filelog-fix 
filelog engine tests improve
 2021-10-31 21:27:48 +03:00
Vitaly Baranov
d29b73e301
Merge pull request #30689 from vitlibar/refactor-log-family 
Refactoring of Log family
 2021-10-31 18:50:08 +03:00
kssenii
2940d9fd19 Squashed commit of the following: 
commit a1acc7ed3485bd158cf42f26b2d2a1bca84b7269
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Sun Oct 31 14:47:25 2021 +0000

    Fix

commit 12a27d445b
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 29 09:24:53 2021 +0000

    Adjutments after merge with master

commit f17e321a73
Merge: 55f1ba857d 11b70a285c
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 29 08:03:25 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 55f1ba857d
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 29 07:38:42 2021 +0000

    Small fixes

commit aeba8104d1
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Thu Oct 28 10:28:05 2021 +0000

    Some more fixes

commit 82f3754b8a
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Wed Oct 27 20:20:15 2021 +0000

    Slightly better

commit 2647b88a66
Merge: 7024f51b55 2bb586bed3
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Wed Oct 27 19:38:19 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 7024f51b55
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Wed Oct 27 18:30:25 2021 +0000

    Review fixes and add test

commit 7de5fca075
Merge: 1ecb6ad05d 855b10261c
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Sun Oct 24 17:04:24 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 1ecb6ad05d
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Sat Oct 23 18:53:59 2021 +0000

    Better

commit 710ee96878
Merge: 28d2485716 158b4c26b7
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Mon Oct 18 08:31:25 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 28d2485716
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 15 17:27:26 2021 +0000

    Some fixes

commit 7c20ca07b7
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 15 12:59:24 2021 +0000

    Fix style check

commit ccb02cac56
Merge: 9e4fe0f3c8 d1138a8a25
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 15 12:36:30 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 9e4fe0f3c8
Merge: 9814cb1b45 daed77038b
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 09:58:15 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 9814cb1b45
Merge: 1c8b1b1133 969999ff10
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 09:49:08 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 1c8b1b1133
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 09:33:07 2021 +0000

    Better

commit e7dbfff5d2
Merge: 3329b668d6 198adc7ecd
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 06:19:15 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
 2021-10-31 17:53:08 +03:00
alexey-milovidov
1285d3bd8c
Merge pull request #30877 from amosbird/projection-fix22 
Fix invalid header in projection block calculation
 2021-10-31 16:30:02 +03:00
mergify[bot]
3cb034880b
Merge branch 'master' into allow_nulls_first_in_key_condition 2021-10-31 12:19:08 +00:00
Alexey Milovidov
c38540786e Reimplement #28639 2021-10-31 15:11:00 +03:00
alexey-milovidov
34009ea4a3
Merge pull request #30310 from amosbird/projection-fix20 
Fix IN set filtering when projection is used.
 2021-10-31 15:00:45 +03:00
Amos Bird
b3a8ad124f
Fix index analysis for set index 2021-10-31 12:35:20 +08:00
Vitaly Baranov
0e8c9b089f Keep indices for StorageStripeLog in memory. 2021-10-31 03:52:41 +03:00
Vitaly Baranov
42596b16bc Make StorageLog.cpp to implement both Log and TinyLog engines. 
Refactoring of implementation of the Log family.
 2021-10-31 03:52:36 +03:00
Amos Bird
d647a2db0c
Fix invalid header in projection block calculation 2021-10-30 17:26:43 +08:00
Kseniia Sumarokova
6035e0498b
Merge pull request #30786 from kssenii/fix-filelog-test 
Fix test 02022
 2021-10-29 22:06:28 +03:00
Anton Popov
0099dfd523 refactoring of SerializationInfo 2021-10-29 20:21:02 +03:00
feng lv
6f12348282 enable modify table comment of some table 2021-10-29 12:31:18 +00:00
Amos Bird
211c3d4d49
Better Projection IN 2021-10-29 20:24:36 +08:00
kssenii
12a27d445b Adjutments after merge with master 2021-10-29 09:29:35 +00:00
tavplubix
f50b2b651b
Merge pull request #30822 from ClickHouse/fix_29052 
Fix ambiguity when extracting auxiliary ZooKeeper name
 2021-10-29 12:15:29 +03:00
kssenii
f17e321a73 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-29 08:03:25 +00:00
Kseniia Sumarokova
1cc1417ba4
Update StorageFileLog.cpp 2021-10-29 09:53:48 +03:00
feng lv
13a305d648 filelog engine tests improve 
fix
 2021-10-29 06:25:21 +00:00
kssenii
03d7476864 One more fix 2021-10-29 00:50:59 +03:00
tavplubix
bb4a8e91f7
Create StorageReplicatedMergeTree.cpp 2021-10-28 20:25:31 +03:00
Alexander Tokmakov
c1310841d9 fix ambiguity when extracting auxiliary zk name 2021-10-28 19:19:41 +03:00
kssenii
7e2ea97e3c Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-28 18:46:21 +03:00
Anton Ivashkin
6cda420b56 Cleanup after fail in localBackup 2021-10-28 17:50:59 +03:00
alexey-milovidov
69bc2a8d2a
Merge pull request #30780 from ClickHouse/remove-cruft 
Remove cruft
 2021-10-28 17:42:18 +03:00
kssenii
d39840376f Review fixes 2021-10-28 15:44:12 +03:00
kssenii
aeba8104d1 Some more fixes 2021-10-28 10:28:05 +00:00
tavplubix
33ffe11d03
Merge pull request #30346 from ClickHouse/fix_some_fetches_may_stuck 
Try fix 'some fetches may stuck'
 2021-10-28 13:10:15 +03:00
kssenii
3d01fbf603 Review fixes 2021-10-28 11:32:37 +03:00
kssenii
37cd91c0bf Fix 2021-10-28 07:59:57 +00:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
alexey-milovidov
fab8da1c64
Merge pull request #30722 from ucasfl/filelog-engine 
remove some unneeded header files
 2021-10-27 23:35:03 +03:00
kssenii
9461e225bb Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-27 22:47:47 +03:00
kssenii
2647b88a66 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-27 19:38:19 +00:00
kssenii
7024f51b55 Review fixes and add test 2021-10-27 19:37:36 +00:00
Azat Khuzhin
a1926b3a55 Fix MergeTreeReaderCompact with empty buffer (max_read_buffer_size=0) 
CI: https://clickhouse-test-reports.s3.yandex.net/30190/0682d80f603d934eda51c93959164ee29eb52c02/functional_stateless_tests_(debug).html#fail1
 2021-10-27 22:25:03 +03:00
tavplubix
ac4a9bcf23
Fix 00975_move_partition_merge_tree (#30717) 
* Revert "Fix race between MOVE PARTITION and merges/mutations for MergeTree"

This reverts commit e9ce859b02.

* fix

* Update StorageMergeTree.cpp

Co-authored-by: alesapin <alesapin@gmail.com>
 2021-10-27 18:19:41 +03:00
Alexander Tokmakov
5e622d4c98 fix another issue 2021-10-27 15:59:26 +03:00
alesapin
2af950d4d5
Merge pull request #30658 from ClickHouse/std_mutex_alter_lock 
Change `alter_lock` from `RWLock` to `std::timed_mutex`
 2021-10-27 11:14:11 +03:00
alesapin
a29711f1d0
Merge pull request #30651 from ClickHouse/fix_intersecting_parts_to_do 
Fix intersecting parts in `parts_to_do`
 2021-10-27 10:52:21 +03:00
alesapin
8749f4a31a
Merge pull request #29043 from nvartolomei/nv/parts-uuid-move-shard-kill 
Part movements between shards improvements and cancel support
 2021-10-27 10:46:00 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
feng lv
4ac120b883 remove some unneeded header files 2021-10-27 04:13:12 +00:00
Dmitry Novik
e971871a95
Merge pull request #30668 from azat/prewhere-always-true-where 
Fix PREWHERE with WHERE in case of always true PREWHERE
 2021-10-26 17:18:50 +03:00
Alexander Tokmakov
6be1e98342 fix 2021-10-26 17:06:48 +03:00
mergify[bot]
ce88a84e88
Merge branch 'master' into nv/parts-uuid-move-shard-kill 2021-10-26 11:09:19 +00:00
alesapin
a679ece0fb Merge branch 'master' into fix_intersecting_parts_to_do 2021-10-26 14:02:05 +03:00
kssenii
2ba3ee830e Partitioned write part 2 2021-10-26 14:00:41 +03:00
Alexander Tokmakov
2e7e195e77 change alter_lock to std::timed_mutex 2021-10-26 13:37:00 +03:00
alexey-milovidov
e528bfdb1a
Merge pull request #30565 from ClickHouse/try-to-fix-30397 
Try to fix #30397
 2021-10-26 09:45:45 +03:00
Azat Khuzhin
16cd70aee9 Fix PREWHERE with WHERE in case of always true PREWHERE 2021-10-26 00:02:41 +03:00
mergify[bot]
70097b5c4e
Merge branch 'master' into fix_intersecting_parts_to_do 2021-10-25 17:21:25 +00:00
kssenii
1d743b9259 Partitoned write 2021-10-25 19:23:44 +03:00
Alexander Tokmakov
5cdd9a4b99 fix intersecting parts in parts_to_do 2021-10-25 19:00:55 +03:00
Alexander Tokmakov
94088f8197 minor fixes 2021-10-25 17:01:36 +03:00
Maksim Kita
772c326f2d
Merge pull request #30500 from kitaisreal/dictionaries-read-multiple-threads 
Dictionaries read support multiple threads
 2021-10-25 15:31:50 +03:00
Alexander Tokmakov
075c3a37f4 Merge branch 'master' into fix_some_fetches_may_stuck 2021-10-25 15:28:54 +03:00
alesapin
28bde879ed
Merge pull request #30455 from kssenii/deprecate-setting 
Deprecate write_final_mark setting
 2021-10-25 10:51:49 +03:00
kssenii
52296e8b04 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-25 00:22:32 +03:00
alexey-milovidov
7200cb8bba
Merge pull request #30611 from azat/readbuffer-real-size 
Replace estimated_size with optional<size> for readFile/createReadBufferFromFileBase
 2021-10-24 23:37:30 +03:00
kssenii
13a2a3d307 Merge branch 'master' of github.com:ClickHouse/ClickHouse into deprecate-setting 2021-10-24 23:05:21 +03:00
kssenii
658b33717f A little nicer 2021-10-24 20:43:34 +03:00
kssenii
7de5fca075 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-24 17:04:24 +00:00
Kseniia Sumarokova
4667ea7883
Merge pull request #29774 from kssenii/remote-connection 
Predefined configuration for table function remote
 2021-10-24 19:38:01 +03:00
Azat Khuzhin
f9b27608f5 Fix reading of columns if offset_in_compressed_file is the same for mark range 
Fixes: 00078_group_by_arrays
 2021-10-24 18:58:22 +03:00
Azat Khuzhin
93c34824c0 Fix explicit 0 size for createReadBufferFromFileBase() 2021-10-24 18:58:22 +03:00
Azat Khuzhin
4599e9191b Fix explicit 0 size for readFile() 2021-10-24 18:58:22 +03:00
Kseniia Sumarokova
7383bdd5fa
Merge pull request #25969 from ucasfl/filelog-engine 
Implementation of a table engine to consume application log files in ClickHouse
 2021-10-24 10:14:05 +03:00
kssenii
1ecb6ad05d Better 2021-10-23 19:00:14 +00:00
Kseniia Sumarokova
00d0665bf2
Merge pull request #30477 from kssenii/postgres-fix-query 
More reliable fetch query in postgres database
 2021-10-23 19:07:34 +03:00
kssenii
e04e37916e Review fixes 2021-10-23 18:20:31 +03:00
kssenii
36d0f67b38 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remote-connection 2021-10-23 17:12:46 +03:00
feng lv
235339d530 fix 2021-10-23 09:35:33 +00:00
kssenii
1fa123ee5c Properly done 2021-10-23 01:33:17 +03:00
Nikolai Kochetov
ee6bc3883f Try to fix 30397 2021-10-22 19:25:48 +03:00
Nikolai Kochetov
e00d55c08d Try to fix 30397 2021-10-22 17:42:20 +03:00
Kseniia Sumarokova
153dd9a895
Update MergeTreeSettings.h 2021-10-22 16:27:01 +03:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
Filatenkov Artur
1c72421e53
Add columns in columns.sql for INFORMATION_SCHEMA (#29637) 
* add columns in columns.sql for INFORMATION_SCHEMA
 2021-10-22 15:18:16 +03:00
Anton Popov
151daf61c1
Merge pull request #30522 from azat/remove-check_columns 
Remove check_columns argument from MergeTree code (false was never passed)
 2021-10-22 14:06:49 +03:00
kssenii
c95a29edac Fix quoting 2021-10-22 10:07:23 +00:00
feng lv
54dc9693fb fix 2021-10-22 07:08:08 +00:00
feng lv
5cb783aed0 fix 2021-10-22 02:36:24 +00:00
Maksim Kita
adaa14f8aa
Merge pull request #30502 from kitaisreal/storage-dictionary-fix-potential-configuration-race 
StorageDictionary fix potential configuration race
 2021-10-22 00:14:11 +03:00
Azat Khuzhin
496aa43a9f Remove check_columns argument from MergeTree code (false was never passed) 2021-10-21 21:24:59 +03:00
kssenii
56f0560c46 Asserts and read till end option 2021-10-21 20:54:03 +03:00
Vitaly Baranov
5695ef55b3
Merge pull request #28125 from vitlibar/fix-writing-storage-log-marks 
Fix writing marks in StorageLog
 2021-10-21 20:10:01 +03:00
Dmitry Novik
58ee783be4
Merge pull request #30293 from amosbird/projection-fix19 
Fix projections with JOIN alias columns
 2021-10-21 17:49:21 +03:00
Maksim Kita
8885748a6c StorageDictionary fix potential configuration race 2021-10-21 17:25:53 +03:00
Maksim Kita
a3a780bbf5 Dictionaries read support multiple threads 2021-10-21 17:17:53 +03:00
Anton Popov
b1a74af44d
Merge pull request #30180 from CurtizJ/remove-sample-by 
Allow to remove `SAMPLE BY` expression
 2021-10-21 17:09:53 +03:00
tavplubix
4c5104b30d
Merge pull request #30432 from ClickHouse/tavplubix-patch-3 
Fix segfault on REPLACE PARTITION if session expired
 2021-10-21 17:08:57 +03:00
Ilya Golshtein
82f33151e7 output_format_avro_rows_in_file fixes per code review 2021-10-21 02:53:39 +03:00
kssenii
e4e157688d Review fixes 2021-10-21 01:57:43 +03:00
Nikita Mikhaylov
56869c0ec4
Merge pull request #30360 from nikitamikhaylov/allow-graphite-final 
Allow to use FINAL clause in SELECT from GraphiteMergeTree
 2021-10-20 19:57:44 +03:00
feng lv
6565e1d4b2 fix conflict 2021-10-20 15:22:16 +00:00
kssenii
0a90be8317 Remove write_final_mark setting 2021-10-20 14:13:04 +00:00
Amos Bird
2f615e9176
Use original_query for projection analysis 2021-10-20 21:30:15 +08:00
Nikolai Kochetov
8986de7e50
Merge pull request #30384 from amosbird/projection-fix21 
Fix crash when minmax_count projection is used with primary key in partition expr
 2021-10-20 16:17:13 +03:00
Kseniia Sumarokova
c692155c7e
Merge pull request #30309 from kssenii/fix-file-symlinks 
Fix symlinks in file table function
 2021-10-20 11:09:51 +03:00
tavplubix
5e24f337a0
Update StorageReplicatedMergeTree.cpp 2021-10-20 09:29:41 +03:00
Kseniia Sumarokova
5324cc8359
Merge pull request #30282 from kssenii/fix-local-less-threads 
Less threads in clickhouse-local, fix Ok. printing
 2021-10-19 22:54:28 +03:00
Anton Popov
f1a5f79849 fix sparse serialization 2021-10-19 19:54:20 +03:00
Amos Bird
05d93796dc
Fix minmax_count projection with primary key in partition expr 2021-10-19 23:36:07 +08:00
Maksim Kita
382c82805f
Merge pull request #30352 from kitaisreal/storage-executable-fix-small-issues 
StorageExecutable fix small issues
 2021-10-19 17:43:11 +03:00
Maksim Kita
0f2e23d775
Update StorageExecutable.cpp 2021-10-19 17:41:59 +03:00
feng lv
05fd6f7caf fix comflict 2021-10-19 12:50:22 +00:00
Nikita Mikhaylov
75487be899 White list of storages that supports final 2021-10-19 12:16:33 +00:00
Alexander Tokmakov
caf972db48 fix 2021-10-19 14:56:04 +03:00
Nikita Mikhaylov
4fbd332bf1 Done 2021-10-19 11:53:55 +00:00
Nikita Mikhaylov
ea2a647724
Merge pull request #30192 from azat/granula-adjust-LOGICAL_ERROR-fix 
Fix reading from MergeTree with max_read_buffer_size=0
 2021-10-19 13:25:55 +03:00
kssenii
e53335bc6f Better way 2021-10-19 08:21:38 +00:00
Nikolai Kochetov
cc7a3ea9ff
Merge pull request #30322 from ClickHouse/update-mergine-sorted-memory-optimization 
Update memory optimisation for MergingSorted.
 2021-10-19 11:12:50 +03:00
kssenii
df5dc4e991 Adjust MergeTreeIndexReader 2021-10-19 10:51:45 +03:00
kssenii
49106f407f Fix some checks 2021-10-19 09:34:27 +03:00
Maksim Kita
fb91b1d92b StorageExecutable fix small issues 2021-10-19 01:04:07 +03:00
Alexander Tokmakov
89fe606d15 try fix 'some fetches may stuck' 2021-10-18 23:16:02 +03:00
Nikita Mikhaylov
d004fd96ed
Merge pull request #30318 from azat/fix-shared-build 
Move SquashingTransform to Interpreters (to fix split build)
 2021-10-18 22:41:01 +03:00
Azat Khuzhin
e576fd17bd Do not allow reading to empty buffer in MergeTreeReaderStream 
Changelog:
- Verify that all rows was read in MergeTreeReaderWide
- Ignore some exceptions for Wide parts
- Take max_rows_to_read into account
- Verify that requested rows is not zero (otherwise it is too tricky)
- Simply verify that buffer is not empty
 2021-10-18 21:19:34 +03:00
Azat Khuzhin
cd4b33c8c9 Verify that all rows was read in MergeTreeReaderCompact 
v0: Use fmt-like style exception in MergeTreeReaderCompact
v2: Update the check
 2021-10-18 21:19:34 +03:00
Azat Khuzhin
41e6df0f4b Remove unused offset_columns from MergeTreeReaderWide::readRows() 2021-10-18 21:19:34 +03:00
Ilya Golshtein
d90302aa3b output_format_avro_rows_in_file 2021-10-18 19:01:06 +03:00
feng lv
ab0d2504e5 fix 2021-10-18 15:44:54 +00:00
kssenii
d5d4817350 Add disk proxies back, add setReadUntilPosition to ReadBuffer 2021-10-18 18:35:11 +03:00
Nikolay Degterinsky
d76976e272 Merge branch 'master' into query_parameters 2021-10-18 17:34:20 +03:00
Anton Popov
d71ffc355a Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-18 15:18:22 +03:00
Nikolai Kochetov
6e479b301a Update memory optimisation for MergingSorted. 2021-10-18 12:54:12 +03:00
Anton Popov
71b474793c
Merge pull request #30174 from CurtizJ/better-interfaces 
Better interfaces for `IDataType` and `ISerialization`
 2021-10-18 12:07:04 +03:00
kssenii
710ee96878 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-18 08:31:25 +00:00
feng lv
81fb4bcf76 fix style 2021-10-18 08:14:54 +00:00
kssenii
3d39ea8e37 Fix WHERE 2021-10-18 11:06:30 +03:00
feng lv
0c628726af fix build 2021-10-18 07:48:58 +00:00
feng lv
1e3df5780c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-18 07:33:11 +00:00
Azat Khuzhin
6d4af3bac1 Move SquashingTransform to Interpreters (to fix split build) 
clickhouse_common_io requires clickhouse_core:

    ld.lld: error: undefined symbol: DB::blocksHaveEqualStructure(DB::Block const&, DB::Block const&)
    >>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
    >>>               src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block&&>(DB::Block&&))
    >>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
    >>>               src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block const&>(DB::Block const&))

while clickhouse_core requires clickhouse_common_io:

    "clickhouse_core" of type SHARED_LIBRARY
      depends on "roaring" (weak)
      depends on "clickhouse_common_io" (weak)
      depends on "clickhouse_common_config" (weak)
      depends on "clickhouse_common_zookeeper" (weak)
      depends on "clickhouse_dictionaries_embedded" (weak)
      depends on "clickhouse_parsers" (weak)

Follow-up for: #30247 (cc @KochetovNicolai)
 2021-10-18 10:28:36 +03:00
feng lv
6cf82f859c fix 2021-10-18 07:27:37 +00:00
feng lv
a32abda322 fix 2021-10-18 07:08:37 +00:00
feng lv
074e02eb14 fix 2021-10-18 06:38:47 +00:00
alexey-milovidov
7beffb04ad
Merge pull request #29495 from ClickHouse/improvement_for_strange_mutations 
Minor improvement for strange mutations
 2021-10-17 20:57:06 +03:00
feng lv
9378b93e41 fix 2021-10-17 14:49:27 +00:00
Kseniia Sumarokova
5633865df1
Update src/Storages/StorageFile.cpp 
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
 2021-10-17 14:16:40 +03:00
Nikolai Kochetov
2a95d7fc87
Merge pull request #29918 from amosbird/projection-improve5 
Add primary key to minmax_count_projection
 2021-10-17 14:04:18 +03:00
kssenii
059fc1de69 Allow symlinks in file storage 2021-10-17 09:11:06 +00:00
Nikolai Kochetov
bfcbf5abe0 Merge branch 'master' into removing-data-streams-folder 2021-10-17 10:42:37 +03:00
alexey-milovidov
4f11cfa59d
Merge pull request #28760 from azat/mutator-forbid-cleaner 
Forbid cleaning of tmp directories that can be used by an active mutation/merge.
 2021-10-17 03:18:01 +03:00
kssenii
4a0bdd296d Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-16 18:35:48 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
alesapin
9418eda122
Merge pull request #30111 from ClickHouse/remove_trash 
Remove trash from MergeTreeReadPool
 2021-10-16 15:41:41 +03:00
Amos Bird
9ceb668a4d
Fix TSan 2021-10-16 18:13:41 +08:00
Amos Bird
77c4a5fa18
Better 2021-10-16 18:13:41 +08:00
Amos Bird
77a2022cf5
Add primary key to minmax_count_projection 2021-10-16 18:13:41 +08:00
Nikolai Kochetov
067eaadadd Merge branch 'master' into removing-data-streams-folder 2021-10-16 09:46:05 +03:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces 
Remove stream interfaces
 2021-10-16 09:34:01 +03:00
feng lv
8b7721ae6f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-16 05:14:11 +00:00
Anton Popov
d2ba3bcaa4
Merge pull request #28363 from amosbird/fastload2 
Speed up part loading for JBOD
 2021-10-16 02:37:32 +03:00
Azat Khuzhin
07e8b2b3c7 Do not try to remove temporary paths that is currently in written by merge/mutation 
v2: rebase against MergeTask
v3: rebase due to conflicts in src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp
v4:
- rebase due to conflicts in src/Storages/MergeTree/MergeTask.cpp
- drop common/scope_guard_safe.h (not used)
 2021-10-16 00:43:52 +03:00
Azat Khuzhin
0b3bf43d61 Rewrite MergeTreeData::clearOldTemporaryDirectories() to use early continue 2021-10-16 00:43:51 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
mergify[bot]
477ea62257
Merge branch 'master' into remove-sample-by 2021-10-15 15:14:17 +00:00
kssenii
759f747080 Fix build 2021-10-15 17:12:34 +03:00
kssenii
7b81766168 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into remote-connection 2021-10-15 12:36:44 +00:00
kssenii
ccb02cac56 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-15 12:36:30 +00:00
kssenii
881ae8617e Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-15 15:09:56 +03:00
kssenii
edfb1a5521 Better performance for smaller number of threads 2021-10-15 15:07:39 +03:00
Nikita Mikhaylov
d1138a8a25
Merge pull request #29691 from azat/fix-merge-mutate-memory-tracking 
Fix memory tracking for merges and mutations
 2021-10-15 13:53:52 +03:00
Nikolai Kochetov
ad8a344b46 Move TTL streams and algo 2021-10-15 13:11:57 +03:00
Maksim Kita
7beaeb8b13
Merge branch 'master' into indexcache 2021-10-15 12:59:09 +03:00
Maksim Kita
27fdeef4b6
Merge pull request #29650 from kitaisreal/add-default-database-visitor-support-dict-get 
AddDefaultDatabaseVisitor support dictGet
 2021-10-15 12:28:58 +03:00
kssenii
3995506d37 Adjust range reader for remote fs reads 2021-10-15 11:36:26 +03:00
Azat Khuzhin
2a7a1d8df5 Avoid losing any allocations context from merges/mutations 2021-10-15 01:43:28 +03:00
Azat Khuzhin
8a209a78d7 Set query_id for mutations/merges 
This will allow to distinguish allocations in trace_log.
 2021-10-15 01:43:28 +03:00
Azat Khuzhin
8cc45bea7b Avoid accounting memory from another mutation/merge 
Before this path it was possible that one merge/mutation may account
memory from another, due to ThreadStatus::untracked_memory.

And this cause flakiness of 01200_mutations_memory_consumption.
 2021-10-15 01:43:28 +03:00
Azat Khuzhin
fd38cbb0df Fix memory tracking for merges and mutations (by destroying earlier) 
It fixes only some tiny allocations, and so it should not affect any
huge mutations/merges.

And plus, this should not be a real fix, since peak_memory_usage is
obtained before even destrying this objects, and destroing objects will
unlikely update peak memory usage (although it is possible).

v0: do this in dtors
v2: do this earlier
 2021-10-15 01:43:27 +03:00
Azat Khuzhin
e4b39ca99c Remove unused Block member from MutateTask 
v0: Fix excessive memory usage in MutateTask
    This is relevant only for 01200_mutations_memory_consumption test, since
    this is just a block.
v2: just remove unused Block member, since other part had been fixed in #29768
 2021-10-15 01:43:27 +03:00
alexey-milovidov
bc1662b9fe
Merge pull request #29946 from azat/log-fix-data-race 
Fix data-race between LogSink::writeMarks() and LogSource in StorageLog
 2021-10-15 01:30:58 +03:00
alexey-milovidov
8d694cbcdc
Merge pull request #29959 from azat/log-data-race-file-checker-fix 
Fix possible data-race between FileChecker and StorageLog/StorageStripeLog
 2021-10-15 01:30:23 +03:00
Anton Popov
7aa6068fb2 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-14 19:44:08 +03:00
mergify[bot]
92dd528c78
Merge branch 'master' into remove_trash 2021-10-14 14:14:07 +00:00
Anton Popov
476d7a411f allow to remove SAMPLE BY expression 2021-10-14 17:05:20 +03:00
Maksim Kita
c1f04452de AddDefaultDatabaseVisitor support dictGet 2021-10-14 16:31:32 +03:00
feng lv
400a31db06 reduce poll timeout 
fix
 2021-10-14 11:45:18 +00:00
Anton Popov
c2d376d0d6
Merge branch 'master' into better-interfaces 2021-10-14 14:33:36 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Nikolai Kochetov
3d3e143c29
Merge pull request #30001 from ClickHouse/remove-streams-from-formats 
Remove streams from formats.
 2021-10-14 12:39:07 +03:00
Maksim Kita
f589084a94
Merge pull request #29899 from Enmk/Governance/dictionary_comment 
Governance/dictionary comment
 2021-10-14 12:35:10 +03:00
Anton Popov
92413aed68 better interfaces for IDataType and ISerialization 2021-10-14 05:36:49 +03:00
feng lv
21f7ecbd9f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-14 02:09:51 +00:00
feng lv
3116767834 fix 2021-10-14 02:09:43 +00:00
Maksim Kita
659a46a972 WriteBufferToRabbitMQProduced fixed queue usage 2021-10-14 00:33:18 +03:00
Maksim Kita
04047f76c7 Fixed tests 2021-10-14 00:33:18 +03:00
Maksim Kita
d55561422f Fixed tests 2021-10-14 00:33:18 +03:00
Maksim Kita
c9b6c2661b Refactor ConcurrentBoundedQueue 2021-10-14 00:33:18 +03:00
Azat Khuzhin
eba6410fd5 Fix data-race between LogSink::writeMarks() and LogSource::readData() in StorageLog 
CI founds [1], TSan report:

    WARNING: ThreadSanitizer: data race (pid=497)
      Write of size 8 at 0x7b1c00e4e8d8 by thread T922:
        4 DB::LogSink::writeMarks() obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:462:72 (clickhouse+0x1665cdaa)
        5 DB::LogSink::consume() obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:324:5 (clickhouse+0x1665c216)

      Previous read of size 8 at 0x7b1c00e4e8d8 by thread T711:
        1 DB::LogSource::readData()::$_0::operator()(bool) const::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:188:26 (clickhouse+0x16661baf)
        9 DB::LogSource::readData() obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:204:20 (clickhouse+0x1665bbcb)

      [1]: https://clickhouse-test-reports.s3.yandex.net/29930/4bc90d1dd7dbd4b8a9b6920d00ca24e8b160358e/stress_test_(thread).html#fail1

Fixes: #29470
v2: rework locking, but this produce deadlock for 01499_log_deadlock test
v3: introduce per-file marks mutex
 2021-10-14 00:17:45 +03:00
Azat Khuzhin
e4db862d85 Fix possible data-race between FileChecker and StorageLog/StorageStripeLog 2021-10-14 00:16:58 +03:00
Nikolay Degterinsky
63faf0212d Merge branch 'master' into query_parameters 2021-10-13 23:55:09 +03:00
Nikolai Kochetov
2957971ee3 Remove some last streams. 2021-10-13 21:22:02 +03:00
alesapin
1801ea4955 Merge branch 'master' into remove_trash 2021-10-13 18:10:31 +03:00
feng lv
168020e0db fix 2021-10-13 14:02:32 +00:00
alesapin
d2dfbb5ab6 Remove trash from MergeTreeReadPool 2021-10-13 15:25:44 +03:00
feng lv
c82fd3491d fix 
fix
 2021-10-13 11:18:19 +00:00
feng lv
868dbc4bad use poll replace select 
fix

fix
 2021-10-13 09:13:35 +00:00
mergify[bot]
cfda5592fe
Merge branch 'master' into fix-writing-storage-log-marks 2021-10-13 07:18:33 +00:00
feng lv
74e1900034 fix 2021-10-13 05:23:00 +00:00
Amos Bird
23602f4607
Speed up part loading for JBOD 2021-10-13 12:33:54 +08:00
feng lv
0409b775bf fix 2021-10-13 02:11:15 +00:00
Nikolay Degterinsky
49c1a52f34 Merge branch 'master' into query_parameters 2021-10-13 03:04:40 +03:00
Azat Khuzhin
e9ce859b02 Fix race between MOVE PARTITION and merges/mutations for MergeTree 
From time to time 00975_move_partition_merge_tree test got failed, the
reason is that there is merge in progress that writes data to the source
table again:

<details>

    2021.10.12 13:20:40.243839 [ 26955 ] {2dbfea50-639e-49ac-9e82-933a00ea04a3} <Debug> executeQuery: (from [::1]:38060) (comment: 00975_move_partition_merge_tree.sql) ALTER TABLE test_move_partition_src MOVE PARTITION 1 TO TABLE test_move_partition_dest;
    2021.10.12 13:20:40.244482 [ 26955 ] {2dbfea50-639e-49ac-9e82-933a00ea04a3} <Debug> test_t1tzb2.test_move_partition_dest (cab3b005-d54b-4cdc-8ab3-b005d54becdc): Cloning part /var/lib/clickhouse/store/467/467d7145-b47e-444e-867d-7145b47ea44e/1_2_2_0/ to /var/lib/clickhouse/store/cab/cab3b005-d54b-4cdc-8ab3-b005d54becdc/tmp_move_from_1_21_21_0
    ...
    2021.10.12 13:20:40.373487 [ 378 ] {} <Trace> test_t1tzb2.test_move_partition_src (467d7145-b47e-444e-867d-7145b47ea44e) (MergerMutator): Merged 6 parts: from 1_2_2_0 to 1_11_11_0
    ...
    2021.10.12 13:20:40.379750 [ 26955 ] {2dbfea50-639e-49ac-9e82-933a00ea04a3} <Debug> test_t1tzb2.test_move_partition_dest (cab3b005-d54b-4cdc-8ab3-b005d54becdc): Cloning part /var/lib/clickhouse/store/467/467d7145-b47e-444e-867d-7145b47ea44e/1_15_15_0/ to /var/lib/clickhouse/store/cab/cab3b005-d54b-4cdc-8ab3-b005d54becdc/tmp_move_from_1_28_28_0

</details>

And also remove cleaning of mutations since this will cause deadlock
after doing MOVE PARTITION under
currently_processing_in_background_mutex.

CI: https://clickhouse-test-reports.s3.yandex.net/0/a59c6b1c8eb47ebf77189a491ee0c3980b38e91c/functional_stateless_tests_
Fixes: 00975_move_partition_merge_tree
 2021-10-13 00:14:24 +03:00
feng lv
b2a17cb320 fix build and conflict 2021-10-12 16:50:50 +00:00
Kseniia Sumarokova
bb15b57753
Merge pull request #30016 from ucasfl/sample-by-tuple 
Fix  crash of sample by tuple()
 2021-10-12 19:46:21 +03:00
Nikolai Kochetov
f2d8cc547f Fix Kafka source. 2021-10-12 18:09:28 +03:00
Nikolai Kochetov
c36bb9206e
Merge pull request #29879 from amosbird/queryidholderfix 
Fix potential leak of query_id_holder
 2021-10-12 15:42:02 +03:00
Nikolai Kochetov
077aba4a97
Merge pull request #29520 from amosbird/projection-improve4 
Get rid of naming limitation of projections.
 2021-10-12 14:25:29 +03:00
Nikolai Kochetov
93b3fcc195 Fix rabbitmq 2021-10-12 13:05:20 +03:00
kssenii
9814cb1b45 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-12 09:49:08 +00:00
Vladimir C
969999ff10
Merge pull request #29544 from Algunenano/join_deadlock 2021-10-12 12:34:04 +03:00
Maksim Kita
91a48e9651
Merge pull request #29896 from kitaisreal/system-data-skipping-indices-added-size 
System table data skipping indices added size
 2021-10-12 12:16:57 +03:00
Maksim Kita
c15b67c182 Fix naming 2021-10-12 11:42:24 +03:00
alesapin
5ac0d36392
Merge pull request #30005 from ClickHouse/more_strict_check_for_intersecting_parts 
More strict check for intersecting parts
 2021-10-12 11:38:09 +03:00
kssenii
e7dbfff5d2 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-12 06:19:15 +00:00
feng lv
b5d69d599e fix sample by tuple() 
add tests

fix

fix

fix
 2021-10-12 05:19:32 +00:00
alexey-milovidov
f1b550cdaf
Merge pull request #29974 from ClickHouse/minimized-toolchain 
Experiment with minimized toolchain (2)
 2021-10-11 22:04:01 +03:00
Alexander Tokmakov
706e2b6b88 more strict check for intersecting parts 2021-10-11 21:42:46 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
kssenii
6d93253a67 Fix tests 2021-10-11 11:34:08 +00:00
kssenii
1de55cfeb4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into remote-connection 2021-10-11 11:33:35 +00:00
Alexey Milovidov
2370a8d6da Fix PVS-Studio 2021-10-11 14:28:46 +03:00
Amos Bird
72bccaa501
Fix path name 2021-10-11 19:12:08 +08:00
Maksim Kita
b0d887a0fe Added tests 2021-10-11 14:00:10 +03:00
Amos Bird
83717b7c3b
Get rid of naming limitation of projections. 2021-10-11 18:32:17 +08:00
Kseniia Sumarokova
20e8074b5e
Merge pull request #29912 from taiyang-li/remove_unused_var 
Remove unused variable in s3cluster
 2021-10-11 12:45:04 +03:00
Maksim Kita
ce0c41e1ad Fixed tests 2021-10-11 11:39:50 +03:00
Maksim Kita
2d069acc22 System table data skipping indices added size 2021-10-11 11:39:50 +03:00
Vasily Nemkov
2c55807b2a Create dictionary with comment 2021-10-11 11:26:59 +03:00
Amos Bird
ebfb013ea1
Fix potential leak of query_id_holder 2021-10-11 15:56:00 +08:00
Vladimir C
dc4bb920d3
Merge branch 'master' into join_deadlock 2021-10-11 10:18:17 +03:00
Vladimir C
6c7fbf0b88
Merge pull request #29956 from azat/fix-tuple-filter 2021-10-11 10:02:47 +03:00
feng lv
a57e97d2e0 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-11 06:09:43 +00:00
feng lv
d8bd1474bc fix 2021-10-11 04:18:21 +00:00
feng lv
4d9604e298 fix 2021-10-11 04:06:31 +00:00
kssenii
448d63a0fc Range read with upper bound offset 2021-10-11 01:53:51 +03:00
alexey-milovidov
0adbcd58c9
Merge pull request #29930 from azat/buffer-fix-data-race 
Fix data-race between flush() and startup() in StorageBuffer
 2021-10-11 01:11:39 +03:00
kssenii
aa4b797808 Turn off tasks stealing for remote disk 2021-10-10 23:22:58 +03:00
Azat Khuzhin
6c33eaee32 Fix filtering by tuple (some conditions was lost during analyzing) 
Fixes: #29281
Fixes: test_cluster_copier/test_three_nodes.py::test
 2021-10-10 22:37:52 +03:00
Nikolai Kochetov
a95c28ec4b
Merge pull request #29898 from ClickHouse/remove-native-stream 
Remove some more streams.
 2021-10-10 21:01:16 +03:00
feng lv
045704274c implement directory watcher 2021-10-10 16:31:32 +00:00
kssenii
eaebd84cd5 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-10 15:17:35 +00:00
Azat Khuzhin
4bc90d1dd7 Fix data-race between flush() and startup() in StorageBuffer 
Stress tests found [1], TSan report:

    ==================
    WARNING: ThreadSanitizer: data race (pid=485)
      Read of size 8 at 0x7b5001280bd8 by thread T567 (mutexes: write M612061890855345680):
        1 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator bool() const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2851:62 (clickhouse+0x159140a6)
        2 bool std::__1::operator!=<DB::BackgroundSchedulePoolTaskInfo>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3447:30 (clickhouse+0x159140a6)
        3 DB::BackgroundSchedulePoolTaskHolder::operator bool() const obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:164:46 (clickhouse+0x159140a6)
        4 DB::StorageBuffer::flush() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:675:10 (clickhouse+0x159140a6)

      Previous write of size 8 at 0x7b5001280bd8 by thread T586 (mutexes: write M191819750614415520):
        2 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator=(std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:34 (clickhouse+0x15913e22)
        3 DB::BackgroundSchedulePoolTaskHolder::operator=() obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:156:110 (clickhouse+0x15913e22)
        4 DB::StorageBuffer::startup() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:668:18 (clickhouse+0x15913e22)
        5 DB::InterpreterCreateQuery::doCreateTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1092:10 (clickhouse+0x149bef7b)
        6 DB::InterpreterCreateQuery::createTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:952:20 (clickhouse+0x149ba9f5)
        7 DB::InterpreterCreateQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1302:16 (clickhouse+0x149c1086)

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/1c9778603ff49563d1d3d0d357de0608167e504d/stress_test_(thread).html

Fixes: #29416
 2021-10-10 04:03:36 +03:00
Maksim Kita
0358d664b9
Merge pull request #29882 from MaxWk/add-alias-for-name 
Add table alias to system.tables and database alias to system.databases
 2021-10-09 12:44:40 +03:00
Vasily Nemkov
67ff0f5dba Refactoring: moved comment out of ASTStorage to ASTCreateQuery 2021-10-09 12:25:27 +03:00
Nikolai Kochetov
939d38b13e Fix gtest build. 2021-10-09 10:01:58 +03:00
taiyang-li
860f74db62 remove unused var 2021-10-09 11:15:13 +08:00
tavplubix
fa1396a9dd
Merge pull request #29790 from ClickHouse/fix-bad-cast-in-parser-create 
Fix bad cast in ParserCreateQuery
 2021-10-08 23:53:38 +03:00
Nikolai Kochetov
c6bce1a4cf Update Native. 2021-10-08 20:21:19 +03:00
Nikolai Kochetov
340b53ef85 Remove some more streams. 2021-10-08 17:03:54 +03:00
kssenii
f3f820c6e0 HTTP read buffer retries 2021-10-08 15:41:55 +03:00
MaxWk
f0e193d5a5 add database alias 2021-10-08 20:25:37 +08:00
Nikolai Kochetov
9eddee5517
Merge pull request #29768 from ClickHouse/remove-merging-streams 
Remove some merging streams
 2021-10-08 13:15:10 +03:00
MaxWk
b081f58db5 add alias 2021-10-08 16:49:02 +08:00
Maksim Kita
717998a37f
Merge pull request #29825 from azat/memory_profiler_step-cleaner-api 
Make memory_profiler_step API cleaner
 2021-10-07 22:44:47 +03:00
tavplubix
de2fd9d762
Merge pull request #29812 from ClickHouse/better_diagnostic_for_optimize 
Better diagnostic for OPTIMIZE
 2021-10-07 22:22:25 +03:00
Kseniia Sumarokova
185d58a7cc
Merge pull request #29766 from ClickHouse/remove_redundant_seeks 
Less seeks in compressed buffers
 2021-10-07 17:04:50 +03:00
Alexander Tokmakov
7e2bc184ec fix another suspicious places, add test 2021-10-07 16:43:49 +03:00
feng lv
4233e2cca4 fix 2021-10-07 13:23:25 +00:00
feng lv
4841b31b56 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-07 13:20:08 +00:00
tavplubix
754e038eec
Merge pull request #29803 from ClickHouse/intersects-previous-part-usability 
Improve usability of error messages when error is caused by sophisticated interventions
 2021-10-07 15:24:17 +03:00
feng lv
bd1ea5cd16 try fix undefine in ifstream 2021-10-07 12:15:36 +00:00
Alexander Tokmakov
5365cc686f Merge branch 'master' into improvement_for_strange_mutations 2021-10-07 13:26:58 +03:00
Alexander Tokmakov
4119848a41 better diagnostic for OPTIMIZE 2021-10-07 13:08:10 +03:00
Vladimir C
0422939893
Merge pull request #29281 from lingtaolf/bugfix/tupleFilter 2021-10-07 12:24:26 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491) 
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-07 11:26:08 +03:00
Azat Khuzhin
71cffbf521 Make memory_profiler_step API cleaner 
Right now to configure memory_profiler_step/total_memory_profiler_step
you need to call:

    MemoryTracker::setOrRaiseProfilerLimit()
    MemoryTracker::setProfilerStep()

But it is easy to forget about setOrRaiseProfilerLimit(), since there is
no even any comments about this.

So instead, make setOrRaiseProfilerLimit() private and call it from
setProfilerStep()
 2021-10-07 10:52:16 +03:00
feng lv
127a0f3591 fix 2021-10-07 07:31:41 +00:00
Maksim Kita
4ec7311d4d
Merge pull request #29738 from kitaisreal/added-function-ngram 
Added function ngram
 2021-10-07 10:21:00 +03:00
feng lv
d9959a4f22 fix 2021-10-07 05:56:09 +00:00
feng lv
28f8e1c297 fix 
update test
 2021-10-07 05:33:12 +00:00
Maksim Kita
b7b39da8cc Fixed build 2021-10-06 23:22:29 +03:00
Nikolai Kochetov
d0c6f11fcb More. 2021-10-06 20:59:27 +03:00
Alexey Milovidov
1455cfeddd Improve usability of error messages when error is caused by sophisticated interventions 2021-10-06 15:28:52 +03:00
Maksim Kita
33353be4ed Fixed tests 2021-10-06 15:12:07 +03:00
kssenii
1f91e2de46 Update install.sh 2021-10-06 12:08:37 +00:00
kssenii
c96b403809 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into remote-connection 2021-10-06 11:54:28 +00:00
feng lv
c801fa65df fix and add new test 
fix

fix
 2021-10-06 10:56:42 +00:00
Nikolai Kochetov
6b619512aa Fix style. 2021-10-06 12:37:56 +03:00
feng lv
acdb8233c3 fix 
fix

fix
 2021-10-06 09:06:33 +00:00
kssenii
5a87b6eb4f Predefined configuration for table function remote 2021-10-06 05:55:09 +00:00
Maksim Kita
90eb74c933 Fixed tests 2021-10-06 00:37:45 +03:00
feng lv
f2a62ef9ee try to fix data race found in DirectoryWatcher 2021-10-05 17:38:18 +00:00
mergify[bot]
ab55225c8f
Merge branch 'master' into remove-merging-streams 2021-10-05 14:16:34 +00:00
Nikolai Kochetov
96d070a5ba Fix tests. 2021-10-05 16:58:24 +03:00
feng lv
0709b2e690 fix 2021-10-05 13:28:45 +00:00
mergify[bot]
3b1f49aac7
Merge branch 'master' into nv/parts-uuid-move-shard-kill 2021-10-05 12:36:02 +00:00
alesapin
9aab0eb0a2 Fixup 2021-10-05 13:41:09 +03:00
alesapin
8186339307 Less seeks in compressed buffers 2021-10-05 12:11:25 +03:00
Vitaly Baranov
8a01b32cba
Merge pull request #28637 from vitlibar/fix-materialized-column-as-sharding-key 
Fix materialized column as sharding key
 2021-10-05 10:53:24 +03:00
feng lv
86d31445fd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-05 03:58:27 +00:00
Alexander Tokmakov
bb32432943 more fixes 2021-10-05 00:13:18 +03:00
Nikolai Kochetov
2001ebbf9d Fix build. 2021-10-04 21:52:31 +03:00
Nikita Mikhaylov
b31d11478b
Merge pull request #29680 from azat/defines 
Cleanup common defines
 2021-10-04 20:13:29 +03:00
Nikita Mikhaylov
36516dd503
Merge pull request #29681 from azat/MemoryTracker-mutate-merge 
Apply max_untracked_memory/memory_profiler_step/memory_profiler_sample_probability during mutate/merge
 2021-10-04 20:04:57 +03:00
Maksim Kita
3fe9fdac96 Added function ngram 2021-10-04 18:53:17 +03:00
Vitaly Baranov
200f655a2f Fix writing marks in StorageLog. 2021-10-04 18:45:04 +03:00
Nikolay Degterinsky
e03bf0c526 Merge branch 'master' into query_parameters 2021-10-04 17:23:17 +03:00
feng lv
873805d4da update poco 
update

fix unbundled build
 2021-10-04 13:55:49 +00:00
Anton Popov
362e84a336
Merge pull request #29429 from Enmk/Governance/database_comment 
Implemented creating databases with comments
 2021-10-04 15:12:33 +03:00
feng lv
45a49b399a fix typo 2021-10-04 08:23:02 +00:00
Vitaly Baranov
1636ee24bb Fix using materialized column as sharding key. 2021-10-04 10:56:42 +03:00
feng lv
f40a17c300 fix bug in updateFileInfos 
update
 2021-10-04 07:45:37 +00:00
Kseniia Sumarokova
5ab17d795f
Merge pull request #29673 from ClickHouse/url-fix-initialization 
Fix initialization of `StorageURLSource`
 2021-10-04 10:23:52 +03:00
feng lv
8c82e8b9bc fix 
fix
 2021-10-03 17:49:00 +00:00
kssenii
dbc114e6fc Fix special build check 2021-10-03 16:49:02 +00:00
kssenii
7ea9037939 Fix 2021-10-03 16:41:12 +00:00
Azat Khuzhin
117e9e77c8 Apply max_untracked_memory/memory_profiler_{step,sample_probability} during mutate/merge 2021-10-03 17:39:07 +03:00
kssenii
7bb2649744 Fix checks 2021-10-03 14:36:32 +00:00
kssenii
0cb7193fde Fix url with failover case 2021-10-03 13:58:03 +00:00
Azat Khuzhin
ae5ee2dd28 Move macros for distributed engine into separate header 2021-10-03 14:34:03 +03:00
Azat Khuzhin
6a9dd9828d Move protocol macros into separate header 
Defines.h is a very common header, so lots of modules will be recompiled
on changes.
Move macros for protocol into separate header, this should significantly
decreases number of units to compile on it's changes.
 2021-10-03 14:34:03 +03:00
feng lv
9683513c52 fix build 2021-10-03 11:03:20 +00:00
feng lv
8a4092c55b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-03 10:00:16 +00:00
feng lv
3a56cc5141 fix 
update test
 2021-10-03 09:49:11 +00:00
Kseniia Sumarokova
ca7395e374
Merge branch 'master' into materialized-postgresql-fix 2021-10-03 11:56:57 +03:00
kssenii
e4d1d43a34 Fixes 2021-10-03 08:55:50 +00:00
Alexey Milovidov
868215d0dc Fix initialization of StorageURLSource 2021-10-03 07:28:28 +03:00
Maksim Kita
395119339f Merge branch 'master' into rename-common 2021-10-02 17:15:10 +03:00
kssenii
6935cc667b Fix tests 2021-10-02 12:52:00 +00:00
kssenii
28aeb30d4a Support full schema list 2021-10-02 11:15:24 +00:00
alexey-milovidov
86d1ce63cf
Merge pull request #29649 from CurtizJ/merging-29376 
Merging #29376
 2021-10-02 12:02:42 +03:00
kssenii
c601db109f Correct nerge 2021-10-02 08:39:15 +00:00
Alexey Milovidov
2ad5dbdfe8 Rename "common" to "base" 2021-10-02 11:30:38 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Anton Popov
bb92ff52d8
Update IMergeTreeDataPart.cpp 2021-10-02 00:50:19 +03:00
Anton Popov
d404efbf7e add test for freeze in-memory parts 2021-10-01 22:05:36 +03:00
Anton Popov
f0363c524d Merge branch 'master' of git://github.com/mo-avatar/ClickHouse into merging-29376 2021-10-01 20:29:33 +03:00
Maksim Kita
57666725f9
Merge pull request #29623 from kitaisreal/bloom-filter-indexes-map-support-fix-unsafe-identifier-cast 
BloomFilter indexes map data type support fix unsafe identifier cast
 2021-10-01 19:28:04 +03:00
Nikita Mikhaylov
b2f7b6fae8
Fix race between MergeTask and storage destruction (#29614) 2021-10-01 18:55:04 +03:00
kssenii
3bc3a01f09 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into materialized-postgresql-fix 2021-10-01 15:53:27 +00:00
feng lv
9c1dcf41b8 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-01 14:47:21 +00:00
feng lv
5bb5e04fb3 fix assert 2021-10-01 14:47:05 +00:00
vdimir
7444d64156
Fix for recursive tuples in MergeTreeWhereOptimizer 2021-10-01 17:01:03 +03:00
vdimir
23ce41e3bb
MergeTreeWhereOptimizer supports constant tuple in both sides of equals 2021-10-01 17:01:03 +03:00
vdimir
64ffacd9e0
MergeTreeWhereOptimizer::tryAnalyzeTuple, fix fuzz 2021-10-01 17:01:02 +03:00
vdimir
e326b12a37
Fix style in MergeTreeWhereOptimizer.cpp 2021-10-01 17:01:02 +03:00
凌涛
f660224a5f
add tests 2021-10-01 17:01:01 +03:00
凌涛
ea68944055
fixed index didn't work in tuple where 2021-10-01 17:01:01 +03:00
Maksim Kita
c793c8e7fa BloomFilter indexes map data type support fix unsafe identifier cast 2021-10-01 14:40:00 +03:00
feng lv
6eeade23fa revert drop table immediately 
remove unused code

fix pvs check

fix
 2021-10-01 11:36:49 +00:00
Vasily Nemkov
2f16806cd0 Implemented creating databases with comments 2021-10-01 11:51:26 +03:00
feng lv
ac9af1ca38 fix wrong offset 2021-10-01 08:24:57 +00:00
feng lv
8c20a184ca optimize event handle 2021-10-01 05:06:19 +00:00
Alexey Milovidov
4271f8c738 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-10-01 01:33:58 +03:00
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings (#29140) 2021-10-01 00:26:24 +03:00
Kseniia Sumarokova
3e0d4faa1e
Merge pull request #29503 from azat/external-db-log-query 
Log queries to external databases (since they may be rewritten)
 2021-09-30 21:08:57 +03:00
feng lv
4390c8f029 fix 2021-09-30 17:13:45 +00:00
feng lv
ee02e52d3d Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-09-30 16:02:31 +00:00
feng lv
1758ff2d32 fix 2021-09-30 16:02:17 +00:00
Alexey Milovidov
551c52c236 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-09-30 17:45:14 +03:00
Raúl Marín
6f2447c027 clang-tidy fix 2021-09-30 15:48:54 +02:00
Raúl Marín
f58742014c Consistent naming 2021-09-30 11:05:21 +02:00
Nikolay Degterinsky
7f3d395bf3 Merge branch 'master' into query_parameters 2021-09-30 02:01:33 +03:00
Azat Khuzhin
a4f2663b82 Log queries to external databases (since they may be rewriten) 
By rewritten means that some expression from WHERE can be removed, yes
there is external_table_strict_query, but sometimes it is too strict.
 2021-09-29 21:56:28 +03:00
Azat Khuzhin
6ddfbf39c8 Cover IS NULL/IS NOT NULL for external dbs in unit tests 2021-09-29 21:45:09 +03:00
Azat Khuzhin
24f55e29d1 Add support for IS NULL/IS NOT NULL for external dbs 2021-09-29 21:45:09 +03:00
Nikolai Kochetov
68f8b9d235 Update ColumnGathererStream 2021-09-29 20:45:01 +03:00
Raúl Marín
0ee5c0bff5 Use RWLock in StorageJoin to avoid deadlocks 2021-09-29 19:30:07 +02:00
Anton Popov
914781052e Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-29 17:37:07 +03:00
mergify[bot]
6a2ac6e871
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-29 13:29:08 +00:00
Vladimir C
27f0f9fcf3
Merge pull request #21320 from arenadata/ADQM-138 2021-09-29 11:53:54 +03:00
Maksim Kita
0279afa738
Merge pull request #29474 from kitaisreal/bloom-filter-indexes-updated-tests 
Bloom filter indexes updated tests
 2021-09-28 22:40:27 +03:00
Maksim Kita
9ecd0a3f4f
Merge pull request #29220 from ClickHouse/full-text-bloom-filter-index-map-data-type 
Merging #28511
 2021-09-28 22:40:17 +03:00
alexey-milovidov
1e6f9ac635
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 20:26:44 +03:00
mo-avatar
6c16ec83eb in memory part lost when freeze 2021-09-28 22:52:01 +08:00
vdimir
71b6c9414c Minor changes related to JOIN ON ORs 2021-09-28 14:11:31 +03:00
vdimir
8e2637aab2 Store all related to one join disjunct in JoinOnClause, pt1 2021-09-28 14:11:31 +03:00
Ilya Golshtein
3766d47f31 ORs in JOINs 2021-09-28 14:11:28 +03:00
Maksim Kita
f2402e1a10 Fixed tests 2021-09-28 12:52:07 +03:00
Maksim Kita
71fb9e6059 Bloom filter indexes updated tests 2021-09-28 12:38:27 +03:00
feng lv
1f90c8dc22 refactor 
fix style
 2021-09-28 09:17:50 +00:00
Maksim Kita
9f9f36b480
Merge pull request #29464 from azat/std-cerr-cout-style-check 
Add std::cerr/std::cout style check
 2021-09-28 12:12:44 +03:00
Vladimir C
6c2db9c29b
Merge pull request #29428 from vdimir/rocksdb-istream-to-source 2021-09-28 12:06:41 +03:00
Azat Khuzhin
9a23b20965 Remove debug from MergeTreeIndexConditionBloomFilter::traverseASTEquals() 2021-09-28 10:28:28 +03:00
feng lv
ef5c71a9ab Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-09-28 02:06:28 +00:00
alexey-milovidov
995913e89c
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 02:27:36 +03:00
Maksim Kita
7861c26eaf
Merge pull request #29431 from kitaisreal/map-bloom-filter-index-map-values-equals-support 
Map bloom filter index mapValues equals function support
 2021-09-27 22:03:53 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views 
Rewrite PushingToViews
 2021-09-27 21:19:11 +03:00
alesapin
ddd80c8e72
Merge pull request #29264 from Enmk/governance/table_comment_update 
Implemented modifying table comments with `ALTER TABLE t MODIFY COMMENT 'value'`
 2021-09-27 18:02:14 +03:00
Maksim Kita
e290f19957 Merge branch 'master' into full-text-bloom-filter-index-map-data-type 2021-09-27 17:50:07 +03:00
Maksim Kita
b28d6e76cc Map bloom filter full text index added support for mapKeys, mapValues 2021-09-27 17:26:25 +03:00
Maksim Kita
cd9c796e6f Map bloom filter index mapValues equals function support 2021-09-27 17:23:56 +03:00
vdimir
aea2b952fd
Merge EmbeddedRocksDBBlockInputStream and EmbeddedRocksDBSource 2021-09-27 16:16:34 +03:00
Kseniia Sumarokova
beda6bee2e
Merge pull request #29276 from kssenii/hdfs-fix-assertion 
Fix !hasPendingData() failed assertion in hdfs read buffer
 2021-09-27 15:47:17 +03:00
feng lv
814f6d5068 fix 2021-09-27 12:09:01 +00:00
Nikita Mikhaylov
3741b8a17e
Merge pull request #29337 from amosbird/projection-fix18 
Fix projection merges and mutations.
 2021-09-27 13:45:13 +03:00
vdimir
8fe17dd3d1
Revert "Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource" 
This reverts commit 3ab217031e50a0b4d19fb24d72088e353cdfa510.
 2021-09-27 12:49:14 +03:00
vdimir
dc7393e810
Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource 2021-09-27 12:49:13 +03:00
vdimir
222ab78931
Convert EmbeddedRocksDBBlockInputStream to ISource 2021-09-27 12:49:13 +03:00
feng lv
30428ad1b0 fix 2021-09-27 08:44:48 +00:00
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
feng lv
56be3fab00 fix 2021-09-27 07:17:16 +00:00
feng lv
1eb96c2fce fix 2021-09-27 04:39:50 +00:00
feng lv
c8062ad397 fix 2021-09-27 04:25:59 +00:00
feng lv
69985ac4ae fix 2021-09-27 02:27:24 +00:00
Alexey Milovidov
1227a383e3 Merge branch 'remove-trash-db-regexp' of github.com:yandex/ClickHouse into remove-trash-db-regexp 2021-09-27 02:17:19 +03:00
Alexey Milovidov
16f0f0af7f Merge remote-tracking branch 'origin/master' into remove-trash-db-regexp 2021-09-27 02:12:13 +03:00
mergify[bot]
64669a28ce
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-26 22:59:23 +00:00
feng lv
da2c8f23cc Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-09-26 15:00:37 +00:00
Nikolai Kochetov
efbd8e4911 Remove ExceptionHandlingSink. 2021-09-26 17:54:59 +03:00
feng lv
6c4b0f2d37 drop table immediately 
fix style

update
 2021-09-26 14:13:12 +00:00
feng lv
30527879b9 fix style 2021-09-26 12:19:19 +00:00
feng lv
7af96b12b1 add meta file 
fix and update tests
 2021-09-26 11:28:46 +00:00
mergify[bot]
15e5204a7c
Merge branch 'master' into issue-29227-rocksdb-truncate 2021-09-26 00:10:30 +00:00
Alexey Milovidov
fb7ce28198 Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-26 02:10:11 +03:00
Kseniia Sumarokova
a7809d3d63
Merge pull request #29365 from kssenii/fix-mysql 
Fix integration test
 2021-09-26 00:52:10 +03:00
alexey-milovidov
e4363872ca
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 19:58:36 +03:00
alexey-milovidov
9dac348893
Update StorageMerge.cpp 2021-09-25 19:41:50 +03:00
kssenii
30b92d24f1 Fix 2021-09-25 14:46:03 +00:00
Amos Bird
34017c942e
More fixes 2021-09-25 13:38:56 +08:00
Alexey Milovidov
4f7830ae3a Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 06:58:30 +03:00
Alexey Milovidov
fe44be522e Fix assert in table function merge with database regexp 2021-09-25 05:48:24 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
Anton Popov
7628273df4
Merge pull request #29266 from CurtizJ/normalize-asts 
Normalize ASTs in ColumnsDescription
 2021-09-25 04:01:54 +03:00
Alexey Milovidov
82d57c6e47 Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 01:38:39 +03:00
feng lv
0009cab962 add virtual columns 2021-09-24 16:44:22 +00:00
Amos Bird
23d3d894e6
Fix projection merges and mutations. 2021-09-24 22:45:50 +08:00
Anton Popov
62dea79b78 better formatting 2021-09-24 14:41:49 +03:00
Anton Popov
4017b1eb89
Update src/Storages/ColumnsDescription.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2021-09-24 14:08:33 +03:00
Nikolai Kochetov
9d3e8fb9c3 Tiny fix. 2021-09-24 13:41:52 +03:00
Nikolai Kochetov
cc53ba283d Try fix other tests. 2021-09-24 13:23:28 +03:00
Vladimir C
604dc8861d
Apply suggestions from code review 
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
 2021-09-24 13:07:24 +03:00
Nikolai Kochetov
e617aeb7a5
Merge pull request #29139 from amosbird/projection-fix17 
Disable projections when ARRAY JOIN is used.
 2021-09-24 11:15:23 +03:00
feng lv
9456c67486 fix 
fix test

fix

fix

fix

exception handle
 2021-09-24 02:59:47 +00:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
Amos Bird
983ee004c7
Add empty key description builder. 2021-09-24 01:35:41 +08:00
Dmitrii Kovalkov
e3ae32f763 Allow nulls first 2021-09-23 19:39:15 +03:00
vdimir
9ca9a66a70
Fix race in StorageSystemRocksDB truncate 2021-09-23 17:03:33 +03:00
vdimir
81a051e3ee
Use dynamic_pointer_cast instead of dynamic_cast in StorageSystemRocksDB 2021-09-23 16:40:10 +03:00
Anton Popov
0390035963 fix unit test 2021-09-23 16:11:31 +03:00
Kruglov Pavel
55553026ee
Merge pull request #29206 from azat/strict-external-queries 
Add ability to disable converting expressions to local filter for external queries
 2021-09-23 13:22:12 +03:00
Nikolai Kochetov
998d29ebc7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-23 13:10:27 +03:00
Maksim Kita
3d79812427 Merge branch 'master' into full-text-bloom-filter-index-map-data-type 2021-09-23 12:38:03 +03:00
Maksim Kita
0341b99c69
Merge pull request #29280 from kitaisreal/full-text-bloom-filter-added-support-for-array-data-type 
FullText bloom filter index added support for Array data type
 2021-09-23 12:34:06 +03:00
Maksim Kita
d9e265df2c Fixed debug message 2021-09-23 12:13:01 +03:00
Maksim Kita
d59986f224
Merge pull request #29278 from kitaisreal/bloom-filter-map-added-support-for-has-function 
Bloom filter map added support for has function
 2021-09-23 12:03:56 +03:00
kssenii
6c1312760f Remove debug 2021-09-23 07:52:52 +00:00
Kseniia Sumarokova
f7ff14b407
Merge pull request #28577 from kssenii/named-collections 
Predefined connections and other parameters for external data sources
 2021-09-23 10:25:23 +03:00
kssenii
d71b6ad657 Fix 2021-09-23 07:21:08 +00:00
Maksim Kita
ebbefd49c3 FullText bloom filter index added support for Array data type 2021-09-23 02:06:54 +03:00
Vasily Nemkov
cb1ca9b33e Implemented modifying table comments with ALTER TABLE t MODIFY COMMENT 'value' 2021-09-23 01:14:00 +03:00
Maksim Kita
d5a7e92b97 Bloom filter map added support for has function 2021-09-23 01:10:14 +03:00
Maksim Kita
1bf375e2b7
Merge pull request #29254 from kitaisreal/map-bloom-filter-index-updated 
Updated bloom filter index for map type to work with mapKeys, mapValues functions
 2021-09-23 00:31:14 +03:00
Nikita Mikhaylov
32aa2ca939
Merge pull request #29257 from nikitamikhaylov/race-merge-task 
Fix segfault in fast test
 2021-09-23 00:12:31 +03:00
Nikolai Kochetov
4f802d1cea Fix test_executable_table_function 2021-09-22 22:31:12 +03:00
hhell
fc166570d8
Fix typo in an error message 
Related: https://github.com/ClickHouse/ClickHouse/issues/20551
 2021-09-22 21:33:15 +03:00
Maksim Kita
93208fca4b Bloom filter for map data type dded support for IN, mapContains functions 2021-09-22 20:25:10 +03:00
Kseniia Sumarokova
b41a446e8b
Delete getEngineConnectionArguments.h 2021-09-22 19:52:55 +03:00
Anton Popov
8943389d03 normalize ASTs in ColumnsDescription 2021-09-22 18:55:20 +03:00
kssenii
a84fc77b52 Fixes 2021-09-22 18:10:25 +03:00
tavplubix
ba82781e11
Merge pull request #29061 from ClickHouse/remove_some_trash 
Fix `system.distributed_ddl_queue`
 2021-09-22 16:58:29 +03:00
mergify[bot]
fd39fc58d7
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-22 13:35:20 +00:00
mergify[bot]
a3f62af09c
Merge branch 'master' into race-merge-task 2021-09-22 12:42:45 +00:00
Nikita Mikhaylov
558c74e826 Done 2021-09-22 11:52:47 +00:00
Nikolay Degterinsky
1b1d90deb8 Merge branch 'master' into query_parameters 2021-09-22 13:53:56 +03:00
Maksim Kita
92fd6b376f Updated bloom filter index for map type to work with mapKeys function 2021-09-22 12:45:48 +03:00
kssenii
b51840a205 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-22 11:45:22 +03:00
feng lv
f637f4f2d1 remove unused variable 
update test

update
 2021-09-22 07:24:18 +00:00
feng lv
dfdf756b6a Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-09-22 02:51:51 +00:00
feng lv
cd6f8078b5 refactor some code 2021-09-22 02:07:59 +00:00
Alexey Milovidov
e9e77b4403 .tech -> .com 2021-09-22 03:22:57 +03:00
Anton Popov
0fbfac056d better diagnostic in ReplicatedMergeTree 2021-09-21 22:42:03 +03:00
Maksim Kita
84e37c2d87 Fixed tests 2021-09-21 21:28:52 +03:00
Maksim Kita
19d70158fe
Merge pull request #29215 from ClickHouse/bloom-filter-index-map-data-type 
Merging #28634
 2021-09-21 20:31:14 +03:00
feng lv
e3fa2e3634 fix 
fix

update tests

update tests
 2021-09-21 16:53:24 +00:00
Nikolai Kochetov
db6f1e198c Add cancell callback to completed executor. 2021-09-21 19:37:32 +03:00
Maksim Kita
6dadc521fb MergeTreeIndexFullText updated for data type map 2021-09-21 16:55:48 +03:00
凌涛
e3e3b84f63 fix tests 2021-09-21 16:55:48 +03:00
凌涛
9461d6dbf2 modify code style 2021-09-21 16:55:47 +03:00
凌涛
e84f1000fb optimze code to make sure only map function will be processed 2021-09-21 16:55:47 +03:00
凌涛
a3e657778a bloom filter skipping index support Map with key of String type 2021-09-21 16:55:07 +03:00
凌涛
8f87d44958 bloom filter skipping index support Map with key of String type 2021-09-21 16:55:07 +03:00
Nikita Mikhaylov
7878d848ab
Merge pull request #29207 from bharatnc/ncb/run-ya-make 
Update outdated ya.make files
 2021-09-21 16:07:56 +03:00
Maksim Kita
15c9315ea1 MergeTreeIndexConditionBloomFilter updated for Map data type 2021-09-21 15:07:59 +03:00
Vladimir C
9abd8aea6d
Merge pull request #29123 from depressed-pho/csv-custom-null 2021-09-21 11:35:21 +03:00
fuwhu
8962672e59 add bloom filter skip index for map data type. 2021-09-21 10:59:16 +03:00
Azat Khuzhin
23b3085c8b Add more tests for external_table_strict_query 2021-09-21 10:53:22 +03:00
bharatnc
51e45050c1 Update outdated ya.make files 
This runs the `utils/generate-ya-make/generate-ya-make.sh` script to
update the ya.make files that are not updated.

I wonder why they were not updated and how it was missed.
 2021-09-20 18:28:07 -07:00
Azat Khuzhin
9c20519f33 Add ability to disable converting expressions to local filter for external queries 
Sometimes it is undesirable to remove any expressions from WHERE, since
this may lead to reading the whole table, and this is pretty heavy job
for MySQL and PostgreSQL (even if you read only one column).

So external_table_strict_query had been introduced to prohibit this and
fail the query instead.
 2021-09-21 02:41:07 +03:00
Nikolai Kochetov
4e7d5191e3 Fix more tests 2021-09-20 22:07:33 +03:00
Anton Popov
55cf857aba
Merge pull request #28707 from azat/max_suspicious_broken_parts_bytes 
Implement max_suspicious_broken_parts_bytes
 2021-09-20 19:55:07 +03:00
Nikolai Kochetov
a790d391c0 Merge branch 'master' into rewrite-pushing-to-views 2021-09-20 19:43:15 +03:00
Nikolai Kochetov
c7533d58ee Add debug output. 2021-09-20 19:09:39 +03:00
Anton Popov
6f9e53197c Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-20 17:17:05 +03:00
alesapin
a249dcc5f5
Merge pull request #28301 from kssenii/materialized-postgresql 
MaterializedPostgreSQL: allow dynamically adding/deleting tables, altering settings
 2021-09-20 14:16:29 +03:00
Maksim Kita
c2088678ed
Merge pull request #28803 from kitaisreal/executable-user-defined-functions 
Executable user defined functions
 2021-09-20 12:11:15 +03:00
Nikolai Kochetov
9398c22aae Fix more tests. 2021-09-20 12:05:34 +03:00
Anton Popov
e3e6258b95
Merge pull request #29163 from CurtizJ/small-refactoring 
Slightly better code near reading of subcolumns
 2021-09-19 16:14:52 +03:00
Anton Popov
f50098a059
fix style check 2021-09-19 00:13:00 +03:00
Anton Popov
1c80a95b36 slightly better code near reading of subcolumns 2021-09-18 22:34:54 +03:00
alexey-milovidov
20d8523a2e
Merge pull request #28888 from azat/mysql-in-fix 
Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )
 2021-09-18 15:53:40 +03:00
feng lv
4fbc7c3f31 remove unused code 2021-09-18 03:01:25 +00:00
Amos Bird
6b3cd34cff
Disable projections when ARRAY JOIN is used. 2021-09-18 08:15:59 +08:00
Alexander Tokmakov
3bca886174 remove some trash 2021-09-17 22:29:10 +03:00
Azat Khuzhin
b108c5e1b0 Implement max_suspicious_broken_parts_bytes 
Usually broken parts are found from just created parts, which are
usually small enough, so it make sense to have a limit in bytes for
broken parts.

Default value was set to 1G.
 2021-09-17 22:06:32 +03:00
Nikolai Kochetov
a8443bef4d Fix build. 2021-09-17 20:52:26 +03:00
Nicolae Vartolomei
9a02061d9c
Rewrite part movement between shards logic and add kill support 
See https://github.com/ClickHouse/ClickHouse/pull/24585#issuecomment-857735081
for extra context about the current implementation.
 2021-09-17 18:11:32 +01:00
Maksim Kita
85a4d4bb50 Added user defined executable functions to system.functions 2021-09-17 18:43:00 +03:00
Maksim Kita
de20e04dfd Added executable user defined functions 2021-09-17 18:42:59 +03:00
feng lv
5f97cad9a4 fix 2021-09-17 15:32:01 +00:00
alesapin
4adc4e9152
Merge pull request #28817 from ClickHouse/fix_part_checker 
Fix rare replicas diverge
 2021-09-17 18:06:27 +03:00
Nicolae Vartolomei
0381c634d4
Add support for user defined identifier on log entries 
Sometimes we want to push a log entry once and only once. Because it is
not possible to create a sequential node in ZooKeeper and store its name
to a well known location in the same transaction we'll do it in the
other order. First somehow generate a unique identifier, then submit a
log entry with that identifier. Later, we can search through log entries
using the identifier we provided to find the node.

Required for part movement between shards.
 2021-09-17 15:32:35 +01:00
Nicolae Vartolomei
c428f433c3
Use do-while control loops for waiting on log processing 
This allows to use the same functions with very short timeouts while
ensuring that the actual state is checked at least once instead of
timing out before even looking at at ZK at least once.
 2021-09-17 15:32:35 +01:00
Vladimir Chebotarev
db516e8c91
Removed sub-optimal mutation notifications in StorageMergeTree when merges are still going (#27552) 
* Removed redundand mutation notifications in `StorageMergeTree` when merges are still going.
 2021-09-17 16:19:36 +03:00
Nikolai Kochetov
a8c3b02598 Merge branch 'master' into rewrite-pushing-to-views 2021-09-17 15:38:11 +03:00
Nikolai Kochetov
6e0ac024a2 Maybe fix some build. 2021-09-17 15:05:54 +03:00
mergify[bot]
c23f1fc160
Merge branch 'master' into fix_part_checker 2021-09-17 07:23:12 +00:00
PHO
5a4985fe27 Do not initialize std::pair with a std::tuple<T1, T2> 
GCC/libstdc++ does not allow this and emits a compilation error. The standard does not say they are supposed to be interchangeable.
 2021-09-17 11:12:05 +09:00
Nikita Mikhaylov
c52b8ec083
Introduced MergeTask and MutateTask (#25165) 
Introduced MergeTask and MutateTask
 2021-09-17 00:19:58 +03:00
Kseniia Sumarokova
e9d44d0848
Merge pull request #28797 from kssenii/rabbit-fix 
Rewrite StorageRabbitMQ a little
 2021-09-16 23:58:51 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
alesapin
2d9fa85c0a
Merge pull request #28889 from azat/fix-mutation-wait 
Fix waiting for mutation with mutations_sync=2
 2021-09-16 19:10:56 +03:00
alesapin
606ab6d3e4 Typo fix 2021-09-16 18:40:25 +03:00
alesapin
8930a0d2bd More precise initialization 2021-09-16 18:39:11 +03:00
Anton Popov
eef436fe22 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-16 18:07:42 +03:00
alesapin
b4cbc86164 Fix bug 2021-09-16 17:59:55 +03:00
kssenii
7b068317c1 Fix build 2021-09-16 13:55:18 +00:00
kssenii
432ca1ed93 Fix tests 2021-09-16 10:46:43 +00:00
kssenii
1650c92407 Better 2021-09-16 11:38:26 +03:00
Nikolai Kochetov
e616732743 Small refactoring. 2021-09-15 22:35:48 +03:00
kssenii
221c09589c Review fixes 2021-09-15 21:11:49 +03:00
kssenii
489a92c067 Review fixes 2021-09-15 18:36:49 +03:00
Anton Popov
f6191b98e7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-15 17:47:22 +03:00
feng lv
dc318000e4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-09-15 14:12:49 +00:00
Nikolay Degterinsky
24e515df05 Merge branch 'master' into query_parameters 2021-09-15 16:15:50 +03:00
Kruglov Pavel
8a67c3cf44
Merge pull request #28636 from amosbird/nullable-index-fix 
Fix nullable/lowcardinality primary key with constant conversion
 2021-09-15 15:56:19 +03:00
kssenii
84c8757049 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-15 12:28:31 +03:00
kssenii
710c8d12dd Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql 2021-09-15 12:28:09 +03:00
Kseniia Sumarokova
fbc048b2bf
Merge branch 'master' into rabbit-fix 2021-09-15 12:11:57 +03:00
kssenii
32c9f76ddc Some fixes 2021-09-15 09:09:24 +00:00
Anton Popov
fc17936c12 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-14 23:02:30 +03:00
mergify[bot]
0cbd4eda60
Merge branch 'master' into fix_part_checker 2021-09-14 19:44:11 +00:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Nikolai Kochetov
7fc830ae2b Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 19:28:41 +03:00
alesapin
5a755d1e7c
Merge pull request #29009 from ClickHouse/improve_error_message 
Better error message on concurrent alters
 2021-09-14 17:21:26 +03:00
mergify[bot]
4993c7c527
Merge branch 'master' into fix_part_checker 2021-09-14 14:13:52 +00:00
Nikolai Kochetov
0e267c50b4 Merge branch 'master' into rewrite-pushing-to-views 2021-09-14 16:13:54 +03:00
Nikita Mikhaylov
58f2504a56 Update ya.make 2021-09-14 12:00:32 +00:00
alesapin
797df22418 Fix build 2021-09-14 14:45:33 +03:00
alesapin
76b8d9d85f Better error message on concurrent alters 2021-09-14 13:57:05 +03:00
alesapin
9441378b39 Add logs to queue because I cannot understand anything without them 2021-09-14 12:04:11 +03:00
feng lv
c58e232385 fix conflict 2021-09-14 06:41:57 +00:00
Kseniia Sumarokova
d3e604e14b
Merge pull request #28603 from kssenii/fix-web-disk-2 
Some fixes for static files disk
 2021-09-14 08:24:24 +03:00
feng lv
19ed9c6dbc fix 2021-09-14 04:50:18 +00:00
feng lv
94ba18e050 fix 2021-09-14 03:15:05 +00:00