thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 10:22:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
124,824 Commits 9,328 Branches 1,783 Tags 2.1 GiB
28b71163b4
Commit Graph

656 Commits

Author SHA1 Message Date
Alexander Gololobov
c0629a38d1 Fix for incorrect result when parallel replicas not completely disabled 2023-07-04 23:50:19 +02:00
Alexander Tokmakov
594ec09edf fix missing metadata_version.txt 2023-06-28 18:19:16 +02:00
Alexey Milovidov
27f41869a9 Remove code that I don't like 2023-06-25 09:11:42 +02:00
Alexey Milovidov
1745535cb1 Add support for ALTER TABLE MODIFY COMMENT 2023-06-23 03:17:45 +02:00
Alexander Tokmakov
b4e455430c
Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-06-17 22:19:21 +03:00
Alexander Tokmakov
b248ba730a
Merge pull request #50997 from vitlibar/use-hash_of_all_files-for-parts 
Use hash_of_all_files to check identity of parts during on-cluster backups
 2023-06-15 16:54:55 +03:00
Vitaly Baranov
6366940a37 Use hash_of_all_files from system.parts to check identity of parts during on-cluster backups. 2023-06-14 20:39:50 +02:00
Anton Popov
d45f07743c fix getting number of mutations 2023-06-12 13:54:07 +00:00
Alexander Tokmakov
46caffd249
Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-06-07 23:30:02 +03:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663) 
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-06-07 20:33:08 +02:00
Alexander Tokmakov
d2aa1779ed Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-06-07 14:58:50 +02:00
Alexander Tokmakov
e148c60d5a
Fixes for MergeTree with readonly disks (#50244) 
* fixes for MergeTree with readonly disks

* Automatic style fix

* Update test.py

* Automatic style fix

* Update test.py

* Update test.py

* Automatic style fix

* Update test.py

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2023-06-02 16:18:32 +03:00
Alexander Tokmakov
82e1d5ae67 Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-05-30 17:40:13 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Alexander Tokmakov
dbf08b25fb better scheduling of merge selecting task 2023-05-23 01:25:17 +02:00
Michael Kolupaev
66cb7f818b Fix flaky tests caused by OPTIMIZE FINAL failing memory budget check 2023-05-16 20:23:55 +00:00
Vitaly Baranov
943707963f Add backup setting "decrypt_files_from_encrypted_disks" 2023-05-16 14:27:27 +02:00
Anton Popov
ff287c7bba add lock 2023-05-10 23:00:49 +00:00
Alexey Milovidov
a8bdb20fc4
Merge pull request #48787 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks [Resubmit]
 2023-05-09 07:58:36 +03:00
Alexey Milovidov
75a2589a42
Merge branch 'master' into deprecate-in-memory-parts 2023-05-06 00:49:59 +03:00
Alexey Milovidov
09f171822e Merge branch 'master' into deprecate-in-memory-parts 2023-05-05 06:43:23 +02:00
Anton Popov
e61d7baa54 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-04 14:03:53 +00:00
Alexander Tokmakov
ff70d26c23
Update StorageMergeTree.cpp 2023-05-03 19:30:44 +03:00
Alexander Tokmakov
f3f6cfdcbb
Update StorageMergeTree.cpp 2023-05-03 17:39:31 +03:00
Alexey Milovidov
f449df85b6 Deprecate in-memory parts 2023-05-03 00:31:09 +02:00
Alexey Milovidov
3fa19c5ee2
Merge branch 'master' into fix_some_tests3 2023-04-27 15:04:15 +03:00
Anton Popov
38622d0770 add settings to delay or throw in case of too many mutations 2023-04-24 18:21:49 +00:00
Alexander Tokmakov
19f43ee7c7 Merge branch 'master' into fix_some_tests3 2023-04-24 16:13:27 +02:00
凌涛
08e81d0a52
Merge branch 'master' into feature/support_settings_allow_suspicious_indices 2023-04-23 10:08:00 +08:00
凌涛
de18bf72b4 modify code 2023-04-21 17:27:26 +08:00
Alexander Tokmakov
70c1000c22 fix some tests 2023-04-14 19:42:17 +02:00
Dmitry Novik
cf5d9a175a Revert "Merge pull request #48760 from ClickHouse/revert-46089-background-memory-tracker" 
This reverts commit a61ed33223, reversing
changes made to 5f01b8a2b5.
 2023-04-14 16:34:19 +02:00
Alexander Gololobov
a7b05589b6 Caches for multiple mutations 2023-04-14 16:12:19 +02:00
Alexander Gololobov
02876ed87d Share PreparedSet-s between task of the same mutation to reduce mem usage 2023-04-14 16:12:18 +02:00
Alexander Tokmakov
af1bf08663
Revert "Add MemoryTracker for the background tasks" 2023-04-13 21:05:02 +03:00
Dmitry Novik
467ecf45e1
Merge pull request #46089 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks
 2023-04-13 13:29:27 +02:00
alesapin
a72ee0b85e Merge remote-tracking branch 'origin/master' into revert-46909-revert-45911-mutations_rename_hang 2023-04-11 20:14:43 +02:00
Dmitry Novik
06e6794fc0 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-04-11 15:29:35 +00:00
Azat Khuzhin
bac7def719 Fix UB (signed integer overflow) in StorageMergeTree::backupData() 
UBsan report:

    /build/src/Storages/StorageMergeTree.cpp:2088:80: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long'
        0 0x30c2ae7b in DB::StorageMergeTree::backupData(DB::BackupEntriesCollector&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::optional<absl::lts_20211102::InlinedVector<std::__1::shared_ptr<DB::IAST>, 7ul, std::__1::allocator<std::__1::shared_ptr<DB::IAST>>>> const&) build_docker/./src/Storages/StorageMergeTree.cpp:2088:80
        1 0x2dd641b5 in DB::BackupEntriesCollector::makeBackupEntriesForTableData(DB::QualifiedTableName const&) build_docker/./src/Backups/BackupEntriesCollector.cpp:703:18
        2 0x2dd5c2a7 in DB::BackupEntriesCollector::makeBackupEntriesForTablesData() build_docker/./src/Backups/BackupEntriesCollector.cpp:676:9
        3 0x2dd5848d in DB::BackupEntriesCollector::run() build_docker/./src/Backups/BackupEntriesCollector.cpp:119:5
        4 0x2dd84da3 in DB::BackupsWorker::doBackup(std::__1::shared_ptr<DB::ASTBackupQuery> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, DB::BackupInfo const&, DB::BackupSettings, std::__1::shared_ptr<DB::IBackupCoordination>, std::__1::shared_ptr<DB::Context const> const&, std::__1::shared_ptr<DB::Context>, bool) build_docker/./src/Backups/BackupsWorker.cpp:359:59
        5 0x2dd82405 in DB::BackupsWorker::startMakingBackup(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Backups/BackupsWorker.cpp:248:13
        6 0x2dd81d0a in DB::BackupsWorker::start(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>) build_docker/./src/Backups/BackupsWorker.cpp:179:16
        7 0x2f4d0d5f in DB::InterpreterBackupQuery::execute() build_docker/./src/Interpreters/InterpreterBackupQuery.cpp:39:30

    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /build/src/Storages/StorageMergeTree.cpp:2088:80 in

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-31 12:29:31 +03:00
Nikita Mikhaylov
301e26b23d
Parallel replicas. Part 2.5 (#47858) 2023-03-22 13:05:38 +01:00
alesapin
8db0d26519
Merge branch 'master' into revert-46909-revert-45911-mutations_rename_hang 2023-03-13 14:48:20 +01:00
Maksim Kita
a762112e15 Analyzer support distributed JOINS and subqueries in IN functions 2023-03-11 11:51:54 +01:00
Dmitry Novik
4f24a7bd11 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-03-10 18:57:45 +00:00
alesapin
4423be8332
Merge branch 'master' into revert-46909-revert-45911-mutations_rename_hang 2023-03-08 12:59:07 +01:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
alesapin
db0f9b6578 Fix one more bug 2023-03-02 14:26:07 +01:00
alesapin
e3961c118a
Revert "Revert "Fix several RENAME COLUMN bugs."" 2023-02-27 12:27:57 +01:00
Alexey Milovidov
8f40bbd5b4
Revert "Fix several RENAME COLUMN bugs." 2023-02-26 05:47:05 +03:00
Alexey Milovidov
eb89ab471f
Merge pull request #45911 from ClickHouse/mutations_rename_hang 
Fix several `RENAME COLUMN` bugs.
 2023-02-26 01:56:40 +03:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog 2023-02-21 11:06:44 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Dmitry Novik
0336f0d8cc
Merge branch 'master' into background-memory-tracker 2023-02-16 20:57:44 +01:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005) 
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-02-16 16:03:16 +03:00
Dmitry Novik
e7a94a28ee
Merge branch 'master' into background-memory-tracker 2023-02-15 19:39:28 +01:00
vdimir
4c1aeb7972
Use overload of PartLog::addNewPart with profile_events 2023-02-14 14:11:12 +00:00
alesapin
808a939ad2 Small rehacktoing 2023-02-10 14:25:19 +01:00
alesapin
fd52979991 Fix snapshot for ordinary merge tree 2023-02-09 14:56:13 +01:00
alesapin
bc58fd156b Merge remote-tracking branch 'origin' into mutations_rename_hang 2023-02-07 20:53:04 +01:00
Dmitry Novik
d43d17f59a Address the review 2023-02-07 16:05:02 +00:00
Dmitry Novik
1e065b32f3 Add MemoryTracker for the background tasks 2023-02-06 17:25:58 +00:00
Alexander Tokmakov
af655eb7dc
Merge pull request #45889 from CheSema/release-merge-blocker-drop-part 
do not block merges when old parts are droping in drop queries
 2023-02-06 19:09:09 +03:00
alesapin
a02e6f3d5b fix bug ordinary merge tree 2023-02-06 14:57:21 +01:00
alesapin
b265300a19 Merge branch 'master' into mutations_rename_hang 2023-02-06 12:25:29 +01:00
alesapin
7a0d0f0c53 Some code which doesn't work 2023-02-03 19:14:49 +01:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log 2023-02-03 20:05:00 +03:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
alesapin
15ddcdc4a6 Fix clang tidy 2023-02-03 12:34:47 +01:00
Frank Chen
d38adfab30 Merge two overridden functions as one 2023-02-03 15:27:45 +08:00
Frank Chen
d3a05a11da Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log 2023-02-03 11:39:43 +08:00
alesapin
b8b0f1c3d3 Make RENAME COLUMN barrier 2023-02-02 21:11:19 +01:00
alesapin
bdc530dead Fix style 2023-02-02 17:30:38 +01:00
alesapin
dea46e58fb Fixes for ordinary merge tree 2023-02-02 17:03:24 +01:00
Sema Checherinda
44be0f41dd
Merge branch 'master' into release-merge-blocker-drop-part 2023-02-02 14:12:40 +01:00
Sema Checherinda
db2aa48c91 do not block merges when old parts are droping in drop queries 2023-02-01 13:30:56 +01:00
Alexander Gololobov
863548114d force_wait flag is no longer needed 2023-01-30 18:38:28 +01:00
Anton Popov
8e3698c91f refactoring of code near merge tree parts 2023-01-25 17:34:09 +00:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Sema Checherinda
962894afc8
Merge pull request #44909 from CheSema/intersect-prev-part 
Do not merge over a gap with outdated undeleted parts
 2023-01-21 11:51:21 +01:00
Sema Checherinda
09f3a5c599 add a comment, add a check, fix test 2023-01-20 12:10:31 +01:00
Frank Chen
f867c4c206 Add an overriden fromCurrentException to simply the code 2023-01-20 16:43:10 +08:00
Frank Chen
38799eb852 Merge two methods as one 2023-01-20 16:35:28 +08:00
Frank Chen
be8ceeb4bb Save stack trace in part_log 2023-01-20 16:22:40 +08:00
Sema Checherinda
a344b526a6
Update src/Storages/StorageMergeTree.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-01-18 15:16:18 +01:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Alexander Tokmakov
72e8615bec formatting of some exception messages 2023-01-17 20:03:56 +01:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern 
Add a column with a message pattern to text_log
 2023-01-17 20:19:32 +03:00
Sema Checherinda
dbe89cd5d8 fix that optimize final waits for currently running merges 2023-01-16 16:47:12 +01:00
Sema Checherinda
90fa1ecd49 make that old_parts_lifetime=0 deletes files instantly at drop/truncate 2023-01-16 16:47:12 +01:00
Sema Checherinda
8f660afab3 style fix 2023-01-16 16:47:12 +01:00
Sema Checherinda
c51f4d7be1 do not merge over a gap with uotdate parts, delete empty parts with respect to old_parts_lifetime 2023-01-16 16:47:11 +01:00
Alexander Tokmakov
2d7773fccc Merge branch 'master' into text_log_add_pattern 2023-01-13 20:33:46 +01:00
Anton Popov
6c4f53bdca Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-11 01:00:57 +00:00
Anton Popov
fcdbc8123d better loading of outdated parts 2023-01-06 11:59:54 +00:00
Alexander Gololobov
2c64f0042a Added force_wait parameter to IStorage::mutate() 2022-12-30 16:23:03 +01:00
Anton Popov
2adf0e9db3 do not wait loading parts at shutdown 2022-12-28 16:07:14 +00:00
Alexander Tokmakov
ca989e9212 less runtime format strings 2022-12-23 19:50:34 +01:00
Anton Popov
0722020cae Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-23 18:15:55 +00:00
alesapin
2fecf420cb
Merge pull request #44513 from ClickHouse/small_polishing 
Minor code polishing
 2022-12-23 13:47:26 +01:00
alesapin
a32fab90d5
Merge pull request #44436 from ClickHouse/fix_bug_with_max_task_count_update 
Fix bug with merge/mutate pool size increase
 2022-12-22 23:48:54 +01:00
alesapin
fa467b3398 Minor code polishing 2022-12-22 14:31:42 +01:00