thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
114,887 Commits 9,328 Branches 1,783 Tags 2.1 GiB
5a44dc26e7
Commit Graph

1458 Commits

Author SHA1 Message Date
Alexey Milovidov
e6b44756dc Fixed build 2023-02-12 07:45:28 +01:00
Alexey Milovidov
e2079b78a3 [RFC] Revert #39737 2023-02-12 07:24:42 +01:00
alesapin
5c83ddabf9 Merge branch 'master' into mutations_rename_hang 2023-02-11 12:18:58 +01:00
robot-ch-test-poll2
4e9d1275c7
Merge pull request #46278 from CheSema/race-check-table 
fix data race between check table request and background checker
 2023-02-11 09:50:48 +01:00
Sema Checherinda
ef133ee30d fix data race between check table query and background part checker thread 2023-02-10 16:22:35 +01:00
alesapin
808a939ad2 Small rehacktoing 2023-02-10 14:25:19 +01:00
Robert Schulze
566fb5b9da
Merge pull request #46006 from ClickHouse/rs/fix-fragile-linking 
Fix terribly broken, fragile and potentially cyclic linking
 2023-02-09 18:19:44 +01:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias 2023-02-09 16:07:45 +01:00
alesapin
9b23068d5e
Merge pull request #45821 from ClickHouse/explicit_drop_part_entry_type 
Make separate DROP_PART log entry type
 2023-02-09 15:11:13 +01:00
Robert Schulze
10af0b3e49
Reduce redundancies 2023-02-07 12:27:23 +00:00
Smita Kulkarni
fb765696b6 Removed state_mutex lock from addSubscriber - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 12:19:58 +01:00
Smita Kulkarni
135615cda6 Removed unwanted check from before callback - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 10:27:45 +01:00
Smita Kulkarni
ef29b61786 Updated callback to return log entry ids - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 10:02:38 +01:00
Smita Kulkarni
f34ef86332 Moved background task trigger before getting log entries to wait - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 08:49:41 +01:00
alesapin
dd3a98e88d Fixes 2023-02-06 22:30:31 +01:00
Dmitry Novik
1e065b32f3 Add MemoryTracker for the background tasks 2023-02-06 17:25:58 +00:00
Alexander Tokmakov
86feaae253 Merge branch 'master' into explicit_drop_part_entry_type 2023-02-06 14:59:59 +01:00
alesapin
b94f9f8006
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-02-06 12:38:39 +01:00
alesapin
b265300a19 Merge branch 'master' into mutations_rename_hang 2023-02-06 12:25:29 +01:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking 
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
 2023-02-05 20:49:34 +00:00
alesapin
51b6154d68 Fix stupid bug 2023-02-03 20:56:09 +01:00
alesapin
f91c10d09d Initial support for version 2023-02-03 20:49:07 +01:00
alesapin
7a0d0f0c53 Some code which doesn't work 2023-02-03 19:14:49 +01:00
Antonio Andelic
8929838fcc Merge branch 'master' into add-support-for-settings-alias 2023-02-03 17:47:00 +00:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log 2023-02-03 20:05:00 +03:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
Smita Kulkarni
123f3cfe1f Used unordered_set for log entries and fixed the callback function - Updated checking of SYSTEM SYNC REPLICA 2023-02-03 08:48:16 +01:00
Frank Chen
d38adfab30 Merge two overridden functions as one 2023-02-03 15:27:45 +08:00
Frank Chen
d3a05a11da Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log 2023-02-03 11:39:43 +08:00
alesapin
bdc530dead Fix style 2023-02-02 17:30:38 +01:00
Antonio Andelic
f613bfd8d2 Merge branch 'master' into add-support-for-settings-alias 2023-02-02 11:15:39 +00:00
Sema Checherinda
41a1dddc7a
Merge branch 'master' into better_postpone_reason 2023-02-01 15:16:34 +01:00
Alexander Gololobov
e705f11ad1
Merge pull request #45786 from ClickHouse/delete_on_cluster 
Support DELETE ON CLUSTER
 2023-02-01 10:52:01 +01:00
Antonio Andelic
ec04cf6876 Merge branch 'master' into add-support-for-settings-alias 2023-02-01 09:01:01 +00:00
Alexander Tokmakov
7862990721 Merge branch 'master' into explicit_drop_part_entry_type 2023-01-31 13:49:04 +01:00
Alexander Tokmakov
2e8100b6e4 make separate DROP_PART log entry type 2023-01-31 13:37:56 +01:00
Alexander Tokmakov
e42cd3163f fix race condition on a part check cancellation 2023-01-30 18:48:20 +01:00
Alexander Gololobov
863548114d force_wait flag is no longer needed 2023-01-30 18:38:28 +01:00
Smita Kulkarni
6cf2498609 Fixed clang-tidy build for comparing strings and updated to notify after adding subscriber if queue is already empty- Updated checking of SYSTEM SYNC REPLICA 2023-01-30 08:18:20 +01:00
SmitaRKulkarni
b22fd40d92
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-27 15:00:47 +01:00
Alexander Tokmakov
f375b1246f
Merge pull request #45363 from ClickHouse/fix_hang_during_drop_in_zero_copy_replication 
Fix no shared id during drop for the fourth time
 2023-01-27 15:59:27 +03:00
Smita Kulkarni
39966457d9 Updated to remove waiting ids till the removed log entry - Updated checking of SYSTEM SYNC REPLICA 2023-01-27 13:36:44 +01:00
alesapin
716d25c33f
Merge pull request #45619 from CurtizJ/refactoring-merge-tree-parts 
Refactoring of code near merge tree parts
 2023-01-27 13:36:33 +01:00
Smita Kulkarni
a95609e0bc Updated to validate & remove each element instead of just last element - Updated checking of SYSTEM SYNC REPLICA 2023-01-27 10:35:12 +01:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
alesapin
84af5eb58b Fxi 2023-01-26 11:42:53 +01:00
alesapin
d5036f2ebc Merge branch 'fix_hang_during_drop_in_zero_copy_replication' of github.com:ClickHouse/ClickHouse into fix_hang_during_drop_in_zero_copy_replication 2023-01-26 11:42:30 +01:00
alesapin
1f61abce46 One useful logging line 2023-01-26 11:41:43 +01:00
Smita Kulkarni
8c885ffeae Updated checking of SYSTEM SYNC REPLICA 
Implementation:
* Updated to wait for current last entry to be processed (after pulling shared log) instead of queue size becoming 0.
* Updated Subscriber to notify both queue size and removed log_entry_id.
 2023-01-26 11:40:09 +01:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Alexander Tokmakov
92c62e2de2
Update src/Storages/StorageReplicatedMergeTree.cpp 2023-01-26 00:28:46 +03:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
alesapin
d0428a9b42 Fxi 2023-01-25 20:30:19 +01:00
Anton Popov
8e3698c91f refactoring of code near merge tree parts 2023-01-25 17:34:09 +00:00
alesapin
55af44ca2f Better fix 2023-01-25 14:40:39 +01:00
alesapin
96d8546b1c Merge branch 'master' into fix_hang_during_drop_in_zero_copy_replication 2023-01-25 13:07:14 +01:00
Alexander Tokmakov
6621834e37 fix 2023-01-24 17:10:54 +01:00
Alexander Tokmakov
6ecae8388e Merge branch 'master' into exception_message_patterns4 2023-01-24 14:42:36 +01:00
Alexey Milovidov
dda6a69f9a
Merge branch 'master' into tavplubix-patch-1 2023-01-24 13:04:25 +03:00
Alexey Milovidov
54bc501d39
Update StorageReplicatedMergeTree.cpp 2023-01-24 13:03:28 +03:00
Alexander Tokmakov
414693feb2 fixes 2023-01-24 00:46:03 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Alexander Tokmakov
910d6dc0ce
Merge pull request #45342 from ClickHouse/exception_message_patterns 
Save message format strings for DB::Exception
 2023-01-20 18:46:52 +03:00
Frank Chen
f867c4c206 Add an overriden fromCurrentException to simply the code 2023-01-20 16:43:10 +08:00
Frank Chen
38799eb852 Merge two methods as one 2023-01-20 16:35:28 +08:00
Frank Chen
be8ceeb4bb Save stack trace in part_log 2023-01-20 16:22:40 +08:00
Frank Chen
1fe9bde594 Set replica name to zero copy exclusive lock 2023-01-20 15:48:33 +08:00
Alexander Tokmakov
7bb65cc002
Update StorageReplicatedMergeTree.cpp 2023-01-19 16:45:41 +03:00
Han Fei
f661dad0e9
Merge pull request #45106 from hanfei1991/hanfei/async-cache 
support cache for async inserts block ids
 2023-01-19 10:59:25 +01:00
Vitaly Baranov
7cdb2c4c7f
Merge pull request #45351 from vitlibar/fix-backup-with-killed-mutations 
Fix backup with killed mutations
 2023-01-18 13:14:27 +01:00
Sergei Trifonov
c443c1ece0
Merge branch 'master' into hanfei/async-cache 2023-01-18 00:19:49 +01:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Alexander Tokmakov
72e8615bec formatting of some exception messages 2023-01-17 20:03:56 +01:00
alesapin
e732f510f0
Merge branch 'master' into fix_hang_during_drop_in_zero_copy_replication 2023-01-17 19:24:36 +01:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern 
Add a column with a message pattern to text_log
 2023-01-17 20:19:32 +03:00
alesapin
69925647eb Fix style 2023-01-17 15:59:55 +01:00
alesapin
f6131101bb Fix no shared id during drop for the fourth time 2023-01-17 15:51:49 +01:00
Vitaly Baranov
692065e5fe Fix backup if mutations got killed during the backup process. 2023-01-17 11:05:34 +01:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Anton Popov
6863cd152f
Merge pull request #42181 from CurtizJ/optimize-loading-parts 
Do not load inactive parts at startup
 2023-01-16 14:38:50 +01:00
Alexander Tokmakov
2d7773fccc Merge branch 'master' into text_log_add_pattern 2023-01-13 20:33:46 +01:00
Han Fei
bcf813fedc Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
 2023-01-13 20:19:30 +01:00
Han Fei
a258a39eb1 Merge branch 'master' into hanfei/async-cache 2023-01-13 20:17:58 +01:00
Alexander Tokmakov
9d5ec474a3
Merge pull request #43998 from evillique/make_system_replicas_parallel 
Make `system.replicas` parallel
 2023-01-13 16:33:36 +03:00
Alexander Tokmakov
51d94314d6
Merge pull request #45235 from ClickHouse/more_verbose_logs_about_replication_log_entries 
More verbose logs about replication log entries
 2023-01-13 15:05:21 +03:00
Raúl Marín
9f121d5d9c
Fix background_fetches_pool_size config reload (#45189) 2023-01-12 23:41:25 +01:00
Alexander Tokmakov
d9b8cf9c04 more verbose logs about replication log entries 2023-01-12 20:32:39 +01:00
Anton Popov
6c4f53bdca Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-11 01:00:57 +00:00
Alexander Tokmakov
881b17492f Merge branch 'master' into fix_get_part_name 2023-01-10 21:39:35 +01:00
Han Fei
bef6168c05 support cache for async inserts block ids 2023-01-10 13:19:12 +01:00
alesapin
616e37ba43
Merge pull request #44847 from ClickHouse/fix_rare_race 
Fix rare race which can lead to queue hang
 2023-01-10 12:09:10 +01:00
Alexander Gololobov
742887ce53
Merge pull request #44718 from ClickHouse/lwd_mutation_always_sync 
Make lightweight deletes always synchronous
 2023-01-09 15:58:02 +01:00
Anton Popov
1e5d6e44a5 fix deadlock at shutdown 2023-01-09 13:07:46 +00:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] (#44955) 
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)""

This reverts commit b8d9066004.

* Fix build

* Better

* Fix test

* Automatic style fix

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-01-09 13:30:32 +01:00
alesapin
da83e97b4c Fix startup issue 2023-01-06 15:36:53 +01:00
Anton Popov
fcdbc8123d better loading of outdated parts 2023-01-06 11:59:54 +00:00
Anton Popov
ed00399eb6 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-05 14:12:02 +00:00
Nikolay Degterinsky
673ba769e5 Merge remote-tracking branch 'upstream/master' into make_system_replicas_parallel 2023-01-05 02:36:07 +00:00
alesapin
2a5b4067e4 Review fixes 2023-01-04 15:15:14 +01:00
alesapin
8ddda46d1b
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-01-04 14:59:46 +01:00
alesapin
39331e07d5 Improve logging 2023-01-02 17:05:08 +01:00
alesapin
6fc63878b4 Fix rare race which can lead to queue hang 2023-01-02 16:57:25 +01:00
Alexander Gololobov
2c64f0042a Added force_wait parameter to IStorage::mutate() 2022-12-30 16:23:03 +01:00
alesapin
5dfefb43a3 Fix deadlock in attach thread 2022-12-29 18:52:20 +01:00
Anton Popov
2adf0e9db3 do not wait loading parts at shutdown 2022-12-28 16:07:14 +00:00
Alexander Tokmakov
ca989e9212 less runtime format strings 2022-12-23 19:50:34 +01:00
Anton Popov
0722020cae Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-23 18:15:55 +00:00
alesapin
2fecf420cb
Merge pull request #44513 from ClickHouse/small_polishing 
Minor code polishing
 2022-12-23 13:47:26 +01:00
alesapin
fa467b3398 Minor code polishing 2022-12-22 14:31:42 +01:00
Nikolay Degterinsky
b9d0d25eb2 Better StorageReplicatedMergeTree::getStatus() 2022-12-21 02:11:25 +00:00
Alexander Tokmakov
cdc3912743 fix incorrect usages of getPartName() 2022-12-20 22:44:27 +01:00
Alexander Tokmakov
3f851d5a09 parallel parts cleanup with zero copy replication 2022-12-20 22:04:26 +01:00
alesapin
2818520572 Fix bug with merge/mutate pool size increase 2022-12-20 14:51:11 +01:00
Han Fei
623af800c2
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-12-14 14:29:04 +01:00
Han Fei
34a589a7d8 create async_blocks zk path for old replicated tables and add a flag "async_insert_deduplicate" 2022-12-14 14:06:12 +01:00
Alexander Tokmakov
371e85d854
Merge pull request #44157 from ClickHouse/fix_session_expired_on_startup 
Ignore "session expired" errors after BC check
 2022-12-13 15:10:36 +03:00
Anton Popov
0b396bab14 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-12 18:08:27 +00:00
Alexander Tokmakov
469406c494 ignore session expired errors after BC check 2022-12-12 14:02:14 +01:00
Han Fei
6740ae9fd7
Merge pull request #43304 from hanfei1991/hanfei/aysnc-insert 
Support dedup for async insert.
 2022-12-09 12:50:48 +01:00
Han Fei
3c11aca040 address comments 2022-12-07 23:40:52 +01:00
Anton Popov
df6d2f2163 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-29 23:21:23 +00:00
Alexander Tokmakov
58d8025a15
Update StorageReplicatedMergeTree.cpp 2022-11-29 16:42:15 +03:00
Alexander Tokmakov
0342e6b236 fix 2022-11-28 16:02:37 +01:00
Alexander Tokmakov
a47e0b8f61 Merge branch 'master' into merge_predicate_uncommitted_blocks_hint 2022-11-28 15:45:34 +01:00
Alexander Tokmakov
f4944664b7 fix 2022-11-25 18:53:46 +01:00
Han Fei
ee5be80d9f Merge branch 'master' into hanfei/aysnc-insert 2022-11-25 17:27:56 +01:00
Alexander Tokmakov
5cc99312d7 provide hint for loading uncommitted blocks in merge predicate 2022-11-25 16:41:20 +01:00
Alexey Milovidov
69d1be50bc
Merge pull request #43446 from ClickHouse/fix-typo-3 
Fix typo
 2022-11-25 15:22:28 +01:00
Sema Checherinda
15a6ce254b
Merge pull request #41145 from CheSema/lock-free-drop-partition 
Delete parts by replacing them with empty parts
 2022-11-25 12:54:30 +01:00
Anton Popov
c65be55da1 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-24 14:30:23 +00:00
Anton Popov
b05a20da1f fix drop in case of zero-copy 2022-11-24 14:29:30 +00:00
Han Fei
28564df6f1
Merge branch 'master' into hanfei/aysnc-insert 2022-11-24 09:12:43 +01:00
Anton Popov
f4d91cd722 fix drop with zero-copy 2022-11-24 03:31:04 +00:00
Sema Checherinda
fb86019db9 Delete parts over replacing them with empty parts 2022-11-23 15:16:08 +00:00
Anton Popov
44b4345996 fix wait in drop 2022-11-23 04:21:27 +00:00
Anton Popov
e3db29aebf better waiting for outdated data parts 2022-11-22 15:15:20 +00:00
Han Fei
32c7fbec55 use another dedup path for async inserts 2022-11-22 15:12:00 +01:00
Alexander Tokmakov
a7efdd7f4e
Update StorageReplicatedMergeTree.cpp 2022-11-21 21:52:48 +03:00
Anton Popov
92ac9385b9 wait loading of outdated parts for partition commands 2022-11-21 17:01:46 +00:00
Alexey Milovidov
56f8c0b84b Fix typo 2022-11-21 15:10:15 +01:00
Han Fei
b80a2d6c89 address comments 2022-11-18 17:22:05 +01:00
Han Fei
14701670b5 Merge branch 'master' into hanfei/aysnc-insert 2022-11-16 17:24:36 +01:00
Antonio Andelic
11aba7597e Check if initialization is done on write 2022-11-16 13:33:22 +00:00
Antonio Andelic
0e5e58bed9 Remove exception if shared ID already created 2022-11-15 12:18:24 +00:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered 2022-11-14 18:23:16 +03:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Igor Nikonov
379113f8f1 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-09 13:03:32 +00:00