thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
103,945 Commits 9,326 Branches 1,783 Tags 2.1 GiB
ffaff7167b
Commit Graph

138 Commits

Author SHA1 Message Date
Nikolay Degterinsky
e03bf0c526 Merge branch 'master' into query_parameters 2021-10-04 17:23:17 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Kevin Michel
aa3f4003c8
Fix hanging DDL queries on Replicated database 
There was a race condition when issuing a DDL query on a replica just
after a new replica was added.

If the DDL query is issued after the new replica adds itself to the
list of replicas, but before the new replica has finished its
recovery, then the first replica adds the new replica to the list of
replicas to wait to confirm the query was replicated.

Meanwhile, the new replica is still in recovery and applies queries
from the /metadata snapshot. When it's done, it bumps its log_ptr
without marking the corresponding log entries (if any) as finished.

The first replica then waits until distributed_ddl_task_timeout
expires and wrongly assumes the query was not replicated.

The issue is fixed by remembering the max_log_ptr at the exact point
where the replica adds itself to the list of replicas, then mark as
finished all queries that happened between that max_log_ptr and the
max_log_ptr of the metadata snapshot used in recovery.

The bug was randomly observed during a downstream test. It can be
reproduced more easily by inserting a sleep of a few seconds at the
end of createReplicaNodesInZooKeeper, enough to have time to issue a
DDL query on the first replica.
 2021-09-24 12:51:52 +02:00
Nikolay Degterinsky
1b1d90deb8 Merge branch 'master' into query_parameters 2021-09-22 13:53:56 +03:00
Alexander Tokmakov
df56e99b87 fixes after review 2021-09-13 22:11:16 +03:00
Nikolay Degterinsky
22f59e1714 Add support for table and database query parameters 2021-09-07 01:19:02 +03:00
Alexander Tokmakov
c8d8f0a38c fix 2021-08-31 15:07:00 +03:00
Amos Bird
de9e591619
Fast load by delaying table startup 2021-08-30 10:15:32 +08:00
Mikhail Filimonov
af6249c8ec
Less include <Common/Stopwatch.h> 2021-08-11 16:20:59 +02:00
Alexander Tokmakov
ff2572c243 fix 2021-08-03 16:28:03 +03:00
Alexander Tokmakov
42a8bb6872 fix assertions in Replicated database 2021-08-02 16:19:11 +03:00
Raúl Marín
383c982715 CH local: Treat localhost:port as a remote database 2021-07-23 13:16:35 +02:00
tavplubix
e2941a8006
Update DatabaseReplicated.cpp 2021-06-01 15:15:13 +03:00
Alexander Tokmakov
94d2aed336 fix tests 2021-06-01 13:24:06 +03:00
Alexander Tokmakov
fe482109da Merge branch 'master' into fix_database_replicated_enqueue_race 2021-06-01 11:49:29 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs 
Poco::File to std::filesystem
 2021-05-31 23:17:02 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Alexander Tokmakov
0035997e02 fix race on enqueue query 2021-05-31 16:31:03 +03:00
kssenii
3dee003f9b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-20 19:20:09 +03:00
Azat Khuzhin
23981ffd0e Introduce IStorage::flushAndShutdown() 2021-05-16 13:23:53 +03:00
kssenii
ab1a05a1f4 Poco::Path to fs::path, less concatination 2021-05-09 14:59:49 +03:00
Maksim Kita
c93666e0de Fixed code review issues 2021-04-26 13:52:41 +03:00
Maksim Kita
a9917a6540 Refactored StorageDictionary 2021-04-26 13:52:41 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Alexander Tokmakov
a5b07f2f03 Merge branch 'master' into test_multiple_nodes 2021-03-29 22:43:25 +03:00
tavplubix
e3cf2d99b7
Update DatabaseReplicated.cpp 2021-03-29 16:44:41 +03:00
Alexander Tokmakov
fdae70df6c fix tests 2021-03-18 15:50:03 +03:00
Alexander Tokmakov
ff79be8af9 fix 2021-03-16 23:01:20 +03:00
tavplubix
6ff8f468f6
Update DatabaseReplicated.cpp 2021-03-15 11:22:42 +03:00
Alexander Tokmakov
ba3d51383c fix 2021-03-10 00:41:04 +03:00
Alexander Tokmakov
83b3e4e0f5 allow to use Cluster from Replicated db 2021-03-09 20:05:24 +03:00
Alexander Tokmakov
00b939e5a3 fix 2021-03-09 13:24:08 +03:00
Alexander Tokmakov
5c5d11cfce enable some tests 2021-03-09 03:48:37 +03:00
Alexander Tokmakov
163e27b299 support query and session settings for distributed DDL 2021-03-09 01:57:53 +03:00
Alexander Tokmakov
5070b5b85b detach permanently setting for Replicated db 2021-03-09 00:34:52 +03:00
Alexander Tokmakov
2022b90919 configurable output mode for distributed DDL 2021-03-08 23:35:09 +03:00
Alexander Tokmakov
2484781070 fix MV recovery in Replicated db 2021-03-08 20:26:38 +03:00
Alexander Tokmakov
2a36d6cb55 review suggestions 2021-02-20 02:41:58 +03:00
Alexander Tokmakov
cf57c3b4a2 update comments 2021-02-15 23:01:21 +03:00
Alexander Tokmakov
9c7cf9e92e remove some debug code 2021-02-15 13:26:34 +03:00
Alexander Tokmakov
320fd6b264 startup without zk 2021-02-15 03:04:46 +03:00
Alexander Tokmakov
f64f9b672b fix 2021-02-12 19:22:01 +03:00
Alexander Tokmakov
c9cf63e958 fix 2021-02-12 01:23:40 +03:00
Alexander Tokmakov
9afb16759e fix 2021-02-11 18:14:38 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Alexander Tokmakov
2c6a0e74fb better replica creation 2021-02-09 18:14:20 +03:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
Alexander Tokmakov
91d0924665 write dictionaries metadata to zk 2021-02-08 12:46:30 +03:00
Alexander Tokmakov
7ce0ef2561 show clusters for replicated db 2021-02-08 12:14:17 +03:00
Alexander Tokmakov
18f6b5bbad add timeouts 2021-02-04 22:41:44 +03:00
Alexander Tokmakov
9da445e740 execute initial query in the same thread 2021-02-01 22:29:47 +03:00
Alexander Tokmakov
7f97a11c84 Merge branch 'master' into database_replicated 2021-01-18 17:09:39 +03:00
Alexander Tokmakov
1a4bd67736 fixes 2020-12-01 20:20:42 +03:00
Alexander Tokmakov
ab197a49c8 better code, fixes 2020-11-30 23:12:43 +03:00
Alexander Tokmakov
f1a52a609b separate DatabaseReplicatedDDLWorker 2020-11-27 17:04:03 +03:00
Alexander Tokmakov
dad21ee684 maintain metadata in zk 2020-11-24 13:24:39 +03:00
Alexander Tokmakov
7ab4445e99 try another approach 2020-11-20 19:06:27 +03:00
Alexander Tokmakov
2283906a11 try support replica recovery 2020-11-19 13:34:45 +03:00
Alexander Tokmakov
b0262b3d06 better replica creation 2020-11-13 21:35:45 +03:00
Alexander Tokmakov
2a6c0b9180 try reuse DDLWorker in DatabaseReplicated 2020-11-12 20:35:29 +03:00
Alexander Tokmakov
d8ae9fcdb4 fixes, add shard name 2020-11-03 14:30:53 +03:00
Alexander Tokmakov
cd14f095ab fix tests 2020-10-26 18:12:16 +03:00
Alexander Tokmakov
478eb0b8a5 fix 2020-10-22 23:32:47 +03:00
Alexander Tokmakov
72dbb43235 Merge branch 'master' into fork-valbaturin-replication-db-engine 2020-10-20 19:14:54 +03:00
Val
e591fe5014 database replicated feedback mechanism prototype 2020-07-04 19:32:23 +03:00
Val
8273248c4e add log_name_to_exec to dbreplicated 2020-06-28 18:43:30 +03:00
Val
e23c7a313e address pr comments 2020-06-27 17:04:12 +03:00
Val
d293e002a7 address pr comments 2020-06-27 17:04:11 +03:00
Val
16e50e33d7 fix typo 2020-06-27 17:04:11 +03:00
Val
67588edcf5 clean up db replicated files and add more tests 2020-06-27 17:04:11 +03:00
Val
82f5281cfe remove redundant includes 2020-06-27 17:04:11 +03:00
Val
4fc4b1d195 db replicated minor enhancements 2020-06-27 17:02:06 +03:00
Val
f928c897cf change replication algorithm, remove zk lock 
In this version of the databaseReplicated sequential persistent zk nodes
are used to order DDL queries. Db replicated ddl queries are executed
in the backgrould pool no matter whether it's proposed by the same
replica or not.
 2020-06-27 17:02:03 +03:00
Val
469f9738df refactor save state in db replicated 2020-06-27 17:01:20 +03:00
Val
a0af67b636 Add one more test for db replicated and fix related bug 2020-06-27 17:01:20 +03:00
Val
31910e9bf1 Use ClientInf::QueryKind to distinguish replicated db log queries 2020-06-27 17:01:18 +03:00
Val
4921dc6dab db replicated refactoring 2020-06-27 16:59:11 +03:00
Val
5e076b464e add replicated db snapshot, integration test, repl alter queries, etc 
add an option to create replicated tables within replicated db without specifying zk path and replica id
add replicated sch pool
disable replication of alter queries for replicated tables in replicated dbs
snapshot prototype. amend of replicated db workflow
add prototype of integration tests for replicated db
 2020-06-27 16:58:15 +03:00
Val
f103e24a09 make db replicated inherited from atomic 2020-06-27 16:52:35 +03:00
Val
d7a354b24d create query fix for replicated dbs 2020-06-27 16:51:47 +03:00
Val
d61259cd7b ddl replication works 2020-06-27 16:51:47 +03:00
Val
5eea58039c fix not initialized last entry in zk 2020-06-27 16:51:47 +03:00
Val
0a860c0c2b log based replicated 2020-06-27 16:51:36 +03:00
Val
319256ef4f an attempt to replicated create query from create query 2020-06-27 16:49:40 +03:00
Val
0a4c1783a1 Make drop work by fixing namespace bug 
data dir wasn't set right. now it's fixed.
add non-replicated table to test sql
 2020-06-27 16:49:40 +03:00
Val
8b0366ff4f an attempt to make something meaningful 2020-06-27 16:49:40 +03:00
Val
0d392bbb34 fix after rebase 2020-06-27 16:49:39 +03:00
Val
d4210d04c1 databasereplicated constructor scratch 2020-06-27 16:49:39 +03:00