thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
102,531 Commits 9,328 Branches 1,783 Tags 2.1 GiB
6979dc9f2f
Commit Graph

1034 Commits

Author SHA1 Message Date
kssenii
bc228f4010 Better way to drop nested table for single storage 2021-04-10 19:26:53 +00:00
kssenii
1c501e7d97 Support replica identity index 2021-04-10 16:10:28 +00:00
kssenii
12f98e8b11 Much better 2021-04-09 10:22:06 +00:00
kssenii
f0be5c6938 Fix style, build, ya check 2021-04-09 08:07:56 +00:00
kssenii
3e41c82df1 Add mutex to tryGetTable only for materialized table (not nested) 2021-04-08 23:38:27 +00:00
kssenii
bd207daa74 Use only atomic database 2021-04-08 22:38:46 +00:00
kssenii
4482a35a3a Check table in a suggested way 2021-04-08 20:39:16 +00:00
kssenii
8586565ce9 Move connection-related filed to Core/PostgreSQL 2021-04-08 12:43:02 +00:00
kssenii
9fa21a6272 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-08 12:18:54 +00:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Kseniia Sumarokova
ad85467128
Merge pull request #22668 from kssenii/postgres-add-bool 
Add bool type in postgres engine
 2021-04-06 13:29:33 +03:00
kssenii
cc2fdc6c62 Add bool type 2021-04-05 14:49:52 +00:00
kssenii
13d1f17d3e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-03 08:00:09 +00:00
kssenii
2de6b99f84 Better names 2021-04-02 16:12:14 +00:00
kssenii
99f0b08ac3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-01 13:48:33 +00:00
kssenii
522d170588 Add default port 2021-04-01 11:30:20 +00:00
kssenii
747967a9bb Better 2021-04-01 10:27:24 +00:00
kssenii
ba5ddd5c32 Do not parse replicas in PoolWithFailover 2021-03-31 14:03:52 +00:00
Maksim Kita
55984e849c
Merge branch 'master' into hierarchy-dictionaries-updated 2021-03-31 15:55:07 +03:00
kssenii
ce05087b1b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-03-30 17:27:10 +00:00
Alexander Tokmakov
a5b07f2f03 Merge branch 'master' into test_multiple_nodes 2021-03-29 22:43:25 +03:00
tavplubix
3c0f5a57ec
Merge pull request #21535 from ClickHouse/distributed_ddl_improvements 
Distributed DDL improvements
 2021-03-29 22:40:11 +03:00
tavplubix
e3cf2d99b7
Update DatabaseReplicated.cpp 2021-03-29 16:44:41 +03:00
kssenii
22b515fbc9 Add namespace, simplify names 2021-03-27 20:14:02 +00:00
kssenii
f40c582e7a Replicas/shards for postgres storage 2021-03-27 20:11:48 +00:00
kssenii
ae868208c2 Use pool with failover in mysql storage 2021-03-27 14:39:45 +00:00
Maksim Kita
9f2f0d1095 Refactored hierarchy dictionaries interface 2021-03-26 18:42:32 +03:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements 2021-03-22 18:13:19 +03:00
kssenii
ed158e2b77 Proper setting 2021-03-20 14:42:03 +00:00
kssenii
4c37ff3fbc Less lambdas 2021-03-20 13:17:30 +00:00
kssenii
91b79f95bc Better templates 2021-03-20 12:24:35 +00:00
kssenii
caffccd73e Fixes 2021-03-20 11:53:41 +00:00
kssenii
6bb81630e5 Small fixes 2021-03-20 10:32:11 +00:00
kssenii
619f0f51fd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-20 09:47:05 +00:00
kssenii
3b32c8fe9a Block on empty connection pool 2021-03-19 21:00:45 +00:00
kssenii
9057aad798 Better version 2021-03-19 08:11:36 +00:00
kssenii
f1ef87d966 Fix 2021-03-18 20:04:54 +00:00
Alexander Tokmakov
ce31d4608b fix tests 2021-03-18 22:13:21 +03:00
Alexander Tokmakov
343b8c906d Merge branch 'master' into test_multiple_nodes 2021-03-18 16:01:04 +03:00
Alexander Tokmakov
fdae70df6c fix tests 2021-03-18 15:50:03 +03:00
kssenii
3903d59d30 Better 2021-03-17 14:34:04 +00:00
kssenii
ae64a24844 Add connection pool 2021-03-17 13:55:47 +00:00
Nikita Mikhaylov
f51b41b7a4
Merge pull request #21710 from kssenii/replica-priorities 
Support replicas priority for postgres dictionary source
 2021-03-17 16:02:00 +03:00
kssenii
87c740730b Rename to MaterializePostgreSQL 2021-03-17 10:52:12 +00:00
Alexander Tokmakov
ff79be8af9 fix 2021-03-16 23:01:20 +03:00
kssenii
f8a99804c1 Add double quotes 2021-03-16 11:15:43 +00:00
kssenii
a03e849a8f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-16 10:06:10 +00:00
tavplubix
6a265f468e
Merge branch 'master' into distributed_ddl_improvements 2021-03-15 11:24:14 +03:00
tavplubix
6ff8f468f6
Update DatabaseReplicated.cpp 2021-03-15 11:22:42 +03:00
kssenii
8eea6a87e9 Support replicas priority for postgres dictionary source 2021-03-14 08:51:24 +00:00
tavplubix
26ebd54eaf
Merge pull request #20961 from havardk/reconnect 
MaterializeMySQL: Attempt to reconnect to MySQL if the connection is lost
 2021-03-10 20:07:11 +03:00
Alexander Tokmakov
ba3d51383c fix 2021-03-10 00:41:04 +03:00
tavplubix
5a6cc876e4
Merge pull request #20626 from ClickHouse/fix_force_drop_materialized_view 
Make force_drop_table work with MaterializedView
 2021-03-09 20:05:51 +03:00
Alexander Tokmakov
83b3e4e0f5 allow to use Cluster from Replicated db 2021-03-09 20:05:24 +03:00
Alexander Tokmakov
00b939e5a3 fix 2021-03-09 13:24:08 +03:00
Alexander Tokmakov
5c5d11cfce enable some tests 2021-03-09 03:48:37 +03:00
Alexander Tokmakov
163e27b299 support query and session settings for distributed DDL 2021-03-09 01:57:53 +03:00
Alexander Tokmakov
5070b5b85b detach permanently setting for Replicated db 2021-03-09 00:34:52 +03:00
Alexander Tokmakov
2022b90919 configurable output mode for distributed DDL 2021-03-08 23:35:09 +03:00
Alexander Tokmakov
2484781070 fix MV recovery in Replicated db 2021-03-08 20:26:38 +03:00
Alexander Tokmakov
e8987f799e Merge branch 'fix_force_drop_materialized_view' into just_another_fix_for_ddl_worker 2021-03-05 15:59:27 +03:00
Haavard Kvaalen
641b1b249e Attempt to reconnect to MySQL 
For MaterializeMySQL databases, attempt to reconnect if the connection
to MySQL is lost.  The existing setting
`max_wait_time_when_mysql_unavailable` is used to control how often we
attempt to reconnect.  This setting can now be set to a negative value
to disable reconnects.
 2021-03-04 08:49:06 +01:00
Haavard Kvaalen
8e95d6e174 Add method to metadata class to start replication 
Previously this was done by the constructor.  This makes it
possible to restart replication with an existing metadata object.
 2021-03-04 08:49:06 +01:00
Haavard Kvaalen
e6711675a1 Make handling of unavailable MySQL consistent 
If MySQL was unavailable when loading an existing database on startup,
we would previously load the database, fail to start synchronization,
but allow queries to the database.  Change this to only allow queries
if the `allows_queries_when_mysql_lost` setting is on, so that the
behavior is consistent with what happens if the connection to MySQL is
lost while ClickHouse is running.

Also retry connection to MySQL if MySQL is unavailable when ClickHouse
is started (we would previously reconnect only if the connection was
lost during the initial dump of existing data).
 2021-03-04 08:49:06 +01:00
kssenii
a3c7402200 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-02 07:07:56 +00:00
tavplubix
10aa26576c
Update DatabaseReplicatedWorker.cpp 2021-03-01 17:11:45 +03:00
Azat Khuzhin
955974a8b5 DDLWorker: avoid NULL dereference on termination and failed zookeeper initialization 
Log snipped shows the problem:

    2021.02.24 04:40:29.349181 [ 39 ] {} <Warning> DDLWorker: DDLWorker is configured to use multiple threads. It's not recommended because queries can be reordered. Also it may cause some unknown issues to appear.
    2021.02.24 04:40:29.349516 [ 39 ] {} <Information> Application: Ready for connections.
    2021.02.24 04:40:29.349602 [ 74 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread
    2021.02.24 04:40:29.349639 [ 73 ] {} <Debug> DDLWorker: Starting DDLWorker thread
    2021.02.24 04:40:29.349698 [ 73 ] {} <Debug> DDLWorker: Started DDLWorker thread
    2021.02.24 04:40:29.352548 [ 73 ] {} <Error> virtual void DB::DDLWorker::initializeMainThread(): Code: 999, e.displayText() = Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
     (Connection loss), Stack trace (when copying this message, always include the lines below):

    0. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0xfe93923 in /usr/bin/clickhouse
    1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0xfe93ba2 in /usr/bin/clickhouse
    2. Coordination::ZooKeeper::connect(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, Poco::Timespan) @ 0xfed3a01 in /usr/bin/clickhouse
    3. Coordination::ZooKeeper::ZooKeeper(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Timespan, Poco::Timespan, Poco::Timespan) @ 0xfed2222 in /usr/bin/clickhouse
    4. zkutil::ZooKeeper::init(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe961cd in /usr/bin/clickhouse
    5. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe97a96 in /usr/bin/clickhouse
    6. void std::__1::allocator_traits<std::__1::allocator<zkutil::ZooKeeper> >::__construct<zkutil::ZooKeeper, Poco::Util::AbstractConfiguration const&, char const (&) [10]>(std::__1::integral_constant<bool, true>, std::__1::allocator<zkutil::ZooKeeper>&, zkutil::ZooKeeper*, Poco::Util::AbstractConfiguration const&, char const (&) [10]) @ 0xed98387 in /usr/bin/clickhouse
    7. DB::Context::getZooKeeper() const @ 0xed75190 in /usr/bin/clickhouse
    8. DB::DDLWorker::getAndSetZooKeeper() @ 0xedb81c9 in /usr/bin/clickhouse
    9. DB::DDLWorker::initializeMainThread() @ 0xedc9eb0 in /usr/bin/clickhouse
    10. DB::DDLWorker::runMainThread() @ 0xedb5d01 in /usr/bin/clickhouse
    11. ThreadFromGlobalPool::ThreadFromGlobalPool<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0xedcafa1 in /usr/bin/clickhouse
    12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x892651f in /usr/bin/clickhouse
    13. ? @ 0x8929fb3 in /usr/bin/clickhouse
    14. start_thread @ 0x8ea7 in /lib/x86_64-linux-gnu/libpthread-2.31.so
    15. __clone @ 0xfddef in /lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.3.1.1)
    ...
    2021.02.24 04:40:30.025278 [ 41 ] {} <Trace> BaseDaemon: Received signal 15
    2021.02.24 04:40:30.025336 [ 41 ] {} <Information> Application: Received termination signal (Terminated)
    ...
    2021.02.24 04:40:30.582078 [ 39 ] {} <Information> Application: Closed all listening sockets.
    2021.02.24 04:40:30.582124 [ 39 ] {} <Information> Application: Closed connections.
    2021.02.24 04:40:30.583770 [ 39 ] {} <Information> Application: Shutting down storages.
    2021.02.24 04:40:30.583932 [ 39 ] {} <Information> Context: Shutdown disk data
    2021.02.24 04:40:30.583951 [ 39 ] {} <Information> Context: Shutdown disk default
    2021.02.24 04:40:30.584163 [ 46 ] {} <Trace> SystemLog (system.query_log): Terminating
    2021.02.24 04:40:30.586025 [ 39 ] {} <Trace> BackgroundSchedulePool/BgSchPool: Waiting for threads to finish.
    2021.02.24 04:40:34.352701 [ 73 ] {} <Debug> DDLWorker: Initialized DDLWorker thread
    2021.02.24 04:40:34.352758 [ 73 ] {} <Debug> DDLWorker: Scheduling tasks
 2021-02-28 05:45:12 +03:00
kssenii
6554965711 Fix build 2021-02-23 10:46:41 +00:00
Kseniia Sumarokova
bf88d693d5
Update fetchPostgreSQLTableStructure.cpp 2021-02-23 11:59:38 +03:00
Kseniia Sumarokova
6c5781cf34
Update fetchPostgreSQLTableStructure.cpp 2021-02-23 11:48:51 +03:00
Kseniia Sumarokova
b2850ec94f
Merge branch 'master' into fix-postgres-numeric 2021-02-23 11:40:28 +03:00
kssenii
4cdb55babe Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-02-22 12:39:35 +00:00
kssenii
ace76bb7fa Better, remove debug logs 2021-02-22 12:35:53 +00:00
kssenii
8a48bb24ce Fix typos, comments 2021-02-21 23:13:58 +00:00
kssenii
427aad80a1 Avoid table does not exist errors if nested is unavailable 2021-02-21 11:48:13 +00:00
tavplubix
aa48d967b7
Update DatabaseAtomic.cpp 2021-02-21 01:13:14 +03:00
kssenii
d590c32708 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-02-20 18:31:26 +00:00
Alexey Milovidov
64e76a4a8d Minor changes in Decimal 2021-02-20 21:13:36 +03:00
tavplubix
103d569885
Merge branch 'master' into fix_force_drop_materialized_view 2021-02-20 20:41:03 +03:00
kssenii
0424770f68 Handle ddl part 1 2021-02-20 16:21:53 +00:00
Alexander Tokmakov
863c099254 fix 2021-02-20 15:57:07 +03:00
kssenii
a38a31c954 Fix typos check 2021-02-20 09:58:24 +00:00
kssenii
f820047cc8 Fix 2021-02-20 06:52:10 +00:00
Alexander Tokmakov
2a36d6cb55 review suggestions 2021-02-20 02:41:58 +03:00
kssenii
3107f82a3b Fix clang tidy 2021-02-18 18:25:49 +00:00
kssenii
2aad067e7c Support conversion for postgres numeric without precision and scale 2021-02-18 12:16:58 +00:00
kssenii
1480e95179 Fix build, fix arcadia 2021-02-18 06:06:37 +00:00
Alexander Tokmakov
13ae988eff make force_drop_table work with materialized view 2021-02-17 17:23:07 +03:00
Alexander Tokmakov
7b54b892b5 fix 2021-02-16 17:05:58 +03:00
kssenii
272431bcae Fix types check, better tests, try fix build 2021-02-15 23:18:53 +00:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
Alexander Tokmakov
cf57c3b4a2 update comments 2021-02-15 23:01:21 +03:00
Alexander Tokmakov
9c7cf9e92e remove some debug code 2021-02-15 13:26:34 +03:00
Alexander Tokmakov
320fd6b264 startup without zk 2021-02-15 03:04:46 +03:00
kssenii
95c93aca41 Better table sync 2021-02-13 21:03:25 +00:00
kssenii
44f4f1a412 Allow to replicate a subset of database tables 2021-02-13 20:09:01 +00:00
kssenii
219dece1d0 Slightly better 2021-02-12 21:15:00 +00:00
kssenii
5bc0010f94 Better 2021-02-12 21:14:36 +00:00
kssenii
ed6d19b8a6 Add PostgreSQLReplica database engine 2021-02-12 21:14:24 +00:00
Alexander Tokmakov
f64f9b672b fix 2021-02-12 19:22:01 +03:00
Alexander Tokmakov
c9cf63e958 fix 2021-02-12 01:23:40 +03:00
Alexander Tokmakov
9afb16759e fix 2021-02-11 18:14:38 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Alexander Tokmakov
2c6a0e74fb better replica creation 2021-02-09 18:14:20 +03:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
Alexander Tokmakov
91d0924665 write dictionaries metadata to zk 2021-02-08 12:46:30 +03:00
Alexander Tokmakov
7ce0ef2561 show clusters for replicated db 2021-02-08 12:14:17 +03:00
Alexander Tokmakov
18f6b5bbad add timeouts 2021-02-04 22:41:44 +03:00
Alexander Tokmakov
066fb4c82b fix 2021-02-03 23:02:37 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Alexander Tokmakov
6456ccf0da better test 2021-02-02 22:39:04 +03:00
Alexander Tokmakov
0073c87d5d fix 2021-02-02 13:32:42 +03:00
Alexander Tokmakov
9da445e740 execute initial query in the same thread 2021-02-01 22:29:47 +03:00
Vitaly Baranov
7c8deae0e1 Fix starting the server with tables having columns' default expressions containing dictGet(). 
Allow getting return type of dictGet() without loading dictionary.
 2021-01-30 19:06:48 +03:00
Alexander Tokmakov
f20d5e3b41 fix 2021-01-26 20:51:25 +03:00
Alexander Tokmakov
3bd4d97353 Merge branch 'master' into database_replicated 2021-01-25 14:19:04 +03:00
tavplubix
4be33e81ee
Merge pull request #19344 from zhang2014/patch-2 
ISSUES-18684 fix MaterializeMySQL integration test failure
 2021-01-25 12:46:53 +03:00
zhang2014
4593ca80ff ISSUES-18684 fix MaterializeMySQL integration failure 2021-01-24 15:50:37 +08:00
kssenii
a98c6d333c Make sure not to log non-existent objects 2021-01-23 12:05:37 +00:00
kssenii
daab2c91bb Better 2021-01-21 21:15:11 +00:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
Alexander Tokmakov
7f97a11c84 Merge branch 'master' into database_replicated 2021-01-18 17:09:39 +03:00
tavplubix
071a6aeb4d
trigger CI 2021-01-18 16:53:33 +03:00
TCeason
a3cc6707b4 add MySQL Var check 
add MySQL var check: log_bin_use_v1_row_events=OFF
 2021-01-17 17:50:35 +08:00
kssenii
2222741e33 Fix style check 2021-01-16 09:17:04 +00:00
Alexey Milovidov
45380d45c8 Fix potential (safe) SQL-injection 2021-01-16 11:15:43 +03:00
kssenii
11ffa6293d Fix integration tests 2021-01-16 08:04:24 +00:00
Alexey Milovidov
593b6f5c83 Add assert 2021-01-16 00:25:58 +03:00
Alexey Milovidov
67f207d8ca More safe connection string 2021-01-16 00:18:51 +03:00
Alexey Milovidov
ef319bfa75 Slightly better 2021-01-15 23:33:04 +03:00
Alexey Milovidov
c16ecb0bdd It feels more correct 2021-01-15 23:18:55 +03:00
Alexey Milovidov
b1b09ed5ae Fix SQL injection 2021-01-15 23:14:51 +03:00
Alexey Milovidov
52cfc1d110 Minor changes 2021-01-15 23:09:31 +03:00
Alexey Milovidov
ba1921f380 Minor changes 2021-01-15 22:59:49 +03:00
Alexey Milovidov
a19e7edd14 Merge branch 'master' into kssenii-pg2ch 2021-01-15 17:33:19 +03:00
kssenii
38a9cba850 Fix 2021-01-11 10:55:38 +00:00
kssenii
c6c6b2d23c Fix style and types check 2021-01-10 13:01:15 +00:00
kssenii
d952b0897e Minor adjustments 2021-01-10 12:06:18 +00:00
kssenii
fc9de76f7d Fix incorrect getCreateTableQueryImpl with multidim arrays 2021-01-10 09:00:35 +00:00
Alexey Milovidov
76149947ef Remove useless headers 2021-01-10 05:48:57 +03:00
kssenii
0088d66d59 Hopefully better array conversions for insert query 2021-01-09 22:42:42 +00:00
Alexey Milovidov
8af19c3251 Fix Arcadia 2021-01-07 15:29:02 +03:00
Alexey Milovidov
60d4db421c Fix Arcadia 2021-01-07 06:45:12 +03:00
alexey-milovidov
72b142a00a
Merge branch 'master' into pg2ch 2021-01-06 23:18:59 +03:00
kssenii
e100aec086 Fix typos check 2020-12-31 11:15:05 +00:00
kssenii
62b06de97e Try fix build 2020-12-30 16:00:05 +00:00
spongedc
494c9ac809 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-29 10:20:06 +08:00
kssenii
fa3297bf72 Fix build, update libpq 2020-12-28 19:16:10 +00:00
kssenii
9b25890674 Generate ya.make, fix fast test 2020-12-28 12:54:52 +00:00
kssenii
8f8920a7ee Add table cache, better drop table 2020-12-27 15:52:15 +00:00
kssenii
00a37404ca Add postgresql database engine 2020-12-27 12:26:29 +00:00
Alexander Tokmakov
42911e2438 implement CREATE OR REPLACE TABLE 2020-12-25 19:32:09 +03:00
Yatsishin Ilya
ce1e2b18a3 Fix codespell warnings. Split style checks. Update style checks docker 2020-12-24 13:17:52 +03:00
spongedc
a277308ac6 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-23 11:56:46 +08:00
Alexander Tokmakov
52ef4289fe enable optimize_on_insert for MaterializeMySQL 2020-12-17 16:05:38 +03:00
tavplubix
7e73b6acd8
Merge pull request #17642 from filimonov/detach-permanently 
DETACH TABLE PERMANENTLY
 2020-12-17 14:26:51 +03:00
Nikolai Kochetov
44d2194326 Try fix integration tests. 2020-12-16 11:52:04 +03:00
spongedc
38c0827881 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-16 12:26:56 +08:00
spongedc
a0083e23d6 Fix compile error 2020-12-16 12:07:50 +08:00
Mikhail Filimonov
9df7ecb8e6
fixes 2020-12-15 18:06:23 +01:00
Mikhail Filimonov
27ff3a5214
Rework using a separate flag file instead of renames 2020-12-15 15:46:13 +01:00
spongedc
dc27e0a4fd Fix error code 2020-12-15 20:29:26 +08:00
spongedc
8b2feaeca2 Support SHOW CREATE VIEW Syntax 2020-12-14 22:37:25 +08:00
Mikhail Filimonov
24abcc970b
Tests and fixes 2020-12-14 15:31:13 +01:00
Mikhail Filimonov
87ea1528cb
support for Lazy database 2020-12-14 15:31:13 +01:00
Mikhail Filimonov
4ba80996b0
Detach permanently for DatabaseConnectionMySQL 2020-12-14 15:31:12 +01:00
Mikhail Filimonov
799141362d
Prevent attempts to recreate the table which was detached permanently 2020-12-14 15:31:11 +01:00
Mikhail Filimonov
f6097cf818
Adding DETACH TABLE PERMANENTLY 2020-12-14 15:28:59 +01:00
Alexander Tokmakov
58a31e4882 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-11 16:14:03 +03:00
tavplubix
5f78280aec
Merge pull request #17915 from zhang2014/better_exception_message 
Better exception message for MaterializeMySQL
 2020-12-10 19:26:20 +03:00
tavplubix
f13de96afc
Merge pull request #17366 from zhang2014/fix/ISSUES-16835 
ISSUES-16835 try fix miss match header with MySQL SHOW statement
 2020-12-09 23:12:47 +03:00
zhang2014
bf30aa5ff1 Code style 2020-12-09 16:50:40 +08:00
zhang2014
ef15b1f1fb Better exception message for MaterializeMySQL 2020-12-09 16:46:54 +08:00
Ivan
0ba712a2d3
Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-08 16:45:51 +03:00
Winter Zhang
972cb41fae
Merge branch 'master' into fix/ISSUES-16835 2020-12-08 15:32:58 +08:00
Alexander Tokmakov
39532f7d9e slightly better DDLWorker initialization and restarting 2020-12-03 21:14:27 +03:00
Alexander Tokmakov
a0ef3a113a Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-02 18:29:02 +03:00
Alexander Tokmakov
1a4bd67736 fixes 2020-12-01 20:20:42 +03:00
Alexander Tokmakov
19c8399eb0 Merge branch 'master' into database_replicated 2020-12-01 17:33:07 +03:00
Vasily Nemkov
87945fb68b Merge branch 'master' into HEAD 2020-11-30 23:25:57 +03:00
Alexander Tokmakov
ab197a49c8 better code, fixes 2020-11-30 23:12:43 +03:00
Alexander Tokmakov
f1a52a609b separate DatabaseReplicatedDDLWorker 2020-11-27 17:04:03 +03:00
zhang2014
a74235397e ISSUES-16835 try fix miss match header with MySQL SHOW statement 2020-11-24 23:31:43 +08:00
Alexander Tokmakov
e82c63e05c fixes 2020-11-24 15:28:54 +03:00
Alexander Tokmakov
dce0bf8efc Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-11-24 15:28:31 +03:00
Alexander Tokmakov
5ccf975a4a Merge branch 'master' into database_replicated 2020-11-24 14:38:02 +03:00
Alexander Tokmakov
dad21ee684 maintain metadata in zk 2020-11-24 13:24:39 +03:00
taichong
a6c2a90756 modify integration test materialize mysql database 
1. fix asan test err

2. reset pymysql conn if ping err

3. finish mysql & ck killed test

4. modify check_query
 2020-11-23 17:55:07 +08:00
Alexey Milovidov
fbbf439338 Merge branch 'master' into Enmk-DateTime64_fixes_comparison 2020-11-23 05:46:17 +03:00
tavplubix
5cc9cb01cd
Merge pull request #16751 from amosbird/globalcontext 
Make global_context consistent.
 2020-11-22 18:46:17 +03:00
Nikolai Kochetov
bac1def5f9
Merge pull request #17134 from abyss7/tcp-port 
Implement tcpPort() function for tests
 2020-11-20 20:32:55 +03:00
Alexander Tokmakov
7ab4445e99 try another approach 2020-11-20 19:06:27 +03:00
tavplubix
3e335c1e95
Merge pull request #17091 from zhang2014/fix/binlog_checksum 
Fix MaterializeMySQL SYNC failure when modify binlog_checksum
 2020-11-20 13:25:14 +03:00
Amos Bird
1d9d586e20
Make global_context consistent. 2020-11-20 18:23:14 +08:00
Alexander Tokmakov
2283906a11 try support replica recovery 2020-11-19 13:34:45 +03:00
Ivan Lezhankin
fdd1c182ab Merge remote-tracking branch 'upstream/master' into tcp-port 2020-11-18 16:22:00 +03:00
Ivan Lezhankin
f897f7c93f Refactor IFunction to execute with const arguments 2020-11-17 16:24:45 +03:00
alexey-milovidov
b494b18748
Merge pull request #17108 from ClickHouse/ya-make-owners 
Add "OWNER" to ya.make files.
 2020-11-17 05:19:46 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
Winter Zhang
f6a205d64b
Merge branch 'master' into fix/binlog_checksum 2020-11-16 21:53:22 +08:00
zhang2014
6ef93ac73c Try fix MaterializeMySQL SYNC with modify binlog_checksum 2020-11-16 21:46:36 +08:00
Vasily Nemkov
63d168d34e Fixed MySQL and DateTime64 integration. 
Also updated other places, like type helpers/dispatchers where DateTime64
was implicitly supported before as Decimal64.
 2020-11-16 09:02:41 +03:00
Alexander Tokmakov
b0262b3d06 better replica creation 2020-11-13 21:35:45 +03:00
Alexander Tokmakov
0731bfb7be Merge branch 'master' into database_replicated 2020-11-12 21:04:18 +03:00
Alexander Tokmakov
2a6c0b9180 try reuse DDLWorker in DatabaseReplicated 2020-11-12 20:35:29 +03:00
taichong
010e12eb4b Replace std::*stringstreams 2020-11-12 13:49:56 +08:00
TCeason
3912ed4965 Merge branch 'master' into add_sync_user_privs_exception 2020-11-12 09:34:08 +08:00
taichong
e2a9fe44b0 modify test error 2020-11-12 08:17:13 +08:00
taichong
5c1a01897a Revise the code according to the review 2020-11-11 19:27:37 +08:00
taichong
d907afa77e throw exception when mysql sync user privs error. 2020-11-11 19:27:37 +08:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexander Tokmakov
62ff00ee8b use WriteBuffer in formatAST(...) 2020-11-09 19:05:40 +03:00
tavplubix
6fb077c2f3
Merge pull request #16704 from TCeason/make_sync_ddl_execute_atomic 
sync MySQL DDL atomicly
 2020-11-09 13:47:02 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
taichong
9ce6c4dcb8 according to review modify code 2020-11-06 15:04:05 +08:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
taichong
fb5e03ebaf sync MySQL DDL atomicly 
Now ClickHouse will write metadata and then execute DDL from MySQL.

But If ClickHouse is crashed and it just begin to execute DDL, we
restart the server and then the sync will may be stopped.

e.g.:

MySQL : create table foo.a(id int key); uuid1:1
ClickHouse : write uuid1:1 to .metadata -> server crashed;

Because of ClickHouse's metadata records: uuid1:1, if MySQL run a
query:`insert into foo.a values(1);` ClickHouse will throw a exception:
the table foo.a is not exist.
 2020-11-05 17:01:14 +08:00
Alexander Tokmakov
d8ae9fcdb4 fixes, add shard name 2020-11-03 14:30:53 +03:00
Alexander Tokmakov
ac32809b6a fix #16482 2020-11-02 19:40:39 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
alexey-milovidov
bf3acecf6f
Merge pull request #16032 from Avogar/mysql_bug 
Don't touch MySQL database if it's unnecessary
 2020-10-29 04:14:26 +03:00
tavplubix
22847613d1
Merge pull request #16437 from ClickHouse/addition_to_16127 
Addition to #16127
 2020-10-28 13:03:13 +03:00
alesapin
8a6be1602b
Merge pull request #16364 from ClickHouse/fix_uuid_mapping_exists 
Minor improvements in DatabaseCatalog
 2020-10-28 10:54:39 +03:00
Alexander Tokmakov
b5ccb5ed5b review suggestions 2020-10-27 23:52:49 +03:00
Alexander Tokmakov
dba8c85ae4 do not wait while holding lock 2020-10-27 22:56:54 +03:00
Alexander Kuzmenkov
6509f1eb39
Merge branch 'master' into minumum 2020-10-27 21:38:23 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
Alexander Tokmakov
cd14f095ab fix tests 2020-10-26 18:12:16 +03:00
Alexander Tokmakov
636da1318b Merge branch 'master' into database_replicated 2020-10-26 15:00:43 +03:00
Alexey Milovidov
d5e277f080 Better exceptions rethrow 2020-10-26 06:35:57 +03:00
Alexander Tokmakov
f8f2d62d70 minor improvements in DatabaseCatalog 2020-10-25 20:27:21 +03:00
Alexander Tokmakov
478eb0b8a5 fix 2020-10-22 23:32:47 +03:00
Alexander Tokmakov
72dbb43235 Merge branch 'master' into fork-valbaturin-replication-db-engine 2020-10-20 19:14:54 +03:00
Azat Khuzhin
32c449c34f Add IDatabaseTablesIterator::reset() interface 2020-10-19 00:16:10 +03:00
alesapin
0bb4480fee
Merge pull request #16031 from ClickHouse/database_atomic_patch 
Minor fix for DatabaseAtomic
 2020-10-16 10:08:04 +03:00
Alexander Tokmakov
2f9611f1ae fix logical error with explicit UUIDs 2020-10-15 19:01:23 +03:00
Pavel Kruglov
ee8a9d2cf1 Don't touch MySQL database if it's unnecessary 2020-10-15 18:57:17 +03:00
Alexander Tokmakov
72b1339656 Revert "Revert "Write structure of table functions to metadata"" 
This reverts commit c65d1e5c70.
 2020-10-14 15:19:29 +03:00
tavplubix
c65d1e5c70
Revert "Write structure of table functions to metadata" 2020-10-14 13:59:29 +03:00
alexey-milovidov
f60ccb4edf
Merge pull request #14295 from ClickHouse/write_structure_of_table_functions 
Write structure of table functions to metadata
 2020-10-13 23:56:09 +03:00
Azat Khuzhin
d16adddb41 Fix drop of materialized view with inner table in Atomic database 
Materialized view drop the inner table (if any) drop the drop() method,
and this will lead to essure recursive drop() from the drop worker
thread, which will hang:

    3  std::__1::condition_variable::wait<> ()
    4  DB::DatabaseCatalog::waitTableFinallyDropped (this=0x7ffff7831500, uuid=...) at ../src/Interpreters/DatabaseCatalog.cpp:828
    5  DB::InterpreterDropQuery::executeToTable (this=this@entry=0x7fff16ff88a0, table_id_=..., query=...) at ../src/Interpreters/InterpreterDropQuery.cpp:135
    6  DB::InterpreterDropQuery::execute (this=this@entry=0x7fff16ff88a0) at ../contrib/libcxx/include/string:1474
    7  DB::executeDropQuery (kind=kind@entry=DB::ASTDropQuery::Drop, global_context=..., target_table_id=...) at ../src/Storages/StorageMaterializedView.cpp:156
    8  DB::StorageMaterializedView::drop (this=0x7ffefc348f40) at ../src/Storages/StorageMaterializedView.cpp:169
    9  DB::DatabaseCatalog::dropTableFinally (this=this@entry=0x7ffff7831500, table=...) at ../src/Interpreters/DatabaseCatalog.cpp:775
    10 DB::DatabaseCatalog::dropTableDataTask (this=0x7ffff7831500) at ../src/Interpreters/DatabaseCatalog.cpp:745

Fix this by dropping the inner table just before scheduling job the the
drop worker thread.
 2020-10-12 21:46:07 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Alexander Tokmakov
6b2da861dc Merge branch 'master' into write_structure_of_table_functions 2020-10-02 20:46:58 +03:00
Alexander Tokmakov
32776da69a Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-10-02 20:09:37 +03:00
tavplubix
2f95609f98
Merge pull request #15448 from ClickHouse/allow_synchronous_drop_for_atomic_db 
Allow synchronous DROP/DETACH TABLE for Atomic
 2020-10-01 01:14:31 +03:00
tavplubix
c9eeb149fd
Merge pull request #15384 from zhang2014/fix/ISSUES-15365 
ISSUES-15365 fix attach mysql database database
 2020-09-30 11:27:15 +03:00
Alexander Tokmakov
e156e6115e add setting 2020-09-30 01:30:34 +03:00
Alexander Tokmakov
5d19ed04f6 allow synchronous drop/detach table for Atomic 2020-09-29 16:47:08 +03:00
zhang2014
9e94c077e7 IESSUES-15365 try fix integration test 2020-09-29 10:39:30 +08:00
zhang2014
e5a6d06820 ISSUES-15365 try fix integration test failure 2020-09-28 21:15:02 +08:00
zhang2014
e40d23cfe5 ISSUES-15365 add ya make 2020-09-28 19:11:40 +08:00
zhang2014
d3047d0783 ISSUES-15365 fix attach mysql database failure 2020-09-28 18:02:30 +08:00
Alexander Tokmakov
b0d99217fb Merge branch 'master' into write_structure_of_table_functions 2020-09-27 14:26:47 +03:00
Alexander Tokmakov
aa5e5013f2 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-09-27 14:11:53 +03:00
Alexander Tokmakov
34addcf61f add review suggestions 2020-09-26 22:18:28 +03:00
alexey-milovidov
e29c4c3cc4
Merge pull request #15024 from ClickHouse/fix_renameat2_einval_on_macos 
Fix EINVAL in renameat2 on MacOS
 2020-09-21 21:08:17 +03:00
tavplubix
7f629c8185
Update DatabaseMemory.cpp 2020-09-21 14:12:45 +03:00
alesapin
5ffee8808a
Alter remove column properties and TTLs (#14742) 2020-09-20 16:27:33 +03:00
Alexander Tokmakov
46aa307d0a fix EINVAL in renameat2 on MacOS 2020-09-19 17:19:06 +03:00
tavplubix
6e0bdaf46d
Merge pull request #14535 from zhang2014/fix/datetime 
ISSUES-4006 support decimal data type for MaterializedMySQL
 2020-09-19 14:05:32 +03:00
Alexander Tokmakov
1ca9a92b21 Merge branch 'master' into write_structure_of_table_functions 2020-09-18 21:09:23 +03:00
tavplubix
0597ad3150
fix typo 2020-09-17 21:47:21 +03:00
Alexander Tokmakov
1c37ac7c9d fix 2020-09-17 21:19:02 +03:00
Alexander Tokmakov
41e99cf261 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-09-17 20:28:50 +03:00
zhang2014
64032e22a2 Merge branch 'master' into fix/datetime 2020-09-18 00:28:09 +08:00
zhang2014
c5b56c24f8 ISSUES-14894 fix MaterializeMySQL temp metadata file exists 2020-09-17 14:44:05 +08:00
Alexander Tokmakov
0ecf8e595f cleanup code, add test 2020-09-15 16:41:53 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Alexander Tokmakov
3671c91c37 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-09-15 12:43:01 +03:00
Alexander Tokmakov
a812f7eb75 allow using Atomic database inside MaterializeMySQL 2020-09-14 22:25:02 +03:00
BohuTANG
00b8d91eb8 Fix mysql replication TableMapEvent optional metadata ignore 2020-09-10 19:46:59 +08:00
zhang2014
094897fe69 Merge branch 'master' into fix/datetime 2020-09-09 23:37:54 +08:00
Alexander Tokmakov
d278d849e4 Merge branch 'master' into write_structure_of_table_functions 2020-09-09 17:12:35 +03:00
Vasily Nemkov
3973a17530
MySql datatypes dateTime64 and decimal (#11512) 2020-09-09 15:18:02 +03:00
Winter Zhang
18b4c5997d
Merge branch 'master' into fix/datetime 2020-09-08 11:32:48 +08:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
zhang2014
b7e9d5e72d ISSUES-4006 support decimal for MaterializedMySQL engine 2020-09-07 17:43:25 +08:00
Alexander Tokmakov
b840d741d0 Merge branch 'master' into write_structure_of_table_functions 2020-09-04 13:00:07 +03:00
zhang2014
dd0feeeaa1 ISSUES-4006 support datetime with precision 2020-09-03 11:03:28 +08:00
Alexey Milovidov
96038e9d54 Fix for Lazy database 2020-09-03 04:06:12 +03:00
Alexander Tokmakov
41dd393824 fixes 2020-09-01 20:55:35 +03:00
Alexander Tokmakov
56695727b2 Merge branch 'master' into write_structure_of_table_functions 2020-09-01 20:15:13 +03:00
Alexander Tokmakov
3a9779adb4 slightly better code 2020-09-01 17:41:49 +03:00
alesapin
a0b4cc78d6 Throw exception on alter for storages created from table functions 2020-09-01 10:26:31 +03:00
zhang2014
4182596961 ISSUES-14231 try fix wrong lexer 2020-08-29 12:28:04 +08:00
alesapin
c1ca09653e Update custom parts of storage AST only if it has extended definition 2020-08-28 14:30:53 +03:00
alesapin
0896b49533 Small fixes 2020-08-28 13:18:56 +03:00
Amos Bird
078b14610d
ALTER MODIFY SAMPLE BY 2020-08-27 22:31:30 +08:00
Alexander Tokmakov
969940b4c9 write table tructure for table function remote(...) 2020-08-26 23:55:40 +03:00
tavplubix
cff92c8ad3
Merge pull request #13820 from BohuTANG/mysql_replica_gtid_issue_4006 
ISSUES-4006 support MySQL GTID based replication #4006
 2020-08-26 15:02:53 +03:00
tavplubix
3b8ac01ed0
Merge pull request #13756 from ClickHouse/test-drop-database-race 
Fix race condition in DROP DATABASE
 2020-08-19 13:27:50 +03:00
Alexander Tokmakov
e686616756 add exclusive DDLGuard for database 2020-08-18 18:15:27 +03:00
BohuTANG
8c9236fefd ISSUES-4006 adapted GTID replication to Materialize MySQL 2020-08-17 22:12:41 +08:00
BohuTANG
adc6cce96e ISSUES-4006 fetch executed_gtid_set from replication position #4006 2020-08-17 15:38:09 +08:00
tavplubix
18ef911fe8
Merge pull request #13672 from zhang2014/refactor/materialize_mysql_database 
ISSUES-4006 split mysql protocol to multiple packets
 2020-08-16 13:55:11 +03:00
zhang2014
fe49bd3414 Merge branch 'master' of github.com:yandex/ClickHouse into refactor/materialize_mysql_database 2020-08-15 16:12:46 +08:00
Alexey Milovidov
52c3857225 Fix server restart after stress test 2020-08-15 06:10:57 +03:00
Alexander Tokmakov
0f1c4bc9c4 attach all system tables 2020-08-13 20:09:40 +03:00
zhang2014
96bd3ac34b ISSUES-4006 split mysql protocol 2020-08-13 20:41:36 +08:00
zhang2014
34f4c8972e ISSUES-4006 split replication packets 2020-08-13 20:07:02 +08:00
zhang2014
a8d3b5f65d ISSUES-4006 better exception message 2020-08-13 10:14:59 +08:00
tavplubix
4aff48dc3f
Merge pull request #10851 from zhang2014/feature/materialize_mysql_database 
ISSUES-4006 support MaterializeMySQL database engine
 2020-08-12 15:59:28 +03:00
zhang2014
2311cda334 ISSUES-4006 convert _sign & _version to materialized column 2020-08-10 22:32:05 +08:00
Alexey Milovidov
0c15f3b6c8 Allow server to startup if there are leftovers from unsuccessfull database creations 2020-08-10 05:58:08 +03:00
zhang2014
933f287db4 ISSUES-4006 resolve review comment & support alter renmae command 2020-08-09 20:10:30 +08:00
zhang2014
6fca8ad7df ISSUES-4006 fix segfault on ClickHouse shutdown when MySQL is not available 2020-08-07 16:49:27 +08:00
zhang2014
d6bba8cbbb ISSUES-4006 add settings for allows_query_when_mysql_lost & move dump event to logger 2020-08-07 14:12:27 +08:00
zhang2014
bd21f8e6b9 ISSUES-4006 fix conflicts commit when merge 2020-08-07 12:55:45 +08:00
zhang2014
e496225641 ISSUES-4006 try fix build failure & review comment 2020-08-07 12:53:33 +08:00
zhang2014
a3bc18bed2 ISSUES-4006 try resolve review comment 2020-08-06 12:03:27 +08:00
Alexey Milovidov
d91471ea00 Merge branch 'master' into stack-frams-size-limit 2020-08-04 15:28:47 +03:00
zhang2014
34f7fac76c ISSUES-4006 try fix test failure 2020-08-04 12:22:12 +08:00
Alexander Tokmakov
cf78a1f8fc try fix build 2020-08-04 10:31:27 +08:00
zhang2014
7ea2eee98a ISSUES-4006 add some check & add comment 2020-08-04 10:31:27 +08:00
zhang2014
c173bd391f ISSUES-4006 try fix build & test failure 2020-08-04 10:31:27 +08:00
zhang2014
6b452cf83d ISSUES-4006 try fix build failure & fix float 32 parser 2020-08-04 10:31:27 +08:00
zhang2014
c0db408c38 ISSUES-4006 try fix build failure 2020-08-04 10:31:27 +08:00
zhang2014
b76f218d90 ISSUES-4006 add mysql dml test 2020-08-04 10:31:27 +08:00
zhang2014
9996166860 ISSUES-4006 try fix check style 2020-08-04 10:31:27 +08:00
zhang2014
bd65a8f213 ISSUES-4006 try fix build failure 2020-08-04 10:31:27 +08:00
zhang2014
9d8234afd7 ISSUES-4006 try fix integration test 2020-08-04 10:31:27 +08:00
zhang2014
c5946f7697 ISSUES-4006 add mysql 8.0 integration test 2020-08-04 10:31:27 +08:00
zhang2014
74f354d840 ISSUES-4006 add some check before create materialize mysql database 2020-08-04 10:31:27 +08:00
zhang2014
95b8afb8eb ISSUES-4006 try fix unit test failure 2020-08-04 10:31:26 +08:00
zhang2014
e471f805b5 ISSUES-4006 try fix build failure & update integration test 2020-08-04 10:31:26 +08:00
zhang2014
1c17878f31 ISSUES-4006 try fix build failure 2020-08-04 10:31:26 +08:00
zhang2014
1930b8f584 ISSUES-4006 support drop materialize mysql database engine 2020-08-04 10:31:26 +08:00
zhang2014
eace0153bd ISSUES-4006 fix test failure 2020-08-04 10:31:26 +08:00
zhang2014
8f28df3514 ISSUES-4006 rename locality_data to MaterializeMySQL 2020-08-04 10:31:26 +08:00
zhang2014
cee16292a6 ISSUES-4006 fix restart failure 2020-08-04 10:31:26 +08:00
zhang2014
094ce895da ISSUES-4006 fix test failure 2020-08-04 10:31:26 +08:00
zhang2014
035760df9d ISSUES-4006 refactor for external ddl 2020-08-04 10:31:26 +08:00
zhang2014
6562597064 ISSUES-4006 refactor interpreter 2020-08-04 10:31:26 +08:00
zhang2014
1583f67681 ISSUES-4006 remove unused code 2020-08-04 10:31:26 +08:00
zhang2014
54c3db0567 ISSUES-4006 some refactor for query convert 2020-08-04 10:31:26 +08:00
zhang2014
5fb92359d2 ISSUES-4006 some refactor for query convert 2020-08-04 10:31:26 +08:00
zhang2014
3afffaf303 ISSUES-4006 rename sign and version column 2020-08-04 10:31:26 +08:00
zhang2014
101a13b5c3 ISSUES-4006 remove unused code 2020-08-04 10:31:26 +08:00
zhang2014
a9e1794bd3 ISSUES-4006 filter sign = -1 rows 2020-08-04 10:31:26 +08:00
zhang2014
9324c0ee49 ISSUES-4006 fix crash when dump data 2020-08-04 10:31:26 +08:00
zhang2014
1df8bfe4fb ISSUES-4006 fix build failure after refactor 2020-08-04 10:31:25 +08:00
zhang2014
4a99ca0d80 ISSUES-4006 refactor 2020-08-04 10:31:25 +08:00
zhang2014
0f75b2d0f0 ISSUES-4006 fix buidl failure 2020-08-04 10:31:25 +08:00
zhang2014
0336a4ad58 ISSUES-4006 suport synchronous MySQL incremental data[part 2] 2020-08-04 10:31:23 +08:00
zhang2014
0c52d425ba ISSUES-4006 suport synchronous MySQL incremental data 2020-08-04 10:29:54 +08:00
zhang2014
7203c6e186 ISSUES-4006 unlock table on exception 2020-08-04 10:29:54 +08:00
zhang2014
86c6be4cd9 ISSUES-4006 fix create query convert 2020-08-04 10:29:52 +08:00
zhang2014
bd18c1cbf8 ISSUES-4006 support dump all mysql data when create database 2020-08-04 10:27:57 +08:00
zhang2014
5d569f56a7 ISSUES-4006 support convert mysql create to clickhouse create 2020-08-04 10:26:23 +08:00
zhang2014
c53a48fe01 ISSUES-4006 support parse MySQL create query 2020-08-04 10:26:23 +08:00
Alexander Tokmakov
5b92f43f12 Merge branch 'master' into database_atomic_improvements 2020-08-03 14:35:40 +03:00
Alexey Milovidov
0fd7af8a63 Merge branch 'master' into stack-frams-size-limit 2020-08-03 00:06:00 +03:00
Alexey Milovidov
41111d1e57 Merge branch 'master' into stack-frams-size-limit 2020-08-02 19:18:33 +03:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Vitaly Baranov
668653600c Use SettingMaxThreads only in Settings, call getNumberOfPhysicalCPUCores() instead of SettingMaxThreads::getAuto(). 2020-07-31 19:11:27 +03:00
Vitaly Baranov
30b34e6a15 Use SettingSeconds only in Settings, use std::chrono::seconds in other places. 2020-07-31 19:11:27 +03:00
Alexey Milovidov
50bbfc848d Merge branch 'master' into stack-frams-size-limit 2020-07-21 14:45:15 +03:00
Alexander Tokmakov
cc25355989 fix 2020-07-18 19:33:39 +03:00
Alexander Tokmakov
c27de9b424 fixes 2020-07-17 16:11:44 +03:00
Alexander Tokmakov
ebf98ba006 implement reneme dictionary query 2020-07-17 00:41:26 +03:00
Alexander Tokmakov
d10b4c504d rename database with dictionaries 2020-07-16 17:25:39 +03:00
Alexander Tokmakov
f6949b2f47 use dictionary ids 2020-07-15 22:25:31 +03:00
Alexander Tokmakov
31e62e713e write uuid for ddl dictionaries 2020-07-14 00:21:01 +03:00
Alexander Tokmakov
15b395da51 fixes 2020-07-13 17:24:00 +03:00
Alexander Tokmakov
445acd0294 Merge branch 'master' into database_atomic_improvements 2020-07-09 16:02:54 +03:00
Alexander Tokmakov
9c4efa1f36 fix, add test 2020-07-09 01:50:15 +03:00
Alexander Tokmakov
7a5d532c7b implement rename database for atomic 2020-07-07 15:11:58 +03:00
Alexander Tokmakov
d1be5ec641 fix symlinks, add short syntax of attach database 2020-07-06 16:41:43 +03:00
Vitaly Baranov
99e9b15e00 Make code clearer: use enum instead of bool internal. 2020-07-05 17:14:29 +03:00
Val
e591fe5014 database replicated feedback mechanism prototype 2020-07-04 19:32:23 +03:00
Vitaly Baranov
ab343132de Fix handling dependency of table with ENGINE=Dictionary on dictionary. 2020-07-03 22:05:53 +03:00
Alexander Tokmakov
383c583bdd add uuid to atomic databases 2020-07-02 23:39:31 +03:00
alexey-milovidov
948af0bfc5
Merge pull request #12045 from ClickHouse/fix_loading_of_tables 
Parse metadata in parallel when loading tables
 2020-06-30 21:16:40 +03:00
Alexander Tokmakov
987e64acfe parse metadata in parallel when loading tables 2020-06-30 03:12:16 +03:00
Val
8273248c4e add log_name_to_exec to dbreplicated 2020-06-28 18:43:30 +03:00
Alexey Milovidov
1462a66d1e Fix typos 2020-06-27 22:05:00 +03:00
Val
e23c7a313e address pr comments 2020-06-27 17:04:12 +03:00
Val
9635ea64be Add desc of propose idatabase method 2020-06-27 17:04:11 +03:00
Val
d293e002a7 address pr comments 2020-06-27 17:04:11 +03:00
Val
16e50e33d7 fix typo 2020-06-27 17:04:11 +03:00
Val
67588edcf5 clean up db replicated files and add more tests 2020-06-27 17:04:11 +03:00
Val
82f5281cfe remove redundant includes 2020-06-27 17:04:11 +03:00
Val
4fc4b1d195 db replicated minor enhancements 2020-06-27 17:02:06 +03:00
Val
f928c897cf change replication algorithm, remove zk lock 
In this version of the databaseReplicated sequential persistent zk nodes
are used to order DDL queries. Db replicated ddl queries are executed
in the backgrould pool no matter whether it's proposed by the same
replica or not.
 2020-06-27 17:02:03 +03:00
Val
469f9738df refactor save state in db replicated 2020-06-27 17:01:20 +03:00
Val
a0af67b636 Add one more test for db replicated and fix related bug 2020-06-27 17:01:20 +03:00
Val
0e9f516738 add comment for replicated db class 2020-06-27 17:01:20 +03:00
Val
31910e9bf1 Use ClientInf::QueryKind to distinguish replicated db log queries 2020-06-27 17:01:18 +03:00
Val
cbcd1bea0e provide better comments and information 2020-06-27 17:00:44 +03:00
Val
4921dc6dab db replicated refactoring 2020-06-27 16:59:11 +03:00
Val
5e076b464e add replicated db snapshot, integration test, repl alter queries, etc 
add an option to create replicated tables within replicated db without specifying zk path and replica id
add replicated sch pool
disable replication of alter queries for replicated tables in replicated dbs
snapshot prototype. amend of replicated db workflow
add prototype of integration tests for replicated db
 2020-06-27 16:58:15 +03:00
Val
f103e24a09 make db replicated inherited from atomic 2020-06-27 16:52:35 +03:00
Val
d7a354b24d create query fix for replicated dbs 2020-06-27 16:51:47 +03:00
Val
d61259cd7b ddl replication works 2020-06-27 16:51:47 +03:00
Val
5eea58039c fix not initialized last entry in zk 2020-06-27 16:51:47 +03:00
Val
0a860c0c2b log based replicated 2020-06-27 16:51:36 +03:00
Val
319256ef4f an attempt to replicated create query from create query 2020-06-27 16:49:40 +03:00
Val
0a4c1783a1 Make drop work by fixing namespace bug 
data dir wasn't set right. now it's fixed.
add non-replicated table to test sql
 2020-06-27 16:49:40 +03:00
Val
8b0366ff4f an attempt to make something meaningful 2020-06-27 16:49:40 +03:00
Val
0d392bbb34 fix after rebase 2020-06-27 16:49:39 +03:00
Val
d4210d04c1 databasereplicated constructor scratch 2020-06-27 16:49:39 +03:00
tavplubix
dfdfc69df1
Merge pull request #11900 from bharatnc/ncb/fix-file-path 
fix double slashed paths in DatabaseOrdinary by switching to std::filesystem::path
 2020-06-26 15:05:15 +03:00
bharatnc
76938b40a4 use std::filesystem::path 2020-06-25 18:30:44 -07:00
Bharat Nallan
325a515e90 remove clang-format tag comments 2020-06-24 12:01:12 -07:00
Bharat Nallan
1244ca5627 fix style check - clang format for lambdas 2020-06-24 07:34:32 -07:00
bharatnc
4f10873c22 cleanup few unused headers 
The following files had some unused headers and caught my eyes, so
cleaning them up:

```bash
programs/obfuscator/Obfuscator.cpp
src/Databases/DatabaseAtomic.cpp
```
 2020-06-23 22:08:05 -07:00
Bharat Nallan
ea40e7b403 construct path using boost::filesystem::path 2020-06-23 18:43:36 -07:00
Bharat Nallan
6a9f3d768a clang format file 2020-06-23 18:31:25 -07:00
Bharat Nallan
ce391c84c5 remove unused imports 2020-06-23 18:11:06 -07:00
alesapin
f262c4f652 Better comments 2020-06-22 12:49:21 +03:00
alesapin
d79982f497 Better locks in Storages 2020-06-18 19:10:47 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
a39d010f94 Merge branch 'master' into consistent_metadata4 2020-06-11 18:47:41 +03:00
alexey-milovidov
89df9915bb
Merge pull request #11592 from ClickHouse/replicated-merge-tree-create-drop-race-garbage 
Fix race conditions in CREATE/DROP of different replicas of ReplicatedMergeTree
 2020-06-11 17:11:38 +03:00
Alexey Milovidov
b72a095e99 Fix race conditions in CREATE/DROP of different replicas of ReplicatedMergeTree 2020-06-11 06:24:52 +03:00
Alexander Tokmakov
9166ddea41 fix segfault in show create table 2020-06-10 15:53:12 +03:00
Alexey Milovidov
475af33319 Avoid too large stack frames 2020-06-08 20:35:45 +03:00
alesapin
ac67d0485c Merge branch 'master' into consistent_metadata4 2020-06-08 15:06:51 +03:00
alesapin
abaf47f0cd Make metadata single structure 2020-06-05 20:29:40 +03:00
Alexander Kuzmenkov
435f53e05a Improve error messages 
Also support fmt for exception messages, the same way we use it in
logging macros.
 2020-06-04 20:50:29 +03:00
Alexander Kuzmenkov
c7d9094a7a
Merge pull request #11259 from ClickHouse/consistent_metadata3 
More consistent metadata for secondary indices
 2020-06-03 12:23:21 +03:00
alesapin
1ab3047201 Merge branch 'master' into consistent_metadata3 2020-06-02 10:37:12 +03:00
Alexey Milovidov
a6bf4b5d70 Fix error with Lazy database 2020-06-02 05:06:16 +03:00
alesapin
b49be4c7f2 Better names 2020-06-01 15:11:23 +03:00
Alexander Kuzmenkov
ca3fb27b09 Report dictionary name on dictionary load errors. 2020-06-01 14:08:14 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexander Tokmakov
3cff070f89 add context to IDatabase methods 2020-05-28 23:10:45 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
7c0c328a35 Progress on task 2020-05-24 00:05:47 +03:00
Alexey Milovidov
d5b7883451 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/' 2020-05-23 22:41:08 +03:00
Alexey Milovidov
7e2fb9ad65 Apply all transformations again 2020-05-23 22:38:30 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
9d24908e53 Progress on task 2020-05-23 20:52:11 +03:00
Alexey Milovidov
533f86278a find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7", \4, \6);/' 2020-05-23 20:00:41 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/' 2020-05-23 19:42:39 +03:00
alexey-milovidov
655b8e1903
Merge pull request #11021 from ClickHouse/fix_10557 
Remove data on DROP DATABASE for DatabaseMemory
 2020-05-21 20:17:46 +03:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible. 
Now after changing Context.h 488 modules will be recompiled instead of 582.
 2020-05-21 01:53:18 +03:00
Alexander Tokmakov
6af9c78476 remove data on DROP DATABASE 2020-05-18 16:51:01 +03:00
alexey-milovidov
6f1d522b6a
Merge pull request #10980 from azat/database-atomic-fixes 
Database atomic fixes
 2020-05-17 23:25:56 +03:00
Azat Khuzhin
fb6b19d57a Fix symlinking data directory for Atomic database 2020-05-17 17:14:51 +03:00
Alexey Milovidov
90cd5d8a3c Remove unused header file 2020-05-17 13:27:13 +03:00
Alexander Tokmakov
791ccd8b05 never throw from destructor 2020-05-17 04:01:26 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
Vitaly Baranov
dd34cd73e2 Move initialization of DatabaseWithDictionaries to constructor. 2020-04-30 20:22:20 +03:00
Alexander Tokmakov
04d4130b0d update yamake 2020-04-24 20:14:10 +03:00
Alexander Tokmakov
15e7ce0053 fixes 2020-04-23 21:00:43 +03:00
Alexander Tokmakov
f413061e0d remove Context from some IDatabase methods 2020-04-23 19:51:48 +03:00
Alexander Tokmakov
04d6b59ac0 Merge branch 'master' into database_atomic 2020-04-23 17:31:37 +03:00
alexey-milovidov
77969d11a5
Merge pull request #10442 from zhang2014/fix/create_database_failure 
Not remove metadata directory when attach database fails
 2020-04-23 01:07:13 +03:00
Alexander Tokmakov
1833ac7f16 fixes 2020-04-22 23:43:10 +03:00
alexey-milovidov
9e14603ae5
Update DatabaseFactory.cpp 2020-04-22 22:37:30 +03:00
zhang2014
a622208284 Not remove metadata directory when attach database fails 2020-04-23 00:03:58 +08:00
Alexander Tokmakov
fefbbd37df Merge branch 'master' into database_atomic 2020-04-22 17:02:30 +03:00
Alexey Milovidov
be22a4b94e Checkpoint 2020-04-22 08:39:31 +03:00
Azat Khuzhin
9173895f85 Fix dictionary name (RELOAD and system.dictionaries) after DETACH/ATTACH 
After DETACH/ATTACH (or at server startup) the following will be broken:
- SYSTEM RELOAD DICTIONARY db_name.dict_name
- system.dictionaries will not have database for those dictionaries

The problem is that after DETACH/ATTACH the dictionary information is
obtained from metadata/$db/$dict.sql, but it does not have database
name.

Fix this by passing database name to the getDictionaryConfigurationFromAST().

Introduced-in: #10234
 2020-04-21 21:03:12 +03:00
Alexander Tokmakov
b29bddac12 Merge branch 'master' into database_atomic 2020-04-20 14:09:09 +03:00
Vitaly Baranov
d62345bbaa Show information about not loaded dictionaries in system.columns, system.tables 
and in "SHOW TABLES" executed in any database with the "Dictionary" engine.
 2020-04-18 12:50:54 +03:00
alexey-milovidov
17e7d4d88a
Merge pull request #10307 from abyss7/arcadia-4 
Changes for auto-sync with Arcadia
 2020-04-17 05:08:34 +03:00
Ivan Lezhankin
e230632645 Changes required for auto-sync with Arcadia 2020-04-16 15:31:57 +03:00
Alexey Milovidov
cdeda4ab91 Fix usage of max_parser_depth setting; remove harmful default function arguments 2020-04-16 04:06:10 +03:00
Alexander Tokmakov
d1024e76bd fix 2020-04-13 18:46:36 +03:00
Alexander Tokmakov
5e6d4b9449 Merge branch 'master' into database_atomic 2020-04-12 16:35:44 +03:00
alexey-milovidov
a4969d89f9
Merge pull request #10164 from azat/system.tables-superfluous-dict-reload 
Avoid superfluous dictionaries load
 2020-04-11 23:05:56 +03:00
Alexander Tokmakov
8b6c836720 fix 2020-04-11 18:38:41 +03:00
Vitaly Baranov
18fbd968ba
Merge pull request #10165 from azat/drop-dictionary-as-table 
Do not break DROP DICTIONARY with DROP TABLE
 2020-04-11 14:49:41 +03:00
Azat Khuzhin
55a143d1a5 Avoid superfluous dictionaries load (system.tables, SHOW CREATE TABLE) 
This patch avoids loading dictionaries for:
- SELECT * FROM system.tables (used by clickhouse-client for completion)
- SHOW CREATE TABLE some_dict

But the dictionary will still be loaded on:
- SHOW CREATE TABLE some_dict (from the database with Dictionary engine)
 2020-04-11 13:26:41 +03:00
Alexander Tokmakov
f0edb65e33 add test 2020-04-11 02:02:15 +03:00
Azat Khuzhin
5b3c635200 Do not break DROP DICTIONARY after DROP TABLE executed on dictionary 2020-04-10 21:10:32 +03:00
Alexander Tokmakov
033d7f9b32 Merge branch 'master' into database_atomic 2020-04-10 17:12:09 +03:00
Alexander Kazakov
497df3086f Merge branch 'master' into timed_rwlock 
Change-Id: I620bfde2121ff013773b001d514b40b1e796a58b
 2020-04-10 11:38:20 +03:00
Alexander Tokmakov
50decc755d fix rename materialized view 2020-04-10 04:35:37 +03:00
Alexander Tokmakov
03ed9d59b0 add symlinks 2020-04-10 03:08:43 +03:00
Alexander Kazakov
26dd6140b2 Added new config settings to control timeouts 
* "lock_acquire_timeout" controls for how long a query will continue to
acquire each lock on its argument tables
 * "lock_acquire_timeout_for_background_operations" is a per-table
setting for storages of *MergeTree family
 2020-04-09 21:10:27 +03:00
zhang2014
9eb96b87db ISSUES-10056 reused evaluateConstantExpressionOrIdentifierAsLiteral 2020-04-09 01:31:28 +08:00
zhang2014
9ed708b902 ISSUES-10056 add some check and support identifier argument for MySQL Database Engine 2020-04-09 01:31:27 +08:00
Alexander Tokmakov
135197aa75 fixes 2020-04-08 04:02:00 +03:00
Alexander Tokmakov
4c48b7dd80 better rename 2020-04-07 18:31:33 +03:00
Alexander Tokmakov
a2101c9b1c better code 2020-04-07 02:22:44 +03:00
Alexander Tokmakov
08bae4668d Merge branch 'master' into database_atomic 2020-04-06 16:18:07 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00