thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
109,614 Commits 9,319 Branches 1,783 Tags 2.1 GiB
e23f624968
Commit Graph

1084 Commits

Author SHA1 Message Date
Alexander Tokmakov
c8d8f0a38c fix 2021-08-31 15:07:00 +03:00
Alexander Tokmakov
b387f05d9c resolve table dependencies on metadata loading 2021-08-30 22:29:22 +03:00
tavplubix
b47f34aa17
Merge pull request #28313 from amosbird/fastload 
Fast data parts loading by delaying table startup process
 2021-08-30 13:53:55 +03:00
Amos Bird
de9e591619
Fast load by delaying table startup 2021-08-30 10:15:32 +08:00
kssenii
9d0444774a Fix tests 2021-08-29 11:50:03 +03:00
kssenii
378e1f74aa Fix tests 2021-08-28 19:51:04 +03:00
kssenii
cc90d787f8 Slightly better 2021-08-28 17:45:15 +03:00
kssenii
fd621381c7 Allow modify some other settings 2021-08-28 17:36:27 +03:00
kssenii
a2b0996ac3 Dynamically remove tables 2021-08-28 17:35:59 +03:00
kssenii
174340074c Dynamically add tables complete 2021-08-28 17:35:44 +03:00
kssenii
4cd62227cf Almost done 2021-08-28 17:34:23 +03:00
tavplubix
c36fe210b8
Revert "Add test for #13398" 2021-08-27 21:07:10 +03:00
kssenii
0d5000f917 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into mysql 2021-08-24 07:30:42 +00:00
kssenii
962f415740 Add test and settings 2021-08-23 22:18:04 +00:00
Vitaly Baranov
63e8bc1f20 Added new commands BACKUP and RESTORE. 2021-08-23 23:41:45 +03:00
Vitaly Baranov
2ac97c7fc3 Make IDatabase::getTablesIterator() const. 2021-08-23 23:41:45 +03:00
kssenii
200d75646a Fix 2021-08-13 20:58:03 +00:00
Maksim Kita
124a87684f Removed some data streams 2021-08-11 23:39:01 +03:00
Mikhail Filimonov
af6249c8ec
Less include <Common/Stopwatch.h> 2021-08-11 16:20:59 +02:00
Maksim Kita
b8b24684b4
Merge pull request #27273 from ClickHouse/remove-streams-from-dicts 
Remove streams from dicts
 2021-08-09 15:58:37 +03:00
Maksim Kita
1db4eddafc
Merge pull request #27225 from kitaisreal/database-replicated-worker-logs-to-keep-race-fix 
DatabaseReplicatedWorker logs_to_keep race fix
 2021-08-06 14:16:36 +03:00
tavplubix
14ef66e49b
Merge pull request #25895 from ClickHouse/improve_create_or_replace 
Improve CREATE OR REPLACE query
 2021-08-06 11:58:46 +03:00
Nikolai Kochetov
39ff5ab478 Merge branch 'master' into remove-streams-from-dicts 2021-08-06 11:46:27 +03:00
Nikolai Kochetov
13f95f3fdf Streams -> Processors for dicts, part 3. 2021-08-06 11:41:45 +03:00
Kseniia Sumarokova
5edd9e0513
Merge pull request #27198 from kssenii/postgres-db-schema 
Support schema for postgres database engine
 2021-08-05 18:30:47 +03:00
Maksim Kita
1ae59b6cce DatabaseReplicatedWorker logs_to_keep race fix 2021-08-05 17:05:03 +03:00
kssenii
e8dd257468 Update DatabaseFactory.cpp 2021-08-05 09:04:00 +00:00
kssenii
1e38d48364 Fixes 2021-08-05 07:25:47 +00:00
kssenii
f06703a7c9 Support schema for postgres database engine 2021-08-04 20:52:45 +00:00
Alexander Tokmakov
ff2572c243 fix 2021-08-03 16:28:03 +03:00
Alexander Tokmakov
42a8bb6872 fix assertions in Replicated database 2021-08-02 16:19:11 +03:00
Alexander Tokmakov
fc9ab2cda7 Merge branch 'master' into rename_materialize_mysql 2021-07-27 22:38:40 +03:00
Alexander Kuzmenkov
ece7d000dd
Merge pull request #26736 from Algunenano/ch_local_remote_localhost 
CH local: Treat localhost:port as a remote database
 2021-07-27 15:30:54 +03:00
Alexander Tokmakov
a7e644f774 support old name 2021-07-26 21:24:32 +03:00
Alexander Tokmakov
63ab38ee09 rename MaterializeMySQL to MaterializedMySQL 2021-07-26 21:17:28 +03:00
kssenii
f3129b80f9 Fix sqlite attach 2021-07-26 07:20:41 +00:00
Raúl Marín
383c982715 CH local: Treat localhost:port as a remote database 2021-07-23 13:16:35 +02:00
mergify[bot]
39f71f4a08
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-19 12:27:38 +00:00
kssenii
dd519f9434 Fix 2021-07-15 19:02:31 +03:00
kssenii
f20ea87bfb Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-sqlite-database-path-check 2021-07-14 20:49:05 +00:00
Kseniia Sumarokova
e42a26a585
Merge pull request #26234 from jasine/master 
convert timestamp and  timestamptz data types to DateTime64 in postgres engine
 2021-07-14 23:42:51 +03:00
kssenii
4fb0762bbb Merge branch 'add-sqlite-database-path-check' of github.com:kssenii/ClickHouse into add-sqlite-database-path-check 2021-07-14 20:08:11 +00:00
kssenii
e875f1e289 Fix yandex check 2021-07-14 20:07:55 +00:00
Kseniia Sumarokova
ef0ebaf55c
Update SQLiteUtils.cpp 2021-07-14 20:20:41 +03:00
kssenii
d62cbbb8a5 Unification 2021-07-14 13:16:30 +00:00
kssenii
b393603b57 Fix build 2021-07-13 22:25:26 +00:00
Nikita Mikhaylov
601d04f55d ignore 2021-07-13 22:16:27 +00:00
kssenii
da09ce3f20 Remove unused 2021-07-13 19:51:02 +00:00
kssenii
a90df59b92 Better 2021-07-13 19:46:06 +00:00
mergify[bot]
d5898ecba6
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-13 14:03:05 +00:00
kssenii
9688efde96 Merge branch 'add-sqlite-database-path-check' of github.com:kssenii/ClickHouse into add-sqlite-database-path-check 2021-07-13 12:40:43 +00:00
kssenii
f86817d57c Check inclusion inside user_files 2021-07-13 12:38:37 +00:00
Kseniia Sumarokova
c615084ba7
Update DatabaseSQLite.cpp 2021-07-13 14:40:07 +03:00
kssenii
48ffd3b81d Add check for sqlite database path 2021-07-13 09:40:46 +00:00
jasine
4ec8d23fce
Merge branch 'ClickHouse:master' into master 2021-07-13 15:12:40 +08:00
kssenii
bcc77fca62 Merge branch 'add-sqlite-support' of github.com:g-arslan/ClickHouse into g-arslan-add-sqlite-support 2021-07-12 22:39:34 +00:00
kssenii
976f4dffda Add table function sqlite, fix type conversions 2021-07-12 21:09:23 +00:00
Kseniia Sumarokova
08ab42483c
Update DatabaseFactory.cpp 2021-07-12 20:40:30 +03:00
jasine
192ee2df98
Update fetchPostgreSQLTableStructure.cpp 2021-07-12 12:39:17 +08:00
kssenii
10611ef92a Normal exceptions 2021-07-10 11:00:59 +00:00
kssenii
18c2abaaf9 Fix database engine attach, more tests 2021-07-10 10:31:21 +00:00
kssenii
d250d79e33 Generate ya make 2021-07-10 08:07:55 +00:00
kssenii
78094a82ab Better 2021-07-10 08:05:23 +00:00
kssenii
1bf9a9f29f Add .cmake 2021-07-09 10:27:53 +00:00
kssenii
ff56f1787c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into g-arslan-add-sqlite-support 2021-07-07 15:19:12 +00:00
Alexander Tokmakov
0f377fc4d8 Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-05 22:07:56 +03:00
kssenii
32b7d7b750 More correct startup on create query 2021-07-04 19:53:05 +00:00
kssenii
b39888f199 Do not allow inconsistency in publication 2021-07-04 08:54:41 +00:00
kssenii
0ce4c565d1 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-07-03 11:48:15 +00:00
kssenii
f01c8edbff Fix access to tables after restart 2021-07-01 07:33:58 +00:00
Kostiantyn Storozhuk
a9510f25f9 Style fix 2021-06-29 15:22:08 +08:00
Kostiantyn Storozhuk
296593716d Reverted constructor change 2021-06-29 14:37:53 +08:00
Kostiantyn Storozhuk
c2c78929cb Implemented MySQL column comments support 2021-06-28 22:44:38 +08:00
Kseniia Sumarokova
ff8c44179c
Update DatabaseAtomic.cpp 2021-06-28 17:37:22 +03:00
Kseniia Sumarokova
a96c7a3a70
Update DatabaseAtomic.cpp 2021-06-28 17:32:03 +03:00
Kseniia Sumarokova
b6b7924413
Update DatabaseAtomic.cpp 2021-06-28 10:20:33 +03:00
kssenii
aa7133c239 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-27 20:08:00 +00:00
kssenii
c3a2fc0584 Review fixes 2021-06-27 19:17:17 +00:00
alexey-milovidov
fb89505656
Merge pull request #25538 from kssenii/pg-dix 
Fix postrges array conversion
 2021-06-27 19:35:24 +03:00
alexey-milovidov
6e25d11e1f
Merge pull request #25582 from azat/atomic-unlink-symlink 
Remove only symlinks during force_restore_data of Atomic engine
 2021-06-27 18:19:59 +03:00
kssenii
3d90f0710c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-26 22:04:47 +00:00
Maksim Kita
3c9ae7b5ba Added more tests 2021-06-26 14:28:13 +03:00
Azat Khuzhin
50d9d3efd5 Remove only symlinks during force_restore_data of Atomic engine 2021-06-23 10:17:31 +03:00
Kseniia Sumarokova
77a3a1416d
Update fetchPostgreSQLTableStructure.cpp 2021-06-21 17:26:14 +03:00
kssenii
0f9fc33a4e Fix postgres arrays 2021-06-21 11:27:10 +00:00
Maksim Kita
2c4c2680f7
Merge pull request #25430 from kitaisreal/database-mysql-rename 
DatabaseMySQL rename
 2021-06-18 01:30:52 +03:00
Maksim Kita
cf77e9f2c6 DatabaseMySQL rename 2021-06-17 22:26:34 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
kssenii
0521851afa Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into postgres-fixes-schema 2021-06-08 17:20:23 +00:00
kssenii
e23bc14582 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-03 19:44:55 +00:00
mergify[bot]
438b45e6c0
Merge branch 'master' into add-sqlite-support 2021-06-03 12:26:12 +00:00
Arslan G
4fe9a24472 Fix review a little, try add simple test 2021-06-02 03:53:30 +03:00
Arslan G
dc0a1001d8 Fix style 2021-06-02 01:21:01 +03:00
Arslan G
6f7dc47b2b Merge branch 'master' into add-sqlite-support 2021-06-02 00:24:29 +03:00
Arslan G
bf4171e0ea Some review fixes 2021-06-01 23:52:12 +03:00
kssenii
12ea45e956 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-01 17:43:34 +00:00
tavplubix
e2941a8006
Update DatabaseReplicated.cpp 2021-06-01 15:15:13 +03:00
Alexander Tokmakov
94d2aed336 fix tests 2021-06-01 13:24:06 +03:00
Alexander Tokmakov
fe482109da Merge branch 'master' into fix_database_replicated_enqueue_race 2021-06-01 11:49:29 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs 
Poco::File to std::filesystem
 2021-05-31 23:17:02 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Alexander Tokmakov
0035997e02 fix race on enqueue query 2021-05-31 16:31:03 +03:00
kssenii
0a3589524e File movement 2021-05-29 00:57:53 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Arslan G
fbc5734302 Clean up code and fix tests 2021-05-28 20:00:26 +03:00
kssenii
a78c9e63e7 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-23 12:07:26 +00:00
kssenii
866b29fb5a Return list fds with Poco, more canonical 2021-05-23 10:56:13 +03:00
kssenii
1c43b333ea Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-22 21:26:48 +03:00
kssenii
3e6d3d4ecb Postgres schema for insert 2021-05-22 08:50:23 +00:00
alexey-milovidov
a25f1910cc
Merge pull request #24320 from ClickHouse/remove-auto-array 
Remove AutoArray
 2021-05-22 02:19:59 +03:00
kssenii
096d117f68 Review fixes 2021-05-22 00:12:46 +03:00
Alexey Milovidov
ea30bb7abf Improve implementation 2021-05-21 04:17:18 +03:00
Azat Khuzhin
4f41ebcae3 Add settings (for connections) for MySQL storage engine 
Default settings are not very efficient, since they do not even reuse
connections.
And when each query requires connection you can have only ~80 QPS, while
by simply enabling connection reuse (connection_auto_close=false) you
can have ~500 QPS (and by increasing connection_pool_size you can have
better QPS throughput).

So this patch allows to pass through some connection related settings
for the StorageMySQL engine, like:
- connection_pool_size=16
- connection_max_tries=3
- connection_auto_close=true

v2: remove connection_pool_default_size
v3: remove num_tries_on_connection_loss
 2021-05-20 22:44:52 +03:00
kssenii
3dee003f9b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-20 19:20:09 +03:00
kssenii
5f083b779d A little better, fix checks 2021-05-18 14:25:19 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
Arslan G
dde9e30043 Merge branch 'master' into add-sqlite-support 2021-05-17 14:38:24 +03:00
Arslan G
ea51840f85 Init SQLite integration 2021-05-17 14:02:35 +03:00
kssenii
1ee77eae89 Better way to distinguish between user query and replication thread query 2021-05-16 21:35:37 +00:00
kssenii
8a4711b716 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-16 20:55:18 +00:00
Azat Khuzhin
23981ffd0e Introduce IStorage::flushAndShutdown() 2021-05-16 13:23:53 +03:00
Azat Khuzhin
f20799f683 Destroy Buffer() tables before others (within one database) 
This will reduce amount of data that may be lost at shutdown.

v2: replace dynamic_cast with IStorage::isBuffer (@kitaisreal)
v3: replace IStorage::isBuffer with IStorage::flush (@alexey-milovidov)
v4: flush() for StorageProxy/StorageTableFunction
 2021-05-16 13:23:53 +03:00
feng lv
c6f8ab9826 fix 2021-05-13 02:05:53 +00:00
kssenii
7c81103ea3 Add replication identifier, allow proper access to _version and _sign columns from nested 2021-05-12 22:48:42 +00:00
kssenii
407db17e2e Remove setting, use native transaction 2021-05-12 22:33:15 +00:00
kssenii
0955e5f0be Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-12 15:11:15 +00:00
kssenii
0527f0ea33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-12 16:54:18 +03:00
Nikolai Kochetov
a1ba67e029
Merge pull request #20202 from amosbird/projection 
Projections
 2021-05-12 09:38:07 +03:00
kssenii
81797393f1 fs::last_write_time, fs::space 2021-05-12 09:19:11 +03:00
Maksim Kita
72d46beca0
Merge pull request #23979 from azat/dict-preallocate 
Reimplement preallocate for hashed/sparse_hashed dictionaries
 2021-05-11 20:15:46 +03:00
feng lv
9035e3a643 fix test and conflict 2021-05-11 16:22:24 +00:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
kssenii
626e87bae5 Some fixes 2021-05-10 13:51:05 +00:00
kssenii
ae1191d0c0 Get rid of metadata file, rely only on lsn 2021-05-10 09:10:02 +00:00
kssenii
4ac023e511 Small fixes 2021-05-10 08:45:52 +00:00
kssenii
993823bcc4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-09 21:26:41 +00:00
kssenii
ab1a05a1f4 Poco::Path to fs::path, less concatination 2021-05-09 14:59:49 +03:00
Azat Khuzhin
80f4059deb Drop unused getDictionaryConfigurationFromAST.h 2021-05-09 14:25:45 +03:00
kssenii
e26cb4bee0 Fix style check, fix yandex check 2021-05-09 13:45:50 +03:00
Alexey Milovidov
3025f9e141 More fixes for PVS-Studio 2021-05-08 19:12:49 +03:00
Alexey Milovidov
8d62c42eb9 A bunch of changes for PVS-Studio 2021-05-08 19:11:50 +03:00
kssenii
5c24f0943c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-08 14:54:44 +00:00
kssenii
266567e04c Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 17:49:05 +03:00
alexey-milovidov
680860aae7
Merge pull request #23912 from azat/CU 
Cleanup IDatabase.h from extra headers
 2021-05-08 16:42:56 +03:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
kssenii
35f999bf04 Poco::createFile to fs::createFile 2021-05-08 02:41:47 +03:00
kssenii
c3eab60b8b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 02:40:17 +03:00
kssenii
140bf7e2ea Less Poco 2021-05-07 15:29:26 +03:00
kssenii
912878ad7f Review fixes 2021-05-07 12:05:34 +00:00
Azat Khuzhin
3672af8bbe Make DatabaseConnectionMySQL self compilable (missing Core/NamesAndTypes.h) 2021-05-06 21:03:03 +03:00
kssenii
b96f047db7 Fix clang tidy 2021-05-06 07:02:13 +00:00
Azat Khuzhin
6348497d1b Cleanup IDatabase.h from extra headers 
From IDatabase.h depends ~300 CU.

- drop unused
- use forward declaration whenever is possible
 2021-05-06 09:07:59 +03:00
kssenii
8900e93bd8 New impl 2021-05-06 00:52:59 +00:00
kssenii
ed5c2321cc Some fixes 2021-05-04 10:57:05 +00:00
kssenii
eff26f9d54 Add comments, small improvements 2021-05-03 21:42:06 +00:00
kssenii
3e3396bd9c Refactor code, add comments 2021-05-03 09:52:32 +00:00
feng lv
39f68bf5ff fix conflict 2021-05-02 16:33:45 +00:00
kssenii
14d355a6ac Fix background update 2021-05-02 11:53:03 +00:00
kssenii
32b2917141 Update s3/* 2021-05-01 18:10:04 +03:00
kssenii
ee06936596 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-01 17:24:31 +03:00
kssenii
14be3a0783 Better reload from snapshot 2021-05-01 11:49:45 +00:00
kssenii
7643d84095 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-30 22:02:51 +00:00
kssenii
e717b4c5c7 Update databases/* 2021-04-30 22:38:07 +03:00
kssenii
6e0cfdaeb5 Update Databases/* part 2 2021-04-30 20:24:36 +03:00
kssenii
1b75f12fcb Update Databases/* 2021-04-30 19:48:42 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Maksim Kita
2e5f97c6ff Fixed code review issues 2021-04-27 22:09:03 +03:00
Maksim Kita
8e508a37b7 Fixed dictionaries_lazy_load during StorageDictionary CREATE query 2021-04-27 12:15:35 +03:00
Maksim Kita
f37599112a Added EXCHANGE DICTIONARIES query support 2021-04-26 23:36:15 +03:00
Maksim Kita
c93666e0de Fixed code review issues 2021-04-26 13:52:41 +03:00
Maksim Kita
43680d8ced Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
561b045991 Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
64f49a981b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
2e3e90b25b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
e5079c87db Added test 2021-04-26 13:52:41 +03:00
Maksim Kita
3cb4d1bdcb Fixed style check 2021-04-26 13:52:41 +03:00
Maksim Kita
a9917a6540 Refactored StorageDictionary 2021-04-26 13:52:41 +03:00
tavplubix
fc2e2848bf
Merge pull request #23413 from TCeason/modify_MaterializeMySQL_var_check 
modify MaterializeMySQL check MySQL var logical
 2021-04-26 09:52:59 +03:00
feng lv
0f5c05854c fix conflict 2021-04-23 12:22:51 +00:00
feng lv
4ffe199d39 Implement table comments 2021-04-23 12:18:23 +00:00
TCeason
51fe9fc771 modify MaterializeMySQL check MySQL var logical 
Supprot for MySQL 5.7.9: https://github.com/ClickHouse/ClickHouse/issues/4006#issuecomment-817622685
 2021-04-21 10:16:04 +08:00
tavplubix
70c89eeff6
Merge pull request #23200 from ClickHouse/fix_prewhere_in_materialize_mysql 
Fix prewhere in MaterializeMySQL
 2021-04-20 20:05:07 +03:00
tavplubix
738d8a757b
Merge pull request #23189 from ClickHouse/minor_fixes_in_attach_query 
Minor fixes in ATTACH query
 2021-04-20 20:04:26 +03:00
alexey-milovidov
cbbff1bc4b
Merge pull request #23319 from ClickHouse/mysql-insert-without-database 
Allow to INSERT into remote MySQL storage with empty database #19281
 2021-04-20 19:30:44 +03:00
tavplubix
bef0ab182e
Merge branch 'master' into fix_prewhere_in_materialize_mysql 2021-04-20 15:38:48 +03:00
Alexander Tokmakov
4108a235f5 make separate functions 2021-04-20 14:22:02 +03:00
Kseniia Sumarokova
b43656bf89
Merge pull request #22697 from TCeason/feature/add_MaterializeMySQL_sync_bytes_judgment 
Add MySQL read history data bytes judgment
 2021-04-20 10:26:42 +03:00
Alexey Milovidov
ac68dd8e97 Fix bad code; allow to INSERT into MySQL without database name 2021-04-20 00:29:37 +03:00
TCeason
472c131420 Add MySQL read history data bytes judgment 2021-04-19 10:51:50 +08:00
kssenii
88b50621d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-18 14:40:37 +00:00
Alexander Tokmakov
0354312294 minor fixes in attach query 2021-04-17 22:43:02 +03:00
Alexander Tokmakov
3b4bd18809 fix prewhere in materialize mysql 2021-04-16 18:51:27 +03:00
Alexander Tokmakov
ebf0a3119e fix data_type_default_nullable on attach 2021-04-16 13:29:33 +03:00
Christian
969b63d1ac
Excludes views from syncing in MaterializeMySQL (#22760) 
* Excludes views from syncing in MaterializeMySQL

* Adds integration test for ignoring MySQL views on initial replication

* Fixes bug in initial integration test for excluding views from replicating to ClickHouse on initial replication

* Fixes bug in initial integration test for excluding views from replicating to ClickHouse on initial replication

* Replace assert with check_query

Co-authored-by: Christian Frøystad <christian@securepractice.no>
Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
 2021-04-14 14:19:07 +03:00
kssenii
01075677cf Better 2021-04-11 20:22:33 +00:00
kssenii
027f67affb Fix checks 2021-04-11 09:54:10 +00:00
kssenii
beae1c5fa4 Use ContextPtr 2021-04-11 08:08:32 +00:00
kssenii
c1adece94f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-11 06:00:18 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
kssenii
bc228f4010 Better way to drop nested table for single storage 2021-04-10 19:26:53 +00:00
kssenii
1c501e7d97 Support replica identity index 2021-04-10 16:10:28 +00:00
kssenii
12f98e8b11 Much better 2021-04-09 10:22:06 +00:00
kssenii
f0be5c6938 Fix style, build, ya check 2021-04-09 08:07:56 +00:00
kssenii
3e41c82df1 Add mutex to tryGetTable only for materialized table (not nested) 2021-04-08 23:38:27 +00:00
kssenii
bd207daa74 Use only atomic database 2021-04-08 22:38:46 +00:00
kssenii
4482a35a3a Check table in a suggested way 2021-04-08 20:39:16 +00:00
kssenii
8586565ce9 Move connection-related filed to Core/PostgreSQL 2021-04-08 12:43:02 +00:00
kssenii
9fa21a6272 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-08 12:18:54 +00:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Kseniia Sumarokova
ad85467128
Merge pull request #22668 from kssenii/postgres-add-bool 
Add bool type in postgres engine
 2021-04-06 13:29:33 +03:00
kssenii
cc2fdc6c62 Add bool type 2021-04-05 14:49:52 +00:00
kssenii
13d1f17d3e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-03 08:00:09 +00:00
kssenii
2de6b99f84 Better names 2021-04-02 16:12:14 +00:00
kssenii
99f0b08ac3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-01 13:48:33 +00:00
kssenii
522d170588 Add default port 2021-04-01 11:30:20 +00:00
kssenii
747967a9bb Better 2021-04-01 10:27:24 +00:00
kssenii
ba5ddd5c32 Do not parse replicas in PoolWithFailover 2021-03-31 14:03:52 +00:00
Maksim Kita
55984e849c
Merge branch 'master' into hierarchy-dictionaries-updated 2021-03-31 15:55:07 +03:00
kssenii
ce05087b1b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-03-30 17:27:10 +00:00
Alexander Tokmakov
a5b07f2f03 Merge branch 'master' into test_multiple_nodes 2021-03-29 22:43:25 +03:00
tavplubix
3c0f5a57ec
Merge pull request #21535 from ClickHouse/distributed_ddl_improvements 
Distributed DDL improvements
 2021-03-29 22:40:11 +03:00
tavplubix
e3cf2d99b7
Update DatabaseReplicated.cpp 2021-03-29 16:44:41 +03:00
kssenii
22b515fbc9 Add namespace, simplify names 2021-03-27 20:14:02 +00:00
kssenii
f40c582e7a Replicas/shards for postgres storage 2021-03-27 20:11:48 +00:00
kssenii
ae868208c2 Use pool with failover in mysql storage 2021-03-27 14:39:45 +00:00
Maksim Kita
9f2f0d1095 Refactored hierarchy dictionaries interface 2021-03-26 18:42:32 +03:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements 2021-03-22 18:13:19 +03:00
kssenii
ed158e2b77 Proper setting 2021-03-20 14:42:03 +00:00
kssenii
4c37ff3fbc Less lambdas 2021-03-20 13:17:30 +00:00
kssenii
91b79f95bc Better templates 2021-03-20 12:24:35 +00:00
kssenii
caffccd73e Fixes 2021-03-20 11:53:41 +00:00
kssenii
6bb81630e5 Small fixes 2021-03-20 10:32:11 +00:00
kssenii
619f0f51fd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-20 09:47:05 +00:00
kssenii
3b32c8fe9a Block on empty connection pool 2021-03-19 21:00:45 +00:00
kssenii
9057aad798 Better version 2021-03-19 08:11:36 +00:00
kssenii
f1ef87d966 Fix 2021-03-18 20:04:54 +00:00
Alexander Tokmakov
ce31d4608b fix tests 2021-03-18 22:13:21 +03:00
Alexander Tokmakov
343b8c906d Merge branch 'master' into test_multiple_nodes 2021-03-18 16:01:04 +03:00
Alexander Tokmakov
fdae70df6c fix tests 2021-03-18 15:50:03 +03:00
kssenii
3903d59d30 Better 2021-03-17 14:34:04 +00:00
kssenii
ae64a24844 Add connection pool 2021-03-17 13:55:47 +00:00
Nikita Mikhaylov
f51b41b7a4
Merge pull request #21710 from kssenii/replica-priorities 
Support replicas priority for postgres dictionary source
 2021-03-17 16:02:00 +03:00
kssenii
87c740730b Rename to MaterializePostgreSQL 2021-03-17 10:52:12 +00:00
Alexander Tokmakov
ff79be8af9 fix 2021-03-16 23:01:20 +03:00
kssenii
f8a99804c1 Add double quotes 2021-03-16 11:15:43 +00:00
kssenii
a03e849a8f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-16 10:06:10 +00:00
tavplubix
6a265f468e
Merge branch 'master' into distributed_ddl_improvements 2021-03-15 11:24:14 +03:00
tavplubix
6ff8f468f6
Update DatabaseReplicated.cpp 2021-03-15 11:22:42 +03:00
kssenii
8eea6a87e9 Support replicas priority for postgres dictionary source 2021-03-14 08:51:24 +00:00
tavplubix
26ebd54eaf
Merge pull request #20961 from havardk/reconnect 
MaterializeMySQL: Attempt to reconnect to MySQL if the connection is lost
 2021-03-10 20:07:11 +03:00
Alexander Tokmakov
ba3d51383c fix 2021-03-10 00:41:04 +03:00
tavplubix
5a6cc876e4
Merge pull request #20626 from ClickHouse/fix_force_drop_materialized_view 
Make force_drop_table work with MaterializedView
 2021-03-09 20:05:51 +03:00
Alexander Tokmakov
83b3e4e0f5 allow to use Cluster from Replicated db 2021-03-09 20:05:24 +03:00
Alexander Tokmakov
00b939e5a3 fix 2021-03-09 13:24:08 +03:00
Alexander Tokmakov
5c5d11cfce enable some tests 2021-03-09 03:48:37 +03:00
Alexander Tokmakov
163e27b299 support query and session settings for distributed DDL 2021-03-09 01:57:53 +03:00
Alexander Tokmakov
5070b5b85b detach permanently setting for Replicated db 2021-03-09 00:34:52 +03:00
Alexander Tokmakov
2022b90919 configurable output mode for distributed DDL 2021-03-08 23:35:09 +03:00
Alexander Tokmakov
2484781070 fix MV recovery in Replicated db 2021-03-08 20:26:38 +03:00
Alexander Tokmakov
e8987f799e Merge branch 'fix_force_drop_materialized_view' into just_another_fix_for_ddl_worker 2021-03-05 15:59:27 +03:00
Haavard Kvaalen
641b1b249e Attempt to reconnect to MySQL 
For MaterializeMySQL databases, attempt to reconnect if the connection
to MySQL is lost.  The existing setting
`max_wait_time_when_mysql_unavailable` is used to control how often we
attempt to reconnect.  This setting can now be set to a negative value
to disable reconnects.
 2021-03-04 08:49:06 +01:00
Haavard Kvaalen
8e95d6e174 Add method to metadata class to start replication 
Previously this was done by the constructor.  This makes it
possible to restart replication with an existing metadata object.
 2021-03-04 08:49:06 +01:00
Haavard Kvaalen
e6711675a1 Make handling of unavailable MySQL consistent 
If MySQL was unavailable when loading an existing database on startup,
we would previously load the database, fail to start synchronization,
but allow queries to the database.  Change this to only allow queries
if the `allows_queries_when_mysql_lost` setting is on, so that the
behavior is consistent with what happens if the connection to MySQL is
lost while ClickHouse is running.

Also retry connection to MySQL if MySQL is unavailable when ClickHouse
is started (we would previously reconnect only if the connection was
lost during the initial dump of existing data).
 2021-03-04 08:49:06 +01:00
kssenii
a3c7402200 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-02 07:07:56 +00:00
tavplubix
10aa26576c
Update DatabaseReplicatedWorker.cpp 2021-03-01 17:11:45 +03:00
Azat Khuzhin
955974a8b5 DDLWorker: avoid NULL dereference on termination and failed zookeeper initialization 
Log snipped shows the problem:

    2021.02.24 04:40:29.349181 [ 39 ] {} <Warning> DDLWorker: DDLWorker is configured to use multiple threads. It's not recommended because queries can be reordered. Also it may cause some unknown issues to appear.
    2021.02.24 04:40:29.349516 [ 39 ] {} <Information> Application: Ready for connections.
    2021.02.24 04:40:29.349602 [ 74 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread
    2021.02.24 04:40:29.349639 [ 73 ] {} <Debug> DDLWorker: Starting DDLWorker thread
    2021.02.24 04:40:29.349698 [ 73 ] {} <Debug> DDLWorker: Started DDLWorker thread
    2021.02.24 04:40:29.352548 [ 73 ] {} <Error> virtual void DB::DDLWorker::initializeMainThread(): Code: 999, e.displayText() = Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
     (Connection loss), Stack trace (when copying this message, always include the lines below):

    0. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0xfe93923 in /usr/bin/clickhouse
    1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0xfe93ba2 in /usr/bin/clickhouse
    2. Coordination::ZooKeeper::connect(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, Poco::Timespan) @ 0xfed3a01 in /usr/bin/clickhouse
    3. Coordination::ZooKeeper::ZooKeeper(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Timespan, Poco::Timespan, Poco::Timespan) @ 0xfed2222 in /usr/bin/clickhouse
    4. zkutil::ZooKeeper::init(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe961cd in /usr/bin/clickhouse
    5. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe97a96 in /usr/bin/clickhouse
    6. void std::__1::allocator_traits<std::__1::allocator<zkutil::ZooKeeper> >::__construct<zkutil::ZooKeeper, Poco::Util::AbstractConfiguration const&, char const (&) [10]>(std::__1::integral_constant<bool, true>, std::__1::allocator<zkutil::ZooKeeper>&, zkutil::ZooKeeper*, Poco::Util::AbstractConfiguration const&, char const (&) [10]) @ 0xed98387 in /usr/bin/clickhouse
    7. DB::Context::getZooKeeper() const @ 0xed75190 in /usr/bin/clickhouse
    8. DB::DDLWorker::getAndSetZooKeeper() @ 0xedb81c9 in /usr/bin/clickhouse
    9. DB::DDLWorker::initializeMainThread() @ 0xedc9eb0 in /usr/bin/clickhouse
    10. DB::DDLWorker::runMainThread() @ 0xedb5d01 in /usr/bin/clickhouse
    11. ThreadFromGlobalPool::ThreadFromGlobalPool<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0xedcafa1 in /usr/bin/clickhouse
    12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x892651f in /usr/bin/clickhouse
    13. ? @ 0x8929fb3 in /usr/bin/clickhouse
    14. start_thread @ 0x8ea7 in /lib/x86_64-linux-gnu/libpthread-2.31.so
    15. __clone @ 0xfddef in /lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.3.1.1)
    ...
    2021.02.24 04:40:30.025278 [ 41 ] {} <Trace> BaseDaemon: Received signal 15
    2021.02.24 04:40:30.025336 [ 41 ] {} <Information> Application: Received termination signal (Terminated)
    ...
    2021.02.24 04:40:30.582078 [ 39 ] {} <Information> Application: Closed all listening sockets.
    2021.02.24 04:40:30.582124 [ 39 ] {} <Information> Application: Closed connections.
    2021.02.24 04:40:30.583770 [ 39 ] {} <Information> Application: Shutting down storages.
    2021.02.24 04:40:30.583932 [ 39 ] {} <Information> Context: Shutdown disk data
    2021.02.24 04:40:30.583951 [ 39 ] {} <Information> Context: Shutdown disk default
    2021.02.24 04:40:30.584163 [ 46 ] {} <Trace> SystemLog (system.query_log): Terminating
    2021.02.24 04:40:30.586025 [ 39 ] {} <Trace> BackgroundSchedulePool/BgSchPool: Waiting for threads to finish.
    2021.02.24 04:40:34.352701 [ 73 ] {} <Debug> DDLWorker: Initialized DDLWorker thread
    2021.02.24 04:40:34.352758 [ 73 ] {} <Debug> DDLWorker: Scheduling tasks
 2021-02-28 05:45:12 +03:00
kssenii
6554965711 Fix build 2021-02-23 10:46:41 +00:00
Kseniia Sumarokova
bf88d693d5
Update fetchPostgreSQLTableStructure.cpp 2021-02-23 11:59:38 +03:00
Kseniia Sumarokova
6c5781cf34
Update fetchPostgreSQLTableStructure.cpp 2021-02-23 11:48:51 +03:00
Kseniia Sumarokova
b2850ec94f
Merge branch 'master' into fix-postgres-numeric 2021-02-23 11:40:28 +03:00
kssenii
4cdb55babe Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-02-22 12:39:35 +00:00
kssenii
ace76bb7fa Better, remove debug logs 2021-02-22 12:35:53 +00:00
kssenii
8a48bb24ce Fix typos, comments 2021-02-21 23:13:58 +00:00
kssenii
427aad80a1 Avoid table does not exist errors if nested is unavailable 2021-02-21 11:48:13 +00:00
tavplubix
aa48d967b7
Update DatabaseAtomic.cpp 2021-02-21 01:13:14 +03:00
kssenii
d590c32708 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-02-20 18:31:26 +00:00
Alexey Milovidov
64e76a4a8d Minor changes in Decimal 2021-02-20 21:13:36 +03:00
tavplubix
103d569885
Merge branch 'master' into fix_force_drop_materialized_view 2021-02-20 20:41:03 +03:00
kssenii
0424770f68 Handle ddl part 1 2021-02-20 16:21:53 +00:00
Alexander Tokmakov
863c099254 fix 2021-02-20 15:57:07 +03:00
kssenii
a38a31c954 Fix typos check 2021-02-20 09:58:24 +00:00
kssenii
f820047cc8 Fix 2021-02-20 06:52:10 +00:00
Alexander Tokmakov
2a36d6cb55 review suggestions 2021-02-20 02:41:58 +03:00
kssenii
3107f82a3b Fix clang tidy 2021-02-18 18:25:49 +00:00
kssenii
2aad067e7c Support conversion for postgres numeric without precision and scale 2021-02-18 12:16:58 +00:00
kssenii
1480e95179 Fix build, fix arcadia 2021-02-18 06:06:37 +00:00
Alexander Tokmakov
13ae988eff make force_drop_table work with materialized view 2021-02-17 17:23:07 +03:00
Alexander Tokmakov
7b54b892b5 fix 2021-02-16 17:05:58 +03:00
kssenii
272431bcae Fix types check, better tests, try fix build 2021-02-15 23:18:53 +00:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
Alexander Tokmakov
cf57c3b4a2 update comments 2021-02-15 23:01:21 +03:00
Alexander Tokmakov
9c7cf9e92e remove some debug code 2021-02-15 13:26:34 +03:00
Alexander Tokmakov
320fd6b264 startup without zk 2021-02-15 03:04:46 +03:00
kssenii
95c93aca41 Better table sync 2021-02-13 21:03:25 +00:00
kssenii
44f4f1a412 Allow to replicate a subset of database tables 2021-02-13 20:09:01 +00:00
kssenii
219dece1d0 Slightly better 2021-02-12 21:15:00 +00:00
kssenii
5bc0010f94 Better 2021-02-12 21:14:36 +00:00
kssenii
ed6d19b8a6 Add PostgreSQLReplica database engine 2021-02-12 21:14:24 +00:00
Alexander Tokmakov
f64f9b672b fix 2021-02-12 19:22:01 +03:00
Alexander Tokmakov
c9cf63e958 fix 2021-02-12 01:23:40 +03:00
Alexander Tokmakov
9afb16759e fix 2021-02-11 18:14:38 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Alexander Tokmakov
2c6a0e74fb better replica creation 2021-02-09 18:14:20 +03:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
Alexander Tokmakov
91d0924665 write dictionaries metadata to zk 2021-02-08 12:46:30 +03:00
Alexander Tokmakov
7ce0ef2561 show clusters for replicated db 2021-02-08 12:14:17 +03:00
Alexander Tokmakov
18f6b5bbad add timeouts 2021-02-04 22:41:44 +03:00
Alexander Tokmakov
066fb4c82b fix 2021-02-03 23:02:37 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Alexander Tokmakov
6456ccf0da better test 2021-02-02 22:39:04 +03:00
Alexander Tokmakov
0073c87d5d fix 2021-02-02 13:32:42 +03:00
Alexander Tokmakov
9da445e740 execute initial query in the same thread 2021-02-01 22:29:47 +03:00
Vitaly Baranov
7c8deae0e1 Fix starting the server with tables having columns' default expressions containing dictGet(). 
Allow getting return type of dictGet() without loading dictionary.
 2021-01-30 19:06:48 +03:00
Alexander Tokmakov
f20d5e3b41 fix 2021-01-26 20:51:25 +03:00
Alexander Tokmakov
3bd4d97353 Merge branch 'master' into database_replicated 2021-01-25 14:19:04 +03:00
tavplubix
4be33e81ee
Merge pull request #19344 from zhang2014/patch-2 
ISSUES-18684 fix MaterializeMySQL integration test failure
 2021-01-25 12:46:53 +03:00
zhang2014
4593ca80ff ISSUES-18684 fix MaterializeMySQL integration failure 2021-01-24 15:50:37 +08:00
kssenii
a98c6d333c Make sure not to log non-existent objects 2021-01-23 12:05:37 +00:00
kssenii
daab2c91bb Better 2021-01-21 21:15:11 +00:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
Alexander Tokmakov
7f97a11c84 Merge branch 'master' into database_replicated 2021-01-18 17:09:39 +03:00
tavplubix
071a6aeb4d
trigger CI 2021-01-18 16:53:33 +03:00
TCeason
a3cc6707b4 add MySQL Var check 
add MySQL var check: log_bin_use_v1_row_events=OFF
 2021-01-17 17:50:35 +08:00
kssenii
2222741e33 Fix style check 2021-01-16 09:17:04 +00:00
Alexey Milovidov
45380d45c8 Fix potential (safe) SQL-injection 2021-01-16 11:15:43 +03:00
kssenii
11ffa6293d Fix integration tests 2021-01-16 08:04:24 +00:00
Alexey Milovidov
593b6f5c83 Add assert 2021-01-16 00:25:58 +03:00
Alexey Milovidov
67f207d8ca More safe connection string 2021-01-16 00:18:51 +03:00
Alexey Milovidov
ef319bfa75 Slightly better 2021-01-15 23:33:04 +03:00
Alexey Milovidov
c16ecb0bdd It feels more correct 2021-01-15 23:18:55 +03:00
Alexey Milovidov
b1b09ed5ae Fix SQL injection 2021-01-15 23:14:51 +03:00
Alexey Milovidov
52cfc1d110 Minor changes 2021-01-15 23:09:31 +03:00
Alexey Milovidov
ba1921f380 Minor changes 2021-01-15 22:59:49 +03:00
Alexey Milovidov
a19e7edd14 Merge branch 'master' into kssenii-pg2ch 2021-01-15 17:33:19 +03:00
kssenii
38a9cba850 Fix 2021-01-11 10:55:38 +00:00
kssenii
c6c6b2d23c Fix style and types check 2021-01-10 13:01:15 +00:00
kssenii
d952b0897e Minor adjustments 2021-01-10 12:06:18 +00:00
kssenii
fc9de76f7d Fix incorrect getCreateTableQueryImpl with multidim arrays 2021-01-10 09:00:35 +00:00
Alexey Milovidov
76149947ef Remove useless headers 2021-01-10 05:48:57 +03:00
kssenii
0088d66d59 Hopefully better array conversions for insert query 2021-01-09 22:42:42 +00:00
Alexey Milovidov
8af19c3251 Fix Arcadia 2021-01-07 15:29:02 +03:00
Alexey Milovidov
60d4db421c Fix Arcadia 2021-01-07 06:45:12 +03:00
alexey-milovidov
72b142a00a
Merge branch 'master' into pg2ch 2021-01-06 23:18:59 +03:00
kssenii
e100aec086 Fix typos check 2020-12-31 11:15:05 +00:00
kssenii
62b06de97e Try fix build 2020-12-30 16:00:05 +00:00
spongedc
494c9ac809 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-29 10:20:06 +08:00
kssenii
fa3297bf72 Fix build, update libpq 2020-12-28 19:16:10 +00:00
kssenii
9b25890674 Generate ya.make, fix fast test 2020-12-28 12:54:52 +00:00
kssenii
8f8920a7ee Add table cache, better drop table 2020-12-27 15:52:15 +00:00
kssenii
00a37404ca Add postgresql database engine 2020-12-27 12:26:29 +00:00
Alexander Tokmakov
42911e2438 implement CREATE OR REPLACE TABLE 2020-12-25 19:32:09 +03:00
Yatsishin Ilya
ce1e2b18a3 Fix codespell warnings. Split style checks. Update style checks docker 2020-12-24 13:17:52 +03:00
spongedc
a277308ac6 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-23 11:56:46 +08:00
Alexander Tokmakov
52ef4289fe enable optimize_on_insert for MaterializeMySQL 2020-12-17 16:05:38 +03:00
tavplubix
7e73b6acd8
Merge pull request #17642 from filimonov/detach-permanently 
DETACH TABLE PERMANENTLY
 2020-12-17 14:26:51 +03:00
Nikolai Kochetov
44d2194326 Try fix integration tests. 2020-12-16 11:52:04 +03:00
spongedc
38c0827881 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-16 12:26:56 +08:00
spongedc
a0083e23d6 Fix compile error 2020-12-16 12:07:50 +08:00
Mikhail Filimonov
9df7ecb8e6
fixes 2020-12-15 18:06:23 +01:00
Mikhail Filimonov
27ff3a5214
Rework using a separate flag file instead of renames 2020-12-15 15:46:13 +01:00
spongedc
dc27e0a4fd Fix error code 2020-12-15 20:29:26 +08:00
spongedc
8b2feaeca2 Support SHOW CREATE VIEW Syntax 2020-12-14 22:37:25 +08:00
Mikhail Filimonov
24abcc970b
Tests and fixes 2020-12-14 15:31:13 +01:00
Mikhail Filimonov
87ea1528cb
support for Lazy database 2020-12-14 15:31:13 +01:00
Mikhail Filimonov
4ba80996b0
Detach permanently for DatabaseConnectionMySQL 2020-12-14 15:31:12 +01:00
Mikhail Filimonov
799141362d
Prevent attempts to recreate the table which was detached permanently 2020-12-14 15:31:11 +01:00
Mikhail Filimonov
f6097cf818
Adding DETACH TABLE PERMANENTLY 2020-12-14 15:28:59 +01:00
Alexander Tokmakov
58a31e4882 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-11 16:14:03 +03:00
tavplubix
5f78280aec
Merge pull request #17915 from zhang2014/better_exception_message 
Better exception message for MaterializeMySQL
 2020-12-10 19:26:20 +03:00
tavplubix
f13de96afc
Merge pull request #17366 from zhang2014/fix/ISSUES-16835 
ISSUES-16835 try fix miss match header with MySQL SHOW statement
 2020-12-09 23:12:47 +03:00
zhang2014
bf30aa5ff1 Code style 2020-12-09 16:50:40 +08:00
zhang2014
ef15b1f1fb Better exception message for MaterializeMySQL 2020-12-09 16:46:54 +08:00
Ivan
0ba712a2d3
Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-08 16:45:51 +03:00
Winter Zhang
972cb41fae
Merge branch 'master' into fix/ISSUES-16835 2020-12-08 15:32:58 +08:00
Alexander Tokmakov
39532f7d9e slightly better DDLWorker initialization and restarting 2020-12-03 21:14:27 +03:00
Alexander Tokmakov
a0ef3a113a Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-02 18:29:02 +03:00
Alexander Tokmakov
1a4bd67736 fixes 2020-12-01 20:20:42 +03:00
Alexander Tokmakov
19c8399eb0 Merge branch 'master' into database_replicated 2020-12-01 17:33:07 +03:00
Vasily Nemkov
87945fb68b Merge branch 'master' into HEAD 2020-11-30 23:25:57 +03:00
Alexander Tokmakov
ab197a49c8 better code, fixes 2020-11-30 23:12:43 +03:00
Alexander Tokmakov
f1a52a609b separate DatabaseReplicatedDDLWorker 2020-11-27 17:04:03 +03:00
zhang2014
a74235397e ISSUES-16835 try fix miss match header with MySQL SHOW statement 2020-11-24 23:31:43 +08:00
Alexander Tokmakov
e82c63e05c fixes 2020-11-24 15:28:54 +03:00
Alexander Tokmakov
dce0bf8efc Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-11-24 15:28:31 +03:00
Alexander Tokmakov
5ccf975a4a Merge branch 'master' into database_replicated 2020-11-24 14:38:02 +03:00
Alexander Tokmakov
dad21ee684 maintain metadata in zk 2020-11-24 13:24:39 +03:00
taichong
a6c2a90756 modify integration test materialize mysql database 
1. fix asan test err

2. reset pymysql conn if ping err

3. finish mysql & ck killed test

4. modify check_query
 2020-11-23 17:55:07 +08:00
Alexey Milovidov
fbbf439338 Merge branch 'master' into Enmk-DateTime64_fixes_comparison 2020-11-23 05:46:17 +03:00
tavplubix
5cc9cb01cd
Merge pull request #16751 from amosbird/globalcontext 
Make global_context consistent.
 2020-11-22 18:46:17 +03:00
Nikolai Kochetov
bac1def5f9
Merge pull request #17134 from abyss7/tcp-port 
Implement tcpPort() function for tests
 2020-11-20 20:32:55 +03:00
Alexander Tokmakov
7ab4445e99 try another approach 2020-11-20 19:06:27 +03:00
tavplubix
3e335c1e95
Merge pull request #17091 from zhang2014/fix/binlog_checksum 
Fix MaterializeMySQL SYNC failure when modify binlog_checksum
 2020-11-20 13:25:14 +03:00
Amos Bird
1d9d586e20
Make global_context consistent. 2020-11-20 18:23:14 +08:00
Alexander Tokmakov
2283906a11 try support replica recovery 2020-11-19 13:34:45 +03:00
Ivan Lezhankin
fdd1c182ab Merge remote-tracking branch 'upstream/master' into tcp-port 2020-11-18 16:22:00 +03:00
Ivan Lezhankin
f897f7c93f Refactor IFunction to execute with const arguments 2020-11-17 16:24:45 +03:00
alexey-milovidov
b494b18748
Merge pull request #17108 from ClickHouse/ya-make-owners 
Add "OWNER" to ya.make files.
 2020-11-17 05:19:46 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
Winter Zhang
f6a205d64b
Merge branch 'master' into fix/binlog_checksum 2020-11-16 21:53:22 +08:00
zhang2014
6ef93ac73c Try fix MaterializeMySQL SYNC with modify binlog_checksum 2020-11-16 21:46:36 +08:00
Vasily Nemkov
63d168d34e Fixed MySQL and DateTime64 integration. 
Also updated other places, like type helpers/dispatchers where DateTime64
was implicitly supported before as Decimal64.
 2020-11-16 09:02:41 +03:00
Alexander Tokmakov
b0262b3d06 better replica creation 2020-11-13 21:35:45 +03:00
Alexander Tokmakov
0731bfb7be Merge branch 'master' into database_replicated 2020-11-12 21:04:18 +03:00
Alexander Tokmakov
2a6c0b9180 try reuse DDLWorker in DatabaseReplicated 2020-11-12 20:35:29 +03:00
taichong
010e12eb4b Replace std::*stringstreams 2020-11-12 13:49:56 +08:00
TCeason
3912ed4965 Merge branch 'master' into add_sync_user_privs_exception 2020-11-12 09:34:08 +08:00
taichong
e2a9fe44b0 modify test error 2020-11-12 08:17:13 +08:00
taichong
5c1a01897a Revise the code according to the review 2020-11-11 19:27:37 +08:00
taichong
d907afa77e throw exception when mysql sync user privs error. 2020-11-11 19:27:37 +08:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexander Tokmakov
62ff00ee8b use WriteBuffer in formatAST(...) 2020-11-09 19:05:40 +03:00
tavplubix
6fb077c2f3
Merge pull request #16704 from TCeason/make_sync_ddl_execute_atomic 
sync MySQL DDL atomicly
 2020-11-09 13:47:02 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
taichong
9ce6c4dcb8 according to review modify code 2020-11-06 15:04:05 +08:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
taichong
fb5e03ebaf sync MySQL DDL atomicly 
Now ClickHouse will write metadata and then execute DDL from MySQL.

But If ClickHouse is crashed and it just begin to execute DDL, we
restart the server and then the sync will may be stopped.

e.g.:

MySQL : create table foo.a(id int key); uuid1:1
ClickHouse : write uuid1:1 to .metadata -> server crashed;

Because of ClickHouse's metadata records: uuid1:1, if MySQL run a
query:`insert into foo.a values(1);` ClickHouse will throw a exception:
the table foo.a is not exist.
 2020-11-05 17:01:14 +08:00
Alexander Tokmakov
d8ae9fcdb4 fixes, add shard name 2020-11-03 14:30:53 +03:00
Alexander Tokmakov
ac32809b6a fix #16482 2020-11-02 19:40:39 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
alexey-milovidov
bf3acecf6f
Merge pull request #16032 from Avogar/mysql_bug 
Don't touch MySQL database if it's unnecessary
 2020-10-29 04:14:26 +03:00
tavplubix
22847613d1
Merge pull request #16437 from ClickHouse/addition_to_16127 
Addition to #16127
 2020-10-28 13:03:13 +03:00
alesapin
8a6be1602b
Merge pull request #16364 from ClickHouse/fix_uuid_mapping_exists 
Minor improvements in DatabaseCatalog
 2020-10-28 10:54:39 +03:00
Alexander Tokmakov
b5ccb5ed5b review suggestions 2020-10-27 23:52:49 +03:00
Alexander Tokmakov
dba8c85ae4 do not wait while holding lock 2020-10-27 22:56:54 +03:00
Alexander Kuzmenkov
6509f1eb39
Merge branch 'master' into minumum 2020-10-27 21:38:23 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
Alexander Tokmakov
cd14f095ab fix tests 2020-10-26 18:12:16 +03:00
Alexander Tokmakov
636da1318b Merge branch 'master' into database_replicated 2020-10-26 15:00:43 +03:00
Alexey Milovidov
d5e277f080 Better exceptions rethrow 2020-10-26 06:35:57 +03:00
Alexander Tokmakov
f8f2d62d70 minor improvements in DatabaseCatalog 2020-10-25 20:27:21 +03:00
Alexander Tokmakov
478eb0b8a5 fix 2020-10-22 23:32:47 +03:00
Alexander Tokmakov
72dbb43235 Merge branch 'master' into fork-valbaturin-replication-db-engine 2020-10-20 19:14:54 +03:00
Azat Khuzhin
32c449c34f Add IDatabaseTablesIterator::reset() interface 2020-10-19 00:16:10 +03:00
alesapin
0bb4480fee
Merge pull request #16031 from ClickHouse/database_atomic_patch 
Minor fix for DatabaseAtomic
 2020-10-16 10:08:04 +03:00
Alexander Tokmakov
2f9611f1ae fix logical error with explicit UUIDs 2020-10-15 19:01:23 +03:00
Pavel Kruglov
ee8a9d2cf1 Don't touch MySQL database if it's unnecessary 2020-10-15 18:57:17 +03:00
Alexander Tokmakov
72b1339656 Revert "Revert "Write structure of table functions to metadata"" 
This reverts commit c65d1e5c70.
 2020-10-14 15:19:29 +03:00
tavplubix
c65d1e5c70
Revert "Write structure of table functions to metadata" 2020-10-14 13:59:29 +03:00
alexey-milovidov
f60ccb4edf
Merge pull request #14295 from ClickHouse/write_structure_of_table_functions 
Write structure of table functions to metadata
 2020-10-13 23:56:09 +03:00
Azat Khuzhin
d16adddb41 Fix drop of materialized view with inner table in Atomic database 
Materialized view drop the inner table (if any) drop the drop() method,
and this will lead to essure recursive drop() from the drop worker
thread, which will hang:

    3  std::__1::condition_variable::wait<> ()
    4  DB::DatabaseCatalog::waitTableFinallyDropped (this=0x7ffff7831500, uuid=...) at ../src/Interpreters/DatabaseCatalog.cpp:828
    5  DB::InterpreterDropQuery::executeToTable (this=this@entry=0x7fff16ff88a0, table_id_=..., query=...) at ../src/Interpreters/InterpreterDropQuery.cpp:135
    6  DB::InterpreterDropQuery::execute (this=this@entry=0x7fff16ff88a0) at ../contrib/libcxx/include/string:1474
    7  DB::executeDropQuery (kind=kind@entry=DB::ASTDropQuery::Drop, global_context=..., target_table_id=...) at ../src/Storages/StorageMaterializedView.cpp:156
    8  DB::StorageMaterializedView::drop (this=0x7ffefc348f40) at ../src/Storages/StorageMaterializedView.cpp:169
    9  DB::DatabaseCatalog::dropTableFinally (this=this@entry=0x7ffff7831500, table=...) at ../src/Interpreters/DatabaseCatalog.cpp:775
    10 DB::DatabaseCatalog::dropTableDataTask (this=0x7ffff7831500) at ../src/Interpreters/DatabaseCatalog.cpp:745

Fix this by dropping the inner table just before scheduling job the the
drop worker thread.
 2020-10-12 21:46:07 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Alexander Tokmakov
6b2da861dc Merge branch 'master' into write_structure_of_table_functions 2020-10-02 20:46:58 +03:00
Alexander Tokmakov
32776da69a Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-10-02 20:09:37 +03:00
tavplubix
2f95609f98
Merge pull request #15448 from ClickHouse/allow_synchronous_drop_for_atomic_db 
Allow synchronous DROP/DETACH TABLE for Atomic
 2020-10-01 01:14:31 +03:00
tavplubix
c9eeb149fd
Merge pull request #15384 from zhang2014/fix/ISSUES-15365 
ISSUES-15365 fix attach mysql database database
 2020-09-30 11:27:15 +03:00
Alexander Tokmakov
e156e6115e add setting 2020-09-30 01:30:34 +03:00
Alexander Tokmakov
5d19ed04f6 allow synchronous drop/detach table for Atomic 2020-09-29 16:47:08 +03:00
zhang2014
9e94c077e7 IESSUES-15365 try fix integration test 2020-09-29 10:39:30 +08:00