thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-04 21:42:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
132,312 Commits 9,185 Branches 1,782 Tags 2.3 GiB
1593d33883
Commit Graph

1448 Commits

Author SHA1 Message Date
Alexey Milovidov
f398fe9fe9 Merge branch 'master' into database-filesystem-remove-catch 2023-07-27 05:00:33 +02:00
Alexander Tokmakov
6bbed6262e
Merge branch 'master' into add_delay_for_replicated 2023-07-26 12:48:48 +03:00
Azat Khuzhin
85082ad8f8
Fix data-race DatabaseReplicated::startupTables()/canExecuteReplicatedMetadataAlter() (#52490) 
CI founds [1]:

    Exception: Sanitizer assert found for instance ==================
    WARNING: ThreadSanitizer: data race (pid=348)
      Write of size 8 at 0x7b58000044a0 by main thread:
        2 DB::DatabaseReplicated::startupTables(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, DB::LoadingStrictnessLevel) build_docker/./src/Databases/DatabaseReplicated.cpp:526:16 (clickhouse+0x1ec45092)
        3 DB::TablesLoader::startupTables() build_docker/./src/Databases/TablesLoader.cpp:87:26 (clickhouse+0x1f9258ab) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        4 DB::loadMetadata(std::__1::shared_ptr<DB::Context>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./src/Interpreters/loadMetadata.cpp:234:12 (clickhouse+0x1fff3834) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        5 DB::Server::main() build_docker/./programs/server/Server.cpp:1615:9 (clickhouse+0x163e7f78) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        6 Poco::Util::Application::run() build_docker/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x257608fe) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        7 DB::Server::run() build_docker/./programs/server/Server.cpp:391:25 (clickhouse+0x163d7d7c) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        8 Poco::Util::ServerApplication::run(int, char**) build_docker/./base/poco/Util/src/ServerApplication.cpp:131:9 (clickhouse+0x25780114) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        9 mainEntryClickHouseServer(int, char**) build_docker/./programs/server/Server.cpp:196:20 (clickhouse+0x163d4c23) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        10 main build_docker/./programs/main.cpp:487:12 (clickhouse+0xdf8c877) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)

      Previous read of size 8 at 0x7b58000044a0 by thread T27 (mutexes: write M0, write M1):
        1 DB::DatabaseReplicated::canExecuteReplicatedMetadataAlter() const build_docker/./src/Databases/DatabaseReplicated.cpp:1303:12 (clickhouse+0x1ec5c5bd)
        2 DB::ReplicatedMergeTreeQueue::shouldExecuteLogEntry() const build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1471:24 (clickhouse+0x2115fb56) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        3 DB::ReplicatedMergeTreeQueue::selectEntryToProcess(DB::MergeTreeDataMergerMutator&, DB::MergeTreeData&) build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1676:13 (clickhouse+0x21163c58) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        4 DB::StorageReplicatedMergeTree::selectQueueEntry() build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3240:26 (clickhouse+0x20823db2) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        5 DB::StorageReplicatedMergeTree::scheduleDataProcessingJob(DB::BackgroundJobsAssignee&) build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3304:65 (clickhouse+0x208240fc) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/52395/0b258dda4ee618a4d002e2b5246d68bbd2c77c7e/integration_tests__tsan__[5_6].html

Add ddl_worker_initialized flag to avoid this race.

Note, that it should be enough to check this flag only in
canExecuteReplicatedMetadataAlter() since only it can be run in parallel
with ctor before it had been finished.

v0: initialize ddl before startupTables()
v2: ddl_worker_initialized

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-07-25 17:35:01 +03:00
Alexey Milovidov
3ba47ae0d4 Merge branch 'master' into database-filesystem-remove-catch 2023-07-24 19:53:42 +02:00
Alexander Tokmakov
faca49a905 Merge branch 'master' into add_delay_for_replicated 2023-07-24 16:07:38 +02:00
Val Doroshchuk
efa638ef3c MaterializedMySQL: Support unquoted utf-8 strings in DDL 
Since ClickHouse does not support unquoted utf-8 strings but MySQL does.

Instead of fixing Lexer to recognize utf-8 chars as TokenType::BareWord,
suggesting to quote all unrecognized tokens before applying any DDL.

Actual parsing and validating the syntax will be done by particular Parser.

If there is any TokenType::Error, the query is unable to be parsed anyway.
Quoting such tokens can provide the support of utf-8 names.

See `tryQuoteUnrecognizedTokens` and `QuoteUnrecognizedTokensTest`.

mysql> CREATE TABLE 道.渠(...

is converted to

CREATE TABLE `道`.`渠`(...

Also fixed the bug with missing * while doing SELECT in full sync because db or table name are back quoted when not needed.
 2023-07-24 11:12:10 +02:00
Azat Khuzhin
8013cb1f78 Remove skip_startup_tables from IDatabase::loadStoredObjects() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-23 08:47:50 +02:00
alesapin
6416fb6eed
Merge branch 'master' into add_delay_for_replicated 2023-07-22 12:11:39 +02:00
Sergei Trifonov
8eaf3b6d94
Merge branch 'master' into async-loader-integration 2023-07-21 21:46:06 +02:00
Nikita Mikhaylov
668062dc29
Merge branch 'master' into no-keep-context-lock-while-calculating-access 2023-07-20 14:42:13 +02:00
Sergei Trifonov
b8a46ff822
Merge branch 'master' into async-loader-integration 2023-07-19 18:45:41 +02:00
serxa
6bca79fdff add auto-resolution for all resolvable deadlock types 2023-07-19 16:19:28 +00:00
Sergei Trifonov
850ab23991
Merge branch 'master' into async-loader-integration 2023-07-17 19:14:54 +02:00
Nikita Mikhaylov
11cc8b4adf
Merge branch 'master' into add_delay_for_replicated 2023-07-17 19:04:29 +02:00
Vitaly Baranov
815a3857de Remove non-const function Context::getClientInfo(). 2023-07-17 15:02:07 +02:00
Val Doroshchuk
4da0782e55 MaterializedMySQL: Add tests to parse db and table names from DDL 2023-07-17 15:01:06 +02:00
Alexey Milovidov
1050fab8de Remove another exception 2023-07-16 23:12:08 +02:00
Alexey Milovidov
72e8303ee7 Remove try/catch from DatabaseFilesystem 2023-07-16 22:49:06 +02:00
Alexander Sapin
30bcc73c63 Merge branch 'master' into add_delay_for_replicated 2023-07-06 14:29:15 +02:00
Alexey Milovidov
5452f2106f
Merge branch 'master' into sunny19930321-fix-attach-table-function-name-normalizer 2023-07-06 03:50:18 +03:00
alesapin
baee73fd96 Make shutdown of replicated tables softer 2023-07-05 18:11:25 +02:00
Sergei Trifonov
bbb5850dec
Merge branch 'master' into async-loader-integration 2023-07-03 20:21:26 +02:00
Kseniia Sumarokova
19508a01a5
Merge branch 'master' into resubmit-51149 2023-07-03 13:12:53 +02:00
Alexey Milovidov
2a7ae8f622
Merge branch 'master' into sunny19930321-fix-attach-table-function-name-normalizer 2023-06-30 12:16:49 +03:00
Alexander Tokmakov
a28728b7e5 Update DatabaseReplicatedWorker.cpp 
(cherry picked from commit 43fc1af1b3)
 2023-06-29 19:04:49 +02:00
kssenii
44246363e2 Fix test 2023-06-29 15:37:36 +02:00
Alexey Milovidov
a4d7e48bc9
Merge branch 'master' into resubmit-51149 2023-06-29 11:07:01 +03:00
Alexander Tokmakov
1ed104417a fix race between executeMetadataAlter and initializeReplication 
(cherry picked from commit 1d47783e85)
 2023-06-27 23:24:02 +02:00
serxa
31b4417d34 use current priority for waits from load jobs 2023-06-27 17:44:28 +00:00
Alexey Milovidov
8610ff828e Merge branch 'fix-attach-table-function-name-normalizer' of github.com:sunny19930321/ClickHouse into sunny19930321-fix-attach-table-function-name-normalizer 2023-06-27 07:58:42 +02:00
Sergei Trifonov
e2798b5221
Merge branch 'master' into async-loader-integration 2023-06-26 19:18:23 +02:00
Alexander Tokmakov
8afb8bf13a disable table structure check for secondary queries from Replicated db 2023-06-22 19:06:28 +02:00
Alexander Tokmakov
50ee424148 do not update digest during recovery 2023-06-21 21:07:59 +02:00
Alexander Tokmakov
9240a82c64
Update DatabaseReplicated.cpp 2023-06-21 21:49:00 +03:00
kssenii
0efaecab51 Revert "Merge pull request #51149 from ClickHouse/revert-48821-localfilefunction" 
This reverts commit a09e6bbb8e, reversing
changes made to ce38d64c5a.
 2023-06-21 00:29:46 +02:00
Sergei Trifonov
02e0d7e204
Merge branch 'master' into async-loader-integration 2023-06-20 12:03:29 +02:00
Sergei Trifonov
5867f39fc7
Merge pull request #50675 from ClickHouse/unify-priorities-pools 
Unify priorities for connection pools
 2023-06-19 17:16:15 +02:00
Alexander Tokmakov
caceb7c862
Revert "Added ability to implicitly use file/hdfs/s3 table functions in clickhouse-local" 2023-06-19 13:21:09 +03:00
Alexey Milovidov
e913a58284
Merge pull request #51049 from ClickHouse/ddl_replication_improvements 
An optimiation for ALTERs and Replicated db with one shard
 2023-06-17 16:57:59 +03:00
kssenii
74f2dea57b Fix typo 2023-06-17 02:11:48 +02:00
kssenii
b902f5d045 Small style corrections 2023-06-17 01:54:38 +02:00
Alexey Milovidov
df9abccfaa
Merge branch 'master' into localfilefunction 2023-06-17 00:45:54 +03:00
Alexander Tokmakov
353b57f13f an optimiation for alters and replicated db 2023-06-15 20:20:11 +02:00
alekseygolub
d72751be27 Added cache invalidation; Fix issues 2023-06-11 15:01:45 +00:00
Alexey Milovidov
2a126aaa76
Merge branch 'master' into fix-attach-table-function-name-normalizer 2023-06-10 13:14:17 +03:00
serxa
2df0e0c669 Unify priorities for connections 2023-06-07 16:25:52 +00:00
serxa
4dd026e1ef do not use table uuid in TablesLoader 2023-06-05 20:12:30 +00:00
serxa
47d70db2de clean 2023-06-03 10:52:23 +00:00
serxa
301dfc006c wait db load before DROP table from Atomic Database 2023-06-03 10:48:46 +00:00
Sergei Trifonov
3ee782a0ec
Merge branch 'master' into async-loader-integration 2023-06-02 18:27:19 +02:00
Alexander Tokmakov
e1d044dfd7
Merge pull request #50392 from ClickHouse/fix_materialized_mysql_poll_timeout 
Fix poll timeout in MaterializedMySQL
 2023-06-01 15:16:59 +03:00
Alexey Milovidov
956c399b2a Remove useless code 2023-06-01 03:04:29 +02:00
serxa
303e3f00c9 add helpers for prioritize+wait calls and fix build 2023-05-31 17:47:36 +00:00
serxa
5048050085 use pool ids instead of priorities in load jobs 2023-05-31 16:53:10 +00:00
serxa
e1045f76de rework async loader initialization and reconfiguration for bg/fg pools 2023-05-31 16:23:41 +00:00
Alexander Tokmakov
f74d7474d2 fix poll timeout in MaterializedMySQL 2023-05-31 15:04:41 +02:00
Sergei Trifonov
fe172f59db
Merge branch 'master' into async-loader-integration 2023-05-26 18:56:08 +02:00
Alexander Gololobov
8996fcb090
Merge pull request #50193 from ClickHouse/fix_for_replicate_delete 
Don't replicate delete through DDL worker if there is just 1 shard
 2023-05-24 22:45:00 +02:00
Alexander Gololobov
de0a074545 Don't replicate delete through DDL worker if there is just 1 shard 2023-05-24 16:10:31 +02:00
alekseygolub
34fe8ba5cd
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-24 09:49:03 +03:00
serxa
87a16d852f fix tidy build 2023-05-23 17:07:30 +00:00
serxa
a9db6bd830 refactoring 2023-05-23 17:04:40 +00:00
serxa
edbc9ba104 fix style 2023-05-23 13:43:23 +00:00
Alexander Tokmakov
3ac7bc90ef
Merge pull request #50108 from ClickHouse/update_replicated_database_settings 
Update default settings for Replicated database
 2023-05-23 12:56:12 +03:00
Alexander Tokmakov
5b768ebd97 update default settings for Replicated database 2023-05-22 19:32:32 +02:00
Sergei Trifonov
994cfaf15b
Merge branch 'master' into async-loader-integration 2023-05-22 19:03:11 +02:00
serxa
4127599a0e fix UNKNOWN_TABLE problem during async startup 2023-05-22 17:02:54 +00:00
Nikolay Degterinsky
7bed59e1d2
Merge pull request #50000 from evillique/add-schema-inference 
Add schema inference to more table engines
 2023-05-22 17:24:30 +02:00
alekseygolub
70b9077bb2
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-21 17:49:29 +03:00
Nikolay Degterinsky
b8be714830 Add schema inference to more table engines 2023-05-19 00:44:27 +00:00
serxa
646bc6b443 fix 2023-05-12 20:27:41 +00:00
serxa
9bf5292415 refact 2023-05-12 19:49:47 +00:00
Sergei Trifonov
a30a0d80f2
Merge branch 'master' into async-loader 2023-05-12 20:17:20 +02:00
Sergei Trifonov
426184ef73
Merge branch 'master' into async-loader-integration 2023-05-12 19:55:24 +02:00
serxa
a7f514e66e fix startup of database w/o tables 2023-05-11 09:09:32 +00:00
serxa
6bbc061ba0 start loader, fix helpers, fix deadlock 2023-05-10 19:56:48 +00:00
Sergei Trifonov
0b2860d822
Merge branch 'master' into async-loader 2023-05-10 14:54:00 +02:00
Azat Khuzhin
051d0e57ff Suppress bugprone-standalone-empty in DatabaseMySQL 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 20:47:52 +02:00
Sergei Trifonov
1198f1fc4c
Merge branch 'master' into async-loader-integration 2023-05-09 14:36:03 +02:00
alesapin
277b42451e
Merge branch 'master' into fix_database_replica_recovery 2023-05-09 13:06:45 +02:00
Sergei Trifonov
b479637e2c
Merge branch 'master' into async-loader-integration 2023-05-09 13:04:36 +02:00
Sergei Trifonov
4f9d443226
Merge branch 'master' into async-loader 2023-05-09 12:04:23 +02:00
Alexander Tokmakov
7250e7fec8 correctly update log pointer during replica recovery 2023-05-08 19:58:43 +02:00
Alexander Tokmakov
14dc7d988f fix 2023-05-08 14:49:45 +02:00
Alexander Tokmakov
6f8455efcb
Update removeWhereConditionPlaceholder.h 2023-05-08 02:26:29 +03:00
Alexander Tokmakov
1224ac9eda fix build 2023-05-08 00:57:13 +02:00
Alexander Tokmakov
abf6c60ad2 Merge branch 'master' into fix_dictionaries_loading_order 2023-05-08 00:31:03 +02:00
Alexander Tokmakov
846abe95e9 fix 2023-05-05 20:50:13 +02:00
alekseygolub
e1151f150f Fix clang build errors 2023-05-05 18:37:25 +00:00
Alexander Tokmakov
dd1bbf7c78 fix another issue with dependencies 2023-05-05 16:27:12 +02:00
alekseygolub
814a3f04cd fix style 2023-05-04 17:12:35 +00:00
alekseygolub
f083372c0c remove extra include 2023-05-04 16:46:51 +00:00
alekseygolub
963d6be120 Added configurations for DatabaseS3 2023-05-04 16:44:08 +00:00
Mal Curtis
5727585d95 s/use_tables_cache/use_table_cache 2023-05-03 23:40:12 +00:00
serxa
3327c0fa1e fix style, get rid of DatabaseOrdinaryThreads* metrics 2023-05-03 19:00:58 +00:00
Alexander Tokmakov
e399903030
Merge pull request #48548 from ClickHouse/clusters_is_active_column 
Add some columns to system.clusters
 2023-05-03 17:42:40 +03:00
alekseygolub
82bb1e8bf2 Fix build and try fix tests 2023-05-02 18:51:35 +00:00
Alexander Tokmakov
495325f664 fix some bad error messages, suppress others 2023-05-02 17:55:42 +02:00
serxa
6eda6e8585 implement async loading interface in database engines 2023-05-02 09:18:50 +00:00
alekseygolub
95522ad7a6 Added DatabaseHDFS 2023-05-01 21:46:17 +00:00
serxa
0f6ff8d381 fix link error 2023-05-01 12:39:17 +00:00
Sergei Trifonov
8dab49f328
Merge branch 'master' into async-loader-integration 2023-05-01 14:04:39 +02:00
serxa
abaf73268b integrate with Server and TablesLoader 2023-05-01 11:56:00 +00:00
Aleksei Golub
1846b76982 Added DatabaseS3 with test 2023-05-01 11:46:55 +03:00
Aleksei Golub
6831eb2001 fix style 2023-04-30 14:51:04 +03:00
Aleksei Golub
57d852a60e Fixed table existence checking 2023-04-30 14:46:11 +03:00
Alexey Milovidov
6dcfdbfaf2
Merge branch 'master' into clusters_is_active_column 2023-04-28 16:16:25 +03:00
Aleksei Golub
e20f92ce0f Fixed exceptions handling; Fixed style; 2023-04-27 21:26:36 +03:00
serxa
37ee4db421 working on async loading in TablesLoader 2023-04-27 14:51:24 +00:00
serxa
879a6e0405 fix build 2023-04-26 18:34:01 +00:00
serxa
3546699c95 wip async tables loading 2023-04-26 18:25:39 +00:00
Aleksei Golub
4606e66068 Fix style 2023-04-26 11:11:41 +03:00
Aleksei Golub
26812f36fb Added read-only database setting; Fixed error messages for filesystem database; added tests 2023-04-26 10:42:20 +03:00
Aleksei Golub
c9f8dd8bfd Replaced Poco::File with std::filesystem 2023-04-26 10:42:20 +03:00
Aleksei Golub
79ca39d920 Fixed exception messages 2023-04-26 10:42:20 +03:00
Aleksei Golub
2d2483d695 Rename DatabaseFileSystem to DatabaseFilesystem 2023-04-26 10:42:19 +03:00
alekseygolub
dfddc25389
Merge branch 'ClickHouse:master' into localfilefunction 2023-04-26 10:41:50 +03:00
Kseniia Sumarokova
a0791e3187
Update DatabasePostgreSQL.cpp 2023-04-25 11:53:59 +02:00
Alexander Tokmakov
332a736a59 Merge branch 'master' into clusters_is_active_column 2023-04-24 15:54:03 +02:00
kssenii
277393fd8d Fix 2023-04-24 14:45:41 +02:00
Aleksei Golub
96553bc3d8 Fix style and tests 2023-04-18 18:05:55 +03:00
Aleksei Golub
267bbcab00 Added ability to implicitly use file table function in clickhouse-local 2023-04-16 15:48:55 +03:00
Kseniia Sumarokova
6a0d9a37ce
Merge branch 'master' into fix-mysql-named-collection 2023-04-14 12:03:51 +02:00
kssenii
ad48e1d010 Fox 2023-04-13 19:36:25 +02:00
Robert Schulze
7a21d5888c
Remove -Wshadow suppression which leaked into global namespace 2023-04-13 08:46:40 +00:00
MikhailBurdukov
2cd3512a5d
ClickHouse startup error when loading a distributed table that depends on a dictionary (#48419) 
* Test for start failure.

* Handling the dictionary as shard key.

* Added integration test and fixed style

* Revert extra test

* Fix style

* Fix style

* Refactoring

* Fix build

* style fix
 2023-04-12 15:06:02 +03:00
Alexander Tokmakov
dfff30e1a9 Merge branch 'master' into clusters_is_active_column 2023-04-11 16:32:15 +02:00
Alexander Tokmakov
9ec91acef3 add some columns to system.clusters 2023-04-07 18:26:23 +02:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
Antonio Andelic
7975df538e Don't replicate KeeperMap DELETE and TRUNCATE 2023-04-05 15:28:35 +00:00
Antonio Andelic
e7d00c8f34 Don't replicate mutations for KeeperMap tables 2023-04-03 14:22:08 +00:00
Alexander Tokmakov
f54fd15112 fix 2023-03-31 12:58:11 +02:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection 
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-29 10:46:59 +02:00
Sergei Solomatov
b143a1a3eb fix query 2023-03-26 11:20:53 +00:00
Nikolay Degterinsky
7ee842848a
Merge pull request #47052 from ClickHouse/evillique-patch-1 
Disable path check in SQLite storage for clickhouse-local
 2023-03-24 23:38:18 +01:00
Alexander Tokmakov
b70216f9f1 fix for Replicated database recovery 2023-03-22 18:12:42 +01:00
Nikolay Degterinsky
ccb0d7df00
Merge branch 'master' into evillique-patch-1 2023-03-21 22:13:12 +01:00
kssenii
bb0beb7449 Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-03-17 13:02:36 +01:00
Nikolay Degterinsky
469cd60e76 Merge remote-tracking branch 'upstream/master' into evillique-patch-1 2023-03-15 01:07:35 +00:00
Roman Vasin
f42f3ee655
Merge branch 'master' into ADQM-639 2023-03-09 09:48:33 +03:00
Alexander Tokmakov
c909d1e3f4
Merge pull request #47314 from ClickHouse/fix_upgrade_check_file_exists 
Fix `File exists` error in Upgrade check
 2023-03-08 17:29:34 +03:00
Alexander Tokmakov
f5cf039190 check if a symlink exists more carefully 2023-03-08 00:04:55 +01:00
Nikolay Degterinsky
990af0fb79
Fix typo 2023-03-06 23:44:41 +01:00
Roman Vasin
95a7e11a47 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-639 2023-03-06 09:11:20 +00:00
kssenii
c06af1f1e7 Fix clang-tidy 2023-03-05 22:12:51 +01:00
kssenii
8f2d75cef8 Fix tests 2023-03-05 12:56:00 +01:00
Kseniia Sumarokova
386663953c
Merge branch 'master' into named-collections-finish 2023-03-03 12:23:38 +01:00
kssenii
cd7cd0526b Fix tests 2023-03-02 21:21:04 +01:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
kssenii
80d017629a Fix test 2023-03-01 17:01:34 +01:00
Nikolay Degterinsky
86744585fd
Disable path check in SQLite storage for clickhouse-local 2023-03-01 06:30:07 +01:00
kssenii
b19264cf9f Remove redundant 2023-02-27 12:32:13 +01:00
Alexey Milovidov
c58468ce8d
Merge branch 'master' into fix-attach-table-function-name-normalizer 2023-02-26 01:30:34 +03:00
Roman Vasin
5c1c5199b9
Merge branch 'master' into ADQM-639 2023-02-22 09:25:17 +03:00
kssenii
a54b011670 Finish for mysql 2023-02-20 21:37:38 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Roman Vasin
3e2b56fabc
Merge branch 'master' into ADQM-639 2023-02-17 21:14:26 +03:00
Roman Vasin
b0ced21c30 Move temp database clean up into new function removeDataPath() 2023-02-14 13:22:15 +00:00
Antonio Andelic
8d16fe5793
Merge branch 'master' into add-support-for-settings-alias 2023-02-13 08:46:00 +01:00
Roman Vasin
bb465ce7dd
Merge branch 'master' into ADQM-639 2023-02-10 17:04:14 +03:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias 2023-02-09 16:07:45 +01:00
Vitaly Baranov
4869d3806c Add setting check_referential_table_dependencies to check referential dependencies on DROP TABLE. 2023-02-08 23:56:59 +01:00
Vitaly Baranov
6555b48132 Disable evaluating the right part of IN operator while analyzing dependencies. 2023-02-07 13:21:30 +01:00
Vitaly Baranov
324b0b6896 Fix DDLDependencyVisitor for an empty table name. 2023-02-07 13:21:18 +01:00
Roman Vasin
9488bd834f
Merge branch 'master' into ADQM-639 2023-02-07 09:45:10 +03:00
Antonio Andelic
12569da984 Merge branch 'master' into add-support-for-settings-alias 2023-02-05 16:08:57 +00:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Antonio Andelic
ec04cf6876 Merge branch 'master' into add-support-for-settings-alias 2023-02-01 09:01:01 +00:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Roman Vasin
16428b91ec Fix removing data on disk for table engines like Log for DatabaseMemory 2023-01-26 09:47:36 +00:00
Alexander Tokmakov
9b670946db Merge branch 'master' into exception_message_patterns5 2023-01-26 00:41:32 +01:00
Alexander Tokmakov
b9e5c586e6
Merge pull request #45566 from ClickHouse/resubmit_45493 
Resubmit "Fix possible in-use table after DETACH"
 2023-01-25 16:39:55 +03:00
Alexander Tokmakov
6eb557b2ba Merge branch 'master' into exception_message_patterns4 2023-01-25 13:49:17 +01:00
Kseniia Sumarokova
5da5327729
Merge pull request #45062 from kssenii/fix-sqlite-data-race-in-init-db 
Fix data race in openSQLiteDB
 2023-01-25 12:32:34 +01:00
Alexander Tokmakov
6ecae8388e Merge branch 'master' into exception_message_patterns4 2023-01-24 14:42:36 +01:00
Alexander Tokmakov
cd8c3bbcc5 fix 2023-01-24 14:36:30 +01:00
Alexander Tokmakov
42a976fe3d Revert "Revert "Merge pull request #45493 from azat/fix-detach"" 
This reverts commit 9dc4f02d15.
 2023-01-24 14:15:51 +01:00
Alexander Tokmakov
9dc4f02d15 Revert "Merge pull request #45493 from azat/fix-detach" 
This reverts commit a182a6b851, reversing
changes made to c47a29a089.
 2023-01-24 12:20:37 +01:00
Alexander Tokmakov
bb4c8e169f check number of parameters in format string 2023-01-23 23:16:16 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Azat Khuzhin
ca8af3d18c Fix possible in-use table after DETACH 
Right now in case of DETACH/ATTACH there can be a window when after the
table had been DETACH'ed someone will still use it, the common example
here is MVs handling.

It happens because TableExclusiveLockHolder does not guards the
shard_ptr of the IStorage, and so if someone holds it, then it can use
it. So if ATTACH will be done for this table then, you can have multiple
instances of it.

It is not possible for DROP, because before using a table, you should
lock it and after table had been DROP'ed you cannot lock it anymore.

So let's do the same for DETACH.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-22 13:18:56 +01:00
Roman Vasin
fa64203225 Add correct data path for temporary tables; Clean temp DB dir on server startup 2023-01-20 16:11:37 +00:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Alexander Tokmakov
522686f78b less empty patterns 2023-01-17 01:19:44 +01:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Alexander Tokmakov
6de4837580 fix 2023-01-13 16:07:20 +01:00
Alexander Tokmakov
b88aae9d5c Merge branch 'master' into fix_44496 2023-01-13 14:05:57 +01:00
kssenii
f412c76eba Fix data race in init sqlite db 2023-01-09 14:15:48 +01:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
Vitaly Baranov
8a5a2f5b3d Up the log level of tables dependencies graphs. 2023-01-06 00:46:49 +01:00
Nikita Mikhaylov
e9c7555365
Use TablesDependencyGraph in DatabaseReplicated recovery process (#44697) 2023-01-04 14:49:39 +01:00
Raúl Marín
0785bf6b29 Do not throw DB::Exception when folders do not exist 2023-01-03 17:05:54 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
Alexander Tokmakov
4f73046520
Implement SYSTEM DROP DATABASE REPLICA (#42807) 2022-12-29 15:34:11 +01:00
alesapin
600bedbff4 Add setting to disallow arguments in ReplicatedMergeTree constructor for DatabaseReplicated 2022-12-24 18:38:14 +01:00
Alexander Tokmakov
9619f4a5f5 fix too aggressive evaluation of args in default column expr 2022-12-23 20:57:19 +01:00
kssenii
853f2ea123 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 11:49:02 +01:00
kssenii
a58b8b8e6c Fix tests 2022-12-22 12:27:42 +01:00
kssenii
d3db1dd6a7 Fix tests 2022-12-22 00:27:22 +01:00
kssenii
1d75f740d7 Fix tests 2022-12-20 22:33:54 +01:00
Alexander Tokmakov
a0d695e618
Update src/Databases/DatabaseOrdinary.cpp 2022-12-19 21:28:39 +03:00
alesapin
c8d9ccf3cd Lock table for share during startup for database ordinary 2022-12-19 18:51:46 +01:00
kssenii
6bb3d06c88 Replace old named collections code for postgresql 2022-12-17 01:30:55 +01:00
Vitaly Baranov
5c1f490b3a Implement referential dependencies for table engine "Distributed" 
and for functions cluster() and clusterAllReplicas().
 2022-12-13 11:09:42 +01:00
Vitaly Baranov
fc0f29821b Use constant evaluation when calculating referential dependencies. 2022-12-12 19:19:18 +01:00
Vitaly Baranov
efbf0f7398 Move DDLDependencyVisitor from header to cpp file. 2022-12-12 19:19:11 +01:00
Vitaly Baranov
0207637f6b Use query context instead of the global context in DDLDependencyVisitor. 2022-12-12 18:22:14 +01:00
Vitaly Baranov
78c433b79d Improve TablesDependencyGraph. 2022-12-12 18:22:04 +01:00
Nikita Taranov
408cea80eb impl 2022-12-03 22:58:46 +01:00
Vitaly Baranov
e1f7f04752
Referential dependencies for RESTORE (#43834) 
* Rename DDLDependencyVisitor -> DDLLoadingDependencyVisitor.

* Move building a loading graph to TablesLoader.

* Implement referential dependencies for tables and use them
when restoring tables from a backup.

* Remove StorageID::operator < (because of its inconsistency with ==).

* Add new tests.

* Fix test.

* Fix memory leak.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2022-12-02 15:05:46 +01:00
Nikita Mikhaylov
9ef54c6ecc
Try to use exchanges during database replicated replica recovery process (#43628) 2022-12-01 13:50:56 +01:00
Alexey Milovidov
56f8c0b84b Fix typo 2022-11-21 15:10:15 +01:00
Vitaly Baranov
ae19af0015 Fix backup of Lazy databases. 2022-11-10 00:27:00 +01:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
Kseniia Sumarokova
cd5c6acbd2
Merge pull request #42960 from kssenii/fix-pg-db-datetime-table-attach 
Fix postgres db engine attaching table with datetime
 2022-11-06 22:39:15 +01:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast() 
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
 2022-11-04 15:52:48 +00:00
Kseniia Sumarokova
6ada8e9e39
Update src/Databases/PostgreSQL/DatabasePostgreSQL.cpp 
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
 2022-11-04 13:37:10 +01:00
kssenii
c9b4bc66b5 Fiux 2022-11-04 12:19:30 +01:00
Vitaly Baranov
c2cc2ccc99 Make as_table_function a child of ASTCreateQuery (to help writing visitors). 2022-10-31 10:50:33 +01:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Alexander Tokmakov
4b371bd20c fix 2022-10-10 16:38:35 +02:00
Alexander Tokmakov
fa1134f299 Merge branch 'master' into fix_loading_dependencies 2022-10-10 16:30:52 +02:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Alexander Tokmakov
014784a9ca Merge branch 'master' into fix_loading_dependencies 2022-10-07 18:58:11 +02:00
Alexander Tokmakov
690ec74bf2 better handling for expressions in dictGet 2022-10-05 20:58:27 +02:00
Robert Schulze
cc92a2d174
Merge branch 'master' into generated-file-cleanup 2022-09-30 09:56:31 +02:00
Nikita Mikhaylov
afe6c99e7d
Update DatabaseReplicated.cpp 2022-09-29 13:23:14 +02:00
Nikita Mikhaylov
ab97c60118 Done 2022-09-29 11:00:23 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Nikita Mikhaylov
bf4d675830
Fix backward incompatibility in Replicated database creation (#41875) 2022-09-28 13:22:19 +02:00
Alexander Tokmakov
922834ccde minor fixes 2022-09-26 19:31:08 +02:00
Frank Chen
a999212082 Update test cases to support both Replicated and non-Replicated database engine 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-24 11:34:42 +08:00
Alexander Tokmakov
65474830ac make tryEnqueueReplicatedDDL virtual 2022-09-16 16:25:32 +02:00
Alexander Tokmakov
03c193ccca fix ON CLUSTER with Replicated database cluster 2022-09-15 21:15:57 +02:00
Alexander Tokmakov
fc73134145 fix race between DROP and short form of ATTACH 2022-09-14 18:03:36 +02:00
Alexey Milovidov
05d0f48d17
Merge pull request #41173 from ClickHouse/remove-some-methods 
Remove some methods
 2022-09-11 07:19:52 +03:00
Alexey Milovidov
fd235919aa Remove some methods 2022-09-10 05:04:40 +02:00
Alexey Milovidov
4301b362ab Fix strange code in DatabaseReplicated 2022-09-10 02:34:37 +02:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
alesapin
4f9faab924 Fix benign race in database replicated worker 2022-08-28 15:07:29 +02:00
Barum Rho
a61827fb78 Convert SQLite int8 column to ClickHouse int64 
SQLite int8 column uses 64 bit integers.
 2022-08-25 16:15:50 -04:00
Alexander Tokmakov
629690c32b fix "incorrect replica identifier" 2022-08-18 16:13:27 +02:00
Alexander Tokmakov
caa270b72a
Merge pull request #39933 from ClickHouse/auto_convert_ordinary_to_atomic 
Add flag that enables automatic conversion from Ordinary to Atomic
 2022-08-17 12:40:49 +03:00
Nikita Mikhaylov
a7c3f23dfb
Merge branch 'master' into auto_convert_ordinary_to_atomic 2022-08-17 00:38:25 +02:00
Alexander Tokmakov
832d7c6e7c Merge branch 'master' into replicated_database_improvements 2022-08-16 15:49:13 +02:00
Alexander Tokmakov
6f5a7c3bf7 fix a bug with symlinks detection 2022-08-15 12:30:47 +02:00
Alexander Tokmakov
848511affb take permanently detached tables into account 2022-08-08 16:06:08 +02:00