Commit Graph

1213 Commits

Author SHA1 Message Date
Alexander Tokmakov
141a3df727 fix 2023-08-03 12:35:14 +02:00
Alexander Tokmakov
c4fb0a0cd5 fix two minor issue 2023-08-02 23:36:54 +02:00
Mike Kot
1335d06b8f
Do not replicate ALTER TABLE t FREEZE on Replicated engine (#52064)
* fix

* do not replicate alter freeze

* fix

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-08-02 14:39:54 +03:00
alesapin
efba3a2113 Add more logging and touch test for materialize mysql 2023-07-31 14:32:09 +02:00
Alexander Tokmakov
6bbed6262e
Merge branch 'master' into add_delay_for_replicated 2023-07-26 12:48:48 +03:00
Azat Khuzhin
85082ad8f8
Fix data-race DatabaseReplicated::startupTables()/canExecuteReplicatedMetadataAlter() (#52490)
CI founds [1]:

    Exception: Sanitizer assert found for instance ==================
    WARNING: ThreadSanitizer: data race (pid=348)
      Write of size 8 at 0x7b58000044a0 by main thread:
        2 DB::DatabaseReplicated::startupTables(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, DB::LoadingStrictnessLevel) build_docker/./src/Databases/DatabaseReplicated.cpp:526:16 (clickhouse+0x1ec45092)
        3 DB::TablesLoader::startupTables() build_docker/./src/Databases/TablesLoader.cpp:87:26 (clickhouse+0x1f9258ab) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        4 DB::loadMetadata(std::__1::shared_ptr<DB::Context>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./src/Interpreters/loadMetadata.cpp:234:12 (clickhouse+0x1fff3834) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        5 DB::Server::main() build_docker/./programs/server/Server.cpp:1615:9 (clickhouse+0x163e7f78) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        6 Poco::Util::Application::run() build_docker/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x257608fe) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        7 DB::Server::run() build_docker/./programs/server/Server.cpp:391:25 (clickhouse+0x163d7d7c) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        8 Poco::Util::ServerApplication::run(int, char**) build_docker/./base/poco/Util/src/ServerApplication.cpp:131:9 (clickhouse+0x25780114) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        9 mainEntryClickHouseServer(int, char**) build_docker/./programs/server/Server.cpp:196:20 (clickhouse+0x163d4c23) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        10 main build_docker/./programs/main.cpp:487:12 (clickhouse+0xdf8c877) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)

      Previous read of size 8 at 0x7b58000044a0 by thread T27 (mutexes: write M0, write M1):
        1 DB::DatabaseReplicated::canExecuteReplicatedMetadataAlter() const build_docker/./src/Databases/DatabaseReplicated.cpp:1303:12 (clickhouse+0x1ec5c5bd)
        2 DB::ReplicatedMergeTreeQueue::shouldExecuteLogEntry() const build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1471:24 (clickhouse+0x2115fb56) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        3 DB::ReplicatedMergeTreeQueue::selectEntryToProcess(DB::MergeTreeDataMergerMutator&, DB::MergeTreeData&) build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1676:13 (clickhouse+0x21163c58) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        4 DB::StorageReplicatedMergeTree::selectQueueEntry() build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3240:26 (clickhouse+0x20823db2) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        5 DB::StorageReplicatedMergeTree::scheduleDataProcessingJob(DB::BackgroundJobsAssignee&) build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3304:65 (clickhouse+0x208240fc) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/52395/0b258dda4ee618a4d002e2b5246d68bbd2c77c7e/integration_tests__tsan__[5_6].html

Add ddl_worker_initialized flag to avoid this race.

Note, that it should be enough to check this flag only in
canExecuteReplicatedMetadataAlter() since only it can be run in parallel
with ctor before it had been finished.

v0: initialize ddl before startupTables()
v2: ddl_worker_initialized

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-07-25 17:35:01 +03:00
Alexander Tokmakov
faca49a905 Merge branch 'master' into add_delay_for_replicated 2023-07-24 16:07:38 +02:00
Val Doroshchuk
efa638ef3c MaterializedMySQL: Support unquoted utf-8 strings in DDL
Since ClickHouse does not support unquoted utf-8 strings but MySQL does.

Instead of fixing Lexer to recognize utf-8 chars as TokenType::BareWord,
suggesting to quote all unrecognized tokens before applying any DDL.

Actual parsing and validating the syntax will be done by particular Parser.

If there is any TokenType::Error, the query is unable to be parsed anyway.
Quoting such tokens can provide the support of utf-8 names.

See `tryQuoteUnrecognizedTokens` and `QuoteUnrecognizedTokensTest`.

mysql> CREATE TABLE 道.渠(...

is converted to

CREATE TABLE `道`.`渠`(...

Also fixed the bug with missing * while doing SELECT in full sync because db or table name are back quoted when not needed.
2023-07-24 11:12:10 +02:00
Azat Khuzhin
8013cb1f78 Remove skip_startup_tables from IDatabase::loadStoredObjects()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-23 08:47:50 +02:00
alesapin
6416fb6eed
Merge branch 'master' into add_delay_for_replicated 2023-07-22 12:11:39 +02:00
Nikita Mikhaylov
668062dc29
Merge branch 'master' into no-keep-context-lock-while-calculating-access 2023-07-20 14:42:13 +02:00
Nikita Mikhaylov
11cc8b4adf
Merge branch 'master' into add_delay_for_replicated 2023-07-17 19:04:29 +02:00
Vitaly Baranov
815a3857de Remove non-const function Context::getClientInfo(). 2023-07-17 15:02:07 +02:00
Val Doroshchuk
4da0782e55 MaterializedMySQL: Add tests to parse db and table names from DDL 2023-07-17 15:01:06 +02:00
Alexander Sapin
30bcc73c63 Merge branch 'master' into add_delay_for_replicated 2023-07-06 14:29:15 +02:00
Alexey Milovidov
5452f2106f
Merge branch 'master' into sunny19930321-fix-attach-table-function-name-normalizer 2023-07-06 03:50:18 +03:00
alesapin
baee73fd96 Make shutdown of replicated tables softer 2023-07-05 18:11:25 +02:00
Kseniia Sumarokova
19508a01a5
Merge branch 'master' into resubmit-51149 2023-07-03 13:12:53 +02:00
Alexey Milovidov
2a7ae8f622
Merge branch 'master' into sunny19930321-fix-attach-table-function-name-normalizer 2023-06-30 12:16:49 +03:00
Alexander Tokmakov
a28728b7e5 Update DatabaseReplicatedWorker.cpp
(cherry picked from commit 43fc1af1b3)
2023-06-29 19:04:49 +02:00
kssenii
44246363e2 Fix test 2023-06-29 15:37:36 +02:00
Alexey Milovidov
a4d7e48bc9
Merge branch 'master' into resubmit-51149 2023-06-29 11:07:01 +03:00
Alexander Tokmakov
1ed104417a fix race between executeMetadataAlter and initializeReplication
(cherry picked from commit 1d47783e85)
2023-06-27 23:24:02 +02:00
Alexey Milovidov
8610ff828e Merge branch 'fix-attach-table-function-name-normalizer' of github.com:sunny19930321/ClickHouse into sunny19930321-fix-attach-table-function-name-normalizer 2023-06-27 07:58:42 +02:00
Alexander Tokmakov
8afb8bf13a disable table structure check for secondary queries from Replicated db 2023-06-22 19:06:28 +02:00
Alexander Tokmakov
50ee424148 do not update digest during recovery 2023-06-21 21:07:59 +02:00
Alexander Tokmakov
9240a82c64
Update DatabaseReplicated.cpp 2023-06-21 21:49:00 +03:00
kssenii
0efaecab51 Revert "Merge pull request #51149 from ClickHouse/revert-48821-localfilefunction"
This reverts commit a09e6bbb8e, reversing
changes made to ce38d64c5a.
2023-06-21 00:29:46 +02:00
Sergei Trifonov
5867f39fc7
Merge pull request #50675 from ClickHouse/unify-priorities-pools
Unify priorities for connection pools
2023-06-19 17:16:15 +02:00
Alexander Tokmakov
caceb7c862
Revert "Added ability to implicitly use file/hdfs/s3 table functions in clickhouse-local" 2023-06-19 13:21:09 +03:00
Alexey Milovidov
e913a58284
Merge pull request #51049 from ClickHouse/ddl_replication_improvements
An optimiation for ALTERs and Replicated db with one shard
2023-06-17 16:57:59 +03:00
kssenii
74f2dea57b Fix typo 2023-06-17 02:11:48 +02:00
kssenii
b902f5d045 Small style corrections 2023-06-17 01:54:38 +02:00
Alexey Milovidov
df9abccfaa
Merge branch 'master' into localfilefunction 2023-06-17 00:45:54 +03:00
Alexander Tokmakov
353b57f13f an optimiation for alters and replicated db 2023-06-15 20:20:11 +02:00
alekseygolub
d72751be27 Added cache invalidation; Fix issues 2023-06-11 15:01:45 +00:00
Alexey Milovidov
2a126aaa76
Merge branch 'master' into fix-attach-table-function-name-normalizer 2023-06-10 13:14:17 +03:00
serxa
2df0e0c669 Unify priorities for connections 2023-06-07 16:25:52 +00:00
Alexander Tokmakov
e1d044dfd7
Merge pull request #50392 from ClickHouse/fix_materialized_mysql_poll_timeout
Fix poll timeout in MaterializedMySQL
2023-06-01 15:16:59 +03:00
Alexey Milovidov
956c399b2a Remove useless code 2023-06-01 03:04:29 +02:00
Alexander Tokmakov
f74d7474d2 fix poll timeout in MaterializedMySQL 2023-05-31 15:04:41 +02:00
Alexander Gololobov
8996fcb090
Merge pull request #50193 from ClickHouse/fix_for_replicate_delete
Don't replicate delete through DDL worker if there is just 1 shard
2023-05-24 22:45:00 +02:00
Alexander Gololobov
de0a074545 Don't replicate delete through DDL worker if there is just 1 shard 2023-05-24 16:10:31 +02:00
alekseygolub
34fe8ba5cd
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-24 09:49:03 +03:00
Alexander Tokmakov
3ac7bc90ef
Merge pull request #50108 from ClickHouse/update_replicated_database_settings
Update default settings for Replicated database
2023-05-23 12:56:12 +03:00
Alexander Tokmakov
5b768ebd97 update default settings for Replicated database 2023-05-22 19:32:32 +02:00
Nikolay Degterinsky
7bed59e1d2
Merge pull request #50000 from evillique/add-schema-inference
Add schema inference to more table engines
2023-05-22 17:24:30 +02:00
alekseygolub
70b9077bb2
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-21 17:49:29 +03:00
Nikolay Degterinsky
b8be714830 Add schema inference to more table engines 2023-05-19 00:44:27 +00:00
Sergei Trifonov
a30a0d80f2
Merge branch 'master' into async-loader 2023-05-12 20:17:20 +02:00