thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 01:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
77,787 Commits 9,322 Branches 1,783 Tags 2.1 GiB
d7939a3b61
Commit Graph

35 Commits

Author SHA1 Message Date
Vitaly Baranov
a140f4127e Fix ExternalLoader without additional mutex. 2021-10-13 10:02:42 +03:00
Azat Khuzhin
2bab572caf Fix lock-order-inversion between periodic dictionary reload and config reload 
Integration tests found [1], TSan report:

    WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=11)
      Cycle in lock order graph: M3152 (0x7b9000000058) => M3153 (0x7b9000000438) => M3152

      Mutex M3153 acquired here while holding mutex M3152 in main thread:
        3 std::__1::lock_guard<std::__1::recursive_mutex>::lock_guard(std::__1::recursive_mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x15716b81)
        4 DB::ExternalLoader::addConfigRepository() const obj-x86_64-linux-gnu/../src/Interpreters/ExternalLoader.cpp:1283:21 (clickhouse+0x15716b81)
        5 DB::Context::loadOrReloadDictionaries() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1453:62 (clickhouse+0x155d7bad)
        6 DB::Server::main()::$_1::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:852:29 (clickhouse+0x9b1944c)
        13 DB::ConfigReloader::reloadIfNewer() obj-x86_64-linux-gnu/../src/Common/Config/ConfigReloader.cpp:137:13 (clickhouse+0x17045e2e)
        14 DB::ConfigReloader::ConfigReloader() obj-x86_64-linux-gnu/../src/Common/Config/ConfigReloader.cpp:33:9 (clickhouse+0x17044e51)
        16 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:803:33 (clickhouse+0x9b0c41d)
        17 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse+0x19ffc08b)
        18 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:405:25 (clickhouse+0x9b03ebe)
        19 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse+0x1a01c246)
        20 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:183:20 (clickhouse+0x9b02576)
        21 main obj-x86_64-linux-gnu/../programs/main.cpp:372:12 (clickhouse+0x9b00a3a)

      Mutex M3152 acquired here while holding mutex M3153 in thread T2:
        3 std::__1::lock_guard<std::__1::mutex>::lock_guard() obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x155d63b8)
        4 DB::Context::getExternalDictionariesLoader() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1337:21 (clickhouse+0x155d63b8)
        5 DB::Context::getExternalDictionariesLoader() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1332:41 (clickhouse+0x155d6359)
        6 DB::DatabaseDictionary::tryGetTable() const obj-x86_64-inux-gnu/../src/Databases/DatabaseDictionary.cpp:76:38 (clickhouse+0x157819ad)
        7 DB::DatabaseCatalog::getTableImpl() const obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.cpp:285:28 (clickhouse+0x1564a1fa)
        8 DB::DatabaseCatalog::getTable() const obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.cpp:656:16 (clickhouse+0x1564fa2a)
        9 DB::JoinedTables::getLeftTableStorage() obj-x86_64-linux-gnu/../src/Interpreters/JoinedTables.cpp:219:40 (clickhouse+0x15eeef45)
        10 DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:321:33 (clickhouse+0x15b792be)
        19 DB::ClickHouseDictionarySource::doInvalidateQuery() const obj-x86_64-linux-gnu/../src/Dictionaries/ClickHouseDictionarySource.cpp:207:36 (clickhouse+0x12872d2d)
        20 DB::ClickHouseDictionarySource::isModified() const obj-x86_64-linux-gnu/../src/Dictionaries/ClickHouseDictionarySource.cpp:144:25 (clickhouse+0x12872534)
        21 DB::IDictionary::isModified() const <null> (clickhouse+0x128ce39b)
        22 DB::ExternalLoader::LoadingDispatcher::reloadOutdated() obj-x86_64-linux-gnu/../src/Interpreters/ExternalLoader.cpp:660:50 (clickhouse+0x157305f7)
        23 DB::ExternalLoader::PeriodicUpdater::doPeriodicUpdates() obj-x86_64-linux-gnu/../src/Interpreters/ExternalLoader.cpp:1248:36 (clickhouse+0x1572fff7)

  [1]: https://clickhouse-test-reports.s3.yandex.net/29856/42ca2b4bb241827edf69bbd6938d6b19c31935f1/integration_tests_(thread).html#fail1
 2021-10-10 04:04:05 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Alexander Tokmakov
d82a794e8c fix logical race on dictionaries loading 2021-09-10 12:54:22 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Azat Khuzhin
4c47d91c94 Attach background thread for dictionary reload to the reload query 2021-05-09 14:30:31 +03:00
Vitaly Baranov
46f4c60839 Small simplification in ExternalLoader. 2021-03-23 15:15:44 +03:00
Vitaly Baranov
090e558da4
Merge pull request #21844 from Avogar/catboost-deadlock 
Fix deadlock in first catboost model execution
 2021-03-23 15:01:04 +03:00
Maksim Kita
a65bfaeab3 Fixed tests 2021-03-18 17:03:22 +03:00
Pavel Kruglov
bf888b7e5f Fix deadlock in first catboost model execution 2021-03-17 21:57:40 +03:00
Nikolai Kochetov
c7236a9359 Try fix tests. 2020-10-27 19:12:53 +03:00
Nikolai Kochetov
71298ba496 Move fix to ExternalLoader::checkLoaded. 2020-10-27 17:21:51 +03:00
Vitaly Baranov
cce970e40c Use join() instead of detach() for loading threads in ExternalLoader. 2020-09-09 03:19:13 +03:00
Alexander Tokmakov
f6949b2f47 use dictionary ids 2020-07-15 22:25:31 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
af2a59e49b Progress on task 2020-05-24 00:50:34 +03:00
Alexey Milovidov
d853bc39c5 Apply all transformations again 2020-05-24 00:41:48 +03:00
Alexey Milovidov
c87fb91532 Progress on task 2020-05-24 00:41:35 +03:00
Alexey Milovidov
93626701b8 Apply all transformations again 2020-05-24 00:27:27 +03:00
Alexey Milovidov
ce0619dabf Progress on task 2020-05-24 00:26:45 +03:00
Alexey Milovidov
cd6d3c7509 Progress on task 2020-05-23 22:55:06 +03:00
Alexey Milovidov
d5b7883451 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/' 2020-05-23 22:41:08 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
7fed65cbe2 Remove duplicate whitespaces (preparation) 2020-05-23 22:14:58 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
241f8c5431 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+ \+ "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+) \+ "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 20:10:21 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Vitaly Baranov
dd34cd73e2 Move initialization of DatabaseWithDictionaries to constructor. 2020-04-30 20:22:20 +03:00
Alexey Milovidov
0a7edce036 Checkpoint 2020-04-22 09:01:33 +03:00
Vitaly Baranov
d62345bbaa Show information about not loaded dictionaries in system.columns, system.tables 
and in "SHOW TABLES" executed in any database with the "Dictionary" engine.
 2020-04-18 12:50:54 +03:00
Vitaly Baranov
d8b76232fe Remove useless logging from ExternalLoader. 2020-04-07 21:39:34 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00