Commit Graph

70433 Commits

Author SHA1 Message Date
Azat Khuzhin
ba97ffb909 Add system.mutations for database filter check 2021-08-09 09:50:50 +03:00
Azat Khuzhin
bf344ecf20 Mark 01710_projection_fetch as long 2021-08-09 09:47:50 +03:00
lehasm
7a45d32da5
Update docs/ru/sql-reference/statements/rename.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-09 08:49:39 +03:00
lehasm
e2ba3f7fd4
Update docs/ru/sql-reference/statements/exchange.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-09 08:49:29 +03:00
lehasm
9893e0ce59
Update docs/ru/sql-reference/statements/attach.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-09 08:49:19 +03:00
lehasm
6f149630c3
Update docs/en/engines/database-engines/atomic.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-09 08:49:09 +03:00
lehasm
dcb750bb35
Update docs/ru/engines/database-engines/atomic.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-09 08:48:45 +03:00
lehasm
7fa4c92408
Update docs/en/engines/database-engines/atomic.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-09 08:48:17 +03:00
Kseniia Sumarokova
a966a440b0
Merge pull request #26677 from sevirov/sevirov-DOCSUP-11550-edit_and_translate_to_russian_materializedpostgresql
DOCSUP-11550: Edit and translate to Russian (MaterializePostgreSQL)
2021-08-09 08:27:18 +03:00
kssenii
a5a60a0451 Fix tests 2021-08-09 05:20:41 +00:00
olgarev
9ce7669e1f Initial 2021-08-09 01:25:22 +00:00
alexey-milovidov
5f84564207
Merge pull request #27378 from amosbird/hyperscansanitizer
Add settings to check hyperscan regexp length.
2021-08-09 02:48:14 +03:00
alexey-milovidov
bd78051c2f
Merge pull request #27428 from azat/thread-pool-no-miss
Do not miss exceptions from the ThreadPool
2021-08-09 02:46:50 +03:00
alexey-milovidov
677fddaa37
Merge pull request #27104 from azat/enable-distributed_push_down_limit
Enable distributed_push_down_limit by default
2021-08-09 02:45:52 +03:00
alexey-milovidov
4fd752c74f
Merge pull request #27427 from azat/fix-dist-zero-nodes
Fix distributed queries with zero shards and aggregation
2021-08-09 02:45:02 +03:00
alexey-milovidov
7b11e11645
Merge pull request #27438 from syominsergey/patch-6
add info about clang version choice
2021-08-09 02:44:25 +03:00
olgarev
86f3905eb5 Upd 2021-08-08 23:23:48 +00:00
olgarev
c449fc95e5 Merge branch 'revolg-DOCSUP-12003-S3_and_HDFS_zero_copy_replication' of https://github.com/olgarev/ClickHouse into revolg-DOCSUP-12003-S3_and_HDFS_zero_copy_replication 2021-08-08 23:21:51 +00:00
olgarev
06e48f39b0 New page - External Disks for Storing Data 2021-08-08 23:19:00 +00:00
alexey-milovidov
67d2a35d4e
Merge pull request #27437 from ClickHouse/dataset-uk-property
Add OpenSky dataset
2021-08-09 00:55:04 +03:00
alexey-milovidov
a090bfc494
Merge pull request #27435 from syominsergey/patch-3
remove extra english text already translated to russian
2021-08-09 00:35:46 +03:00
alexey-milovidov
2d251abda3
Merge pull request #27434 from syominsergey/patch-2
fix description of changing url when cloning project
2021-08-09 00:35:13 +03:00
Sergei Semin
578e55184e
add info about clang version choice 2021-08-09 00:32:36 +03:00
kssenii
7991bb6e83 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal 2021-08-08 21:30:17 +00:00
alexey-milovidov
bf72ac0854
Merge pull request #25492 from morozovsk/patch-6
Documentation: manipulating projections
2021-08-09 00:29:15 +03:00
alexey-milovidov
4edc272f5a
Merge pull request #27436 from syominsergey/patch-4
translate some text from english to russian in developer-instruction.md
2021-08-09 00:24:33 +03:00
Alexey Milovidov
d159c83fb8 Add OpenSky dataset 2021-08-09 00:02:25 +03:00
Sergei Semin
8b0cfdda68
translate some text from english to russian in developer-instruction.md 2021-08-08 23:40:45 +03:00
alexey-milovidov
f18e1aa2de
Update coordinates.md 2021-08-08 23:38:51 +03:00
alexey-milovidov
63f624c0b9
Update coordinates.md 2021-08-08 23:38:06 +03:00
Sergei Semin
f55e113755
remove extra english text already translated to russian 2021-08-08 23:36:02 +03:00
alexey-milovidov
9661dd9232
Merge pull request #27432 from ClickHouse/dataset-uk-property
Add UK Property Price Paid data
2021-08-08 23:31:26 +03:00
Sergei Semin
8a320967e6
fix description of changing url when cloning project 2021-08-08 23:26:35 +03:00
alexey-milovidov
a5daf2d2c4
Merge pull request #27252 from ClickHouse/roaring-memory-tracker
Track memory consumed by Roaring Bitmaps.
2021-08-08 22:19:30 +03:00
Alexey Milovidov
844c0cf0ab Add UK Property Paid data 2021-08-08 22:16:53 +03:00
Alexey Milovidov
0f5d3963bd Add UK Property Paid data 2021-08-08 22:16:11 +03:00
Alexey Milovidov
b368c12400 Add UK Property Paid data 2021-08-08 22:15:18 +03:00
Alexey Milovidov
7631242a3b Add UK Property Paid data 2021-08-08 22:14:01 +03:00
Alexey Milovidov
e45a09e2f5 Fix wrong PR #27184 2021-08-08 21:09:39 +03:00
Kseniia Sumarokova
ba220c8e3a
Merge branch 'master' into subquery-operators 2021-08-08 20:18:41 +03:00
kssenii
0f67acf648 Add test 2021-08-08 20:16:22 +03:00
kssenii
5abe33e1a3 Better 2021-08-08 20:12:12 +03:00
mergify[bot]
bd313279e8
Merge branch 'master' into hyperscansanitizer 2021-08-08 17:03:05 +00:00
Azat Khuzhin
2bfdf0d5a1 check-style: check system.projection_parts for database filter 2021-08-08 19:31:57 +03:00
Azat Khuzhin
940a537c66 Fix 01710_projection_fetch flakiness 2021-08-08 19:31:57 +03:00
Azat Khuzhin
0677a6c6cb Enable distributed_push_down_limit by default 2021-08-08 19:30:50 +03:00
Azat Khuzhin
038241b6ed Add new index data skipping minmax index format for proper Nullable support
Note, that it cannot be done w/o new extension, since index does not
have any header.

v2: use IDisk interface for existence check
v3: remove extra file existence check
v4: fix MATERIALIZE INDEX
2021-08-08 19:30:12 +03:00
Azat Khuzhin
91d7f3daa7 Fix lock-order-inversion while notifying about finished mutations
TSAN reports [1]:

  WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=36)
    Cycle in lock order graph: M16388 (0x7b7400011d68) => M1030334152907497744 (0x000000000000) => M16388

    Mutex M1030334152907497744 acquired here while holding mutex M16388 in thread T4:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x15220cd9)
      4 DB::StorageMergeTree::getIncompleteMutationsStatus(long, std::__1::set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std:>
      5 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0::operator()() const obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:464:36 (clickhouse+0x1521e2b1)
      6 void std::__1::condition_variable::wait<DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>(std::__1::unique_lock<std::__1::mutex>&, DB::StorageMergeTree::waitForMutation(lon>
      7 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:469:29 (clickhouse+0x1521e2b1)
      8 DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:496:9 (clickhouse+0x15221738)
      9 DB::InterpreterAlterQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterAlterQuery.cpp:113:16 (clickhouse+0x141182f6)
      10 DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:560:32 (clickhouse+0x149152f6)
      11 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:909:30 (clickhous>
      12 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:313:24 (clickhouse+0x153270af)

      Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

    Mutex M16388 acquired here while holding mutex M1030334152907497744 in thread T59:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::unique_lock<std::__1::mutex>::unique_lock(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse+0x15226c31)
      4 DB::StorageMergeTree::selectPartsToMutate(std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::shared_ptr<DB::RWLockImpl::LockHolderImpl>&) obj-x86_64-linux->
      5 DB::StorageMergeTree::scheduleDataProcessingJob(DB::IBackgroundJobExecutor&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1060:28 (clickhouse+0x15228b10)
      6 DB::BackgroundJobsExecutor::scheduleJob() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:229:17 (clickhouse+0x14f441dc)
      7 DB::IBackgroundJobExecutor::backgroundTaskFunction() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:10 (clickhouse+0x14f438ed)

  [1]: https://clickhouse-test-reports.s3.yandex.net/27248/4f3b80ff33c846465983aa2bc9ae9490e1118b15/fuzzer_tsan/report.htmlfail1
2021-08-08 19:28:29 +03:00
Azat Khuzhin
7964355ecf Fix mutation stuck on invalid partitions in non-replicated MergeTree
v2: Do not try to process empty mutations
    Found with flaky check [1].

      [1]: https://clickhouse-test-reports.s3.yandex.net/27248/66e8c0833392c20ba8dba3780f2b0d5c18f8194e/functional_stateless_tests_flaky_check_(address).html#fail1
2021-08-08 19:28:15 +03:00
Azat Khuzhin
5139067631 Guard BackgroundJobsExecutor from thread termination in case of uncaught exception 2021-08-08 19:27:58 +03:00