Commit Graph

69806 Commits

Author SHA1 Message Date
olgarev
c449fc95e5 Merge branch 'revolg-DOCSUP-12003-S3_and_HDFS_zero_copy_replication' of https://github.com/olgarev/ClickHouse into revolg-DOCSUP-12003-S3_and_HDFS_zero_copy_replication 2021-08-08 23:21:51 +00:00
olgarev
06e48f39b0 New page - External Disks for Storing Data 2021-08-08 23:19:00 +00:00
alexey-milovidov
67d2a35d4e
Merge pull request #27437 from ClickHouse/dataset-uk-property
Add OpenSky dataset
2021-08-09 00:55:04 +03:00
alexey-milovidov
a090bfc494
Merge pull request #27435 from syominsergey/patch-3
remove extra english text already translated to russian
2021-08-09 00:35:46 +03:00
alexey-milovidov
2d251abda3
Merge pull request #27434 from syominsergey/patch-2
fix description of changing url when cloning project
2021-08-09 00:35:13 +03:00
Sergei Semin
578e55184e
add info about clang version choice 2021-08-09 00:32:36 +03:00
alexey-milovidov
bf72ac0854
Merge pull request #25492 from morozovsk/patch-6
Documentation: manipulating projections
2021-08-09 00:29:15 +03:00
alexey-milovidov
4edc272f5a
Merge pull request #27436 from syominsergey/patch-4
translate some text from english to russian in developer-instruction.md
2021-08-09 00:24:33 +03:00
Alexey Milovidov
d159c83fb8 Add OpenSky dataset 2021-08-09 00:02:25 +03:00
Sergei Semin
8b0cfdda68
translate some text from english to russian in developer-instruction.md 2021-08-08 23:40:45 +03:00
alexey-milovidov
f18e1aa2de
Update coordinates.md 2021-08-08 23:38:51 +03:00
alexey-milovidov
63f624c0b9
Update coordinates.md 2021-08-08 23:38:06 +03:00
Sergei Semin
f55e113755
remove extra english text already translated to russian 2021-08-08 23:36:02 +03:00
alexey-milovidov
9661dd9232
Merge pull request #27432 from ClickHouse/dataset-uk-property
Add UK Property Price Paid data
2021-08-08 23:31:26 +03:00
Sergei Semin
8a320967e6
fix description of changing url when cloning project 2021-08-08 23:26:35 +03:00
alexey-milovidov
a5daf2d2c4
Merge pull request #27252 from ClickHouse/roaring-memory-tracker
Track memory consumed by Roaring Bitmaps.
2021-08-08 22:19:30 +03:00
Alexey Milovidov
844c0cf0ab Add UK Property Paid data 2021-08-08 22:16:53 +03:00
Alexey Milovidov
0f5d3963bd Add UK Property Paid data 2021-08-08 22:16:11 +03:00
Alexey Milovidov
b368c12400 Add UK Property Paid data 2021-08-08 22:15:18 +03:00
Alexey Milovidov
7631242a3b Add UK Property Paid data 2021-08-08 22:14:01 +03:00
Alexey Milovidov
e45a09e2f5 Fix wrong PR #27184 2021-08-08 21:09:39 +03:00
mergify[bot]
bd313279e8
Merge branch 'master' into hyperscansanitizer 2021-08-08 17:03:05 +00:00
Azat Khuzhin
2bfdf0d5a1 check-style: check system.projection_parts for database filter 2021-08-08 19:31:57 +03:00
Azat Khuzhin
940a537c66 Fix 01710_projection_fetch flakiness 2021-08-08 19:31:57 +03:00
Azat Khuzhin
0677a6c6cb Enable distributed_push_down_limit by default 2021-08-08 19:30:50 +03:00
Azat Khuzhin
91d7f3daa7 Fix lock-order-inversion while notifying about finished mutations
TSAN reports [1]:

  WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=36)
    Cycle in lock order graph: M16388 (0x7b7400011d68) => M1030334152907497744 (0x000000000000) => M16388

    Mutex M1030334152907497744 acquired here while holding mutex M16388 in thread T4:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x15220cd9)
      4 DB::StorageMergeTree::getIncompleteMutationsStatus(long, std::__1::set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std:>
      5 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0::operator()() const obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:464:36 (clickhouse+0x1521e2b1)
      6 void std::__1::condition_variable::wait<DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>(std::__1::unique_lock<std::__1::mutex>&, DB::StorageMergeTree::waitForMutation(lon>
      7 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:469:29 (clickhouse+0x1521e2b1)
      8 DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:496:9 (clickhouse+0x15221738)
      9 DB::InterpreterAlterQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterAlterQuery.cpp:113:16 (clickhouse+0x141182f6)
      10 DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:560:32 (clickhouse+0x149152f6)
      11 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:909:30 (clickhous>
      12 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:313:24 (clickhouse+0x153270af)

      Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

    Mutex M16388 acquired here while holding mutex M1030334152907497744 in thread T59:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::unique_lock<std::__1::mutex>::unique_lock(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse+0x15226c31)
      4 DB::StorageMergeTree::selectPartsToMutate(std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::shared_ptr<DB::RWLockImpl::LockHolderImpl>&) obj-x86_64-linux->
      5 DB::StorageMergeTree::scheduleDataProcessingJob(DB::IBackgroundJobExecutor&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1060:28 (clickhouse+0x15228b10)
      6 DB::BackgroundJobsExecutor::scheduleJob() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:229:17 (clickhouse+0x14f441dc)
      7 DB::IBackgroundJobExecutor::backgroundTaskFunction() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:10 (clickhouse+0x14f438ed)

  [1]: https://clickhouse-test-reports.s3.yandex.net/27248/4f3b80ff33c846465983aa2bc9ae9490e1118b15/fuzzer_tsan/report.htmlfail1
2021-08-08 19:28:29 +03:00
Azat Khuzhin
7964355ecf Fix mutation stuck on invalid partitions in non-replicated MergeTree
v2: Do not try to process empty mutations
    Found with flaky check [1].

      [1]: https://clickhouse-test-reports.s3.yandex.net/27248/66e8c0833392c20ba8dba3780f2b0d5c18f8194e/functional_stateless_tests_flaky_check_(address).html#fail1
2021-08-08 19:28:15 +03:00
Azat Khuzhin
5139067631 Guard BackgroundJobsExecutor from thread termination in case of uncaught exception 2021-08-08 19:27:58 +03:00
Azat Khuzhin
83d871b701 Do not miss exceptions from the ThreadPool 2021-08-08 19:25:43 +03:00
Azat Khuzhin
702d9955c0 Fix distributed queries with zero shards and aggregation 2021-08-08 19:22:49 +03:00
Nikolai Kochetov
57a7a20759
Update 02002_row_level_filter_bug.sh 2021-08-08 18:25:53 +03:00
mergify[bot]
8d05b7c5e5
Merge branch 'master' into roaring-memory-tracker 2021-08-08 15:19:57 +00:00
alexey-milovidov
560e71dcfa
Merge pull request #27424 from azat/WithMergeableStateAfterAggregationAndLimit-tests-fixes
Fix tests for WithMergeableStateAfterAggregationAndLimit
2021-08-08 18:14:04 +03:00
Amos Bird
a59ce96242
Skip test in fasttest 2021-08-08 22:30:08 +08:00
Artur
7a36b7a212 correct tests 2021-08-08 12:22:33 +00:00
Azat Khuzhin
f19ff3d61c Fix 02003_WithMergeableStateAfterAggregationAndLimit_LIMIT_BY_LIMIT_OFFSET 2021-08-08 10:27:26 +03:00
Azat Khuzhin
3be3c503aa Fix some comments 2021-08-08 09:58:07 +03:00
Azat Khuzhin
b8a679158c Fix 01951_distributed_push_down_limit reference 2021-08-08 09:36:36 +03:00
fuwhu
34e6f5c66b Correct the key data type used in mapContains. 2021-08-08 10:22:39 +08:00
alexey-milovidov
a7e26506cf
Merge pull request #26951 from ianton-ru/zero_copy_by_default
Set allow_remote_fs_zero_copy_replication to true by default
2021-08-08 04:46:49 +03:00
alexey-milovidov
ba3fbbaab0
Update ReplicatedMergeTreeMergeStrategyPicker.cpp 2021-08-08 04:43:59 +03:00
alexey-milovidov
e6e9a27d8f
Merge pull request #27377 from evillique/bz2
Add support for bzip2 compression method
2021-08-08 04:40:13 +03:00
Alexey Milovidov
3f8c1a1433 Add a test 2021-08-08 04:36:34 +03:00
Alexey Milovidov
e7afb7e159 Merge branch 'master' into roaring-memory-tracker 2021-08-08 04:33:33 +03:00
alexey-milovidov
8ac1f3b73d
Update projection.md 2021-08-08 04:24:23 +03:00
mergify[bot]
106bddb290
Merge branch 'master' into patch-6 2021-08-08 01:22:50 +00:00
alexey-milovidov
da803b56c1
Merge pull request #27380 from amosbird/logformattedquery
Add setting to log formatted query into system.query_log
2021-08-08 04:09:19 +03:00
alexey-milovidov
6811eb5e39
Merge pull request #27214 from azat/tests-replicated-database-macro
Allow parallel execution of *.sql tests with ReplicatedMergeTree (by using {database} macro)
2021-08-08 04:05:59 +03:00
alexey-milovidov
d2efe3869f
Merge pull request #26984 from azat/keeper-server-fix
Remove test_keeper_server usage (for {operation/session}_timeout_ms)
2021-08-08 04:04:42 +03:00
alexey-milovidov
c5207fc237
Merge pull request #26466 from azat/optimize-dist-select
Rework SELECT from Distributed optimizations
2021-08-08 03:59:32 +03:00