thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 08:32:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
70,050 Commits 9,168 Branches 1,781 Tags 2.1 GiB
f9a1bf4156
Commit Graph

70050 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Azat Khuzhin
0677a6c6cb Enable distributed_push_down_limit by default 2021-08-08 19:30:50 +03:00
Azat Khuzhin
038241b6ed Add new index data skipping minmax index format for proper Nullable support 
Note, that it cannot be done w/o new extension, since index does not
have any header.

v2: use IDisk interface for existence check
v3: remove extra file existence check
v4: fix MATERIALIZE INDEX
 2021-08-08 19:30:12 +03:00
Azat Khuzhin
91d7f3daa7 Fix lock-order-inversion while notifying about finished mutations 
TSAN reports [1]:

  WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=36)
    Cycle in lock order graph: M16388 (0x7b7400011d68) => M1030334152907497744 (0x000000000000) => M16388

    Mutex M1030334152907497744 acquired here while holding mutex M16388 in thread T4:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x15220cd9)
      4 DB::StorageMergeTree::getIncompleteMutationsStatus(long, std::__1::set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std:>
      5 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0::operator()() const obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:464:36 (clickhouse+0x1521e2b1)
      6 void std::__1::condition_variable::wait<DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>(std::__1::unique_lock<std::__1::mutex>&, DB::StorageMergeTree::waitForMutation(lon>
      7 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:469:29 (clickhouse+0x1521e2b1)
      8 DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:496:9 (clickhouse+0x15221738)
      9 DB::InterpreterAlterQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterAlterQuery.cpp:113:16 (clickhouse+0x141182f6)
      10 DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:560:32 (clickhouse+0x149152f6)
      11 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:909:30 (clickhous>
      12 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:313:24 (clickhouse+0x153270af)

      Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

    Mutex M16388 acquired here while holding mutex M1030334152907497744 in thread T59:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::unique_lock<std::__1::mutex>::unique_lock(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse+0x15226c31)
      4 DB::StorageMergeTree::selectPartsToMutate(std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::shared_ptr<DB::RWLockImpl::LockHolderImpl>&) obj-x86_64-linux->
      5 DB::StorageMergeTree::scheduleDataProcessingJob(DB::IBackgroundJobExecutor&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1060:28 (clickhouse+0x15228b10)
      6 DB::BackgroundJobsExecutor::scheduleJob() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:229:17 (clickhouse+0x14f441dc)
      7 DB::IBackgroundJobExecutor::backgroundTaskFunction() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:10 (clickhouse+0x14f438ed)

  [1]: https://clickhouse-test-reports.s3.yandex.net/27248/4f3b80ff33c846465983aa2bc9ae9490e1118b15/fuzzer_tsan/report.htmlfail1
 2021-08-08 19:28:29 +03:00
Azat Khuzhin
7964355ecf Fix mutation stuck on invalid partitions in non-replicated MergeTree 
v2: Do not try to process empty mutations
    Found with flaky check [1].

      [1]: https://clickhouse-test-reports.s3.yandex.net/27248/66e8c0833392c20ba8dba3780f2b0d5c18f8194e/functional_stateless_tests_flaky_check_(address).html#fail1
 2021-08-08 19:28:15 +03:00
Azat Khuzhin
5139067631 Guard BackgroundJobsExecutor from thread termination in case of uncaught exception 2021-08-08 19:27:58 +03:00
Azat Khuzhin
83d871b701 Do not miss exceptions from the ThreadPool 2021-08-08 19:25:43 +03:00
Azat Khuzhin
702d9955c0 Fix distributed queries with zero shards and aggregation 2021-08-08 19:22:49 +03:00
Nikolai Kochetov
57a7a20759
Update 02002_row_level_filter_bug.sh 2021-08-08 18:25:53 +03:00
mergify[bot]
8d05b7c5e5
Merge branch 'master' into roaring-memory-tracker 2021-08-08 15:19:57 +00:00
alexey-milovidov
560e71dcfa
Merge pull request #27424 from azat/WithMergeableStateAfterAggregationAndLimit-tests-fixes 
Fix tests for WithMergeableStateAfterAggregationAndLimit
 2021-08-08 18:14:04 +03:00
Amos Bird
a59ce96242
Skip test in fasttest 2021-08-08 22:30:08 +08:00
Artur
7a36b7a212 correct tests 2021-08-08 12:22:33 +00:00
Azat Khuzhin
f19ff3d61c Fix 02003_WithMergeableStateAfterAggregationAndLimit_LIMIT_BY_LIMIT_OFFSET 2021-08-08 10:27:26 +03:00
Azat Khuzhin
3be3c503aa Fix some comments 2021-08-08 09:58:07 +03:00
Azat Khuzhin
b8a679158c Fix 01951_distributed_push_down_limit reference 2021-08-08 09:36:36 +03:00
fuwhu
34e6f5c66b Correct the key data type used in mapContains. 2021-08-08 10:22:39 +08:00
alexey-milovidov
a7e26506cf
Merge pull request #26951 from ianton-ru/zero_copy_by_default 
Set allow_remote_fs_zero_copy_replication to true by default
 2021-08-08 04:46:49 +03:00
alexey-milovidov
ba3fbbaab0
Update ReplicatedMergeTreeMergeStrategyPicker.cpp 2021-08-08 04:43:59 +03:00
alexey-milovidov
e6e9a27d8f
Merge pull request #27377 from evillique/bz2 
Add support for bzip2 compression method
 2021-08-08 04:40:13 +03:00
Alexey Milovidov
3f8c1a1433 Add a test 2021-08-08 04:36:34 +03:00
Alexey Milovidov
e7afb7e159 Merge branch 'master' into roaring-memory-tracker 2021-08-08 04:33:33 +03:00
alexey-milovidov
8ac1f3b73d
Update projection.md 2021-08-08 04:24:23 +03:00
mergify[bot]
106bddb290
Merge branch 'master' into patch-6 2021-08-08 01:22:50 +00:00
alexey-milovidov
da803b56c1
Merge pull request #27380 from amosbird/logformattedquery 
Add setting to log formatted query into system.query_log
 2021-08-08 04:09:19 +03:00
alexey-milovidov
6811eb5e39
Merge pull request #27214 from azat/tests-replicated-database-macro 
Allow parallel execution of *.sql tests with ReplicatedMergeTree (by using {database} macro)
 2021-08-08 04:05:59 +03:00
alexey-milovidov
d2efe3869f
Merge pull request #26984 from azat/keeper-server-fix 
Remove test_keeper_server usage (for {operation/session}_timeout_ms)
 2021-08-08 04:04:42 +03:00
alexey-milovidov
c5207fc237
Merge pull request #26466 from azat/optimize-dist-select 
Rework SELECT from Distributed optimizations
 2021-08-08 03:59:32 +03:00
alexey-milovidov
a60d60dec1
Merge pull request #27153 from CurtizJ/legacy-tuple-name 
Implement `legacy_column_name_of_tuple_literal` in a less intrusive way
 2021-08-08 01:16:31 +03:00
Azat Khuzhin
c62de36e4c Mark more tests as long 2021-08-08 00:55:40 +03:00
Braulio Valdivielso
a833d1cf9e fixup! change http error response 2021-08-07 19:57:19 +01:00
mergify[bot]
9869c8bb10
Merge branch 'master' into zero_copy_by_default 2021-08-07 16:53:44 +00:00
alexey-milovidov
b8f4d480a3
Merge pull request #27081 from ClickHouse/fix_logs_in_nuraft 
Fix excessive logging in NuRaft on server shutdown
 2021-08-07 19:48:53 +03:00
alexey-milovidov
6b6b4f114b
Merge pull request #27089 from ianton-ru/fix_test_merge_tree_s3_failover 
Fix test_merge_tree_s3_failover with debug build
 2021-08-07 19:46:37 +03:00
alexey-milovidov
ac3abb8fd2
Update test.py 2021-08-07 19:46:19 +03:00
mergify[bot]
589505e76b
Merge branch 'master' into legacy-tuple-name 2021-08-07 16:42:20 +00:00
alexey-milovidov
af1d41a3de
Merge pull request #27228 from myrrc/improvement/tester-changes 
Using formatted string literals in clickhouse-test, extracted sort key functions and stacktraces printer
 2021-08-07 19:35:46 +03:00
Alexey Milovidov
161aa40206 Fix build; add comments 2021-08-07 19:30:59 +03:00
alexey-milovidov
4b52c37681
Merge pull request #27249 from azat/fix-WithMergeableStateAfterAggregationAndLimit 
Fix WithMergeableStateAfterAggregationAndLimit with LIMIT BY and LIMIT OFFSET
 2021-08-07 19:28:59 +03:00
alexey-milovidov
d9eab713aa
Merge pull request #27324 from Algunenano/fix_01300 
Fix 01300_client_save_history_when_terminated_long
 2021-08-07 19:27:29 +03:00
alexey-milovidov
22bc7eb170
Merge pull request #27294 from Algunenano/better_kafka_errors 
Improve kafka integration test error messages
 2021-08-07 19:26:31 +03:00
alexey-milovidov
7512b61dd9
Merge pull request #27300 from Algunenano/flaky_00738 
Improve 00738_lock_for_inner_table stability
 2021-08-07 19:25:02 +03:00
Alexey Milovidov
c9d66defa0 Merge branch 'master' into roaring-memory-tracker 2021-08-07 19:01:31 +03:00
alexey-milovidov
83c0f89dcd
Merge pull request #27327 from qoega/update-zlib-ng-2.0.5 
Try update contrib/zlib-ng
 2021-08-07 18:51:15 +03:00
Dmitriy
27f489b41f Correction of the description 
Скорректировал описание.
 2021-08-07 18:50:24 +03:00
alexey-milovidov
5523e5e153
Update Settings.h 2021-08-07 18:48:52 +03:00
alexey-milovidov
24cc073a22
Merge pull request #27328 from ClickHouse/fix_partition_id_validation 
Fix partition id validation
 2021-08-07 18:45:19 +03:00
alexey-milovidov
833b232bae
Merge pull request #27383 from kssenii/fix-test 
Fix flacky test
 2021-08-07 18:40:57 +03:00
alexey-milovidov
fa5bcb2e30
Merge pull request #27301 from ClickHouse/fix-unknown-column-with-sampling 
Fix unknown column with sampling
 2021-08-07 18:39:19 +03:00
alexey-milovidov
fad30f6949
Merge pull request #27361 from myrrc/bugfix/error-parsing-proc-meminfo 
Fixing reading of /proc/meminfo when kB suffix is not present
 2021-08-07 18:38:15 +03:00
Nikolay Degterinsky
b5a6e18511 Small fixes 2021-08-07 15:15:49 +00:00