Commit Graph

85717 Commits

Author SHA1 Message Date
Antonio Andelic
cdedcee0ce
Merge pull request #35746 from ClickHouse/use-common-io-thread-pool
Use common IOThreadPool for S3 and URL
2022-03-31 08:41:54 +02:00
Azat Khuzhin
ff0e46c394 clickhouse-keeper: finish session on client EOF (fixes heap-use-after-free)
CI founds [1]:

    WARNING: ThreadSanitizer: heap-use-after-free (pid=1)
      Read of size 8 at 0x7b48001e0088 by thread T7 (mutexes: write M2588):
        ...
        1 DB::KeeperTCPHandler::runImpl()::$_0::operator()() const build_docker/../src/Server/KeeperTCPHandler.cpp:384:14 (clickhouse+0x194f974f)
        ...
        8 DB::KeeperDispatcher::setResponse() build_docker/../src/Coordination/KeeperDispatcher.cpp:222:9 (clickhouse+0x1999de82)
        9 DB::KeeperDispatcher::responseThread() build_docker/../src/Coordination/KeeperDispatcher.cpp:161:18 (clickhouse+0x1999d7a7)
        10 DB::KeeperDispatcher::initialize()::$_1::operator()() const build_docker/../src/Coordination/KeeperDispatcher.cpp:273:54 (clickhouse+0x199a55d1)

      Previous write of size 8 at 0x7b48001e0088 by thread T3:
        0 operator delete(void*, unsigned long) <null> (clickhouse+0xa0dad50)
        1 DB::KeeperTCPHandler::~KeeperTCPHandler() build_docker/../src/Server/KeeperTCPHandler.cpp:648:1 (clickhouse+0x194f8ee6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35772/38873d726d22b15d4c5a284410956927b2e2f95f/integration_tests__thread__actions__[4/4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-31 09:31:34 +03:00
Brendan Cox
87fff8b2d3 remove ATOMIC_FLAG_INIT: depreciated in C++20 and warns in clang-14 2022-03-30 21:45:48 -07:00
taiyang-li
70087bc959 Merge branch 'master' into improve_access_type 2022-03-31 12:19:17 +08:00
taiyang-li
f6ccda8097 Merge branch 'master' into hdfs_config_prefix 2022-03-31 12:16:05 +08:00
taiyang-li
c11f563d47 Merge branch 'master' into remote_filter_for_hive 2022-03-31 10:54:55 +08:00
alesapin
b078288709 Merge branch 'startup-sanity-checks' of github.com:ClickHouse/ClickHouse into startup-sanity-checks 2022-03-31 00:29:57 +02:00
alesapin
ae3bcbb99b Fix logs space check 2022-03-31 00:22:36 +02:00
Nikolay Degterinsky
f055d7b692
Merge pull request #34055 from palegre-tiny/groupSortedArray
Add groupSortedArray() function
2022-03-31 01:20:15 +03:00
alesapin
81ce991494 Merge branch 'master' into startup-sanity-checks 2022-03-31 00:08:43 +02:00
Kruglov Pavel
4ec3c35e14
Merge pull request #35755 from Avogar/fix-custom-to-string
Fix bug in conversion from custom types to string
2022-03-31 00:06:48 +02:00
alesapin
5a55913f45
Merge pull request #35720 from ClickHouse/parallel_stateful
Try to run stateful tests in parallel
2022-03-30 23:42:44 +02:00
alesapin
10db099d2f One more try to resurrect build hash 2022-03-30 23:27:46 +02:00
Alexander Gololobov
1d7059175b
Merge pull request #35681 from tonickkozlov/patch-2
[test] Adds clean-up for functional test 02167
2022-03-30 23:09:51 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439)
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-30 22:47:51 +02:00
Azat Khuzhin
8b84829e93 ci: replace directory system log tables artifacts with tsv
Changelog:
- remove query_thread_log (query_log should be enough)
- for s3 storage there /var/lib/clickhouse/data/system/*_log cannot be
  used, replace them with plain TSVWithNamesAndTypes, also note, that
  after schema inference it is even pretty easy to use
- support replicated database env correctly

Co-authored-by: tavplubix <tavplubix@clickhouse.com>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-30 21:50:55 +03:00
Azat Khuzhin
38873d726d clickhouse-keeper: fix implementation of server with poll()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-30 21:30:12 +03:00
kssenii
98e85dd68e Fix build 2022-03-30 20:27:04 +02:00
kssenii
ac4897f4f9 Fix 2022-03-30 20:16:41 +02:00
Alexander Gololobov
8f7329d7dd
Removed debug print 2022-03-30 20:10:25 +02:00
alesapin
02574bd05b
Merge pull request #35766 from ClickHouse/resurrect_official_flag
Resurrect build hash
2022-03-30 19:34:19 +02:00
Azat Khuzhin
9820230e95 clickhouse-keeper: correctly handle closed client connection
This will avoid noisy message like:

    DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-30 20:29:04 +03:00
alesapin
b388bb8448
Merge pull request #35718 from ClickHouse/fail_build_check_in_error_status
Fail CI checks in case of errors in checks (not failed tests)
2022-03-30 19:28:22 +02:00
alesapin
fcd4e0be65 Black 2022-03-30 19:15:27 +02:00
mergify[bot]
78bcf2655f
Merge branch 'master' into startup-sanity-checks 2022-03-30 17:07:35 +00:00
tavplubix
82a32194bf
Merge pull request #35768 from ClickHouse/tavplubix-patch-2
Fix 00484_preferred_max_column_in_block_size_bytes
2022-03-30 19:17:12 +03:00
Dmitry Novik
597c1a6523
Merge pull request #35719 from ClickHouse/memory-overcommit-test-fix
Try to fix test_global_overcommit_tracker flakyness
2022-03-30 18:15:40 +02:00
kssenii
c1ae5814a7 Merge master 2022-03-30 17:59:36 +02:00
Maksim Kita
dd7ae2b9aa Fix enable LLVM for JIT compilation in CMake 2022-03-30 17:43:52 +02:00
tavplubix
c34d512586
Update 00484_preferred_max_column_in_block_size_bytes.sql 2022-03-30 18:36:37 +03:00
kssenii
d810853b78 Fix path in test 2022-03-30 17:35:59 +02:00
kssenii
099fa0e668 Merge master 2022-03-30 17:33:32 +02:00
mergify[bot]
4813f0b305
Merge branch 'master' into is-secure-client-info 2022-03-30 15:12:56 +00:00
kssenii
b30948dd53 Relax test 2022-03-30 17:12:23 +02:00
tavplubix
60ecd49313
Merge pull request #35754 from ClickHouse/fix-ci-script-style-check
Fix stylecheck
2022-03-30 18:06:33 +03:00
alesapin
8980994faa Resurrect build hash 2022-03-30 16:29:13 +02:00
Alexander Gololobov
6e1036de50
Fix user file path extraction
It turned out that $CLICHOUSE_CLIENT could produce more verbose output that was not properly parsed. I made a change that should fix this parsing.
2022-03-30 16:12:13 +02:00
Maksim Kita
12ec3522e6
Merge pull request #35745 from azat/fix-tests-v22.3
Tests fixes
2022-03-30 15:44:25 +02:00
Kruglov Pavel
771f34d793
Merge pull request #35691 from Avogar/fix-matview
Fix creating materialized view with subquery after server restart
2022-03-30 14:57:14 +02:00
tavplubix
cf8332e109
Update 00170_s3_cache.reference 2022-03-30 14:38:21 +03:00
Antonio Andelic
7bc5731025 Merge branch 'master' into is-secure-client-info 2022-03-30 11:26:05 +00:00
Antonio Andelic
87bab4f2ea Fix style 2022-03-30 11:25:58 +00:00
Antonio Andelic
a3ad99be03 Create CallbackRunner in Storage 2022-03-30 11:24:50 +00:00
avogar
af4bfec051 Fix bug in conversion from custom types to string 2022-03-30 11:19:03 +00:00
Nikolai Kochetov
d3697625a9 Fix stylecheck 2022-03-30 10:55:49 +00:00
Vladimir C
31c367d3cd
Merge pull request #35651 from amosbird/columntransformerrename 2022-03-30 12:37:30 +02:00
tavplubix
531942ce5c
Update 00170_s3_cache.sql 2022-03-30 12:43:34 +03:00
mergify[bot]
9a391f2fed
Merge branch 'master' into parallel_stateful 2022-03-30 09:40:12 +00:00
alesapin
b19ecf60ad Fix style 2022-03-30 11:30:21 +02:00
alesapin
b5bce6d4dd
Merge pull request #35722 from ClickHouse/new_branch
Resurrect automatic labelling
2022-03-30 11:26:41 +02:00