Commit Graph

84028 Commits

Author SHA1 Message Date
Raúl Marín
8137f13fcd Improve ANY performance by acting over batches 2022-02-20 02:43:28 +01:00
mergify[bot]
1ed69119b0
Merge branch 'master' into PR202202190800 2022-02-20 01:11:21 +00:00
mergify[bot]
a08aceffb2
Merge branch 'master' into master 2022-02-20 01:11:17 +00:00
alexey-milovidov
b75fbaa6a9
Merge pull request #34758 from ClickHouse/changelog-22.2
Changelog for version 22.2
2022-02-19 23:47:39 +03:00
Alexey Milovidov
ce56733e1a Update changelog 2022-02-19 21:47:17 +01:00
Alexey Milovidov
f07f8113d7 Editing the changelog 2022-02-19 21:00:58 +01:00
Amos Bird
84d111cde9
Better ReadBuffer asynchronicity 2022-02-20 01:08:21 +08:00
alexey-milovidov
73fe35e552
Update Volnitsky.h 2022-02-19 19:53:36 +03:00
alexey-milovidov
82728e03b5
Update Volnitsky.h 2022-02-19 19:52:38 +03:00
Alexey Milovidov
e1a2e376a3 Submodules are mandatory 2022-02-19 17:44:24 +01:00
bharatnc
a4cf3303f9 fix fuzzer segfault 2022-02-19 08:41:21 -08:00
alexey-milovidov
25c75649fe
Update build.md 2022-02-19 19:20:47 +03:00
alexey-milovidov
d8af2dfba0
Update build.md 2022-02-19 19:19:34 +03:00
alexey-milovidov
42313befd3
Update build.md 2022-02-19 19:19:07 +03:00
Xudong Zhang
9c4cc70dae update document: add _timestamp_ms of kafka 2022-02-19 23:49:37 +08:00
Vitaly Baranov
874b2c8dcb
Merge pull request #34596 from vitlibar/allow-restrictive-without-permissive
Allow restrictive row policies without permissive
2022-02-19 21:45:28 +07:00
Kruglov Pavel
a0ae68db99
Fix tests 2022-02-19 17:41:56 +03:00
Nikolay Degterinsky
c3c1344f0b
Merge pull request #34718 from amosbird/logfilecmd
Allow only to specify --log-file without configuration
2022-02-19 16:11:27 +03:00
Maksim Kita
0bfac173f9
Merge pull request #34745 from bharatnc/ncb/fixes-to-s2
fix s2 functions having const columns
2022-02-19 12:20:25 +01:00
Alexey Milovidov
bb1e3ebba1 Fix year on the website, thanks Vova VG 2022-02-19 11:50:12 +01:00
Azat Khuzhin
9b753c84f0 Print only total profile events for --profile-events-delay-ms=-1
There was serveral issues:
1) there can be events for multiple threads in one block
2) lack of filter for thread_id = 0
3) thread_id was included into comparator

Fixes: #34719
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-19 13:49:41 +03:00
Azat Khuzhin
0025110a16 Replace exit() with abort() in case of NuRaft errors
CI founds [1]:

    WARNING: ThreadSanitizer: data race (pid=99)
      Write of size 8 at 0x7b14002192b0 by thread T27:
        12 std::__1::map<>
        13 DB::OpenedFileCache::~OpenedFileCache() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:27:7 (clickhouse+0xac66de6)
        14 cxa_at_exit_wrapper(void*) crtstuff.c (clickhouse+0xaa3646f)
        15 decltype(*(std::__1::forward<nuraft::raft_server*&>(fp0)).*fp()) std::__1::__invoke<void ()(), nuraft::raft_server*&, void>()

      Previous read of size 8 at 0x7b14002192b0 by thread T37 (mutexes: write M732116415018761216):
        4 DB::OpenedFileCache::get() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:47:37 (clickhouse+0xac66784)

    Thread T27 'nuraft_commit' (tid=193, running) created by main thread at:
    ...
    Thread T37 'MergeMutate' (tid=204, running) created by main thread at:
    ...

But it also reports that the mutex was already destroyed:

    Mutex M732116415018761216 is already destroyed.

The problem is that [nuraft can call `exit()`](1707a7572a/src/handle_commit.cxx (L157-L158)) which will call atexit handlers:

    2022.02.17 22:54:03.495450 [ 193 ] {} <Error> RaftInstance: background committing thread encounter err Memory limit (total) exceeded: would use 56.56 GiB (attempt to allocate chunk of 8192 bytes), maximum: 55.82 GiB, exiting to protect the system

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/5a8cf3ac98808dadf125068a33ed9c622998a484/fuzzer_astfuzzertsan,actions//report.html

Let's replace exit() with abort() to avoid this.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-19 13:05:08 +03:00
Anton Popov
5ccee4e778
Merge pull request #34739 from azat/fix-fsync_part_directory
Fix ENOENT with fsync_part_directory and Vertical merge
2022-02-19 12:21:07 +03:00
Ilya Yatsishin
108e12b530
Merge pull request #34729 from vdimir/add-trusted-taiyang-li 2022-02-19 11:44:45 +03:00
cnmade
bb49867955 Translate zh/engines/table-engines/integrations/sqlite: remove bak file 2022-02-19 15:21:08 +08:00
cnmade
da5cb8950d Translate zh/engines/table-engines/integrations/sqlite: translate to zh 2022-02-19 15:20:34 +08:00
cnmade
8f6a3f4d7f Translate zh/engines/table-engines/integrations/sqlite: reimport files 2022-02-19 15:14:22 +08:00
cnmade
a7dda35f7f Translate zh/engines/table-engines/integrations/sqlite: rename old file 2022-02-19 15:11:43 +08:00
bharatnc
233b4486ed add tests 2022-02-18 22:42:39 -08:00
bharatnc
b7678c1bc1 fix s2 functions having const columns 2022-02-18 22:42:27 -08:00
Azat Khuzhin
fef5f146e7 Fix ENOENT with fsync_part_directory and Vertical merge
fsync of the temporary part directory is superfluous anyway, and besides
that directory is not exists at that time, that will lead to ENOENT
error:

    2022.02.18 17:02:51.634565 [ 35639 ] {} <Error> void DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(DB::TaskRuntimeDataPtr) [Queue = DB::MergeMutateRuntimeQueue]: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/data/system/text_log/tmp_merge_202202_1864_3192_14/, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception() @ 0xb26ecfa in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
    1. DB::throwFromErrnoWithPath() @ 0xb2700ea in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
    2. DB::LocalDirectorySyncGuard::LocalDirectorySyncGuard() @ 0x14905531 in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
    3. DB::DiskLocal::getDirectorySyncGuard() const @ 0x148af3e3 in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
    4. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x157bef13 in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug

Note, that IMergeTreeDataPart::renameTo() anyway will have fsync for the
directory.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-19 07:50:59 +03:00
bharatnc
d9acbf480c fix - skip fast tests 2022-02-18 18:45:15 -08:00
bharatnc
8076dd8cf5 add tests 2022-02-18 16:35:07 -08:00
bharatnc
7c9a581af6 convert const cols to full cols 2022-02-18 16:34:50 -08:00
Mikhail f. Shiryaev
d2cbc5af40
Add patch type for stable/lts releases 2022-02-19 00:13:07 +01:00
Mikhail f. Shiryaev
df9165bbff
Refactoring of Release, add Repo
- Do not pass Namespace to Release.do
- Decouple repository to another class
2022-02-19 00:13:00 +01:00
Mikhail f. Shiryaev
0960fbad52
Set selease_branch only once as Release property 2022-02-19 00:10:21 +01:00
Mikhail f. Shiryaev
d2a8cff508
Add release_branch checker 2022-02-19 00:01:42 +01:00
Mikhail f. Shiryaev
0f1565a3a2
Clean-up the code a little bit 2022-02-19 00:01:39 +01:00
alexey-milovidov
9630087131
Update app.py 2022-02-19 00:26:43 +03:00
alexey-milovidov
37fbf7c329
Merge branch 'master' into add-trusted-taiyang-li 2022-02-19 00:25:52 +03:00
Nikolai Kochetov
40a48c5670 Enable async writes to S3. 2022-02-18 18:32:04 +00:00
Nikolai Kochetov
e4d5db6161
Merge pull request #34717 from azat/merge-mutate-memory-tracker
Fix possible memory_tracker use-after-free (for async s3 writes) for merges/mutations
2022-02-18 19:28:43 +01:00
Maksim Kita
3dd6b91cbe
Merge pull request #34731 from kitaisreal/performance-tests-fix-h3
Performance tests fix H3
2022-02-18 19:21:38 +01:00
Vladimir C
fcec52f49c
Merge pull request #34612 from vdimir/fix_aarch64/01681_arg_min_max_if_fix
Adjust 01681_arg_min_max_if_fix
2022-02-18 18:35:52 +01:00
Vladimir C
9b7d011ee7
Merge pull request #34529 from vdimir/join-nullable-on-pipeline
Apply join_use_nulls on types before join
2022-02-18 18:34:44 +01:00
alexey-milovidov
b9a377c890
Update build.md 2022-02-18 20:01:44 +03:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Maksim Kita
80b0efb367 Performance tests fix H3 2022-02-18 15:57:54 +00:00
Maksim Kita
f3dca476ae Updated documentation 2022-02-18 15:35:18 +00:00