Commit Graph

119317 Commits

Author SHA1 Message Date
Azat Khuzhin
16165d9498 Improve error messages for decompressor
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 13:10:40 +02:00
Azat Khuzhin
c374653c7b Revert "Merge pull request #52138 from azat/decompressor-inode"
This reverts commit 6524031348, reversing
changes made to 9bf114f9a3.

This was not a good idea, since the underlying problem was that
`/proc/self/exe` was pointing to `qemu-$ARCH-static` (because the code
uses realpath() over normal interface readlink(), which is not caught by
the qemu linux-user).

And this means that later, it will try to overwrite incorrect binary and
then execute some garbage.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 13:10:29 +02:00
Alexey Milovidov
44929b7d28
Merge pull request #52154 from ClickHouse/fix-02497_storage_file_reader_selection
Fix test 02497_storage_file_reader_selection
2023-07-17 06:45:48 +03:00
Alexey Milovidov
60908ac269
Merge pull request #51991 from ClickHouse/test-cleanup
Check that functional tests cleanup their tables
2023-07-17 01:33:54 +03:00
Alexey Milovidov
59ecdbd53e Fix test 02497_storage_file_reader_selection 2023-07-16 22:30:04 +02:00
Alexey Milovidov
09b300cf07
Merge pull request #52143 from ClickHouse/remove-default-value
Remove default argument value
2023-07-16 19:22:30 +03:00
robot-ch-test-poll2
1f1fcd027a
Merge pull request #52130 from zvonand/zvonand-issue-42138
Small fix for toDateTime64() for dates after 2283-12-31
2023-07-16 18:11:28 +02:00
robot-ch-test-poll2
6524031348
Merge pull request #52138 from azat/decompressor-inode
Skip protection from double decompression if inode from maps cannot be obtained
2023-07-16 17:20:43 +02:00
Alexey Milovidov
9bf114f9a3
Merge pull request #52145 from ClickHouse/fix-order-by-tuple-of-window-functions
Fix ORDER BY tuple of WINDOW functions
2023-07-16 17:43:07 +03:00
Alexey Milovidov
71d6206fc4
Fix Python 2023-07-16 17:42:22 +03:00
Azat Khuzhin
20a671b8cf Skip protection from double decompression if inode from maps cannot be obtained
Under some circumstances, like using qemu-$ARCH-static, /proc/self/maps
will not contain information about /proc/self/exe.

Well, strictly speaking it does contains, however qemu will not pass it
to the user program:

<details>

<summary>strace</summary>

    $ sudo strace -s10000 -f arch-chroot . /qemu-riscv64-static /clickhouse
    ...
    execve("/qemu-riscv64-static", ["/qemu-riscv64-static", "/clickhouse"], 0x7fffffffe458 /* 20 vars */) = 0
    readlinkat(AT_FDCWD, "/proc/self/exe", "/qemu-riscv64-static", 4096) = 20
    openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 4
    [pid  3126] read(4, "00010000-00111000 r--p 00000000 fe:01 30312571                           /clickhouse\n00111000-00119000 r--p 00100000 fe:01 30312571                           /clickhouse\n00119000-0011a000 rw-p 00108000 fe:01 30312571                           /clickhouse\n0011a000-0013d000 rw-p 00000000 00:00 0 \n4000000000-4000001000 ---p 00000000 00:00 0 \n4000001000-4000801000 rw-p 00000000 00:00 0 \n4000801000-400081a000 r--p 00000000 fe:01 30316932                       /lib/riscv64-linux-gnu/ld-2.32.so\n400081a000-400081b000 ---p 00000000 00:00 0 \n400081b000-400081c000 r--p 00019000 fe:01 30316932                       /lib/riscv64-linux-gnu/ld-2.32.so\n400081c000-400081e000 rw-p 0001a000 fe:01 30316932                       /lib/riscv64-linux-gnu/ld-2.32.so\n400081e000-400081f000 r--p 00000000 00:00 0 \n400081f000-4000922000 r--p 00000000 fe:01 30316935 /lib/riscv64-linux-gnu/libc-2.32.so\n4000922000-4000926000 r--p 00102000 fe:01 30316935                       /lib/riscv64-linux-gnu/libc-2.32.so\n4000926000-4000928000 rw-p 00106000 fe:01 30316935                       /lib/riscv64-linux-gnu/libc-2.32.so\n4000928000-400092d000 rw-p 00000000 00:00 0 \n400092d000-40009af000 r--p 00000000 fe:01 30316943                       /lib/riscv64-linux-gnu/libm-2.32.so\n40009af000-40009b0000 r--p 00081000 fe:01 30316943                       /lib/riscv64-linux-gnu/libm-2.32.so\n40009b0000-40009b1000 rw-p 00082000 fe:01 30316943 /lib/riscv64-linux-gnu/libm-2.32.so\n40009b1000-40009c5000 r--p 00000000 fe:01 30316946                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n40009c5000-40009c6000 r--p 00013000 fe:01 30316946                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n40009c6000-40009c7000 rw-p 00014000 fe:01 30316946                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n40009c7000-40009cb000 rw-p 00000000 00:00 0 \n40009cb000-40009cd000 r--p 00000000 fe:01 30316939                       /lib/riscv64-linux-gnu/libdl-2.32.so\n40009cd000-40009ce000 r--p 00001000 fe:01 30316939                       /lib/riscv64-linux-gnu/libdl-2.32.so\n40009ce000-40009cf000 rw-p 00002000 fe:01 30316939                       /lib/riscv64-linux-gnu/libdl-2.32.so\n40009cf000-40009d1000 rw-p 00000000 00:00 0 \n7fffe8000000-7fffeffff000 rwxp 00000000 00:00 0 \n7fffeffff000-7ffff0000000 ---p 00000000 00:00 0 \n7ffff0000000-7ffff0021000 rw-p 00000000 00:00 0 \n7ffff0021000-7ffff4000000 ---p 00000000 00:00 0 \n7ffff6b4b000-7ffff6b5b000 rw-p 00000000 00:00 0 \n7ffff71ff000-7ffff7200000 ---p 00000000 00:00 0 \n7ffff7200000-7ffff7a00000 rw-p 00000000 00:00 0\n7ffff7a00000-7ffff7a3c000 r--p 00000000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7a3c000-7ffff7c74000 r-xp 0003c000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7c74000-7ffff7d77000 r--p 00274000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7d77000-7ffff7dce000 r--p 00377000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7dce000-7ffff7df7000 rw-p 003ce000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7df7000-7ffff7e0c000 rw-p 00000000 00:00 0                          [heap]\n7ffff7e0c000-7ffff7e70000 rw-p 00000000 00:00 0                          [heap]\n7ffff7f42000-7ffff7ff9000 rw-p 00000000 00:00 0 \n7ffff7ff9000-7ffff7ffd000 r--p 00000000 00:00 0                          [vvar]\n7ffff7ffd000-7ffff7fff000 r-xp 00000000 00:00 0                          [vdso]\n7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0                          [stack]\nffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]\n", 4096) = 3608
    [pid  3126] read(4, "", 1024)           = 0
    [pid  3126] close(4)                    = 0
    [pid  3126] write(3, "10000-111000 r-xp 00000000 fe:01 30312571", 41) = 41
    [pid  3126] write(3, "                                /clickhouse\n", 44) = 44
    [pid  3126] write(3, "111000-119000 r--p 00100000 fe:01 30312571", 42) = 42
    [pid  3126] write(3, "                               /clickhouse\n", 43) = 43
    [pid  3126] write(3, "119000-11a000 rw-p 00108000 fe:01 30312571", 42) = 42
    [pid  3126] write(3, "                               /clickhouse\n", 43) = 43
    [pid  3126] write(3, "11a000-13d000 rw-p 00000000 00:00 0", 35) = 35
    [pid  3126] write(3, "                                      \n", 39) = 39
    [pid  3126] write(3, "4000000000-4000001000 ---p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "4000001000-4000801000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              [stack]\n", 38) = 38
    [pid  3126] write(3, "4000801000-400081a000 r-xp 00000000 fe:01 30316932", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/ld-2.32.so\n", 57 <unfinished ...>
    [pid  3127] <... clock_nanosleep resumed>0x7ffff79ff060) = 0
    [pid  3126] <... write resumed>)        = 57
    [pid  3127] clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000},  <unfinished ...>
    [pid  3126] write(3, "400081a000-400081b000 ---p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "400081b000-400081c000 r--p 00019000 fe:01 30316932", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/ld-2.32.so\n", 57) = 57
    [pid  3126] write(3, "400081c000-400081e000 rw-p 0001a000 fe:01 30316932", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/ld-2.32.so\n", 57) = 57
    [pid  3126] write(3, "400081e000-400081f000 r-xp 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "400081f000-4000922000 r-xp 00000000 fe:01 30316935", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libc-2.32.so\n", 59) = 59
    [pid  3126] write(3, "4000922000-4000926000 r--p 00102000 fe:01 30316935", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libc-2.32.so\n", 59) = 59
    [pid  3126] write(3, "4000926000-4000928000 rw-p 00106000 fe:01 30316935", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libc-2.32.so\n", 59) = 59
    [pid  3126] write(3, "4000928000-400092d000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "400092d000-40009af000 r-xp 00000000 fe:01 30316943", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libm-2.32.so\n", 59) = 59
    [pid  3126] write(3, "40009af000-40009b0000 r--p 00081000 fe:01 30316943", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libm-2.32.so\n", 59) = 59
    [pid  3126] write(3, "40009b0000-40009b1000 rw-p 00082000 fe:01 30316943", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libm-2.32.so\n", 59) = 59
    [pid  3126] write(3, "40009b1000-40009c5000 r-xp 00000000 fe:01 30316946", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n", 65) = 65
    [pid  3126] write(3, "40009c5000-40009c6000 r--p 00013000 fe:01 30316946", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n", 65) = 65
    [pid  3126] write(3, "40009c6000-40009c7000 rw-p 00014000 fe:01 30316946", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n", 65) = 65
    [pid  3126] write(3, "40009c7000-40009cb000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "40009cb000-40009cd000 r-xp 00000000 fe:01 30316939", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libdl-2.32.so\n", 60) = 60
    [pid  3126] write(3, "40009cd000-40009ce000 r--p 00001000 fe:01 30316939", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libdl-2.32.so\n", 60) = 60
    [pid  3126] write(3, "40009ce000-40009cf000 rw-p 00002000 fe:01 30316939", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libdl-2.32.so\n", 60) = 60
    [pid  3126] write(3, "40009cf000-40009d1000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-16 08:17:14 +02:00
robot-ch-test-poll
3b0a4040cd
Merge pull request #51876 from evillique/fix-mongodb-inserts
Fix inserts into MongoDB tables
2023-07-16 05:28:43 +02:00
Alexey Milovidov
d900aa467f
Merge pull request #51992 from ClickHouse/fix-test_extreme_deduplication
Fix test_extreme_deduplication
2023-07-16 05:54:44 +03:00
Alexey Milovidov
2be26de2db
Merge pull request #52142 from ClickHouse/remove-detection-of-flaky-test
There is no point in detecting flaky tests
2023-07-16 05:52:04 +03:00
robot-clickhouse-ci-2
ac3cc1c2ff
Merge pull request #45671 from ClibMouse/feature/interval-kql-style-formatting
Implement KQL-style formatting for Interval
2023-07-16 04:06:54 +02:00
robot-ch-test-poll
67492bc2ed
Merge pull request #52122 from azat/keeper-stat-zxid
Print Zxid in keeper stat command in hex (so as ZooKeeper)
2023-07-16 03:28:53 +02:00
Alexey Milovidov
9a421369d7
Merge pull request #49709 from amosbird/fix_49695
Fix projection with optimize_aggregators_of_group_by_keys
2023-07-16 04:10:54 +03:00
Alexey Milovidov
66b66db39d Fix ORDER BY tuple of WINDOW functions 2023-07-16 02:52:06 +02:00
Alexey Milovidov
7e6551dbd0
Merge branch 'master' into fix-mongodb-inserts 2023-07-16 01:47:52 +03:00
Andrey Zvonov
f2e68d1d7e
Merge branch 'master' into zvonand-issue-42138 2023-07-16 01:06:41 +03:00
Alexey Milovidov
095624fbd1 Remove default argument value 2023-07-15 23:33:11 +02:00
Alexey Milovidov
c57dbf3410 Merge branch 'test-cleanup' of github.com:ClickHouse/ClickHouse into test-cleanup 2023-07-15 23:12:29 +02:00
Alexey Milovidov
4f6a0c2fac Loosen the check 2023-07-15 23:11:22 +02:00
robot-clickhouse
44fc93ba63 Automatic style fix 2023-07-15 21:10:21 +00:00
Alexey Milovidov
e837f3cb9c Merge branch 'master' into test-cleanup 2023-07-15 23:07:36 +02:00
Alexey Milovidov
90f9d156d2
Merge branch 'master' into fix-test_extreme_deduplication 2023-07-16 00:05:40 +03:00
Alexey Milovidov
2ad7d121ae
Merge pull request #52085 from azat/tests/test_extreme_deduplication
Fix test_extreme_deduplication flakiness
2023-07-16 00:02:18 +03:00
Alexey Milovidov
20b77e946a There is no point in detecting flaky tests 2023-07-15 23:00:23 +02:00
Alexey Milovidov
84f7a6a6d7
Merge pull request #52083 from azat/tests/test_keeper_s3_snapshot
Fix flakiness of test_keeper_s3_snapshot flakiness
2023-07-15 23:46:47 +03:00
robot-ch-test-poll3
9e7361a0f6
Merge pull request #52135 from ucasfl/alias
Add array_agg as alias of groupArray for PostgreSQL compatibility
2023-07-15 21:14:38 +02:00
robot-ch-test-poll1
3b2b7d75c9
Merge pull request #52094 from zvonand/zvonand-docs-dt-parsing-short-string
Small docs update for DateTime, DateTime64
2023-07-15 19:31:49 +02:00
robot-ch-test-poll1
78994c8a96
Merge pull request #52090 from zvonand/zvonand-docs-22948
Small docs update for toYearWeek() function
2023-07-15 19:31:19 +02:00
Alexey Milovidov
47a33efc73
Merge branch 'master' into tests/test_keeper_s3_snapshot 2023-07-15 19:27:06 +03:00
Alexey Milovidov
edbd05f332
Merge branch 'master' into tests/test_extreme_deduplication 2023-07-15 19:20:32 +03:00
Alexey Milovidov
0eab3276f5
Merge branch 'master' into keeper-stat-zxid 2023-07-15 19:07:48 +03:00
robot-clickhouse
d365fade94
Merge pull request #52128 from azat/distributed-ddl-version
Bump version of the distributed_ddl_entry_format_version to 5 by default
2023-07-15 18:07:01 +02:00
flynn
290c145acf Add array_agg as alias of groupArray for PostgreSQL compatibility 2023-07-15 14:53:46 +00:00
Ilya Yatsishin
d058715cfc
Merge pull request #51186 from ClibMouse/jemalloc_s390x 2023-07-15 14:17:04 +02:00
Vitaly Baranov
42936c8939
Merge pull request #51448 from azat/backups-native-copy
Add ability to disable native copy for S3
2023-07-15 08:41:10 +02:00
Vitaly Baranov
2f23b6a593
Merge branch 'master' into backups-native-copy 2023-07-15 08:40:17 +02:00
Azat Khuzhin
7d3b28f9c3 Bump version of the distributed_ddl_entry_format_version to 5 by default
This will enable the following features for distributed DDL queries:
- opentelemetry support (#41484)
- initial_query_id pass through (#50015)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-15 08:06:21 +02:00
zvonand
761cc7afa5 add tests 2023-07-15 03:17:18 +02:00
zvonand
cbe0a0e748 upd threshold due to dt64 range change 2023-07-15 03:04:56 +02:00
Alexey Milovidov
1b9bcae68e
Merge pull request #52129 from ClickHouse/revert-51291-ADQM-868
Revert "Millisecond and microsecond support in date_diff / age functions"
2023-07-14 23:41:19 +03:00
Alexey Milovidov
7284749d05
Merge pull request #52126 from ClickHouse/alexey-milovidov-patch-7
Add RISC-V 64 to the docs
2023-07-14 23:27:41 +03:00
Alexey Milovidov
6ff9c19129
Merge pull request #52125 from ClickHouse/docs-clang-16
Update build.md for clang-16
2023-07-14 23:27:28 +03:00
Alexey Milovidov
9307e60c20
Revert "Millisecond and microsecond support in date_diff / age functions" 2023-07-14 23:21:26 +03:00
Azat Khuzhin
6abdde12ca Print Zxid in keeper stat command in hex (so as ZooKeeper)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-14 21:17:09 +02:00
Alexey Milovidov
94112895a1
Add RISC-V 64 to the docs 2023-07-14 21:09:58 +03:00
Alexey Milovidov
32bc5866ea Update build.md for clang-16 2023-07-14 20:04:58 +02:00