thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 20:02:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
129,309 Commits 9,330 Branches 1,783 Tags 2.1 GiB
63adf4d20c
Commit Graph

244 Commits

Author SHA1 Message Date
Alexey Milovidov
735fe5b189
Merge pull request #44282 from nickitat/set_mem_limit_for_fuzz_tests 
Set memory limit for server in AST fuzz tests
 2022-12-27 16:05:26 +03:00
Alexey Milovidov
44de7195d1
Merge branch 'master' into fix-fuzzer-html 2022-12-27 15:35:54 +03:00
Alexey Milovidov
9645941cdf
Merge pull request #44616 from ClickHouse/sanitizer-out-of-memory 
Do not fail the AST fuzzer if sanitizer is out of memory
 2022-12-27 15:15:26 +03:00
Alexey Milovidov
05c27f4e15
Merge pull request #44603 from ClickHouse/runlog 
Rename `runlog.log` to `run.log` in tests
 2022-12-27 15:14:05 +03:00
Alexey Milovidov
b85409f976 Do not fail if sanitizer is out of memory 2022-12-26 22:46:37 +01:00
Alexey Milovidov
a0753cbca2 Disable analyzer in fuzz and stress tests 2022-12-26 19:04:10 +01:00
Alexey Milovidov
6ba360be71 Rename runlog.log to run.log 2022-12-26 16:29:32 +01:00
Alexey Milovidov
fe73b0f648
Update run-fuzzer.sh 2022-12-25 22:14:25 +03:00
Alexey Milovidov
31bd775073
Update run-fuzzer.sh 2022-12-25 19:53:26 +03:00
Alexey Milovidov
51503fc9e4 Fuzzer HTML: fix trash 2022-12-25 16:19:42 +01:00
Nikita Taranov
c79c80f841 better 2022-12-20 19:56:29 +01:00
Nikita Taranov
7058a5e7ed impl 2022-12-15 18:37:04 +01:00
Alexander Tokmakov
cdbd1cb158 fix 2022-11-26 18:45:50 +01:00
Alexander Tokmakov
a982e62f65 fix 2022-11-25 19:01:24 +01:00
Alexander Tokmakov
98e6c76a60 fix 2022-11-25 16:58:18 +01:00
Alexander Tokmakov
1e1caf1440
Update run-fuzzer.sh 2022-11-25 15:06:52 +03:00
Alexander Tokmakov
e125f4016b fix 2022-11-24 21:37:08 +01:00
Alexander Tokmakov
162c7db972
Update run-fuzzer.sh 2022-11-24 14:29:48 +03:00
Alexander Tokmakov
f3b7193635 remove code that I do not understand 2022-11-23 21:49:44 +01:00
Alexander Tokmakov
9edb4d779b fix 2022-11-09 12:18:29 +01:00
Alexander Tokmakov
236f3329bc
Update run-fuzzer.sh 2022-11-08 22:54:58 +03:00
Dmitry Novik
a3c1049f9d
Update run-fuzzer.sh 2022-11-08 19:42:13 +01:00
Alexander Tokmakov
e659fb5d85
Update run-fuzzer.sh 2022-11-08 19:11:44 +03:00
alesapin
de6014e4a5
Update run-fuzzer.sh 2022-11-07 22:54:08 +01:00
Alexander Tokmakov
77ea58d539
Update run-fuzzer.sh 2022-10-21 17:58:20 +03:00
Alexander Tokmakov
dc99e67d3f
Update run-fuzzer.sh 2022-10-14 17:45:17 +03:00
Anton Popov
4ac5df2540 drop all fuzzed tables 2022-09-28 14:52:10 +00:00
Anton Popov
c8beae8237 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-28 12:47:02 +00:00
Azat Khuzhin
ced30a2072 Switch to llvm/clang 15 
It had been released few hours ago, and I want to check how clang-15
generates DWARF-5

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-10 21:38:34 +02:00
Azat Khuzhin
25e3bebd9d Rework core collecting on CI (eliminate gcore usage) 
gcore is a gdb command, that internally uses gdb to dump the core.

However with proper configuration of limits (core_dump.size_limit) it
should not be required, althought some issues is possible:
- non standard kernel.core_pattern
- sanitizers

So yes, gcore is more "universal" (you don't need to configure any
`kernel_pattern`), but it is ad-hoc, and it has drawbacks -
**it does not work when gdb fails**. For example gdb may fail with
`Dwarf Error: DW_FORM_strx1 found in non-DWO CU` in case of DWARF-5 [1].

  [1]: https://github.com/ClickHouse/ClickHouse/pull/40772#issuecomment-1236331323.

Let's try to switch to more native way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-04 22:07:16 +02:00
Alexey Milovidov
91ed3ebe9a Remove the remaining parts of "unbundled" build 2022-08-27 02:18:47 +02:00
Anton Popov
0ba886f763 enable fuzzing of table definitions 2022-08-11 12:24:15 +00:00
Anton Popov
e347fa2f45 add fuzzer for table definition 2022-08-11 02:34:10 +00:00
Yakov Olkhovskiy
09e6b1a497
run clickhouse to decompress 2022-08-09 09:16:03 -04:00
Alexander Tokmakov
3bc583c22b
Revert "copy self-extracting to output" 2022-08-08 20:13:40 +03:00
Yakov Olkhovskiy
72be640cfb clickhouse may be compressed - run once to decompress 2022-07-29 15:04:50 -04:00
Alexander Tokmakov
5ba304476d
Merge pull request #37560 from azat/tests-scripts-proper-wait 
Proper wait of the clickhouse-server in tests
 2022-06-17 18:23:30 +03:00
Alexey Milovidov
c1169019d2 Merge branch 'master' into llvm-14 2022-05-29 02:29:02 +02:00
Alexey Milovidov
e500ec5fd8
Update run-fuzzer.sh 2022-05-28 21:25:20 +03:00
Alexey Milovidov
f7a5b1fdaf Remove margin in test reports and change the font 2022-05-28 03:24:56 +02:00
Azat Khuzhin
478b03c5cb tests/fuzzer: use "clickhouse stop" to stop the server 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 07:18:51 +03:00
Alexey Milovidov
73b52859df Merge branch 'master' into llvm-14 2022-04-16 00:46:58 +02:00
kssenii
d59d4eda4f Fix tests 2022-04-06 01:58:16 +02:00
Alexey Milovidov
93edf05259
Merge branch 'master' into llvm-14 2022-04-01 04:49:03 +03:00
alesapin
b838a7dcb0 Remove outdated links from CI 2022-03-28 15:53:22 +02:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Alexey Milovidov
d0716b035f
Revert "Change timezone in Docker" 2022-03-13 04:03:06 +03:00
Alexey Milovidov
6b9129c494 Change timezone in Docker 2022-03-11 23:21:45 +01:00
Alexey Milovidov
aa8f282ad9 Switch to clang/llvm 14 2022-02-19 18:43:02 +01:00
Azat Khuzhin
4965586e7e
More gdb introspection on CI (#34517) 
* test/fuzzer: preserve core

This may help to capture things like in [1]:

    Failed assertion: "bin->low_bits_full > bin->low_bits_empty" Received signal 6 Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/19216f4c0ae0f72108c147f958a708b521ad27dc/fuzzer_astfuzzerdebug,actions//report.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: do not run 'info locals' since 'backtrace full' includes it

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: try capture backtrace from all threads

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test/stress: fix path for core artifacts

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #33389

* test/fuzzer: store core file in artifacts

v2: fix report, because of undefined variable CORE_LINK
v3: fix case when there is no core file

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-13 15:02:15 +03:00
Mikhail f. Shiryaev
2d02eab750
Revert "Merge pull request #34373 from ClickHouse/docker-tz" 
This reverts commit efd8044ab7, reversing
changes made to 4bb69bcb15.
 2022-02-08 15:34:01 +01:00
Mikhail f. Shiryaev
4a857d7d18
Use UTC in docker images 2022-02-07 12:35:32 +01:00
Mikhail f. Shiryaev
c465593706
Force rebuild images once again in master 2022-01-13 19:03:34 +01:00
Mikhail f. Shiryaev
e314eed992
Force rebuild images in CI 2022-01-13 17:49:23 +01:00
Mikhail f. Shiryaev
2a83a1c05d
Build dependent images from the current version 2022-01-12 17:07:52 +01:00
alesapin
333dfab6a6 Add retries to AST fuzzer 2021-12-28 16:12:51 +03:00
alesapin
118004015a Longer timeout for server start in fuzzer 2021-12-24 14:56:51 +03:00
Alexander Tokmakov
bd5931e5af improve gdb script 2021-12-15 13:21:21 +03:00
Alexander Tokmakov
dfa31a6577 fix 2021-12-10 22:38:34 +03:00
Alexander Tokmakov
254d6c5d59 fix 2021-12-10 20:07:28 +03:00
Alexander Tokmakov
3ff8aff1b5 try to fix attaching gdb 2021-12-10 18:03:57 +03:00
tavplubix
85e53b1b1f
Try fix attaching gdb in tests (#32448) 
* attach gdb with sudo

* fix

* Update run.sh
 2021-12-10 00:12:45 +03:00
Kruglov Pavel
ef42cd6821
Merge pull request #32138 from Avogar/fix-apply-lambda 
Fix 'APPLY lambda' parsing
 2021-12-09 17:09:11 +03:00
tavplubix
f4cbbfd8c0
Update run-fuzzer.sh 2021-12-09 14:17:55 +03:00
tavplubix
de0a9a84a9
Update run-fuzzer.sh 2021-12-09 13:51:39 +03:00
Kruglov Pavel
1df2383d00
Update run-fuzzer.sh 2021-12-09 12:17:31 +03:00
avogar
51bb37715f Fix fuzzer 2021-12-08 21:29:50 +03:00
Mikhail f. Shiryaev
e97e172dd8
Merge pull request #31822 from ClickHouse/rollback-28016 
Rollback changes from #28016: ru.archive.ubuntu.com
 2021-11-29 10:27:59 +01:00
Mikhail f. Shiryaev
5408d6232f
Make archive.ubuntu.com mirror configurable for build 2021-11-26 10:55:41 +01:00
alesapin
5815429059 Shorter stress and fuzzer tests 2021-11-25 17:54:50 +03:00
Azat Khuzhin
40be97c1dd Improve fuzzer report in case of fuzzer killed 
This will avoid Exception from fuzzer in description like in [1], that
someone may think that is related to the failure, which is obviously
don't in case it was KILLed.

  [1]: https://clickhouse-test-reports.s3.yandex.net/31259/a49e10c78860a451ff9cc39df6049c88234d507a/fuzzer_ubsan/report.html#fail1
 2021-11-12 01:01:10 +03:00
alesapin
4eca493ab8 Fix fuzzer on master 2021-10-29 13:47:29 +03:00
alesapin
ffc8dbd085
Update run-fuzzer.sh 2021-10-28 23:06:04 +03:00
alesapin
313584e553
Update run-fuzzer.sh 2021-10-28 19:12:47 +03:00
alesapin
0651a1f0e9
Update run-fuzzer.sh 2021-10-28 17:03:40 +03:00
alesapin
542196128b Trying to make fuzzer check less hardcoded 2021-10-28 12:29:50 +03:00
Azat Khuzhin
071a5af96e Revert "docker: add pandas/clickhouse_driver into test images" 
This reverts commit e07a6f3fc0.
 2021-10-12 21:06:26 +03:00
alexey-milovidov
268c155b7d
Merge pull request #29856 from azat/clickhouse-test-python-client 
Rewrite clickhouse-test to use python clickhouse_driver
 2021-10-09 17:58:40 +03:00
Azat Khuzhin
e07a6f3fc0 docker: add pandas/clickhouse_driver into test images 2021-10-08 00:09:37 +03:00
Azat Khuzhin
aeb670dbe0 Simplify obtaining of server pid in fuzzer tests 2021-10-05 21:05:49 +03:00
Nikita Mikhaylov
731d6c5755
Merge pull request #29513 from azat/fuzzer-fix-server-pid 
Fix server pid (hence exit code and attaching with gdb) in fuzzer tests
 2021-10-04 19:59:24 +03:00
Azat Khuzhin
74a7a0603d test/fuzzer: print stacktrace 
To get the stacktrace for errors w/o it [1]:

    SET max_parser_depth = 4000

    Got boring AST
    Code: 306. DB::Exception: Maximum parse depth (1000) exceeded. Consider rising max_parser_depth parameter. (TOO_DEEP_RECURSION)

  [1]: https://clickhouse-test-reports.s3.yandex.net/29513/769bfbe71f22c392458eced655080a0bbd442e02/fuzzer_ubsan/fuzzer.log
 2021-10-02 12:49:44 +03:00
Azat Khuzhin
769bfbe71f Fix server pid in fuzzer tests 
Previously it was the pid of the subshell 40 while it should be the pid
of the clickhouse-server 39:

Here we see that the server pid is 39:

    2021-09-28 11:02:34	 + pgrep -f clickhouse-server
    2021-09-28 11:02:34	 39

Here we see that the 40 is the pid of subshell:

    2021-09-28 11:02:45	 ch/docker/test/fuzzer/run-fuzzer.sh: line 90:    39 Killed                  clickhouse-server --config-file db/config.xml -- --path db 2>&1
    2021-09-28 11:02:45	         40 Done                    | tail -100000 > server.log

And here we see that server_pid variable is 40:

    2021-09-28 11:02:45	 + server_exit_code=0
    2021-09-28 11:02:45	 + wait 40

v2: wait in background to call wait in foreground and ensure that the
process is alive, since w/o job control this is the only way to obtain
the exit code
 2021-10-01 21:05:30 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
mergify[bot]
e3b3439535
Merge branch 'master' into fix-build-clang-13-2 2021-09-10 12:05:54 +00:00
Sergei Semin
b739d1cc92 make changes analog to https://github.com/ClickHouse/ClickHouse/pull/20495/files but for clang-13 2021-09-09 23:51:47 +03:00
alesapin
bdd26f2a19 Merge branch 'master' into move_docker_images_to_separate_repo 2021-09-08 12:36:36 +03:00
alesapin
0eac4529e0 Preparation 2021-09-06 14:33:39 +03:00
mergify[bot]
6ccf905133
Merge branch 'master' into try-llvm-12 2021-09-03 16:13:13 +00:00
Vladimir C
4cbc1aba19
Disable SC2001 shellcheck in docker/test/fuzzer/run-fuzzer.sh 2021-08-31 11:02:52 +03:00
vdimir
bc785bff25
Fix sed argument in test/fuzzer/run-fuzzer.sh 2021-08-30 14:25:08 +03:00
Ilya Yatsishin
7f7ecb0c60
Merge branch 'master' into try-llvm-12 2021-08-26 18:58:49 +03:00
vdimir
92f073a435
Call generate-test-j2.py in run-fuzzer.sh before calculating NEW_TESTS_OPT 2021-08-25 16:05:16 +03:00
vdimir
b9b0b17ad0
chmod +x docker/test/fuzzer/generate-test-j2.py 2021-08-25 11:20:42 +03:00
vdimir
0e37203dcd
Use jinja template tests in fuzzer 2021-08-25 11:18:30 +03:00
Yatsishin Ilya
91289983c6 Use ru.archive.ubuntu.com as default one is not responding from CI 2021-08-23 11:22:39 +03:00
mergify[bot]
bde834a8c0
Merge branch 'master' into try-llvm-12 2021-08-19 18:21:29 +00:00
Alexander Kuzmenkov
1f48166bfb
Update run-fuzzer.sh 2021-08-16 16:55:20 +03:00
Yatsishin Ilya
9145dd3fd0 Merge branch 'master' into try-llvm-12 2021-08-06 19:14:01 +03:00
Alexander Kuzmenkov
dbf0ed31e8 record server exit code in fuzzer 2021-07-22 17:05:47 +03:00