thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 09:22:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
131,047 Commits 9,297 Branches 1,782 Tags 2.1 GiB
1d9c0db1f6
Commit Graph

249 Commits

Author SHA1 Message Date
Alexey Milovidov
8f7a84c964 Print fatal messages in Fuzzer 2022-12-30 22:16:30 +01:00
Alexey Milovidov
234193acc9 Print fatal messages in Fuzzer 2022-12-30 22:13:47 +01:00
Alexey Milovidov
79ae191611 A few improvements to AST Fuzzer 2022-12-29 18:55:28 +01:00
Alexey Milovidov
f30272279c Whitespace 2022-12-28 12:41:44 +01:00
alesapin
f93c52ac69 Add +x flag for run-fuzzer.sh 2022-12-27 19:48:58 +01:00
Alexey Milovidov
735fe5b189
Merge pull request #44282 from nickitat/set_mem_limit_for_fuzz_tests 
Set memory limit for server in AST fuzz tests
 2022-12-27 16:05:26 +03:00
Alexey Milovidov
44de7195d1
Merge branch 'master' into fix-fuzzer-html 2022-12-27 15:35:54 +03:00
Alexey Milovidov
9645941cdf
Merge pull request #44616 from ClickHouse/sanitizer-out-of-memory 
Do not fail the AST fuzzer if sanitizer is out of memory
 2022-12-27 15:15:26 +03:00
Alexey Milovidov
05c27f4e15
Merge pull request #44603 from ClickHouse/runlog 
Rename `runlog.log` to `run.log` in tests
 2022-12-27 15:14:05 +03:00
Alexey Milovidov
b85409f976 Do not fail if sanitizer is out of memory 2022-12-26 22:46:37 +01:00
Alexey Milovidov
a0753cbca2 Disable analyzer in fuzz and stress tests 2022-12-26 19:04:10 +01:00
Alexey Milovidov
6ba360be71 Rename runlog.log to run.log 2022-12-26 16:29:32 +01:00
Alexey Milovidov
fe73b0f648
Update run-fuzzer.sh 2022-12-25 22:14:25 +03:00
Alexey Milovidov
31bd775073
Update run-fuzzer.sh 2022-12-25 19:53:26 +03:00
Alexey Milovidov
51503fc9e4 Fuzzer HTML: fix trash 2022-12-25 16:19:42 +01:00
Nikita Taranov
c79c80f841 better 2022-12-20 19:56:29 +01:00
Nikita Taranov
7058a5e7ed impl 2022-12-15 18:37:04 +01:00
Alexander Tokmakov
cdbd1cb158 fix 2022-11-26 18:45:50 +01:00
Alexander Tokmakov
a982e62f65 fix 2022-11-25 19:01:24 +01:00
Alexander Tokmakov
98e6c76a60 fix 2022-11-25 16:58:18 +01:00
Alexander Tokmakov
1e1caf1440
Update run-fuzzer.sh 2022-11-25 15:06:52 +03:00
Alexander Tokmakov
e125f4016b fix 2022-11-24 21:37:08 +01:00
Alexander Tokmakov
162c7db972
Update run-fuzzer.sh 2022-11-24 14:29:48 +03:00
Alexander Tokmakov
f3b7193635 remove code that I do not understand 2022-11-23 21:49:44 +01:00
Alexander Tokmakov
9edb4d779b fix 2022-11-09 12:18:29 +01:00
Alexander Tokmakov
236f3329bc
Update run-fuzzer.sh 2022-11-08 22:54:58 +03:00
Dmitry Novik
a3c1049f9d
Update run-fuzzer.sh 2022-11-08 19:42:13 +01:00
Alexander Tokmakov
e659fb5d85
Update run-fuzzer.sh 2022-11-08 19:11:44 +03:00
alesapin
de6014e4a5
Update run-fuzzer.sh 2022-11-07 22:54:08 +01:00
Alexander Tokmakov
77ea58d539
Update run-fuzzer.sh 2022-10-21 17:58:20 +03:00
Alexander Tokmakov
dc99e67d3f
Update run-fuzzer.sh 2022-10-14 17:45:17 +03:00
Anton Popov
4ac5df2540 drop all fuzzed tables 2022-09-28 14:52:10 +00:00
Anton Popov
c8beae8237 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-28 12:47:02 +00:00
Azat Khuzhin
ced30a2072 Switch to llvm/clang 15 
It had been released few hours ago, and I want to check how clang-15
generates DWARF-5

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-10 21:38:34 +02:00
Azat Khuzhin
25e3bebd9d Rework core collecting on CI (eliminate gcore usage) 
gcore is a gdb command, that internally uses gdb to dump the core.

However with proper configuration of limits (core_dump.size_limit) it
should not be required, althought some issues is possible:
- non standard kernel.core_pattern
- sanitizers

So yes, gcore is more "universal" (you don't need to configure any
`kernel_pattern`), but it is ad-hoc, and it has drawbacks -
**it does not work when gdb fails**. For example gdb may fail with
`Dwarf Error: DW_FORM_strx1 found in non-DWO CU` in case of DWARF-5 [1].

  [1]: https://github.com/ClickHouse/ClickHouse/pull/40772#issuecomment-1236331323.

Let's try to switch to more native way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-04 22:07:16 +02:00
Alexey Milovidov
91ed3ebe9a Remove the remaining parts of "unbundled" build 2022-08-27 02:18:47 +02:00
Anton Popov
0ba886f763 enable fuzzing of table definitions 2022-08-11 12:24:15 +00:00
Anton Popov
e347fa2f45 add fuzzer for table definition 2022-08-11 02:34:10 +00:00
Yakov Olkhovskiy
09e6b1a497
run clickhouse to decompress 2022-08-09 09:16:03 -04:00
Alexander Tokmakov
3bc583c22b
Revert "copy self-extracting to output" 2022-08-08 20:13:40 +03:00
Yakov Olkhovskiy
72be640cfb clickhouse may be compressed - run once to decompress 2022-07-29 15:04:50 -04:00
Alexander Tokmakov
5ba304476d
Merge pull request #37560 from azat/tests-scripts-proper-wait 
Proper wait of the clickhouse-server in tests
 2022-06-17 18:23:30 +03:00
Alexey Milovidov
c1169019d2 Merge branch 'master' into llvm-14 2022-05-29 02:29:02 +02:00
Alexey Milovidov
e500ec5fd8
Update run-fuzzer.sh 2022-05-28 21:25:20 +03:00
Alexey Milovidov
f7a5b1fdaf Remove margin in test reports and change the font 2022-05-28 03:24:56 +02:00
Azat Khuzhin
478b03c5cb tests/fuzzer: use "clickhouse stop" to stop the server 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 07:18:51 +03:00
Alexey Milovidov
73b52859df Merge branch 'master' into llvm-14 2022-04-16 00:46:58 +02:00
kssenii
d59d4eda4f Fix tests 2022-04-06 01:58:16 +02:00
Alexey Milovidov
93edf05259
Merge branch 'master' into llvm-14 2022-04-01 04:49:03 +03:00
alesapin
b838a7dcb0 Remove outdated links from CI 2022-03-28 15:53:22 +02:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Alexey Milovidov
d0716b035f
Revert "Change timezone in Docker" 2022-03-13 04:03:06 +03:00
Alexey Milovidov
6b9129c494 Change timezone in Docker 2022-03-11 23:21:45 +01:00
Alexey Milovidov
aa8f282ad9 Switch to clang/llvm 14 2022-02-19 18:43:02 +01:00
Azat Khuzhin
4965586e7e
More gdb introspection on CI (#34517) 
* test/fuzzer: preserve core

This may help to capture things like in [1]:

    Failed assertion: "bin->low_bits_full > bin->low_bits_empty" Received signal 6 Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/19216f4c0ae0f72108c147f958a708b521ad27dc/fuzzer_astfuzzerdebug,actions//report.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: do not run 'info locals' since 'backtrace full' includes it

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: try capture backtrace from all threads

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test/stress: fix path for core artifacts

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #33389

* test/fuzzer: store core file in artifacts

v2: fix report, because of undefined variable CORE_LINK
v3: fix case when there is no core file

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-13 15:02:15 +03:00
Mikhail f. Shiryaev
2d02eab750
Revert "Merge pull request #34373 from ClickHouse/docker-tz" 
This reverts commit efd8044ab7, reversing
changes made to 4bb69bcb15.
 2022-02-08 15:34:01 +01:00
Mikhail f. Shiryaev
4a857d7d18
Use UTC in docker images 2022-02-07 12:35:32 +01:00
Mikhail f. Shiryaev
c465593706
Force rebuild images once again in master 2022-01-13 19:03:34 +01:00
Mikhail f. Shiryaev
e314eed992
Force rebuild images in CI 2022-01-13 17:49:23 +01:00
Mikhail f. Shiryaev
2a83a1c05d
Build dependent images from the current version 2022-01-12 17:07:52 +01:00
alesapin
333dfab6a6 Add retries to AST fuzzer 2021-12-28 16:12:51 +03:00
alesapin
118004015a Longer timeout for server start in fuzzer 2021-12-24 14:56:51 +03:00
Alexander Tokmakov
bd5931e5af improve gdb script 2021-12-15 13:21:21 +03:00
Alexander Tokmakov
dfa31a6577 fix 2021-12-10 22:38:34 +03:00
Alexander Tokmakov
254d6c5d59 fix 2021-12-10 20:07:28 +03:00
Alexander Tokmakov
3ff8aff1b5 try to fix attaching gdb 2021-12-10 18:03:57 +03:00
tavplubix
85e53b1b1f
Try fix attaching gdb in tests (#32448) 
* attach gdb with sudo

* fix

* Update run.sh
 2021-12-10 00:12:45 +03:00
Kruglov Pavel
ef42cd6821
Merge pull request #32138 from Avogar/fix-apply-lambda 
Fix 'APPLY lambda' parsing
 2021-12-09 17:09:11 +03:00
tavplubix
f4cbbfd8c0
Update run-fuzzer.sh 2021-12-09 14:17:55 +03:00
tavplubix
de0a9a84a9
Update run-fuzzer.sh 2021-12-09 13:51:39 +03:00
Kruglov Pavel
1df2383d00
Update run-fuzzer.sh 2021-12-09 12:17:31 +03:00
avogar
51bb37715f Fix fuzzer 2021-12-08 21:29:50 +03:00
Mikhail f. Shiryaev
e97e172dd8
Merge pull request #31822 from ClickHouse/rollback-28016 
Rollback changes from #28016: ru.archive.ubuntu.com
 2021-11-29 10:27:59 +01:00
Mikhail f. Shiryaev
5408d6232f
Make archive.ubuntu.com mirror configurable for build 2021-11-26 10:55:41 +01:00
alesapin
5815429059 Shorter stress and fuzzer tests 2021-11-25 17:54:50 +03:00
Azat Khuzhin
40be97c1dd Improve fuzzer report in case of fuzzer killed 
This will avoid Exception from fuzzer in description like in [1], that
someone may think that is related to the failure, which is obviously
don't in case it was KILLed.

  [1]: https://clickhouse-test-reports.s3.yandex.net/31259/a49e10c78860a451ff9cc39df6049c88234d507a/fuzzer_ubsan/report.html#fail1
 2021-11-12 01:01:10 +03:00
alesapin
4eca493ab8 Fix fuzzer on master 2021-10-29 13:47:29 +03:00
alesapin
ffc8dbd085
Update run-fuzzer.sh 2021-10-28 23:06:04 +03:00
alesapin
313584e553
Update run-fuzzer.sh 2021-10-28 19:12:47 +03:00
alesapin
0651a1f0e9
Update run-fuzzer.sh 2021-10-28 17:03:40 +03:00
alesapin
542196128b Trying to make fuzzer check less hardcoded 2021-10-28 12:29:50 +03:00
Azat Khuzhin
071a5af96e Revert "docker: add pandas/clickhouse_driver into test images" 
This reverts commit e07a6f3fc0.
 2021-10-12 21:06:26 +03:00
alexey-milovidov
268c155b7d
Merge pull request #29856 from azat/clickhouse-test-python-client 
Rewrite clickhouse-test to use python clickhouse_driver
 2021-10-09 17:58:40 +03:00
Azat Khuzhin
e07a6f3fc0 docker: add pandas/clickhouse_driver into test images 2021-10-08 00:09:37 +03:00
Azat Khuzhin
aeb670dbe0 Simplify obtaining of server pid in fuzzer tests 2021-10-05 21:05:49 +03:00
Nikita Mikhaylov
731d6c5755
Merge pull request #29513 from azat/fuzzer-fix-server-pid 
Fix server pid (hence exit code and attaching with gdb) in fuzzer tests
 2021-10-04 19:59:24 +03:00
Azat Khuzhin
74a7a0603d test/fuzzer: print stacktrace 
To get the stacktrace for errors w/o it [1]:

    SET max_parser_depth = 4000

    Got boring AST
    Code: 306. DB::Exception: Maximum parse depth (1000) exceeded. Consider rising max_parser_depth parameter. (TOO_DEEP_RECURSION)

  [1]: https://clickhouse-test-reports.s3.yandex.net/29513/769bfbe71f22c392458eced655080a0bbd442e02/fuzzer_ubsan/fuzzer.log
 2021-10-02 12:49:44 +03:00
Azat Khuzhin
769bfbe71f Fix server pid in fuzzer tests 
Previously it was the pid of the subshell 40 while it should be the pid
of the clickhouse-server 39:

Here we see that the server pid is 39:

    2021-09-28 11:02:34	 + pgrep -f clickhouse-server
    2021-09-28 11:02:34	 39

Here we see that the 40 is the pid of subshell:

    2021-09-28 11:02:45	 ch/docker/test/fuzzer/run-fuzzer.sh: line 90:    39 Killed                  clickhouse-server --config-file db/config.xml -- --path db 2>&1
    2021-09-28 11:02:45	         40 Done                    | tail -100000 > server.log

And here we see that server_pid variable is 40:

    2021-09-28 11:02:45	 + server_exit_code=0
    2021-09-28 11:02:45	 + wait 40

v2: wait in background to call wait in foreground and ensure that the
process is alive, since w/o job control this is the only way to obtain
the exit code
 2021-10-01 21:05:30 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
mergify[bot]
e3b3439535
Merge branch 'master' into fix-build-clang-13-2 2021-09-10 12:05:54 +00:00
Sergei Semin
b739d1cc92 make changes analog to https://github.com/ClickHouse/ClickHouse/pull/20495/files but for clang-13 2021-09-09 23:51:47 +03:00
alesapin
bdd26f2a19 Merge branch 'master' into move_docker_images_to_separate_repo 2021-09-08 12:36:36 +03:00
alesapin
0eac4529e0 Preparation 2021-09-06 14:33:39 +03:00
mergify[bot]
6ccf905133
Merge branch 'master' into try-llvm-12 2021-09-03 16:13:13 +00:00
Vladimir C
4cbc1aba19
Disable SC2001 shellcheck in docker/test/fuzzer/run-fuzzer.sh 2021-08-31 11:02:52 +03:00
vdimir
bc785bff25
Fix sed argument in test/fuzzer/run-fuzzer.sh 2021-08-30 14:25:08 +03:00
Ilya Yatsishin
7f7ecb0c60
Merge branch 'master' into try-llvm-12 2021-08-26 18:58:49 +03:00
vdimir
92f073a435
Call generate-test-j2.py in run-fuzzer.sh before calculating NEW_TESTS_OPT 2021-08-25 16:05:16 +03:00
vdimir
b9b0b17ad0
chmod +x docker/test/fuzzer/generate-test-j2.py 2021-08-25 11:20:42 +03:00
vdimir
0e37203dcd
Use jinja template tests in fuzzer 2021-08-25 11:18:30 +03:00