thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 11:02:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,103 Commits 9,189 Branches 1,782 Tags 2.2 GiB
dea95e6c33
Commit Graph

466 Commits

Author SHA1 Message Date
alesapin
e4e6cccbf8 Remove garbage 2022-12-27 12:59:09 +01:00
Alexey Milovidov
a0753cbca2 Disable analyzer in fuzz and stress tests 2022-12-26 19:04:10 +01:00
Alexey Milovidov
91c1f9de8f Faster server startup after stress test 2022-12-26 18:13:50 +01:00
alesapin
211c6062bc Fix 2022-12-25 19:37:02 +01:00
Alexey Milovidov
f22fc35900 Minimal changes 2022-12-25 14:18:05 +01:00
alesapin
bd19c6b699 Make BC check optional (if env var set) 2022-12-24 17:46:22 +01:00
Alexander Tokmakov
f2b177880f print stats in tests 2022-12-23 16:04:29 +01:00
Alexander Tokmakov
3f336190a7
Update run.sh 2022-12-14 14:21:21 +03:00
Alexander Tokmakov
f68a407636
Update docker/test/stress/run.sh 
Co-authored-by: Alexander Gololobov <440544+davenger@users.noreply.github.com>
 2022-12-13 18:58:02 +03:00
Alexander Tokmakov
49f3e8e19b
Update run.sh 2022-12-13 18:47:13 +03:00
Alexander Tokmakov
371e85d854
Merge pull request #44157 from ClickHouse/fix_session_expired_on_startup 
Ignore "session expired" errors after BC check
 2022-12-13 15:10:36 +03:00
Alexander Tokmakov
469406c494 ignore session expired errors after BC check 2022-12-12 14:02:14 +01:00
Sema Checherinda
9a3aecd4c1 set max-tries bigger when backward compatibility check runs, old version still drop parts at shutdown 2022-12-05 16:05:14 +01:00
Sema Checherinda
dc52a40c04 decrease stop max-tries back to default, do not clear old parts at shutdown 2022-12-05 16:05:14 +01:00
Alexander Tokmakov
ac578efc05
Update run.sh 2022-11-29 20:16:59 +03:00
Azat Khuzhin
ca16f5136f stress: use --max-tries only since 22.12 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 09:40:39 +01:00
Azat Khuzhin
2c36b52050 stress: increase timeout for server waiting after TERM 
Greater timeout after TERM may reduce about of KILL, let's try.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 09:40:39 +01:00
Alexey Milovidov
69d1be50bc
Merge pull request #43446 from ClickHouse/fix-typo-3 
Fix typo
 2022-11-25 15:22:28 +01:00
alesapin
f64f0461fe
Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key 
Enable compress marks and primary key for CI
 2022-11-24 12:24:04 +01:00
alesapin
41d3825770 Merge branch 'Enable_compress_marks_and_primary_key' of https://github.com/ClickHouse/ClickHouse into Enable_compress_marks_and_primary_key 2022-11-22 19:12:00 +01:00
alesapin
31142808de Disable compressed marks and index in stress tests 2022-11-22 19:10:51 +01:00
Alexey Milovidov
b7e4547f53
Merge branch 'master' into fix-typo-3 2022-11-22 18:30:06 +01:00
Alexey Milovidov
56f8c0b84b Fix typo 2022-11-21 15:10:15 +01:00
Alexey Milovidov
17e2189199 Fix backward compatibility check 2022-11-21 12:51:51 +01:00
alesapin
2dbf07f58a Splitting checks 2022-11-18 17:38:17 +01:00
Igor Nikonov
b12ebab65c Enable keeper fault injection and retries for insert queries in 
funcntional tests
 2022-11-11 18:07:15 +00:00
Alexander Tokmakov
acbad99b24 remove abandonable_lock part 2 2022-11-02 14:28:49 +01:00
Alexey Milovidov
4f582e19a8
Merge pull request #42479 from ClickHouse/Avogar-patch-1 
Add error "Destination table is myself" to exception list in BC check
 2022-10-25 04:55:12 +02:00
Alexey Milovidov
c4fd827442
Merge pull request #42483 from ClickHouse/Avogar-patch-2 
Get current clickhouse version without sending query in BC check
 2022-10-25 04:54:24 +02:00
Azat Khuzhin
f0aed68893 ci/stress: fix memory limits overrides for hung check 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-22 20:53:18 +02:00
Alexander Tokmakov
f958c155f3
Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-19 17:25:24 +03:00
Antonio Andelic
10b176fe19
Merge pull request #41854 from azat/tests/stress/fix-dbg-sym-bc 
tests: fix debug symbols (and possible crashes) for backward compatiblity check
 2022-10-19 16:18:41 +02:00
Kruglov Pavel
2ca0d37e83
Get current clickhouse version without sending query in BC check 2022-10-19 13:25:03 +02:00
Kruglov Pavel
632965212f
Add error "Destination table is myself" to exception list in BC check 2022-10-19 12:52:32 +02:00
Alexander Tokmakov
4211605093 disable zk fault injections in bc check 2022-10-03 13:13:29 +02:00
Alexander Tokmakov
8741785a72 temporarily disable ThreadFuzzer with tsan 2022-09-29 18:08:35 +02:00
Azat Khuzhin
9b20468198 tests: fix debug symbols (and possible crashes) for backward compatiblity check 
stress running previous version of the server w/o correct debug symbols
right now, since nobody restore clickhouse.debug file, and this can lead
to the following issues, like in [1]:
- incorrect stack traces
- gdb crashes
- clickhouse crashes, due to non-robust internal DWARF parser (probably)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/41730/8cc53a48ae99a765085f44a75fa49314d1f1cc7d/stress_test__ubsan_.html

Right now I decided not to rework the script to make it less error
prone, but simply fix the problem.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-27 22:50:28 +02:00
alesapin
69f3a66538 Keep the most important log in stress tests 2022-09-27 11:16:10 +02:00
Antonio Andelic
97cf045203
Merge pull request #41721 from ClickHouse/collect-correctly-logs-in-stress-test 
Collect logs in Stress test using clickhouse-local
 2022-09-27 08:43:44 +02:00
Antonio Andelic
eb78761a7e Collect necessary 2022-09-26 16:30:01 +00:00
Antonio Andelic
6f4a636e8f Remove wildcard 2022-09-26 11:21:53 +00:00
Antonio Andelic
8fde8b2c56 Try with multiple calls 2022-09-26 11:03:24 +00:00
Antonio Andelic
c60d9db687
Merge branch 'master' into ignore-attach-thread-keeper-errors 2022-09-26 08:38:48 +02:00
Antonio Andelic
5ff1bcd553
Merge branch 'master' into collect-correctly-logs-in-stress-test 2022-09-26 08:38:38 +02:00
alesapin
06e0f554d8 Fix fetch to local disk 2022-09-23 16:46:53 +02:00
Antonio Andelic
1d93c56d1a Collect logs using clickhouse-local 2022-09-23 10:54:16 +00:00
Antonio Andelic
a17a3e1de1 Ignore Keeper hardware errors 2022-09-23 08:23:57 +00:00
kssenii
46f74aaba9 Update stress/run.sh 2022-09-12 20:10:35 +02:00
Alexander Tokmakov
e77b9e4d0c
Merge pull request #40775 from azat/ci/core-dumps-rework 
Rework core collecting on CI (eliminate gcore usage)
 2022-09-09 20:20:10 +03:00
Alexey Milovidov
7776512b04
Merge pull request #41002 from azat/ci/fix-oom-check 
ci/stress: clear dmesg before run to fix "OOM in dmesg" check
 2022-09-06 06:41:36 +03:00
Alexander Tokmakov
b264be3c63
Merge branch 'master' into zookeeper_client_fault_injection 2022-09-05 22:13:09 +03:00
Azat Khuzhin
2724b67537 ci/stress: clear dmesg before run to fix "OOM in dmesg" check 
CI: https://s3.amazonaws.com/clickhouse-test-reports/40772/afa137ae2b6108e72c2d6e43556a04548afa2ea9/stress_test__ubsan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-05 15:51:36 +02:00
Azat Khuzhin
25e3bebd9d Rework core collecting on CI (eliminate gcore usage) 
gcore is a gdb command, that internally uses gdb to dump the core.

However with proper configuration of limits (core_dump.size_limit) it
should not be required, althought some issues is possible:
- non standard kernel.core_pattern
- sanitizers

So yes, gcore is more "universal" (you don't need to configure any
`kernel_pattern`), but it is ad-hoc, and it has drawbacks -
**it does not work when gdb fails**. For example gdb may fail with
`Dwarf Error: DW_FORM_strx1 found in non-DWO CU` in case of DWARF-5 [1].

  [1]: https://github.com/ClickHouse/ClickHouse/pull/40772#issuecomment-1236331323.

Let's try to switch to more native way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-04 22:07:16 +02:00
Alexander Tokmakov
8bdb589c2b Merge branch 'master' into zookeeper_client_fault_injection 2022-08-29 13:34:57 +02:00
alesapin
133ca01447 Merge branch 'master' into stress_s3 2022-08-29 11:25:28 +02:00
Azat Khuzhin
ebc61a36e0 tests/stress: improve OOM detection (add separate check by dmesg) 
Right now if you will look at the OOM errors:
- OOM killer (or signal 9) in clickhouse-server.log
- Backward compatibility check: OOM messages in clickhouse-server.log

Most of them are not real, but just clickhouse server got KILLed by
clickhouse stop, #40678 may imporove the situation, but to definitely
sure that there was OOM let's look at dmesg.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 12:46:58 +02:00
Azat Khuzhin
3b519c5d44 tests/stress: capture stacktrace of server hungs if pid was removed already 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-26 22:06:32 +03:00
alesapin
704c4b2c5b Stop thread fuzzer on shutdown 2022-08-26 11:54:54 +02:00
alesapin
3ff6489fae Merge branch 'master' into stress_s3 2022-08-25 13:14:58 +02:00
alesapin
ad692f732a Merge branch 'master' into stress_s3 2022-08-25 13:13:30 +02:00
alesapin
35f9815b8e Fix backward comp check 2022-08-24 14:43:02 +02:00
Azat Khuzhin
50bddc43dc tests/stress: ignore NETLINK_ERROR from checkPermissionsImpl 
Since now with --privileged it has CAP_SYS_ADMIN and tries to
communicate via netlink.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-19 14:07:06 +02:00
Alexander Tokmakov
3d253ec51b
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-18 21:23:50 +03:00
alesapin
922818bbd9 Merge branch 'stress_s3' of github.com:ClickHouse/ClickHouse into stress_s3 2022-08-18 14:46:50 +02:00
alesapin
932ea146f5 Merge branch 'master' into stress_s3 2022-08-18 13:14:47 +02:00
alesapin
86b1e33eed Disable cache on writes 2022-08-17 19:00:53 +02:00
alesapin
600d22851f Grep dangerous S3 errors 2022-08-17 12:43:11 +02:00
alesapin
0433b801d2 Configure properly 2022-08-17 12:27:15 +02:00
Alexander Tokmakov
ae000e9125
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-17 12:48:54 +03:00
alesapin
1ec6627a70 Fix tables creation 2022-08-16 18:28:17 +02:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Alexander Tokmakov
589c3408d2
Merge pull request #40234 from ClickHouse/better_message_on_restore_covered 
Better error message when restoring covered parts
 2022-08-15 22:01:48 +03:00
alesapin
243bd492fa Trying to fix it 2022-08-15 20:55:11 +02:00
alesapin
96722a13bb Merge branch 'master' into stress_s3 2022-08-15 20:20:31 +02:00
Alexander Tokmakov
edaff70010 better error message when restoring covered parts 2022-08-15 13:53:14 +02:00
Alexander Tokmakov
467ef7bbc2
Update run.sh 2022-08-12 14:30:18 +03:00
Alexander Tokmakov
b9d18182f2 fix 2022-08-11 15:27:26 +02:00
kssenii
5c3227ba56 Merge master 2022-08-10 12:00:34 +02:00
kssenii
0dda03c94b Fix checks 2022-08-10 00:06:58 +02:00
Azat Khuzhin
3772415588 tests/stress: add dmesg output (to see OOM details) 
max_server_memory_usage already set to 75%, so OOM should not happens,
the reason is that because RSS does not match with memory tracker
statistics:

    2022.08.05 12:36:57.869896 [ 82524 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 64.69 GiB, peak 65.26 GiB, will set to 62.80 GiB (RSS), difference: -1.89 GiB
    ...
    2022.08.05 12:37:00.213440 [ 82334 ] {} <Error> void DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(DB::TaskRuntimeDataPtr) [Queue = DB::MergeMutateRuntimeQueue]: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 64.68 GiB (attempt to allocate chunk of 1298794 bytes), maximum: 51.44 GiB. OvercommitTracker decision: Memory overcommit isn't used. Waiting time or orvercommit denominator are set to zero.. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-06 12:34:38 +03:00
kssenii
7a9b0bc47f Merge master 2022-08-05 01:48:52 +02:00
Kruglov Pavel
235649cb98
Merge pull request #39458 from Avogar/fix-cancel-insert-into-function 
Fix WriteBuffer finalize when cancel insert into function
 2022-08-04 13:02:08 +02:00
kssenii
d462782d1a Fix checks 2022-08-02 14:27:45 +02:00
Alexander Tokmakov
e5c47cb26f
Update run.sh 2022-08-02 12:10:53 +03:00
Alexander Tokmakov
ecf7ce1f74 Merge branch 'master' into zookeeper_client_fault_injection 2022-08-01 20:49:01 +02:00
Alexander Tokmakov
3cc20f05ba
Update run.sh 2022-08-01 20:47:14 +03:00
kssenii
e5f4a619ed Merge master 2022-07-31 20:24:40 +03:00
avogar
573d9c1b57 Fix stress test 2022-07-29 18:30:40 +00:00
Mikhail f. Shiryaev
c6b251d0cb
Revert "Merge pull request #38106 from ClickHouse/tsan_less_flaky_tests" 
This reverts commit 0ae0cc54aa, reversing
changes made to 1d7cf28cab.
 2022-07-22 12:14:23 +02:00
Dmitry Novik
a09422de7b
Merge pull request #38642 from ClickHouse/group-by-use-nulls 
Add group_by_use_nulls setting
 2022-07-20 14:42:42 +02:00
Alexander Tokmakov
840ace663b
Update run.sh 2022-07-20 13:19:53 +03:00
Dmitry Novik
50989bdb68
Merge branch 'master' into group-by-use-nulls 2022-07-19 14:58:01 +02:00
kssenii
8ad26b3111 Merge master 2022-07-16 22:50:03 +02:00
Alexander Tokmakov
b770f9903d
Update run.sh 2022-07-15 20:03:00 +03:00
Alexander Tokmakov
5c24cd979b
Merge pull request #39256 from ClickHouse/bc_check_fix_stacks_in_gdb 
Fix stacktraces in gdb in BC check
 2022-07-15 17:54:18 +03:00
Alexander Tokmakov
77487e996c fix stacktraces in gdb in bc check 2022-07-15 12:23:28 +02:00
kssenii
15a2d60f23 Fix checks 2022-07-14 22:04:39 +02:00
Alexander Tokmakov
f8e23c1deb
Update run.sh 2022-07-14 17:29:08 +03:00
Alexander Tokmakov
991493e6fc
Update run.sh 2022-07-13 15:22:36 +03:00
Dmitry Novik
5f65b45269
Merge branch 'master' into group-by-use-nulls 2022-07-12 22:36:04 +02:00
Alexander Tokmakov
948f15eff3
Update run.sh 2022-07-12 14:01:53 +03:00
Alexander Tokmakov
2b886ddc2e
Update run.sh 2022-07-08 16:47:54 +03:00
Alexander Tokmakov
57b8a6a542
Update run.sh 2022-07-08 13:53:35 +03:00
Dmitry Novik
0b18b4ed43 Use group_by_use_nulls in tests more often 2022-07-07 18:58:46 +00:00
Dmitry Novik
d1df66687b
Merge branch 'master' into group-by-use-nulls 2022-07-07 20:54:38 +02:00
Vladimir C
db838f1343
Merge pull request #35796 from vdimir/full-sorting-merge-join 2022-07-07 19:16:49 +02:00
Kruglov Pavel
8ffd6cd22c
Merge pull request #38717 from Avogar/improve-bc-check 
Try to improve backward compatibility check
 2022-07-07 18:52:34 +02:00
vdimir
b0c0ec370d
Add join_algorithm='full_sorting_merge' to stress tests 2022-07-06 14:27:34 +00:00
Kruglov Pavel
a3a94a1c09
Avoid false positive red sanitizer asserts check in stress test 2022-07-06 14:32:17 +02:00
Kruglov Pavel
45ac74617b
Fix 2022-07-04 20:38:21 +02:00
Dmitry Novik
614ce26ee7 Add group_be_use_nulls setting to stress tests 2022-07-04 18:09:48 +00:00
Kruglov Pavel
65dd81586b
Update run.sh 2022-07-04 17:30:22 +02:00
avogar
17da24b584 Try to improve backward compatibility check 2022-07-04 13:02:22 +00:00
Azat Khuzhin
8e9c1eaa73 tests/stress: redirect gdb output on stop hang to gdb.log 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 14:46:17 +03:00
Alexander Tokmakov
b238ed4093 fix stress tests 2022-07-01 17:19:00 +02:00
Alexander Tokmakov
3ff26939fc
Update run.sh 2022-06-29 15:46:40 +03:00
Alexander Tokmakov
428628238f Merge branch 'master' into deprecate_ordinary_database 2022-06-28 15:40:33 +02:00
Kruglov Pavel
25100ad5e5
Merge pull request #38520 from ClickHouse/Avogar-patch-1 
Fix application errors grep in stress test
 2022-06-28 14:26:22 +02:00
Alexander Tokmakov
411512ed6f
Update run.sh 2022-06-28 14:28:06 +03:00
Kruglov Pavel
f7419e0703
Fix application errors grep in stress test 2022-06-28 13:12:21 +02:00
Alexander Tokmakov
dbf2763788 automatically convert system database to Atomic 2022-06-23 21:38:43 +02:00
Alexander Tokmakov
b5a977ad54
Merge pull request #38213 from azat/stress-thread-fuzzer-v2 
tests/stress: fix TSan detection
 2022-06-20 16:46:31 +03:00
Azat Khuzhin
dce71bbe06 tests/stress: fix TSan detection 
+ test -f package_folder/clickhouse-client_22.7.1.1+tsan_all.deb package_folder/clickhouse-common-static-dbg_22.7.1.1+tsan_amd64.deb package_folder/clickhouse-common-static_22.7.1.1+tsan_amd64.deb package_folder/clickhouse-keeper_22.7.1.1+tsan_amd64.deb package_folder/clickhouse-server_22.7.1.1+tsan_all.deb
    /run.sh: line 13: test: too many arguments

Follow-up for: #38207 (cc @tavplubix)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-20 08:10:03 +03:00
Azat Khuzhin
e307a728ca tests: add no-stress tag 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-20 07:48:10 +03:00
Azat Khuzhin
e79c24deda docker/test/stress/stress: apply black 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-20 07:48:10 +03:00
Azat Khuzhin
d154288fc6 tests/stress: fix TSan detection (enables thread fuzzer for non-TSan builds) 
Follow-up for: #38106 (cc @tavplubix @alesapin)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-19 19:27:25 +03:00
alesapin
0ae0cc54aa
Merge pull request #38106 from ClickHouse/tsan_less_flaky_tests 
Some fixes for tests with tsan
 2022-06-18 14:22:43 +02:00
Alexander Tokmakov
3e8aa2dc47
Update run.sh 2022-06-16 13:24:21 +03:00
Alexander Tokmakov
9cac78b498 make tests with tsan less flaky 2022-06-15 19:54:46 +02:00
Mikhail f. Shiryaev
4d45839f17
Generalize setup_minio.sh 
- Cleanup apt garbage in the container
- Download the same binary in docker and setup_minio.sh
- Use binary and not rpm for minio
- Sort packages in Dockerfile's
 2022-06-10 15:04:45 +02:00
Kruglov Pavel
83418ebba2
Merge pull request #37712 from vdimir/duplicate_download_previous_release 
Get rid of duplicate download_previous_release
 2022-06-07 12:51:34 +02:00
Kruglov Pavel
196863ade5
Ignore PART_IS_TEMPORARILY_LOCKED error in BC check 2022-06-02 13:33:27 +02:00
Vladimir C
2622791b9e
upd 2022-06-01 16:17:17 +02:00
mergify[bot]
3866c27850
Merge branch 'master' into stress_s3 2022-06-01 12:02:33 +00:00
vdimir
4d336d0228
Get rid of duplicate download_previous_release 2022-05-31 17:50:29 +00:00
vdimir
6d2fab49f4
Add join_algorithm='parallel_hash' to stress test 2022-05-23 10:18:22 +02:00
Alexander Tokmakov
425b31ba1b
Update stress 2022-05-12 15:04:12 +03:00
Alexander Tokmakov
c23bc01196 Merge branch 'master' into zookeeper_client_fault_injection 2022-05-11 17:20:42 +02:00
alesapin
af99183950 Disable thread fuzzer after server restart 2022-05-10 15:16:21 +02:00
mergify[bot]
5c742cf2d5
Merge branch 'master' into stress_s3 2022-05-10 12:26:26 +00:00
Alexander Tokmakov
9346c9363a
Merge pull request #37052 from ClickHouse/fix_server_termination_stress_tests 
Print stacks if we cannot terminate server in stress tests
 2022-05-10 13:12:44 +03:00
Alexander Tokmakov
17c954bea7 fix shellcheck 2022-05-10 11:48:55 +02:00
Alexander Tokmakov
92d423c478 print stacks if wa cannot terminate server in stress tests 2022-05-09 19:43:51 +02:00
Alexander Tokmakov
b647f8f9bf
Update run.sh 2022-05-09 17:48:10 +03:00
Alexey Milovidov
419687ac50
Update run.sh 2022-05-09 10:17:49 +02:00
Mikhail f. Shiryaev
b14dca2fac
Improve stress tests report a little bit 2022-05-09 10:17:40 +02:00
alesapin
85dda249b8 Don't download stateful table 2022-05-02 00:11:05 +02:00
alesapin
51345fa25b Trying stress on top of S3 2022-05-01 19:07:02 +02:00
Kruglov Pavel
4e2b1cd981
Update run.sh 2022-04-20 16:48:54 +02:00
Kruglov Pavel
cd77f927b6
Fix putting logs into test_output 2022-04-20 15:06:09 +02:00