thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-30 03:22:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
120,041 Commits 9,189 Branches 1,782 Tags 2.2 GiB
cf01800a63
Commit Graph

432 Commits

Author SHA1 Message Date
avogar
64df6ca520 Split stress test and backward compatibility check 2023-01-03 19:22:31 +00:00
Alexey Milovidov
6e946cc757 Less OOM in stress test 2023-01-01 23:22:26 +01:00
Alexey Milovidov
0b324ab836 Modern tools 2023-01-01 20:53:06 +01:00
Alexey Milovidov
e2c00cefb8
Merge pull request #44810 from ClickHouse/stress-test-is-a-trash 
Fix total trash in stress test
 2023-01-01 01:09:02 +03:00
Alexey Milovidov
7434a2cbbb
Merge pull request #44807 from ClickHouse/fix-typo-in-stress-test 
Retry packages download if GitHub returned HTTP 500.
 2023-01-01 01:08:48 +03:00
Alexey Milovidov
a7ccc618c6 Fix total trash in stress test 2022-12-31 15:40:06 +01:00
Alexey Milovidov
78ed8a6d69 GitHub is unreliable; copy-paste code from Stackoverflow to solve it 2022-12-31 14:45:44 +01:00
Alexey Milovidov
d4a35f7682 Fix typo in stress test 2022-12-31 14:31:40 +01:00
alesapin
acad2fac63 Remove database ordinary from stress test 2022-12-30 18:48:25 +01:00
Alexey Milovidov
e54bf36a3e
Merge pull request #44722 from ClickHouse/rename_file 
Rename log in stress test
 2022-12-30 01:44:48 +03:00
Alexey Milovidov
f423dced5a Debug deadlock in stress test 2022-12-29 20:06:09 +01:00
alesapin
9eeb7db2f3 Rename log in stress test 2022-12-29 19:33:32 +01:00
alesapin
3eda778707 Better 2022-12-28 14:29:27 +01:00
Alexey Milovidov
135a32c606
Merge branch 'master' into faster-startup-bc-check 2022-12-27 23:17:24 +03:00
Alexey Milovidov
bfe895975e
Merge pull request #44575 from ClickHouse/minimal-changes 
Non-significant changes
 2022-12-27 15:34:27 +03:00
alesapin
338c554525 Push 2022-12-27 13:00:14 +01:00
alesapin
e4e6cccbf8 Remove garbage 2022-12-27 12:59:09 +01:00
Alexey Milovidov
a0753cbca2 Disable analyzer in fuzz and stress tests 2022-12-26 19:04:10 +01:00
Alexey Milovidov
91c1f9de8f Faster server startup after stress test 2022-12-26 18:13:50 +01:00
alesapin
211c6062bc Fix 2022-12-25 19:37:02 +01:00
Alexey Milovidov
f22fc35900 Minimal changes 2022-12-25 14:18:05 +01:00
alesapin
bd19c6b699 Make BC check optional (if env var set) 2022-12-24 17:46:22 +01:00
Alexander Tokmakov
f2b177880f print stats in tests 2022-12-23 16:04:29 +01:00
Alexander Tokmakov
3f336190a7
Update run.sh 2022-12-14 14:21:21 +03:00
Alexander Tokmakov
f68a407636
Update docker/test/stress/run.sh 
Co-authored-by: Alexander Gololobov <440544+davenger@users.noreply.github.com>
 2022-12-13 18:58:02 +03:00
Alexander Tokmakov
49f3e8e19b
Update run.sh 2022-12-13 18:47:13 +03:00
Alexander Tokmakov
371e85d854
Merge pull request #44157 from ClickHouse/fix_session_expired_on_startup 
Ignore "session expired" errors after BC check
 2022-12-13 15:10:36 +03:00
Alexander Tokmakov
469406c494 ignore session expired errors after BC check 2022-12-12 14:02:14 +01:00
Sema Checherinda
9a3aecd4c1 set max-tries bigger when backward compatibility check runs, old version still drop parts at shutdown 2022-12-05 16:05:14 +01:00
Sema Checherinda
dc52a40c04 decrease stop max-tries back to default, do not clear old parts at shutdown 2022-12-05 16:05:14 +01:00
Alexander Tokmakov
ac578efc05
Update run.sh 2022-11-29 20:16:59 +03:00
Azat Khuzhin
ca16f5136f stress: use --max-tries only since 22.12 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 09:40:39 +01:00
Azat Khuzhin
2c36b52050 stress: increase timeout for server waiting after TERM 
Greater timeout after TERM may reduce about of KILL, let's try.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 09:40:39 +01:00
Alexey Milovidov
69d1be50bc
Merge pull request #43446 from ClickHouse/fix-typo-3 
Fix typo
 2022-11-25 15:22:28 +01:00
alesapin
f64f0461fe
Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key 
Enable compress marks and primary key for CI
 2022-11-24 12:24:04 +01:00
alesapin
41d3825770 Merge branch 'Enable_compress_marks_and_primary_key' of https://github.com/ClickHouse/ClickHouse into Enable_compress_marks_and_primary_key 2022-11-22 19:12:00 +01:00
alesapin
31142808de Disable compressed marks and index in stress tests 2022-11-22 19:10:51 +01:00
Alexey Milovidov
b7e4547f53
Merge branch 'master' into fix-typo-3 2022-11-22 18:30:06 +01:00
Alexey Milovidov
56f8c0b84b Fix typo 2022-11-21 15:10:15 +01:00
Alexey Milovidov
17e2189199 Fix backward compatibility check 2022-11-21 12:51:51 +01:00
alesapin
2dbf07f58a Splitting checks 2022-11-18 17:38:17 +01:00
Igor Nikonov
b12ebab65c Enable keeper fault injection and retries for insert queries in 
funcntional tests
 2022-11-11 18:07:15 +00:00
Alexander Tokmakov
acbad99b24 remove abandonable_lock part 2 2022-11-02 14:28:49 +01:00
Alexey Milovidov
4f582e19a8
Merge pull request #42479 from ClickHouse/Avogar-patch-1 
Add error "Destination table is myself" to exception list in BC check
 2022-10-25 04:55:12 +02:00
Alexey Milovidov
c4fd827442
Merge pull request #42483 from ClickHouse/Avogar-patch-2 
Get current clickhouse version without sending query in BC check
 2022-10-25 04:54:24 +02:00
Azat Khuzhin
f0aed68893 ci/stress: fix memory limits overrides for hung check 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-22 20:53:18 +02:00
Alexander Tokmakov
f958c155f3
Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-19 17:25:24 +03:00
Antonio Andelic
10b176fe19
Merge pull request #41854 from azat/tests/stress/fix-dbg-sym-bc 
tests: fix debug symbols (and possible crashes) for backward compatiblity check
 2022-10-19 16:18:41 +02:00
Kruglov Pavel
2ca0d37e83
Get current clickhouse version without sending query in BC check 2022-10-19 13:25:03 +02:00
Kruglov Pavel
632965212f
Add error "Destination table is myself" to exception list in BC check 2022-10-19 12:52:32 +02:00
Alexander Tokmakov
4211605093 disable zk fault injections in bc check 2022-10-03 13:13:29 +02:00
Alexander Tokmakov
8741785a72 temporarily disable ThreadFuzzer with tsan 2022-09-29 18:08:35 +02:00
Azat Khuzhin
9b20468198 tests: fix debug symbols (and possible crashes) for backward compatiblity check 
stress running previous version of the server w/o correct debug symbols
right now, since nobody restore clickhouse.debug file, and this can lead
to the following issues, like in [1]:
- incorrect stack traces
- gdb crashes
- clickhouse crashes, due to non-robust internal DWARF parser (probably)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/41730/8cc53a48ae99a765085f44a75fa49314d1f1cc7d/stress_test__ubsan_.html

Right now I decided not to rework the script to make it less error
prone, but simply fix the problem.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-27 22:50:28 +02:00
alesapin
69f3a66538 Keep the most important log in stress tests 2022-09-27 11:16:10 +02:00
Antonio Andelic
97cf045203
Merge pull request #41721 from ClickHouse/collect-correctly-logs-in-stress-test 
Collect logs in Stress test using clickhouse-local
 2022-09-27 08:43:44 +02:00
Antonio Andelic
eb78761a7e Collect necessary 2022-09-26 16:30:01 +00:00
Antonio Andelic
6f4a636e8f Remove wildcard 2022-09-26 11:21:53 +00:00
Antonio Andelic
8fde8b2c56 Try with multiple calls 2022-09-26 11:03:24 +00:00
Antonio Andelic
c60d9db687
Merge branch 'master' into ignore-attach-thread-keeper-errors 2022-09-26 08:38:48 +02:00
Antonio Andelic
5ff1bcd553
Merge branch 'master' into collect-correctly-logs-in-stress-test 2022-09-26 08:38:38 +02:00
alesapin
06e0f554d8 Fix fetch to local disk 2022-09-23 16:46:53 +02:00
Antonio Andelic
1d93c56d1a Collect logs using clickhouse-local 2022-09-23 10:54:16 +00:00
Antonio Andelic
a17a3e1de1 Ignore Keeper hardware errors 2022-09-23 08:23:57 +00:00
kssenii
46f74aaba9 Update stress/run.sh 2022-09-12 20:10:35 +02:00
Alexander Tokmakov
e77b9e4d0c
Merge pull request #40775 from azat/ci/core-dumps-rework 
Rework core collecting on CI (eliminate gcore usage)
 2022-09-09 20:20:10 +03:00
Alexey Milovidov
7776512b04
Merge pull request #41002 from azat/ci/fix-oom-check 
ci/stress: clear dmesg before run to fix "OOM in dmesg" check
 2022-09-06 06:41:36 +03:00
Alexander Tokmakov
b264be3c63
Merge branch 'master' into zookeeper_client_fault_injection 2022-09-05 22:13:09 +03:00
Azat Khuzhin
2724b67537 ci/stress: clear dmesg before run to fix "OOM in dmesg" check 
CI: https://s3.amazonaws.com/clickhouse-test-reports/40772/afa137ae2b6108e72c2d6e43556a04548afa2ea9/stress_test__ubsan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-05 15:51:36 +02:00
Azat Khuzhin
25e3bebd9d Rework core collecting on CI (eliminate gcore usage) 
gcore is a gdb command, that internally uses gdb to dump the core.

However with proper configuration of limits (core_dump.size_limit) it
should not be required, althought some issues is possible:
- non standard kernel.core_pattern
- sanitizers

So yes, gcore is more "universal" (you don't need to configure any
`kernel_pattern`), but it is ad-hoc, and it has drawbacks -
**it does not work when gdb fails**. For example gdb may fail with
`Dwarf Error: DW_FORM_strx1 found in non-DWO CU` in case of DWARF-5 [1].

  [1]: https://github.com/ClickHouse/ClickHouse/pull/40772#issuecomment-1236331323.

Let's try to switch to more native way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-04 22:07:16 +02:00
Alexander Tokmakov
8bdb589c2b Merge branch 'master' into zookeeper_client_fault_injection 2022-08-29 13:34:57 +02:00
alesapin
133ca01447 Merge branch 'master' into stress_s3 2022-08-29 11:25:28 +02:00
Azat Khuzhin
ebc61a36e0 tests/stress: improve OOM detection (add separate check by dmesg) 
Right now if you will look at the OOM errors:
- OOM killer (or signal 9) in clickhouse-server.log
- Backward compatibility check: OOM messages in clickhouse-server.log

Most of them are not real, but just clickhouse server got KILLed by
clickhouse stop, #40678 may imporove the situation, but to definitely
sure that there was OOM let's look at dmesg.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 12:46:58 +02:00
Azat Khuzhin
3b519c5d44 tests/stress: capture stacktrace of server hungs if pid was removed already 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-26 22:06:32 +03:00
alesapin
704c4b2c5b Stop thread fuzzer on shutdown 2022-08-26 11:54:54 +02:00
alesapin
3ff6489fae Merge branch 'master' into stress_s3 2022-08-25 13:14:58 +02:00
alesapin
ad692f732a Merge branch 'master' into stress_s3 2022-08-25 13:13:30 +02:00
alesapin
35f9815b8e Fix backward comp check 2022-08-24 14:43:02 +02:00
Azat Khuzhin
50bddc43dc tests/stress: ignore NETLINK_ERROR from checkPermissionsImpl 
Since now with --privileged it has CAP_SYS_ADMIN and tries to
communicate via netlink.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-19 14:07:06 +02:00
Alexander Tokmakov
3d253ec51b
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-18 21:23:50 +03:00
alesapin
922818bbd9 Merge branch 'stress_s3' of github.com:ClickHouse/ClickHouse into stress_s3 2022-08-18 14:46:50 +02:00
alesapin
932ea146f5 Merge branch 'master' into stress_s3 2022-08-18 13:14:47 +02:00
alesapin
86b1e33eed Disable cache on writes 2022-08-17 19:00:53 +02:00
alesapin
600d22851f Grep dangerous S3 errors 2022-08-17 12:43:11 +02:00
alesapin
0433b801d2 Configure properly 2022-08-17 12:27:15 +02:00
Alexander Tokmakov
ae000e9125
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-17 12:48:54 +03:00
alesapin
1ec6627a70 Fix tables creation 2022-08-16 18:28:17 +02:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Alexander Tokmakov
589c3408d2
Merge pull request #40234 from ClickHouse/better_message_on_restore_covered 
Better error message when restoring covered parts
 2022-08-15 22:01:48 +03:00
alesapin
243bd492fa Trying to fix it 2022-08-15 20:55:11 +02:00
alesapin
96722a13bb Merge branch 'master' into stress_s3 2022-08-15 20:20:31 +02:00
Alexander Tokmakov
edaff70010 better error message when restoring covered parts 2022-08-15 13:53:14 +02:00
Alexander Tokmakov
467ef7bbc2
Update run.sh 2022-08-12 14:30:18 +03:00
Alexander Tokmakov
b9d18182f2 fix 2022-08-11 15:27:26 +02:00
kssenii
5c3227ba56 Merge master 2022-08-10 12:00:34 +02:00
kssenii
0dda03c94b Fix checks 2022-08-10 00:06:58 +02:00
Azat Khuzhin
3772415588 tests/stress: add dmesg output (to see OOM details) 
max_server_memory_usage already set to 75%, so OOM should not happens,
the reason is that because RSS does not match with memory tracker
statistics:

    2022.08.05 12:36:57.869896 [ 82524 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 64.69 GiB, peak 65.26 GiB, will set to 62.80 GiB (RSS), difference: -1.89 GiB
    ...
    2022.08.05 12:37:00.213440 [ 82334 ] {} <Error> void DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(DB::TaskRuntimeDataPtr) [Queue = DB::MergeMutateRuntimeQueue]: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 64.68 GiB (attempt to allocate chunk of 1298794 bytes), maximum: 51.44 GiB. OvercommitTracker decision: Memory overcommit isn't used. Waiting time or orvercommit denominator are set to zero.. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-06 12:34:38 +03:00
kssenii
7a9b0bc47f Merge master 2022-08-05 01:48:52 +02:00
Kruglov Pavel
235649cb98
Merge pull request #39458 from Avogar/fix-cancel-insert-into-function 
Fix WriteBuffer finalize when cancel insert into function
 2022-08-04 13:02:08 +02:00
kssenii
d462782d1a Fix checks 2022-08-02 14:27:45 +02:00
Alexander Tokmakov
e5c47cb26f
Update run.sh 2022-08-02 12:10:53 +03:00