thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-21 15:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,839 Commits 9,143 Branches 1,778 Tags 2.1 GiB
301ac5dab7
Commit Graph

149839 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Azat Khuzhin
301ac5dab7 Fix possible data-race StorageKafka with statistics_interval_ms>0 
The problem here is that ignorelist did not work by some reason, if I
will look at the ignored functions it should not contain any TSan
interseption code, while it does:

    $ lldb-13 clickhouse
    (lldb) target create "clickhouse"
    disas -n rd_avg_rollover
    Current executable set to '/home/azat/ch/tmp/tsan-test/clickhouse' (x86_64).
    (lldb) disas -n rd_avg_rollover
    clickhouse`rd_kafka_stats_emit_avg:
    clickhouse[0x1cbf84a7] <+39>:  leaq   0x30(%r15), %r12
    clickhouse[0x1cbf84ab] <+43>:  movq   %r12, %rdi
    clickhouse[0x1cbf84ae] <+46>:  callq  0x1ccdad40                ; rdk_thread_mutex_lock at tinycthread.c:111
    clickhouse[0x1cbf84b3] <+51>:  leaq   0x58(%r15), %rdi
    clickhouse[0x1cbf84b7] <+55>:  callq  0x71b5390                 ; __tsan_read4
    clickhouse[0x1cbf84bc] <+60>:  cmpl   $0x0, 0x58(%r15)
    clickhouse[0x1cbf84c1] <+65>:  je     0x1cbf8595                ; <+277> [inlined] rd_avg_rollover + 238 at rdavg.h
    clickhouse[0x1cbf84c7] <+71>:  leaq   -0xc8(%rbp), %rdi
    clickhouse[0x1cbf84ce] <+78>:  xorl   %esi, %esi
    clickhouse[0x1cbf84d0] <+80>:  callq  0x1ccdac80                ; rdk_thread_mutex_init at tinycthread.c:62
    clickhouse[0x1cbf84d5] <+85>:  leaq   0x5c(%r15), %rdi
    clickhouse[0x1cbf84d9] <+89>:  callq  0x71b5390                 ; __tsan_read4

    (lldb) disas -n rd_avg_calc
    clickhouse`rd_kafka_broker_ops_io_serve:
    clickhouse[0x1cbdf086] <+1990>: leaq   0x5a4(%rbx), %rdi
    clickhouse[0x1cbdf08d] <+1997>: callq  0x71b5390                 ; __tsan_read4
    clickhouse[0x1cbdf092] <+2002>: cmpl   $0x0, 0x5a4(%rbx)
    clickhouse[0x1cbdf099] <+2009>: je     0x1cbdf12b                ; <+2155> [inlined] rd_kafka_broker_timeout_scan + 719 at rdkafka_broker.c

I guess the reason is that they had been inlined

So now rd_avg_calc() guarded with a mutex.

Refs: https://github.com/ClickHouse/librdkafka/pull/11
Fixes: https://github.com/ClickHouse/ClickHouse/issues/60939
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-08 07:43:29 +02:00
Yakov Olkhovskiy
3319a5e6f8
Merge pull request #67341 from ClickHouse/fix-race-tcphandler 
Fix: data race in TCPHandler on socket timeouts settings
 2024-08-08 02:50:24 +00:00
Alexey Milovidov
e0975213bc
Merge pull request #66821 from ClickHouse/randomize-trace_profile_events 
Randomize `trace_profile_events`
 2024-08-08 02:05:13 +00:00
Alexey Milovidov
d60cb9d3a6
Merge pull request #67960 from azat/tests/03002_part_log_rmt_fetch__error 
tests: fix 03002_part_log_rmt_fetch_merge_error flakiness
 2024-08-08 01:43:23 +00:00
Alexey Milovidov
e07069c500
Merge pull request #68002 from Algunenano/no-parallel-5 
Remove some no-parallel tags from tests (Part 5)
 2024-08-08 01:42:58 +00:00
Alexey Milovidov
31dd584df3
Merge pull request #68011 from ClickHouse/fix-test-02845_threads_count_in_distributed_queries 
Fix test `02845_threads_count_in_distributed_queries`
 2024-08-08 01:42:45 +00:00
Alexey Milovidov
ae6139e66c
Merge pull request #68007 from ClickHouse/qoega-patch-7 
Follow up for #67843
 2024-08-08 01:38:54 +00:00
Nikita Mikhaylov
ef31d9301b
Merge pull request #68014 from ClickHouse/revert-67904-bump-rocksdb3 
Revert "Bump rocksdb from v8.10 to v9.4 + enable jemalloc and liburing"
 2024-08-08 00:07:49 +00:00
Nikita Mikhaylov
ff8ce505d7
Revert "Bump rocksdb from v8.10 to v9.4 + enable jemalloc and liburing" 2024-08-08 00:20:16 +02:00
Nikita Fomichev
755b06b455
Merge pull request #67793 from ClickHouse/flaky-test_reload_after_fail_by_timer 
Integration tests: fix flaky `test_dictionaries_update_and_reload::test_reload_after_fail_by_timer`
 2024-08-07 21:36:54 +00:00
Azat Khuzhin
85427030aa tests: fix 03002_part_log_rmt_fetch_merge_error flakiness 
CI found [1]:

    --- /usr/share/clickhouse-test/queries/0_stateless/03002_part_log_rmt_fetch_merge_error.reference	2024-08-07 05:09:42.058643403 +0200
    +++ /tmp/clickhouse-test/0_stateless/03002_part_log_rmt_fetch_merge_error.stdout	2024-08-07 05:54:45.514083455 +0200
    @@ -6,5 +6,7 @@
     after
     rmt_master	NewPart	0	1
     rmt_master	MergeParts	0	1
    +rmt_master	RemovePart	0	1
     rmt_slave	MergeParts	1	0
     rmt_slave	DownloadPart	0	2
    +rmt_slave	RemovePart	0	1

    MergeTree settings used in test: --ratio_of_defaults_for_sparse_serialization 1.0 --prefer_fetch_merged_part_size_threshold 3517855074 --vertical_merge_algorithm_min_rows_to_activate 1000000 --vertical_merge_algorithm_min_columns_to_activate 100 --allow_vertical_merges_from_compact_to_wide_parts 0 --min_merge_bytes_to_use_direct_io 10737418240 --index_granularity_bytes 7659983 --merge_max_block_size 17667 --index_granularity 48465 --min_bytes_for_wide_part 1073741824 --marks_compress_block_size 58048 --primary_key_compress_block_size 18342 --replace_long_file_name_to_hash 0 --max_file_name_length 36 --min_bytes_for_full_part_storage 536870912 --compact_parts_max_bytes_to_buffer 148846831 --compact_parts_max_granules_to_buffer 140 --compact_parts_merge_max_bytes_to_prefetch_part 4513530 --cache_populated_by_fetch 1 --concurrent_part_removal_threshold 8 --old_parts_lifetime 10

The reason is old_parts_lifetime=10

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/67511/881d57a7644057b586e4cdb95ebb8785d912d4c5/stateless_tests__msan__%5B3_4%5D.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-07 23:06:14 +02:00
Alexey Milovidov
1fcd8320cf Merge branch 'fix-test-02845_threads_count_in_distributed_queries' into randomize-trace_profile_events 2024-08-07 22:24:39 +02:00
Alexey Milovidov
92d5d5e8dc Merge branch 'master' into randomize-trace_profile_events 2024-08-07 22:24:20 +02:00
Alexey Milovidov
9fdc746c44 Fix test 02845_threads_count_in_distributed_queries 2024-08-07 22:23:21 +02:00
Alexey Milovidov
8b3e85dd8d
Merge pull request #67785 from arenadata/ADQM-FixedTestRaceCondition 
Fixed session_log related tests race condition on logout
 2024-08-07 20:01:48 +00:00
Ilya Yatsishin
7341dcefd4
Follow up for #67843 2024-08-07 20:36:23 +02:00
Nikita Mikhaylov
722fb1f512
Merge pull request #68005 from den-crane/patch-72 
Docs: Fix heading level
 2024-08-07 18:21:27 +00:00
Yakov Olkhovskiy
7706bc853d Merge branch 'master' into fix-race-tcphandler 2024-08-07 18:13:20 +00:00
Ilya Yatsishin
ac4bed2f42
Merge pull request #67843 from qoega/improve-integration-tests-3 
Improve integration tests 3
 2024-08-07 18:06:37 +00:00
Yarik Briukhovetskyi
e3abcc7ae9
Merge pull request #68004 from ClickHouse/qoega-patch-7 
Minor fixes in tables.md
 2024-08-07 17:43:35 +00:00
Kseniia Sumarokova
315fd5496a
Merge pull request #65386 from skyoct/feat-s3-field 
Feat add _etag for object storage
 2024-08-07 17:35:43 +00:00
Denny Crane
3b48a1a92c
Update http.md 2024-08-07 14:01:40 -03:00
Nikita Mikhaylov
9d95d31320
Merge pull request #67950 from ClickHouse/flaky-flaky-system-processes 
Fix flaky `02558_system_processes_elapsed `
 2024-08-07 16:58:39 +00:00
Ilya Yatsishin
4fb1febe48
Update table.md 2024-08-07 18:51:24 +02:00
Alexander Tokmakov
a523125411
Merge pull request #66410 from ClickHouse/reject_poco 
Add logs and metrics about rejected connections in Poco
 2024-08-07 16:40:59 +00:00
vdimir
5e9c3c222b
Merge pull request #67883 from canhld94/optimize_join_engine 
Join engine support OPTIMIZE query
 2024-08-07 16:24:50 +00:00
Raúl Marín
35f827e55a Make 03147_table_function_loop parallelizable 2024-08-07 18:19:15 +02:00
Raúl Marín
a109e78776 03148_async_queries_in_query_log_errors is parallelizable 2024-08-07 18:17:08 +02:00
Raúl Marín
58b15c71d9 03156_default_multiquery_split is parallelizable 2024-08-07 18:15:19 +02:00
Raúl Marín
c7b94ca43a 03164_adapting_parquet_reader_output_size is parallelizable 2024-08-07 18:11:51 +02:00
Raúl Marín
09964cee56 Fix 03168_query_log_privileges_not_empty 2024-08-07 18:09:06 +02:00
Raúl Marín
db9ba01886 Leave some notes 2024-08-07 17:58:54 +02:00
Raúl Marín
ff2e8b65bd 03171_hashed_dictionary_short_circuit_bug_fix is parallelizable 2024-08-07 17:58:44 +02:00
Raúl Marín
bf111b65fd 03201_avro_negative_block_size_arrays is parallelizable 2024-08-07 17:48:46 +02:00
Alexey Milovidov
9c2b51e3d6
Merge pull request #65860 from ClickHouse/atomic-database-in-clickhouse-local 
Use `Atomic` database by default in `clickhouse-local`
 2024-08-07 15:18:54 +00:00
Robert Schulze
681014d99d
Merge pull request #67904 from rschu1ze/bump-rocksdb3 
Bump rocksdb from v8.10 to v9.4 + enable jemalloc and liburing
 2024-08-07 14:34:48 +00:00
Max K.
ebc4ef393b
Merge pull request #67986 from ClickHouse/revert-67934-ci_strict_job_timeout 
Revert "CI: Strict job timeout 1.5h for tests, 2h for builds"
 2024-08-07 14:33:16 +00:00
max-vostrikov
7dc55faf73
Merge pull request #67979 from ClickHouse/docs-for-lightweight_mutation_projection_mode 
docs for lightweight_mutation_projection_mode option
 2024-08-07 14:31:02 +00:00
kssenii
364e973ef7 Ping CI 2024-08-07 16:23:47 +02:00
Max K.
b8fec3bce4
Merge pull request #67923 from ClickHouse/ci_buddy_with_channel_dispatcher 
CI: CiBuddy with channel dispatcher
 2024-08-07 14:21:02 +00:00
Dmitry Novik
27c54d7cce
Merge pull request #67985 from ClickHouse/fix-memory-overcommit-docs 
Fix documentation for memory overcommit
 2024-08-07 14:19:27 +00:00
Nikita Fomichev
ed94f94657
Merge pull request #67970 from ClickHouse/fix_docs_for_groupconcat_function 
fix docks for groupConcat function
 2024-08-07 14:17:55 +00:00
Max K.
9a812fc535
Revert "CI: Strict job timeout 1.5h for tests, 2h for builds" 2024-08-07 15:55:03 +02:00
Robert Schulze
de9cdedeb0
Merge pull request #67961 from rschu1ze/wreck-a-test 
Fix timeout of 02310_clickhouse_local_INSERT_progress_profile_events
 2024-08-07 13:35:47 +00:00
Dmitry Novik
ecba21bfe7
Fix documentation for memory overcommit 2024-08-07 15:23:19 +02:00
Kruglov Pavel
e809dbed60
Merge pull request #67171 from compasses/fix-memory-leak-nullkey-distinct 
fix memory leak when exception happend during count distinct for null key
 2024-08-07 12:47:52 +00:00
Kruglov Pavel
81b0caf85c
Merge pull request #67911 from Avogar/validate-data-types-in-alter 
Validate data types in ALTER ADD/MODIFY COLUMN
 2024-08-07 12:41:10 +00:00
Nikita Taranov
04d988d31b
Merge pull request #67917 from ClickHouse/follow_up_67235 
Follow up #67235
 2024-08-07 12:13:29 +00:00
maxvostrikov
37a6bd25f6 squash! docs for lightweight_mutation_projection_mode option starting with 24.7 we can lightweight delete in tables with projections. Fixed docs stating it is not possible 2024-08-07 14:06:02 +02:00
maxvostrikov
c39cdcffaf docs for lightweight_mutation_projection_mode option 
starting with 24.7 we can lightweight delete in tables with projections. Fixed docs stating it is not possible
 2024-08-07 14:03:53 +02:00