Commit Graph

121021 Commits

Author SHA1 Message Date
Yarik Briukhovetskyi
11016d4c5f
Revert "Rewrite uniq to count" 2023-07-25 16:46:50 +02:00
Sanjam Panda
c75b5bc740
Update TwoLevelStringHashTable.h 2023-07-25 20:12:22 +05:30
Azat Khuzhin
85082ad8f8
Fix data-race DatabaseReplicated::startupTables()/canExecuteReplicatedMetadataAlter() (#52490)
CI founds [1]:

    Exception: Sanitizer assert found for instance ==================
    WARNING: ThreadSanitizer: data race (pid=348)
      Write of size 8 at 0x7b58000044a0 by main thread:
        2 DB::DatabaseReplicated::startupTables(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, DB::LoadingStrictnessLevel) build_docker/./src/Databases/DatabaseReplicated.cpp:526:16 (clickhouse+0x1ec45092)
        3 DB::TablesLoader::startupTables() build_docker/./src/Databases/TablesLoader.cpp:87:26 (clickhouse+0x1f9258ab) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        4 DB::loadMetadata(std::__1::shared_ptr<DB::Context>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./src/Interpreters/loadMetadata.cpp:234:12 (clickhouse+0x1fff3834) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        5 DB::Server::main() build_docker/./programs/server/Server.cpp:1615:9 (clickhouse+0x163e7f78) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        6 Poco::Util::Application::run() build_docker/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x257608fe) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        7 DB::Server::run() build_docker/./programs/server/Server.cpp:391:25 (clickhouse+0x163d7d7c) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        8 Poco::Util::ServerApplication::run(int, char**) build_docker/./base/poco/Util/src/ServerApplication.cpp:131:9 (clickhouse+0x25780114) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        9 mainEntryClickHouseServer(int, char**) build_docker/./programs/server/Server.cpp:196:20 (clickhouse+0x163d4c23) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        10 main build_docker/./programs/main.cpp:487:12 (clickhouse+0xdf8c877) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)

      Previous read of size 8 at 0x7b58000044a0 by thread T27 (mutexes: write M0, write M1):
        1 DB::DatabaseReplicated::canExecuteReplicatedMetadataAlter() const build_docker/./src/Databases/DatabaseReplicated.cpp:1303:12 (clickhouse+0x1ec5c5bd)
        2 DB::ReplicatedMergeTreeQueue::shouldExecuteLogEntry() const build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1471:24 (clickhouse+0x2115fb56) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        3 DB::ReplicatedMergeTreeQueue::selectEntryToProcess(DB::MergeTreeDataMergerMutator&, DB::MergeTreeData&) build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1676:13 (clickhouse+0x21163c58) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        4 DB::StorageReplicatedMergeTree::selectQueueEntry() build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3240:26 (clickhouse+0x20823db2) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)
        5 DB::StorageReplicatedMergeTree::scheduleDataProcessingJob(DB::BackgroundJobsAssignee&) build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3304:65 (clickhouse+0x208240fc) (BuildId: 7d4ce55d33d4c3e3df9fd39b304e67e53eb61a63)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/52395/0b258dda4ee618a4d002e2b5246d68bbd2c77c7e/integration_tests__tsan__[5_6].html

Add ddl_worker_initialized flag to avoid this race.

Note, that it should be enough to check this flag only in
canExecuteReplicatedMetadataAlter() since only it can be run in parallel
with ctor before it had been finished.

v0: initialize ddl before startupTables()
v2: ddl_worker_initialized

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-07-25 17:35:01 +03:00
Azat Khuzhin
79d0343bec
tests: fix 01821_join_table_race_long flakiness (#52559)
By grouping multiple queries into one clickhouse-client invocation,
since each execve of the binary can take ~1 second in debug builds.

But this slightly changes the logic, so be aware.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-07-25 17:34:40 +03:00
Sanjam Panda
ee37bdda92
Merge branch 'master' into external_aggregation_fix 2023-07-25 19:57:36 +05:30
Alexander Tokmakov
a6cf769222
Merge pull request #52558 from azat/tests/00719_parallel_ddl_table
tests: fix 00719_parallel_ddl_table flakiness in debug builds
2023-07-25 17:25:24 +03:00
Alexander Tokmakov
7d871dadd7
Merge pull request #52556 from azat/tests/fix-01035_avg_weighted_long
tests: fix 01035_avg_weighted_long flakiness
2023-07-25 17:21:29 +03:00
Anton Popov
2c7c38950d better check for lightweight deletes 2023-07-25 14:21:12 +00:00
Val Doroshchuk
bd09ad6736 MaterializedMySQL: Fix typos in tests 2023-07-25 16:20:24 +02:00
Julian Maicher
b91852de3a fix(docs): Document correct MODIFY COLUMN REMOVE syntax 2023-07-25 16:07:29 +02:00
Vasily Nemkov
6add283933
Merge branch 'master' into fix_insert_into_buffer_table 2023-07-25 16:06:42 +02:00
Anton Popov
073dea6eef
Merge pull request #49160 from CurtizJ/flush-async-insert-queue
Allow to flush asynchronous insert queue
2023-07-25 15:58:20 +02:00
Anton Popov
4531b105ac
Merge branch 'master' into flush-async-insert-queue 2023-07-25 15:48:01 +02:00
flynn
5262221718
Merge branch 'master' into operator 2023-07-25 21:21:19 +08:00
robot-clickhouse
d7de8bf797 Automatic style fix 2023-07-25 13:03:12 +00:00
Alexander Tokmakov
328d0a5269 fix 2023-07-25 14:50:27 +02:00
Yakov Olkhovskiy
e133db7937
Merge pull request #51472 from bigo-sg/substring_index
Add function substring_index, as in spark or mysql
2023-07-25 08:43:13 -04:00
Anton Popov
568afbbec3 added field with refcount to system.remote_data_paths table 2023-07-25 12:33:15 +00:00
Kruglov Pavel
da02f3189d
Merge pull request #50452 from DanRoscigno/update-acid-docs
update for min_chunk_bytes_for_parallel_parsing
2023-07-25 14:15:55 +02:00
Robert Schulze
ed5393ef03
Stabilize tests 2023-07-25 12:09:13 +00:00
Smita Kulkarni
22fec136c1 Create new StorageView with substituted parameters for every SELECT query of a parameterized view 2023-07-25 14:04:55 +02:00
Robert Schulze
1a965b20a3
Merge pull request #52309 from rschu1ze/qc-event-descr
Minor: Update description of events "QueryCacheHits/Misses"
2023-07-25 13:54:06 +02:00
Antonio Andelic
93e5d7f51c Fix flaky 00995_exception_while_insert 2023-07-25 11:42:22 +00:00
Kruglov Pavel
f8c90d5964
Make better 2023-07-25 13:36:57 +02:00
robot-ch-test-poll
8964de808e
Merge pull request #52147 from ucasfl/any_value
Add any_value as a compatibility alias for any
2023-07-25 13:05:49 +02:00
Andrey Zvonov
155b90c780
oops 2023-07-25 13:47:59 +03:00
Nikolai Kochetov
c960b3b533
Merge pull request #52533 from ClickHouse/Avogar-patch-2
Cancel execution in PipelineExecutor in case of exception in graph->updateNode
2023-07-25 12:39:22 +02:00
Michael Kolupaev
5ee71bd643 Work around the clang bug 2023-07-25 10:26:26 +00:00
Andrey Zvonov
15b3a637e9
Merge branch 'master' into revert-52450-remove-to-decimal-string 2023-07-25 13:21:00 +03:00
zvonand
6b8752f293 fix error message 2023-07-25 12:19:35 +02:00
Michael Kolupaev
dfdf5de972 Fixes 2023-07-25 10:16:28 +00:00
Michael Kolupaev
db5cb96050 Start over when falling back to non-dictionary encoding 2023-07-25 10:16:28 +00:00
Michael Kolupaev
8184a289e5 Partially reimplement Parquet encoder to make it faster and parallelizable 2023-07-25 10:16:28 +00:00
Smita Kulkarni
94ff9a4696 Merge branch 'master' into Fix_parameterized_view_with_cte_multiple_usage 2023-07-25 12:08:38 +02:00
zvonand
d500e75569 fix 2023-07-25 12:07:47 +02:00
Yarik Briukhovetskyi
5f767b0dfa
Merge pull request #52004 from JackyWoo/rewrite_uniq_to_count
Rewrite uniq to count
2023-07-25 11:41:00 +02:00
Azat Khuzhin
2efbeab5af tests: fix 00719_parallel_ddl_table flakiness in debug builds
In debug bulds each client invocation takes ~1 second, and on CI it can
take more if the node is under some load, so let's decrease number of
iterations.

Anyway CI runs each test ~1K times daily, and if there will be something
even this number of iterations should be enough.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-25 10:43:40 +02:00
Azat Khuzhin
b02e290d55 tests: fix 01035_avg_weighted_long flakiness
Use one clickhouse-client invocation instead of 300, in debug builds it
is significant - each spawn is ~1 second

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-25 10:38:16 +02:00
Azat Khuzhin
3e3adc7fec tests: increase throttling for 01923_network_receive_time_metric_insert
In debug builds launching the client can take a while, so let's increase
the throttling to avoid flakiness

CI: https://s3.amazonaws.com/clickhouse-test-reports/52490/9e2526a5f04861fcfac49c2ce85560d08c68af66/stateless_tests__debug__[1_5].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-25 10:31:18 +02:00
Roman Vasin
ab55c294bf Add assert for STANDARD storage class 2023-07-25 07:25:27 +00:00
robot-ch-test-poll
bd5d93e439
Merge pull request #52536 from ClickHouse/Fix_disallow_concurrent_backup_restore_test
Fix disallow_concurrency test for backup and restore
2023-07-25 09:10:27 +02:00
robot-ch-test-poll1
4e5ed9b475
Merge pull request #52534 from ClickHouse/analyzer_fix_01951_distributed_push_down_limit
Make 01951_distributed_push_down_limit analyzer agnostic
2023-07-25 07:45:21 +02:00
robot-ch-test-poll4
fe6cd14a48
Merge pull request #52527 from Algunenano/mergetree_cleanup
Remove unused code
2023-07-25 07:41:42 +02:00
Alexey Milovidov
21382afa2b Check for punctuation 2023-07-25 06:10:04 +02:00
Alexey Milovidov
90a6a018b0
Merge branch 'master' into 50727_currentDatabase_alias 2023-07-25 06:51:28 +03:00
Alexey Milovidov
ecdafeaf83
Merge pull request #52175 from bigo-sg/comment_improve_ch_to_arrow
Add comments for https://github.com/ClickHouse/ClickHouse/pull/52112
2023-07-25 06:47:48 +03:00
Alexey Milovidov
0d0ec4cdf3
Merge branch 'master' into iaadeflate_upgrade_qpl_v1.2.0 2023-07-25 06:47:15 +03:00
flynn
059025f452
Merge branch 'master' into operator 2023-07-25 11:21:40 +08:00
flynn
8388ee067a
Merge branch 'master' into any_value 2023-07-25 11:21:19 +08:00
Sanjam Panda
f8787508ec
Merge branch 'master' into external_aggregation_fix 2023-07-25 08:35:05 +05:30