thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 18:02:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
107,643 Commits 9,328 Branches 1,783 Tags 2.1 GiB
1b207d9fa0
Commit Graph

11394 Commits

Author SHA1 Message Date
kssenii
4a1bdcf24c Merge remote-tracking branch 'upstream/master' into storage-rabbitmq-fix-flush 2023-01-31 18:11:26 +01:00
Anton Popov
1c4174750d fix mistake after refactoring 2023-01-31 15:48:14 +00:00
Raúl Marín
7c31cb7adc Proper includes for ConnectionTimeoutsContext.h 2023-01-31 16:11:32 +01:00
Alexander Tokmakov
7862990721 Merge branch 'master' into explicit_drop_part_entry_type 2023-01-31 13:49:04 +01:00
Alexander Tokmakov
2e8100b6e4 make separate DROP_PART log entry type 2023-01-31 13:37:56 +01:00
Alexander Tokmakov
51b0f66e6b
Merge pull request #45793 from ClickHouse/fix_part_check_cancellation 
Fix race condition on a part check cancellation
 2023-01-31 14:15:17 +03:00
Kseniia Sumarokova
7d53f8bbf6
Merge pull request #45591 from bharatnc/ncb/odbc-connection-pool-fixes 
fixes to use the odbc_bridge_use_connection_pooling setting correctly
 2023-01-31 11:31:59 +01:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache" 
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
 2023-01-31 09:54:34 +00:00
Alexander Gololobov
f9133f3d85
Merge pull request #45791 from ClickHouse/fix_drop_hung 
Fix bug in tables drop which can lead to potential query hung
 2023-01-31 10:11:11 +01:00
Bharat Nallan
f1d6e3b908
Merge branch 'master' into ncb/odbc-connection-pool-fixes 2023-01-30 15:49:04 -08:00
Ilya Yatsishin
8d6df7ed58
Merge pull request #45785 from Algunenano/generate_random_types 2023-01-30 23:03:12 +01:00
Alexander Gololobov
4eea3de654 Do not restrict count() query to 1 thread in isStorageTouchedByMutations() 2023-01-30 22:29:28 +01:00
Alexander Tokmakov
e42cd3163f fix race condition on a part check cancellation 2023-01-30 18:48:20 +01:00
Alexander Gololobov
863548114d force_wait flag is no longer needed 2023-01-30 18:38:28 +01:00
Vitaly Baranov
2e3a3cc4dc
Merge pull request #45701 from vitlibar/add-setting-allow-head-object-request 
Add new S3 setting allow_head_object_request
 2023-01-30 17:41:09 +01:00
Bharat Nallan Chakravarthy
0d2e4fcc15 set use_connection_pooling from settings everywhere 2023-01-30 08:22:48 -08:00
alesapin
be58d5d1af Fix bug in tables drop which can lead to potential query hung 2023-01-30 17:00:28 +01:00
Raúl Marín
997fad4e8d GenerateRandom: Add support for maps 2023-01-30 15:23:22 +01:00
Raúl Marín
05d8cfc714 GenerateRandom: Add support to IPv4 and IPv6 types 2023-01-30 14:18:25 +01:00
Vitaly Baranov
38910412c4
Merge pull request #42244 from AVMusorin/fix_backup_restore_num_files 
Added num_processed_files and processed_files_size for backup and restore processes
 2023-01-30 09:24:49 +01:00
Smita Kulkarni
6cf2498609 Fixed clang-tidy build for comparing strings and updated to notify after adding subscriber if queue is already empty- Updated checking of SYSTEM SYNC REPLICA 2023-01-30 08:18:20 +01:00
alesapin
631c8fb155
Merge branch 'master' into add-setting-allow-head-object-request 2023-01-29 14:58:27 +01:00
SmitaRKulkarni
809354dc30
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-29 11:28:21 +01:00
Alexey Milovidov
49a59ff9bd Fix bad comment 2023-01-29 03:57:59 +01:00
Smita Kulkarni
9b457a57a9 Fixed clang-tidy build - Updated checking of SYSTEM SYNC REPLICA 2023-01-28 21:43:41 +01:00
Smita Kulkarni
2bbe08ce8b Updated destructor to notify even when queue empty - Updated checking of SYSTEM SYNC REPLICA 2023-01-27 22:37:42 +01:00
Sergei Trifonov
5c2686fbf0
Merge branch 'master' into io-scheduler-integration 2023-01-27 19:56:23 +01:00
serxa
21e871a0a4 integrate IO scheduler with buffers 2023-01-27 18:47:22 +00:00
Vitaly Baranov
af74c008c4 Use one request to implement S3ObjectStorage::getObjectMetadata instead of two ones. 2023-01-27 18:42:37 +01:00
kssenii
9c4a97bbd1 Merge remote-tracking branch 'upstream/master' into storage-rabbitmq-fix-flush 2023-01-27 17:17:09 +01:00
Anton Popov
d8fe2bcbaa
Merge branch 'master' into randomize-mt-settings 2023-01-27 16:03:00 +01:00
Anton Popov
3f1db52d51 Merge remote-tracking branch 'upstream/master' into compact-parts-vertical-merge 2023-01-27 14:38:37 +00:00
Vitaly Baranov
d02b255b55 Add new setting "allow_head_object_request" to S3::RequestSettings. 2023-01-27 15:10:34 +01:00
Vitaly Baranov
aea9ccdb60 Pass request settings to S3::getObjectInfo(). 2023-01-27 15:10:09 +01:00
Vitaly Baranov
a8304525ed Move getObjectInfo() to a separate header. 2023-01-27 15:09:38 +01:00
Nikolai Kochetov
313947e0b0
Merge branch 'master' into issue_45590 2023-01-27 15:08:28 +01:00
SmitaRKulkarni
b22fd40d92
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-27 15:00:47 +01:00
Anton Popov
f10e82355e
Merge pull request #45630 from CurtizJ/fix-performance-columns-description 
Fix performance of short queries with `Array` columns
 2023-01-27 14:59:55 +01:00
Alexander Tokmakov
ec0738ef31
Merge pull request #45692 from ClickHouse/tavplubix-patch-3 
Fix build
 2023-01-27 16:39:56 +03:00
Anton Popov
9c1717b42d fix computation of granularity in vertical merges 2023-01-27 13:35:04 +00:00
Alexander Tokmakov
f375b1246f
Merge pull request #45363 from ClickHouse/fix_hang_during_drop_in_zero_copy_replication 
Fix no shared id during drop for the fourth time
 2023-01-27 15:59:27 +03:00
Nikolai Kochetov
f6ad6296ba Restore. 2023-01-27 12:45:41 +00:00
Alexander Tokmakov
1494262a38
Update DataPartStorageOnDiskFull.cpp 2023-01-27 15:42:41 +03:00
Nikolai Kochetov
b716e2d754 A bit cleaner fix. 2023-01-27 12:37:23 +00:00
Smita Kulkarni
39966457d9 Updated to remove waiting ids till the removed log entry - Updated checking of SYSTEM SYNC REPLICA 2023-01-27 13:36:44 +01:00
alesapin
716d25c33f
Merge pull request #45619 from CurtizJ/refactoring-merge-tree-parts 
Refactoring of code near merge tree parts
 2023-01-27 13:36:33 +01:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4 
Better formatting for exception messages 2
 2023-01-27 15:31:52 +03:00
Smita Kulkarni
a95609e0bc Updated to validate & remove each element instead of just last element - Updated checking of SYSTEM SYNC REPLICA 2023-01-27 10:35:12 +01:00
Amos Bird
716516f84a
Fix test 2023-01-27 16:58:41 +08:00
Amos Bird
fd60954081
Fix key description on duplicate primary keys 
This can happen in projections. See
https://github.com/ClickHouse/ClickHouse/issues/45590 for details.
 2023-01-27 13:01:36 +08:00
Anton Popov
fb3f7409fa allow vertical merges from compact to wide parts 2023-01-26 22:18:04 +00:00
Alexander Tokmakov
061026c307 fix 2023-01-26 19:01:29 +01:00
Alexey Milovidov
d942a4ddef Allow LowCardinality in GenerateRandom 2023-01-26 15:35:20 +01:00
Alexander Tokmakov
067b1f5f13 Merge branch 'master' into exception_message_patterns4 2023-01-26 15:20:58 +01:00
Alexander Tokmakov
f676e9eaff
Merge pull request #45613 from azat/replicated-improve-log 
Improve "at least part X is missing" error message
 2023-01-26 16:48:18 +03:00
Anton Popov
92fa39798b Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-26 13:29:23 +00:00
Anton Popov
b58b73b0e7
Merge pull request #45529 from CurtizJ/fix-storage-s3-race 
Try to fix test `test_storage_s3/test.py::test_wrong_s3_syntax` (race in `StorageS3`)
 2023-01-26 14:21:32 +01:00
Anton Popov
87af86e9a3 fix projections 2023-01-26 13:11:31 +00:00
Alexander Tokmakov
14db798191 fix 2023-01-26 13:56:16 +01:00
Anton Popov
b52cf17fa6
Merge pull request #45612 from ClickHouse/CurtizJ-patch-7 
Remove useless code
 2023-01-26 12:38:52 +01:00
alesapin
84af5eb58b Fxi 2023-01-26 11:42:53 +01:00
alesapin
d5036f2ebc Merge branch 'fix_hang_during_drop_in_zero_copy_replication' of github.com:ClickHouse/ClickHouse into fix_hang_during_drop_in_zero_copy_replication 2023-01-26 11:42:30 +01:00
alesapin
1f61abce46 One useful logging line 2023-01-26 11:41:43 +01:00
Smita Kulkarni
8c885ffeae Updated checking of SYSTEM SYNC REPLICA 
Implementation:
* Updated to wait for current last entry to be processed (after pulling shared log) instead of queue size becoming 0.
* Updated Subscriber to notify both queue size and removed log_entry_id.
 2023-01-26 11:40:09 +01:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Antonio Andelic
6fe9e9a67f
Merge pull request #45617 from ClickHouse/fix-bfngram-index 
Correctly check types when using N-gram bloom filter index
 2023-01-26 09:05:56 +01:00
Alexander Tokmakov
9b670946db Merge branch 'master' into exception_message_patterns5 2023-01-26 00:41:32 +01:00
Mikhail f. Shiryaev
120c31d57f
Merge pull request #45634 from ClickHouse/23.2-prepare 
Update version after release
 2023-01-26 00:10:53 +01:00
Mikhail f. Shiryaev
53d89844be
Update version to 23.2.1.1 2023-01-25 23:57:29 +01:00
Anton Popov
79ea6a1c9c fix performance of short queries 2023-01-25 22:20:01 +00:00
Alexander Tokmakov
92c62e2de2
Update src/Storages/StorageReplicatedMergeTree.cpp 2023-01-26 00:28:46 +03:00
Han Fei
0221b50028
Merge pull request #45601 from hanfei1991/hanfei/async-cache-conflict 
fix race of async-insert-blockids-cache
 2023-01-25 22:28:42 +01:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
Anton Popov
0f46c1ecc6 fix build 2023-01-25 20:13:54 +00:00
alesapin
d0428a9b42 Fxi 2023-01-25 20:30:19 +01:00
Kruglov Pavel
cd3d3fdd68
Merge pull request #45483 from Avogar/fix-hdfs-cluster 
Fix schema inference from insertion table in hdfsCluster
 2023-01-25 20:24:51 +01:00
Azat Khuzhin
00c341850f Improve "at least part X is missing" error message 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-25 19:53:45 +01:00
Anton Popov
a455083b87 Merge remote-tracking branch 'upstream/master' into refactoring-merge-tree-parts 2023-01-25 18:01:12 +00:00
Kruglov Pavel
83a212a128
Merge branch 'master' into more-fsync-fixes 2023-01-25 18:40:41 +01:00
Anton Popov
8e3698c91f refactoring of code near merge tree parts 2023-01-25 17:34:09 +00:00
Antonio Andelic
7fa906487c Fix ngram index with null 2023-01-25 16:40:04 +00:00
Anton Popov
d9aabb48c3 remove useless code 2023-01-25 16:11:15 +00:00
alesapin
55af44ca2f Better fix 2023-01-25 14:40:39 +01:00
Alexander Tokmakov
b9e5c586e6
Merge pull request #45566 from ClickHouse/resubmit_45493 
Resubmit "Fix possible in-use table after DETACH"
 2023-01-25 16:39:55 +03:00
Alexander Tokmakov
6eb557b2ba Merge branch 'master' into exception_message_patterns4 2023-01-25 13:49:17 +01:00
Robert Schulze
59528cfca0
Merge pull request #45460 from ClickHouse/inv-index-cleanup 
Cleanup of inverted index
 2023-01-25 13:23:38 +01:00
alesapin
96d8546b1c Merge branch 'master' into fix_hang_during_drop_in_zero_copy_replication 2023-01-25 13:07:14 +01:00
Han Fei
8c4bbcd0fc fix race of async-insert-blockids-cache 2023-01-25 12:41:01 +01:00
Sergei Trifonov
0d1ea05ff6
Merge pull request #45007 from ClickHouse/cancellable-mutex-integration 
Fast shared mutex integration
 2023-01-25 11:15:46 +01:00
Kseniia Sumarokova
2b8b1ad5d4
Additional check in MergeTreeReadPool (#45515) 
* Check ranges

* Check equality just in case

* Check under ndebug
 2023-01-24 23:56:26 +01:00
Alexander Tokmakov
6621834e37 fix 2023-01-24 17:10:54 +01:00
avogar
07ad750366 Fix tests 2023-01-24 15:08:19 +00:00
Alexander Tokmakov
6ecae8388e Merge branch 'master' into exception_message_patterns4 2023-01-24 14:42:36 +01:00
Alexander Tokmakov
42a976fe3d Revert "Revert "Merge pull request #45493 from azat/fix-detach"" 
This reverts commit 9dc4f02d15.
 2023-01-24 14:15:51 +01:00
Azat Khuzhin
54ef885a61 Fsync all small files at once after mutation 
Everything else is handled in MergedBlockOutputStream::finalizePart()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-24 14:08:37 +01:00
Azat Khuzhin
4b775ccdd5 Do fsync all files at once for fetched parts to decrease latency 
For filessystems like ext4, fsync of one file will handle all operations
before, so this can be pretty time consuming.

And in case of you write multiple files in a loop, and at the end of
each iteration sync each file, then during writing of this file there
can be other operations in journal, and hence more work for fsync.

Let's call fsync for all files at once instead, like
MergedBlockOutputStream does.

Hope that keeping all file buffers till the end, will not cause troubles
(buffering and so forth).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-24 14:08:37 +01:00
Azat Khuzhin
b370e3faa5 Move fsync inside transaction callback in DataPartStorageOnDisk::rename() 
Otherwise, it is useless.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-24 14:08:37 +01:00
Anton Popov
5c0307bc6a fix race in StorageURL and StorageHDFS 2023-01-24 12:34:43 +00:00
Alexander Tokmakov
9dc4f02d15 Revert "Merge pull request #45493 from azat/fix-detach" 
This reverts commit a182a6b851, reversing
changes made to c47a29a089.
 2023-01-24 12:20:37 +01:00
Robert Schulze
9ff2bfcbf5
Merge remote-tracking branch 'origin/master' into inv-index-cleanup 
src/Interpreters/GinFilter.cpp
	src/Interpreters/InterpreterCreateQuery.cpp
	src/Storages/MergeTree/MergeTreeData.cpp
	src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp
	src/Storages/MergeTree/MergeTreeIndexInverted.cpp
 2023-01-24 10:09:42 +00:00
Alexey Milovidov
dda6a69f9a
Merge branch 'master' into tavplubix-patch-1 2023-01-24 13:04:25 +03:00
Alexey Milovidov
54bc501d39
Update StorageReplicatedMergeTree.cpp 2023-01-24 13:03:28 +03:00
Anton Popov
8a1ea62aec fix race on cancelation of query in StorageS3 2023-01-24 01:12:01 +00:00
Alexander Tokmakov
414693feb2 fixes 2023-01-24 00:46:03 +01:00
Alexander Tokmakov
bb4c8e169f check number of parameters in format string 2023-01-23 23:16:16 +01:00
Anton Popov
b987ed5037
Merge pull request #45499 from CurtizJ/add-fields-system-formats 
Add fields to table `system.formats`
 2023-01-23 23:08:35 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Robert Schulze
7adec3d8a4
Update src/Storages/MergeTree/MergeTreeIndexInverted.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-01-23 20:53:25 +01:00
Robert Schulze
d8af5bc2ef
Update src/Storages/MergeTree/MergeTreeIndexInverted.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-01-23 20:53:17 +01:00
Robert Schulze
0ab7ae6c17
Merge pull request #43797 from ClickHouse/query-result-cache 
Query result cache [experimental]
 2023-01-23 19:54:35 +01:00
Maksim Kita
7b48c75e82
Merge pull request #45485 from kitaisreal/remove-unnecessary-get-total-row-count-function-calls 
Remove unnecessary getTotalRowCount function calls
 2023-01-23 21:02:51 +03:00
Anton Popov
08b2e524c7 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-23 15:34:53 +00:00
Kseniia Sumarokova
8ec39ab6f3
Merge pull request #45442 from kssenii/minor-changes 
Minor improvements around reading from remote
 2023-01-23 15:46:30 +01:00
Alexander Tokmakov
a182a6b851
Merge pull request #45493 from azat/fix-detach 
Fix possible in-use table after DETACH
 2023-01-23 17:36:45 +03:00
Aleksandr
2caeed901b
Merge branch 'master' into fix_backup_restore_num_files 2023-01-23 13:43:20 +01:00
Sergei Trifonov
0fbfa17863
Merge branch 'master' into cancellable-mutex-integration 2023-01-23 12:44:09 +01:00
Azat Khuzhin
51019bc9f3 Fix a race between Distributed table creation and INSERT into it 
Initializing queues for pending on-disk files for async INSERT cannot be
done after table had been attached and visible to user, since it
initializes the per-table counter, that is used during INSERT.

Now there is a window, when this counter is not initialized and it will
start from the beginning, and this could lead to CANNOT_LINK error:

    Destination file /data/clickhouse/data/urls_v1/urls_in/shard6_replica1/13129817.bin is already exist and have different inode

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-23 09:55:43 +01:00
Anton Popov
8a39337c7c add fields to table system.formats 2023-01-23 02:04:31 +00:00
Robert Schulze
340f406553
Merge branch 'master' into query-result-cache 2023-01-22 13:21:36 +01:00
Robert Schulze
531e64dc39
Merge branch 'master' into inv-index-cleanup 2023-01-22 13:20:38 +01:00
Azat Khuzhin
ca8af3d18c Fix possible in-use table after DETACH 
Right now in case of DETACH/ATTACH there can be a window when after the
table had been DETACH'ed someone will still use it, the common example
here is MVs handling.

It happens because TableExclusiveLockHolder does not guards the
shard_ptr of the IStorage, and so if someone holds it, then it can use
it. So if ATTACH will be done for this table then, you can have multiple
instances of it.

It is not possible for DROP, because before using a table, you should
lock it and after table had been DROP'ed you cannot lock it anymore.

So let's do the same for DETACH.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-22 13:18:56 +01:00
Alexander Gololobov
91fb1eab8c
Merge pull request #45373 from ClickHouse/throttler-metrics 
Add detailed profile events for throttling
 2023-01-22 11:20:45 +01:00
Alexander Tokmakov
1174eaa132
Merge pull request #45492 from azat/revert/dist/async-INSERT-metrics 
Revert "Merge pull request #44922 from azat/dist/async-INSERT-metrics"
 2023-01-22 00:45:10 +03:00
Azat Khuzhin
a55798626a Revert "Merge pull request #44922 from azat/dist/async-INSERT-metrics" 
There are the following problems with this patch:
- Looses files on exception
- Existing current_batch.txt on startup leads to ENOENT error and hung
  of distributed sends without ATTACH/DETACH
- Race between creating the queue for sending at table startup and
  INSERT, if it had been created from INSERT, then it will not be
  initialized from disk

They were addressed in #45491, but it makes code more cmoplex and plus
since, likely, the release is comming, it is better to revert the
change.

This reverts commit 94604f71b7, reversing
changes made to 80f6a45376.
 2023-01-21 22:42:00 +01:00
Maksim Kita
19f1bae5ed
Merge pull request #45254 from kitaisreal/planner-small-fixes 
Planner small fixes
 2023-01-21 19:54:17 +03:00
Nikolai Kochetov
b877c484d2
Merge pull request #45481 from ClickHouse/fix-deadlock-with-allow_asynchronous_read_from_io_pool_for_merge_tree 
Fix possible deadlock with allow_asynchronous_read_from_io_pool_for_merge_tree in case of exception from ThreadPool::schedule
 2023-01-21 12:05:34 +01:00
Sema Checherinda
962894afc8
Merge pull request #44909 from CheSema/intersect-prev-part 
Do not merge over a gap with outdated undeleted parts
 2023-01-21 11:51:21 +01:00
Maksim Kita
47385a19e7 Remove unnecessary getTotalRowCount function calls 2023-01-21 11:27:07 +01:00
avogar
eed1db7e07 Fix schema inference in hdfsCluster 2023-01-20 21:17:35 +00:00
Anton Popov
41a199e175
Fix crash when ListObjects request fails (#45371) 2023-01-20 20:10:23 +01:00
Nikolai Kochetov
dcd84c152a Fix possible deadlock with allow_asynchronous_read_from_io_pool_for_merge_tree in case of exception from ThreadPool::schedule 2023-01-20 18:57:47 +00:00
Alexander Tokmakov
910d6dc0ce
Merge pull request #45342 from ClickHouse/exception_message_patterns 
Save message format strings for DB::Exception
 2023-01-20 18:46:52 +03:00
Sema Checherinda
b76b612d23
fix typo 2023-01-20 14:55:58 +01:00
Robert Schulze
1a966a9590
Fix bad comparison 2023-01-20 13:05:06 +00:00
Sema Checherinda
02f22f04e8
fix typos 2023-01-20 13:35:23 +01:00
kssenii
8d20af8127 Fix 2023-01-20 13:34:23 +01:00
Robert Schulze
bfc3b4f5ca
Suffix "GinFilter" --> "Inverted" 2023-01-20 12:02:35 +00:00
Robert Schulze
0738b2499c
Use GinFilters typedef where possible 2023-01-20 11:52:04 +00:00
Robert Schulze
0b77f07f67
Remove superfluous check (the same is checked in MergeTreeIndices.cpp) 2023-01-20 11:50:35 +00:00
Robert Schulze
d2c830ec39
Cosmetics 2023-01-20 11:49:08 +00:00
Robert Schulze
72973076c9
Rename MergeTreeIndexGin.h/cpp to MergeTreeIndexInverted.h/cpp 2023-01-20 11:42:36 +00:00
Robert Schulze
1ef2704539
Cosmetics 2023-01-20 11:39:23 +00:00
Robert Schulze
463cc843de
"segment file" --> "segment metadata file" 2023-01-20 11:26:22 +00:00
Robert Schulze
58df3953bb
Move some code around (no other changes) 2023-01-20 11:24:23 +00:00
Robert Schulze
3267ac2787
Prefix more typedefs in DB namespace with "Gin" 2023-01-20 11:19:07 +00:00
Robert Schulze
919b67f117
Cosmetics 2023-01-20 11:15:28 +00:00
Sema Checherinda
09f3a5c599 add a comment, add a check, fix test 2023-01-20 12:10:31 +01:00
Robert Schulze
98e117dca6
SegmentDictionary --> GinSegmentDictionary, also move typedef 2023-01-20 11:09:49 +00:00
Robert Schulze
908fa83f72
Move some typedefs around 2023-01-20 11:08:19 +00:00
Robert Schulze
44618927f9
Inline two short methods + uppercase 2023-01-20 11:04:35 +00:00
Robert Schulze
5c3cc5283f
"term dictionary" --> "dictionary" 2023-01-20 10:53:41 +00:00
Robert Schulze
be936b257c
Make version enum private 2023-01-20 10:48:43 +00:00
Robert Schulze
0653f86de9
Various cosmetic cleanups 2023-01-20 10:45:35 +00:00
Maksim Kita
23e26032ca
Merge pull request #45399 from aalexfvk/alexfvk/mdb-21326_fix_system_dictionaries_when_dictionary_with_bad_structure 
Fix select from system.dictionaries when there is dictionary with bad structure
 2023-01-20 13:36:32 +03:00
Aleksandr Musorin
838acb22b7
added num_processed_files and processed_files_size 2023-01-20 10:20:41 +01:00
Frank Chen
f867c4c206 Add an overriden fromCurrentException to simply the code 2023-01-20 16:43:10 +08:00
Frank Chen
38799eb852 Merge two methods as one 2023-01-20 16:35:28 +08:00
Frank Chen
be8ceeb4bb Save stack trace in part_log 2023-01-20 16:22:40 +08:00
Frank Chen
1fe9bde594 Set replica name to zero copy exclusive lock 2023-01-20 15:48:33 +08:00
Frank Chen
fb8452aa49 Add last_exception_time to replication_queue 2023-01-20 15:20:57 +08:00
Maksim Kita
2c56b0b2b9 Planner small fixes 2023-01-19 19:05:49 +01:00
kssenii
4ce8950712 Minor changes 2023-01-19 17:53:10 +01:00
larryluogit
52ae33dba7
Merge branch 'master' into ftsearch 2023-01-19 11:34:11 -05:00
Aleksei Filatov
afada0ecb3 Fix review notes 2023-01-19 17:02:57 +03:00
Alexander Tokmakov
7bb65cc002
Update StorageReplicatedMergeTree.cpp 2023-01-19 16:45:41 +03:00
Han Fei
f661dad0e9
Merge pull request #45106 from hanfei1991/hanfei/async-cache 
support cache for async inserts block ids
 2023-01-19 10:59:25 +01:00
Ilya Yatsishin
d16b59b662
Merge pull request #45422 from Avogar/fix-s3-cluser-si 2023-01-19 10:36:54 +01:00
avogar
117ec13c9e Fix s3Cluster schema inference when structure from insertion table is used 2023-01-18 20:33:50 +00:00
Aleksei Filatov
5e9340f682 Add integration test 2023-01-18 17:50:38 +03:00
Aleksei Filatov
7f4a01b903 Add handling of bad dictionary structure 2023-01-18 17:27:03 +03:00
Sema Checherinda
ae1dfb9ce5
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-01-18 15:21:11 +01:00
Sema Checherinda
a344b526a6
Update src/Storages/StorageMergeTree.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-01-18 15:16:18 +01:00
Antonio Andelic
8f8b14148a
Merge pull request #45215 from ClickHouse/fix-crash-kv-store 
Fix crash when prepared set with different type used in KV stores
 2023-01-18 13:27:40 +01:00
Vitaly Baranov
7cdb2c4c7f
Merge pull request #45351 from vitlibar/fix-backup-with-killed-mutations 
Fix backup with killed mutations
 2023-01-18 13:14:27 +01:00
Han Fei
e51123c9b0 fix data race 2023-01-18 13:11:07 +01:00
Antonio Andelic
cfba9b19eb
Merge pull request #45360 from azat/dist/fix-startup-race 
Fix race in Distributed table startup
 2023-01-18 10:09:54 +01:00
Antonio Andelic
f57ee043ae
Merge pull request #45319 from ClickHouse/disable-prewhere-in-merge-different-types 
Disable PREWHERE in storage Merge if types don't match
 2023-01-18 10:02:06 +01:00
Antonio Andelic
f3469ee077
Merge branch 'master' into dist/fix-startup-race 2023-01-18 09:44:52 +01:00
Sergei Trifonov
c443c1ece0
Merge branch 'master' into hanfei/async-cache 2023-01-18 00:19:49 +01:00
Robert Schulze
4f90824347
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-17 22:49:53 +00:00
Anton Popov
f40fd7a151
Add checks for compilation of regexps (#45356) 2023-01-17 23:46:04 +01:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Alexander Tokmakov
72e8615bec formatting of some exception messages 2023-01-17 20:03:56 +01:00
serxa
ce7e22b87b add detailed profile events for throttling 2023-01-17 18:29:24 +00:00
alesapin
e732f510f0
Merge branch 'master' into fix_hang_during_drop_in_zero_copy_replication 2023-01-17 19:24:36 +01:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern 
Add a column with a message pattern to text_log
 2023-01-17 20:19:32 +03:00
alesapin
69925647eb Fix style 2023-01-17 15:59:55 +01:00
alesapin
f6131101bb Fix no shared id during drop for the fourth time 2023-01-17 15:51:49 +01:00
Han Fei
8a74238fe0 improve 2023-01-17 15:47:52 +01:00
Kruglov Pavel
582aa8b770
Merge pull request #45253 from Avogar/fix-s3-heap-use-after-free 
Fix heap-use-after-free in reading from s3
 2023-01-17 15:32:26 +01:00
HarryLeeIBM
e7add8218f Addressed more review comments and ClangTidy errors 2023-01-17 06:29:13 -08:00
Azat Khuzhin
54fc6859ae Fix race in Distributed table startup 
Before this patch it was possible to have multiple directory monitors
for the same directory, one from the INSERT context, another one on
storage startup().

Here are an example of logs for this scenario:

    2022.12.07 12:12:27.552485 [ 39925 ] {a47fcb32-4f44-4dbd-94fe-0070d4ea0f6b} <Debug> DDLWorker: Executed query: DETACH TABLE inc.dist_urls_in
    ...
    2022.12.07 12:12:33.228449 [ 4408 ] {20c761d3-a46d-417b-9fcd-89a8919dd1fe} <Debug> executeQuery: (from 0.0.0.0:0, user: ) /* ddl_entry=query-0000089229 */ ATTACH TABLE inc.dist_urls_in (stage: Complete)
    ... this is the DirectoryMonitor created from the context of INSERT for the old StoragePtr that had not been destroyed yet (becase of "was 1" this can be done only from the context of INSERT) ...
    2022.12.07 12:12:35.556048 [ 39536 ] {} <Trace> inc.dist_urls_in.DirectoryMonitor: Files set to 173 (was 1)
    2022.12.07 12:12:35.556078 [ 39536 ] {} <Trace> inc.dist_urls_in.DirectoryMonitor: Bytes set to 29750181 (was 71004)
    2022.12.07 12:12:35.562716 [ 39536 ] {} <Trace> Connection (i13.ch:9000): Connected to ClickHouse server version 22.10.1.
    2022.12.07 12:12:35.562750 [ 39536 ] {} <Debug> inc.dist_urls_in.DirectoryMonitor: Sending a batch of 10 files to i13.ch:9000 (0.00 rows, 0.00 B bytes).
    ... this is the DirectoryMonitor that created during ATTACH ...
    2022.12.07 12:12:35.802080 [ 39265 ] {} <Trace> inc.dist_urls_in.DirectoryMonitor: Files set to 173 (was 0)
    2022.12.07 12:12:35.802107 [ 39265 ] {} <Trace> inc.dist_urls_in.DirectoryMonitor: Bytes set to 29750181 (was 0)
    2022.12.07 12:12:35.834216 [ 39265 ] {} <Debug> inc.dist_urls_in.DirectoryMonitor: Sending a batch of 10 files to i13.ch:9000 (0.00 rows, 0.00 B bytes).
    ...
    2022.12.07 12:12:38.532627 [ 39536 ] {} <Trace> inc.dist_urls_in.DirectoryMonitor: Sent a batch of 10 files (took 2976 ms).
    ...
    2022.12.07 12:12:38.601051 [ 39265 ] {} <Error> inc.dist_urls_in.DirectoryMonitor: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in file_size: No such file or directory ["/data6/clickhouse/data/inc/dist_urls_in/shard13_replica1/66827403.bin"], Stack trace (when copying this message, always include the lines below):
    ...
    2022.12.07 12:12:54.132837 [ 4408 ] {20c761d3-a46d-417b-9fcd-89a8919dd1fe} <Debug> DDLWorker: Executed query: ATTACH TABLE inc.dist_urls_in

And eventually both monitors (for a short period of time, one replaces
another) are trying to process the same batch (current_batch.txt), and
one of them fails because such file had been already removed.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-17 14:51:00 +01:00
Vitaly Baranov
692065e5fe Fix backup if mutations got killed during the backup process. 2023-01-17 11:05:34 +01:00
Vitaly Baranov
b13498d9ba
Merge pull request #45288 from vitlibar/fix-s3-requests-without-region 
Fix s3 requests without region
 2023-01-17 09:24:59 +01:00
Antonio Andelic
76eb3e3b3c Fix test 2023-01-17 07:34:39 +00:00
Alexander Tokmakov
522686f78b less empty patterns 2023-01-17 01:19:44 +01:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Alexander Tokmakov
e7899825e6 save format strings for DB::Exceptions 2023-01-16 23:20:33 +01:00
Sema Checherinda
dbe89cd5d8 fix that optimize final waits for currently running merges 2023-01-16 16:47:12 +01:00