thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
125,683 Commits 9,315 Branches 1,783 Tags 2.1 GiB
978a535849
Commit Graph

14530 Commits

Author SHA1 Message Date
Robert Schulze
e954ffe939
Cleanups 2023-09-26 09:51:48 +00:00
Robert Schulze
9fff447716
Re-enable clang-tidy checks 2023-09-26 09:34:12 +00:00
Robert Schulze
fa40732395
Merge pull request #54766 from rschu1ze/is_deterministic 
Added field "is_deterministic" to system.functions
 2023-09-26 11:05:03 +02:00
János Benjamin Antal
036aa9f756
Merge pull request #54869 from usurai/ceil 
Update ceilRowsToCompleteGranules to use binary search.
 2023-09-26 09:46:20 +02:00
VanDarkholme7
9d74502488
Merge branch 'ClickHouse:master' into missing_subcolumns 2023-09-26 13:48:16 +08:00
VanDarkholme7
7dda1646fd fix 2023-09-26 02:32:05 +00:00
Nikolay Degterinsky
987906680f
Revert "Fix NATS high cpu usage" 2023-09-26 03:36:14 +02:00
Kseniia Sumarokova
5057a31bac
Merge pull request #54844 from kssenii/materialized-postrgesql-safer-parts 
Fix "Invalid number of rows in Chunk" in MaterializedPostgreSQL
 2023-09-25 23:37:12 +02:00
Robert Schulze
6f3d7fcc86
Better lint 2023-09-25 17:11:32 +00:00
Robert Schulze
0c490465d4
Stabilize tests 2023-09-25 17:07:48 +00:00
Robert Schulze
d170304359
Parallelize Annoy index creation + minor test fix 2023-09-25 17:07:23 +00:00
Robert Schulze
5233adecb0
Merge remote-tracking branch 'rschu1ze/master' into is_deterministic 2023-09-25 16:27:50 +00:00
Kruglov Pavel
af8b1e4978
Merge pull request #54855 from Avogar/obsolete-format-settings 
Move obsolete format settings to separate section
 2023-09-25 18:00:31 +02:00
Anton Popov
0ebc22e851
Merge branch 'master' into long-column-names 2023-09-25 16:55:14 +02:00
usurai
330bcff239 Update ceilRowsToCompleteGranules to use binary search. 
Replace the linear method in ceilRowsToCompleteGranules with a binary search.
 2023-09-25 22:52:00 +08:00
Azat Khuzhin
7a761730fa Respect max_block_size while reading form system.stack_trace 
Reading from system.stack_trace can be very slow, especially if you read
all columns (there are some optimizations to avoid doing some actions),
that requires:
- reading /proc/self/task
- reading /proc/self/task/*/comm
- sending signal to each of the thread (to read query_id and trace_id)

And even if you have some conditions (for which there are some
optimizations, i.e. not sending signal to the thread which is the
slowest operation under load) it will still send signals to every thread
and wait for result.

This patch implements reading from system.stack_trace with respect to
max_block_size, that will make queries with lower max_block_size and
LIMIT works significantly faster.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-24 14:01:24 +02:00
Azat Khuzhin
01d22805ef Remove indent for anonymous namespace 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-24 14:01:24 +02:00
Azat Khuzhin
e0bca06a5d Add some logging for preparing rows for system.stack_trace 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-24 14:00:43 +02:00
robot-clickhouse
320e4c47f3
Merge pull request #54943 from amosbird/fix-54941 
Rebuild minmax_count_projection when partition key gets modified
 2023-09-23 22:11:52 +02:00
robot-clickhouse-ci-2
2a26557391
Merge pull request #54399 from vahpetr/fix-nats-high-cpu-usage 
Fix NATS high cpu usage
 2023-09-23 21:36:21 +02:00
Sergei Trifonov
c1489d52b7
Merge pull request #54618 from ClickHouse/io-scheduling-throttler 
add IO scheduling node to limit bandwidth
 2023-09-23 21:26:17 +02:00
Alexey Milovidov
e89a8e4d13
Merge pull request #53492 from ClickHouse/fix-prewhere-lost-column 
Fix: moved to prewhere condition actions can lose column
 2023-09-23 22:24:00 +03:00
serxa
44acb53ad3 Merge branch 'master' into io-scheduling-throttler 2023-09-23 10:51:18 +00:00
Sergei Trifonov
4032fc17bb
Merge pull request #54780 from ucasfl/comment 
update comment
 2023-09-23 12:48:48 +02:00
Amos Bird
81804ce9ba
Rebuild minmax_count_projection when partition key gets modified 2023-09-23 17:59:06 +08:00
robot-ch-test-poll2
3c93a939a2
Merge pull request #54936 from ClickHouse/pufit/s3-yet-another-num-streams-adjustment 
Set a minimum limit of `num_streams` in StorageS3
 2023-09-23 02:59:50 +02:00
robot-clickhouse-ci-2
d98234dc9d
Merge pull request #54803 from Avogar/ephemeral-columns-from-files 
Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files
 2023-09-22 23:24:42 +02:00
Alexey Milovidov
783d1873b2
Merge pull request #54839 from rschu1ze/vector-search-test-refactoring 
Refactor and split up vector search tests
 2023-09-23 00:10:47 +03:00
robot-ch-test-poll3
1334202af4
Merge pull request #54863 from zvonand/zvonand-refactor-globs 
Refactor and simplify multi-directory globs
 2023-09-22 23:09:05 +02:00
pufit
13eee0e950 Set a minimum limit of num_streams in StorageS3 2023-09-22 14:13:20 -04:00
Nikolai Kochetov
cf40723179
Merge pull request #54895 from amosbird/allow-nullable-primary-keys-for-projections 
Always allow nullable keys for projections
 2023-09-22 19:09:24 +02:00
Alexander Tokmakov
21ffbd88de
Merge pull request #54859 from ClickHouse/fix_zero_copy_locks 
Fix zero copy locks with hardlinks
 2023-09-22 18:27:03 +02:00
Kruglov Pavel
4d675dbad4
Merge pull request #54825 from azat/fix-virtual-columns-filtering 
Fix filtering parts with indexHint for non analyzer (resubmit)
 2023-09-22 18:20:16 +02:00
Dmitry Novik
0f8afdbbe7 Merge remote-tracking branch 'origin/master' into fix-prewhere-lost-column 2023-09-22 14:50:05 +00:00
Alexander Tokmakov
f54c3c6a9f
Update StorageReplicatedMergeTree.cpp 2023-09-22 15:07:55 +02:00
Robert Schulze
e9c0dbd33e
Merge remote-tracking branch 'rschu1ze/master' into tableau-online-information-schema 2023-09-22 11:03:06 +00:00
Robert Schulze
9a91f97ac4
Merge pull request #53831 from rschu1ze/clang-17 
Switch CI to Clang 17
 2023-09-22 12:28:35 +02:00
Kseniia Sumarokova
163961115b
Merge pull request #54842 from kssenii/storage-rabbitmq-minor-change 
Add some logging to StorageRabbitMQ
 2023-09-22 11:51:27 +02:00
zvonand
2eb63649bd fix old workflows 2023-09-22 00:30:46 +02:00
Robert Schulze
877e4f3aab
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 20:21:12 +00:00
pufit
99c1d76604 Fix division by zero in StorageS3 2023-09-21 15:55:42 -04:00
Robert Schulze
a46e0deb85
Resolve bad caching of INCBIN'd external data 2023-09-21 17:41:21 +00:00
Michael Kolupaev
be0c512329
Fix virtual columns having incorrect values after ORDER BY (#54811) 
Fix virtual columns having incorrect values after ORDER BY
 2023-09-21 10:14:28 -07:00
Michael Kolupaev
9af9b4a085
Enable connection pooling for s3 table function (#54812) 
Enable connection pooling for s3 table function
 2023-09-21 09:27:20 -07:00
Amos Bird
973cd5e972
Always allow nullable keys for projections 2023-09-21 22:46:16 +08:00
Robert Schulze
5209bd2d51
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 14:45:55 +00:00
zvonand
e2b40749a6 the same for HDFS 2023-09-21 16:45:47 +02:00
Robert Schulze
f5137dd0b4
More clang-tidy fixes 2023-09-21 14:40:57 +00:00
zvonand
bc5f141a6c fix {..} for File 2023-09-21 16:40:38 +02:00
Anton Popov
4d7ecf39ac check file collisions on all alters 2023-09-21 14:16:03 +00:00
slvrtrn
2b8a263355 Change referential_constraints to an empty table 2023-09-21 16:11:08 +02:00
pufit
bd387f6d2c
Merge pull request #54815 from ClickHouse/pufit/optimal-num-streams-for-s3-storage 
Adjusting `num_streams` by expected work in StorageS3
 2023-09-21 08:41:39 -04:00
slvrtrn
e49ddbb678 Merge remote-tracking branch 'origin' into tableau-online-information-schema 2023-09-21 14:25:04 +02:00
Igor Nikonov
d260b7abe4
Merge pull request #54519 from ClickHouse/parallel-replicas-not-enough-replicas 
Disable parallel replicas on shards with not enough nodes
 2023-09-21 14:07:25 +02:00
Robert Schulze
44b7fa1030
Merge remote-tracking branch 'rschu1ze/master' into is_deterministic 2023-09-21 09:29:44 +00:00
Robert Schulze
c2a593baf9
Split tests 2023-09-21 09:17:38 +00:00
Vitaly Baranov
bf4662aca7
Merge pull request #53336 from azat/backups/s3-throttling 
Fix throttling of BACKUPs from/to S3 (in case native copy was not used) and in some other places as well
 2023-09-21 10:42:28 +02:00
zvonand
a05bb020d4 do all the same for HDFS + remove setting 2023-09-21 01:50:41 +02:00
pufit
4a2f7976f0 Resolve PR issues 2023-09-20 19:43:02 -04:00
zvonand
5f67788c03 update docs and comments 2023-09-21 00:16:17 +02:00
zvonand
9eccb12183 small refactor of storageFile globs 2023-09-20 23:59:10 +02:00
Robert Schulze
e95f9ee814
Merge remote-tracking branch 'rschu1ze/master' into is_deterministic 2023-09-20 21:17:57 +00:00
Alexander Tokmakov
f8062d77e1 fix rare data loss 2023-09-20 22:38:25 +02:00
Alexander Tokmakov
ec80068611 add a test for no such key 2023-09-20 22:37:24 +02:00
Alexander Tokmakov
18acc0c19d
Merge pull request #54641 from ClickHouse/followup_54550 
Follow-up to #54550
 2023-09-20 19:44:56 +02:00
pufit
20105958a8 add reserve 2023-09-20 13:37:06 -04:00
pufit
71c7e3c81e Add logging, fix thread name length 2023-09-20 13:33:25 -04:00
Azat Khuzhin
c706101891 Fix throttling of BACKUPs from/to S3 (in case native copy was not used) 
In some cases native copy is not possible, and such requests should be
throttled.

v0: copyS3FileNativeWithFallback
v2: revert v0 and pass write_settings
v3: pass read_settings to copyFile()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-20 18:28:43 +02:00
avogar
3e08800cb5 Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files 2023-09-20 16:25:55 +00:00
avogar
54cee196bd Move obsolete format settings to separate section 2023-09-20 16:00:28 +00:00
Robert Schulze
2c91e52da1
Merge pull request #54600 from rschu1ze/fix-annoy-crash 
Annoy/Usearch index: Fix LOGICAL_ERROR during build-up with default values
 2023-09-20 17:05:16 +02:00
pufit
729c8aa29f fix glob iterator estimated objects 2023-09-20 10:41:47 -04:00
kssenii
5d66189d87 Fix 2023-09-20 16:39:24 +02:00
kssenii
3ef2c37b92 Add some logging to StorageRabbitMQ 2023-09-20 15:54:11 +02:00
pufit
a1c98bc8c9 fix build 2023-09-20 09:50:35 -04:00
Igor Nikonov
2d446fc1ca Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas 2023-09-20 13:40:08 +00:00
slvrtrn
fb0ba86489 Merge remote-tracking branch 'origin' into tableau-online-information-schema 2023-09-20 15:14:40 +02:00
Robert Schulze
a4eeed50b6
Mark obsolete settings in system.merge_tree_settings as is_obsolete = 1 2023-09-20 12:55:53 +00:00
Azat Khuzhin
9c3fb64106 Check type of the filter expressions while filtering by virtual columns 
This should fix filtering by virtual columns with non-uint8 types, i.e.
queries like:

    SELECT * FROM data WHERE 1.0

Fixes: 02346_full_text_search
Fixes: 00990_hasToken_and_tokenbf
v2: move check out from is_const to filterBlockWithQuery(), since in is_const there is no way to validate sets.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-20 13:36:25 +02:00
flynn
dfa95ecc2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into comment 2023-09-20 11:16:33 +00:00
Nikolai Kochetov
0bc41bab74
Merge pull request #54544 from kitaisreal/key-condition-in-function-different-types 
KeyCondition IN function support different types
 2023-09-20 12:54:01 +02:00
alesapin
2ff36b6e0c
Merge pull request #54768 from ClickHouse/better_exception_in_check_data_part 
Better exception message in checkDataPart
 2023-09-20 12:04:46 +02:00
Robert Schulze
f3bf8388b6
Incorporate feedback 2023-09-20 10:00:00 +00:00
SmitaRKulkarni
d8adf05de2
Added a new column _block_number (#47532) 
Added a new virtual column _block_number which is persisted on merges when allow_experimental_block_number_column is enabled
 2023-09-20 11:31:12 +02:00
Azat Khuzhin
d9a634eb0f Fix filtering parts with indexHint for non analyzer 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit ffa82e9297)
 2023-09-20 11:29:35 +02:00
pufit
34aecc0bf3 Adjusting num_streams by expected work in StorageS3 2023-09-19 23:05:48 -04:00
slvrtrn
636fc506aa Replace information_schema view columns aliases with uppercase 2023-09-19 23:42:40 +02:00
Igor Nikonov
e28f4e819c
Merge branch 'master' into parallel-replicas-not-enough-replicas 2023-09-19 23:05:24 +02:00
Azat Khuzhin
c439c4bca2
Revert "Fix filtering parts with indexHint for non analyzer" 2023-09-19 21:39:21 +02:00
Han Fei
eb86c113e5
Merge pull request #54184 from priera/fix_mv_insertion_when_duplicated_parts 
Inserting only non-duplicate chunks in MV
 2023-09-19 20:41:45 +02:00
Kruglov Pavel
d555fdffa5
Merge pull request #54449 from azat/parts-prune-indexHint 
Fix filtering parts with indexHint for non analyzer
 2023-09-19 19:15:41 +02:00
slvrtrn
e4e9727572 Update information_schema docs 2023-09-19 14:14:17 +02:00
alesapin
9dcc41cf83 Fxi 2023-09-19 13:47:32 +02:00
alesapin
28b71163b4 Merge branch 'master' into better_exception_in_check_data_part 2023-09-19 13:45:56 +02:00
alesapin
7b1cf9d912 Revert "Better check for exceptions in checkData of non replicated merge tree" 
This reverts commit 363ee7cae5.
 2023-09-19 13:45:34 +02:00
alesapin
363ee7cae5 Better check for exceptions in checkData of non replicated merge tree 2023-09-19 13:41:55 +02:00
priera
b92e37d2df Fixing PR comments 2023-09-19 12:15:01 +02:00
priera
94d737264c inserting non-duplicate chunks 2023-09-19 12:15:00 +02:00
serxa
311db94640 default max_burst equals to max_speed as in Throttler 2023-09-19 10:10:21 +00:00
Maksim Kita
c7ddbab9bc Updated implementation 2023-09-19 13:06:19 +03:00