Alexander Tokmakov
59d28f3787
fix
2023-09-01 22:40:08 +02:00
Alexander Tokmakov
bba5a1d480
update zero copy locks correctly
2023-09-01 20:10:23 +02:00
Sergei Trifonov
cf5ea46713
Merge pull request #53338 from azat/throttling-fixes
...
Fix IO throttling during copying whole directories
2023-09-01 18:29:15 +02:00
Alexey Milovidov
f8c2ea986d
Update autogenerated version to 23.9.1.1 and contributors
2023-09-01 17:46:49 +02:00
Nikolai Kochetov
46087fd3e5
Refactor.
2023-09-01 15:45:08 +00:00
Victor Krasnov
52cce27e81
Merge branch 'master' into ADQM-987
2023-09-01 18:00:20 +03:00
Nikolai Kochetov
4cbfca78e4
Fixing 01748_partition_id_pruning
2023-09-01 14:29:31 +00:00
Yakov Olkhovskiy
361b21b416
fix fuzzers, cmake refactor, add target fuzzers
2023-09-01 14:20:50 +00:00
robot-clickhouse-ci-2
ae4f6c9683
Merge pull request #54137 from kssenii/s3-queue-better-default-path
...
Better path for storage `S3Queue` metadata
2023-09-01 16:01:38 +02:00
Anton Popov
033dba4772
Merge pull request #54126 from ClickHouse/CurtizJ-patch-8
...
Small improvements in `getAlterMutationCommandsForPart`
2023-09-01 14:10:45 +02:00
kssenii
9eb1dfcd12
Refactor buffers reading from object storage
2023-09-01 14:03:07 +02:00
Nikolai Kochetov
291bd76f1d
Merge pull request #54133 from ClickHouse/fixing-01086_odbc_roundtrip-with-analyzer
...
Fixing 01086_odbc_roundtrip with analyzer.
2023-09-01 12:54:50 +02:00
Kseniia Sumarokova
c51b6ca60c
Update StorageS3Queue.cpp
2023-08-31 18:29:06 +02:00
kssenii
da185fc8b1
Better paths for s3 queue metadata
2023-08-31 18:16:07 +02:00
Nikolai Kochetov
dc062856b0
Fixing 01086_odbc_roundtrip with analyzer.
2023-08-31 15:23:27 +00:00
Anton Popov
af70fdfcfa
mute noisy log message
2023-08-31 15:05:35 +00:00
Igor Nikonov
c3caead859
Merge pull request #52688 from Algunenano/better_zk_retries_on_insert
...
Improve insert retries on keeper session expiration
2023-08-31 17:00:02 +02:00
pufit
1bc6b40a12
Merge pull request #35102 from tonickkozlov/bloom-filter-index/deduplicate
...
Prune bloom filter indices based on data cardinality
2023-08-31 10:04:19 -04:00
Anton Kozlov
af21620ab5
Update src/Storages/MergeTree/MergeTreeIndexGranuleBloomFilter.h
...
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:06:21 +01:00
Anton Kozlov
d3123da824
Update src/Storages/MergeTree/MergeTreeIndexAggregatorBloomFilter.cpp
...
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:05:57 +01:00
Anton Kozlov
3f6ef6b5d9
Update src/Storages/MergeTree/MergeTreeIndexGranuleBloomFilter.h
...
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:05:41 +01:00
Anton Kozlov
e6895688d7
Update src/Storages/MergeTree/MergeTreeIndexAggregatorBloomFilter.cpp
...
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:05:28 +01:00
Anton Popov
77111a229d
fix clang-tidy
2023-08-31 12:59:21 +00:00
Raúl Marín
4703c463aa
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-31 14:37:02 +02:00
Antonio Andelic
0148e15aee
Merge pull request #53880 from ClickHouse/archive-improvements-2
...
Improve schema inference for archives
2023-08-30 21:11:34 +02:00
Alexander Tokmakov
83c5e2fba6
Revert "Fix bug on mutations with subcolumns of type JSON in predicates of UPDATE and DELETE queries."
2023-08-30 12:56:17 +02:00
Antonio Andelic
f406019413
Apply PR comments
2023-08-30 09:26:01 +00:00
Antonio Andelic
ddb58217d4
Merge branch 'master' into archive-improvements-2
2023-08-30 07:43:25 +00:00
Antonio Andelic
36fb7cfbd1
Merge pull request #54012 from ClickHouse/refactor-async-insert-with-dedup
...
Refactor logic around async insert with deduplication
2023-08-30 09:37:55 +02:00
Azat Khuzhin
51e8b16916
Fix IO throttling during copying whole directories
...
v2: pass local context during ALTER TABLE MOVE PARTITION TO VOLUME/DISK as well
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-29 23:13:46 +02:00
Nikita Mikhaylov
97efd14e90
Merge pull request #54009 from ClickHouse/change-istorage-interface
...
Change IStorage interface by random walk, no goal in particular
2023-08-29 22:22:53 +02:00
Sergei Trifonov
802579f3f1
Merge pull request #49618 from ClickHouse/concurrency-control-controllable
...
Make concurrency control controllable
2023-08-29 19:44:51 +02:00
Anton Popov
8197580746
fix tests and better interface
2023-08-29 17:23:52 +00:00
Antonio Andelic
e96c665be7
Refactor logic around async insert with dedup
2023-08-29 14:48:43 +00:00
Alexander Tokmakov
17f3754193
Merge pull request #53851 from ClickHouse/distributed_no_flush_on_shutdown
...
Follow-up to #53501
2023-08-29 16:34:22 +02:00
Nikita Mikhaylov
96cabcae3a
Change interface
2023-08-29 16:26:48 +02:00
Anton Popov
7bc5627912
fix typos
2023-08-29 14:21:58 +00:00
Anton Popov
02391eafe1
refactoring of reading from MergeTree tables
2023-08-29 13:40:12 +00:00
Anton Popov
6ce4a82a04
Merge pull request #53677 from VanDarkholme7/mutations_subcolumns
...
Fix bug on mutations with subcolumns of type JSON in predicates of UPDATE and DELETE queries.
2023-08-29 15:08:00 +02:00
Antonio Andelic
a0dd350e5e
Fix assert
2023-08-29 09:59:16 +00:00
János Benjamin Antal
a1a45ee905
Merge pull request #53766 from ilejn/exceptions_kafka_consumers
...
system.kafka_consumers exception info improvements
2023-08-29 10:22:26 +02:00
Antonio Andelic
8708ebd7fb
Correctly check files
2023-08-29 07:40:08 +00:00
Raúl Marín
174f7a4e63
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-28 16:27:45 +02:00
Alexander Tokmakov
d41eca1dcc
rename new method
2023-08-28 16:01:00 +02:00
Antonio Andelic
9b99f25d75
Improve schema inference
2023-08-28 13:11:52 +00:00
Robert Schulze
4c78c08847
Add test
2023-08-28 13:02:24 +00:00
Davit Vardanyan
9190fb2ba3
Fix: Deserialize
2023-08-28 16:18:12 +04:00
Igor Nikonov
a2686af496
Merge pull request #53862 from ClickHouse/cleanup-parallel-replicas-0
...
Remove unused parallel replicas coordinator in query info
2023-08-28 13:30:23 +02:00
Antonio Andelic
5a0c2ca108
Merge branch 'master' into archive-improvements-2
2023-08-28 08:34:42 +00:00
VanDarkholme7
bac257743b
Merge branch 'ClickHouse:master' into mutations_subcolumns
2023-08-28 14:10:31 +08:00
Igor Nikonov
7368bf65b3
Remove unused parallel replicas coordinator in query info
2023-08-27 22:12:39 +00:00
Victor Krasnov
87c90767d9
Merge branch 'master' into ADQM-987
2023-08-27 09:14:42 +03:00
alesapin
0e601b856f
Merge pull request #53751 from Algunenano/clang18
...
Support clang-18 (Wmissing-field-initializers)
2023-08-25 23:27:06 +02:00
Victor Krasnov
921f0c8b5d
Merge branch 'master' into ADQM-987
2023-08-25 21:20:23 +03:00
Alexander Tokmakov
32efbe77d1
Merge pull request #53811 from ClickHouse/fix_02443_2
...
Fix a bug in attach partition
2023-08-25 20:16:53 +03:00
Alexander Tokmakov
9ab545e28c
do not wait for flush on shutdown
2023-08-25 19:09:10 +02:00
Alexander Gololobov
684d5aab88
Merge pull request #53797 from ClickHouse/add_profile_event_for_lock
...
Add event for tracking data parts lock
2023-08-25 18:51:49 +02:00
Kseniia Sumarokova
34ac113af6
Merge pull request #53121 from kssenii/fix-cache-logical-error
...
Fix possible error from cache "Read unexpected size"
2023-08-25 12:16:24 +02:00
Kruglov Pavel
d80be4673b
Merge pull request #53692 from Avogar/cache-count
...
Cache number of rows in files for count in file/s3/url/hdfs/azure functions
2023-08-25 09:08:52 +02:00
Antonio Andelic
f0e800092f
Merge pull request #53796 from ClickHouse/fix-filtered-url-paths
...
Fix reading from `url` with all filtered paths
2023-08-25 08:49:33 +02:00
Yakov Olkhovskiy
415a993c91
fix fuzzers build
2023-08-24 23:22:39 +00:00
alesapin
3a289534cb
Fxi
2023-08-24 20:48:10 +02:00
Alexander Tokmakov
cbbb81f5cc
Merge pull request #53405 from canhld94/limit_concurent_background_optimize
...
Limit number of concurrent background partition optimize
2023-08-24 20:49:38 +03:00
Alexander Tokmakov
75d32bfe77
Merge pull request #53750 from ClickHouse/Add_retries_for_azure_retryable_errors
...
Add retries for azure retryable errors
2023-08-24 20:26:42 +03:00
Alexander Tokmakov
e1d135dc01
fix another bug in attach partition
2023-08-24 19:10:25 +02:00
Raúl Marín
6490213207
Merge remote-tracking branch 'blessed/master' into clang18
2023-08-24 17:35:26 +02:00
Kruglov Pavel
44db5fa992
Merge branch 'master' into cache-count
2023-08-24 17:21:18 +02:00
avogar
c4f8113a78
Fix tests
2023-08-24 15:05:45 +00:00
Robert Schulze
45d924c62c
Merge pull request #53616 from wzb5212/skip_idx_bug_fix
...
Fix number of dropped granules in EXPLAIN PLAN index=1
2023-08-24 16:46:47 +02:00
Antonio Andelic
6eb400c6ae
Fix reading from url with all filtered paths
2023-08-24 13:37:11 +00:00
alesapin
415b3826e0
Add event for tracking data parts lock
2023-08-24 15:28:38 +02:00
Ilya Golshtein
f58b4a812d
exceptions_kafka_consumers: code review suggestions are addressed
2023-08-24 13:10:21 +00:00
Arthur Passos
2bade7db08
Add global proxy setting ( #51749 )
...
* initial impl
* fix env ut
* move ut directory
* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider
* minor adjustment
* add a few tests, still incomplete
* add proxy support for url table function
* use proxy for select from url as well
* remove optional from return type, just returns empty config
* fix style
* style
* black
* ohg boy
* rm in progress file
* god pls don't let me kill anyone
* ...
* add use_aws guards
* remove hard coded s3 proxy resolver
* add concurrency-mt-unsafe
* aa
* black
* add logging back
* revert change
* imrpove code a bit
* helper functions and separate tests
* for some reason, this env test is not working..
* formatting
* :)
* clangtidy
* lint
* revert some stupid things
* small test adjusmtments
* simplify tests
* rename test
* remove extra line
* freaking style change
* simplify a bit
* fix segfault & remove an extra call
* tightly couple proxy provider with context..
* remove useless include
* rename config prefix parameter
* simplify provider a bit
* organize provider a bit
* add a few comments
* comment out proxy env tests
* fix nullptr in unit tests
* make sure old storage proxy config is properly covered without global context instance
* move a few functions from class to anonymous namespace
* fix no fallback for specific storage conf
* change API to accept http method instead of bool
* implement http/https distinction in listresolver, any still not implemented
* implement http/https distinction in remote resolver
* progress on code, improve tests and add url function working test
* use protcol instead of method for http and https
* small fix
* few more adjustments
* fix style
* black
* move enum to proxyconfiguration
* wip
* fix build
* fix ut
* delete atomicroundrobin class
* remove stale include
* add some tests.. need to spend some more time on the design..
* change design a bit
* progress
* use existing context for tests
* rename aux function and fix ut
* ..
* rename test
* try to simplify tests a bit
* simplify tests a bit more
* attempt to fix tests, accept more than one remote resolver
* use proper log id
* try waiting for resolver
* proper wait logic
* black
* empty
* address a few comments
* refactor tests
* remove old tests
* baclk
* use RAII to set/unset env
* black
* clang tidy
* fix env proxy not respecting any
* use log trace
* fix wrong logic in getRemoteREsolver
* fix wrong logic in getRemoteREsolver
* fix test
* remove unwanted code
* remove ClientConfigurationperRequest and auxilary classes
* remove unwanted code
* remove adapter test
* few adjustments and add test for s3 storage conf with new proxy settings
* black
* use chassert for context
* Add getenv comment
2023-08-24 16:07:26 +03:00
Antonio Andelic
8e1d38d377
Merge branch 'master' into archive-improvements-2
2023-08-24 13:03:36 +00:00
Raúl Marín
1c98568df9
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-24 12:22:42 +02:00
Robert Schulze
8c9fade58c
Merge remote-tracking branch 'rschu1ze/master' into skip_idx_bug_fix
2023-08-24 10:17:34 +00:00
Kseniia Sumarokova
a4f39fe134
Merge branch 'master' into fix-cache-logical-error
2023-08-24 12:00:18 +02:00
Alexander Tokmakov
011c7f7064
Revert "Planner prepare filters for analysis"
2023-08-24 12:52:03 +03:00
Kseniia Sumarokova
187706558a
Merge pull request #51622 from kssenii/system-sync-cache
...
Add `system sync filesystem cache` command
2023-08-24 11:42:32 +02:00
Ilya Golshtein
901240eede
exceptions_kafka_consumers: clang-tidy improvement
2023-08-24 08:44:08 +00:00
Ilya Golshtein
ec0c12960a
exceptions_kafka_consumers: stylecheck
2023-08-23 23:10:25 +00:00
Ilya Golshtein
e7213ef5b9
exceptions_kafka_consumers: new test, MV exceptions propagated
2023-08-23 22:46:04 +00:00
Kruglov Pavel
685ffc5856
Fix style
2023-08-23 22:51:14 +02:00
Kruglov Pavel
f7e1abd774
Merge branch 'master' into cache-count
2023-08-23 22:31:49 +02:00
Kruglov Pavel
063c986680
Merge pull request #53637 from Avogar/fast-count-from-files
...
Optimize count from files in most input formats
2023-08-23 22:17:29 +02:00
avogar
894513f6cd
Fix tests
2023-08-23 18:43:08 +00:00
Nikolai Kochetov
3a4a7d70a2
Merge pull request #52762 from kitaisreal/planner-prepare-filters-for-analysis
...
Planner prepare filters for analysis
2023-08-23 18:59:43 +02:00
Anton Kozlov
a5b6a91b54
[bloom filter] Write hashes for bloom filter into a hash set for optimal sizing
2023-08-23 14:17:55 +00:00
Victor Krasnov
8b2342c205
Minor code cleanup: remove some redundant includes of InterpreterAlterQuery.h
2023-08-23 14:16:36 +00:00
Raúl Marín
93dac0c880
Support clang-18 (Wmissing-field-initializers)
2023-08-23 15:53:45 +02:00
Smita Kulkarni
262c39329b
Updated to retry loading parts for azure TransportException
2023-08-23 15:50:11 +02:00
Smita Kulkarni
1a074ecf93
Updated to not mark part as broken for azure TransportException in loadDataPart
2023-08-23 15:33:01 +02:00
Smita Kulkarni
7fdf90ccf1
Added azure TransportException to retryable errors
2023-08-23 15:21:37 +02:00
avogar
68e3af56d4
Address comments
2023-08-23 13:19:15 +00:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count
2023-08-23 15:18:02 +02:00
Kruglov Pavel
7e362a2110
Merge branch 'master' into fast-count-from-files
2023-08-23 15:13:20 +02:00
Robert Schulze
bea651e828
Merge pull request #53657 from rschu1ze/cache-improvements
...
Expose more settings from caches
2023-08-23 14:58:45 +02:00
Robert Schulze
a093f519fc
Merge remote-tracking branch 'rschu1ze/master' into skip_idx_bug_fix
2023-08-23 12:37:46 +00:00
Robert Schulze
5dec6f19f3
Merge pull request #51325 from xinhuitian/fix-annoy-index-update
...
Do not reset Annoy index during build-up with > 1 mark
2023-08-23 14:31:06 +02:00
robot-ch-test-poll1
c22ffa6195
Merge pull request #53529 from Avogar/filter-files-all-table-functions
...
Use filter by file/path before reading in url/file/hdfs table functins
2023-08-23 14:21:23 +02:00
Robert Schulze
761268f86a
Merge remote-tracking branch 'rschu1ze/master' into skip_idx_bug_fix
2023-08-23 10:55:04 +00:00
Kruglov Pavel
e193aec583
Merge branch 'master' into fast-count-from-files
2023-08-23 12:15:34 +02:00
kssenii
325e2d3172
Merge remote-tracking branch 'upstream/master' into system-sync-cache
2023-08-23 11:44:19 +02:00
priera
fd51839b97
basic implementation
2023-08-23 11:19:01 +02:00
Antonio Andelic
83d4b819f3
Better support for reading from archives
2023-08-23 08:10:30 +00:00
xinhuitian
920762e66d
fix USearch
2023-08-23 10:54:56 +08:00
Tian Xinhui
7cf057e716
Merge branch 'ClickHouse:master' into fix-annoy-index-update
2023-08-23 10:33:14 +08:00
pufit
926533306c
Merge pull request #53668 from ClickHouse/pufit/fix_s3_threads
...
Limiting the number of parsing threads for the S3 source
2023-08-22 19:09:33 -04:00
Kruglov Pavel
a83a868acd
Update src/Storages/HDFS/StorageHDFS.cpp
...
Co-authored-by: Dmitry Novik <mrnovikd@gmail.com>
2023-08-22 19:08:04 +02:00
Robert Schulze
d5a7277a59
Better parameter name
2023-08-22 15:43:13 +00:00
pufit
e42da9411b
Fix variables names
2023-08-22 11:23:10 -04:00
Raúl Marín
abfd48bcd2
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-22 17:01:22 +02:00
Nikolai Kochetov
5ee9b56040
Do not change the snapshot after at reading from MT
2023-08-22 13:50:51 +00:00
Victor Krasnov
258f9927b4
Merge branch 'master' into ADQM-987
2023-08-22 16:49:32 +03:00
Kruglov Pavel
de960f3c35
Fix style
2023-08-22 15:22:17 +02:00
Kruglov Pavel
a34c9c3ebc
Fix build
2023-08-22 15:21:54 +02:00
Kruglov Pavel
67c5c0203b
Merge branch 'master' into fast-count-from-files
2023-08-22 15:03:48 +02:00
avogar
89b402f4e2
Fix
2023-08-22 13:01:07 +00:00
avogar
7f9e81d504
Clean up
2023-08-22 12:55:00 +00:00
alesapin
b4aaeb1a82
Fix bad log line format
2023-08-22 14:53:35 +02:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count
2023-08-22 14:42:22 +02:00
avogar
b4145aeddc
Cache number of rows in files for count in file/s3/url/hdfs/azure functions
2023-08-22 11:59:59 +00:00
Robert Schulze
f2fb7365e6
Add settings for 1. size_ratio (all caches) and 2. policy (index caches)
...
1. is an expert-level setting, default is 0.5, applies only to SLRU.
Also, I noticed that we expose cache policy settings for the mark and
the uncompresed cache but not for the index mark and the index
uncompressed cache. Changed that as well, it simplifies the code a bit.
2023-08-22 11:04:29 +00:00
Ilya Golshtein
7cd8237a42
exceptions_kafka_consumers: initial
2023-08-22 10:26:52 +00:00
Robert Schulze
acaa2eee58
Merge pull request #53556 from ClibMouse/feature/big-endian-UUID-correction
...
Change Big Endian-UUID to work the same as Little Endian-UUID
2023-08-22 11:57:44 +02:00
alesapin
1a5a9fca87
Merge pull request #53605 from irenjj/feat_53538
...
Add column 'name' to system.clusters as an alias to cluster.
2023-08-22 11:34:21 +02:00
VanDarkholme7
2638cc69d7
fix bug on DELETE and UPDATE with JSON subcolumns
2023-08-22 09:26:17 +00:00
Tian Xinhui
76016d9593
Merge branch 'master' into fix-annoy-index-update
2023-08-22 16:58:29 +08:00
pufit
9d454d9afc
Merge branch 'master' into pufit/fix_s3_threads
...
# Conflicts:
# src/Storages/StorageS3.cpp
# src/Storages/StorageS3.h
# src/Storages/StorageURL.cpp
# src/Storages/StorageURL.h
2023-08-21 21:32:15 -04:00
pufit
98a701e2c1
Limiting number of parsing threads for S3 source
2023-08-21 21:21:03 -04:00
Robert Schulze
a20ceef398
Merge pull request #53447 from davvard/master
...
USearch HNSW index for ANN search
2023-08-21 23:34:26 +02:00
Michael Kolupaev
2f4d433e69
Parquet filter pushdown
2023-08-21 14:15:52 -07:00
Nikolai Kochetov
4222ac528d
Merge branch 'master' into planner-prepare-filters-for-analysis
2023-08-21 16:48:33 +00:00
Kruglov Pavel
aebbf00a5a
Fix
2023-08-21 17:47:30 +02:00
Kruglov Pavel
a2f81bf4ea
Fix typo
2023-08-21 15:20:44 +02:00
Kruglov Pavel
dfdd26ee71
Fix build
2023-08-21 15:04:03 +02:00
Kruglov Pavel
88aee95122
Merge branch 'master' into fast-count-from-files
2023-08-21 14:46:33 +02:00
avogar
584bd57f5c
Clean up
2023-08-21 12:43:11 +00:00
avogar
47304bf7aa
Optimize count from files in most input formats
2023-08-21 12:30:52 +00:00
Kruglov Pavel
9d9bdde0e3
Merge branch 'master' into filter-files-all-table-functions
2023-08-21 14:26:59 +02:00
Kruglov Pavel
9869196021
Update VirtualColumnUtils.cpp
2023-08-21 14:22:48 +02:00
Antonio Andelic
ed15babddd
Merge pull request #53198 from ClickHouse/improve-archive-reading
...
Improve reading from archives
2023-08-21 14:11:25 +02:00
Victor Krasnov
a7813260c0
Merge branch 'master' into ADQM-987
2023-08-21 14:45:13 +03:00
Kruglov Pavel
21413636ac
Remove debug logging
2023-08-21 13:30:08 +02:00
Robert Schulze
2c330516b8
Another compile fix
2023-08-21 11:12:39 +00:00
Duc Canh Le
2c07065f81
increase the default value
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-08-21 11:04:24 +00:00
Raúl Marín
f2d3c9c281
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-21 12:14:31 +02:00
Robert Schulze
066ec559ad
Do not reset Annoy index during build-up with > 1 mark
2023-08-21 10:05:43 +00:00
王智博
8bb0f18019
bad size of mark skip idx
2023-08-20 22:57:53 +08:00
Kseniia Sumarokova
404fb0588a
Merge branch 'master' into fix-cache-logical-error
2023-08-20 16:28:13 +02:00