thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 18:02:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
115,148 Commits 9,328 Branches 1,783 Tags 2.1 GiB
a91c793684
Commit Graph

256 Commits

Author SHA1 Message Date
SmitaRKulkarni
a91c793684
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-18 09:24:25 +02:00
Smita Kulkarni
fd58eac75a Fixed max_threads datatype issue for builds 2023-05-13 10:22:37 +02:00
Smita Kulkarni
792565d858 Updated to ULL 2023-05-12 17:23:37 +02:00
Smita Kulkarni
ef1100bb90 Added include to fix build issue 2023-05-11 14:44:36 +02:00
SmitaRKulkarni
5c030c428c
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-10 09:51:38 +02:00
Alexey Milovidov
a2c4b8e23d Disable mmap for server 2023-05-10 03:16:52 +02:00
Michael Kolupaev
3bd1489f18 Propagate input_format_parquet_preserve_order to parallelizeOutputAfterReading() 2023-05-05 04:20:27 +00:00
Michael Kolupaev
eb3b774ad0 Better control over Parquet row group size 2023-05-04 14:59:55 -07:00
Smita Kulkarni
8205398f31 Fixed comment 2023-05-02 16:31:39 +02:00
Smita Kulkarni
a5d47ea489 Fixed build issues 2023-04-30 19:01:06 +02:00
Smita Kulkarni
307aa127d4 Updated to calculate and send max_parsing_threads 2023-04-25 13:27:20 +02:00
Smita Kulkarni
b70878aa0e Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-04-24 19:39:36 +02:00
Igor Nikonov
8603807b57 Use generic way to parallelize output for file() 
+ disable parallelization for storage Null
 2023-04-15 12:35:24 +00:00
Igor Nikonov
1187534545 Simpler way to resize pipeline 2023-04-09 21:26:39 +00:00
Igor Nikonov
78038a3c2c Fix: do not resize pipeline when there is no files to process (globs expands to empty set) 2023-04-07 11:34:04 +00:00
Igor Nikonov
96213fa464 Fix header 2023-04-06 22:17:09 +00:00
Igor Nikonov
2e139c21d2 Parallel reading in FROM file() 2023-04-06 21:57:03 +00:00
SmitaRKulkarni
d9c67a3380
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-03-30 09:35:03 +02:00
Azat Khuzhin
33b13549ad Separate out DirectoryMonitorSource as DistributedAsyncInsertSource 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Smita Kulkarni
3e9ab1276a Updated to reduce parallel parsing based on number of files - When reading from multiple files reduce parallel parsing 2023-02-22 14:56:44 +01:00
Smita Kulkarni
3a6dea5e16 When reading from multiple files displace parallel parsing 
Implementation:
* Added a new parameter to getInput & getInputFormat to disable parallel parsing.
* Currently this is used only by StorageFile as we have not seen degradation for other storages reading from multiple paths.
 2023-02-21 17:03:00 +01:00
Sergei Trifonov
0d1ea05ff6
Merge pull request #45007 from ClickHouse/cancellable-mutex-integration 
Fast shared mutex integration
 2023-01-25 11:15:46 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Sergei Trifonov
0fbfa17863
Merge branch 'master' into cancellable-mutex-integration 2023-01-23 12:44:09 +01:00
Anton Popov
f40fd7a151
Add checks for compilation of regexps (#45356) 2023-01-17 23:46:04 +01:00
serxa
693489a8ad review fixes 2023-01-12 15:51:04 +00:00
Kruglov Pavel
29240ef380
Merge pull request #43927 from pufit/mmap-for-storage-file 
Added mmap for StorageFile
 2023-01-11 21:25:02 +01:00
Kruglov Pavel
ce6962614d
Merge branch 'master' into mmap-for-storage-file 2023-01-10 17:34:01 +01:00
Maksim Kita
fbba28b31e Analyzer aggregation without column fix 2023-01-10 16:49:55 +01:00
pufit
2d942af7b4 Fix codestyle, fix test. 2022-12-16 11:55:50 -05:00
pufit
b7df684762 Enum settings, fix else branch. 2022-12-15 18:08:19 -05:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines 2022-12-15 18:44:35 +01:00
pufit
5c52f26823 ya fix. 2022-12-12 00:39:08 -05:00
pufit
6979dc9f2f dummy fix, additional test 2022-12-11 17:36:30 -05:00
pufit
1d6e77a29a Move reader selection logic back to StorageFile. 2022-12-11 16:15:41 -05:00
pufit
e38a93c45a Fix UB, fix test. 2022-12-10 22:26:07 -05:00
pufit
2d87cc1a6c Add storage_file_read_method setting. 2022-12-08 18:02:29 -05:00
pufit
76401ad0b9 Test and codestyle fix. 2022-12-07 23:17:10 -05:00
pufit
9b46baa17d Rewrite StorageFile buffer creation with createReadBufferFromFileBase. 
Add file descriptor support for `createReadBufferFromFileBase`.
Fix file_size overflow in `createReadBufferFromFileBase`.
Fix `MMapReadBufferFromFileWithCache` file_size definition.
 2022-12-07 22:31:32 -05:00
pufit
084e465d84 Use mmap only on regular files. 2022-12-04 23:39:23 -05:00
pufit
bc7a76a486 Added mmap for StorageFile 2022-12-04 17:27:28 -05:00
kssenii
5e01441f61 Show progress bar while reading from s3 table function 2022-11-21 17:56:02 +01:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines 2022-11-03 13:22:06 +01:00
avogar
8e13d1f1ec Improve and refactor Kafka/StorageMQ/NATS and data formats 2022-10-28 16:41:10 +00:00
SmitaRKulkarni
96c8260230
Merge branch 'master' into 36316_Support_glob_for_recursive_directory_traversal 2022-10-24 18:34:19 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Smita Kulkarni
91433e5b9c Added ** glob support for recursive directory traversal to filesystem and S3. 
Implementation:
* Updated parseGlob to not add ‘/‘ restriction when ** is used.
* Updated S3 & filesystem to fetch files and not use regex match if glob is **.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02459_glob_for_recursive_directory_traversal.sh
 2022-10-17 09:04:25 +02:00
Alexey Milovidov
ab4db2d0c4 Fix 5/6 of trash 2022-09-19 08:50:53 +02:00
avogar
5ab87f1da4 Small refactoring 2022-08-19 16:42:23 +00:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00