Raúl Marín
48ab0721a8
Unify lightweight mutation control
2024-04-02 11:52:14 +02:00
Alexey Milovidov
a56c113c83
Fix for clang-19
2024-03-22 22:12:59 +01:00
Anton Popov
95611889dc
fix tests
2024-03-01 18:44:33 +00:00
Anton Popov
9f8ac76345
Merge remote-tracking branch 'upstream/master' into HEAD
2024-02-28 14:42:37 +00:00
Kruglov Pavel
46a6b84a5a
Merge branch 'master' into auto-format-detection
2024-01-25 22:11:07 +01:00
Maksim Kita
2a327107b6
Updated implementation
2024-01-25 14:31:49 +03:00
avogar
617cc514b7
Try to detect file format automatically during schema inference if it's unknown
2024-01-23 18:59:39 +00:00
Nikolai Kochetov
9c25cb6692
Cleanup
2024-01-02 18:08:04 +00:00
Nikolai Kochetov
c808b03e55
Remove unneeded code
2024-01-02 17:27:33 +00:00
Nikolai Kochetov
5521e5d9b1
Refactor StorageHDFS and StorageFile virtual columns filtering
2023-12-29 15:58:01 +00:00
avogar
ee7af95bc0
Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union
2023-12-08 20:29:28 +00:00
avogar
4d9a1b50f9
Add information about new _size virtual column in file/s3/url/hdfs/azure table functions
2023-11-28 18:15:07 +00:00
avogar
007353a2dd
Add _size virtual column to s3/file/hdfs/url/azureBlobStorage engines
2023-11-22 18:12:36 +00:00
avogar
6934e27e8b
Add union mode for schema inference to infer union schema of files with different schemas
2023-10-20 20:46:41 +00:00
avogar
2d8f33bfa2
Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header
2023-09-11 14:55:37 +00:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count
2023-08-23 15:18:02 +02:00
robot-ch-test-poll1
c22ffa6195
Merge pull request #53529 from Avogar/filter-files-all-table-functions
...
Use filter by file/path before reading in url/file/hdfs table functins
2023-08-23 14:21:23 +02:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count
2023-08-22 14:42:22 +02:00
avogar
b4145aeddc
Cache number of rows in files for count in file/s3/url/hdfs/azure functions
2023-08-22 11:59:59 +00:00
Michael Kolupaev
2f4d433e69
Parquet filter pushdown
2023-08-21 14:15:52 -07:00
avogar
4c32097df3
Use filter by file/path before reading in url/file/hdfs table functions, reduce code duplication
2023-08-17 16:54:43 +00:00
Anton Popov
ff137773e7
Merge branch 'master' into formats-with-subcolumns
2023-08-02 15:24:56 +02:00
robot-ch-test-poll1
abc48a0b73
Merge pull request #52627 from ClickHouse/fix_build_with_clang15
...
Fix build with clang-15
2023-07-27 09:49:29 +02:00
Alexander Tokmakov
74f3e76b18
fix build with clang-15
2023-07-26 17:15:28 +02:00
Kruglov Pavel
0d34e97dbe
Merge branch 'master' into formats-with-subcolumns
2023-07-26 13:30:35 +02:00
avogar
98aa6b317f
Support reading subcolumns from file/s3/hdfs/url/azureBlobStorage table functions
2023-07-04 21:17:26 +00:00
avogar
c679dd400e
Make better
2023-06-23 13:43:40 +00:00
avogar
cf082f2f9a
Use read_bytes/total_bytes_to_read for progress bar in s3/file/url/... table functions
2023-06-22 17:24:43 +00:00
avogar
3209ebe34b
Improve progress bar for file/s3/hdfs/url table functions. Step 1
2023-06-16 15:51:18 +00:00
Kruglov Pavel
bf28074d32
Merge branch 'master' into allow-skip-empty-files
2023-06-08 12:36:18 +02:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed ( #50663 )
...
* Correctly disable async insert when it's not used
* Better
* Add comment
* Better
* Fix tests
---------
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
avogar
d4efbbfbd3
Allow to skip empty files in file/s3/url/hdfs table functions
2023-05-30 19:32:24 +00:00
Azat Khuzhin
79b83c4fd2
Remove superfluous includes of logger_userful.h from headers
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 17:59:30 +02:00
Anton Popov
5c0307bc6a
fix race in StorageURL and StorageHDFS
2023-01-24 12:34:43 +00:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Azat Khuzhin
4e3135383b
Fix clang-tidy warnings (from clang-15)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-10 21:38:35 +02:00
avogar
9b1a267203
Refactor, remove TTL, add size limit, add system table and system query
2022-08-05 16:20:15 +00:00
avogar
ee54c4f9b7
Add some fixes and add settings in docs
2022-06-30 12:41:56 +00:00
avogar
5155262a16
Add some additional information to cache keys
2022-06-27 12:43:24 +00:00
avogar
d37ad2e6de
Implement cache for schema inference for file/s3/hdfs/url
2022-06-21 13:02:48 +00:00
Nikolai Kochetov
5b4658aa5e
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-05-30 09:47:35 +00:00
Nikolai Kochetov
1b85f2c1d6
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-05-25 16:27:40 +02:00
msaf1980
fda6ddeffa
cleanup StorageHDFS (unused variables)
2022-05-25 19:23:05 +05:00
Nikolai Kochetov
56feef01e7
Move some resources
2022-05-20 19:49:31 +00:00
avogar
68bb07d166
Better naming
2022-05-13 18:39:19 +00:00
avogar
b17fec659a
Improve performance and memory usage for select of subset of columns for some formats
2022-05-13 13:51:28 +00:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable
...
... IStorage has deleted copy ctor / assignment already
2022-05-03 09:07:08 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
...
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
previously allowed.
Hence, this change
- removes shared_ptr_helper and as a result all inherited create() methods,
- instead, Storage objects are now created using make_shared<>() by the
caller (for that to work, many constructors had to be made public), and
- all Storage classes were marked as noncopyable using boost::noncopyable.
In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common
2022-04-29 10:26:35 +08:00
