avogar
617cc514b7
Try to detect file format automatically during schema inference if it's unknown
2024-01-23 18:59:39 +00:00
Nikolai Kochetov
8936c8376a
Use predicate in getTaskIteratorExtension.
2024-01-02 17:14:16 +00:00
Kruglov Pavel
7e362a2110
Merge branch 'master' into fast-count-from-files
2023-08-23 15:13:20 +02:00
avogar
47304bf7aa
Optimize count from files in most input formats
2023-08-21 12:30:52 +00:00
avogar
60b0b88d50
Clean up
2023-08-17 16:59:57 +00:00
avogar
4c32097df3
Use filter by file/path before reading in url/file/hdfs table functions, reduce code duplication
2023-08-17 16:54:43 +00:00
avogar
98aa6b317f
Support reading subcolumns from file/s3/hdfs/url/azureBlobStorage table functions
2023-07-04 21:17:26 +00:00
avogar
70a8fd2c50
Fix schema inference with named collection, refactor Cluster table functions
2023-05-12 13:58:45 +00:00
avogar
2949ceced1
Fix adding structure to cluster table functions, make it better
2023-04-24 13:20:04 +00:00
avogar
447189a6ca
Better
2023-04-21 17:54:09 +00:00
avogar
944f54aadf
Finish urlCluster, refactor code, reduce code duplication
2023-04-21 17:24:37 +00:00
kssenii
13f29a7242
Better
2023-03-28 18:57:24 +02:00
kssenii
ab0dedf0c8
Simplify code around storage s3 configuration
2023-02-06 16:23:17 +01:00
avogar
117ec13c9e
Fix s3Cluster schema inference when structure from insertion table is used
2023-01-18 20:33:50 +00:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] ( #44955 )
...
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535 )""
This reverts commit b8d9066004
2023-01-09 13:30:32 +01:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference
...
Fix schema inference in s3Cluster and improve in hdfsCluster
2022-11-01 14:00:21 +01:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
avogar
3bb1161859
Better naming
2022-10-18 13:05:40 +00:00
Kruglov Pavel
3d9f46a1e7
Merge branch 'master' into s3-cluster-schema-inference
2022-10-14 22:07:54 +02:00
Alexander Tokmakov
b8d9066004
Revert "Resurrect parallel distributed insert select with s3Cluster ( #41535 )"
...
This reverts commit 860e34e760
2022-10-07 15:53:30 +02:00
Robert Schulze
78be400ac0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup
2022-10-06 12:22:43 +00:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster ( #41535 )
2022-10-06 13:47:32 +02:00
avogar
c74b5c8126
Fix schema inference in s3Cluster and improve in hdfsCluster
2022-09-30 16:59:17 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
kssenii
420ac4eb43
s3 header auth in ast
2022-09-13 15:13:28 +02:00
Alexander Tokmakov
f9f85a0e8b
Revert "Parallel distributed insert select from *Cluster table functions ( #39107 )"
...
This reverts commit d3cc234986
2022-08-24 15:17:15 +03:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions ( #39107 )
2022-08-15 12:41:17 +02:00
Amos Bird
093d315756
partition pruning for s3
2022-05-24 18:57:55 +08:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable
...
... IStorage has deleted copy ctor / assignment already
2022-05-03 09:07:08 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
...
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
previously allowed.
Hence, this change
- removes shared_ptr_helper and as a result all inherited create() methods,
- instead, Storage objects are now created using make_shared<>() by the
caller (for that to work, many constructors had to be made public), and
- all Storage classes were marked as noncopyable using boost::noncopyable.
In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
kssenii
4e50da722b
Better s3 settings
2022-04-04 16:14:56 +02:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function ( #35544 )
2022-03-29 00:46:35 +02:00
Anton Popov
a20922b2d3
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-11-09 15:36:25 +03:00
Alexey Milovidov
8b4a6a2416
Remove cruft
2021-10-28 02:10:39 +03:00
Alexey Milovidov
fe6b7c77c7
Rename "common" to "base"
2021-10-02 10:13:14 +03:00
Anton Popov
e36736b50c
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-08-02 22:52:02 +03:00
vdimir
dccc379d39
Fix use after free in AsyncDrain connection from S3Cluster
2021-07-23 10:40:03 +03:00
Anton Popov
3ed7f5a6cc
dynamic subcolumns: add snapshot for storage
2021-07-09 06:15:41 +03:00
Raúl Marín
bfc122df64
Fix some typos in Storage classes
2021-06-28 19:03:56 +02:00
Maksim Kita
67e9b85951
Merge ext into common
2021-06-16 23:28:41 +03:00
feng lv
c6f8ab9826
fix
2021-05-13 02:05:53 +00:00
Amos Bird
cd6414639e
add metadata_snapshot to getQueryProcessingStage
2021-05-11 18:12:26 +08:00
feng lv
4ffe199d39
Implement table comments
2021-04-23 12:18:23 +00:00
Nikita Mikhaylov
3954eff276
do not use cluster from config on remote replicas
2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
a15757a9c9
better renaming
2021-04-13 22:39:43 +03:00
