Anton Popov
fe5fff0347
Merge pull request #43329 from xiedeyantu/support_nested_column
...
s3 table function can support select nested column using {column_name}.{subcolumn_name}
2022-11-29 22:27:19 +01:00
xiedeyantu
304b6ebf3a
s3 table function can support select nested column using {column_name}.{subcolumn_name}
2022-11-23 23:36:12 +08:00
Antonio Andelic
9a8d82f0fc
Merge branch 'master' into fix-describe-delta-lake-hudi
2022-11-21 08:23:33 +00:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling""
2022-11-17 17:35:04 +01:00
Alexander Tokmakov
9011a18234
Revert "S3 request per second rate throttling"
2022-11-16 22:33:48 +03:00
Sergei Trifonov
8eedd1e046
Merge branch 'master' into disk-s3-throttler
2022-11-08 15:00:56 +01:00
serxa
6d5d9ff421
rename ReadWriteSettings -> RequestSettings
2022-11-08 13:48:23 +00:00
serxa
2daec0b45e
S3 request per second rate throttling + refactoring
2022-11-07 18:05:40 +00:00
Daniil Rubin
99d57325bb
Merge branch 'master' into hudi
2022-11-03 19:29:23 +00:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference
...
Fix schema inference in s3Cluster and improve in hdfsCluster
2022-11-01 14:00:21 +01:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Antonio Andelic
7c7395575f
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-10-17 11:02:36 +00:00
Kruglov Pavel
3d9f46a1e7
Merge branch 'master' into s3-cluster-schema-inference
2022-10-14 22:07:54 +02:00
avogar
c74b5c8126
Fix schema inference in s3Cluster and improve in hdfsCluster
2022-09-30 16:59:17 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Daniil Rubin
95b5d2d29d
Merge branch 'master' into hudi
2022-09-28 14:24:55 +03:00
Daniil Rubin
13e50c771c
Fix bug, use static method
2022-09-28 11:03:43 +00:00
Antonio Andelic
655bde8e3f
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-09-27 07:28:26 +00:00
Kseniia Sumarokova
9e97ca1df7
Update src/Storages/StorageS3.h
...
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2022-09-22 17:59:26 +02:00
Antonio Andelic
9fdd2f2e61
Basic implementation for S3 snapshot upload
2022-09-15 09:11:55 +00:00
Kseniia Sumarokova
2cd6ceb203
Update StorageS3.h
2022-09-13 16:30:28 +02:00
kssenii
420ac4eb43
s3 header auth in ast
2022-09-13 15:13:28 +02:00
avogar
9b1a267203
Refactor, remove TTL, add size limit, add system table and system query
2022-08-05 16:20:15 +00:00
avogar
5155262a16
Add some additional information to cache keys
2022-06-27 12:43:24 +00:00
avogar
d37ad2e6de
Implement cache for schema inference for file/s3/hdfs/url
2022-06-21 13:02:48 +00:00
Nikolai Kochetov
5b4658aa5e
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-05-30 09:47:35 +00:00
alesapin
c7b16065e1
Merge with master
2022-05-25 21:47:05 +02:00
alesapin
6f5c86e55e
Merge branch 'master' into i_object_storage
2022-05-25 20:49:01 +02:00
Nikolai Kochetov
1b85f2c1d6
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-05-25 16:27:40 +02:00
Kseniia Sumarokova
b50d4549c9
Merge pull request #37356 from amosbird/partition-prune-for-s3
...
"Partition pruning" for s3
2022-05-25 11:03:07 +02:00
Amos Bird
093d315756
partition pruning for s3
2022-05-24 18:57:55 +08:00
Nikolai Kochetov
56feef01e7
Move some resources
2022-05-20 19:49:31 +00:00
avogar
68bb07d166
Better naming
2022-05-13 18:39:19 +00:00
avogar
b17fec659a
Improve performance and memory usage for select of subset of columns for some formats
2022-05-13 13:51:28 +00:00
alesapin
5dcd25be23
Initial implementation
2022-05-12 00:04:54 +02:00
Anton Popov
e911900054
remove last mentions of data streams
2022-05-09 19:15:24 +00:00
Kruglov Pavel
c6e26f6128
Merge pull request #36434 from Avogar/fix-s3-cluster
...
Fix bug in s3Cluster schema inference
2022-05-05 13:08:49 +02:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable
...
... IStorage has deleted copy ctor / assignment already
2022-05-03 09:07:08 +02:00
Kruglov Pavel
c3570b9f7e
Merge branch 'master' into fix-s3-cluster
2022-05-02 14:15:21 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
...
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
previously allowed.
Hence, this change
- removes shared_ptr_helper and as a result all inherited create() methods,
- instead, Storage objects are now created using make_shared<>() by the
caller (for that to work, many constructors had to be made public), and
- all Storage classes were marked as noncopyable using boost::noncopyable.
In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common
2022-04-29 10:26:35 +08:00
Kruglov Pavel
7fb0f2284d
Merge branch 'master' into fix-s3-cluster
2022-04-21 13:39:52 +02:00
Sergei Trifonov
34842f209c
merge master and fix conflicts
2022-04-20 18:35:56 +02:00
avogar
9a81b67ba5
Fix bug in s3Cluster schema inference
2022-04-19 18:23:04 +00:00
kssenii
ef2b238444
Fix
2022-04-08 12:30:24 +02:00
Sergei Trifonov
01425df758
fix build
2022-04-07 16:07:12 +02:00
Sergei Trifonov
9c2804b924
merge from master + resolve conflicts
2022-04-07 11:22:01 +02:00
kssenii
4e50da722b
Better s3 settings
2022-04-04 16:14:56 +02:00
kssenii
98e85dd68e
Fix build
2022-03-30 20:27:04 +02:00
kssenii
099fa0e668
Merge master
2022-03-30 17:33:32 +02:00
