Commit Graph

123 Commits

Author SHA1 Message Date
Robert Schulze
e583099158
Fix build, pt. V 2022-05-04 15:50:52 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing
Parallel parsing of hive text format
2022-05-03 15:56:28 +02:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable
... IStorage has deleted copy ctor / assignment already
2022-05-03 09:07:08 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
taiyang-li
090fd72884 fix bug 2022-04-11 11:19:31 +08:00
taiyang-li
7e89f760f3 remove useless code 2022-04-09 10:43:58 +08:00
taiyang-li
70f4503ba5 use global context for cache 2022-04-09 00:28:07 +08:00
taiyang-li
cd807da838 finish test 2022-04-09 00:15:33 +08:00
taiyang-li
e319df1799 finish dev 2022-04-08 23:58:56 +08:00
taiyang-li
2c99ef0ecc refactor HiveTableMetadata 2022-04-08 23:04:24 +08:00
taiyang-li
2e6f0db825 first commit 2022-04-08 15:12:24 +08:00
taiyang-li
87507ec9e8 fix conflicts 2022-04-07 20:52:54 +08:00
taiyang-li
d7c79c3a54 merge master and solve conflicts 2022-04-07 20:48:16 +08:00
taiyang-li
e9de38c52b fix bug 2022-04-07 20:45:07 +08:00
taiyang-li
2dc420c66b rename some symbols in hivefile 2022-04-07 15:48:42 +08:00
taiyang-li
4763a39802 merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 15:45:28 +08:00
taiyang-li
046a2ba51c rename some symboles 2022-04-07 15:35:08 +08:00
taiyang-li
ad074fee91 merge use_minmax_index and solve conflict 2022-04-07 15:19:45 +08:00
taiyang-li
f02d769343 fix build error 2022-04-07 14:29:35 +08:00
taiyang-li
acc7046d54 remove some useless virtual and rename some functions in HiveFile 2022-04-07 11:46:57 +08:00
taiyang-li
df00bd214d merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 11:18:24 +08:00
taiyang-li
2ef316801c Merge branch 'master' into use_minmax_index 2022-04-07 10:53:25 +08:00
taiyang-li
0b0c8ef09e add integration tests 2022-04-06 18:47:34 +08:00
taiyang-li
acb9f1632e suppoort skip splits in orc and parquet 2022-04-06 16:40:22 +08:00
taiyang-li
43e8af697a fix code style 2022-04-06 11:41:16 +08:00
taiyang-li
38f149b533 optimize trivial count hive query 2022-04-04 15:28:26 +08:00
taiyang-li
4e2d5f1841 Merge remote-tracking branch 'bigo-sg/use_minmax_index' into optimize_trivial_hive_query 2022-04-04 10:42:28 +08:00
taiyang-li
cbfc0f6bac fix typo 2022-04-04 10:42:22 +08:00
Kseniia Sumarokova
d3b3294872
Merge pull request #35365 from bigo-sg/improve_access_type
Improve check access in table functions
2022-04-01 10:47:02 +02:00
taiyang-li
16bb4c4ad0 respect remote_url_allow_hosts for hive 2022-03-30 15:33:59 +08:00
taiyang-li
0af6fdb576 fix building 2022-03-30 11:28:21 +08:00
taiyang-li
b79cec6806 Merge branch 'use_minmax_index' of https://github.com/bigo-sg/ClickHouse into use_minmax_index 2022-03-25 23:33:49 +08:00
taiyang-li
eee8949150 fix code 2022-03-25 23:33:46 +08:00
taiyang-li
4aaa361f2e Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-25 22:48:03 +08:00
李扬
9cc528b01f
Update HiveFile.h 2022-03-23 21:57:58 +08:00
taiyang-li
ae3d55c6a2 merge master and fix conflict 2022-03-23 14:31:12 +08:00
taiyang-li
68d5b538aa fix build error 2022-03-23 11:15:42 +08:00
lgbo-ustc
967d5a8055 Merge remote-tracking branch 'ck/master' into hive_column_pruning_bug 2022-03-21 19:52:06 +08:00
taiyang-li
49b6f3dfc5 merge master and fix conflict 2022-03-21 15:05:43 +08:00
taiyang-li
bf05b94940 fix build 2022-03-21 15:03:28 +08:00
taiyang-li
7d50bd1eb3 add access type hive 2022-03-21 11:19:45 +08:00
lgbo-ustc
f7aa40af5b update codes 2022-03-21 09:25:20 +08:00
lgbo-ustc
e78cfe3b26 update codes 2022-03-18 15:07:52 +08:00
lgbo-ustc
abfaa82bca fixed hive query bugs 2022-03-15 12:01:34 +08:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
Kseniia Sumarokova
e6ee891c9c
Merge pull request #34957 from bigo-sg/hive_random_access_file_cache
Optimization for first time to read a random access readbuffer in hive
2022-03-10 11:36:22 +01:00