thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
125,433 Commits 9,326 Branches 1,783 Tags 2.1 GiB
5595d2d20b
Commit Graph

213 Commits

Author SHA1 Message Date
taiyang-li
70f4503ba5 use global context for cache 2022-04-09 00:28:07 +08:00
taiyang-li
cd807da838 finish test 2022-04-09 00:15:33 +08:00
taiyang-li
e319df1799 finish dev 2022-04-08 23:58:56 +08:00
taiyang-li
2c99ef0ecc refactor HiveTableMetadata 2022-04-08 23:04:24 +08:00
taiyang-li
2e6f0db825 first commit 2022-04-08 15:12:24 +08:00
taiyang-li
87507ec9e8 fix conflicts 2022-04-07 20:52:54 +08:00
taiyang-li
d7c79c3a54 merge master and solve conflicts 2022-04-07 20:48:16 +08:00
taiyang-li
e9de38c52b fix bug 2022-04-07 20:45:07 +08:00
taiyang-li
2dc420c66b rename some symbols in hivefile 2022-04-07 15:48:42 +08:00
taiyang-li
4763a39802 merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 15:45:28 +08:00
taiyang-li
046a2ba51c rename some symboles 2022-04-07 15:35:08 +08:00
taiyang-li
ad074fee91 merge use_minmax_index and solve conflict 2022-04-07 15:19:45 +08:00
taiyang-li
f02d769343 fix build error 2022-04-07 14:29:35 +08:00
taiyang-li
acc7046d54 remove some useless virtual and rename some functions in HiveFile 2022-04-07 11:46:57 +08:00
taiyang-li
df00bd214d merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 11:18:24 +08:00
taiyang-li
2ef316801c Merge branch 'master' into use_minmax_index 2022-04-07 10:53:25 +08:00
taiyang-li
0b0c8ef09e add integration tests 2022-04-06 18:47:34 +08:00
taiyang-li
acb9f1632e suppoort skip splits in orc and parquet 2022-04-06 16:40:22 +08:00
taiyang-li
43e8af697a fix code style 2022-04-06 11:41:16 +08:00
taiyang-li
38f149b533 optimize trivial count hive query 2022-04-04 15:28:26 +08:00
taiyang-li
4e2d5f1841 Merge remote-tracking branch 'bigo-sg/use_minmax_index' into optimize_trivial_hive_query 2022-04-04 10:42:28 +08:00
taiyang-li
cbfc0f6bac fix typo 2022-04-04 10:42:22 +08:00
Kseniia Sumarokova
d3b3294872
Merge pull request #35365 from bigo-sg/improve_access_type 
Improve check access in table functions
 2022-04-01 10:47:02 +02:00
taiyang-li
16bb4c4ad0 respect remote_url_allow_hosts for hive 2022-03-30 15:33:59 +08:00
taiyang-li
0af6fdb576 fix building 2022-03-30 11:28:21 +08:00
taiyang-li
b79cec6806 Merge branch 'use_minmax_index' of https://github.com/bigo-sg/ClickHouse into use_minmax_index 2022-03-25 23:33:49 +08:00
taiyang-li
eee8949150 fix code 2022-03-25 23:33:46 +08:00
taiyang-li
4aaa361f2e Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-25 22:48:03 +08:00
李扬
9cc528b01f
Update HiveFile.h 2022-03-23 21:57:58 +08:00
taiyang-li
ae3d55c6a2 merge master and fix conflict 2022-03-23 14:31:12 +08:00
taiyang-li
68d5b538aa fix build error 2022-03-23 11:15:42 +08:00
lgbo-ustc
967d5a8055 Merge remote-tracking branch 'ck/master' into hive_column_pruning_bug 2022-03-21 19:52:06 +08:00
taiyang-li
49b6f3dfc5 merge master and fix conflict 2022-03-21 15:05:43 +08:00
taiyang-li
bf05b94940 fix build 2022-03-21 15:03:28 +08:00
taiyang-li
7d50bd1eb3 add access type hive 2022-03-21 11:19:45 +08:00
lgbo-ustc
f7aa40af5b update codes 2022-03-21 09:25:20 +08:00
lgbo-ustc
e78cfe3b26 update codes 2022-03-18 15:07:52 +08:00
lgbo-ustc
abfaa82bca fixed hive query bugs 2022-03-15 12:01:34 +08:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
Kseniia Sumarokova
e6ee891c9c
Merge pull request #34957 from bigo-sg/hive_random_access_file_cache 
Optimization for first time to read a random access readbuffer in hive
 2022-03-10 11:36:22 +01:00
Kseniia Sumarokova
1eb2bae792
Merge pull request #34954 from bigo-sg/hive_read_columns_pruning 
read columns pruning for hive
 2022-03-08 10:17:24 +01:00
lgbo-ustc
256e92ffee Merge remote-tracking branch 'ck/master' into hive_random_access_file_cache 2022-03-08 14:14:40 +08:00
lgbo-ustc
a8cfc2458a update codes 2022-03-08 11:55:15 +08:00
Kseniia Sumarokova
5511f2f6e6
Merge pull request #34940 from bigo-sg/hive_client_connection_pool 
Use connection pool in HiveMetastoreClient
 2022-03-07 17:14:56 +01:00
Kseniia Sumarokova
28b9ec01c0
Merge pull request #34945 from bigo-sg/hive_bug_fixed 
unexpected result when use `in` in hive query
 2022-03-07 17:13:11 +01:00
lgbo-ustc
8ae5296ee8 fixed compile errors 2022-03-07 17:26:48 +08:00
lgbo-ustc
cfeedd2cb5 fixed code style 2022-03-07 12:28:31 +08:00
lgbo-ustc
c37eedd887 update codes 2022-03-07 10:30:54 +08:00
lgbo-ustc
75a50a30c4 update codes 2022-03-07 09:43:53 +08:00
lgbo-ustc
d907b70cc4 update codes: get actual read block 2022-03-07 09:26:05 +08:00
lgbo-ustc
f4d8fb46c5 update codes 2022-03-07 09:26:05 +08:00
lgbo-ustc
62c1bd5ae9 hive read columns pruning 2022-03-07 09:26:05 +08:00
Anton Popov
c1fdcf7a64 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-01 20:21:39 +03:00
lgbo-ustc
ca470e1b94 lazy initialization about getting hive metadata in HiveStorage 2022-03-01 19:04:44 +08:00
lgbo-ustc
5ed41bda9b fixed code style 2022-03-01 17:20:32 +08:00
lgbo-ustc
91a45d799e optimization for first time to read a random access readbuffer 2022-03-01 15:22:07 +08:00
lgbo-ustc
99cd25d70e add new table function: hive() 2022-02-28 20:51:33 +08:00
lgbo-ustc
6473767c99 fixed code style 2022-02-28 17:10:56 +08:00
lgbo-ustc
5885cfd869 fixed bug : unexpected result when using in clause for filtering partitions 2022-02-28 16:47:50 +08:00
lgbo-ustc
c5e02be44e fixed code-style 2022-02-28 15:22:54 +08:00
lgbo-ustc
2176d74cd1 Use connection pool in HiveMetastoreClient 
1. remove lock for hive metastore client access
2. auo reconnect when connection is broken
 2022-02-28 15:11:38 +08:00
taiyang-li
a4baec6d26 fix building 2022-02-16 15:12:43 +08:00
taiyang-li
afcb295273 fix compile error 2022-02-16 14:51:56 +08:00
taiyang-li
f19f0d847f fix code style 2022-02-16 12:23:06 +08:00
taiyang-li
dec083ab44 add minmax index for hivengine; remove libhdfspp dependency for hive engine 2022-02-16 12:19:16 +08:00
Anton Popov
18940b8637 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-09 23:38:38 +03:00
feng lv
6325d4d9b0 continue of #34317 
fix

fix
 2022-02-06 08:59:17 +00:00
Anton Popov
e8ce091e68 Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-21 20:11:18 +03:00
lgbo-ustc
717871d9ce code style 
1. remove unused error codes
 2022-01-04 17:21:29 +08:00
lgbo-ustc
45c86757bf change LRUCache to LRUResourceCache 2022-01-04 15:19:09 +08:00
taiyang-li
8e675e9e0a fix building 2022-01-04 15:06:19 +08:00
taiyang-li
2d6eb40879 fix style 2021-12-27 15:31:24 +08:00
lgbo-ustc
078521496a modifications based on pr review 2021-12-27 15:10:48 +08:00
lgbo-ustc
ccc39bad80 move IRemoteFileMetadata info Storages/Cache 2021-12-23 15:56:33 +08:00
lgbo-ustc
d205f9ecab codes refactor 2021-12-23 11:50:26 +08:00
taiyang-li
cff3c20742 Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table 2021-12-21 15:01:23 +08:00
taiyang-li
2fd63f9381 rename some symboles 2021-12-21 15:00:47 +08:00
taiyang-li
40aa359201 refactor storage hive 2021-12-21 14:44:31 +08:00
lgbo-ustc
2e2354be40 1. add tryDel and getOrTrySet in LRUCache 
2. fixed bugs in LRUCache for evicting elements
 2021-12-21 10:39:59 +08:00
lgbo-ustc
71353ceede refactor metadata class factory 2021-12-20 20:33:41 +08:00
taiyang-li
7192458601 fix code typo 2021-12-20 18:10:15 +08:00
taiyang-li
a0b90fa9ec fix bugs and add some docs 2021-12-14 18:44:15 +08:00
taiyang-li
ca3f7425a4 fix code 2021-12-14 17:37:31 +08:00
taiyang-li
02b877c978 move remote buffer from src/IO to src/Storages 2021-12-10 10:27:53 +08:00
taiyang-li
0eef092ded fix building 2021-12-09 16:57:33 +08:00
lgbo-ustc
dd70209623 fixed code style 2021-12-06 18:19:36 +08:00
lgbo-ustc
5913d67553 fixed code style 2021-12-06 16:51:49 +08:00
lgbo-ustc
3c56a5deac refector 
1. Make the lru cache pilicy in RemoteReadBufferCache into Common/UnreleasableLRUCache.h
2. If try to use local cached file fail, the RemoteReadBuffer::create() return the original ReadBuffer
3. Remove LocalFileReader
4. RemoteCacheController does not hold a context now, and the download process starts in RemoteReadBufferCache
   once a new RemoteCacheController is create successly
5. Make an abstract class RemoteFileMetaDataBase for descripting a remote file meta data.
6. Remote file meta data need to  a version, can cache multi-version for a file now
 2021-12-06 16:17:32 +08:00
lgbo-ustc
01940c3f01 coding refinement 
1. make remote file meta data be a abstract class
2. make cache controller download process be started by RemoteReadBufferCache
 2021-12-06 10:30:59 +08:00
taiyang-li
c678c8101e fix some bugs 2021-12-04 16:41:35 +08:00
taiyang-li
9ec8272186 refactor hive text input format 2021-12-02 16:14:25 +08:00
lgbo-ustc
aaa5d8f002 rewrite thread pool in remote file cache 2021-12-01 10:45:25 +08:00
taiyang-li
d213500a3e remove blank at end of line 2021-11-30 18:23:24 +08:00
lgbo-ustc
b023dd34c0 fixed code style 2021-11-30 17:11:58 +08:00
taiyang-li
440fa9b69c implement getMissingValues for ORC/Parquet/Arrow 2021-11-30 15:44:59 +08:00
taiyang-li
cacf516e3e calculate column value by default expression & apply defaults_for_omitted_fields_ in ArrowColumnToCHColumn 2021-11-30 14:52:26 +08:00
taiyang-li
d0546f5c9c apply swithc use_local_cache_for_remote_fs 2021-11-29 18:39:50 +08:00
taiyang-li
03e7f0f6f7 extract hivemetastore client from context 2021-11-29 18:23:57 +08:00
taiyang-li
c7a0fe467f fix code stylke 2021-11-29 16:22:43 +08:00
taiyang-li
fd4462db64 fix code style 2021-11-29 13:01:03 +08:00
taiyang-li
e67407b5c5 fix code style 2021-11-29 11:30:11 +08:00
taiyang-li
08cc4306b6 fix typos 2021-11-27 18:56:43 +08:00
taiyang-li
4d750c9dc5 fix code typo 2021-11-24 17:52:02 +08:00
taiyang-li
13226c9bbf fix code typo 2021-11-24 16:35:11 +08:00
taiyang-li
f9edfcec61 fix build error 2021-11-22 17:31:53 +08:00
taiyang-li
d22854e82b add compile swiych USE_HIVE 2021-11-22 10:52:10 +08:00
lgbo-ustc
a66a23c2e7 code modification based on the pr comment 2021-11-12 13:17:58 +08:00
taiyang-li
deef4d4dbe add options read_bool_as_uint8 when parse csv 2021-11-11 11:49:54 +08:00
taiyang-li
c47bb97fce compile storage hive only USE_PARQUET AND USE_ORC enalbed 2021-11-09 20:20:45 +08:00
taiyang-li
b042e2b8f8 fix pvs check 2021-11-09 14:38:46 +08:00
taiyang-li
bb0feaa191 remove defined(ARCADIA_BUILD) 2021-11-08 18:40:41 +08:00
taiyang-li
dd98fc74b6 fix typos 2021-11-06 11:01:58 +08:00
taiyang-li
7b5d7a0f02 add missing source files 2021-11-06 10:31:15 +08:00