192 Commits

Author	SHA1	Message	Date
taiyang-li	2d6eb40879	fix style	2021-12-27 15:31:24 +08:00
lgbo-ustc	078521496a	modifications based on pr review	2021-12-27 15:10:48 +08:00
lgbo-ustc	ccc39bad80	move IRemoteFileMetadata info Storages/Cache	2021-12-23 15:56:33 +08:00
lgbo-ustc	d205f9ecab	codes refactor	2021-12-23 11:50:26 +08:00
taiyang-li	cff3c20742	Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table	2021-12-21 15:01:23 +08:00
taiyang-li	2fd63f9381	rename some symboles	2021-12-21 15:00:47 +08:00
taiyang-li	40aa359201	refactor storage hive	2021-12-21 14:44:31 +08:00
lgbo-ustc	2e2354be40	1. add tryDel and getOrTrySet in LRUCache ... 2. fixed bugs in LRUCache for evicting elements	2021-12-21 10:39:59 +08:00
lgbo-ustc	71353ceede	refactor metadata class factory	2021-12-20 20:33:41 +08:00
taiyang-li	7192458601	fix code typo	2021-12-20 18:10:15 +08:00
taiyang-li	a0b90fa9ec	fix bugs and add some docs	2021-12-14 18:44:15 +08:00
taiyang-li	ca3f7425a4	fix code	2021-12-14 17:37:31 +08:00
taiyang-li	02b877c978	move remote buffer from src/IO to src/Storages	2021-12-10 10:27:53 +08:00
taiyang-li	0eef092ded	fix building	2021-12-09 16:57:33 +08:00
lgbo-ustc	dd70209623	fixed code style	2021-12-06 18:19:36 +08:00
lgbo-ustc	5913d67553	fixed code style	2021-12-06 16:51:49 +08:00
lgbo-ustc	3c56a5deac	refector ... 1. Make the lru cache pilicy in RemoteReadBufferCache into Common/UnreleasableLRUCache.h 2. If try to use local cached file fail, the RemoteReadBuffer::create() return the original ReadBuffer 3. Remove LocalFileReader 4. RemoteCacheController does not hold a context now, and the download process starts in RemoteReadBufferCache once a new RemoteCacheController is create successly 5. Make an abstract class RemoteFileMetaDataBase for descripting a remote file meta data. 6. Remote file meta data need to a version, can cache multi-version for a file now	2021-12-06 16:17:32 +08:00
lgbo-ustc	01940c3f01	coding refinement ... 1. make remote file meta data be a abstract class 2. make cache controller download process be started by RemoteReadBufferCache	2021-12-06 10:30:59 +08:00
taiyang-li	c678c8101e	fix some bugs	2021-12-04 16:41:35 +08:00
taiyang-li	9ec8272186	refactor hive text input format	2021-12-02 16:14:25 +08:00
lgbo-ustc	aaa5d8f002	rewrite thread pool in remote file cache	2021-12-01 10:45:25 +08:00
taiyang-li	d213500a3e	remove blank at end of line	2021-11-30 18:23:24 +08:00
lgbo-ustc	b023dd34c0	fixed code style	2021-11-30 17:11:58 +08:00
taiyang-li	440fa9b69c	implement getMissingValues for ORC/Parquet/Arrow	2021-11-30 15:44:59 +08:00
taiyang-li	cacf516e3e	calculate column value by default expression & apply defaults_for_omitted_fields_ in ArrowColumnToCHColumn	2021-11-30 14:52:26 +08:00
taiyang-li	d0546f5c9c	apply swithc use_local_cache_for_remote_fs	2021-11-29 18:39:50 +08:00
taiyang-li	03e7f0f6f7	extract hivemetastore client from context	2021-11-29 18:23:57 +08:00
taiyang-li	c7a0fe467f	fix code stylke	2021-11-29 16:22:43 +08:00
taiyang-li	fd4462db64	fix code style	2021-11-29 13:01:03 +08:00
taiyang-li	e67407b5c5	fix code style	2021-11-29 11:30:11 +08:00
taiyang-li	08cc4306b6	fix typos	2021-11-27 18:56:43 +08:00
taiyang-li	4d750c9dc5	fix code typo	2021-11-24 17:52:02 +08:00
taiyang-li	13226c9bbf	fix code typo	2021-11-24 16:35:11 +08:00
taiyang-li	f9edfcec61	fix build error	2021-11-22 17:31:53 +08:00
taiyang-li	d22854e82b	add compile swiych USE_HIVE	2021-11-22 10:52:10 +08:00
lgbo-ustc	a66a23c2e7	code modification based on the pr comment	2021-11-12 13:17:58 +08:00
taiyang-li	deef4d4dbe	add options read_bool_as_uint8 when parse csv	2021-11-11 11:49:54 +08:00
taiyang-li	c47bb97fce	compile storage hive only USE_PARQUET AND USE_ORC enalbed	2021-11-09 20:20:45 +08:00
taiyang-li	b042e2b8f8	fix pvs check	2021-11-09 14:38:46 +08:00
taiyang-li	bb0feaa191	remove defined(ARCADIA_BUILD)	2021-11-08 18:40:41 +08:00
taiyang-li	dd98fc74b6	fix typos	2021-11-06 11:01:58 +08:00
taiyang-li	7b5d7a0f02	add missing source files	2021-11-06 10:31:15 +08:00