162 Commits

Author	SHA1	Message	Date
avogar	78069765bb	Try to remove code duplication	2022-05-20 13:20:52 +00:00
avogar	2d4b4b9008	Fix inserting defaults for missing values in columnar formats	2022-05-16 14:19:44 +00:00
avogar	02bd5f6542	Finalize write buffers in case of exception	2022-05-06 17:30:18 +00:00
Robert Schulze	330212e0f4	Remove inherited create() method + disallow copying ... The original motivation for this commit was that shared_ptr_helper used std::shared_ptr<>() which does two heap allocations instead of make_shared<>() which does a single allocation. Turned out that 1. the affected code (--> Storages/) is not on a hot path (rendering the performance argument moot ...) 2. yet copying Storage objects is potentially dangerous and was previously allowed. Hence, this change - removes shared_ptr_helper and as a result all inherited create() methods, - instead, Storage objects are now created using make_shared<>() by the caller (for that to work, many constructors had to be made public), and - all Storage classes were marked as noncopyable using boost::noncopyable. In sum, we are (likely) not making things faster but the code becomes cleaner and harder to misuse.	2022-05-02 08:46:52 +02:00
Amos Bird	4a5e4274f0	base should not depend on Common	2022-04-29 10:26:35 +08:00
Kruglov Pavel	56cfa55bd5	Fix style	2022-04-20 16:35:57 +02:00
avogar	f31f019252	Fix	2022-04-19 19:25:41 +00:00
avogar	1f252cedfe	Make better	2022-04-19 19:16:47 +00:00
avogar	1c065f8c7a	Some refactoring around schema inference with globs	2022-04-13 17:02:48 +00:00
Antonio Andelic	84eef61d17	Pull under reader mutex	2022-04-06 06:39:56 +00:00
Antonio Andelic	a3ad99be03	Create CallbackRunner in Storage	2022-03-30 11:24:50 +00:00
Antonio Andelic	1173eab15e	Use same IOThreadPool for URL and S3	2022-03-30 08:15:20 +00:00
avogar	ecc57b436c	Check remote_url_allow_hosts before schema inference in URL engine	2022-03-25 13:30:45 +00:00
Antonio Andelic	103a3fa140	Merge branch 'master' into parallel-downloading-url-engine	2022-03-17 10:11:58 +00:00
Antonio Andelic	44de5a581c	Fix split build	2022-03-17 10:09:16 +00:00
Antonio Andelic	cc623bbf5e	Fix redirect_url tests	2022-03-17 08:03:22 +00:00
Antonio Andelic	0979155f2f	Address PR comments	2022-03-16 15:45:17 +00:00
Anton Popov	0ba78c3c3a	Merge remote-tracking branch 'upstream/master' into HEAD	2022-03-16 15:28:09 +00:00
Antonio Andelic	84f86bf3da	Limit number of bytes read per worker	2022-03-16 12:34:14 +00:00
Antonio Andelic	2291b7c1e1	Merge branch 'master' into parallel-downloading-url-engine	2022-03-16 09:59:52 +00:00
Nikolai Kochetov	97aa6c82ce	Reload remote_url_allow_hosts after config update.	2022-03-15 13:00:31 +00:00
Antonio Andelic	556fe2bcc5	Don't send RANGE with HEAD call	2022-03-14 12:46:22 +00:00
Antonio Andelic	9dda2863d3	Split download threads when multiple URLs are used	2022-03-14 09:27:09 +00:00
Antonio Andelic	f5d3a8a31d	Polishing	2022-03-11 13:38:19 +00:00
Antonio Andelic	28e9508c4e	Improve shared pool and add settings	2022-03-11 08:33:34 +00:00
Antonio Andelic	cce318273b	Use a shared IO thread pool	2022-03-11 08:33:34 +00:00
Antonio Andelic	df0f5e20d0	Refactor	2022-03-11 08:33:34 +00:00
Antonio Andelic	e051587fc2	Format code	2022-03-11 08:33:34 +00:00
Antonio Andelic	29c32ed831	Refactor code	2022-03-11 08:33:34 +00:00
Antonio Andelic	0a1a3a230e	Add support for parallel http range requests	2022-03-11 08:33:34 +00:00
Antonio Andelic	58557b9bec	Test check for ranges support	2022-03-11 08:33:34 +00:00
Anton Popov	df3b07fe7c	Merge remote-tracking branch 'upstream/master' into HEAD	2022-03-03 22:25:28 +00:00
Kseniia Sumarokova	b11b34dc8c	Merge pull request #34849 from kssenii/fix-too-many-columns ... Fix reading too many columns for s3 and url storages	2022-03-03 13:57:22 +01:00
Maksim Kita	b1a956c5f1	clang-tidy check performance-move-const-arg fix	2022-03-02 18:15:27 +00:00
kssenii	9b64a8fe39	Fix odbc bridge	2022-02-28 14:29:05 +01:00
Anton Popov	fcdebea925	Merge remote-tracking branch 'upstream/master' into HEAD	2022-02-25 13:41:30 +03:00
kssenii	cfad79bf74	Remove redundant	2022-02-25 09:37:48 +01:00
kssenii	003b807b00	Fix	2022-02-23 20:33:05 +01:00
Kruglov Pavel	dd863ca2a0	Merge branch 'master' into fix-url-globs	2022-02-16 12:45:31 +03:00
Frank Chen	7b7bc8f6c3	Address review comment	2022-02-12 10:46:01 +08:00
mergify[bot]	f099048dd1	Merge branch 'master' into urlengine	2022-02-11 11:03:21 +00:00
Frank Chen	c8cddd7077	Fix compression for URL engine	2022-02-11 17:45:49 +08:00
Anton Popov	18940b8637	Merge remote-tracking branch 'upstream/master' into HEAD	2022-02-09 23:38:38 +03:00
avogar	84087c1554	Improve schema inference with globs	2022-02-09 19:16:15 +03:00
Kruglov Pavel	e0e36c256e	Fix style	2022-02-09 14:44:46 +03:00
avogar	7baf23f35f	Fix bug URL engine	2022-02-09 13:53:54 +03:00
kssenii	e8a8f46864	Fix style check	2022-02-08 21:58:03 +01:00
Kruglov Pavel	cae1517693	Fix build	2022-02-08 17:31:02 +03:00
Kruglov Pavel	720310fcf1	Merge branch 'master' into http-buffer-skip-not-found-url-for-globs	2022-02-08 16:56:22 +03:00
kssenii	dc5f035265	Fix	2022-02-08 13:28:10 +01:00