Commit Graph

240 Commits

Author SHA1 Message Date
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
Alexey Milovidov
71926a3a97 Fix surprisingly bad code in function "file" 2021-12-13 07:57:54 +03:00
Anton Popov
6f4d9a53b2 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-12-01 15:54:33 +03:00
Raúl Marín
051dddd8df Reduce dependencies on ASTIdentifier.h
Goes from rebuilding 483 objects to 165 when it's modified
2021-11-26 16:49:40 +01:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
Kruglov Pavel
1bbe0df945
Fix perf test 2021-11-16 11:41:50 +03:00
avogar
8f7be52ac4 Update test, remove commented code 2021-11-15 14:58:18 +03:00
avogar
e5661b9b62 Minor change 2021-11-11 21:09:21 +03:00
avogar
c521a9131a Small refactoring of WriteBiffer-s 2021-11-11 02:11:18 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
avogar
2dd4393ca1 Small refactoring in formats 2021-11-03 20:07:05 +03:00
kssenii
03d7476864 One more fix 2021-10-29 00:50:59 +03:00
kssenii
d39840376f Review fixes 2021-10-28 15:44:12 +03:00
kssenii
1d743b9259 Partitoned write 2021-10-25 19:23:44 +03:00
Kseniia Sumarokova
c692155c7e
Merge pull request #30309 from kssenii/fix-file-symlinks
Fix symlinks in file table function
2021-10-20 11:09:51 +03:00
Kseniia Sumarokova
5633865df1
Update src/Storages/StorageFile.cpp
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2021-10-17 14:16:40 +03:00
kssenii
059fc1de69 Allow symlinks in file storage 2021-10-17 09:11:06 +00:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Nikolai Kochetov
2957971ee3 Remove some last streams. 2021-10-13 21:22:02 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
Anton Popov
4c388e3d84 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-09-09 14:10:16 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
kssenii
c5ee335c68 Fix tests 2021-08-22 21:41:09 +03:00
kssenii
14afb3c509 File progress for local fix 2021-08-22 20:33:55 +03:00
Anton Popov
e36736b50c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-02 22:52:02 +03:00
Nikolai Kochetov
9b5a816b43 Merge branch 'master' into output-streams-to-processors 2021-07-26 18:03:11 +03:00
Nikolai Kochetov
0eb563dc1b Fix more tests. 2021-07-26 17:47:29 +03:00
Nikolai Kochetov
fa1c223269 Fix some tests. 2021-07-26 13:08:40 +03:00
Alexey Milovidov
2f2dce399a Fix style 2021-07-25 01:26:21 +03:00
Alexey Milovidov
215505b985 Remove unused error code 2021-07-24 19:51:56 +03:00
Alexey Milovidov
774f6d2617 Allow to read file descriptor multiple times in File storage 2021-07-24 19:50:03 +03:00
BoloniniD
2fa02061b7 Fix parallel reads 2021-07-24 15:31:35 +03:00
BoloniniD
07c57edbfc Merge branch 'master' of github.com:ClickHouse/ClickHouse into pipe_reading 2021-07-23 23:39:38 +03:00
Nikolai Kochetov
9c92f43359 Update storages. 2021-07-23 22:33:59 +03:00
Nikolai Kochetov
2dc5c89b66 Update Storage::write 2021-07-23 17:25:35 +03:00
Nikolai Kochetov
f38de35b14 Rename some constants. 2021-07-21 19:13:17 +03:00
Nikolai Kochetov
179ec05a72 Remove some streams. 2021-07-20 21:18:43 +03:00
BoloniniD
49ee24e824 What if we replace unique_lock with shared? 2021-07-20 19:22:41 +03:00
Anton Popov
f99374cca6 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-07-20 18:20:21 +03:00
Nikita Mikhaylov
4d3f828beb
Merge pull request #26314 from kssenii/fix-hdfs-crash
Fix hdfs crash
2021-07-20 15:01:56 +03:00
BoloniniD
a9deeb6a78 Trying to fix freeze on union all 2021-07-18 15:55:24 +03:00
Alexey Milovidov
261a220227 Remove some code 2021-07-17 21:06:46 +03:00
BoloniniD
19b673fb3b Style fix for error message 2021-07-17 15:30:03 +03:00
kssenii
61652acede Add test 2021-07-14 22:49:16 +03:00
BoloniniD
407652457f Added fstat for table fd + more fixes 2021-07-09 12:20:11 +03:00
Anton Popov
3ed7f5a6cc dynamic subcolumns: add snapshot for storage 2021-07-09 06:15:41 +03:00
Anton Popov
072e65b728 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-07-07 17:20:38 +03:00
BoloniniD
0f584c3e76 Fix style and add check for seekability 2021-07-07 14:21:32 +03:00
BoloniniD
077c147e8f First changes after review 2021-07-06 13:23:39 +03:00
BoloniniD
e988f35d86 Added read counter to checker 2021-07-04 21:15:40 +03:00
BoloniniD
bbe33f061e Merge branch 'master' of github.com:ClickHouse/ClickHouse into pipe_reading 2021-07-04 15:36:49 +03:00
BoloniniD
6f5b4ba49e First look 2021-07-01 17:04:10 +03:00
Raúl Marín
bfc122df64 Fix some typos in Storage classes 2021-06-28 19:03:56 +02:00
Anton Popov
205a23282b Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-06-07 19:38:22 +03:00
kssenii
de277f8ac4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into progress-bar 2021-06-01 07:58:19 +00:00
Anton Popov
0bdf9d207c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-05-26 00:25:47 +03:00
kssenii
31107816bb fs::absolute to fs::canonical 2021-05-24 20:37:54 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
kssenii
e9c3bc1836 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into progress-bar 2021-05-16 20:48:35 +00:00
kssenii
c25be65ebf Refactor progress bar, adjust progress indication for reading from file 2021-05-14 06:16:31 +00:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
Anton Popov
e44706911e dynamic columns: better getting of sample block 2021-05-05 02:02:54 +03:00
kssenii
c33f358a9a Better way to track progress, display progress for ch-client 2021-04-28 19:12:28 +00:00
kssenii
563f8a97c6 Fix build, typos check 2021-04-27 17:01:41 +00:00
kssenii
eeb71672a0 Change in Storages/* 2021-04-27 16:49:37 +03:00
kssenii
3a0372535e Show progress only with --progress option 2021-04-27 13:21:42 +00:00
kssenii
1135c47082 Fixes 2021-04-27 11:47:57 +00:00
kssenii
5efd075ad2 Slightly bette 2021-04-27 00:24:04 +00:00
kssenii
ffcfe5f9a1 Progress bar for file table engine 2021-04-26 21:02:52 +00:00
feng lv
4ffe199d39 Implement table comments 2021-04-23 12:18:23 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Nikita Mikhailov
71a2c85d9a better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
af85ce2fd1 better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
37f48d13b4 add test 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
282775b10c save 2021-04-06 22:23:16 +03:00
keenwolf
1a4961499b the Native format is not supported after test 2021-04-06 22:23:16 +03:00
keenwolf
b95f54edce Small fix 2021-04-06 22:23:16 +03:00
keenwolf
139bb678a9 Branch with Format isColumnOriented() or not 2021-04-06 22:23:15 +03:00
keenwolf
de365a5852 fix 2021-04-06 22:23:15 +03:00
keenwolf
6f9bad682f Only read needed columns for formats as parquet etc 2021-04-06 22:23:15 +03:00
M0r64n
2f7d0ba926 Replace direct truncate with O_TRUNC flag 2021-02-20 03:27:23 +04:00
M0r64n
75117389ec Add a couple of QOL file engine settings 2021-02-16 18:50:11 +04:00
Alexey Milovidov
1a90c569cb Merge branch 'master' into revert-15259-revert-14962-log-avoid-deadlock 2021-01-01 21:39:08 +03:00
Nikita Mikhailov
ec0f70b8ef cleanup 2020-12-30 08:39:08 +03:00
Nikita Mikhailov
60b4a36c4a arcadia fix + live view fix + cleanup 2020-12-30 07:50:58 +03:00
Nikita Mikhailov
c5f92e5096 better formatfactory 2020-12-30 06:07:30 +03:00
alexey-milovidov
69719f743d
Merge branch 'master' into revert-15259-revert-14962-log-avoid-deadlock 2020-12-22 00:41:52 +03:00
nikitamikhaylov
12e624fd9a fix tests 2020-12-15 00:56:48 +03:00
nikitamikhaylov
f7ac8bf542 rebase and fix tests 2020-12-15 00:56:48 +03:00
Nikita Mikhaylov
5a47928431 save 2020-12-15 00:56:47 +03:00
Nikita Mikhaylov
f40f3ced2a fix JSONEachRowArray 2020-12-15 00:56:46 +03:00
Nikita Mikhaylov
e0addac6fc save changes 2020-12-15 00:56:46 +03:00
Alexander Tokmakov
04ceaa66f2 implement ATTACH TABLE name FROM 'path/to/data/ 2020-12-08 17:15:23 +03:00
tavplubix
67099f28ac
Merge pull request #16591 from ClickHouse/aku/create-file
Support `SETTINGS` clause for File engine
2020-11-09 14:15:42 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Alexander Kuzmenkov
3c60f6cec2 make a separate settings collection + some cleanup 2020-11-07 11:53:39 +03:00
Alexander Kuzmenkov
2efbcbaa14 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-05 14:28:56 +03:00
Alexander Kuzmenkov
14f31f5117 add support of settings to URL, fix url and file table functions 2020-11-05 14:28:20 +03:00
Alexander Tokmakov
ac32809b6a fix #16482 2020-11-02 19:40:39 +03:00
Alexander Kuzmenkov
99ee127620 Support SETTINGS clause for File engine
Accept the usual user settings related to file formats.

Most of the diff are the mechanistic code changes required to allow
providing the required FormatSettings to the format factory. The File
engine then extracts these settings from the `CREATE` query, and specifies
them when creating the format parser.
2020-11-02 10:50:38 +03:00
Alexander Tokmakov
72b1339656 Revert "Revert "Write structure of table functions to metadata""
This reverts commit c65d1e5c70.
2020-10-14 15:19:29 +03:00
tavplubix
c65d1e5c70
Revert "Write structure of table functions to metadata" 2020-10-14 13:59:29 +03:00
alexey-milovidov
f60ccb4edf
Merge pull request #14295 from ClickHouse/write_structure_of_table_functions
Write structure of table functions to metadata
2020-10-13 23:56:09 +03:00
Alexey Milovidov
5b482f4191 Cleanups 2020-10-10 19:31:10 +03:00
Azat Khuzhin
b838214a35 Pass non-const SelectQueryInfo (and drop mutable qualifiers) 2020-10-02 22:42:35 +03:00
alesapin
17b86dec96 Recursive defaults 2020-10-02 15:38:50 +03:00
Alexander Tokmakov
b0d99217fb Merge branch 'master' into write_structure_of_table_functions 2020-09-27 14:26:47 +03:00
alexey-milovidov
47d150b116
Revert "Revert "Avoid deadlocks in Log/TinyLog"" 2020-09-25 02:29:16 +03:00
alexey-milovidov
13529265c4
Revert "Avoid deadlocks in Log/TinyLog" 2020-09-25 02:28:57 +03:00
Alexey Milovidov
111acdc63b Continuation 2020-09-19 14:02:11 +03:00
Alexander Tokmakov
1ca9a92b21 Merge branch 'master' into write_structure_of_table_functions 2020-09-18 21:09:23 +03:00
Alexey Milovidov
7fb4dfea2c Small improvements for IStorage::rename 2020-09-17 22:50:43 +03:00
Azat Khuzhin
138e953429 Fix SIGSEGV for an attempt to INSERT into StorageFile(fd) 2020-09-17 01:26:34 +03:00
Alexander Tokmakov
ee7b8a797a better code, fix Distributed format of StorageFile 2020-09-02 18:07:53 +03:00
Alexander Tokmakov
b521ca9b18 save structure for all table functions 2020-08-28 00:05:52 +03:00
Nikolai Kochetov
20e63d2271 Refactor Pipe [part 6] 2020-08-06 15:24:05 +03:00
Nikolai Kochetov
2cca4d5fcf Refactor Pipe [part 2]. 2020-08-03 16:54:14 +03:00
Nikita Mikhaylov
270551e599 done 2020-07-07 14:45:20 +03:00
alesapin
c9fa5d2ec3 Better naming 2020-06-19 18:39:41 +03:00
alesapin
d79982f497 Better locks in Storages 2020-06-18 19:10:47 +03:00
alesapin
aab4ce6394 Truncate with metadata 2020-06-18 13:29:13 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
alesapin
af2fe2ba55 Compilable setColumns, setConstraints, setIndices 2020-06-15 19:55:33 +03:00
alesapin
f981649213 Fix pushing to views stream and refactor virtuals 2020-04-28 13:38:57 +03:00
alesapin
4badd0fd28 Better code 2020-04-27 20:46:51 +03:00
alesapin
18c550df15 Better virtuals logic 2020-04-27 16:55:30 +03:00
Alexander Tokmakov
dd1590830b Merge branch 'master' into database_atomic 2020-04-08 22:00:46 +03:00
Vitaly Baranov
e573549945 Rework access rights for table functions. 2020-04-07 23:31:59 +03:00
Alexander Tokmakov
4c48b7dd80 better rename 2020-04-07 18:31:33 +03:00
Alexander Tokmakov
08bae4668d Merge branch 'master' into database_atomic 2020-04-06 16:18:07 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00