Commit Graph

57 Commits

Author SHA1 Message Date
Kruglov Pavel
7873b4475f
Merge branch 'master' into autodetect-format 2022-01-25 10:56:52 +03:00
avogar
a6740d2f9a Detect format and schema for stdin in clickhouse-local 2022-01-25 10:25:37 +03:00
avogar
1f49acc164 Better naming 2022-01-24 16:28:36 +03:00
Kruglov Pavel
a7df9cd53a
Merge branch 'master' into formats-with-suffixes 2022-01-14 21:03:49 +03:00
avogar
89a181bd19 Make better 2022-01-14 18:16:18 +03:00
Kruglov Pavel
5a908e8edd
Merge branch 'master' into formats-with-suffixes 2022-01-14 16:45:20 +03:00
avogar
2d7b1bfa5e Detect format in S3/HDFS/URL table engines 2022-01-13 16:14:18 +03:00
zhongyuankai
878e44eb97 auto format by file extension 2022-01-08 21:47:14 +08:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
frank chen
898db5b468 Resolve review comments
Signed-off-by: frank chen <frank.chen021@outlook.com>
2021-12-03 19:47:05 +08:00
cgp
18504f545a move InputCreatorFunc to InputCreator 2021-11-12 00:34:59 +08:00
avogar
872cca550a Make better 2021-10-20 15:47:20 +03:00
mergify[bot]
0a4360c43e
Merge branch 'master' into tsv-csv 2021-10-20 11:57:06 +00:00
avogar
7007286088 Fix WithNamesAndTypes parallel parsing, add new tests, small refactoring 2021-10-20 14:48:54 +03:00
Nikolai Kochetov
a92dc0a826 Update obsolete comments. 2021-10-19 12:58:10 +03:00
Azat Khuzhin
50231460af Use forward declaration for Buffer<> in generic headers
- changes in ReadHelpers.h -- recompiles 1000 modules
- changes in FormatFactor.h -- recompiles 100 modules
2021-10-16 12:03:24 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Anton Popov
99175f7acc minor enhancements in async inserts 2021-09-16 20:55:34 +03:00
Pavel Kruglov
92e6df7b89 Allow data in square brackets in JSONAsString format 2021-06-23 16:17:34 +03:00
Nikolai Kochetov
dbaa6ffc62 Rename ContextConstPtr to ContextPtr. 2021-06-01 15:20:52 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Nikita Mikhailov
37f48d13b4 add test 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
d4ec52f732 shellchek 2020-12-30 18:21:58 +03:00
Nikita Mikhailov
c5f92e5096 better formatfactory 2020-12-30 06:07:30 +03:00
Nikita Mikhailov
c3288c3fbf Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-28 15:09:37 +03:00
nikitamikhaylov
27f647f93d done 2020-12-23 01:01:05 +03:00
nikitamikhaylov
12e624fd9a fix tests 2020-12-15 00:56:48 +03:00
nikitamikhaylov
4ff1be6e25 better 2020-12-15 00:56:47 +03:00
Nikita Mikhaylov
e0addac6fc save changes 2020-12-15 00:56:46 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
tavplubix
67099f28ac
Merge pull request #16591 from ClickHouse/aku/create-file
Support `SETTINGS` clause for File engine
2020-11-09 14:15:42 +03:00
Alexander Kuzmenkov
3c60f6cec2 make a separate settings collection + some cleanup 2020-11-07 11:53:39 +03:00
Alexander Kuzmenkov
99ee127620 Support SETTINGS clause for File engine
Accept the usual user settings related to file formats.

Most of the diff are the mechanistic code changes required to allow
providing the required FormatSettings to the format factory. The File
engine then extracts these settings from the `CREATE` query, and specifies
them when creating the format parser.
2020-11-02 10:50:38 +03:00
Alexey Milovidov
2e0a979e3a Fix inconsistency in FormatFactory 2020-10-29 06:39:43 +03:00
Mikhail Filimonov
f6b00f2cb6
After CR fixes 2020-10-12 07:52:09 +02:00
Mikhail Filimonov
c37a456c49
Refactoring WriteCallback->RowOutputFormatParams 2020-10-12 07:52:06 +02:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
hcz
602535396d Refactor, move function declarations 2020-09-01 16:36:27 +08:00
hexiaoting
ddbfe0ce21 Deprecate ODBCDriver format 2020-08-18 11:29:40 +08:00
Azat Khuzhin
15be6a0dd5 Fix quirks for PrettyCompactMonoBlock
- Fix PrettyCompactMonoBlock for clickhouse-local (broken in 20.3+,
  fails with an error `Unknown format PrettyCompactMonoBlock`, after #6239)
- Fix extremes/totals with PrettyCompactMonoBlock (even before 20.3
  breakage they were simply ignored)
2020-08-06 11:59:45 +03:00
Maxim Sabyanin
40f7ec71d3 add setting output_format_pretty_grid_charset
This setting allows to chose charset for printing grids (either UTF-8 or
ASCII).
2020-07-10 22:25:49 +03:00
Nikita Mikhaylov
717f63923a
Merge branch 'master' into orc_output_format 2020-06-23 15:07:05 +04:00
Alexey Milovidov
3f96bdaa24 Minor changes after merge 2020-06-21 15:16:08 +03:00
Avogar
93aee32ae4 Add ORCBlockOutputFormat 2020-06-14 18:35:32 +03:00
MovElb
b4b5c90343 squash 2020-05-30 20:05:05 +03:00
FawnD2
f0201f7aca Move ORC to Input only formats 2020-05-04 18:04:20 +03:00
FawnD2
112758b99d Merge branch 'master' into arrow-io-format 2020-05-04 00:53:17 +03:00