Commit Graph

276 Commits

Author SHA1 Message Date
Kruglov Pavel
bf28074d32
Merge branch 'master' into allow-skip-empty-files 2023-06-08 12:36:18 +02:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
Michael Kolupaev
b51064a508 Get rid of SeekableReadBufferFactory, add SeekableReadBuffer::readBigAt() instead 2023-06-01 18:48:30 -07:00
avogar
0b62be649f Add docs, fix style 2023-05-31 17:52:29 +00:00
Kruglov Pavel
0beca0336d
Merge pull request #49112 from ClickHouse/Avogar-patch-3
Fix possible terminate called for uncaught exception in some places
2023-05-31 16:55:43 +02:00
avogar
d4efbbfbd3 Allow to skip empty files in file/s3/url/hdfs table functions 2023-05-30 19:32:24 +00:00
avogar
88e4c93abc Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-22 19:19:57 +00:00
Nikolay Degterinsky
d4b89cb643
Merge pull request #49356 from Ziy1-Tan/vcol
Support for `_path` and `_file` virtual columns for table function `url`.
2023-05-22 18:10:32 +02:00
avogar
3ee8de792c Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-11 12:46:20 +00:00
Michael Kolupaev
3bd1489f18 Propagate input_format_parquet_preserve_order to parallelizeOutputAfterReading() 2023-05-05 04:20:27 +00:00
Michael Kolupaev
eb3b774ad0 Better control over Parquet row group size 2023-05-04 14:59:55 -07:00
Ziy1-Tan
1bb0d1519e Fix style
Signed-off-by: Ziy1-Tan <ajb459684460@gmail.com>
2023-05-02 16:54:14 +08:00
Ziy1-Tan
c93ceedbef Fix style
Signed-off-by: Ziy1-Tan <ajb459684460@gmail.com>
2023-05-02 10:38:37 +08:00
Ziy1-Tan
2c159061ed Support _path and _file virtual columns for table function url. 2023-05-01 21:40:30 +08:00
Kruglov Pavel
75a3b6c322
Fix build 2023-04-24 21:08:53 +02:00
Kruglov Pavel
8ff864cd8b
Fix 2023-04-24 19:12:50 +02:00
avogar
c503f6532c Add more finalize() to avoid terminate 2023-04-24 15:11:36 +00:00
avogar
0097230611 Better 2023-04-21 17:35:17 +00:00
avogar
0805b517ee Fix parsing failover options 2023-04-21 17:28:14 +00:00
avogar
944f54aadf Finish urlCluster, refactor code, reduce code duplication 2023-04-21 17:24:37 +00:00
avogar
c949f0ebf5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-04-21 14:13:33 +02:00
avogar
86686fbbc3 Fix conflicts 2023-04-21 14:11:18 +02:00
Michael Kolupaev
87be78e6de Better 2023-04-17 04:58:32 +00:00
Michael Kolupaev
e133633359 Parallel decoding with one row group per thread 2023-04-17 04:58:32 +00:00
Michael Kolupaev
683077890f Highly questionable refactoring (getInputMultistream() nonsense) 2023-04-17 04:58:32 +00:00
Michael Kolupaev
2d4fe85513 Something 2023-04-17 04:58:32 +00:00
kssenii
bb0beb7449 Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-03-17 13:02:36 +01:00
Antonio Andelic
a70ca31884 Merge branch 'master' into fix-url-progress-bar 2023-03-09 10:17:33 +00:00
kssenii
8f2d75cef8 Fix tests 2023-03-05 12:56:00 +01:00
Konstantin Bogdanov
1bbf5acd47
Pass headers from StorageURL to WriteBufferFromHTTP (#46996)
* Pass headers from StorageURL to WriteBufferFromHTTP

* Add a test

* Lint

* `time.sleep(1)`

* Start echo server earlier

* Add proper handling for mock server start

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-03-03 13:55:52 +01:00
Antonio Andelic
f540f7f6f9 Fix some tests 2023-03-01 12:45:00 +00:00
Antonio Andelic
45dc5dc25d No progress bar if no size 2023-02-28 15:25:28 +00:00
Antonio Andelic
56a126f7af Fix progress bar with URL 2023-02-24 14:49:14 +00:00
Robert Schulze
10af0b3e49
Reduce redundancies 2023-02-07 12:27:23 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
2023-02-05 20:49:34 +00:00
attack204
7ed6bad097
Merge branch 'master' into urlCluster 2023-02-02 21:12:20 +08:00
Anton Popov
5c0307bc6a fix race in StorageURL and StorageHDFS 2023-01-24 12:34:43 +00:00
attack204
1f4139718a fix:style 2023-01-19 16:19:39 +08:00
attack204
f549380867 fix:style 2023-01-19 16:10:59 +08:00
attack204
7bd0010c50 fix conflict 2023-01-19 13:18:07 +08:00
attack204
31fa490822 fix conflict 2023-01-19 10:45:14 +08:00
attack204
e312cfa794 feature:urlCluster 2023-01-19 10:19:04 +08:00
Alexander Tokmakov
2d7773fccc Merge branch 'master' into text_log_add_pattern 2023-01-13 20:33:46 +01:00
kssenii
fcb042d80c Fix cland tidy 2023-01-05 12:04:07 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
Kruglov Pavel
966f57ef68
Merge pull request #42777 from Avogar/improve-streaming-engines
Refactor and Improve streaming engines Kafka/RabbitMQ/NATS and data formats
2023-01-02 15:59:06 +01:00
Alexander Tokmakov
ca989e9212 less runtime format strings 2022-12-23 19:50:34 +01:00
kssenii
8d9bf77588 Fix tests 2022-12-19 13:56:23 +01:00
kssenii
326747b555 Fix tests 2022-12-19 12:16:50 +01:00
kssenii
0c63ce9731 Remove some more old code 2022-12-17 00:34:29 +01:00