Commit Graph

64 Commits

Author SHA1 Message Date
Kruglov Pavel
46a6b84a5a
Merge branch 'master' into auto-format-detection 2024-01-25 22:11:07 +01:00
Maksim Kita
2a327107b6 Updated implementation 2024-01-25 14:31:49 +03:00
avogar
617cc514b7 Try to detect file format automatically during schema inference if it's unknown 2024-01-23 18:59:39 +00:00
Nikolai Kochetov
3e3fed1cbe Add reading step to URL 2024-01-02 15:18:13 +00:00
Nikita Mikhaylov
04d167c6d9 Better 2023-12-05 13:34:37 +01:00
avogar
2d8f33bfa2 Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header 2023-09-11 14:55:37 +00:00
Nikolay Degterinsky
7321f5e543 Better 2023-08-10 06:34:10 +00:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
vdimir
023d14a894 Support transformQueryForExternalDatabase for analyzer 2023-03-22 08:59:04 +01:00
Robert Schulze
10af0b3e49
Reduce redundancies 2023-02-07 12:27:23 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
2023-02-05 20:49:34 +00:00
Bharat Nallan Chakravarthy
0d2e4fcc15 set use_connection_pooling from settings everywhere 2023-01-30 08:22:48 -08:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Alexey Milovidov
6e564b18bf
Merge pull request #40600 from FrankChen021/check_url_arg
Validate the CompressionMethod parameter of URL table engine
2022-08-27 19:29:55 +03:00
Frank Chen
c9ea4f9f77 Change compression_method from String to CompressionMethod 2022-08-25 19:18:04 +08:00
Alexey Milovidov
d157c6a7a0
Update StorageXDBC.cpp 2022-08-21 16:53:48 +03:00
Alexey Milovidov
01700078d9
Merge branch 'master' into xdbc-constraints 2022-08-20 21:26:07 +03:00
avogar
59c1c472cb Better exception messages on wrong table engines/functions argument types 2022-06-23 20:04:06 +00:00
avogar
68bb07d166 Better naming 2022-05-13 18:39:19 +00:00
avogar
b17fec659a Improve performance and memory usage for select of subset of columns for some formats 2022-05-13 13:51:28 +00:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Alexey Milovidov
9fedcbde78 Merge branch 'master' into xdbc-constraints 2022-03-20 19:50:43 +01:00
Anton Popov
df3b07fe7c Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-03 22:25:28 +00:00
kssenii
9b64a8fe39 Fix odbc bridge 2022-02-28 14:29:05 +01:00
Alexey Milovidov
ccfc41bce5 Allow constraints for ODBC and JDBC tables 2022-02-12 13:50:05 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
kssenii
9080d2ea52 Fix tests 2021-11-02 07:27:13 +00:00
kssenii
2ba3ee830e Partitioned write part 2 2021-10-26 14:00:41 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Azat Khuzhin
a4f2663b82 Log queries to external databases (since they may be rewriten)
By rewritten means that some expression from WHERE can be removed, yes
there is external_table_strict_query, but sometimes it is too strict.
2021-09-29 21:56:28 +03:00
Anton Popov
e36736b50c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-02 22:52:02 +03:00
Nikolai Kochetov
2dc5c89b66 Update Storage::write 2021-07-23 17:25:35 +03:00
Anton Popov
3ed7f5a6cc dynamic subcolumns: add snapshot for storage 2021-07-09 06:15:41 +03:00
Anton Popov
205a23282b Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-06-07 19:38:22 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Anton Popov
0bdf9d207c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-05-26 00:25:47 +03:00
Anton Popov
e44706911e dynamic columns: better getting of sample block 2021-05-05 02:02:54 +03:00
Anton Popov
644df6be7d dynamic subcolumns: wip 2021-04-24 07:09:01 +03:00
feng lv
4ffe199d39 Implement table comments 2021-04-23 12:18:23 +00:00
kssenii
5dc2dfa437 Fix 2021-04-17 09:00:58 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
tavplubix
67099f28ac
Merge pull request #16591 from ClickHouse/aku/create-file
Support `SETTINGS` clause for File engine
2020-11-09 14:15:42 +03:00
Alexander Kuzmenkov
14f31f5117 add support of settings to URL, fix url and file table functions 2020-11-05 14:28:20 +03:00
Azat Khuzhin
b838214a35 Pass non-const SelectQueryInfo (and drop mutable qualifiers) 2020-10-02 22:42:35 +03:00
Nikolai Kochetov
2cca4d5fcf Refactor Pipe [part 2]. 2020-08-03 16:54:14 +03:00
alesapin
4c0879ae30 Better logging in storages 2020-06-19 20:17:13 +03:00