thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
116,038 Commits 9,326 Branches 1,784 Tags 2.1 GiB
60dccff77c
Commit Graph

711 Commits

Author SHA1 Message Date
Antonio Andelic
3a6ea861d8 Extract common argument parsing logic 2023-02-13 12:27:49 +00:00
flynn
d3dd9421da refactor and get rid of s3 2023-02-13 08:29:22 +00:00
flynn
37a850bd36 Merge branch 'master' of github.com:ClickHouse/ClickHouse into iceberg 2023-02-10 08:47:14 +00:00
flynn
db15634a01 fix conflict 2023-02-10 08:41:04 +00:00
Robert Schulze
e490ec91d9
Merge branch 'master' into rs/fix-fragile-linking 2023-02-09 11:33:59 +01:00
Ilya Yatsishin
9f480b8fa2
Merge pull request #46132 from azat/file-fd-schema-inference 2023-02-08 23:23:32 +01:00
robot-clickhouse-ci-1
99274f1db1
Merge pull request #46143 from ClickHouse/evillique-patch-2 
Interpret `cluster_name` identifier in `s3Cluster` function as literal
 2023-02-08 14:06:16 +01:00
Robert Schulze
6ff232d782
Merge branch 'master' into rs/fix-fragile-linking 2023-02-08 12:51:12 +01:00
Azat Khuzhin
0b37344b26 Use BAD_ARGUMENTS over LOGICAL_ERROR for schema inference error file() over fd 
Otherwise the following leads to SIGSEGV in debug/sanitizers builds:

  echo '0000000000Custom NULL representation0000000000' | clickhouse-local -q "desc file('-', 'TSV')"

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-08 05:36:49 +01:00
Nikolay Degterinsky
4040b9c91b
Interpret cluster_name identifier in s3Cluster function as literal 2023-02-08 00:39:19 +01:00
Robert Schulze
10af0b3e49
Reduce redundancies 2023-02-07 12:27:23 +00:00
kssenii
ab0dedf0c8 Simplify code around storage s3 configuration 2023-02-06 16:23:17 +01:00
flynn
0dd8a61a8e fix conflict 2023-02-06 03:25:12 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking 
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
 2023-02-05 20:49:34 +00:00
attack204
7ed6bad097
Merge branch 'master' into urlCluster 2023-02-02 21:12:20 +08:00
FFFFFFFHHHHHHH
eca344eff1
Update src/TableFunctions/TableFunctionFormat.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-02-01 14:12:36 +08:00
fhbai
d0fc3d9d4e allow a three-argument version for table function format 2023-02-01 11:17:14 +08:00
Bharat Nallan
f1d6e3b908
Merge branch 'master' into ncb/odbc-connection-pool-fixes 2023-01-30 15:49:04 -08:00
Bharat Nallan Chakravarthy
0d2e4fcc15 set use_connection_pooling from settings everywhere 2023-01-30 08:22:48 -08:00
flynn
bb95093eee remove unused file 2023-01-29 15:05:03 +00:00
flynn
6d9dc83064 fix 2023-01-29 10:15:13 +00:00
flynn
9b517cdc76 fix conflict 2023-01-29 08:34:56 +00:00
flynn
ed6fcfe0d1 refactor 2023-01-29 08:28:48 +00:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Alexander Tokmakov
9b670946db Merge branch 'master' into exception_message_patterns5 2023-01-26 00:41:32 +01:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
Kruglov Pavel
cd3d3fdd68
Merge pull request #45483 from Avogar/fix-hdfs-cluster 
Fix schema inference from insertion table in hdfsCluster
 2023-01-25 20:24:51 +01:00
Alexander Tokmakov
414693feb2 fixes 2023-01-24 00:46:03 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
avogar
73538117eb Fix and add a test 2023-01-23 16:24:00 +00:00
attack204
1ad4d47787 Merge branch 'master' of github.com:attack204/ClickHouse 2023-01-21 09:36:16 +08:00
attack204
d971e0e75d
Merge branch 'master' into master 2023-01-21 09:34:15 +08:00
attack204
0c92c9917e Fix schema inference from insertion table in urlCluster 2023-01-21 09:33:37 +08:00
avogar
eed1db7e07 Fix schema inference in hdfsCluster 2023-01-20 21:17:35 +00:00
Alexander Tokmakov
910d6dc0ce
Merge pull request #45342 from ClickHouse/exception_message_patterns 
Save message format strings for DB::Exception
 2023-01-20 18:46:52 +03:00
attack204
406bd09b40
Merge branch 'master' into master 2023-01-19 17:50:08 +08:00
attack204
1f4139718a fix:style 2023-01-19 16:19:39 +08:00
attack204
f549380867 fix:style 2023-01-19 16:10:59 +08:00
attack204
99bdef71c8 fix:style 2023-01-19 15:12:47 +08:00
attack204
31fa490822 fix conflict 2023-01-19 10:45:14 +08:00
attack204
e312cfa794 feature:urlCluster 2023-01-19 10:19:04 +08:00
avogar
117ec13c9e Fix s3Cluster schema inference when structure from insertion table is used 2023-01-18 20:33:50 +00:00
flynn
2fb32dc56c fix and add test 2023-01-18 08:33:55 +00:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Alexander Tokmakov
522686f78b less empty patterns 2023-01-17 01:19:44 +01:00
Maksim Kita
80f6a45376
Merge pull request #44641 from ClickHouse/vdimir/view_explain_2 
Function viewExplain accept SELECT and settings
 2023-01-16 13:39:53 +03:00
avogar
abfb6b096f Better exception message 2023-01-13 14:23:30 +00:00
vdimir
f881a82417
Fix viewExplain, add testcases 2023-01-13 12:19:25 +00:00
avogar
82ff1fd343 Add tests and docs 2023-01-12 22:29:23 +00:00
avogar
5d6ae2c3d5 Use structure from insertion table in generateRandom func 2023-01-12 18:33:47 +00:00
vdimir
8b65ed200c
Review fixes 2023-01-11 10:53:13 +00:00
vdimir
c0b12d51bd
Function viewExplain accept SELECT and settings 2023-01-10 15:42:59 +00:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
Kseniia Sumarokova
a2a9c7dec5
Update TableFunctionURL.cpp 2023-01-05 21:35:52 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
SmitaRKulkarni
16fca48f5b
Disable functions in readonly for inserts (#44290) 
Implementation:
* Added a new parameter to ITableFunction::execute to check if query is insert query.
* Used the new parameter in InterpreterInsertQuery::getTable function when calling ITableFunction::execute .
Testing:
* Added test case for INSERT in 02473_functions_in_readonly_mode.sh.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-01-03 15:25:04 +01:00
kssenii
6bd4f8c029 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-20 21:17:28 +01:00
Raúl Marín
45d27f461b
Merge branch 'master' into perf_experiment 2022-12-20 09:07:48 +00:00
kssenii
6bb3d06c88 Replace old named collections code for postgresql 2022-12-17 01:30:55 +01:00
kssenii
2ce5af421e Replace old named collections code for mongo 2022-12-17 00:50:25 +01:00
kssenii
30547d2dcd Replace old named collections code for url 2022-12-17 00:24:05 +01:00
Kruglov Pavel
e3aa2d9d56
Merge pull request #43695 from Avogar/better-using-structure-in-table-functions 
Improve automatic usage of structure from insertion table in table functions file/hdfs/s3
 2022-12-16 15:45:30 +01:00
kssenii
fae817863c Apply new code of named collections to s3 2022-12-13 22:19:09 +01:00
avogar
4aa3e23270 Improve automatic usage of structure from insertion table in table functions file/hdfs/s3 2022-11-25 19:33:47 +00:00
Raúl Marín
4aa29b6a63 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-22 19:09:00 +01:00
Kruglov Pavel
98d6b96c82
Merge pull request #42033 from mark-polokhov/BSONEachRow 
Add BSONEachRow input/output format
 2022-11-22 14:45:21 +01:00
Kseniia Sumarokova
5c90d5aa7e
Merge pull request #43253 from xiedeyantu/fix-s3-glob 
fix s3 support question mark wildcard
 2022-11-22 14:26:56 +01:00
Raúl Marín
8776b2f566 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-22 11:45:36 +01:00
Antonio Andelic
843401c3a3
Merge pull request #43323 from ClickHouse/fix-describe-delta-lake-hudi 
Fix `DESCRIBE` for `deltaLake` and `hudi` table functions
 2022-11-22 09:09:39 +01:00
Kruglov Pavel
406c50277b
Merge pull request #42779 from Avogar/add-format-func-doc 
Add documentation for 'format' table function
 2022-11-21 15:16:26 +01:00
Raúl Marín
24c7496d70 Why am i getting this warnings? 2022-11-21 11:05:56 +01:00
Raúl Marín
ed0c174c0c Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-21 11:02:31 +01:00
Antonio Andelic
9a8d82f0fc Merge branch 'master' into fix-describe-delta-lake-hudi 2022-11-21 08:23:33 +00:00
Alexey Milovidov
b3a06175e7 Add a comment 2022-11-20 07:47:20 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
xiedeyantu
c258d3ac8b fix s3 support question mark wildcard 2022-11-18 12:11:22 +08:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
Raúl Marín
97d6fc3071 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-17 11:48:46 +01:00
Antonio Andelic
41b557f926 Disable DeltaLake and hudi table functions in readonly mode 2022-11-17 09:16:52 +00:00
Vitaly Baranov
8e99f5fea3 Move maskSensitiveInfoInQueryForLogging() to src/Parsers/ 2022-11-14 18:55:19 +01:00
Kruglov Pavel
831abbaff9
Fix build 2022-11-14 17:29:30 +01:00
flynn
5bd8e2338b
Merge branch 'master' into hudi-udtf 2022-11-14 22:28:04 +08:00
Kruglov Pavel
58b7f6ca04
Merge pull request #42320 from Avogar/improve-schema-inference-insert-select 
Improve using structure from insertion table in table functions
 2022-11-14 14:51:27 +01:00
Alexey Milovidov
127631ee47
Merge branch 'master' into perf_experiment 2022-11-12 18:58:25 +01:00
Kruglov Pavel
0270f6003c
Merge branch 'master' into add-format-func-doc 2022-11-11 14:05:34 +01:00
zzsmdfj
3835373644 to add_oss_function_and_StorageOSS 2022-11-11 16:40:10 +08:00
flynn
d96d11899b fix 2022-11-11 06:14:51 +00:00
flynn
eebc5349cc add documentation 2022-11-11 02:42:18 +00:00
flynn
6f6a1f5d76 fix 2022-11-11 02:30:05 +00:00
avogar
9e89af28c6 Refactor BSONEachRow format, fix bugs, support more data types, support parallel parsing and schema inference 2022-11-10 20:15:14 +00:00
flynn
fc7311aab7 fix 2022-11-10 15:09:55 +00:00
flynn
592f6b31d9 fix and add test 2022-11-09 10:04:53 +00:00
flynn
1a8dd85a48
Update src/TableFunctions/TableFunctionDelta.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-11-09 17:11:20 +08:00
flynn
b7cb7ab6db
Update src/TableFunctions/TableFunctionDelta.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-11-09 17:11:11 +08:00
flynn
75d3907e79
Update src/TableFunctions/TableFunctionDelta.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-11-09 17:10:57 +08:00
flynn
5eafe3fc46 fix comment 2022-11-09 08:03:32 +00:00
flynn
ee856fc458 Add table function hudi and deltaLake 2022-11-09 08:01:05 +00:00
SmitaRKulkarni
c563e124c2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-11-07 09:50:05 +01:00
Kruglov Pavel
83c5c44b11
Merge branch 'master' into improve-schema-inference-insert-select 2022-11-03 18:41:51 +01:00
Smita Kulkarni
e6a387e7b2 Updated registerFunction factory method and removed table_functions_allowed_in_readonly_mode - 42414 Enable table functions in readonly mode 2022-11-03 12:31:07 +01:00
Nikolay Degterinsky
30ad1a6826
Merge branch 'master' into perf_experiment 2022-11-03 02:18:21 +03:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
avogar
9d9bca3dbb Add embedded documentation 2022-11-01 14:08:34 +00:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference 
Fix schema inference in s3Cluster and improve in hdfsCluster
 2022-11-01 14:00:21 +01:00
Smita Kulkarni
b689f7ab70 Updated registerFunction method to take TableFunctionFactoryData as input , moved table_functions_allowed_in_readonly_mode inside TableFunctionFactory- 42414 Enable table functions in readonly mode 2022-11-01 10:13:10 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
Smita Kulkarni
6a8fa2d4a5 Added new field allow_readonly in system.table_functions to allow using table functions in readonly mode. 
Implementation:
* Added a new field allow_readonly to table system.table_functions.
* Updated to use new field allow_readonly to allow using table functions in readonly mode.
* Added TableFunctionProperties similar to AggregateFunctionProperties.
* The functions allowed in readonly mode are in this set table_functions_allowed_in_readonly_mode.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02473_functions_in_readonly_mode.sh
Documentation:
* Updated the english documentation for Table Functions.
 2022-10-26 18:45:23 +02:00
Raúl Marín
6e0a9452e7 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-10-25 15:25:06 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Raúl Marín
a6aad33e89 More style 2022-10-20 11:43:55 +02:00
Kruglov Pavel
3d9f46a1e7
Merge branch 'master' into s3-cluster-schema-inference 2022-10-14 22:07:54 +02:00
Kruglov Pavel
f1dd247181
Merge branch 'master' into improve-schema-inference-insert-select 2022-10-14 17:16:58 +02:00
avogar
10335aec13 Improve using structure from insertion table in table functions 2022-10-14 15:09:35 +00:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
Vladimir C
fa254bcec3 Fix clang-tidy in TableFunctionExplain.cpp 2022-10-03 19:14:40 +02:00
vdimir
06f504c166 doc viewExplain 2022-10-03 19:14:40 +02:00
vdimir
3fdea908ff Use ASTExplain in viewExplain 2022-10-03 19:14:40 +02:00
vdimir
6028643e04 Add function viewExplain 2022-10-03 19:14:40 +02:00
avogar
addd36f436 Remove logging 2022-09-30 17:00:39 +00:00
avogar
c74b5c8126 Fix schema inference in s3Cluster and improve in hdfsCluster 2022-09-30 16:59:17 +00:00
Robert Schulze
f24fab7747
Fix some #include atrocities 2022-09-28 13:49:28 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Alexey Milovidov
a3b28b441b
Merge pull request #34892 from evillique/new_parser 
Implement operator precedence parser
 2022-09-27 00:07:04 +03:00
Nikolay Degterinsky
e9c4968211 Merge remote-tracking branch 'master' into new_parser 2022-09-14 11:08:48 +00:00
kssenii
420ac4eb43 s3 header auth in ast 2022-09-13 15:13:28 +02:00
Alexey Milovidov
fd235919aa Remove some methods 2022-09-10 05:04:40 +02:00
Vitaly Baranov
cfb250c46b Fix checking access rights for DESCRIBE TABLE url. 2022-09-04 18:58:39 +02:00
Nikolay Degterinsky
33c070a5e0 Merge remote-tracking branch 'upstream/master' into new_parser 2022-09-01 13:08:36 +00:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented 
All new table functions must be documented
 2022-08-28 17:05:20 +03:00
Alexey Milovidov
5b57c91ad6 Rename table function MeiliSearch to meilisearch 2022-08-28 02:08:51 +02:00
Alexey Milovidov
6139cc8f7a Add Documentation to FunctionFactory 2022-08-27 22:06:57 +02:00
Alexey Milovidov
e7054debc1
Merge pull request #40649 from ClickHouse/embedded-documentation 
A prototype for embedded documentation
 2022-08-27 21:28:11 +03:00
Alexey Milovidov
6e564b18bf
Merge pull request #40600 from FrankChen021/check_url_arg 
Validate the CompressionMethod parameter of URL table engine
 2022-08-27 19:29:55 +03:00
Kseniia Sumarokova
4e5385298a
Update src/TableFunctions/TableFunctionURL.cpp 2022-08-26 14:22:29 +02:00
Alexey Milovidov
8afbfebd25 A prototype for embedded documentation 2022-08-26 03:41:52 +02:00
avogar
cef2972af4 Better format detection for url table function/engine 2022-08-25 18:31:06 +00:00
Frank Chen
ef49cacefe Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:32:22 +08:00
Frank Chen
c9ea4f9f77 Change compression_method from String to CompressionMethod 2022-08-25 19:18:04 +08:00
Alexey Milovidov
01700078d9
Merge branch 'master' into xdbc-constraints 2022-08-20 21:26:07 +03:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache 
Add schema inference cache for s3/hdfs/file/url
 2022-08-18 13:07:50 +02:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
avogar
e1ff996ec3 Allow to specify structure hints in schema inference 2022-08-16 09:46:57 +00:00
Nikolay Degterinsky
bd9b51ad72 Merge remote-tracking branch 'upstream/master' into new_parser 2022-08-15 21:18:57 +00:00
Constantine Peresypkin
b9d7cd6a5d add settings for executable table func 
SELECT * FROM executable('<script name>', '<format>', '<columns>', (SELECT * FROM table), SETTINGS max_command_execution_time=100, command_termination_timeout=100)

fixes #38908
 2022-07-30 20:02:56 +02:00
Nikolay Degterinsky
fd5a06c827 Merge remote-tracking branch 'upstream/master' into new_parser 2022-07-20 13:41:12 +00:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache 2022-07-20 13:30:42 +02:00