thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-05 14:02:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
114,497 Commits 9,193 Branches 1,782 Tags 2 GiB
0c0ea7bfc0
Commit Graph

3363 Commits

Author SHA1 Message Date
avogar
b8156f459f Better solution 2023-02-23 18:13:35 +00:00
avogar
54622566df Add setting to change parquet version 2023-02-23 16:14:10 +00:00
Kruglov Pavel
35887911e3
Merge pull request #46284 from Avogar/null-as-default-all-formats 
Extend setting `input_format_null_as_default` for more formats
 2023-02-23 14:48:22 +01:00
Kseniia Sumarokova
30a8eb0c2f
Merge pull request #46670 from kssenii/hide-disk-setting 
Hide arguments of custom disk merge tree setting
 2023-02-23 11:42:53 +01:00
Alexey Milovidov
c0bc549e77
Merge pull request #46709 from azat/bg-pool-no-query 
BackgroundSchedulePool should not have any query context
 2023-02-23 10:24:46 +03:00
pufit
0fcda19d20 Fix description typo 2023-02-22 18:27:23 -05:00
Alexander Tokmakov
e660c0838c fix multipart requests 2023-02-22 17:54:35 +01:00
avogar
e0931dbdbe Enable input_format_json_ignore_unknown_keys_in_named_tuple by default 2023-02-22 16:40:53 +00:00
Azat Khuzhin
2ca47a6eb6 BackgroundSchedulePool should not have any query context 
BackgroundSchedulePool is used for some peridic jobs, not from the query
context, i.e. flush of Buffer table.

And for such jobs there cannot be any query context, and more
importantly it will not work correctly since that query_context will
eventually expires.

And this is the reason of this failures [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/46668/015991bc5e20c787851050c2eaa13f0fef3aac00/stateless_tests_flaky_check__asan_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-22 10:50:51 +01:00
Alexander Tokmakov
592af6d652 fix incomplete interst through http 2023-02-22 02:34:03 +01:00
kssenii
1f0ab8d427 Hide disk setting arguemtns 2023-02-21 19:24:01 +01:00
Igor Nikonov
bf05082984
Merge branch 'master' into keeper-retries-by-default 2023-02-21 17:59:55 +01:00
Alexey Milovidov
848286eca7
Merge pull request #40945 from arthurpassos/add_select_final_mt_setting 
Add user setting to force select final at query level
 2023-02-21 19:58:45 +03:00
Kruglov Pavel
ef0d6becba
Merge branch 'master' into null-as-default-all-formats 2023-02-21 16:52:39 +01:00
Azat Khuzhin
a110e0f022 Allow to ignore errors while pushing to MATERILIZED VIEW 
This can be useful in the following scenarious:

- you want to duplicate the data to another table and you don't care
  about the errors
- you want to duplicate system.*_log to another server, you are adding
  materialized view that will push to Distributed table, but you don't
  want to miss original blocks in the local system.*_log
- you want to push some data to a 3d party service, using i.e. URL
  engine.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-21 16:28:38 +01:00
Alexander Gololobov
5354b0d069
Merge pull request #46140 from ClickHouse/split_prewhere_dag 
Split PREWHERE DAG into multiple read steps
 2023-02-21 14:37:09 +01:00
Alexey Milovidov
67d808f49a Add settings changes history 2023-02-21 00:30:38 +01:00
Alexey Milovidov
4d0ba39e7f Merge branch 'master' into keeper-retries-by-default 2023-02-21 00:27:37 +01:00
Kruglov Pavel
9866ecfe8b
Merge branch 'master' into null-as-default-all-formats 2023-02-20 20:49:30 +01:00
Nikolai Kochetov
ee597b939f Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 19:46:52 +00:00
Arthur Passos
4f2fcf763b
Merge branch 'master' into add_select_final_mt_setting 2023-02-20 16:22:14 -03:00
Nikolai Kochetov
2a8460d962 Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 13:54:05 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
pufit
ec49fbec23
Update src/Core/ServerSettings.h 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-02-19 16:49:01 -05:00
Arthur Passos
3d9b23e5f3
Merge branch 'master' into add_select_final_mt_setting 2023-02-19 15:01:41 -03:00
Alexey Milovidov
17992b178a
Merge pull request #45364 from nickitat/aggr_partitions_independently 
Add option to aggregate partitions independently
 2023-02-19 17:44:18 +03:00
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer planner fixes before enable by default
 2023-02-19 13:49:18 +03:00
Nikita Taranov
f70044f34b Merge branch 'master' into aggr_partitions_independently 2023-02-18 13:19:05 +00:00
pufit
4496db66b2 Implement system.server_settings 2023-02-17 23:46:09 -05:00
Igor Nikonov
486501296c Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-17 14:40:02 +00:00
Antonio Andelic
ab51c1d975
Merge branch 'master' into custom-key-parallel-replicas 2023-02-17 15:06:41 +01:00
Maksim Kita
11965e59e2 Analyzer Planner disable by default 2023-02-16 19:46:24 +01:00
Alexander Gololobov
a6061e022e Merge branch 'master' into split_prewhere_dag 2023-02-16 14:43:58 +01:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005) 
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-02-16 16:03:16 +03:00
Maksim Kita
70eb41c6a8 Analyzer add single_join_prefer_left_table setting 2023-02-16 12:17:03 +01:00
Maksim Kita
96fb630231 Analyzer Planner enable by default 2023-02-16 12:17:01 +01:00
Alexey Milovidov
77f60311da
Merge branch 'master' into add_select_final_mt_setting 2023-02-15 15:35:37 +03:00
Igor Nikonov
9c25706711 Update settings history 2023-02-14 14:29:25 +00:00
Antonio Andelic
adde580756
Merge branch 'master' into custom-key-parallel-replicas 2023-02-14 14:09:12 +01:00
Igor Nikonov
49763f7dc2 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-14 12:13:22 +00:00
Kruglov Pavel
d67e7e47f5
Update src/Core/Settings.h 2023-02-13 13:28:46 +01:00
Antonio Andelic
8d16fe5793
Merge branch 'master' into add-support-for-settings-alias 2023-02-13 08:46:00 +01:00
Alexander Gololobov
7634b9f56e Added settings to enable the new logic 2023-02-12 21:54:23 +01:00
Alexey Milovidov
3ae526d4ba Enable retries for INSERT by default in case of ZooKeeper session loss 2023-02-12 04:57:53 +01:00
Alexey Milovidov
6e0dab71ed
Merge pull request #46188 from bigo-sg/rewrite_array_exists 
Rewrite array exists to has
 2023-02-12 05:53:22 +03:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Alexey Milovidov
44bd95a410
Merge pull request #46167 from ClickHouse/rs/reject-dos-patterns 
Reject hyperscan regexes which are prone to ReDoS
 2023-02-11 06:04:03 +03:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy 
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
 2023-02-10 21:38:50 +01:00
Kruglov Pavel
2a3cb8b4ee
Merge pull request #45340 from Avogar/parquet-fixed-binary 
Support FixedSizeBinary type in Parquet/Arrow
 2023-02-10 18:31:20 +01:00
avogar
d1efd02480 Extend setting input_format_null_as_default for more formats 2023-02-10 16:41:09 +00:00
Alexander Tokmakov
8eae498ec6
Merge pull request #45936 from vitlibar/add-setting-check-referential-deps-on-drop 
Check referential dependencies on DROP TABLE
 2023-02-10 16:21:00 +03:00
kssenii
8458c7821b Review comments fix 2023-02-10 12:06:24 +01:00
Robert Schulze
74937cf27b
Reject DoS-prone hyperscan regexes 2023-02-09 17:17:35 +00:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias 2023-02-09 16:07:45 +01:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
Robert Schulze
e490ec91d9
Merge branch 'master' into rs/fix-fragile-linking 2023-02-09 11:33:59 +01:00
Kruglov Pavel
b5f90c608a
Merge branch 'master' into parquet-fixed-binary 2023-02-09 11:26:38 +01:00
taiyang-li
af7a6abf80 fix bugs 2023-02-09 15:15:08 +08:00
taiyang-li
48f0fb0d52 rewrite array exists to has 2023-02-09 12:06:42 +08:00
Vitaly Baranov
4869d3806c Add setting check_referential_table_dependencies to check referential dependencies on DROP TABLE. 2023-02-08 23:56:59 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Vladimir C
247d6c1edd
Merge pull request #44730 from bigo-sg/optimize_aggregate_function_withif 2023-02-08 15:26:47 +01:00
Robert Schulze
6ff232d782
Merge branch 'master' into rs/fix-fragile-linking 2023-02-08 12:51:12 +01:00
Kruglov Pavel
4e2918cee3
Merge branch 'master' into parquet-fixed-binary 2023-02-08 12:31:13 +01:00
Kruglov Pavel
0e5ceae896
Merge pull request #45296 from bigo-sg/improve_multi_if 
Improve performance of function multiIf by columnar executing
 2023-02-08 11:33:12 +01:00
Robert Schulze
10af0b3e49
Reduce redundancies 2023-02-07 12:27:23 +00:00
Han Fei
021e6e90cc
Merge pull request #45631 from hanfei1991/hanfei/regexp-refine 
RegExpTreeDict use re2 engines when processing heavy regexps
 2023-02-07 13:21:22 +01:00
Igor Nikonov
5709c9832c Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-07 11:49:25 +00:00
Robert Schulze
08c1f8346e
Merge remote-tracking branch 'origin/master' into rs/fix-fragile-linking 2023-02-07 11:22:00 +00:00
kssenii
6cea67d58b Cleanup 2023-02-06 18:11:33 +01:00
Kseniia Sumarokova
38c001ca42
Merge pull request #45957 from xiedeyantu/s3_file_not_found 
Throw an error on no files satisfying S3 wildcard
 2023-02-06 12:32:12 +01:00
Antonio Andelic
f67e5505ab Merge branch 'master' into custom-key-parallel-replicas 2023-02-06 11:12:39 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking 
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
 2023-02-05 20:49:34 +00:00
Antonio Andelic
12569da984 Merge branch 'master' into add-support-for-settings-alias 2023-02-05 16:08:57 +00:00
Nikita Taranov
b983b363f8 Merge branch 'master' into aggr_partitions_independently 2023-02-04 18:24:31 +00:00
xiedeyantu
f13eedd644 change settings name 2023-02-04 22:11:14 +08:00
kssenii
cce23cee51 Fix build 2023-02-04 13:49:32 +01:00
Han Fei
9ea3de14ce use re2 by default 2023-02-04 10:53:54 +01:00
Alexey Milovidov
22bd0b6f69
Merge pull request #38983 from CurtizJ/randomize-mt-settings 
Allow to randomize merge tree settings in tests
 2023-02-04 02:59:52 +01:00
kssenii
536c02ca8a Adding custom field continued 2023-02-03 21:05:18 +01:00
kssenii
5f124263a6 Revert changes with complex SettingsChanges 2023-02-03 21:04:45 +01:00
Antonio Andelic
60e5ee84c2 Extract common code 2023-02-03 18:44:56 +00:00
Nikolai Kochetov
310a4c69af Aggregate Projections analysis using query plan [In progress] 2023-02-03 18:30:06 +00:00
Antonio Andelic
3a8ba689f2 Update tests and system.settings 2023-02-03 18:26:31 +00:00
Antonio Andelic
8929838fcc Merge branch 'master' into add-support-for-settings-alias 2023-02-03 17:47:00 +00:00
kssenii
12b62de374 Add field CustomType 2023-02-03 17:07:02 +01:00
Igor Nikonov
38a0ef8743 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-03 15:13:06 +00:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
Alexey Milovidov
c91b5c264f
Merge pull request #45940 from ClickHouse/fix-uring 
Fix IO URing
 2023-02-03 15:36:29 +03:00
Igor Nikonov
7b922784e8 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-03 10:29:37 +00:00
Alexey Milovidov
700503d64a Fix IO URing 2023-02-03 07:00:40 +01:00
xiedeyantu
562642ab7f add settings s3_allow_throw_if_mismatch_files 2023-02-03 12:27:13 +08:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Antonio Andelic
a39e4e24c6
Merge branch 'master' into optimize_parquet_reader 2023-02-02 14:18:00 +01:00
Antonio Andelic
f613bfd8d2 Merge branch 'master' into add-support-for-settings-alias 2023-02-02 11:15:39 +00:00
Igor Nikonov
8f873d5682 Disable optimize_duplicate_order_by_and_distinct 
+ tests
 2023-02-01 21:13:44 +00:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Igor Nikonov
d3d5e67a36 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-01 12:03:06 +00:00
liuneng
17fc22a21e add parquet max_block_size setting 2023-02-01 18:29:20 +08:00
Robert Schulze
b512316586
Merge pull request #45682 from ClickHouse/rename-qrc-to-qc 
Rename "Query Result Cache" to "Query Cache"
 2023-02-01 11:23:29 +01:00
Antonio Andelic
ec04cf6876 Merge branch 'master' into add-support-for-settings-alias 2023-02-01 09:01:01 +00:00
Alexey Milovidov
bf9f62dcbd
Merge pull request #38456 from sauliusvl/iouring 
Re-introduce io_uring read method
 2023-02-01 06:41:32 +03:00
Arthur Passos
a8489fbef3 improve docs 2023-01-31 15:29:30 -03:00
Arthur Passos
2c15ede37c Change setting name from force_select_final to final 2023-01-31 15:21:00 -03:00
taiyang-li
7825ee95e9 merge master and solve conflicts 2023-01-31 20:49:59 +08:00
Antonio Andelic
c99efa75b7
Merge branch 'master' into custom-key-parallel-replicas 2023-01-31 11:58:30 +01:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache" 
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
 2023-01-31 09:54:34 +00:00
Robert Schulze
659a64a1d9
Merge pull request #45679 from ClickHouse/untangle-query-result-cache-settings 
Untangle configuration of query result cache
 2023-01-31 10:44:14 +01:00
Igor Nikonov
f8bcf78593 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-01-30 23:12:09 +00:00
Nikita Taranov
3ce725023c fix style & build 2023-01-30 17:11:56 +00:00
Nikita Taranov
962f163dbd add settings 2023-01-30 17:11:52 +00:00
Robert Schulze
abac59477b
Update src/Core/Settings.h 
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
 2023-01-30 17:12:01 +01:00
Robert Schulze
b42383f0d4
Update src/Core/Settings.h 
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
 2023-01-30 17:11:53 +01:00
Arthur Passos
48e5211454
Merge branch 'ClickHouse:master' into add_select_final_mt_setting 2023-01-30 13:01:36 -03:00
Robert Schulze
27b978381b
Use more speaking setting names 2023-01-30 14:20:43 +00:00
taiyang-li
c45111349f Merge branch 'master' into improve_multi_if 2023-01-30 16:32:55 +08:00
Alexey Milovidov
c3c1ed01d9
Merge branch 'master' into bg-pool-catch 2023-01-29 06:19:38 +03:00
taiyang-li
7f3114d806 merge master and solve conflicts 2023-01-29 09:59:16 +08:00
Saulius Valatka
ac2c921bdf add initial io_uring support 2023-01-28 21:54:44 +02:00
Kruglov Pavel
96700abbe1
Merge pull request #45678 from azat/formats/json-parse-tupels 
Add ability to ignore unknown keys in JSON object for named tuples
 2023-01-27 21:11:05 +01:00
Igor Nikonov
101a0cc90b Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-01-27 19:08:38 +00:00
Igor Nikonov
300f78df96
Merge pull request #45567 from ClickHouse/enable-remove-redundant-sorting 
Enable query_plan_remove_redundant_sorting optimization by default
 2023-01-27 19:14:36 +01:00
Anton Popov
d8fe2bcbaa
Merge branch 'master' into randomize-mt-settings 2023-01-27 16:03:00 +01:00
Robert Schulze
e30c38cab3
Fix up some leftovers 2023-01-27 10:43:43 +00:00
Antonio Andelic
193661f11e Remove unused field 2023-01-27 10:08:22 +00:00
Antonio Andelic
defc9108f0 Extend support for aliases to constraints and profiles 2023-01-27 09:39:10 +00:00
Azat Khuzhin
1a8437f2c9 Add ability to ignore unknown keys in JSON object for named tuples 
This can be useful in case your input JSON is complex, while you need
only few fields in it.

This behaviour is controlled by the
input_format_json_ignore_unknown_keys_in_named_tuple setting name, that
is turned OFF by default.

This will, almost, allow to parse gharchive dataset without jq. "almost"
because of two things:
- Tuple cannot be Nullable, so such keys with Tuple type in ClickHouse
  cannot be `null` in JSON
- You cannot use dot.dot notation to extract columns for file() engine,
  only tupleElement()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-27 10:01:08 +01:00
Igor Nikonov
4480d8ae60 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-01-26 22:16:24 +00:00
Robert Schulze
7d97fab2dc
Untangle query result cache settings 
This PR modularizes the configuration options of the query result cache
(QRC).

1. Add a feature toggle "allow_experimental_query_result_cache = false"
   which controls whether the experimental QRC can be used at all. This
   configuration option will be removed once the QRC becomes
   non-experimental.

2. Modularize the existing two configuration settings
   "enable_experimental_query_result_cache" and
   "enable_experimental_query_result_cache_passive_usage" into three
   orthogonal settings:

   - "use_query_result_cache = false" controls whether a query (or all
     queries of the session) utilize the query result cache

   - settings "use_query_result_cache_passive_usage = true" and
     "use_query_result_cache_active_usage = true" control the way a
     query (or all queries of the session) utilize the query result
     cache: "passive" use is reading, "active" use is writing.
 2023-01-26 21:17:48 +00:00
Alexander Tokmakov
067b1f5f13 Merge branch 'master' into exception_message_patterns4 2023-01-26 15:20:58 +01:00
Igor Nikonov
ba92705ea1 Fix duplicated setting in changes history 2023-01-26 14:20:44 +00:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
Anton Popov
92fa39798b Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-26 13:29:23 +00:00
Antonio Andelic
07b2c5ce0f
Merge branch 'master' into custom-key-parallel-replicas 2023-01-26 12:42:04 +01:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Kseniia Sumarokova
297516a084
Merge pull request #45614 from kssenii/rename-some-setting 
Rename setting `max_query_cache_size`
 2023-01-26 10:46:03 +01:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
Igor Nikonov
eea076cee0 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-01-25 20:16:41 +00:00
Igor Nikonov
2b5206c223 Merge remote-tracking branch 'origin/master' into enable-remove-redundant-sorting 2023-01-25 16:32:14 +00:00
Igor Nikonov
fc23ff6ad2 Add entity into setting changes history for compatibility mode 2023-01-25 16:28:54 +00:00
kssenii
29de87e145 Rename setting 2023-01-25 16:36:36 +01:00
Alexander Tokmakov
6eb557b2ba Merge branch 'master' into exception_message_patterns4 2023-01-25 13:49:17 +01:00
Vladimir C
cfefaebe8f
Merge pull request #45558 from ClickHouse/vdimir/remove-ast-fuse 2023-01-25 11:25:36 +01:00
Alexander Tokmakov
dd57215934 Merge branch 'master' into exception_message_patterns4 2023-01-24 17:03:12 +01:00
Kruglov Pavel
23c12ac8ee
Merge branch 'master' into parquet-fixed-binary 2023-01-24 16:51:05 +01:00
vdimir
0e09d0a00c
Remove FuseSumCountAggregatesVisitor 2023-01-24 14:58:26 +00:00
Kruglov Pavel
4bd3f0e5ef
Merge pull request #44953 from Avogar/tsv-csv-detect-header 
Detect header in CSV/TSV/CustomSeparated files automatically
 2023-01-24 15:13:52 +01:00
Alexander Tokmakov
6ecae8388e Merge branch 'master' into exception_message_patterns4 2023-01-24 14:42:36 +01:00
Igor Nikonov
41b94b4954 Enable query_plan_remove_redundant_sorting optimization by default 2023-01-24 13:38:21 +00:00
Igor Nikonov
7108189b45 Merge remote-tracking branch 'origin/master' into revert-45414-revert-43905-igor/remove_redundant_order_by 2023-01-24 11:46:24 +00:00
Igor Nikonov
ba5fff2a8a Disable the optimization to avoid sort.xml perf test fail in other PRs 2023-01-24 11:42:56 +00:00
Antonio Andelic
95853af459 Merge branch 'master' into custom-key-parallel-replicas 2023-01-24 10:49:40 +00:00
Antonio Andelic
37b62b3a58 Use Map for custom_key 2023-01-24 10:46:47 +00:00
Kruglov Pavel
cd1cd904a7
Merge branch 'master' into tsv-csv-detect-header 2023-01-23 23:49:56 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Kruglov Pavel
478a552a0a
Merge branch 'master' into tsv-csv-detect-header 2023-01-23 21:47:17 +01:00
Anton Popov
08b2e524c7 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-23 15:34:53 +00:00
Kruglov Pavel
01ddf326ac
Merge branch 'master' into parquet-fixed-binary 2023-01-23 15:31:45 +01:00
Igor Nikonov
f807b4b315 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-01-22 18:30:30 +00:00
Antonio Andelic
44ef00dc05 Merge branch 'master' into custom-key-parallel-replicas 2023-01-22 14:20:58 +00:00
Robert Schulze
340f406553
Merge branch 'master' into query-result-cache 2023-01-22 13:21:36 +01:00
Anton Popov
41a199e175
Fix crash when ListObjects request fails (#45371) 2023-01-20 20:10:23 +01:00
Kruglov Pavel
28ddcc2432
Merge branch 'master' into tsv-csv-detect-header 2023-01-20 15:08:38 +01:00
Antonio Andelic
fc9c53f963 Merge branch 'master' into custom-key-parallel-replicas 2023-01-20 11:18:30 +00:00
larryluogit
52ae33dba7
Merge branch 'master' into ftsearch 2023-01-19 11:34:11 -05:00
Igor Nikonov
1866f990de
Revert "Revert "Remove redundant sorting"" 2023-01-18 20:12:34 +01:00
Igor Nikonov
7ed8fec94f
Revert "Remove redundant sorting" 2023-01-18 18:38:25 +01:00
Robert Schulze
4f90824347
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-17 22:49:53 +00:00
Igor Nikonov
0cfa08df7a Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2023-01-17 16:28:17 +00:00
Antonio Andelic
a85565c4a6 Merge branch 'master' into custom-key-parallel-replicas 2023-01-17 07:37:18 +00:00
avogar
5bf4704e7a Support FixedSizeBinary type in Parquet/Arrow 2023-01-16 21:01:31 +00:00
Kruglov Pavel
e9d6590926
Merge branch 'master' into tsv-csv-detect-header 2023-01-16 17:50:24 +01:00
Kruglov Pavel
bdb3517512
Merge pull request #45231 from Avogar/json-tuples 
Insert default values in case of missing tuple elements in JSONEachRow
 2023-01-16 17:49:50 +01:00
avogar
1c0941d72a Add docs and examples 2023-01-16 16:46:41 +00:00
avogar
3ea80b0f54 Merge branch 'master' of github.com:ClickHouse/ClickHouse into tsv-csv-detect-header 2023-01-16 15:14:25 +00:00
Igor Nikonov
a34991cb65 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2023-01-16 12:14:02 +00:00
Antonio Andelic
57e771226c Add range mode for custom key 2023-01-16 10:11:21 +00:00
Antonio Andelic
36cf9debae Merge branch 'master' into custom-key-parallel-replicas 2023-01-16 08:11:08 +00:00
Robert Schulze
099e30ef2a
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-16 08:04:49 +00:00
taiyang-li
4448bfe8a7 try to improve performance of function multiif 2023-01-16 15:08:12 +08:00
Vladimir C
eefbffcc5b
Merge pull request #45230 from ClickHouse/vdimir/semi_join_null_const_bug 2023-01-13 13:22:57 +01:00
vdimir
023162df1d
fix clang-tidy style 2023-01-13 11:25:07 +00:00
Antonio Andelic
e64f07e72b Merge branch 'master' into custom-key-parallel-replicas 2023-01-13 08:44:53 +00:00
avogar
87b934c472 Insert default values in case of missing tuple elements in JSONEachRow 2023-01-12 16:36:44 +00:00
vdimir
81891f7bde
Do not compare empty const columns in checkBlockStructure 2023-01-12 16:31:29 +00:00
Anton Popov
ca80240d84 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-12 12:13:45 +00:00
Kruglov Pavel
05a11ff4a4
Merge branch 'master' into tsv-csv-detect-header 2023-01-12 12:35:18 +01:00
Arthur Passos
1b3c7766f8
Merge branch 'master' into add_select_final_mt_setting 2023-01-12 08:31:04 -03:00
Antonio Andelic
ecbffa80b6 Add READ_TASKS mode 2023-01-12 07:56:15 +00:00
Kruglov Pavel
fee76a8919
Better comment 2023-01-11 23:00:43 +01:00
Kruglov Pavel
b56d355b96
Fix pissible stack-use-after-return in LimitReadBuffer 2023-01-11 22:59:36 +01:00
李扬
ef11a6c281
Merge branch 'master' into optimize_aggregate_function_withif 2023-01-11 10:40:38 +08:00
avogar
035dc33707 Fix builds 2023-01-10 19:22:18 +00:00
Kruglov Pavel
ce6962614d
Merge branch 'master' into mmap-for-storage-file 2023-01-10 17:34:01 +01:00
Antonio Andelic
347ccab8f9
Merge branch 'master' into custom-key-parallel-replicas 2023-01-10 16:31:40 +01:00
Han Fei
6ed4570f73
Merge branch 'master' into regexp-tree-dictionary 2023-01-10 15:36:30 +01:00
Antonio Andelic
fbe4808b6b Add support for custom key in parallel replicas 2023-01-10 13:13:43 +00:00
Robert Schulze
923fa2c15a
Fix review comments, pt. II 2023-01-10 10:21:08 +00:00
Robert Schulze
0c3b034887
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-10 09:02:41 +00:00
Yakov Olkhovskiy
2559c49810 make only IPv4 as numeric, fix hash functions for IPv6, add more hash functions to test 2023-01-10 02:03:06 +00:00
Robert Schulze
ad4884b394
Fix review comments, pt. I 2023-01-09 21:42:51 +00:00
Igor Nikonov
7298b39a18 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2023-01-09 10:08:48 +00:00
Igor Nikonov
cec7dc5a5a Renaming according to setting 2023-01-09 10:06:47 +00:00
Igor Nikonov
e29cd5d640 Rename settings to remove_redundant_sorting 2023-01-09 09:50:00 +00:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types 
Replace domain IP types (IPv4, IPv6) with native
 2023-01-07 12:01:21 +01:00
Robert Schulze
45dbcf88e5
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-06 20:42:48 +00:00
Dmitry Novik
9d44c7a737
Merge pull request #44541 from ClickHouse/cleanup-resolved-function 
Followup #43761
 2023-01-06 18:09:32 +01:00
kssenii
2a030c1dc0 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-01-06 15:58:27 +01:00
Igor Nikonov
48b58a9fed Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2023-01-06 13:55:26 +00:00
avogar
7fcdb08ec6 Detect header in CSV/TSV/CustomSeparated files automatically 2023-01-05 22:57:25 +00:00
Azat Khuzhin
ae2bd711d9 Catch exceptions in BackgroundSchedulePool 
I saw the case when one of users of BackgroundSchedulePool
(DirectoryMonitor for async INSERT into Distributed table) throws (in
a very unlikely scenaior, which wich #44922 may help), and this case the
task is leaved in an undefined state, and instead of fix this I put
assert to catch such code instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-04 21:49:47 +01:00
Igor Nikonov
1edfdfda4d Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2023-01-04 12:34:40 +00:00
李扬
02ad53a5dc
Merge branch 'master' into optimize_aggregate_function_withif 2023-01-04 11:06:50 +08:00
Han Fei
00e717d7ce some improvement 2023-01-03 21:41:51 +01:00
Dmitry Novik
a84318af5f
Merge branch 'master' into cleanup-resolved-function 2023-01-03 17:52:29 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
taiyang-li
eb698ea322 change as request 2023-01-03 11:28:35 +08:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types 2023-01-02 21:58:53 +03:00
Kruglov Pavel
f6deea1365
Try fix build 2023-01-02 16:51:11 +01:00
Kruglov Pavel
ebbd47012e
Merge pull request #44657 from ClickHouse/Avogar-patch-1 
Enable input_format_json_read_objects_as_strings by default
 2023-01-02 16:14:33 +01:00
Robert Schulze
8381d8b467
query_result_cache_keep_seconds_alive --> query_result_cache_ttl 2023-01-02 09:38:40 +00:00
Robert Schulze
ba55fd3ad2
Cosmetics 2023-01-02 09:30:44 +00:00
Robert Schulze
e9e04166d9
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-02 08:20:27 +00:00
HarryLeeIBM
6a5fef632d Added more tests and experimental flag, etc. 2022-12-31 18:05:01 -08:00
taiyang-li
f398811d37 rewrite aggregate functions with if 2022-12-30 17:34:37 +08:00
Kruglov Pavel
9736bdad72
Remove unnecessary header 2022-12-28 14:46:27 +01:00
Kruglov Pavel
11b9fb4a3e
Add settings change in history 2022-12-27 23:42:19 +01:00
Kruglov Pavel
009938d521
Enable input_format_json_read_objects_as_strings by default 2022-12-27 23:40:09 +01:00
Kruglov Pavel
f582cb4c9f
Merge branch 'master' into fix-comp-setting 2022-12-27 16:53:24 +01:00
Alexey Milovidov
462e7f76bf Attempt to fix the "system.stack_trace" test 2022-12-27 13:28:25 +01:00
alesapin
600bedbff4 Add setting to disallow arguments in ReplicatedMergeTree constructor for DatabaseReplicated 2022-12-24 18:38:14 +01:00
Dmitry Novik
e481c0bae5 Cleanup code 2022-12-23 18:23:01 +00:00
kssenii
853f2ea123 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 11:49:02 +01:00
Dmitry Novik
cff882d506 Merge remote-tracking branch 'origin/master' into refector-function-node 2022-12-22 21:34:29 +00:00
Igor Nikonov
a7aa3a4d9e Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2022-12-22 11:22:01 +00:00
Kseniia Sumarokova
993fb387d8
Merge pull request #44289 from kssenii/write-throw-cache-fix-writing 
Fix possible unneeded cache write interruption in write-through cache
 2022-12-22 11:31:29 +01:00
Yakov Olkhovskiy
a8cb29da4b
Merge branch 'master' into refactoring-ip-types 2022-12-21 23:56:24 -05:00
kssenii
d3db1dd6a7 Fix tests 2022-12-22 00:27:22 +01:00
Kruglov Pavel
524e6cd799
Merge branch 'master' into fix-comp-setting 2022-12-21 10:48:19 +01:00
kssenii
6bd4f8c029 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-20 21:17:28 +01:00
Igor Nikonov
5cd4767838 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2022-12-20 17:56:59 +00:00
Dmitry Novik
4793412887
Merge branch 'master' into refector-function-node 2022-12-20 18:26:19 +01:00
Kruglov Pavel
643a35bed1
Merge pull request #44019 from Avogar/refactor-schema-inference 
Refactor and improve schema inference for text formats
 2022-12-20 17:29:03 +01:00
Kruglov Pavel
bd3efc458c
Merge branch 'master' into fix-comp-setting 2022-12-20 14:54:22 +01:00
Anton Popov
56187c5f0e Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-20 13:37:23 +00:00
Kseniia Sumarokova
b8bf28eee0
Merge branch 'master' into write-throw-cache-fix-writing 2022-12-20 11:40:49 +01:00
Nikolai Kochetov
ce56ae61c6
Merge pull request #44335 from ClickHouse/check-s3-part-upload-settings 
Check s3 part upload settings
 2022-12-19 21:12:06 +01:00
Igor Nikonov
65ab584e0f Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2022-12-19 19:50:10 +00:00
Igor Nikonov
1e53b3be38 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by 2022-12-19 16:44:22 +00:00