thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-27 18:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
118,055 Commits 9,188 Branches 1,782 Tags 2.1 GiB
bd88a2195a
Commit Graph

43194 Commits

Author SHA1 Message Date
Kseniia Sumarokova
c3227fd44f
Merge pull request #51161 from kssenii/more-metrics-in-cache 
More profile events for fs cache
 2023-06-20 12:44:00 +02:00
Maksim Kita
7e5017dd31 Fixed tests 2023-06-20 11:42:22 +03:00
Maksim Kita
aa28a1f259 Fixed tests 2023-06-20 11:42:22 +03:00
Maksim Kita
6e26fde707 Updated tests 2023-06-20 11:42:22 +03:00
Maksim Kita
129b97c727 JIT infrastructure refactoring 2023-06-20 11:42:22 +03:00
zvonand
4754d7863c small 2023-06-20 03:21:03 +02:00
Andrey Zvonov
07191ce10d
Merge branch 'master' into zvonand-implicit-tz 2023-06-20 00:24:53 +02:00
Jordi Villar
6182a71c4d
Merge branch 'master' into fix-summing-mt-dt64 2023-06-19 21:15:23 +02:00
Nikolai Kochetov
7280078bc6 Refactoring a bit more. 2023-06-19 19:01:56 +00:00
kssenii
7f9cd56f61 One more 2023-06-19 19:22:45 +02:00
Han Fei
959f66eb8a
Merge pull request #50697 from kitaisreal/sort-block-improvements 
Sort block improvements
 2023-06-19 19:02:03 +02:00
Anton Popov
1ef19dbe4d drop cache on check of broken part 2023-06-19 16:32:51 +00:00
kssenii
ec86028fd8 Change to microseconds 2023-06-19 17:48:56 +02:00
kssenii
0ca2916163 More metrics in cache 2023-06-19 17:22:41 +02:00
Sergei Trifonov
5867f39fc7
Merge pull request #50675 from ClickHouse/unify-priorities-pools 
Unify priorities for connection pools
 2023-06-19 17:16:15 +02:00
Alexander Gololobov
44f1f036e8 Use multi-read requests for list and get 2023-06-19 16:47:17 +02:00
avogar
cf8f277c32 Add total_bytes_to_read to TCP protocol for better Progress bar 2023-06-19 13:58:08 +00:00
avogar
d492acbcd2 Fix tests 2023-06-19 13:36:29 +00:00
Nikolai Kochetov
8e7d06e0a4 Remove isReady from FutureSet iface. 2023-06-19 12:56:24 +00:00
Alexey Milovidov
f9558345e8
Merge pull request #51138 from ClickHouse/print-changed-settings 
Print changed settings on fatal errors
 2023-06-19 15:50:44 +03:00
Nikolai Kochetov
f3398f9908 Try to make more consisten type check for sets. 2023-06-19 12:17:52 +00:00
Raúl Marín
8fdcca4326
Merge branch 'master' into parallel_replicas_analyzer 2023-06-19 13:54:59 +02:00
Nikita Taranov
90ced1aebc use int64 2023-06-19 13:48:52 +02:00
Sema Checherinda
a2d5c91a28 cancel temp part with exception context when deduplication triggered 2023-06-19 12:52:20 +02:00
Nikolai Kochetov
880c9aea09 Fixing style. 2023-06-19 10:47:43 +00:00
Alexander Tokmakov
a00842498e Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-06-19 12:37:17 +02:00
Alexander Tokmakov
caceb7c862
Revert "Added ability to implicitly use file/hdfs/s3 table functions in clickhouse-local" 2023-06-19 13:21:09 +03:00
Maksim Kita
4c72995537 Fixed build 2023-06-19 13:02:35 +03:00
Jordi Villar
5e77fa6c87
Merge branch 'master' into fix-summing-mt-dt64 2023-06-19 08:26:36 +02:00
Alexey Milovidov
dcd4cfa5c2 Fix build 2023-06-19 03:28:59 +02:00
jinjunzh
004755914e change to use generic MSan API to bypass uninitialized memory check 2023-06-18 20:55:10 -04:00
Sema Checherinda
e3f892f84a fix gtest with MemoryWriteBuffer, do not mute exception in ReplicatedMergeMutateTaskBase 2023-06-19 00:10:42 +02:00
Alexey Milovidov
9ae4915655 Code simplification 2023-06-18 21:08:37 +02:00
Sema Checherinda
2bbfa279ec fix gtest_lru_file_cache, fix remove from pending 2023-06-18 21:05:00 +02:00
Alexey Milovidov
b19b87a3e5 Whitespaces 2023-06-18 20:42:27 +02:00
Alexey Milovidov
a6dde6ab62 Print changed settings on fatal errors 2023-06-18 20:38:25 +02:00
Sema Checherinda
ce7cd67981 finalize CascadeWriteBuffer 2023-06-18 17:35:12 +02:00
Maksim Kita
977cd65cfb Fixed build 2023-06-18 12:52:08 +03:00
Maksim Kita
2f3dce5fe6 Fixed tests 2023-06-18 12:52:08 +03:00
Maksim Kita
7d17a70d17 Fixed tests 2023-06-18 12:52:08 +03:00
Maksim Kita
8d3212363e Fixed tests 2023-06-18 12:52:08 +03:00
Maksim Kita
ca298d66cb Sort block improvements 2023-06-18 12:52:08 +03:00
Alexander Gololobov
59c11cb150 Make more parallel get requests to ZooKeeper 2023-06-18 08:26:37 +02:00
Alexey Milovidov
ce38d64c5a
Merge pull request #50856 from ClickHouse/relax-too-many-parts 
Relax "too many parts" further
 2023-06-18 01:49:46 +03:00
Sema Checherinda
642734e512 fix shell commands source 2023-06-17 23:48:09 +02:00
Robert Schulze
5311ebe1d2
Make query cache work effective with nested queries. 2023-06-17 20:27:25 +00:00
Alexander Tokmakov
b4e455430c
Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-06-17 22:19:21 +03:00
Robert Schulze
24717ed908
Factorize check if query cache exists 2023-06-17 18:24:20 +00:00
Robert Schulze
69e7c1cc82
Introduce convenience ctor for Key construction during read 2023-06-17 18:22:15 +00:00
Alexey Milovidov
c7a18c640e Slightly better diagnostics 2023-06-17 17:20:44 +02:00
Alexey Milovidov
61e423c5df Fix typo 2023-06-17 17:14:49 +02:00
Alexey Milovidov
98bc124317 Fix typo 2023-06-17 17:13:19 +02:00
Alexey Milovidov
e913a58284
Merge pull request #51049 from ClickHouse/ddl_replication_improvements 
An optimiation for ALTERs and Replicated db with one shard
 2023-06-17 16:57:59 +03:00
Alexey Milovidov
9fc5b9c2c5
Merge branch 'master' into relax-too-many-parts 2023-06-17 16:14:34 +03:00
Alexey Milovidov
c2e4a7714f
Merge pull request #48821 from alekseygolub/localfilefunction 
Added ability to implicitly use file/hdfs/s3 table functions in clickhouse-local
 2023-06-17 15:55:13 +03:00
Robert Schulze
6e8af09289
Provide query cache reader with dummy block 2023-06-17 12:48:29 +00:00
Robert Schulze
0adc98865d
Add variable for access to found key components 2023-06-17 12:43:14 +00:00
Robert Schulze
840e73eb74
Update #includes 2023-06-17 12:28:18 +00:00
Robert Schulze
83e2196a17
Make is_shared const 2023-06-17 12:26:53 +00:00
Michael Kolupaev
03baa19e88 Fix ParallelReadBuffer seek 2023-06-16 17:50:51 -07:00
Michael Kolupaev
c5536e4906 Fix ColumnConst::forEachSubcolumn 2023-06-16 17:48:36 -07:00
kssenii
74f2dea57b Fix typo 2023-06-17 02:11:48 +02:00
kssenii
b902f5d045 Small style corrections 2023-06-17 01:54:38 +02:00
Alexey Milovidov
df9abccfaa
Merge branch 'master' into localfilefunction 2023-06-17 00:45:54 +03:00
Alexey Milovidov
a18dc73271 Fix error 2023-06-16 23:12:01 +02:00
Alexey Milovidov
688d5d59af Merge branch 'master' into relax-too-many-parts 2023-06-16 23:08:29 +02:00
Alexey Milovidov
ec952d401d
Merge pull request #51114 from ClickHouse/clang-17-3 
Compatibility with clang-17
 2023-06-17 00:07:45 +03:00
Alexey Milovidov
d94e47a930 Compatibility with clang-17 2023-06-16 23:07:19 +02:00
Alexey Milovidov
4e45e39e25
Merge pull request #51099 from ClickHouse/fix_use_local_vars_after_exception 
Fix for part_names_mutex used after destruction
 2023-06-16 23:52:50 +03:00
Alexey Milovidov
782889a051
Merge pull request #51042 from ClickHouse/sys_zookeep_parallel_list 
Make multiple parallel list requests when reading from system.zookeeper
 2023-06-16 23:46:53 +03:00
Nikolai Kochetov
afa74f697c Refactor a bit. 2023-06-16 19:38:50 +00:00
jinjunzh
5f8ef27513 initialize buffer with memset 2023-06-16 14:37:01 -04:00
Michael Kolupaev
10d597676c Refactor IColumn::forEachSubcolumn to make it slightly harder to implement incorrectly 2023-06-16 11:08:12 -07:00
Michael Kolupaev
c2900177a2 Fix race condition in ColumnLowCardinality caused by subtly broken copy-on-write for the dictionary 2023-06-16 11:06:28 -07:00
Sema Checherinda
e8d4dedbcc fix build and tests 2023-06-16 19:52:52 +02:00
Alexander Gololobov
8289d8fefa Fix for part_names_mutex used after destruction 2023-06-16 18:46:07 +02:00
ltrk2
e20cea8901 Implement endianness-independent serialization for quantileTiming 2023-06-16 09:22:38 -07:00
avogar
56789af0fa Remove loggig 2023-06-16 15:55:05 +00:00
avogar
3209ebe34b Improve progress bar for file/s3/hdfs/url table functions. Step 1 2023-06-16 15:51:18 +00:00
Salvatore Mesoraca
6b49968bc4
Generate safe IVs 
IVs should never be re-used when using CTR.
The previous implementation had a 50% probability to generate
an IV that was already used after 65536 generations.
This happens because std::mt19937 is seeded using a 32bit integer
and returns a 32 bits integer.
Because of the birthday problem, collisions have a 50% chance after
only 2^16 IV's generations.
CH uses one IV per file and, for most use cases, 65k files are
not as many as they seem.
Also, the initial entropy is gathered using std::random_device
which is "best-effort" and it's allowed by the standard to
return a fixed sequence of numbers.
 2023-06-16 17:09:08 +02:00
Nikita Taranov
841d58f6ba
Small fix in MergeTreePrefetchedReadPool 2023-06-16 16:58:01 +02:00
Sema Checherinda
47484faace do abort/cancel/deletion merge/mutate task with exception context 2023-06-16 16:48:21 +02:00
Sema Checherinda
cf31fdb4d5 no finalize at d-tor in compression 2023-06-16 16:38:18 +02:00
Sema Checherinda
b428f88f22 finalize hashing buffer 2023-06-16 16:38:18 +02:00
Sema Checherinda
d21fc22607 fix 02428_combinators_with_over_statement 2023-06-16 16:38:18 +02:00
Sema Checherinda
1cb02e2710 do call finalize for all buffers 2023-06-16 16:38:18 +02:00
Sema Checherinda
9a4962bd68 add error log message when no finalize called before d-tor 2023-06-16 16:38:18 +02:00
Nikolai Kochetov
df1ea0b572
Merge pull request #50489 from ClickHouse/vdimir/alter_moving_garbage 
Cleanup moving parts
 2023-06-16 15:39:32 +02:00
Vitaly Baranov
f7638de213
Fix crash in BackupCoordinationStageSync::setError(). (#51012) 2023-06-16 15:32:30 +02:00
Kruglov Pavel
dcc3efe4b7
Merge pull request #50364 from Avogar/allow-skip-empty-files 
Allow to skip empty files in file/s3/url/hdfs table functions
 2023-06-16 14:59:49 +02:00
Alexander Tokmakov
9260a1bf2e
Merge pull request #51006 from ClickHouse/followup_50448 
Follow-up to #50448
 2023-06-16 15:32:12 +03:00
Andrey Zvonov
43e572bd84
typo in SettingsFields.h 2023-06-16 14:55:34 +03:00
Kruglov Pavel
f8ddfb1fd8
Merge branch 'master' into allow-skip-empty-files 2023-06-16 13:23:41 +02:00
Kruglov Pavel
11f176dd19
Merge pull request #50712 from KevinyhZou/bug_fix_csv_parse_by_tab_delimiter 
Support CSVInputFormat to read csv file by whitespace & tab delimiter
 2023-06-16 13:16:22 +02:00
Andrey Zvonov
b3476e53fc
Update SettingsFields.cpp 2023-06-16 13:44:17 +03:00
zvonand
112a3e6aca revert tz validation optimization 2023-06-16 12:19:01 +02:00
Manas Alekar
508e4906c0 Avoid double-catching the exceptions. 2023-06-16 01:47:28 -07:00
Manas Alekar
f610645c07 Move outfile checks before query starts running. 2023-06-16 01:10:11 -07:00
alekar
b70803c71b
Merge branch 'master' into improve-outfile 2023-06-16 01:08:48 -07:00
zvonand
4155d13d69 merge master and resolve conflict 2023-06-16 02:20:32 +02:00
zvonand
e60506f41b changes requested in review 2023-06-16 02:16:04 +02:00
Han Fei
6bbd0d144d
Merge pull request #50951 from ZhiguoZh/20230607-toyear-fix 
Optimization of predicates with toYear/toYYYYMM based on a general solution
 2023-06-16 00:32:55 +02:00
Jordi Villar
8ae05b7e18
Merge branch 'master' into fix-summing-mt-dt64 2023-06-15 21:46:02 +02:00
Robert Schulze
74cb79769b
Merge pull request #50925 from arenadata/ADQM-812 
Implement support of syslog format in the parseDateTimeBestEffort() function
 2023-06-15 21:04:50 +02:00
Alexander Tokmakov
353b57f13f an optimiation for alters and replicated db 2023-06-15 20:20:11 +02:00
Alexander Gololobov
f08fd758fa Make multiple parallel list requests 2023-06-15 18:45:10 +02:00
Nikolai Kochetov
1cb7ba4895 Revert "Trying to fix a test." 
This reverts commit d0d39c8f61.
 2023-06-15 16:42:38 +00:00
Nikolai Kochetov
d34b6e5f35 Try to fix analyzer tests. 2023-06-15 16:27:51 +00:00
Nikolay Degterinsky
b963369920
Merge pull request #50865 from jmaicher/fix/50864/verification-cooldown-cache-entry 
Fix type of LDAP server params hash in cache entry
 2023-06-15 16:21:44 +02:00
Zhiguo Zhou
ff6629d1d1 Enhance safety of function generateOptimizedDateFilterAST 
This commit checks the corner case where the comparator is none
of equals, notEquals, less, lessOrEquals, greater, greaterOrEquals,
and throws LOGICAL_ERROR exception if so.
 2023-06-15 22:04:13 +08:00
Alexander Tokmakov
b248ba730a
Merge pull request #50997 from vitlibar/use-hash_of_all_files-for-parts 
Use hash_of_all_files to check identity of parts during on-cluster backups
 2023-06-15 16:54:55 +03:00
Zhiguo Zhou
d780d0bab1 fix style 
Move error throws in Transform to FunctionDateOrDateTimeToSomething.
 2023-06-15 21:48:02 +08:00
Han Fei
9e81b2fd5d fix style 2023-06-15 21:48:02 +08:00
Zhiguo Zhou
d14299eb09 The general optimization of predicates with date/datetime converters 
As is suggested in issue #15257, the function preimage is a general
solution to the optimization problem with predicates containing the
date and datetime converters. This commit implements the idea by
adding the new methods, hasInformationAboutPreimage and getPreimage,
to IFunction/IFunctionBase, and having the specific convert functions
define their own preimage. Moreover, we added a new pass in the
TreeOptimizer and a new AST visitor for in-place rewriting the AST
with the converters' preimage.

Specifically, the optimization is applied to toYear and toYYYYMM.
 2023-06-15 21:48:02 +08:00
Alexander Tokmakov
dc0a224f52 fix 2023-06-15 15:05:17 +02:00
avogar
870f3d1270 Fix comments 2023-06-15 12:59:46 +00:00
Kseniia Sumarokova
c15e7b93cb
Merge pull request #50976 from kssenii/fix-data-lakes-too-many-head-requests 
Fix data lakes slowness because of synchronous head requests
 2023-06-15 13:33:09 +02:00
Kseniia Sumarokova
31e08635bf
Merge branch 'master' into unify-priorities-pools 2023-06-15 12:51:45 +02:00
Igor Nikonov
1113a7c524
Merge pull request #50214 from azat/parallelize_output_from_storages-fix 
Disable parallelize_output_from_storages for processing MATERIALIZED VIEWs and storages with one block only
 2023-06-15 12:48:54 +02:00
Kseniia Sumarokova
b7fbc4dd8e
Merge pull request #50977 from valbok/empty-table-overrides 
MaterializedMySQL: Keep parentheses for empty table overrides
 2023-06-15 12:48:04 +02:00
Alexander Tokmakov
a018d9ca11 try to fix false-positive 'part is lost forever' 2023-06-15 12:25:16 +02:00
Kruglov Pavel
7aea4a1f10
Merge branch 'master' into allow-skip-empty-files 2023-06-15 12:07:24 +02:00
Antonio Andelic
c1faf42481
Merge pull request #50967 from baibaichen/feature/fix_build_clang15 
fix build issue on clang 15
 2023-06-15 10:32:38 +02:00
Kseniia Sumarokova
c786fbf8bd
Add comment 2023-06-15 10:22:02 +02:00
Kseniia Sumarokova
c8619ee6e4
Merge pull request #50974 from kssenii/iceberg-metadata-fix 
Fix iceberg V2 optional metadata parsing
 2023-06-15 09:31:36 +02:00
KevinyhZou
953f40aa3b
Merge branch 'master' into bug_fix_csv_parse_by_tab_delimiter 2023-06-15 10:25:19 +08:00
Michael Kolupaev
badde0fde2 Print git hash when crashing 2023-06-14 15:22:44 -07:00
Thomas Panetti
dbdf77ce51
Merge branch 'master' into mysql_compatible_types 2023-06-14 13:05:54 -07:00
Jiebin Sun
fcadb851c8
Maintain per-thread timer_id rather than create/delete frequently (#48778) 
* Maintain per-thread timer_id rather than create/delete frequently

The QueryProfiler will frequently create/delete timer_id globally, which
will result in heavy kernel lock contention.
The idea is to maintain thread-local timer_id. Before create the
timer_id, it should check whether there is a timer_id already. And we
could stop the timer by timer_settime() rather than delete the timer_id
with timer_delete().

Apply the patch and run clickbench on latest 65d671b7c7 ClickHouse with
SPR 112 x 2 vCPUs. Query 4, 0, 5, 3, 15, 32 have 17.5%, 14.4%, 8.3%, 7.9%,
7.1%, 5.8% performance gain. The overall geomean has got 2.5%
performance gain.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

* Pack the timer and delete the timer_id when thread terminates

Pack the timer and related methods into the class. Delete the timer_id
when the thread terminates.

According to the issue (ClickHouse#49965),
all of the SSB queries benefit from this optimization, some have even got
improved by ~30% and the overall QPS could be significantly improved by ~18%.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

* Update src/Common/QueryProfiler.cpp

Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>

* Update src/Common/QueryProfiler.cpp

Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>

* Fix the review issue of QueryProfiler Timer from PR
https://github.com/ClickHouse/ClickHouse/pull/48778.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

* Update src/Common/QueryProfiler.cpp

Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>

* Add two separate CurrentMetrics for created and active timers
in QueryProfiler.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

---------

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
 2023-06-14 21:22:09 +02:00
Vitaly Baranov
6366940a37 Use hash_of_all_files from system.parts to check identity of parts during on-cluster backups. 2023-06-14 20:39:50 +02:00
Nikolai Kochetov
d0d39c8f61 Trying to fix a test. 2023-06-14 18:10:30 +00:00
Azat Khuzhin
e9c9db9335 Disable parallelize_output_from_storages for all system tables 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-06-14 19:11:23 +03:00
Azat Khuzhin
3c4f638bd7 Disable parallelize_output_from_storages for storages with only one block 
$ clickhouse-benchmark -i 10000 -d 0 --parallelize_output_from_storages=0 -q "select * from values('foo')"
    Loaded 1 queries.

    Queries executed: 10000.

    localhost:9000, queries 10000, QPS: 2800.490, RPS: 2800.490, MiB/s: 0.032, result RPS: 2800.490, result MiB/s: 0.032.

    0.000%          0.000 sec.
    10.000%         0.000 sec.
    20.000%         0.000 sec.
    30.000%         0.000 sec.
    40.000%         0.000 sec.
    50.000%         0.000 sec.
    60.000%         0.000 sec.
    70.000%         0.000 sec.
    80.000%         0.000 sec.
    90.000%         0.000 sec.
    95.000%         0.000 sec.
    99.000%         0.001 sec.
    99.900%         0.001 sec.
    99.990%         0.001 sec.

    $ clickhouse-benchmark -i 10000 -d 0 --parallelize_output_from_storages=1 -q "select * from values('foo')"
    Loaded 1 queries.

    Queries executed: 10000.

    localhost:9000, queries 10000, QPS: 1259.805, RPS: 1259.805, MiB/s: 0.014, result RPS: 1259.805, result MiB/s: 0.014.

    0.000%          0.001 sec.
    10.000%         0.001 sec.
    20.000%         0.001 sec.
    30.000%         0.001 sec.
    40.000%         0.001 sec.
    50.000%         0.001 sec.
    60.000%         0.001 sec.
    70.000%         0.001 sec.
    80.000%         0.001 sec.
    90.000%         0.001 sec.
    95.000%         0.001 sec.
    99.000%         0.001 sec.
    99.900%         0.001 sec.
    99.990%         0.003 sec.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-06-14 19:11:23 +03:00
Azat Khuzhin
3e419730c3 Disable parallelize_output_from_storages for processing MATERIALIZED VIEWs 
Adding more processors for parallelize_output_from_storages is not a
costless operation (I've experienced some issues in production because
of this), and it is not easy to fix in a normal way, so let's disable it
for now.

Before this patch:
- INSERT INTO input SELECT * FROM numbers(10e6) SETTINGS parallelize_output_from_storages=1, min_insert_block_size_rows=1000
  0 rows in set. Elapsed: 3.648 sec. Processed 20.00 million rows, 120.00 MB (5.48 million rows/s., 32.90 MB/s.)

- INSERT INTO input SELECT * FROM numbers(10e6) SETTINGS parallelize_output_from_storages=0, min_insert_block_size_rows=1000
  0 rows in set. Elapsed: 1.851 sec. Processed 20.00 million rows, 120.00 MB (10.80 million rows/s., 64.82 MB/s.)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-06-14 19:11:23 +03:00
Thomas Panetti
04772215cb
Merge branch 'master' into mysql_compatible_types 2023-06-14 08:03:52 -07:00
Dmitry Novik
1d88b16830
Merge pull request #50584 from ClickHouse/analyzer-optimizations-on-shards 
Analyzer: Do not apply Query Tree optimizations on shards
 2023-06-14 14:31:46 +02:00
Sema Checherinda
7d59af1f77 test compressed write to S3 2023-06-14 14:28:57 +02:00
Val Doroshchuk
e7c5991b39 MaterializedMySQL: Keep parenthesises for empty table overrides 
Empty table overrides are formatted without any parenthesises,
but they are required by a parser,
and it is not possible to parse empty table overrides without it.

:) CREATE DATABASE db ... TABLE OVERRIDE t1()

CREATE DATABASE db
...
TABLE OVERRIDE `t1`

This query will be saved to metadata and ClickHouse will not be able to start up, since
table overrides require ().
 2023-06-14 13:37:49 +02:00
Nikolai Kochetov
5163927b7d
Merge pull request #50923 from ClickHouse/read-in-order-proj-bug 
Do not apply projection if read-in-order was enabled.
 2023-06-14 13:13:25 +02:00
Nikita Taranov
a00e42f75b
Fix erroneous sort_description propagation in CreatingSets (#50955) 2023-06-14 13:00:47 +02:00
kssenii
827ac17dc2 Fix 2023-06-14 12:59:06 +02:00
kssenii
47a6db596e Better 2023-06-14 11:37:57 +02:00
kssenii
2a3ef3941e Fix 2023-06-14 11:31:28 +02:00
Kseniia Sumarokova
e2d8299b23
Merge pull request #50952 from nickitat/fix_remote_read_perf_degr 
Fix logic in `AsynchronousBoundedReadBuffer::seek`
 2023-06-14 11:25:26 +02:00
Nikolai Kochetov
1663905acd Hold context in pipeline resources. 2023-06-14 08:48:16 +00:00
Robert Schulze
2643fd2c25
Merge pull request #50689 from arenadata/ADQM-871 
Added connection string to clickhouse-client
 2023-06-14 10:39:32 +02:00
Robert Schulze
edd29492c3
Merge pull request #50893 from rschu1ze/snowflake-crash 
Fix LOGICAL_ERROR in snowflakeToDateTime*()
 2023-06-14 09:42:21 +02:00
Chang Chen
86694847c6 using Reader instead of typename CapnpType::Reader 2023-06-14 15:22:32 +08:00
Manas Alekar
0a1d0c4abd Fix one stateless test. 2023-06-13 23:37:01 -07:00
alekar
dc3c4e6528
Merge branch 'master' into improve-outfile 2023-06-13 21:34:32 -07:00
Chang Chen
e281026e00 fix build issue on clang 15 2023-06-14 12:29:55 +08:00
Alexey Gerasimchuck
868c3bd45d minor change 2023-06-14 04:29:08 +00:00
Thomas Panetti
a502967e1a
Merge branch 'master' into mysql_compatible_types 2023-06-13 20:30:16 -07:00
kevinyhzou
f3b99156ac review fix 2023-06-14 10:48:21 +08:00
pufit
91d794cf0a
Merge pull request #50150 from JackyWoo/support_redis 
Add Redis table Engine/Function
 2023-06-13 21:34:14 -04:00
Alexey Gerasimchuck
f1b5d47ce2 corrections after second review iteration 2023-06-14 01:26:39 +00:00
Nikita Taranov
1bc5598aa7 impl 2023-06-13 20:02:50 +02:00
alekar
3f9ded7b13
Merge branch 'master' into improve-outfile 2023-06-13 10:09:11 -07:00
Victor Krasnov
f01b96f9f9
Merge branch 'master' into ADQM-812 2023-06-13 19:52:10 +03:00
Robert Schulze
76f69f2b44
Revert overengineering 2023-06-13 15:52:06 +00:00
Kruglov Pavel
39ba925f8b
Merge branch 'master' into allow-skip-empty-files 2023-06-13 17:17:26 +02:00
Thomas Panetti
fa4bc6b7c5
Merge branch 'master' into mysql_compatible_types 2023-06-13 08:13:15 -07:00
Dmitry Novik
23c478f8b5
Merge branch 'master' into parallel_replicas_analyzer 2023-06-13 17:09:18 +02:00
Smita Kulkarni
332893344d Updated lock for accessing azure blob storage iterator 2023-06-13 16:50:10 +02:00
Kruglov Pavel
607f337d67
Merge pull request #50592 from Avogar/max-bytes-to-read-in-schema-inference 
Add setting to limit the number of bytes to read in schema inference
 2023-06-13 16:47:57 +02:00
Kruglov Pavel
2bfb15cf81
Merge pull request #50620 from Avogar/increase-bitmap-max-array-sixe 
Increase max array size in group bitmap
 2023-06-13 16:47:32 +02:00
avogar
2e1f56ae33 Address comments 2023-06-13 14:43:50 +00:00
Nikita Mikhaylov
52a460df67
Tests with parallel replicas are no more "always green" (#50896) 2023-06-13 16:43:35 +02:00
Kruglov Pavel
8fdcd91c38
Merge pull request #49752 from Avogar/better-capnproto-3 
Refactor CapnProto format to improve input/output performance
 2023-06-13 16:20:38 +02:00
Anton Popov
79f3300709
Merge pull request #50726 from CurtizJ/enable-mutations-throttling 
Enable settings for mutations throttling by default
 2023-06-13 14:57:36 +02:00
Dmitry Novik
f4ed10c0a2
Update src/Storages/StorageReplicatedMergeTree.cpp 2023-06-13 14:44:39 +02:00
Nikolai Kochetov
eddd932636 Do not apply projection if read-in-order was enabled. 2023-06-13 12:34:26 +00:00
Robert Schulze
3e3b8ff5f6
More robustness 2023-06-13 12:24:31 +00:00
Kruglov Pavel
cbed327077
Merge pull request #50635 from Avogar/skip-trailing-empty-lines 
Allow to skip trailing empty lines in CSV/TSV/CustomSeparated formats
 2023-06-13 12:43:43 +02:00
Julian Maicher
6201947b45
Merge branch 'master' into fix/50864/verification-cooldown-cache-entry 2023-06-13 11:51:53 +02:00
Kseniia Sumarokova
0ab3dc9261
A bit safer UserDefinedSQLFunctionVisitor (#50913) 
* Update UserDefinedSQLFunctionVisitor.cpp

* Update UserDefinedSQLFunctionVisitor.cpp

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-06-13 11:25:13 +02:00
Michael Kolupaev
72f2832129
Slightly more information in error message about cached disk (#50897) 2023-06-13 11:07:05 +02:00
Robert Schulze
79bc884733
Stabilize tests 2023-06-13 08:56:22 +00:00
Sema Checherinda
2c018f5261 do not call finalize after exception 2023-06-13 10:48:31 +02:00
Manas Alekar
bf69755ada Address some usability issues with INTO OUTFILE usage. 2023-06-13 01:26:11 -07:00
Kseniia Sumarokova
9a3cea379f
Merge pull request #50723 from kssenii/fix-data-race-in-read-buffer 
Fix data race in log message of cached buffer
 2023-06-13 10:25:18 +02:00
Robert Schulze
d8d352b2c5
Merge pull request #50912 from rschu1ze/annoy-docs 
Update Annoy docs
 2023-06-13 10:03:50 +02:00
Robert Schulze
8358d29ac7
Merge pull request #50405 from ClibMouse/feature/reservoir-sampler-big-endian-support 
Implement big-endian support for the deterministic reservoir sampler
 2023-06-13 09:55:23 +02:00
Alexey Gerasimchuck
e3a13111ae
Merge branch 'master' into ADQM-871 2023-06-13 14:05:13 +10:00
Victor Krasnov
b3ef2a4860 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-812 2023-06-13 03:55:38 +00:00
Alexey Gerasimchuck
2395b25f9e Changes after review 2023-06-13 01:55:34 +00:00
JackyWoo
9d548315e8
Merge branch 'master' into support_redis 2023-06-13 09:34:32 +08:00
JackyWoo
959fde4491 add notifications in docs 2023-06-13 09:33:38 +08:00
Vitaly Baranov
23cff1fc32
Merge pull request #50889 from vitlibar/fix-checking-lock-file-too-often-while-writing-backup 
Fix checking the lock file too often while writing a backup
 2023-06-13 02:42:51 +02:00
ltrk2
002c15823c Perform in-place endianness transform because of padding 2023-06-12 16:44:46 -07:00
Alexey Milovidov
4e7cd2da01
Merge pull request #50811 from ClickHouse/tavplubix-patch-6 
Don't mark a part as broken on `Poco::TimeoutException`
 2023-06-12 23:17:37 +03:00
Alexey Milovidov
5cdf893f3a
Merge pull request #50835 from ClickHouse/rename-async-metrics 
Replace CPU CGroups metrics to one
 2023-06-12 23:15:05 +03:00
Robert Schulze
4f39ee51ae
Update Annoy docs 2023-06-12 20:06:57 +00:00
Robert Schulze
7745f7da73
Merge branch 'master' into annoy-misc 2023-06-12 21:46:27 +02:00
Nikolai Kochetov
7025232175 Fixing cache for sets. 2023-06-12 18:59:07 +00:00
ltrk2
edb4a644b1 Update FunctionsCodingIP.cpp 2023-06-12 14:23:22 -04:00
ltrk2
a4285d56b2 Fix compilation error on big-endian platforms 2023-06-12 14:23:22 -04:00
Kruglov Pavel
e4838725e3
Merge branch 'master' into allow-skip-empty-files 2023-06-12 20:03:23 +02:00
Dmitry Novik
d05f89f8f5 Fix style 2023-06-12 17:33:15 +00:00
Dmitry Novik
326a3a3e8d Use query tree to rewrite the query 2023-06-12 16:51:40 +00:00
Dmitry Novik
d606e2190c
Merge branch 'master' into parallel_replicas_analyzer 2023-06-12 18:49:54 +02:00
Robert Schulze
65d83e45cb
Fix crash in snowflakeToDateTime(), follow-up to #50834 2023-06-12 16:21:49 +00:00
Vitaly Baranov
5aa0566767 Fix checking the lock file too often while writing a backup. 2023-06-12 18:13:26 +02:00
Thomas Panetti
8683f5e48d
Merge branch 'master' into mysql_compatible_types 2023-06-12 08:19:59 -07:00
Robert Schulze
128e8c20d5
Merge pull request #50709 from arenadata/ADQM-867 
Added numeric arguments support to some Date/DateTime conversion functions
 2023-06-12 17:08:14 +02:00
Robert Schulze
3617316026
Merge branch 'master' into non-experimental-qpl-deflate 2023-06-12 16:01:45 +02:00
Dmitry Novik
26c9bda144 Add a comment 2023-06-12 13:54:45 +00:00
Anton Popov
d45f07743c fix getting number of mutations 2023-06-12 13:54:07 +00:00
Alexander Tokmakov
01c7d2fe71
Prostpone check of outdated parts (#50676) 
* prostpone check of outdated parts

* Update ReplicatedMergeTreePartCheckThread.cpp
 2023-06-12 16:53:26 +03:00
Alexander Tokmakov
5db3b393d8
Update MergeTreeData.cpp 2023-06-12 16:22:33 +03:00
Han Fei
d47cdd4eb6
Merge pull request #50605 from ClickHouse/revert-50467-revert-50430-hanfei/fix-crossjoin-filter-pushdown 
Revert "Revert "make filter push down through cross join"" and supress a test
 2023-06-12 15:12:58 +02:00
Anton Popov
32caf87163
Merge pull request #50104 from amosbird/fix_43107_47549 
Proper mutation of skip indices and projections
 2023-06-12 15:09:18 +02:00
Kruglov Pavel
873cee9451
Merge pull request #49626 from alekseygolub/renamefile 
Added option to rename files, loaded via TableFunctionFile, after successful processing
 2023-06-12 15:01:22 +02:00
Kruglov Pavel
edd47a2281
Merge branch 'master' into skip-trailing-empty-lines 2023-06-12 13:57:15 +02:00
Kruglov Pavel
e03cd725b0
Merge pull request #50602 from Avogar/null-as-default-schema-inference 
Respect setting input_format_as_default in schema inference
 2023-06-12 13:45:52 +02:00
Kruglov Pavel
da68980b8d
Merge branch 'master' into max-bytes-to-read-in-schema-inference 2023-06-12 13:45:31 +02:00
Kruglov Pavel
24d70a2afd
Fix 2023-06-12 13:37:59 +02:00
avogar
5cec4c3161 Fallback to parsing big integer from String instead of exception in Parquet format 2023-06-12 11:34:40 +00:00
Robert Schulze
6da002e250
Merge pull request #50834 from rschu1ze/non-const-tz 
Add compat setting for non-const timezones
 2023-06-12 13:33:55 +02:00
Nikolai Kochetov
eb16745033 Collect sets from indexHint actions as well. 2023-06-12 11:04:07 +00:00
Alexander Tokmakov
676ba2fbde
Update MergeTreeData.cpp 2023-06-12 12:30:38 +03:00
Julian Maicher
c378c3fcbb Fix type of LDAP server params hash in cache entry 
In 1ed7ad57d9, we switched from (`size_t`, usually 64bit) to SipHash (128bit) and forgot to change the type of the cache entry. This broke the caching of successful LDAP authentication requests (verification cooldown).

Fixes #50864
 2023-06-12 10:58:35 +02:00
Kseniia Sumarokova
ee829061f4
Merge branch 'master' into fix-data-race-in-read-buffer 2023-06-12 10:32:54 +02:00
Antonio Andelic
220378cff9
Merge pull request #50751 from ClickHouse/keeper-reject-unknown-operation 
Avoid storing logs in Keeper containing unknown operation
 2023-06-12 09:08:57 +02:00
kevinyhzou
911f8ad8dc use whitespace or tab as field delimiter 2023-06-12 11:57:52 +08:00
JackyWoo
ef40c029a5 fix style 2023-06-12 11:54:42 +08:00
JackyWoo
7bcaf8b233 fix build error 2023-06-12 10:15:32 +08:00
JackyWoo
a1641aa25d
Merge branch 'master' into support_redis 2023-06-12 09:53:06 +08:00
Alexander Tokmakov
3797a4202c
Update MergeTreeData.cpp 2023-06-11 23:38:31 +03:00
Dmitry Kardymon
9fd3f197c6
Merge branch 'master' into ADQM-867 2023-06-11 22:06:37 +03:00
Alexey Milovidov
598501011f Relax "too many parts" further 2023-06-11 17:51:54 +02:00
alekseygolub
d72751be27 Added cache invalidation; Fix issues 2023-06-11 15:01:45 +00:00
Robert Schulze
8222861f2e
Merge remote-tracking branch 'rschu1ze/master' into non-experimental-qpl-deflate 2023-06-11 14:39:30 +00:00
Robert Schulze
2146c35608
Fix style 2023-06-11 07:40:39 +00:00
Nikolay Degterinsky
dc9001a969
Merge pull request #50061 from evillique/update-mongo 
Update MongoDB protocol & fix inserts
 2023-06-10 22:52:15 +02:00
Antonio Andelic
5b4078f0c1 Merge branch 'master' into keeper-reject-unknown-operation 2023-06-10 17:19:17 +00:00
Robert Schulze
72b9d75a84
Add compat setting for non-const timezones 
SQL function toTimezone() converts a Date or DateTime into another
timezone. The problem is that the timezone is part of the Date /
DateTime type but not part of the internal representation (value). This
led to the fact that toTimeZone() wqith non-const timezones produced
wrong and misleading results until #48471 (shipped with v23.4) enforced
a const timezone.

Unfortunately, this PR also broke existing table definitions with
non-const timezones, e.g. in ALIAS expressions. So while #48471
addressed the issue appropriately, it is really backwards-incompatible.

This PR adds a setting to toggle the behavior and makes it also part of
the compatibility profile.
 2023-06-10 16:56:42 +00:00
alesapin
572c9dec4e
Merge pull request #50812 from ClickHouse/Rename_azure_blob_table_function 
Rename azure_blob_storage to azureBlobStorage
 2023-06-10 18:10:59 +02:00
Alexey Milovidov
ed9c7c9841
Merge pull request #50711 from kssenii/get-rid-of-uneeded-code 
Refactor reading from object storages
 2023-06-10 19:10:25 +03:00
Alexey Milovidov
ddd2257cf5 Normalize with respect to CGroups 2023-06-10 18:03:17 +02:00
Alexey Milovidov
a91b84e60c Slightly better 2023-06-10 18:00:47 +02:00
Alexey Milovidov
e785c6796d Replace CGroups CPU metrics to one 2023-06-10 17:54:46 +02:00
Alexey Milovidov
7cb6f3c722 Rename Cpu to CPU and Cfs to CFS 2023-06-10 17:43:42 +02:00
tpanetti
cb8c20722b Rename setting and description for MySQL compatible types 
This renames the setting for MySQL compatible types from output_format_mysql_types to use_mysql_types_in_show_columns
 2023-06-10 08:38:10 -07:00
Alexey Milovidov
3c023febc4
Merge pull request #50698 from kitaisreal/reverse-transform-small-improvement 
ReverseTransform small improvement
 2023-06-10 14:59:23 +03:00
Nikolay Degterinsky
9ad8e022a8
Merge branch 'master' into update-mongo 2023-06-10 10:58:02 +02:00
Hongbin Ma
41c34aaf5e optimize parquet write performance for parallel threads 
fix CI

fix review comments and CI
 2023-06-09 19:09:58 -07:00
Thomas Panetti
5769d7feca
Merge branch 'master' into mysql_compatible_types 2023-06-09 13:52:16 -07:00
tpanetti
96d7b2efc9 Disable fasttest for MySQL Compatibility Type Conversion and refactor style for DataTypeNumberBase 2023-06-09 13:50:30 -07:00
Nikolay Degterinsky
c71edb6c79 Fix style 2023-06-09 20:50:24 +00:00