thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-03 21:12:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
115,701 Commits 9,188 Branches 1,782 Tags 2.2 GiB
780e9df08d
Commit Graph

41840 Commits

Author SHA1 Message Date
Kseniia Sumarokova
1c04085e8f
Update MergeTreeWriteAheadLog.h 2023-05-17 18:15:51 +02:00
kssenii
f2dbcb5146 Better fix 2023-05-17 16:27:06 +02:00
Han Fei
ed1d036151
Merge pull request #49884 from azat/dist-fix-async-block-processing 
Fix processing pending batch for Distributed async INSERT after restart
 2023-05-17 15:19:42 +02:00
avogar
7443dc925c Fix possible Logical error on bad Nullable parsing for text formats 2023-05-17 13:12:00 +00:00
AVMusorin
7df4820af7
Fix metrics WriteBufferFromS3Bytes, WriteBufferFromS3Microseconds and WriteBufferFromS3RequestsErrors 
Ref: https://github.com/ClickHouse/ClickHouse/pull/45188
 2023-05-17 14:50:38 +02:00
Alexander Tokmakov
36c31e1d79
Improve concurrent parts removal with zero copy replication (#49630) 
* improve concurrent parts removal

* fix

* fix
 2023-05-17 14:07:34 +03:00
Alexander Tokmakov
1e529263d0
Merge branch 'master' into Follow_up_Backup_Restore_concurrency_check_node_2 2023-05-17 13:57:50 +03:00
Vitaly Baranov
6c8a923c9d
Merge branch 'master' into write-encrypted-to-backup 2023-05-17 12:37:05 +02:00
Kseniia Sumarokova
edceda494d
Merge branch 'master' into add-more-logging-for-cache 2023-05-17 12:24:59 +02:00
Kseniia Sumarokova
3787b7f127
Update Metadata.cpp 2023-05-17 12:16:18 +02:00
Azat Khuzhin
fdfb1eda55 Fix {Local,Remote}ReadThrottlerSleepMicroseconds metric values 
And also update the test, since now you could have slightly less sleep
intervals, if query spend some time in other places.

But what is important is that query_duration_ms does not exceeded
calculated delay.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-17 12:12:39 +02:00
Azat Khuzhin
7383da0c52 Fix per-query remote throttler 
remote throttler by some reason had been overwritten by the global one
during reloads, likely this is for graceful reload of this option, but
it breaks per-query throttling, remove this logic.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-17 12:12:39 +02:00
Azat Khuzhin
3c80e30f02 Fix per-query IO/BACKUPs throttling settings (when default profile has them) 
When some of this settings was set for default profile (in
users.xml/users.yml), then it will be always used regardless of what
user passed.

Fix this by not inherit per-query throttlers, for this they should be
reset before making query context and they should not be initialized as
before in Context::makeQueryContext(), since makeQueryContext() called
too early, when user settings was not read yet.

But there we had also initialization of per-server throttling, move this
into the ContextSharedPart::configureServerWideThrottling(), and call it
once we have ServerSettings set.

Also note, that this patch makes the following settings - server
settings:
- max_replicated_fetches_network_bandwidth_for_server
- max_replicated_sends_network_bandwidth_for_server
But this change should not affect anybody, since it is done with
compatiblity (i.e. if this setting is set in users profile it will be
read from it as well as a fallback).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-17 12:12:39 +02:00
Igor Nikonov
7d647c50c7
Merge branch 'master' into clearable_hash_set_without_zero_storage 2023-05-17 11:29:01 +02:00
FFFFFFFHHHHHHH
fd1e6557e1
Merge branch 'master' into dot_product 2023-05-17 14:40:06 +08:00
fhbai
c104354894 fix 2023-05-17 14:39:30 +08:00
Alexey Gerasimchuck
29b10ae336 reordered options 2023-05-17 04:06:01 +00:00
Alexey Gerasimchuck
4a6c7254e8 --multiquery <sql> -> -n -q <sql> syntax sugar 2023-05-17 03:43:35 +00:00
Vitaly Baranov
f4ac4c3f9d Corrections after review. 2023-05-17 03:23:16 +02:00
Yakov Olkhovskiy
0a44a69dc8 remove unnecessary header 2023-05-17 00:22:13 +00:00
Yakov Olkhovskiy
282297b677 binary encoding of IPv6 in protobuf 2023-05-16 23:46:01 +00:00
Han Fei
3ead9e627e Merge branch 'master' into hanfei/fix-optimize-regexp-prefix 2023-05-16 22:31:01 +02:00
serxa
abacf1f990 add missing quota_key in operator== for connections 2023-05-16 19:14:54 +00:00
serxa
b12eefc694 fix timeout units and log message 2023-05-16 18:57:04 +00:00
Alexander Tokmakov
242a3fc520 Merge branch 'master' into less_zookeeper_requests 2023-05-16 18:24:11 +02:00
Alexander Tokmakov
0da82945ac fix 2023-05-16 18:18:48 +02:00
Alexander Tokmakov
3d26232cc0
Merge pull request #49918 from ClickHouse/remove_unused_code 
Remove unused code
 2023-05-16 18:53:49 +03:00
kssenii
724949927b Add logging 2023-05-16 17:36:48 +02:00
Antonio Andelic
4bc5a76fa7
Add Compose request for GCS (#49693) 
* Add compose request

* Check if outcome is successful

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-16 17:20:06 +02:00
Dmitry Novik
2287dd8633
Merge pull request #49800 from ClickHouse/fix-adding-cast 
Analyzer: apply _CAST to constants only once
 2023-05-16 17:05:02 +02:00
Igor Nikonov
dea5cbcf4e
Slightly update comment 2023-05-16 16:39:00 +02:00
vdimir
1f55c320b4 Fix style 2023-05-16 16:23:53 +02:00
vdimir
ca005ecea1 Update comment about filtering nulls in asof join 2023-05-16 16:23:53 +02:00
vdimir
a7bb8f412f Allow ASOF JOIN over nullable right column 2023-05-16 16:23:53 +02:00
Kruglov Pavel
4530f38fdf
Merge branch 'master' into urlCluster 2023-05-16 16:21:23 +02:00
alesapin
50a536bba8 Remove unused code 2023-05-16 15:26:24 +02:00
Han Fei
ea59761809 fix OptimizeRegularExpression 2023-05-16 15:25:04 +02:00
Azat Khuzhin
68138395eb Fix parameterized views when query parameter used multiple times in the query 
Example:

    CREATE VIEW view AS
    SELECT *
    FROM system.one
    WHERE dummy = {k1:Int}+1 OR dummy = {k1:Int}+2
                   ^^                    ^^

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-16 15:13:21 +02:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
Vitaly Baranov
b068f0b619 Fix build. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
2ec94a42b7 Remove default parameters from virtual functions. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
943707963f Add backup setting "decrypt_files_from_encrypted_disks" 2023-05-16 14:27:27 +02:00
Vitaly Baranov
019493efa3 Fix throttling in backups. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
5198997fd8 Remove ReadSettings from backup entries. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
7cea264230 Fix whitespaces. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
c48c20fac8 Use combined checksums for encrypted immutable files. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
517e119e03 Move checksum calculation to IBackupEntry. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
002fd19cb7 Move the common part of BackupIO_* to BackupIO_Default. 2023-05-16 14:27:23 +02:00
Vitaly Baranov
c92219f01b BACKUP now writes encrypted data for tables on encrypted disks. 2023-05-16 14:26:33 +02:00
Vitaly Baranov
cc50fcc60a Remove the 'temporary_file_' argument from BackupEntryFromImmutableFile's constructor. 2023-05-16 14:25:37 +02:00
Vitaly Baranov
bc880db5d9 Add functions to read/write encrypted files from IDisk. 2023-05-16 14:25:37 +02:00
Vitaly Baranov
101aa6eff0 Add function copyS3FileFromDisk(). 2023-05-16 14:25:37 +02:00
Vitaly Baranov
69114cb550 Add function getBlobPath() to IDisk interface to allow copying to/from disks which are not built on top of IObjectStorage. 2023-05-16 14:25:36 +02:00
Vitaly Baranov
fd2731845c Simplify interface of IBackupWriter: Remove supportNativeCopy() function. 2023-05-16 14:25:36 +02:00
Smita Kulkarni
9a2645a729 Fixed clang build 2023-05-16 14:09:38 +02:00
kssenii
d4ea3ea045 Fix 2023-05-16 13:54:13 +02:00
Kruglov Pavel
9dbe9507e7
Fix style 2023-05-16 12:55:20 +02:00
Sergei Trifonov
1f9135e4ab
Merge branch 'master' into async-loader-workloads 2023-05-16 12:50:09 +02:00
alesapin
93bd09ddd6
Merge branch 'master' into fix_another_zero_copy_bug 2023-05-16 12:24:52 +02:00
avogar
a3dfa40eab Fix 2023-05-16 10:07:21 +00:00
Kruglov Pavel
b414760d43
Merge pull request #49673 from Avogar/fiber-local-var 
Fix assert in SpanHolder::finish() with fibers
 2023-05-16 11:59:33 +02:00
alesapin
0b4ab70dd9
Merge pull request #49891 from hanfei1991/hanfei/chassert-1 
use chassert in MergeTreeDeduplicationLog to have better log info
 2023-05-16 11:50:11 +02:00
Sema Checherinda
03c51208d1
Merge pull request #44869 from CheSema/multi_part_upload 
rework WriteBufferFromS3, add tests, add abortion
 2023-05-16 10:52:01 +02:00
serxa
e1509ca59d fix style 2023-05-16 08:23:06 +00:00
serxa
f6d7145a72 fix tests 2023-05-15 23:05:13 +00:00
Robert Schulze
59bc3e25be
Merge pull request #49824 from AVMusorin/allow-alias-column-kafka 
KafkaEngine: Allow usage of Alias column type
 2023-05-15 23:40:03 +02:00
FFFFFFFHHHHHHH
11b94a626a
Fix aggregate function kolmogorovSmirnovTest (#49768) 2023-05-15 23:20:29 +02:00
Sergei Trifonov
cbc15bf35a
Add DynamicResourceManager and FairPolicy into scheduling subsystem (#49671) 
* Add `DynamicResourceManager` and `FairPolicy` into scheduling subsystem

* fix test

* fix tidy build
 2023-05-15 23:13:17 +02:00
Alexander Tokmakov
c9d6ee3c98
Merge pull request #49874 from azat/build/fix 
Fix "reference to local binding" after fixes for clang-17
 2023-05-15 23:25:18 +03:00
Vitaly Baranov
801cacc13f
Merge pull request #49831 from vitlibar/fix-setting-null-in-profile-def 
Fix setting NULL in profile definition
 2023-05-15 22:24:49 +02:00
Sergei Trifonov
567a099097
Merge branch 'master' into async-loader-workloads 2023-05-15 21:07:53 +02:00
serxa
e4a653b41c Multiple pools support for AsyncLoader 2023-05-15 18:49:34 +00:00
Vitaly Baranov
bf3336a84e
Merge pull request #47640 from ilejn/row_policy_template 
Row policy for database
 2023-05-15 20:05:15 +02:00
Alexander Tokmakov
65bc702b0b fix 2023-05-15 20:02:30 +02:00
Michael Kolupaev
91db148513 Fix AsynchronousReadIndirectBufferFromRemoteFS breaking on short seeks 2023-05-15 11:02:24 -07:00
Igor Nikonov
0d585746de Fixes 
+ disallow usage of the same column in sorting prefix and INTERPOLATE
+ sorting prefix is empty if use_with_fill_by_sorting_prefix is off
 2023-05-15 17:08:00 +00:00
Han Fei
4137a5e058 use chassert in MergeTreeDeduplicationLog to have better log info 2023-05-15 18:51:16 +02:00
avogar
a551d1f042 Fix build 2023-05-15 16:45:45 +00:00
avogar
334f062fa0 fix style 2023-05-15 16:39:26 +00:00
avogar
f9e0eb47d7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-15 16:35:24 +00:00
Kruglov Pavel
900aca5f0a
Delete unneded files 2023-05-15 18:33:09 +02:00
Kruglov Pavel
bfcaf95aed
Delete unneded files 2023-05-15 18:32:54 +02:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Kruglov Pavel
550d7a01bc
Try to fix build 2023-05-15 17:55:02 +02:00
avogar
78064d0622 Better comments 2023-05-15 15:52:14 +00:00
avogar
b23afdc533 Fix build for aarch64-darwin 2023-05-15 15:48:00 +00:00
Igor Nikonov
97e1513b22
Merge branch 'master' into clearable_hash_set_without_zero_storage 2023-05-15 17:42:10 +02:00
Igor Nikonov
4d168400ae Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-15 14:40:46 +00:00
vdimir
07de815d96
Merge pull request #49836 from arthurpassos/add_extract_kv_max_number_of_pairs_safeguard 2023-05-15 16:31:01 +02:00
Anton Popov
512b27ef27
Merge pull request #49873 from amosbird/fix_49839 
Fix a bug with projections and the aggregate_functions_null_for_empty setting (for query_plan_optimize_projection)
 2023-05-15 15:58:42 +02:00
Azat Khuzhin
f2a023140e Fix processing pending batch for Distributed async INSERT after restart 
After abnormal server restart current_batch.txt (that contains list of
files to send to the remote shard), may not have all files, if it was
terminated between unlink .bin files and truncation of current_batch.txt

But it should be fixed in a more reliable way, though to backport the
patch I kept it simple.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-15 15:57:30 +02:00
AVMusorin
418a61a68c
Allow using Alias column type for KafkaEngine 
```
create table kafka
(
 a UInt32,
 a_str String Alias toString(a)
) engine = Kafka;

create table data
(
  a UInt32;
  a_str String
) engine = MergeTree
order by tuple();

create materialized view data_mv to data
(
  a UInt32,
  a_str String
) as
select a, a_str from kafka;
```
Alias type works as expected in comparison with MATERIALIZED/EPHEMERAL
or column with default expression.

Ref: https://github.com/ClickHouse/ClickHouse/pull/47138

Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
 2023-05-15 15:39:58 +02:00
Sema Checherinda
dccdb3e678 work with comments on PR 2023-05-15 14:41:51 +02:00
Arthur Passos
e8f971aa2b use LIMIT_EXCEEDED instead of TOO_LARGE_MAP_SIZE 2023-05-15 09:25:10 -03:00
Arthur Passos
b06e34a77f Accept key value delimiter as part of value 2023-05-15 13:52:47 +02:00
Azat Khuzhin
665545ec45 Fix "reference to local binding" after fixes for clang-17 
Follow-up for: #49851 (cc @alexey-milovidov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-15 12:45:20 +02:00
Alexander Tokmakov
25912a2673
Merge pull request #49876 from JackyWoo/fix_typo 
fix typo
 2023-05-15 13:32:58 +03:00
Kruglov Pavel
558eda4146
Merge pull request #49412 from azat/block-use-dense-hash-map 
Switch Block::NameMap to google::dense_hash_map over HashMap
 2023-05-15 12:22:55 +02:00
JackyWoo
8d1bcb5c2f fix typo 2023-05-15 16:51:20 +08:00
Amos Bird
4764259f60
Fix a bug with projections and the aggregate_functions_null_for_empty 
setting (for query_plan_optimize_projection)

Fix a bug with projections and the aggregate_functions_null_for_empty
setting. This was already fixed in PR #42198 but got forgotten after
using query_plan_optimize_projection.
 2023-05-15 14:17:16 +08:00
Alexey Milovidov
1db35384d9 Support bitCount for big integers 2023-05-15 03:30:03 +02:00
alekar
528e68bfc4
Merge branch 'master' into fix-osx-setsockopt-errors 2023-05-14 15:35:55 -07:00
Igor Nikonov
b2dddf7cf8 Fix style 2023-05-14 16:06:35 +00:00
Igor Nikonov
e743693fb3 Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-14 15:56:12 +00:00
Igor Nikonov
93384d6546 Fix suffix generation (see added test in commit 3355894) 2023-05-14 15:52:49 +00:00
Sergei Trifonov
8f20085d9a
Merge pull request #48923 from ClickHouse/async-loader 
Add AsyncLoader with dependency tracking and runtime prioritization
 2023-05-14 15:12:39 +02:00
robot-clickhouse
33ca77b4ca
Merge pull request #49843 from azat/joinGet-non-deterministic 
[RFC] Mark joinGet() as non deterministic (so as dictGet)
 2023-05-14 11:12:12 +02:00
alekar
2631d3db20
Merge branch 'master' into fix-osx-setsockopt-errors 2023-05-13 23:03:17 -07:00
Manas Alekar
c87b33a24d Fix error on OS X regarding resetting timeouts. 
This happens when remote disconnects due to inactivity. It seems
to work on Linux, likely due to difference in SO_LINGER, maybe a
different default timeout on Darwin.

Verified manually using clickhouse cloud using following process:

1. Connect to instance.
2. Run `show tables`.
3. Wait 6 minutes.
4. Run `show tables`.

With this fix, the EINVAL is not reported, and client will simply
reconnect.
 2023-05-13 22:55:27 -07:00
Alexey Milovidov
4f7bcf01f6
Merge pull request #49858 from ucasfl/bit-hamming 
bitHammingDistance support String and FixedString data type
 2023-05-14 08:28:01 +03:00
Alexey Milovidov
b4b5b4dcc5 Amend 2023-05-14 02:52:23 +02:00
Alexey Milovidov
0ca36d4f89 Merge branch 'master' into clang-17 2023-05-14 01:57:40 +02:00
Ilya Golshtein
ccd82842a0 row_policy_template - rollback tweaking found policy for database 2023-05-13 22:30:02 +00:00
Ilya Golshtein
9ef610040f row_policy_template - table * is respected 2023-05-13 22:30:02 +00:00
Ilya Golshtein
6852ae0d93 row_policy_template - external code review changes 2023-05-13 22:30:02 +00:00
Ilya Golshtein
7fa6ea4ccc row policy template: tests and code cleanup, code review changes 2023-05-13 22:30:02 +00:00
Ilya Golshtein
f0d21a9100 row policy template: initial, works, restrictive rules fix 2023-05-13 22:30:02 +00:00
Igor Nikonov
629a1a6668 Minor cleanup 2023-05-13 22:11:23 +00:00
flynn
2f88605c3d remove space 
format
 2023-05-13 14:03:21 +00:00
flynn
2ffd00df8a bitHammingDistance support String and FixedString data type 2023-05-13 13:56:36 +00:00
Azat Khuzhin
a96067987e Mark joinGet() as non deterministic (so as dictGet) 
joinGet() should not be considered as deterministic function, since
shards could have different data in tables.

Also since now there is allow_nondeterministic_mutations, it could be
used as a workaround for this backward incompatible change.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-13 08:12:53 +02:00
Alexey Milovidov
a2d1cc1333
Merge pull request #49844 from MikhailBurdukov/in_with_tuple 
TYPE_MISMATCH exception for in operator with  single column tuples.
 2023-05-13 04:21:12 +03:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Alexey Milovidov
f6144ee32b
Revert "Make Pretty formats even prettier." 2023-05-13 02:45:07 +03:00
Alexey Milovidov
c306deb1ae
Merge pull request #49087 from Eridanus117/suffix_size 
improvement(parser): size suffixes in quota creation parameters
 2023-05-13 01:56:10 +03:00
Alexey Milovidov
146e3c391a
Merge pull request #49785 from evillique/local-query-parameters 
Add query parameters to clickhouse-local
 2023-05-13 01:07:53 +03:00
Alexey Milovidov
12be14b195
Merge pull request #49819 from azat/fix-clock 
Use CLOCK_MONOTONIC_RAW over CLOCK_MONOTONIC on Linux (fixes non monotonic clock)
 2023-05-13 01:06:27 +03:00
Alexey Milovidov
efc5e69aaf
Merge pull request #49833 from HarryLeeIBM/hlee-s390x-simple-agg 
Fix DefaultHash64 for s390x
 2023-05-13 01:00:36 +03:00
Anton Popov
401030e2be
Merge pull request #49757 from CurtizJ/add-forgotten-lock 
Add forgotten lock (addition to #49117)
 2023-05-12 23:55:09 +02:00
Sergei Trifonov
a30a0d80f2
Merge branch 'master' into async-loader 2023-05-12 20:17:20 +02:00
Igor Nikonov
51c69b6fbd Comments regarding zero values in clearable hash table 2023-05-12 17:36:35 +00:00
MikhailBurdukov
8c7b63f1e7 Fix 2023-05-12 16:42:01 +00:00
Igor Nikonov
d3c408aaf1 ClearableHashSet w/o zero storage for StringRef 2023-05-12 16:32:23 +00:00
serxa
195eafcaed fix disable test 2023-05-12 14:51:33 +00:00
Dmitry Novik
d2e185d86d Analyzer: do not optimize GROUP BY keys with ROLLUP and CUBE 2023-05-12 14:36:54 +00:00
Arthur Passos
b1549a19a5 Use 0 as unlimited 2023-05-12 11:19:35 -03:00
Dmitry Novik
5fdaef851c Handle constants carefully 2023-05-12 14:14:40 +00:00
avogar
93dd3a8ec9 Remove not used cpp file 2023-05-12 14:03:37 +00:00
avogar
3d00f4cdf6 Remove unneded files 2023-05-12 14:03:04 +00:00
avogar
70a8fd2c50 Fix schema inference with named collection, refactor Cluster table functions 2023-05-12 13:58:45 +00:00
Arthur Passos
1e3b7af97a Add setting to limit the max number of pairs produced by extractKeyValuePairs 2023-05-12 10:26:05 -03:00
HarryLeeIBM
809fdc7b16 Fix DefaultHash64 for s390x 2023-05-12 06:00:47 -07:00
serxa
5807737804 temporarily disable JobFailure test 2023-05-12 12:02:45 +00:00
Robert Schulze
b9c185af44
Merge pull request #49678 from azat/build/llvm-16 
Switch to LLVM/clang 16 (16.0.3)
 2023-05-12 13:47:36 +02:00
Antonio Andelic
e0a3ac41b0
Merge pull request #49776 from azat/settings-aliases 
Fix settings aliases in native protocol
 2023-05-12 13:30:16 +02:00
Anton Popov
3351ef7398
Merge pull request #49789 from CurtizJ/fix-array-map-tuple 
Fix `arrayMap` with array of tuples with single argument
 2023-05-12 13:27:40 +02:00
Igor Nikonov
5513c74c96
Merge pull request #49677 from ClickHouse/enable-distinct-in-order 
Enable distinct in order after fix #49636
 2023-05-12 13:21:09 +02:00
Sema Checherinda
a1ee7d52e1
Merge pull request #49432 from CheSema/lost-blobs 
all s3-blobs removed when merge aborted, remove part from failed fetch without unlock keper
 2023-05-12 13:19:27 +02:00
alesapin
d147cb105c
Merge pull request #49755 from CurtizJ/fix-unit-test 
Fix unit test `ExceptionFromWait`
 2023-05-12 13:08:48 +02:00
Sema Checherinda
5706e05757 adding more comments 2023-05-12 13:00:56 +02:00
vdimir
94bfb1171d
Merge pull request #49784 from ClickHouse/vdimir/analyzer_prewhere_and_sample 2023-05-12 12:25:36 +02:00
Alexander Gololobov
7c7565f094
Merge pull request #49737 from ClickHouse/fix_lwd_and_object 
Ignore LWD column in checkPartDynamicColumns
 2023-05-12 12:20:07 +02:00
Sema Checherinda
22f7aa8d89 make special build pass 2023-05-12 12:00:15 +02:00
Han Fei
07341a04fd
Merge pull request #49666 from hanfei1991/hanfei/regexp-dict-read 
Support `dictionary` table function for `RegExpTreeDictionary`
 2023-05-12 11:40:46 +02:00
Nikita Mikhaylov
66badcd59c
Merge pull request #49725 from vitlibar/incremental-backup-append-table-def 
Fix writing appended files to incremental backups
 2023-05-12 11:37:49 +02:00
Robert Schulze
922420420c
Merge pull request #49300 from ClickHouse/rs/functdocs 
Introduce more fields for in-source function documentation
 2023-05-12 11:36:04 +02:00
Vitaly Baranov
eb62030fa4 Fix assigning a setting to NULL in settings profile's definition. 2023-05-12 11:04:30 +02:00
Azat Khuzhin
fa2074e67b Use CLOCK_MONOTONIC_RAW over CLOCK_MONOTONIC on Linux (fixes non monotonic clock) 
There is no bug in Linux, the issue that CLOCK_MONOTONIC returns values
less then previous calls likely happens due to adjtime(3) (NTP), since
CLOCK_MONOTONIC is affected by it, and I've seen lots of slight time
modifications due to NTP on the servers. And even on my desktop (I also
have NTP enabled):

    CLOCK_MONOTONIC:     189292.803 (2 days +  4h 34m 52s)
    CLOCK_MONOTONIC_RAW: 189290.016 (2 days +  4h 34m 50s)

However on Linux there is CLOCK_MONOTONIC_RAW, it is similar to
CLOCK_MONOTONIC, but does not affected by the adjtime(3).

About performance, it is the same:

    CLOCK_MONOTONIC 10e6:     real=0m0.191s user=0m0.190s sys=0m0.000s
    CLOCK_MONOTONIC_RAW 10e6: real=0m0.191s user=0m0.191s sys=0m0.000s

Ops/s:
- AMD Threadripper: 52.3e6
- Xeon Silver 4216 2.10: 46.5e6

Fixes: c5d631ca54
Fixes: #29811 (cc @tavplubix)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-12 10:59:53 +02:00
Robert Schulze
d15f19912f
Merge pull request #49198 from ClibMouse/s390x_reinterpretas_fix 
Fix reinterpretAs*() on big endian machines
 2023-05-12 10:33:50 +02:00
Azat Khuzhin
9c42f7f113 Fix settings aliases in native protocol 
The initial test (tests/queries/0_stateless/02539_settings_alias.sh)
works only because of the clickhouse-client, while in native protocol
aliases does not work.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-12 05:56:32 +02:00
Azat Khuzhin
2c40dd6a4c Switch Block::NameMap to google::dense_hash_map over HashMap 
Since HashMap creates 2^8 elements by default, while dense_hash_map
should be good here.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-12 05:52:57 +02:00
Dmitry Novik
0c0ea7bfc0 Analyzer: apply _CAST to constants only once 2023-05-11 23:07:48 +00:00
Alexey Milovidov
76f7f5750d
Merge pull request #49799 from rschu1ze/demange 
Typo: demange.cpp --> demangle.cpp
 2023-05-12 02:01:49 +03:00
Vitaly Baranov
c4036b8361
Merge pull request #49381 from ClickHouse/user-grants-from-config 
Specify grants for users in `users.xml`
 2023-05-12 00:15:52 +02:00
Kseniia Sumarokova
f5b959624d
Merge pull request #49733 from ClickHouse/nickitat-patch-11 
Do not allocate own buffer in CachedOnDiskReadBufferFromFile when `use_external_buffer == true`
 2023-05-12 00:05:25 +02:00
Kseniia Sumarokova
41fa0a54e8
Merge pull request #49729 from kssenii/minor-change-move-log 
Move assertions after logging
 2023-05-11 23:47:26 +02:00
Robert Schulze
8ca804d40e
Typo: demange.cpp --> demangle.cpp 2023-05-11 21:32:12 +00:00
Vitaly Baranov
478140f8b4 Fix LimitSeekableReadBuffer. 2023-05-11 22:58:05 +02:00
Alexey Milovidov
ea979b40a9
Merge pull request #49717 from ClickHouse/disable-mmap-for-server 
Disable mmap for StorageFile in clickhouse-server
 2023-05-11 23:53:01 +03:00
Han Fei
ef74e64336 address comments 2023-05-11 22:18:08 +02:00
Suzy Wang
70db49cdeb
Merge branch 'master' into s390x_reinterpretas_fix 2023-05-11 14:41:57 -04:00
Robert Schulze
8997c6ef95
Merge pull request #49750 from ClickHouse/rs/msan-randomStringUTF8 
Fix msan issue in randomStringUTF8(<uneven number>)
 2023-05-11 18:49:12 +02:00
Kruglov Pavel
084a696b54
Merge branch 'master' into fiber-local-var 2023-05-11 17:55:25 +02:00
Vitaly Baranov
ec011b9113
Merge branch 'master' into user-grants-from-config 2023-05-11 17:08:06 +02:00
Anton Popov
292cfe12af
Merge branch 'master' into fix-unit-test 2023-05-11 16:56:21 +02:00
Anton Popov
d21dc829d9
Merge branch 'master' into add-forgotten-lock 2023-05-11 16:56:13 +02:00
Anton Popov
84aa97b738 fix arrayMap with array of tuples with single argument 2023-05-11 14:52:01 +00:00
Azat Khuzhin
e5c4eb35e8 Merge remote-tracking branch 'u/master' into build/llvm-16 
* u/master:
  enable used flags's reinit only when the hash talbe rehash
  Fix build of libfiu on clang-16
  fix flaky test 02504_regexp_dictionary_ua_parser
  fix convertation
  Fix test that expected CH to apply a wrong optimization
  ActionsDAG: remove wrong optimization
 2023-05-11 16:32:25 +02:00
Azat Khuzhin
00fdfa115f Suppress MSan warning in NgramDistanceImpl::unrollLowering() 
NgramDistanceImpl::unrollLowering() relies on the fact that PODArray has
padding and it is OK to access more items.

Here is an MSan report:

    ==656==WARNING: MemorySanitizer: use-of-uninitialized-value
        0 0x557fd825485f in DB::NgramDistanceImpl<4ul, char8_t, false, true, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, DB::PODArray<float, 4096ul, Allocator<false, false>, 63ul, 64ul>&) (/usr/bin/clickhouse+0x124d885f) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)
        1 0x557fd824eb83 in DB::FunctionsStringSimilarity<DB::NgramDistanceImpl<4ul, char8_t, false, true, false>, DB::NameNgramSearchCaseInsensitive>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName>> const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x124d2b83) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)
        2 0x557fd50023b7 in DB::FunctionToExecutableFunctionAdaptor::executeImpl() const (/usr/bin/clickhouse+0xf2863b7) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)

      Uninitialized value was stored to memory at
        0 0x557fd4f8da5a in __msan_memcpy (/usr/bin/clickhouse+0xf211a5a) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)
        1 0x557fd8253803 in DB::NgramDistanceImpl<4ul, char8_t, false, true, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, DB::PODArray<float, 4096ul, Allocator<false, false>, 63ul, 64ul>&) (/usr/bin/clickhouse+0x124d7803) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)
        2 0x557fd824eb83 in DB::FunctionsStringSimilarity<DB::NgramDistanceImpl<4ul, char8_t, false, true, false>, DB::NameNgramSearchCaseInsensitive>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName>> const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x124d2b83) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)
        3 0x557fd50023b7 in DB::FunctionToExecutableFunctionAdaptor::executeImpl() const (/usr/bin/clickhouse+0xf2863b7) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)

      Uninitialized value was stored to memory at
        0 0x557fd4f8da5a in __msan_memcpy (/usr/bin/clickhouse+0xf211a5a) (BuildId: 76773125d8739591c75d4f4d263a2ffe7ca96855)
        1 0x5580061699f5 in detail::memcpySmallAllowReadWriteOverflow15Impl(char*, char const*, long) build_docker/./src/Common/memcpySmall.h:42:13
        2 0x5580061699f5 in memcpySmallAllowReadWriteOverflow15(void*, void const*, unsigned long) build_docker/./src/Common/memcpySmall.h:57:5
        3 0x5580061699f5 in DB::ColumnString::replicate(DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&) const build_docker/./src/Columns/ColumnString.cpp:462:13
        4 0x558005d3fae4 in DB::ColumnConst::convertToFullColumn() const build_docker/./src/Columns/ColumnConst.cpp:48:18

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-11 16:30:17 +02:00
Sergei Trifonov
1263c58f64
Merge branch 'master' into async-loader 2023-05-11 16:10:07 +02:00
Nikolay Degterinsky
3160483c17 Add query parameters to clickhouse-local 2023-05-11 12:56:59 +00:00
avogar
3ee8de792c Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-11 12:46:20 +00:00
vdimir
fd7374fbc5
Analyzer: fix column not found for optimized prewhere with sample by 2023-05-11 12:38:46 +00:00
Eridanus
990c224540 improvement(parser): add support for size suffixes in quota creation 
statement parameters

This commit adds support for creating quotas with size suffixes in the
parameter list of a quota creation statement. The parser now recognizes
size suffixes such as "KB", "MB", and "GB" and correctly converts them
to their corresponding values in bytes. This feature improves the
usability of the quota creation statement and makes it more flexible.
The commit also includes additional tests to ensure that the new syntax
is parsed correctly.
Attention: we did not support string literal with double like
execute_time = '1.5'
 2023-05-11 20:04:53 +08:00
SmitaRKulkarni
9bf6175919
Merge pull request #47584 from aiven-sal/aiven-sal/having_pr 
ActionsDAG: fix wrong optimization
 2023-05-11 14:03:46 +02:00
Kseniia Sumarokova
183b4be796
Merge branch 'master' into nickitat-patch-11 2023-05-11 13:57:33 +02:00
Igor Nikonov
9865d7c7ec
Merge branch 'master' into enable-distinct-in-order 2023-05-11 13:49:55 +02:00
Han Fei
6b3160f7ce trivial change 2023-05-11 13:49:02 +02:00
Sema Checherinda
98769bb433
Merge pull request #49749 from CheSema/convertations_to_timespan 
fix convertation
 2023-05-11 12:11:16 +02:00
Robert Schulze
9db78792d0
Fix MsgPackRowInputFormat.cpp build 2023-05-11 10:00:32 +00:00
Robert Schulze
c2a4d89b6f
Fix style 2023-05-11 09:29:05 +00:00
Robert Schulze
1f3b1c47af
Merge branch 'master' into rs/msan-randomStringUTF8 2023-05-11 10:34:00 +02:00
Alexander Gololobov
1a91b6293d
Merge branch 'master' into fix_lwd_and_object 2023-05-11 09:39:57 +02:00
Sema Checherinda
8470aa8fbb
Merge branch 'master' into convertations_to_timespan 2023-05-11 08:24:40 +02:00
Sema Checherinda
66e3712dca
Merge branch 'master' into lost-blobs 2023-05-11 08:23:25 +02:00
Sema Checherinda
9724688608 rename field, do not fetch PreActive 2023-05-11 08:22:37 +02:00
Sema Checherinda
9ef14d5ed4
Merge branch 'master' into multi_part_upload 2023-05-11 08:21:02 +02:00
Alexey Milovidov
cfd3f76217
Merge branch 'master' into disable-mmap-for-server 2023-05-11 06:17:32 +03:00
lgbo-ustc
a07359fbe8 enable used flags's reinit only when the hash talbe rehash 2023-05-11 11:06:13 +08:00
Alexey Milovidov
91ffe8be73
Merge branch 'master' into build/llvm-16 2023-05-11 05:38:40 +03:00
Alexey Milovidov
3591eab432
Merge pull request #49762 from ClickHouse/fix-typo-10 
Fix typo
 2023-05-11 05:16:25 +03:00
Alexey Milovidov
aed031c3a3 Fix typo 2023-05-11 04:15:54 +02:00
Anton Popov
ff287c7bba add lock 2023-05-10 23:00:49 +00:00
Igor Nikonov
268d7d70fc Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-10 21:22:24 +00:00
Igor Nikonov
72f4aed64b Add setting use_with_fill_by_sorting_prefix 2023-05-10 21:08:29 +00:00
Anton Popov
db52652337 fix unit test 2023-05-10 21:00:56 +00:00
Suzy Wang
d267c914c3
Merge branch 'master' into s390x_reinterpretas_fix 2023-05-10 16:08:50 -04:00
Suzy Wang
24b6ff47ac fix format and some more fix for fixedstring 2023-05-10 13:06:30 -07:00
Azat Khuzhin
d56e1423ef
Merge branch 'master' into build/llvm-16 2023-05-10 21:23:26 +02:00
Sema Checherinda
8bc9a32d19 fix special build 2023-05-10 18:45:59 +00:00
Sema Checherinda
60bf45f863 Update WriteBufferFromS3BufferAllocationPolicy.cpp 2023-05-10 18:31:58 +00:00
Sema Checherinda
cd449cca38 WriteBufferFromS3BufferAllocationPolicy for FixedSize and Exp policy 2023-05-10 18:31:58 +00:00
Sema Checherinda
7fbf87be17 rework WriteBufferFromS3, squashed 2023-05-10 18:31:47 +00:00
Robert Schulze
9795d5403f
Merge branch 'master' into rs/msan-randomStringUTF8 2023-05-10 20:16:49 +02:00
Alexander Gololobov
049ef9dda5
Merge branch 'master' into fix_lwd_and_object 2023-05-10 19:54:26 +02:00
Sema Checherinda
f2ad1122a1 fix convertation 2023-05-10 17:50:42 +00:00
Robert Schulze
374dbd9c39
Fix msan issue in randomStringUTF8(<uneven number>) 2023-05-10 17:49:23 +00:00
Igor Nikonov
5143b37d06 Fix clang-tidy build 2023-05-10 17:03:48 +00:00
Kruglov Pavel
289af626ad
Merge branch 'master' into fiber-local-var 2023-05-10 18:21:46 +02:00
Sergei Trifonov
2169f077be
Merge branch 'master' into async-loader 2023-05-10 18:02:11 +02:00
Han Fei
2273884ded address comments 2023-05-10 17:42:08 +02:00
Alexey Milovidov
adad880c14
Merge pull request #49716 from CurtizJ/fix-sparse-short-circuit 
Fix short circuit functions and mutations with sparse arguments
 2023-05-10 18:11:33 +03:00
Alexey Milovidov
4a4eb5b171
Merge pull request #49660 from CurtizJ/fix-sparse-columns-reload 
Fix reading from sparse columns after restart
 2023-05-10 18:08:22 +03:00
Alexey Milovidov
33a58aacea
Merge pull request #49719 from ClickHouse/make-projections-non-experimental 
Make projections production-ready
 2023-05-10 18:07:02 +03:00
Igor Nikonov
49971276d3
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-10 16:29:28 +02:00
Azat Khuzhin
a0cc5b7e47 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-10 16:28:36 +02:00
Alexander Gololobov
e6f4e5cd10 Ignore LWD column in checkPartDynamicColumns 2023-05-10 16:10:04 +02:00
Han Fei
d17aa828b3
Merge pull request #49361 from hanfei1991/hanfei/fiu 
support named fault injection and add a simple test
 2023-05-10 15:51:51 +02:00
Sergei Trifonov
6029bdeefa
Merge branch 'master' into async-loader 2023-05-10 15:04:49 +02:00
vdimir
1b7f54e886
Merge pull request #46467 from bigo-sg/first_value 2023-05-10 14:56:14 +02:00
Sergei Trifonov
0b2860d822
Merge branch 'master' into async-loader 2023-05-10 14:54:00 +02:00
avogar
a7aec49fbb Fix indents 2023-05-10 12:43:29 +00:00
Nikita Taranov
95bcccaf84
Do not allocate own buffer in CachedOnDiskReadBufferFromFile when use_external_buffer == true 2023-05-10 14:42:49 +02:00
Igor Nikonov
a0ef2e272f Fix suffix generation for empty result set 2023-05-10 12:41:33 +00:00
avogar
10e6f5b59a Fix indents 2023-05-10 12:31:34 +00:00
avogar
0cf6b9f145 Inherit context from parent fiber 2023-05-10 12:27:13 +00:00
Roman Vasin
632c0eeefc
Add support of temporary tables in FROM part of ATTACH|REPLACE PARTITION FROM (#49436) 
* Add temp tables after FROM

* Resolve works fine with temp and non-temp tables

* Add usage of temporary table in  00626_replace_partition_from_table test

* Update documentation

* Add tests for ReplicatedMergeTree

* Move tests into 02731_replace_partition_from_temporary_table
 2023-05-10 15:25:12 +03:00
Han Fei
505d9be7b2 address comments 2023-05-10 13:56:17 +02:00
kssenii
acae08714b Move assertions after logging 2023-05-10 13:03:34 +02:00
Kseniia Sumarokova
1ea53b5885
Merge pull request #49690 from kssenii/add-logging-for-try-reserve 
Add some logging
 2023-05-10 12:59:56 +02:00
Igor Nikonov
37bc0becc9 Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-10 10:53:10 +00:00
Igor Nikonov
93161f70d1 Correct suffix handling for ranges 2023-05-10 10:45:43 +00:00
Nikolay Degterinsky
6b2daff663
Merge pull request #48880 from alekar/48716 
Enhance usability of INTO OUTFILE clause for SELECT queries
 2023-05-10 12:35:22 +02:00
Ilya Yatsishin
a4f7bfa62d
Merge pull request #49718 from ClickHouse/fix-typo-9 
Fix typo
 2023-05-10 12:32:37 +02:00
Azat Khuzhin
2f9a7cba8a Fix stack-use-after-scope in KafkaProducer::produce() 
ASan report:

    ==1==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f7540186b20 at pc 0x55656ef458ad bp 0x7f754114acf0 sp 0x7f754114a4b8
    READ of size 2 at 0x7f7540186b20 thread T2 (TCPHandler)
        0 0x55656ef458ac in __asan_memcpy (/usr/bin/clickhouse+0x11b028ac) (BuildId: 6ba63057df561f6dc16fb8fd7623fd858bf327d1)
        1 0x55659f62daf8 in rd_kafka_msg_new00 build_docker/./contrib/librdkafka/src/rdkafka_msg.c:192:3
        2 0x55659f62daf8 in rd_kafka_msg_new0 build_docker/./contrib/librdkafka/src/rdkafka_msg.c:257:8
        3 0x55659f630635 in rd_kafka_producev build_docker/./contrib/librdkafka/src/rdkafka_msg.c:642:23
        4 0x55659f4a865c in cppkafka::Producer::do_produce() build_docker/./contrib/cppkafka/src/producer.cpp:128:19
        5 0x55659f4a8306 in cppkafka::Producer::produce() build_docker/./contrib/cppkafka/src/producer.cpp:69:5
        6 0x5565961e748c in DB::KafkaProducer::produce() build_docker/./src/Storages/Kafka/KafkaProducer.cpp:70:23
        7 0x5565962155b2 in DB::MessageQueueSink::consume(DB::Chunk) build_docker/./src/Storages/MessageQueueSink.cpp:64:23
        8 0x5565980b9e8e in DB::SinkToStorage::onConsume(DB::Chunk) build_docker/./src/Processors/Sinks/SinkToStorage.cpp:18:5

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-10 12:15:17 +02:00
Antonio Andelic
c6904805f4
Merge branch 'master' into fix-cnf-new-analyzer 2023-05-10 08:56:14 +02:00
Alexey Milovidov
0eae23ba2d
Merge pull request #49715 from ClickHouse/remove-unused-errorcodes 
Remove not used ErrorCodes
 2023-05-10 06:43:16 +03:00
Alexey Milovidov
8a6e07f0ea Make projections production-ready 2023-05-10 03:35:13 +02:00
Alexey Milovidov
56ea17c922 Fix typo 2023-05-10 03:21:36 +02:00
Alexey Milovidov
8a23d8b389 Disable mmap for server 2023-05-10 03:17:57 +02:00
Alexey Milovidov
a2c4b8e23d Disable mmap for server 2023-05-10 03:16:52 +02:00
robot-clickhouse-ci-2
c698d2af99
Merge pull request #49702 from ClickHouse/nickitat-patch-11 
Enable `do_not_evict_index_and_mark_files` by default
 2023-05-10 02:26:38 +02:00
Anton Popov
862008da0d
Merge pull request #49663 from CurtizJ/fix-race-context-copy 
Fix race in `Context::createCopy`
 2023-05-10 02:11:35 +02:00
Anton Popov
8ff5fda628 fix short curcuit function with sparse arguments 2023-05-09 23:54:13 +00:00
Alexey Milovidov
f20a1893f5
Merge branch 'master' into enable-distinct-in-order 2023-05-10 02:08:32 +03:00
Sergei Trifonov
dc5664f36a
Merge branch 'master' into remove-unused-errorcodes 2023-05-10 00:49:55 +02:00
Anton Popov
996fcfe120 fix column ttl with sparse columns 2023-05-09 21:37:43 +00:00
alekar
67ad681e41
Merge branch 'master' into 48716 2023-05-09 14:34:32 -07:00
Alexander Tokmakov
6f53ee289d fix another issue, add a test 2023-05-09 23:26:56 +02:00
Suzy Wang
1b21f13605 ip encoding fix 2023-05-09 13:57:22 -07:00
kssenii
e35c669892 Fix 2023-05-09 22:51:47 +02:00
alekar
605991d4d5
Merge branch 'master' into 48716 2023-05-09 13:38:11 -07:00
Suzy Wang
ce471a2e8b Updated code as suggested 2023-05-09 13:31:54 -07:00
Yakov Olkhovskiy
815cc8de9b removed unnecessary check for thread's joinability 2023-05-09 19:43:39 +00:00
Azat Khuzhin
9b3bcd0870 Fix stack-use-after-scope in gtest_resource_manager_static 
Reordering of the static variables should be enough to fix this:

    [ RUN      ] IOResourceStaticResourceManager.Prioritization
    =================================================================
    ==13==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f7d8d621970 at pc 0x5636b80dcbb2 bp 0x7f7c48e47dd0 sp 0x7f7c48e47dc8
    READ of size 8 at 0x7f7d8d621970 thread T3975 (ThreadPool)
        #0 0x5636b80dcbb1 in IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_1::operator()(long) const build_docker/./src/IO/Resource/tests/gtest_resource_manager_static.cpp:78:13
        #1 0x5636b80dcbb1 in IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0::operator()() const build_docker/./src/IO/Resource/tests/gtest_resource_manager_static.cpp:95:21
        #2 0x5636b80dcbb1 in decltype(std::declval<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&>()()) std::__1::__invoke[abi:v15000]<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        #3 0x5636b80dcbb1 in decltype(auto) std::__1::__apply_tuple_impl[abi:v15000]<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/tuple:1789:1
        #4 0x5636b80dcbb1 in decltype(auto) std::__1::apply[abi:v15000]<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&, std::__1::tuple<>&) build_docker/./contrib/llvm-project/libcxx/include/tuple:1798:1
        #5 0x5636b80dcbb1 in ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()::operator()() build_docker/./src/Common/ThreadPool.h:228:13
        #6 0x5636b80dcbb1 in decltype(std::declval<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()&>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        #7 0x5636b80dcbb1 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'()&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9
        #8 0x5636b80dcbb1 in std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12
        #9 0x5636b80dcbb1 in void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0>(IOResourceStaticResourceManager_Prioritization_Test::TestBody()::$_0&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16
        #10 0x5636ea219310 in std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16
        #11 0x5636ea219310 in std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12
        #12 0x5636ea219310 in ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) build_docker/./src/Common/ThreadPool.cpp:416:13
        #13 0x5636ea2258ac in void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const build_docker/./src/Common/ThreadPool.cpp:180:73
        #14 0x5636ea2258ac in decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        #15 0x5636ea2258ac in void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/thread:284:5
        #16 0x5636ea2258ac in void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>>(void*) build_docker/./contrib/llvm-project/libcxx/include/thread:295:5
        #17 0x7f7d8fcf1608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608) (BuildId: 7b4536f41cdaa5888408e82d0836e33dcf436466)
        #18 0x7f7d8fc16132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)

    Address 0x7f7d8d621970 is located in stack of thread T0 at offset 368 in frame
        #0 0x5636b80d9bef in IOResourceStaticResourceManager_Prioritization_Test::TestBody() build_docker/./src/IO/Resource/tests/gtest_resource_manager_static.cpp:46

      This frame has 10 object(s):
        [32, 36) 'requests_per_thead' (line 48)
        [48, 208) 't' (line 49)
        [272, 296) 'ref.tmp' (line 51)
        [336, 352) 'last_priority' (line 74)
        [368, 376) 'check' (line 75) <== Memory access at offset 368 is inside this variable
        [400, 424) 'name' (line 83)
        [464, 512) 'ref.tmp13' (line 87)
        [544, 560) 'c' (line 101)
        [576, 600) 'ref.tmp32' (line 101)
        [640, 664) 'ref.tmp41' (line 102)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:32:13 +02:00
Azat Khuzhin
d8dd50a9c6 Fix misc-misplaced-const clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:27:21 +02:00
Azat Khuzhin
0610865811 Fix readability-redundant-string-cstr clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:22:05 +02:00
Azat Khuzhin
d03ae2abfa Fix modernize-loop-convert clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:19:37 +02:00
avogar
930c8c3043 Use own fiber local implementation 2023-05-09 18:56:09 +00:00
Azat Khuzhin
051d0e57ff Suppress bugprone-standalone-empty in DatabaseMySQL 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 20:47:52 +02:00
Azat Khuzhin
be39e8e566 Merge remote-tracking branch 'u/master' into build/llvm-16 
Conflicts:
	docker/test/codebrowser/Dockerfile
 2023-05-09 20:40:43 +02:00
Azat Khuzhin
833652b7c9 Revert "Suppress clang-analyzer-cplusplus.NewDelete in MsgPackRowInputFormat" 
Let's try to revert this quirk during upgrading to clang 16.

This reverts commit c1e70169d2.
 2023-05-09 20:36:39 +02:00
Han Fei
ddce47f79e refine table source for regexp tree dictionary 2023-05-09 20:17:54 +02:00
kssenii
c20aa0786d More logging 2023-05-09 19:29:57 +02:00
kssenii
abc909fb21 Merge remote-tracking branch 'upstream/master' into add-logging-for-try-reserve 2023-05-09 19:13:58 +02:00
kssenii
13b77f1dac Fix 2023-05-09 19:13:39 +02:00
Anton Popov
d88224dc43 fix build 2023-05-09 16:30:43 +00:00
Nikita Taranov
7cd6db3fb3
Enable do_not_evict_index_and_mark_files by default 2023-05-09 18:04:47 +02:00
Dmitry Novik
1da54538f1
Merge branch 'master' into fix-cnf-new-analyzer 2023-05-09 17:31:05 +02:00
kssenii
75aa775d2b Add logging 2023-05-09 15:58:20 +02:00
serxa
459181568c remove not used ErrorCodes 2023-05-09 11:54:25 +00:00
alesapin
277b42451e
Merge branch 'master' into fix_database_replica_recovery 2023-05-09 13:06:45 +02:00
Azat Khuzhin
a794177dc3 Use __powerpc64__ over __ppc64__ (clang 16 stops declaring __ppc64__) 
From the release notes [1]:

    To match GCC, __ppc64__ is no longer defined on PowerPC64 targets. Use __powerpc64__ instead.

  [1]: https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 12:20:50 +02:00
Sergei Trifonov
4f9d443226
Merge branch 'master' into async-loader 2023-05-09 12:04:23 +02:00
vdimir
6bb552377f
Merge pull request #49085 from ClibMouse/saveIntervals 2023-05-09 11:33:05 +02:00
Kseniia Sumarokova
abde991cc4
Merge pull request #49675 from ClickHouse/cache-cleanup-after-locking-rework 
Some cache cleanup after rework locking
 2023-05-09 11:29:45 +02:00
Robert Schulze
3ab4a46c7b
Merge pull request #49658 from saitama951/clickhouse 
Fix Optimization Issue for T64 library for S390x
 2023-05-09 09:11:09 +02:00
Robert Schulze
d1429630f5
Merge pull request #49651 from jinjunzh/iaadeflate_idxd_config_amend 
add missing idxd-config module for qpl-deflate codec
 2023-05-09 08:32:45 +02:00
FFFFFFFHHHHHHH
2856816641
Merge branch 'master' into dot_product 2023-05-09 14:20:30 +08:00
Alexey Milovidov
3556c26075
Merge pull request #49434 from ClickHouse/ins 
Make INSERT do more things in parallel to avoid getting bottlenecked on one thread
 2023-05-09 08:10:15 +03:00
Alexey Milovidov
a8bdb20fc4
Merge pull request #48787 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks [Resubmit]
 2023-05-09 07:58:36 +03:00
FFFFFFFHHHHHHH
4a10f4b3d0
Merge branch 'master' into dot_product 2023-05-09 12:06:28 +08:00
FFFFFFFHHHHHHH
79398f612f fix style 2023-05-09 11:50:38 +08:00
Alexey Milovidov
9a73a04fe3
Merge pull request #42587 from ClickHouse/compress-marks-primary-key-by-default 
Compress marks and primary key by default
 2023-05-09 06:43:10 +03:00
Alexey Milovidov
dda6e9175c
Merge pull request #49647 from ClickHouse/optimize-opentelemetry-span-log 
Optimize `opentelemetry_span_log`
 2023-05-09 06:40:49 +03:00
fhbai
a7e04b7576 fix return type 2023-05-09 11:36:15 +08:00
Han Fei
64b67b9a37 Merge branch 'master' into hanfei/fiu 2023-05-09 01:14:30 +02:00
Yakov Olkhovskiy
70516f2f45 close pipe in run() if exception happens 2023-05-08 23:00:20 +00:00
Alexander Tokmakov
aa829c4ddc better partitions hint in mutations finalization 2023-05-09 00:23:21 +02:00
Kseniia Sumarokova
bfe9537012
Merge pull request #49426 from kssenii/remove-dependency-from-context 
Remove dependency from DB::Context in remote/cache readers
 2023-05-08 23:25:12 +02:00