thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 16:42:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
97,105 Commits 9,168 Branches 1,781 Tags 2.1 GiB
21ab379026
Commit Graph

30826 Commits

Author SHA1 Message Date
vdimir
031aaf3a45
Add Creating/FilterBySetsOnTheFlyStep for full sorting join 2022-08-29 09:46:57 +00:00
vdimir
c5bc7b0a0c
Resize pipeline after full sort join 2022-08-29 09:46:56 +00:00
Azat Khuzhin
f9812d9917 Fix memory leak while pushing to MVs w/o query context (from Kafka/...) 
While pushign to MVs, there is a low-level code that create
ThreadGroupStatus/ThreadStatus, it is required to gather some metrics
for system.query_views_log.

But, one should not use ThreadGroupStatus of the MainThreadStatus, since
this structure can hold some state, that may not be cleaned, plus this
may be racy, instead it is better to create new ThreadGroupStatus and
attach it instead.

Also this place misses detachQuery(), and because of this it leaks
ThreadGroupStatus::finished_threads_counters_memory. But it is only the
problem pushing to MVs is done w/o query context (i.e. from Kafka/...),
since when it has query context detachQuery() will be called eventually.

Before this patch series, when I've tried the reproducer with
500 MVs attached to Kafka engine (that @den-crane suggested), jemalloc
report looks like this:

    $ ../jeprof --text ~/ch/tmp/upstream/clickhouse-binary --base jeprof.44384.0.i0.heap jeprof.44384.167.i167.heap
    Using local file /home/azat/ch/tmp/upstream/clickhouse-binary.
    Using local file jeprof.44384.167.i167.heap.
    Total: 915.6 MB
       910.7  99.5%  99.5%    910.7  99.5% Snapshot (inline)
         9.5   1.0% 100.5%      9.5   1.0% std::__1::__libcpp_operator_new (inline)
         0.5   0.1% 100.6%      0.5   0.1% DB::TasksStatsCounters::create

And with focus to this place:

    $ ../jeprof --focus Snapshot --text ~/ch/tmp/upstream/clickhouse-binary --base jeprof.44384.0.i0.heap jeprof.44384.167.i167.heap
    Using local file /home/azat/ch/tmp/upstream/clickhouse-binary.
    Using local file jeprof.44384.167.i167.heap.
    Total: 915.6 MB
       910.7 100.0% 100.0%    910.7 100.0% Snapshot (inline)
         0.0   0.0% 100.0%    910.7 100.0% DB::QueryPipeline::reset
         0.0   0.0% 100.0%    910.7 100.0% DB::StorageKafka::streamToViews
         0.0   0.0% 100.0%    910.7 100.0% DB::StorageKafka::threadFunc
         0.0   0.0% 100.0%    910.7 100.0% ProfileEvents::Counters::getPartiallyAtomicSnapshot
         0.0   0.0% 100.0%    910.7 100.0% ~ThreadStatus
         0.0   0.0% 100.0%    910.7 100.0% ~ViewRuntimeData
         0.0   0.0% 100.0%    910.7 100.0% ~ViewRuntimeStats (inline)

Actually this report does not looks great (you understand it because I
stripped it), because --text does not that smart, but if you will use
--pdf for the report you will see the stacktrace (will attach pdf to the
pull request).

But after this patch series the process RSS does not goes beyond
~700MiB.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:36:33 +02:00
Azat Khuzhin
6da5707f8f Fix possible missing detachQuery() in case of exception in readers 
This can create leaks, since detachQuery() responsible for cleaning,
i.e. ThreadGroupStatus::finished_threads_counters_memory

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:30:17 +02:00
Azat Khuzhin
b16891da8d Avoid using of ThreadGroupStatus of the MainThreadStatus 
One should not use MainThreadStatus, since ThreadGroupStatus can hold
some states, and it is better not to play with this, since this may
create leaks.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:30:17 +02:00
Azat Khuzhin
9fff08eac7 WriteBufferFromS3: remove unused ThreadGroupStatus 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:30:17 +02:00
alesapin
133ca01447 Merge branch 'master' into stress_s3 2022-08-29 11:25:28 +02:00
alesapin
4ed375ca5b
Merge pull request #40720 from ClickHouse/fix_benign_race 
Fix benign race in database replicated worker
 2022-08-29 11:13:07 +02:00
Azat Khuzhin
269453a646 Avoid leaking of ThreadGroupStatus::finished_threads_counters_memory 
Cleanup them in ThreadStatus::detachQuery(), anyway they cannot be
received after by the client.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:08:50 +02:00
alesapin
7ce0afc0df
Merge pull request #40670 from Avogar/kafka 
Add setting to disable limit on kafka_num_consumers
 2022-08-29 10:53:35 +02:00
Amos Bird
15a69bce84
Use index when row_policy_filter is always false 2022-08-29 16:44:32 +08:00
Antonio Andelic
b4af21b6fb Small fixes 2022-08-29 07:06:51 +00:00
Amos Bird
abf51bccfd
Disable projection when grouping set is used. 2022-08-29 14:35:48 +08:00
Konstantin Morozov
6636bdec2a refactoring: fix format 2022-08-29 07:44:40 +03:00
Cory Levy
d0ace9a8fb Add the %z format descriptor to formatDateTime 2022-08-28 22:28:11 -04:00
Alexey Milovidov
71f6c52c2d
Merge pull request #40727 from amosbird/column-transformer-fix1 
Correct format of APPLY transformer param
 2022-08-29 03:48:04 +03:00
Alexey Milovidov
307e8f2da9
Merge pull request #40729 from ClickHouse/remove-useless-method 
Remove useless method
 2022-08-29 03:46:34 +03:00
Alexey Milovidov
ae97e880d7
Merge pull request #40710 from azat/DWARF-5 
Support for DWARF-5 in in house DWARF parser
 2022-08-29 00:13:33 +03:00
Alexey Milovidov
27782ceef8 Remove useless method 2022-08-28 22:33:42 +02:00
Konstantin Morozov
38a85ade27 refactoring: small up 2022-08-28 22:33:12 +03:00
Konstantin Morozov
b4de735a90 refactoring: fix space 2022-08-28 20:55:08 +03:00
Konstantin Morozov
08e9e799d0 refactoring: fix 2022-08-28 20:53:00 +03:00
Konstantin Morozov
75bd61fad5 Merge remote-tracking branch 'origin/ref/remove-unnecessary-allocation' into ref/remove-unnecessary-allocation 
# Conflicts:
#	src/Access/ContextAccess.cpp
#	src/Access/ContextAccess.h
 2022-08-28 20:50:29 +03:00
Konstantin Morozov
7cd2821aed refactoring: template make helper 2022-08-28 20:42:55 +03:00
Amos Bird
d1fbe51b81
Correct format of APPLY transformer param 2022-08-29 01:21:12 +08:00
Konstantin Morozov
4050ab819e refactoring: remove unnecessary allocation 2022-08-28 17:23:17 +03:00
Konstantin Morozov
d4d80dd3c1 refactoring: remove unnecessary allocation 2022-08-28 17:09:13 +03:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented 
All new table functions must be documented
 2022-08-28 17:05:20 +03:00
Alexey Milovidov
00074a5e14
Merge pull request #40711 from ClickHouse/all_new_functions_must_be_documented 
All new functions must be documented
 2022-08-28 17:05:05 +03:00
Azat Khuzhin
e03870bc8b Common/Dwarf: fix clang-tidy 2022-08-28 15:51:46 +02:00
alesapin
4f9faab924 Fix benign race in database replicated worker 2022-08-28 15:07:29 +02:00
Kseniia Sumarokova
a0bc5b6ea4
Merge pull request #40705 from azat/stress/max_read_buffer_size 
Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
 2022-08-28 12:41:16 +02:00
Robert Schulze
df934d8762
Merge pull request #40217 from zvonand/zvonand-minmax 
Fix conversion Date32 / DateTime64 / Date to narrow types
 2022-08-28 09:42:39 +02:00
Alexey Milovidov
d3a7945b1b
Merge pull request #40646 from ClickHouse/force-documentation-3 
Every profile event must have documentation
 2022-08-28 04:10:11 +03:00
Alexey Milovidov
5b57c91ad6 Rename table function MeiliSearch to meilisearch 2022-08-28 02:08:51 +02:00
Alexey Milovidov
1ff535a128 One step back 2022-08-28 02:00:09 +02:00
Alexey Milovidov
8a4bacc4b7 Update example 2022-08-28 00:30:14 +02:00
Alexey Milovidov
6ac9eabbd4 Register more aliases correctly 2022-08-28 00:17:13 +02:00
Alexey Milovidov
a6d99e795e Register more aliases correctly 2022-08-28 00:14:38 +02:00
Alexey Milovidov
e266688576 Remove something stupid 2022-08-27 23:58:34 +02:00
Alexey Milovidov
421d28542b Register all function aliases correctly 2022-08-27 23:53:51 +02:00
Alexey Milovidov
90cf0bb6e4 Fix style 2022-08-27 23:40:24 +02:00
Alexey Milovidov
365a600fdb Merge branch 'force-documentation-3' of github.com:ClickHouse/ClickHouse into force-documentation-3 2022-08-27 22:28:54 +02:00
Alexey Milovidov
6b2e227c8b Fix integration test 2022-08-27 22:28:38 +02:00
Azat Khuzhin
ee5696bb32 Add support for DWARF-5 
I have to do this, since I have libraries compiled with DWARF5 (i.e.
glibc).

ClickHouse changes:
- use camel_case
- add NOLINT
- avoid using folly:: (use std:: instead)
- avoid using boost:: (use std:: instead)

Refs: 490b287ca3
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 22:28:15 +02:00
Alexey Milovidov
6139cc8f7a Add Documentation to FunctionFactory 2022-08-27 22:06:57 +02:00
Alexey Milovidov
f86242c17e
Merge pull request #40699 from ClickHouse/remove-unbundled 
Remove the remaining parts of "unbundled" build
 2022-08-27 21:54:42 +03:00
Alexey Milovidov
e7054debc1
Merge pull request #40649 from ClickHouse/embedded-documentation 
A prototype for embedded documentation
 2022-08-27 21:28:11 +03:00
Alexey Milovidov
a6387ac0f7
Merge branch 'master' into force-documentation-3 2022-08-27 19:31:44 +03:00
Alexey Milovidov
6e564b18bf
Merge pull request #40600 from FrankChen021/check_url_arg 
Validate the CompressionMethod parameter of URL table engine
 2022-08-27 19:29:55 +03:00
Alexey Milovidov
7569cbb381
Merge pull request #40643 from ClickHouse/fix-recursive-buffer 
Fix stack overflow in recursive Buffer tables.
 2022-08-27 19:27:24 +03:00
Alexey Milovidov
779fac512d Revert wrong modification 2022-08-27 15:55:28 +02:00
Alexey Milovidov
2a931c69b7
Merge pull request #40682 from ClickHouse/remove-accurate-underscore-cast 
Remove `accurate_Cast` and `accurate_CastOrNull`, which are unused
 2022-08-27 16:53:30 +03:00
Alexey Milovidov
e04f69717b
Merge pull request #40673 from ClickHouse/vdimir/tmp-file-metrics-doc 
Add doc to ExternalSort/Agg/Join ProfileEvents
 2022-08-27 16:52:40 +03:00
Alexey Milovidov
36ca126a26
Merge pull request #40679 from ClickHouse/fix-typo 
Fix typo
 2022-08-27 16:52:21 +03:00
Azat Khuzhin
c801469e9e Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks 
The problem is that the buffer size validated after marks reading in
MergeTreeReaderStream::init(), since it requires to read marks first.

And later it is passed to AsynchronousReadBufferFromFileDescriptor,
which throws LOGICAL_ERROR because buffer_size < alignment.

Fix this my simply disallow such values for max_read_buffer_size (I
thougt about modifying createReadBufferFromFileBase(), but it is not
used for remote reads -- remote_fs_buffer_size).

Fixes: #40669
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 14:12:07 +02:00
Alexander Gololobov
6fc8f1e9e0
Merge pull request #40645 from ClickHouse/force-documentation-2 
Every current metric must have documentation
 2022-08-27 10:44:47 +02:00
Duc Canh Le
1691db9ce6
Merge branch 'master' into ch_canh_fix_json 2022-08-27 09:23:04 +08:00
Alexey Milovidov
91ed3ebe9a Remove the remaining parts of "unbundled" build 2022-08-27 02:18:47 +02:00
Azat Khuzhin
3c478da003 Move committing InMemory parts to WAL out of NOEXCEPT_SCOPE() 
Since this commit can definitelly throw (i.e. due to ENOSPC).

Note, that it should be safe, since rollback() will call dropPart() for
those parts.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-26 23:37:59 +02:00
Alexander Gololobov
89be7a5a25
Update ProfileEvents.cpp 2022-08-26 22:55:42 +02:00
Kseniia Sumarokova
11b82ffc5e
Merge branch 'master' into fix-race 2022-08-26 22:55:14 +02:00
Kseniia Sumarokova
0cf76dab77
Merge pull request #40608 from ClickHouse/kssenii-patch-4 
Fix stress test after #40420
 2022-08-26 22:54:37 +02:00
Alexey Milovidov
53343e0fc2 Remove accurate_Cast and accurate_CastOrNull, which are unused 2022-08-26 21:27:30 +02:00
Alexey Milovidov
c9858099ea
Update ProfileEvents.cpp 2022-08-26 22:10:10 +03:00
Alexander Gololobov
4fef408104
Merge branch 'master' into force-documentation-3 2022-08-26 20:29:06 +02:00
Alexey Milovidov
6940bdab3b
Merge branch 'master' into fix-recursive-buffer 2022-08-26 21:25:11 +03:00
Alexey Milovidov
02bfebf1d3 Fix typo 2022-08-26 20:08:33 +02:00
Alexey Milovidov
1ad44acce8
Merge pull request #40644 from ClickHouse/force-documentation 
Every setting must have documentation
 2022-08-26 21:06:57 +03:00
Dmitry Novik
5b7fe91675 Avoid deadlock in case of new query and OOM 2022-08-26 17:02:31 +02:00
Antonio Andelic
bff2b232ef Cache deltas for path 2022-08-26 14:03:38 +00:00
FArthur-cmd
f3d0083e57 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy-2 2022-08-26 13:53:53 +00:00
vdimir
c5a9b64080
Add doc to ExternalSort/Agg/Join ProfileEvents 2022-08-26 13:41:53 +00:00
Alexey Milovidov
46594e47bc Add comment 2022-08-26 15:28:18 +02:00
Alexey Milovidov
c2dffebe26
Merge branch 'master' into force-documentation-2 2022-08-26 16:10:58 +03:00
avogar
e31bba7731 Add docs 2022-08-26 12:52:15 +00:00
avogar
b8061532ac Add setting to disable limit on kafka_num_consumers 2022-08-26 12:49:39 +00:00
Kseniia Sumarokova
fcb53aacdb
Merge pull request #40642 from barumrho/barumrho/sqlite-int8-conversion 
Convert SQLite int8 column to ClickHouse int64
 2022-08-26 14:33:20 +02:00
Kseniia Sumarokova
4e5385298a
Update src/TableFunctions/TableFunctionURL.cpp 2022-08-26 14:22:29 +02:00
kssenii
470e18677f Ger rid of cache lock 2022-08-26 14:13:36 +02:00
Antonio Andelic
26d68dce23 Fix style 2022-08-26 10:55:07 +00:00
Antonio Andelic
dae65178b7 Improve preprocessing performance 2022-08-26 10:46:14 +00:00
Kruglov Pavel
e2e1ca26e9
Merge pull request #40628 from Avogar/fix-to-fixed-string 
Fix short circuit execution of toFixedString function
 2022-08-26 12:26:16 +02:00
Kruglov Pavel
58538d3f90
Merge pull request #40636 from Avogar/fix-url-format-detection 
Better format detection for url table function/engine
 2022-08-26 12:25:55 +02:00
Vladimir C
e067629e0d
Merge pull request #40239 from vdimir/vdimir/tmp-file-metrics 
More metrics for on-disk temporary data
 2022-08-26 11:28:01 +02:00
Robert Schulze
1c8c83ccf3
Merge pull request #40620 from zvonand/zvonand-b58 
Base58 fix handling leading 0 / '1'
 2022-08-26 10:18:26 +02:00
Alexander Gololobov
c7a792d929
small grammar fix 2022-08-26 10:03:43 +02:00
Alexander Gololobov
856f83caa1
Removed unnecessary commas 2022-08-26 10:00:20 +02:00
Lloyd-Pottiger
0d33327eda
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-26 15:36:21 +08:00
Frank Chen
cf1081eada Exception safe from destructor 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-26 11:18:16 +08:00
Frank Chen
2e874b384d
Update src/Storages/StorageURL.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-08-26 10:34:46 +08:00
Alexey Milovidov
a029116fe3 Add a comment 2022-08-26 04:01:34 +02:00
Alexey Milovidov
8afbfebd25 A prototype for embedded documentation 2022-08-26 03:41:52 +02:00
何李夫
2ec087192e
Merge branch 'master' into master 2022-08-26 09:09:29 +08:00
Alexey Milovidov
5b5a16320b Suggestions from @azat 2022-08-26 03:00:56 +02:00
Alexey Milovidov
91bd6f1d46 Write the documentation for all ProfileEvents. Enforce that documentation exists. 2022-08-26 01:35:03 +02:00
Alexey Milovidov
6659438d87 Every current metric must have documentation 2022-08-26 00:33:25 +02:00
Alexey Milovidov
a1acdf18ec Write the documentation 2022-08-26 00:14:46 +02:00
Kseniia Sumarokova
a8abf58f13
Update FileSegment.cpp 2022-08-25 23:15:24 +02:00
Barum Rho
a61827fb78 Convert SQLite int8 column to ClickHouse int64 
SQLite int8 column uses 64 bit integers.
 2022-08-25 16:15:50 -04:00
Alexander Tokmakov
b8e3a582f7 do not try to strartup dropped replica 2022-08-25 22:14:39 +02:00
Alexey Milovidov
ebe7d2d25e Fix stack overflow on recursive Buffer tables and system.tables 2022-08-25 21:26:32 +02:00
Azat Khuzhin
09aaa2b5dd Do not obtain storage snapshot for each INSERT block (improves performance) 
It depends on how much parts do you have, but for some workload with
InMemory only parts w/o merges, I got 5% increase.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-25 21:02:27 +02:00
Alexander Gololobov
6a69e08799
Merge pull request #40559 from ClickHouse/lwd_vertical_merge_fix 
Fix vertical merge of parts with lightweight deleted rows
 2022-08-25 20:47:44 +02:00
avogar
cef2972af4 Better format detection for url table function/engine 2022-08-25 18:31:06 +00:00
Alexey Milovidov
fcba079baa
Merge pull request #40572 from Avogar/fix-map-bug 
Fix possible logical error in arrayElement function with Map
 2022-08-25 20:17:22 +03:00
Alexander Tokmakov
eae2667a1c
Merge pull request #40148 from ClickHouse/replicate-tables-readonly 
`ReplicatedMergeTree` tables as readonly until attach is finished
 2022-08-25 19:39:19 +03:00
Alexander Tokmakov
a6f55b1430 fix 2022-08-25 16:27:12 +02:00
avogar
1844873dbc Fix short circuit execution of toFixedString function 2022-08-25 16:15:22 +02:00
zvonand
3b164f0b12 style fix 2022-08-25 17:07:41 +03:00
Vladimir C
ddde5096ef
Merge branch 'master' into vdimir/tmp-file-metrics 2022-08-25 15:23:35 +02:00
alesapin
10fdc1e286 Fix stupid bug 2022-08-25 14:52:01 +02:00
zvonand
b3360c169d Fix handling zeros/'1's at the beginning of input 2022-08-25 15:35:39 +03:00
Frank Chen
ef49cacefe Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:32:22 +08:00
zvonand
23357350d3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-minmax 2022-08-25 15:32:04 +03:00
Frank Chen
92f7ca3616 Move TracingContextOnThread::current() out of class for simplicity 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:23:56 +08:00
Alexander Tokmakov
0aea4c7184 fix 2022-08-25 14:22:27 +02:00
Frank Chen
bb00dcc19b Remove using namespace from header 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:20:13 +08:00
Frank Chen
c9ea4f9f77 Change compression_method from String to CompressionMethod 2022-08-25 19:18:04 +08:00
Kseniia Sumarokova
fac87d9c4d
Update FileSegment.cpp 2022-08-25 13:18:03 +02:00
alesapin
3ff6489fae Merge branch 'master' into stress_s3 2022-08-25 13:14:58 +02:00
alesapin
7d1e3ecfa3
Merge pull request #40506 from ClickHouse/fix_bad_exception 
Fix lost errors in AWS CPP SDK
 2022-08-25 13:07:35 +02:00
Kseniia Sumarokova
2089ce14dd
Merge branch 'master' into fix-race 2022-08-25 13:03:29 +02:00
Kseniia Sumarokova
44577ea8f6
Update FileSegment.cpp 2022-08-25 13:02:41 +02:00
Kseniia Sumarokova
7e0ff0dfaf
Update FileSegment.cpp 2022-08-25 12:53:51 +02:00
Kseniia Sumarokova
b0f5fd9bf6
Update FileSegment.cpp 2022-08-25 12:41:31 +02:00
alesapin
c4f8811701
Merge pull request #40395 from ClickHouse/data_source_description 
Add copy shortcut for backups if dest and source support it
 2022-08-25 12:36:30 +02:00
Kseniia Sumarokova
4b62d0f75f
Merge pull request #40511 from maks-buren630501/master 
Fix bug (issues 40262)
 2022-08-25 12:18:23 +02:00
pzhdfy
e85e885d40 add blank line 2022-08-25 18:15:01 +08:00
Kseniia Sumarokova
f6eea03120
Merge pull request #40569 from kssenii/add-comments 
Add some logs to fs cache
 2022-08-25 12:06:48 +02:00
zvonand
24dfa0c64a tryfix darwin 2022-08-25 12:28:15 +03:00
Kruglov Pavel
66e677b307
Merge pull request #40546 from Avogar/fix-inferring-datetimes 
Fix possible error 'Decimal math overflow' while parsing DateTime64
 2022-08-25 11:18:30 +02:00
Kruglov Pavel
25d7d05148
Merge pull request #40567 from ClickHouse/Avogar-patch-1 
Better exception messages in FunctionArrayMapped
 2022-08-25 11:14:06 +02:00
vdimir
414872b1b9 Rename logger.warning_supress_regex -> warning_supress_regexp 2022-08-25 11:00:01 +02:00
vdimir
19cb068f64 Add config option warning_supress_regex 2022-08-25 11:00:01 +02:00
Antonio Andelic
f90612f577 Remove shutdown assert 2022-08-25 07:07:54 +00:00
Frank Chen
221a71f801 Fix typo 2022-08-25 15:05:07 +08:00
Frank Chen
4c1fe0d623 validate parameter 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 14:49:17 +08:00
helifu
b52e098a40 Fix a crash while the grpc port is conflict 2022-08-25 10:56:53 +08:00
Frank Chen
558b179cb1 Fix typo 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 10:14:18 +08:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
zvonand
3797e7cfbe Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-minmax 2022-08-25 01:57:49 +03:00
Kseniia Sumarokova
ba4dfe8308
Update MaterializedPostgreSQLConsumer.cpp 2022-08-25 00:23:59 +02:00
Kseniia Sumarokova
fec58f7d1b
Merge pull request #40578 from ClickHouse/kssenii-patch-4 
Fix after #40420
 2022-08-24 23:53:26 +02:00
Alexey Milovidov
aaafb797c0
Merge pull request #40541 from ClickHouse/paranoid_check_for_s3_uploads 
Add paranoid option to check blobs to S3 after upload
 2022-08-24 23:51:22 +02:00
zvonand
a61fd73c88 fix test fails 2022-08-25 00:26:44 +03:00
Alexander Tokmakov
3cd26aafe4 remove wrong code for skipping mutations in MergeTree 2022-08-24 20:58:59 +02:00
Kseniia Sumarokova
4e7578cf3f
Merge pull request #40565 from FrankChen021/url_engine 
Fix segment fault when writing data to URL table engine
 2022-08-24 20:36:45 +02:00
Antonio Andelic
797e21761a Address PR comments 2022-08-24 18:32:03 +00:00
Nikita Taranov
ac34a17551
Merge branch 'master' into control_block_size_in_aggregator 2022-08-24 20:25:28 +02:00
Kseniia Sumarokova
588be578c3
Merge branch 'master' into add-comments 2022-08-24 19:47:20 +02:00
Antonio Andelic
da0d6056f4 Merge branch 'master' into replicate-tables-readonly 2022-08-24 17:29:33 +00:00
kssenii
ee2f9caa16 Fix 2022-08-24 18:50:22 +02:00
zvonand
5c00428940 updated as was before 2022-08-24 19:46:17 +03:00
vdimir
91788f29e8
Upd TemporaryFileOnDisk 2022-08-24 16:15:54 +00:00
vdimir
7194df1184
Move back TemporaryFile -> TemporaryFileOnDisk 2022-08-24 16:14:11 +00:00
vdimir
0349c85017
Use getCompressedBytes in BufferingToFileTransform and TemporaryFileStream 2022-08-24 16:14:10 +00:00
vdimir
1e24a731f7
Move Disks/TemporaryFileOnDisk -> Common/TemporaryFile 2022-08-24 16:14:10 +00:00
vdimir
463320f15a
Fix TemporaryFileOnDisk 2022-08-24 16:14:09 +00:00
vdimir
8701a909a2
enoughSpaceInDirectory works if file is not created yet 2022-08-24 16:14:09 +00:00
vdimir
51c44424cc
More metrics for temp files 2022-08-24 16:14:09 +00:00
vdimir
cd4038d511
Use TemporaryFileOnDisk instead of Poco::TemporaryFile 2022-08-24 16:14:08 +00:00
vdimir
1321ac87b5
Minor fixes 2022-08-24 16:14:07 +00:00
vdimir
7e0c9062c7
Add ProfileEvents::ExternalSort(Un)CompressedBytes 2022-08-24 16:14:07 +00:00
vdimir
fbc35f066b
Remove unsused ctors of BackupEntryFromImmutableFile 2022-08-24 16:14:07 +00:00
Lloyd-Pottiger
9d7ace3286
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-24 23:52:34 +08:00
alesapin
51afaedbf2 Better defaults 2022-08-24 17:51:26 +02:00
alesapin
669a48e302 Merge branch 'data_source_description' of github.com:ClickHouse/ClickHouse into data_source_description 2022-08-24 17:48:15 +02:00
alesapin
571778ad25
Update src/Backups/BackupIO_Disk.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-08-24 17:45:26 +02:00
alesapin
814bc37f0e Use DiskPtr 2022-08-24 17:45:20 +02:00
Lloyd-Pottiger
dc64341876 add tests 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-24 23:42:13 +08:00
Alexander Gololobov
c9177d2cb3
Merge pull request #40475 from ClickHouse/allow-conversion-from-string-datetime64-to-date-date32 
Allow conversion from String with DateTime64 to Date and Date32
 2022-08-24 17:33:20 +02:00
alesapin
14fb1bcab3
Merge branch 'master' into stress_s3 2022-08-24 17:31:17 +02:00
Kseniia Sumarokova
3b982f6b8b
Update FileCache.cpp 2022-08-24 17:29:43 +02:00
Alexander Gololobov
6c256cbe4a
Merge pull request #40474 from ClickHouse/allow-to-parse-date-as-datetime 
Allow parsing Date as DateTime and DateTime64
 2022-08-24 17:23:44 +02:00
alesapin
ac54f15c06 Remove more whitespaces 2022-08-24 17:06:27 +02:00
alesapin
f2be18b3a6 Remove trailing 2022-08-24 16:57:59 +02:00
alesapin
69084f5d7a Merge branch 'master' into stress_s3 2022-08-24 16:53:30 +02:00
alesapin
96b32c6380 Fix batch delete 2022-08-24 16:51:21 +02:00
Kseniia Sumarokova
aa0ed28a0b
Update FileSegment.cpp 2022-08-24 16:43:44 +02:00
Kseniia Sumarokova
985276f8e6
Merge branch 'master' into add-comments 2022-08-24 16:33:35 +02:00
alesapin
f5f0934c13
Update PocoHTTPClient.h 2022-08-24 16:15:43 +02:00
zvonand
b4564099f9 fix inherited parts 2022-08-24 17:07:30 +03:00
Kruglov Pavel
e6e7f5db93
Merge pull request #40491 from mini4/fix-settings-input_format_tsv_skip_first_lines 
Fix bug in settings input_format_tsv_skip_first_lines of format TSV
 2022-08-24 15:57:45 +02:00
Kseniia Sumarokova
6d0907ebef
Update FileSegment.cpp 2022-08-24 15:55:07 +02:00
Kseniia Sumarokova
2cea295a7a
More logs 2022-08-24 15:53:54 +02:00
alesapin
eeb0a3584e Merge branch 'master' into data_source_description 2022-08-24 15:26:18 +02:00
alesapin
e3672d1d40 Fix memsan 2022-08-24 15:25:10 +02:00
Alexander Gololobov
1c2dd50ca5 Fix vertical merge of parts with lightweight deleted rows 2022-08-24 15:18:33 +02:00
alesapin
f0b733313f Merge branch 'master' into fix_bad_exception 2022-08-24 14:44:40 +02:00
alesapin
cac3005f4d Merge branch 'master' into paranoid_check_for_s3_uploads 2022-08-24 14:43:11 +02:00
Alexander Tokmakov
b8abecb908
Merge pull request #40577 from ClickHouse/revert-39107-parallel-insert-select-s3-cluster 
Revert "Parallel distributed insert select from *Cluster table functions"
 2022-08-24 15:26:03 +03:00
Alexander Tokmakov
9820c3fce0
Merge pull request #40545 from ClickHouse/fix_restart_replica_race 
Fix race on system restart replica
 2022-08-24 15:18:36 +03:00
Alexander Tokmakov
f9f85a0e8b Revert "Parallel distributed insert select from *Cluster table functions (#39107)" 
This reverts commit d3cc234986.
 2022-08-24 15:17:15 +03:00
Kseniia Sumarokova
3e8a2d9093
Update FileSegment.cpp 2022-08-24 14:16:53 +02:00
Kruglov Pavel
109f384e5d
Fix typo 2022-08-24 13:40:19 +02:00
Kruglov Pavel
0781e8b4f7
Merge pull request #40534 from Avogar/nested-in-avro 
Support reading Array(Record) into flatten nested table in Avro
 2022-08-24 13:33:12 +02:00
avogar
fa320f65f6 Fix possible logical error in arrayElement function with Map 2022-08-24 11:01:39 +00:00
kssenii
2b792a2bdd Add more logs 2022-08-24 12:40:44 +02:00
Kruglov Pavel
860d0baa48
Better exception messages in FunctionArrayMapped 2022-08-24 12:28:44 +02:00
Kseniia Sumarokova
38c4094f89
Update MaterializedPostgreSQLConsumer.h 2022-08-24 12:25:03 +02:00
Kseniia Sumarokova
de9d254290
Update MaterializedPostgreSQLConsumer.cpp 2022-08-24 12:24:44 +02:00
Kseniia Sumarokova
4486f26d51
Update MaterializedPostgreSQLConsumer.cpp 2022-08-24 12:23:08 +02:00
Frank Chen
1cf3659f26 Fix style checking 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-24 18:03:14 +08:00
Frank Chen
b432b0d100
Merge branch 'master' into url_engine 2022-08-24 17:59:21 +08:00
Frank Chen
335293549b Fix segment fault when sending data to remote HTTP server via URL table engine 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-24 17:38:16 +08:00
Frank Chen
dd0f2cbe6b Fix style checking 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-24 17:04:02 +08:00
Frank Chen
03007606ba Fix format 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-24 17:01:31 +08:00
Frank Chen
cd19366b44 Move classes into DB::OpenTelemetry namespace 2022-08-24 16:41:40 +08:00
Frank Chen
efc6a60a60 Clean code 2022-08-24 15:59:44 +08:00
kgurjev
f62c2c3221 Fix bug in settings input_format_tsv_skip_first_lines of format TSV 2022-08-24 10:02:57 +03:00
zvonand
6ad02fd687 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-minmax 2022-08-24 01:12:38 +03:00
zvonand
e257f9d0cd update docs, tests + small fixes 2022-08-24 01:09:14 +03:00
Amos Bird
d58de79f57
Revert "Better logging support." 
This reverts commit a774dd708d.
 2022-08-24 00:58:42 +08:00
Amos Bird
a8d8293466
Fix unused columns introduced by with stmt 2022-08-24 00:40:01 +08:00
Amos Bird
a774dd708d
Better logging support. 2022-08-24 00:40:00 +08:00
zvonand
42f86442ab updated tests + toStartOfWeek 2022-08-23 18:33:09 +03:00
Kruglov Pavel
72f02bd6eb
Merge pull request #40414 from Avogar/improve-schema-inference-cache 
Improve schema inference cache, respect format settings that can change the schema
 2022-08-23 17:04:58 +02:00
avogar
de489ba1f7 Fix possible error 'Decimal math overflow' while parsing DateTime64 2022-08-23 14:42:23 +00:00
Andrey Zvonov
52159b77d0
Merge branch 'master' into zvonand-minmax 2022-08-23 17:18:57 +03:00
zvonand
6abe09a725 updated tests + teStartOfWeek 2022-08-23 17:13:39 +03:00
Alexander Tokmakov
77178a18fa fix race on system restart replica 2022-08-23 16:10:44 +02:00
Vladimir C
3ee52eb25f
Merge pull request #40335 from liql2007/master 
Fix bugs in MergeJoin when 'not_processed' is not null
 2022-08-23 16:02:23 +02:00
Duc Canh Le
e42e78436e
Merge branch 'master' into ch_canh_fix_json 2022-08-23 21:53:12 +08:00
Duc Canh Le
3bb6dbbed4 fix fold dimensions with null type 2022-08-23 21:52:53 +08:00
Lloyd-Pottiger
52ec268d75 format check 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-23 21:26:15 +08:00
alesapin
07286dd042 Add paranoid option to check blobs to S3 after upload 2022-08-23 15:12:09 +02:00
Lloyd-Pottiger
ac3cf0beb4 add read_only option 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-23 20:58:57 +08:00
Alexey Milovidov
34b5d267f3
Merge pull request #40461 from ClickHouse/dashboard 
Dashboard
 2022-08-23 15:34:28 +03:00
Kseniia Sumarokova
d814a56277
Merge pull request #40420 from kssenii/fix-cache-bug 
Fix incorrect fallback to skip cache in case of very high concurrency level
 2022-08-23 14:18:38 +02:00
alesapin
a646db4665 Better 2022-08-23 13:56:41 +02:00
avogar
29a887578b Fix 2022-08-23 11:42:57 +00:00
alesapin
eef80b0d06 Fix message 2022-08-23 13:23:53 +02:00
Maksim Buren
0827e87d45
delete miss log 2022-08-23 14:20:32 +03:00
alesapin
d97f3b012a Better 2022-08-23 13:20:05 +02:00
maks-buren630501
b2b97e53ea Fix bug whith wrong postgresql answer 2022-08-23 14:13:16 +03:00
avogar
581e569d04 Support reading Array(Record) into flatten nested table in Avro 2022-08-23 11:05:02 +00:00
alesapin
802d56df49 Merge branch 'master' into fix_bad_exception 2022-08-23 12:22:05 +02:00
Robert Schulze
635cc7617e
Merge pull request #40464 from ClickHouse/less-string-ref 
Reduce some usage of StringRef
 2022-08-23 12:16:00 +02:00
Kruglov Pavel
54ed9029b8
Merge pull request #40482 from ClickHouse/Avogar-patch-1 
Fix bug in HadoopSnappyReadBuffer
 2022-08-23 11:47:59 +02:00
Maksim Kita
2514ebc18f
Merge pull request #40502 from kitaisreal/dns-resolver-remove-hints 
DNSResolver remove AI_V4MAPPED, AI_ALL hints
 2022-08-23 11:41:09 +02:00
Kseniia Sumarokova
4711f57763
Update PostgreSQLReplicationHandler.cpp 2022-08-23 10:43:12 +02:00
Robert Schulze
60aa1d99f2
Merge pull request #38900 from Algunenano/avoid_printing_settings 
Avoid printing SETTINGS twice when formatting a query
 2022-08-23 09:28:01 +02:00
pzhdfy
d81462c095 check all ArgumentsDataTypes and use equals 2022-08-23 15:08:35 +08:00
Antonio Andelic
a87a762b0c
Merge pull request #39973 from ClickHouse/keeper-listen-host 
Support `interserver_listen_host` in Keeper
 2022-08-23 09:03:08 +02:00
Maksim Buren
9e54792359
Troubleshooting racing 
consumer_task use replication_handler_initialized in thread start in method assertInitialized(), by this sometimes assertInitialized throw error because replication_handler_initialized was still "false".
 2022-08-23 09:34:03 +03:00
Antonio Andelic
0765495be1
Merge pull request #39986 from Lloyd-Pottiger/feat/support-ttl-for-embeddedrocksdb 
Support TTL for EmbeddedRocksDB
 2022-08-23 08:27:47 +02:00
alesapin
61d014f6ec
Merge branch 'master' into fix_bad_exception 2022-08-23 00:35:29 +02:00
alesapin
780f39b108 Fix lost errors in AWS CPP SDK 2022-08-23 00:34:39 +02:00
Maksim Kita
11ba15dde6
Merge branch 'master' into dns-resolver-remove-hints 2022-08-22 22:23:35 +02:00
Alexey Milovidov
0252695c6e
Merge pull request #40224 from ClickHouse/alexey-milovidov-patch-4 
Check what will be if I enable `concurrency-mt-unsafe` in `clang-tidy`
 2022-08-22 23:12:01 +03:00
Alexey Milovidov
7c8e540dea Implement code review suggestions 2022-08-22 21:55:41 +02:00
Maksim Kita
191f39ae8d DNSResolver remove AI_V4MAPPED, AI_ALL hints 2022-08-22 21:03:25 +02:00
Robert Schulze
ec9ef741fc
Merge branch 'master' into less-string-ref 2022-08-22 20:48:16 +02:00
Nikita Taranov
b31342ec2c fix 2022-08-22 19:29:48 +02:00
Kruglov Pavel
3666b25e6b
Fix 2022-08-22 18:48:32 +02:00
Kruglov Pavel
2636c032c4
Use loop instead of recursion 2022-08-22 18:48:02 +02:00
Raúl Marín
f814908364 Improvements based on comments from the PR 2022-08-22 17:39:33 +02:00
Lloyd-Pottiger
85a1a36167 remove read_only option 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-22 23:31:55 +08:00
Kseniia Sumarokova
f85e5f8808
Merge branch 'master' into fix-cache-bug 2022-08-22 16:54:07 +02:00
zvonand
d789dabc9d Fixed , updated docs 2022-08-22 17:36:56 +03:00
zvonand
e5b4fb4191 fix some issues 2022-08-22 17:36:56 +03:00
zvonand
a7a1269e60 updated tests + improve logic 2022-08-22 17:36:55 +03:00
zvonand
537fb8c4ee fix 1 2022-08-22 17:36:55 +03:00
Raúl Marín
cda9138d88 Merge remote-tracking branch 'blessed/master' into avoid_printing_settings 2022-08-22 16:07:17 +02:00
Alexey Milovidov
a3e4753d83
Merge pull request #40466 from ClickHouse/remove-some-trash 
Remove some trash
 2022-08-22 15:59:50 +03:00
Alexander Gololobov
d370c45137
Merge pull request #40476 from ClickHouse/test-24093 
Add a test for #24093
 2022-08-22 14:37:25 +02:00
alesapin
b4d6afd5b3 Black 2022-08-22 13:51:48 +02:00
Maksim Kita
2b0c62ab80
Merge pull request #40353 from kitaisreal/proxy-resolver-stop-on-successful-request 
Proxy resolver stop on successful request
 2022-08-22 13:43:02 +02:00
Arthur Passos
f8e2ab0a20 Use FileReader::GetRecordBatchReader instead of FileReader::ReadRowGroup to parse Parquet 2022-08-22 08:21:32 -03:00
Kruglov Pavel
ae81341ab5
Fix hadoop unit test 2022-08-22 12:42:18 +02:00
Kruglov Pavel
019686f6e1
Fix bug in HadoopSnappyReadBuffer 2022-08-22 12:40:58 +02:00
Duc Canh Le
e0541f3f86 fix columnObject crash with array of variadic dimension elems 2022-08-22 18:34:53 +08:00
Azat Khuzhin
ec2e671d20 Remove ThreadStatus::untracked_memory_limit_increase 
It looks useless nowadays, because operator new cannot throw
MEMORY_LIMIT_EXCEEDED today, and so any code that works on Exception is
likely safe.

Refs: #40249
Refs: #24483
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-22 13:26:18 +03:00
kssenii
74087d57c4 Less notify 2022-08-22 12:00:14 +02:00
Robert Schulze
756b704f81
build: fix build 2022-08-22 08:55:53 +00:00
Antonio Andelic
78cbfa70ef Revert ZooKeeper changes 2022-08-22 06:25:13 +00:00
Alexey Milovidov
e9d0c50486 Add a test for #24093 2022-08-22 02:32:45 +02:00
Alexey Milovidov
6ae9e0f64f Fix error 2022-08-22 01:46:46 +02:00
Alexey Milovidov
6444afac52 Merge branch 'master' into dashboard 2022-08-22 01:30:21 +02:00
Alexey Milovidov
75c0ed528d Fix test 2022-08-22 01:21:32 +02:00
Alexey Milovidov
2faa23112c Fix test 2022-08-22 00:39:06 +02:00
Alexey Milovidov
c4a60b8123 Allow conversion from String with DateTime64 to Date and Date32 2022-08-22 00:23:10 +02:00
kssenii
26384b2543 Fix 2022-08-22 00:02:05 +02:00
Alexey Milovidov
bddf1cc312 Allow to parse Date as DateTime and DateTime64 2022-08-21 23:12:41 +02:00
Alexey Milovidov
996aa2d126 Remove some trash 2022-08-21 20:46:33 +02:00
Alexey Milovidov
7cd4619d9f
Merge pull request #34551 from ClickHouse/xdbc-constraints 
Allow constraints for ODBC and JDBC tables
 2022-08-21 21:35:06 +03:00
Alexey Milovidov
8dd5c215e0
Merge pull request #40462 from ClickHouse/revert-39000-avx-enablement 
Revert "Avx enablement"
 2022-08-21 21:32:45 +03:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning 2022-08-21 18:24:17 +00:00
Robert Schulze
4caf2c4c33
Reduce some usage of StringRef 
See #39535 and #39300
 2022-08-21 18:10:32 +00:00
Robert Schulze
3aa5acdb35
Merge pull request #40391 from ClickHouse/reduce-StringRef 
Reduce some usage of StringRef
 2022-08-21 20:04:57 +02:00
Antonio Andelic
af02c76a3c
Merge branch 'master' into keeper-listen-host 2022-08-21 18:43:55 +02:00
alesapin
354f4e90eb Remove redundant lines 2022-08-21 18:21:01 +02:00
alesapin
704d7fdc41 Fix copy and use disks 2022-08-21 18:18:35 +02:00
Alexey Milovidov
d157c6a7a0
Update StorageXDBC.cpp 2022-08-21 16:53:48 +03:00
Alexey Milovidov
6fbff6d1f6
Revert "Avx enablement" 2022-08-21 15:47:02 +03:00
FArthur-cmd
f59735d4f9 fix style 2022-08-21 12:45:35 +00:00
FArthur-cmd
fb7785c740 add support queries with parameters, add setting to avoid memory overflow 2022-08-21 11:07:03 +00:00
Alexey Milovidov
9635fa817b Embedded dashboard 2022-08-21 12:11:25 +02:00
alesapin
78ba732b31 Merge branch 'master' into data_source_description 2022-08-21 12:11:21 +02:00
Robert Schulze
bba04db0b4
Try to fix tests 2022-08-21 09:23:37 +00:00
FArthur-cmd
1e8910f048 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy-2 2022-08-21 08:45:03 +00:00
Alexey Milovidov
ab91c99495
Merge branch 'master' into control_block_size_in_aggregator 2022-08-20 21:28:27 +03:00
Alexey Milovidov
01700078d9
Merge branch 'master' into xdbc-constraints 2022-08-20 21:26:07 +03:00
Alexey Milovidov
0db45afb08
Merge branch 'master' into alexey-milovidov-patch-4 2022-08-20 19:58:31 +03:00
Alexey Milovidov
3649aac593 Fix style 2022-08-20 17:59:51 +02:00
alesapin
8fd3088459 Commit missed files 2022-08-20 17:21:03 +02:00
alesapin
e4d486fbba
Merge pull request #40346 from ClickHouse/fix_ttl_vertical_merge_bug 
Fix bug with multiple TTL merges on expired column
 2022-08-20 17:09:32 +02:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Kseniia Sumarokova
d335f264d9
Merge pull request #40413 from kssenii/minor-change 
FileCache.cpp tiny refactor
 2022-08-20 14:59:26 +02:00
Alexey Milovidov
6f39e3e039
Merge pull request #40419 from ClickHouse/fix-parallel-replicas 
Fix obvious trash in Parallel replicas
 2022-08-20 14:10:14 +03:00
Frank Chen
024fef1aec
Merge branch 'master' into tracing_context_propagation 2022-08-20 14:23:36 +08:00
Frank Chen
54a530a4e0 Make sure right toString is linked 2022-08-20 14:22:33 +08:00
Frank Chen
0569608192 Use toString 2022-08-20 12:30:39 +08:00
Sema Checherinda
1f8ce6d84e
Merge pull request #40292 from CheSema/fix-heap-overflow-httpchunkedreadbuffer 
fix heap buffer overflow by limiting http chunk size
 2022-08-20 01:01:29 +02:00
Maksim Kita
f80cd63198
Merge branch 'master' into proxy-resolver-stop-on-successful-request 2022-08-19 23:02:16 +02:00
Azat Khuzhin
28f2607d50 Fix rare race in updating peak of memory tracker 
Previously it was possible that allocation, that had been done with
locked MEMORY_LIMIT_EXCEEDED (LockMemoryExceptionInThread), will update
the peak to the amount that will include some other allocations.

Consider the following example (T1/T2 - different threads):

Initial state:

    MemoryTracker::amount = 0

Possible race:

    LockMemoryExceptionInThread locker;
T1: MemoryTracker::allocImpl(1MiB)      T2: MemoryTracker::allocImpl(1PiB)
                                            will_be = 1T, amount = 1T
    will_be = 1T+1MiB, amount=1T+1MiB
    peak = will_be
                                            amount = 0 // revert due to exceeding limit
                                            throw(MEMORY_LIMIT_EXCEEDED)

So as you can see T1 includes allocations from T2 into the peak.

Note, that almost every place that has memory_usage column uses
MemoryTracker::peak (not MemoryTracker::amount), this is true for
query_log (for part_log column name is peak_memory_usage, and obviously
it uses MemoryTracker::peak).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-19 20:57:12 +02:00
Nikita Mikhaylov
4a453f0776 Fix obvious trash 2022-08-19 18:08:13 +00:00
avogar
5ab87f1da4 Small refactoring 2022-08-19 16:42:23 +00:00
avogar
612ffaffde Make schema inference cache better, respect format settings that can change the schema 2022-08-19 16:39:13 +00:00
kssenii
e641664115 Refactor 2022-08-19 18:15:18 +02:00
alesapin
d8664c3227 Add shortcut for backups 2022-08-19 16:58:30 +02:00
Antonio Andelic
d07fd218a2 Fix initialization 2022-08-19 14:56:17 +00:00
Alexey Milovidov
2d1628439b
Merge pull request #40357 from ClickHouse/22.9-prepare 
Update version after release
 2022-08-19 15:51:30 +03:00
Duc Canh Le
24682556a4
Fix storage merge on view cannot use primary index (#40233) 
* fix storage merge on view cannot use index

* minor fix code + tests

* fix view stage

* fix stage

* fix style

* fix tests
 2022-08-19 14:24:22 +02:00
Antonio Andelic
eeb716d36b Remove unused error code 2022-08-19 12:03:02 +00:00
alesapin
7b460b5f85 Small refactoring 2022-08-19 13:31:25 +02:00
Antonio Andelic
ee3df77fda Simplify logic 2022-08-19 11:12:20 +00:00
Nikita Taranov
1b6e7b9ca2
Merge branch 'master' into sort_mode_rename 2022-08-19 12:31:59 +02:00
Robert Schulze
77e64935e1
Reduce some usage of StringRef 2022-08-19 09:56:59 +00:00
Antonio Andelic
5e2c8869a9 Merge branch 'master' into replicate-tables-readonly 2022-08-19 09:41:11 +00:00
Antonio Andelic
3f7d28cc6b Address PR comments 2022-08-19 09:39:59 +00:00
Sema Checherinda
f8480c26e7 do not trigger allocator to return null under sanitizers 2022-08-19 11:22:34 +02:00
Antonio Andelic
a8f3b62bf6 Use global zookeeper 2022-08-19 08:49:51 +00:00
Alexander Tokmakov
8cbdd39c81
Update ActionsDAG.cpp 2022-08-19 11:34:15 +03:00
Antonio Andelic
6f533313e9 Restart everything during attach 2022-08-19 08:17:02 +00:00
Alexander Tokmakov
a90d2ad35e
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-19 11:12:40 +03:00
Robert Schulze
24615059fb
Merge pull request #40319 from ClickHouse/custom-error-code-in-throwif 
Allow custom error code in throwIf
 2022-08-19 09:49:35 +02:00
Kruglov Pavel
2546773ffb
Merge pull request #40347 from ClickHouse/Avogar-patch-1 
Fix setting description
 2022-08-18 21:52:38 +02:00
Kruglov Pavel
b67cb9e378
Merge pull request #40173 from Avogar/arrow-dict 
Improve and fix dictionaries in Arrow format
 2022-08-18 20:54:55 +02:00
Alexander Tokmakov
3d253ec51b
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-18 21:23:50 +03:00
Alexander Tokmakov
b9e090699a
Merge branch 'master' into fix_incorrect_replica_idenifier 2022-08-18 21:18:10 +03:00
Alexander Tokmakov
e311d06eaa
Merge pull request #40246 from ClickHouse/parallel_drop_ranges_execution 
Allow parallel execution of disjoint drop ranges
 2022-08-18 21:17:41 +03:00
Alexey Milovidov
d62d3ad461 Update version to 22.9.1.1 2022-08-18 19:39:03 +02:00
Robert Schulze
5037ce547f
Only allow 8/16/32-bit signed integers as custom error code data type 
The internally used data type for error code is "int", and by
disallowing unsigned integers or integers >= 64 bit we avoid truncation
issues.
 2022-08-18 15:16:08 +00:00
Maksim Kita
61a465db19 Proxy resolver stop on successful request 2022-08-18 16:49:38 +02:00
Alexander Tokmakov
629690c32b fix "incorrect replica identifier" 2022-08-18 16:13:27 +02:00
Sema Checherinda
128e1fec3d Memory don't do alignment by itself, Allocator does 2022-08-18 16:01:30 +02:00
alesapin
a2e08299f8
Update src/Storages/MergeTree/IMergedBlockOutputStream.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-08-18 15:36:55 +02:00
Kseniia Sumarokova
3ae9420592
Merge pull request #40337 from yinpeiqi/fix-typo 
fix typo
 2022-08-18 15:00:56 +02:00