Commit Graph

49984 Commits

Author SHA1 Message Date
Azat Khuzhin
114ea9b1eb Fix accounting for new/delete from different threads for VariableContext::Thread
MemoryTracker assumes that for VariableContext::Thread new/delete may be
called from different threads, hence the amount of memory can go
negative.

However the MemoryTracker is nested, so even if the negative amount is
allowed for VariableContext::Thread it does not allowed for anything
upper, and hence the MemoryTracking will not be decremented properly.

Fix this, by passing initial size to the parent free.

This should fix memory drift for HTTP queries.
2020-10-23 21:07:52 +03:00
Azat Khuzhin
72d7b6117e Use total_memory_tracker when there is no other MemoryTracker object.
This should significantly reduce the MemoryTracking drift, test shows
that there is 0 drift after query storm (100 queries, via http/tcp/tcp
in one session).

TL;DR;

To track memory, clickhouse creates memory tracker object for each
thread **explicitly**, but until it is not created the memory
allocations are not under account.
There should not be lot of allocations w/o memory tracker, since most of
the time it is created early enough, but even this maybe enough to
trigger some problems.

Plus sometimes it is not possible to create it, for example some 3d
party library does not allow to do this explicitly:
- for example before #15740 allocations from librdkafka threads,
- or even worse, poco threads, they don't have any routines to do this.
This won't be a problem for `MemoryTracking` metric if the deallocation
will be done from the same thread w/o memory tracker (or vise versa),
but this is not always true.

NOTE, that this will slow down per-thread allocations w/o memory
tracker, since before this patch there were no memory tracking for them
while now they will be accounted in total_memory_tracker, and for
total_memory_tracker max_untracked_memory is always reached.
But this should not be significant.
2020-10-23 21:07:52 +03:00
alexey-milovidov
840d96255e
Merge pull request #16199 from azat/clang10-build-fix
Link dbms w/ atomic to fix undefined reference in unbundled build
2020-10-23 20:17:37 +03:00
alesapin
bc331a8029 Better set 2020-10-23 19:34:41 +03:00
alesapin
b28894c43b More compatible way 2020-10-23 19:31:51 +03:00
robot-clickhouse
f2b506785b Update version_date.tsv after release 20.10.2.20 2020-10-23 18:20:43 +03:00
sevirov
b8e754d68f
DOCSUP-3121: Update the SHOW DATABASES and SHOW TABLES descriptions (#16115)
* Update show.md

Update descriptions of show databases and show tables.

* Update show.md

Update 'identical query'.

* Update show.md

Внес поправки согласно комментариям.

* Update show.md

Внес поправки и перевел на русский язык.

* Update show.md

Внес мелкие поправки.

Co-authored-by: Dmitriy <sevirov@yandex-team.ru>
2020-10-23 18:11:02 +03:00
Aleksandr Karo
ba0513b031
Fix broken link in replication docs 2020-10-23 17:49:56 +03:00
Alexander Kuzmenkov
69b16fc8ef
Update tests.md 2020-10-23 17:39:09 +03:00
Alexander Kuzmenkov
77654eeee8 fixup 2020-10-23 17:35:03 +03:00
Alexander Kuzmenkov
b0a14a41c3 fixup 2020-10-23 17:28:55 +03:00
tavplubix
84b453f4be
Update InterpreterDropQuery.cpp 2020-10-23 17:19:02 +03:00
alesapin
f0ef7dd1e9 Change default to zero 2020-10-23 16:31:45 +03:00
alesapin
a73a908264 Optionally upload clickhouse binary 2020-10-23 16:31:00 +03:00
Alexander Tokmakov
337fe7b81f add comment 2020-10-23 15:53:58 +03:00
Nikita Mikhaylov
cb4945d8b0
Merge pull request #16205 from azat/do-not-cache-dictGet
Do not cache dictionary for dictGet*/dictHas*
2020-10-23 15:47:11 +03:00
Alexander Kuzmenkov
d17a49f127 Add more context to error messages 2020-10-23 15:20:07 +03:00
Alexander Kuzmenkov
28c9e66dc1
Merge pull request #15978 from ucasFL/interval-op
INTERVAL operator support string literal
2020-10-23 14:44:05 +03:00
sundy-li
f407504a7a try use cmake version for croaring instead of amalgamation.sh 2020-10-23 19:42:21 +08:00
alesapin
59ca0d071c Fix comment 2020-10-23 14:17:37 +03:00
alesapin
69f2f9838e Add ability to specify PRIMARY KEY without ORDER BY 2020-10-23 14:16:27 +03:00
Alexander Kuzmenkov
4476117ac6
Merge pull request #15144 from ClickHouse/aku/numa-perf
Perf test: bind server to one NUMA node
2020-10-23 14:10:35 +03:00
Alexander Kuzmenkov
acc5e3906a
Merge pull request #15476 from ClickHouse/aku/fasttest-debug
fix some tests for split build
2020-10-23 14:05:03 +03:00
alesapin
033525e27e
Merge pull request #16267 from den-crane/patch-6
Fix for incorrect error message
2020-10-23 10:43:34 +03:00
alexey-milovidov
f0064a3b09
Merge pull request #16223 from zhang2014/feature/add_mysql_binlog_check
Add MySQL BinLog file check tool
2020-10-23 09:22:17 +03:00
alexey-milovidov
0b002f393f
Merge pull request #16275 from ucasFL/docs
fix small document error
2020-10-23 09:19:20 +03:00
alexey-milovidov
121ebfdc81
Merge pull request #16249 from traceon/access-storage-added-log
Add a log message after an access storage is added
2020-10-23 09:17:10 +03:00
alexey-milovidov
6cd7d46f93
Update AccessControlManager.cpp 2020-10-23 09:13:26 +03:00
alexey-milovidov
40f6748880
Update adopters.md 2020-10-23 08:59:57 +03:00
feng lv
1a562ee329 fix small document error 2020-10-23 02:32:42 +00:00
Vitaliy Zakaznikov
e743d2fd26 * Fixing tests in the tests/testflows/ldap/external_user_directory/tests/authentications.py suite
* Moving TestFlows runs to use classic output format for stdout
* Moving to TestFlows 1.6.57
* Updating LDAP test code styling
2020-10-22 19:14:58 -04:00
alexey-milovidov
10d2070fbe
Merge pull request #16191 from ClickHouse/blind_fix_of_read_in_order
Blind performance fix of read_in_order_many_parts
2020-10-23 01:51:54 +03:00
Denny Crane
b310d0abb4
Fix for incorrect error message
max_[table/partition]_size_to_drop can be changed without restart since https://github.com/ClickHouse/ClickHouse/pull/7779
2020-10-22 15:22:17 -03:00
Alexander Kuzmenkov
a1a7bc0217
Merge pull request #16192 from filimonov/clickhouse-local-segfault
Add setTemporaryStorage to clickhouse-local to make OPTIMIZE work
2020-10-22 20:47:28 +03:00
Alexander Kuzmenkov
ef39def7ff
Apply suggestions from code review
Co-authored-by: filimonov <1549571+filimonov@users.noreply.github.com>
2020-10-22 20:46:17 +03:00
Alexander Kuzmenkov
b77e776db4 fixes 2020-10-22 20:30:50 +03:00
Denis Glazachev
a6439aba44 More specific log messages for each access storage type 2020-10-22 20:47:52 +04:00
Alexander Kuzmenkov
0275e9c010 work around docker weirdness 2020-10-22 19:42:31 +03:00
Amos Bird
ccf59c6412
Fix definitely wrong bug 2020-10-22 22:56:45 +08:00
alesapin
1d07ece5d1 Fix clang tidy warning 2020-10-22 17:15:56 +03:00
Alexander Kuzmenkov
ac7af67ede fixup 2020-10-22 15:52:43 +03:00
alesapin
4b65d5469f Fix some unrelated performance issues in select parts for merge 2020-10-22 15:41:01 +03:00
Mikhail Filimonov
77f66e5a09
Split test into 2 2020-10-22 11:02:14 +02:00
alesapin
945dd3aa59 Merge branch 'master' into blind_fix_of_read_in_order 2020-10-22 10:37:37 +03:00
Mikhail Filimonov
111b553ee5
Better 2020-10-22 09:37:03 +02:00
alexey-milovidov
0082c56b67
Merge pull request #16238 from ClickHouse/stateless_test_flaky_check
Changes in image for stateless test flaky check
2020-10-22 07:05:47 +03:00
alexey-milovidov
ecddc18648
Merge pull request #16239 from ClickHouse/fix_deb_image
Remove checksum check from arrow package
2020-10-22 07:05:08 +03:00
alexey-milovidov
d879b6a601
Merge pull request #16247 from vzakaznikov/fix_test_01446_json_strings_each_row
Fixing tests/queries/0_stateless/01446_json_strings_each_row.sql test
2020-10-22 07:03:51 +03:00
alexey-milovidov
f351b52851
Update AccessControlManager.cpp 2020-10-22 07:03:17 +03:00
alexey-milovidov
0433bc2034
Merge pull request #16248 from azat/nullptr_t-build-fix
Use std::nullptr_t over nullptr_t (build fix)
2020-10-22 07:02:38 +03:00