Commit Graph

49853 Commits

Author SHA1 Message Date
Azat Khuzhin
6e5b04fad8 Make 01540_MemoryTracking integration
01540_MemoryTracking is failing on CI for the following reasons:
- log_queries (fixed, by adding log_queries=0)
- profilers (fixed)
- but what can't be fixed is metric_log and so on, so we need separate
  instance with separate configuration (sigh).
2020-10-24 02:27:29 +03:00
Azat Khuzhin
3f594ed3ad Add a test for memory drift in user memory tracker (max_memory_usage_for_user) 2020-10-23 22:13:34 +03:00
Azat Khuzhin
6c42ad564c Add a test for MemoryTracking drift
v2: disable query profiling and logging in 01540_MemoryTracking
    (This should make MemoryTracker drift zero).
2020-10-23 21:11:22 +03:00
Azat Khuzhin
0cccf3049a Fix parent memory tracker during query detaching 2020-10-23 21:07:53 +03:00
Azat Khuzhin
114ea9b1eb Fix accounting for new/delete from different threads for VariableContext::Thread
MemoryTracker assumes that for VariableContext::Thread new/delete may be
called from different threads, hence the amount of memory can go
negative.

However the MemoryTracker is nested, so even if the negative amount is
allowed for VariableContext::Thread it does not allowed for anything
upper, and hence the MemoryTracking will not be decremented properly.

Fix this, by passing initial size to the parent free.

This should fix memory drift for HTTP queries.
2020-10-23 21:07:52 +03:00
Azat Khuzhin
72d7b6117e Use total_memory_tracker when there is no other MemoryTracker object.
This should significantly reduce the MemoryTracking drift, test shows
that there is 0 drift after query storm (100 queries, via http/tcp/tcp
in one session).

TL;DR;

To track memory, clickhouse creates memory tracker object for each
thread **explicitly**, but until it is not created the memory
allocations are not under account.
There should not be lot of allocations w/o memory tracker, since most of
the time it is created early enough, but even this maybe enough to
trigger some problems.

Plus sometimes it is not possible to create it, for example some 3d
party library does not allow to do this explicitly:
- for example before #15740 allocations from librdkafka threads,
- or even worse, poco threads, they don't have any routines to do this.
This won't be a problem for `MemoryTracking` metric if the deallocation
will be done from the same thread w/o memory tracker (or vise versa),
but this is not always true.

NOTE, that this will slow down per-thread allocations w/o memory
tracker, since before this patch there were no memory tracking for them
while now they will be accounted in total_memory_tracker, and for
total_memory_tracker max_untracked_memory is always reached.
But this should not be significant.
2020-10-23 21:07:52 +03:00
alexey-milovidov
840d96255e
Merge pull request #16199 from azat/clang10-build-fix
Link dbms w/ atomic to fix undefined reference in unbundled build
2020-10-23 20:17:37 +03:00
robot-clickhouse
f2b506785b Update version_date.tsv after release 20.10.2.20 2020-10-23 18:20:43 +03:00
sevirov
b8e754d68f
DOCSUP-3121: Update the SHOW DATABASES and SHOW TABLES descriptions (#16115)
* Update show.md

Update descriptions of show databases and show tables.

* Update show.md

Update 'identical query'.

* Update show.md

Внес поправки согласно комментариям.

* Update show.md

Внес поправки и перевел на русский язык.

* Update show.md

Внес мелкие поправки.

Co-authored-by: Dmitriy <sevirov@yandex-team.ru>
2020-10-23 18:11:02 +03:00
Alexander Kuzmenkov
69b16fc8ef
Update tests.md 2020-10-23 17:39:09 +03:00
Nikita Mikhaylov
cb4945d8b0
Merge pull request #16205 from azat/do-not-cache-dictGet
Do not cache dictionary for dictGet*/dictHas*
2020-10-23 15:47:11 +03:00
Alexander Kuzmenkov
28c9e66dc1
Merge pull request #15978 from ucasFL/interval-op
INTERVAL operator support string literal
2020-10-23 14:44:05 +03:00
Alexander Kuzmenkov
4476117ac6
Merge pull request #15144 from ClickHouse/aku/numa-perf
Perf test: bind server to one NUMA node
2020-10-23 14:10:35 +03:00
Alexander Kuzmenkov
acc5e3906a
Merge pull request #15476 from ClickHouse/aku/fasttest-debug
fix some tests for split build
2020-10-23 14:05:03 +03:00
alesapin
033525e27e
Merge pull request #16267 from den-crane/patch-6
Fix for incorrect error message
2020-10-23 10:43:34 +03:00
alexey-milovidov
f0064a3b09
Merge pull request #16223 from zhang2014/feature/add_mysql_binlog_check
Add MySQL BinLog file check tool
2020-10-23 09:22:17 +03:00
alexey-milovidov
0b002f393f
Merge pull request #16275 from ucasFL/docs
fix small document error
2020-10-23 09:19:20 +03:00
alexey-milovidov
121ebfdc81
Merge pull request #16249 from traceon/access-storage-added-log
Add a log message after an access storage is added
2020-10-23 09:17:10 +03:00
alexey-milovidov
6cd7d46f93
Update AccessControlManager.cpp 2020-10-23 09:13:26 +03:00
alexey-milovidov
40f6748880
Update adopters.md 2020-10-23 08:59:57 +03:00
feng lv
1a562ee329 fix small document error 2020-10-23 02:32:42 +00:00
Vitaliy Zakaznikov
e743d2fd26 * Fixing tests in the tests/testflows/ldap/external_user_directory/tests/authentications.py suite
* Moving TestFlows runs to use classic output format for stdout
* Moving to TestFlows 1.6.57
* Updating LDAP test code styling
2020-10-22 19:14:58 -04:00
alexey-milovidov
10d2070fbe
Merge pull request #16191 from ClickHouse/blind_fix_of_read_in_order
Blind performance fix of read_in_order_many_parts
2020-10-23 01:51:54 +03:00
Denny Crane
b310d0abb4
Fix for incorrect error message
max_[table/partition]_size_to_drop can be changed without restart since https://github.com/ClickHouse/ClickHouse/pull/7779
2020-10-22 15:22:17 -03:00
Alexander Kuzmenkov
a1a7bc0217
Merge pull request #16192 from filimonov/clickhouse-local-segfault
Add setTemporaryStorage to clickhouse-local to make OPTIMIZE work
2020-10-22 20:47:28 +03:00
Alexander Kuzmenkov
ef39def7ff
Apply suggestions from code review
Co-authored-by: filimonov <1549571+filimonov@users.noreply.github.com>
2020-10-22 20:46:17 +03:00
Alexander Kuzmenkov
b77e776db4 fixes 2020-10-22 20:30:50 +03:00
Denis Glazachev
a6439aba44 More specific log messages for each access storage type 2020-10-22 20:47:52 +04:00
Alexander Kuzmenkov
0275e9c010 work around docker weirdness 2020-10-22 19:42:31 +03:00
alesapin
1d07ece5d1 Fix clang tidy warning 2020-10-22 17:15:56 +03:00
Alexander Kuzmenkov
ac7af67ede fixup 2020-10-22 15:52:43 +03:00
alesapin
4b65d5469f Fix some unrelated performance issues in select parts for merge 2020-10-22 15:41:01 +03:00
Mikhail Filimonov
77f66e5a09
Split test into 2 2020-10-22 11:02:14 +02:00
alesapin
945dd3aa59 Merge branch 'master' into blind_fix_of_read_in_order 2020-10-22 10:37:37 +03:00
Mikhail Filimonov
111b553ee5
Better 2020-10-22 09:37:03 +02:00
alexey-milovidov
0082c56b67
Merge pull request #16238 from ClickHouse/stateless_test_flaky_check
Changes in image for stateless test flaky check
2020-10-22 07:05:47 +03:00
alexey-milovidov
ecddc18648
Merge pull request #16239 from ClickHouse/fix_deb_image
Remove checksum check from arrow package
2020-10-22 07:05:08 +03:00
alexey-milovidov
d879b6a601
Merge pull request #16247 from vzakaznikov/fix_test_01446_json_strings_each_row
Fixing tests/queries/0_stateless/01446_json_strings_each_row.sql test
2020-10-22 07:03:51 +03:00
alexey-milovidov
f351b52851
Update AccessControlManager.cpp 2020-10-22 07:03:17 +03:00
alexey-milovidov
0433bc2034
Merge pull request #16248 from azat/nullptr_t-build-fix
Use std::nullptr_t over nullptr_t (build fix)
2020-10-22 07:02:38 +03:00
alexey-milovidov
adeba6bdd8
Merge pull request #15074 from amosbird/btc
Extend trivial count optimization.
2020-10-22 02:50:57 +03:00
Denis Glazachev
691b28e98b Add a log message after a storage is added 2020-10-22 02:43:02 +04:00
Azat Khuzhin
6f78243e32 Use std::nullptr_t over nullptr_t 2020-10-22 01:21:06 +03:00
alexey-milovidov
cfaf82ba0c
Merge pull request #16135 from hexiaoting/dev_rankCorr
Fix inconsistent behaviour of rankCorr function
2020-10-22 01:15:50 +03:00
alexey-milovidov
222a4d2e9b
Update AggregateFunctionRankCorrelation.h 2020-10-22 01:15:24 +03:00
alexey-milovidov
2ae32e3d5c
Merge pull request #16101 from azat/http-memory-tracking-fixes
Fix memory tracker settings (for HTTP queries)
2020-10-22 00:00:47 +03:00
Mikhail Filimonov
67cbb55d63
Fix compilation 2020-10-21 22:16:13 +02:00
alesapin
369365bdb0 Fix run.sh 2020-10-21 23:11:35 +03:00
alesapin
6a57bc8c0b
Merge pull request #16215 from ClickHouse/simplify_ttl_move_test
Simplify test_concurrent_alter_with_ttl_move
2020-10-21 22:48:04 +03:00
tavplubix
3c53d47840
Merge pull request #16189 from ClickHouse/fix_assertion_table_function_file
Fix assertion in table function file()
2020-10-21 22:38:24 +03:00