Azat Khuzhin
6403cbee46
Fix arrayDifference() for consts
2021-09-11 22:39:30 +03:00
Azat Khuzhin
3c75844a79
Fix SIGSEGV for arrayCompact() with consts
...
arrayCompact() implements useDefaultImplementationForConstants() but it
is a no-op for FunctionArrayMapped, fix this.
CI report [1]:
[1]: https://clickhouse-test-reports.s3.yandex.net/28890/baac2e561cfb05bc174f0bd25571e607d19c7463/fuzzer_asan/report.html#fail1
2021-09-11 22:39:20 +03:00
alexey-milovidov
648d561242
Merge pull request #28828 from azat/nested-combinators
...
Relax nested function for If-combinator check (but forbid nested identical combinators)
2021-09-11 15:53:19 +03:00
Kseniia Sumarokova
d420de6448
Merge pull request #28827 from azat/client-improvements
...
client: print full query in case of failures (test hints mismatch or server exception)
2021-09-11 15:33:16 +03:00
Maksim Kita
48e8e24552
Merge pull request #28882 from ClickHouse/async-metric-skip-some-exceptions
...
Skip "no data" exception when reading thermal sensors
2021-09-11 11:36:23 +03:00
Maksim Kita
aef6112af4
Merge pull request #28881 from azat/mysql-sync-thread-joinable
...
Join MaterializedMySQLSyncThread only if it is joinable
2021-09-11 11:35:43 +03:00
Vladimir C
aee967de3d
Merge pull request #28194 from ka1bi4/romanzhukov-DOCSUP-12543-bin-unbin-functions
2021-09-11 10:47:17 +03:00
Vladimir C
a581603964
Apply suggestions from code review
2021-09-11 10:46:55 +03:00
Alexey Milovidov
7e084304fb
Skip "no data" exception when reading thermal sensors
2021-09-11 10:03:06 +03:00
Azat Khuzhin
ff17de531c
Join MaterializedMySQLSyncThread only if it is joinable
...
It is possible to trigger
MaterializedMySQLSyncThread::stopSynchronization() from the same thread
in case of some exception at startup, when some interpreter holds the
storage refcnt, and later MaterializedMySQLSyncThread will try to join
itself from the main thread.
Here is a stack trace for example:
<details>
4 0x000000000f7ae45c in Poco::Event::wait (this=0x7f1b90c38170) at ../contrib/poco/Foundation/include/Poco/Event.h:97
5 ThreadFromGlobalPool::join (this=0x7f1b90c23120) at ../src/Common/ThreadPool.h:210
6 DB::MaterializeMySQLSyncThread::stopSynchronization (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:229
7 DB::MaterializeMySQLSyncThread::~MaterializeMySQLSyncThread (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:85
8 0x000000000f816dc8 in DB::DatabaseMaterializeMySQL<DB::DatabaseAtomic>::~DatabaseMaterializeMySQL (this=0x7f1b8024c918) at ../src/Databases/MySQL/DatabaseMaterializeMySQL.h:21
11 std::__1::shared_ptr<DB::IDatabase>::~shared_ptr (this=<optimized out>) at ../contrib/libcxx/include/memory:3212
12 0x000000000f8726a6 in DB::InterpreterCreateQuery::createTable (this=<optimized out>, create=...) at ../src/Interpreters/InterpreterCreateQuery.cpp:952
13 0x000000000f87735c in DB::InterpreterCreateQuery::execute (this=0x7f1aeef59860) at ../src/Interpreters/InterpreterCreateQuery.cpp:1225
14 0x000000000fe22253 in DB::executeQueryImpl (begin=<optimized out>, end=<optimized out>, context=..., internal=true, stage=DB::QueryProcessingStage::Complete, has_query_tail=<optimized out>, istr=<optimized out>) at ../src/Interpreters/executeQuery.cpp:574
15 0x000000000fe208e3 in DB::executeQuery (query=..., context=..., internal=<optimized out>, stage=DB::QueryProcessingStage::FetchColumns, may_have_embedded_data=<optimized out>) at ../src/Interpreters/executeQuery.cpp:933
16 0x000000000faafcde in DB::MySQLInterpreter::InterpreterMySQLDDLQuery<DB::MySQLInterpreter::InterpreterCreateImpl>::execute (this=<optimized out>) at ../src/Interpreters/MySQL/InterpretersMySQLDDLQuery.h:75
17 0x000000000faade78 in DB::InterpreterExternalDDLQuery::execute (this=<optimized out>) at ../src/Interpreters/InterpreterExternalDDLQuery.cpp:64
18 0x000000000fe22253 in DB::executeQueryImpl (begin=<optimized out>, end=<optimized out>, context=..., internal=true, stage=DB::QueryProcessingStage::Complete, has_query_tail=<optimized out>, istr=<optimized out>) at ../src/Interpreters/executeQuery.cpp:574
19 0x000000000fe208e3 in DB::executeQuery (query=..., context=..., internal=<optimized out>, stage=DB::QueryProcessingStage::FetchColumns, may_have_embedded_data=<optimized out>) at ../src/Interpreters/executeQuery.cpp:933
20 0x000000000f7ba406 in DB::tryToExecuteQuery (query_to_execute=..., query_context=..., database=..., comment=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:69
21 0x000000000f7d4b88 in DB::dumpDataForTables() (connection=..., query_prefix=..., database_name=..., mysql_database_name=..., context=..., is_cancelled=..., need_dumping_tables=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:326
22 DB::MaterializeMySQLSyncThread::prepareSynchronized(DB::MaterializeMetadata&)::$_1::operator()() const (this=<optimized out>) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:391
29 DB::commitMetadata() (function=..., persistent_tmp_path=..., persistent_path=...) at ../src/Databases/MySQL/MaterializeMetadata.cpp:197
30 0x000000000f80a000 in DB::MaterializeMetadata::transaction(DB::MySQLReplication::Position const&, std::__1::function<void ()> const&) (this=0x7f1b6375c3d8, position=..., fun=...) at ../src/Databases/MySQL/MaterializeMetadata.cpp:230
31 0x000000000f7b2231 in DB::MaterializeMySQLSyncThread::prepareSynchronized (this=0x7f1b8024ca68, metadata=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:388
32 0x000000000f7b169c in DB::MaterializeMySQLSyncThread::synchronization (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:180
33 0x000000000f7d4074 in DB::MaterializeMySQLSyncThread::startSynchronization()::$_0::operator()() const (this=<optimized out>) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:236
</details>
2021-09-11 09:27:12 +03:00
Azat Khuzhin
c6b9816d45
tests: mark 00763_lock_buffer as long
...
https://clickhouse-test-reports.s3.yandex.net/28827/58266e4ed6db77109e6237ecd2612d34c0b19516/functional_stateless_tests_flaky_check_(address).html#fail1
2021-09-11 09:15:38 +03:00
Azat Khuzhin
f7540dad56
tests: update 01305_replica_create_drop_zookeeper after client query printing
2021-09-11 09:13:23 +03:00
alexey-milovidov
28911a959f
Merge pull request #28431 from azat/thread-pool-joinable
...
[RFC] Do not allow ThreadFromGlobalPool::join() from the spawned/occupated thread
2021-09-11 05:44:56 +03:00
alexey-milovidov
d50335041d
Merge pull request #28874 from azat/heavy-build-heavy-system
...
Relax ENABLE_CHECK_HEAVY_BUILDS restrictions on systems under pressure (increase RLIMIT_CPU)
2021-09-11 05:29:32 +03:00
Azat Khuzhin
017dc9fa2a
Remove unused ILLEGAL_AGGREGATION from AggregateFunctionIf
2021-09-11 00:48:17 +03:00
Azat Khuzhin
58266e4ed6
tests/queries/0_stateless/01175_distributed_ddl_output_mode_long: use unique tmp file
2021-09-10 23:23:24 +03:00
Azat Khuzhin
4336038711
tests: update more references after client started printing query
2021-09-10 23:19:16 +03:00
Azat Khuzhin
c362e7c541
Introduce IAggregateFunctionCombinator::supportsNesting()
2021-09-10 22:41:38 +03:00
Azat Khuzhin
d8bf2f81ec
Allow Array/OrNull combinators regardless nesting
2021-09-10 22:35:25 +03:00
Azat Khuzhin
4e9b2c5d69
Relax ENABLE_CHECK_HEAVY_BUILDS restrictions on systems under pressure (increase RLIMIT_CPU)
...
The problem was with compiling src/AggregateFunctions/AggregateFunctionQuantile.cpp
2021-09-10 21:05:41 +03:00
Azat Khuzhin
1e7714d56d
Forbid nested identical (only) combinators
...
Nested identical combinators (i.e. uniqCombinedIfIf) is not
supported (since they even don't work -- silently).
But non-identical does supported and works, for example
uniqCombinedIfMergeIf, it is useful in case when the underlying
storage stores AggregateFunction(uniqCombinedIf) and in SELECT you
need to filter aggregation result based on another column for
example.
2021-09-10 21:03:24 +03:00
Azat Khuzhin
60df1e8800
Relax nested function for If-combinator check
...
Second If combinator can be useful to add condition based on another
column, when Merge combinator is applied for example (i.e.
uniqCombinedIfMergeIf(state_of_merge_that_is_stored_in_table, event_date = today())).
2021-09-10 21:03:24 +03:00
Azat Khuzhin
c4a24c7696
tests: update reference after client started printing query
2021-09-10 21:02:55 +03:00
Azat Khuzhin
09f43d21a8
client: print query on client exception only in non-interactive mode
2021-09-10 21:02:55 +03:00
Azat Khuzhin
f0f31ab8f3
client: print full query in case of server exception
2021-09-10 21:02:54 +03:00
Azat Khuzhin
af5072db50
client: print full query in case of test hints failures
2021-09-10 21:02:54 +03:00
Vladimir C
5b967d91ba
Merge pull request #28716 from olgarev/revolg-DOCSUP-13742-partitions_in_s3_table_function
2021-09-10 17:57:58 +03:00
Anton Popov
0bb74f8eaf
Merge pull request #28762 from CurtizJ/fix-nested-1
...
Fix usage of nested columns with non-array columns with the same prefix [2]
2021-09-10 17:34:43 +03:00
Maksim Kita
43102e8427
Merge pull request #28833 from kitaisreal/executable-added-option-send-chunk-header
...
Executable added option send_chunk_header
2021-09-10 14:44:34 +03:00
Maksim Kita
0b37de02e0
Fixed tests
2021-09-10 12:47:12 +03:00
Maksim Kita
ce739d1c32
Merge pull request #28834 from azat/llvm-shared-ci
...
Build w/ ENABLE_EMBEDDED_COMPILER in shared build on CI
2021-09-10 12:30:11 +03:00
Maksim Kita
4eaa24c498
Merge pull request #28846 from azat/fix-heavy-builds-check
...
Fix ENABLE_CHECK_HEAVY_BUILDS for UBsan build
2021-09-10 12:29:33 +03:00
Maksim Kita
2e535cf6fd
Merge pull request #28835 from kitaisreal/shell-command-source-fix-logging
...
ShellCommandSource fix logging
2021-09-10 12:27:19 +03:00
alesapin
6243b64ff5
Merge pull request #28656 from ClickHouse/move_docker_images_to_separate_repo
...
Move docker images to a separate repo
2021-09-10 10:24:21 +03:00
Vladimir C
27cd75eaa1
Apply suggestions from code review
2021-09-10 09:31:49 +03:00
Azat Khuzhin
1ec0295c6f
Increase RLIMIT_CPU for ENABLE_CHECK_HEAVY_BUILDS to fix UBsan build
...
Right now CI fails to compile UBsan while trying to compile src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o:
2021-09-09 23:02:28 FAILED: src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o
2021-09-09 23:02:28 prlimit --as=10000000000 --data=5000000000 --cpu=600 /usr/bin/ccache /usr/bin/clang++-12 -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=7 -DAWS_SDK_VERSION_PATCH=231 -DBOOST_ASIO_STANDALONE=1 -DCARES_STATICLIB -DENABLE_OPENSSL_ENCRYPTION -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DUSE_REPLXX=1 -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -DZ_TLS=__thread -D__CLANG_SUPPORT_DYN_ANNOTATION__ -I../contrib/sentry-native/include -Iincludes/configs -I../base/glibc-compatibility/memcpy -I../src -Isrc -Isrc/Core/include -I../base/common/.. -Ibase/common/.. -I../contrib/cityhash102/include -I../contrib/cctz/include -Icontrib/zlib-ng -I../contrib/zlib-ng -I../base/pcg-random/. -I../contrib/aws-c-common/include -I../contrib/aws-c-event-stream/include -Icontrib/aws-s3-cmake/include -I../base/mysqlxx/.. -I../contrib/mariadb-connector-c/include -Icontrib/mariadb-connector-c/include -Icontrib/grpc/third_party/cares/cares -I../contrib/grpc/third_party/cares/cares -I../contrib/abseil-cpp -I../contrib/libpq -I../contrib/libpq/include -I../contrib/libstemmer_c/include -I../contrib/wordnet-blast -I../contrib/lemmagen-c/include -isystem ../contrib/datasketches-cpp/common/include -isystem ../contrib/datasketches-cpp/theta/include -isystem ../contrib/libcxx/include -isystem ../contrib/libcxxabi/include -isystem ../contrib/libunwind/include -isystem ../contrib/libpqxx/include -isystem ../contrib/rocksdb/include -isystem ../contrib/orc/c++/include -isystem contrib/orc/c++/include -isystem ../contrib/AMQP-CPP/include -isystem ../contrib/libuv/include -isystem ../contrib/AMQP-CPP -isystem ../contrib/s2geometry/src/s2 -isystem ../contrib/libhdfs3/include -isystem ../contrib/miniselect/include -isystem ../contrib/pdqsort -isystem ../contrib/croaring/cpp -isystem ../contrib/croaring/include -isystem ../contrib/fast_float/include -isystem ../contrib/msgpack-c/include -isystem ../contrib/aws/aws-cpp-sdk-s3/include -isystem ../contrib/aws/aws-cpp-sdk-core/include -isystem ../contrib/xz/src/liblzma/api -isystem ../contrib/zstd/lib -isystem ../contrib/re2 -isystem ../contrib/boost -isystem ../contrib/poco/Net/include -isystem ../contrib/poco/Foundation/include -isystem ../contrib/poco/NetSSL_OpenSSL/include -isystem ../contrib/poco/Crypto/include -isystem ../contrib/boringssl/include -isystem ../contrib/poco/Util/include -isystem ../contrib/poco/JSON/include -isystem ../contrib/poco/XML/include -isystem ../contrib/replxx/include -isystem ../contrib/fmtlib-cmake/../fmtlib/include -isystem ../contrib/double-conversion -isystem ../contrib/dragonbox/include -isystem contrib/re2_st -isystem ../contrib/NuRaft/include -isystem ../contrib/grpc/include -isystem ../contrib/protobuf/src -isystem src/Server/grpc_protos -isystem ../contrib/s2geometry/src -isystem ../contrib/sqlite-amalgamation -isystem ../contrib/libc-headers/x86_64-linux-gnu -isystem ../contrib/libc-headers -g -O2 -fdebug-prefix-map=/build=. -specs=/usr/share/dpkg/no-pie-compile.specs -Wdate-time -g -fno-omit-frame-pointer -DSANITIZER -fsanitize=undefined -fno-sanitize-recover=all -fno-sanitize=float-divide-by-zero -fsanitize-blacklist=/build/tests/ubsan_suppressions.txt -fdiagnostics-color=always -Xclang -fuse-ctor-homing -fsized-deallocation -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -falign-functions=32 -Wall -Wno-unused-command-line-argument -fdiagnostics-absolute-paths -fexperimental-new-pass-manager -Werror -Wextra -Wpedantic -Wno-vla-extension -Wno-zero-length-array -Wno-c11-extensions -Wcomma -Wconditional-uninitialized -Wcovered-switch-default -Wdeprecated -Wembedded-directive -Wempty-init-stmt -Wextra-semi-stmt -Wextra-semi -Wgnu-case-range -Winconsistent-missing-destructor-override -Wnewline-eof -Wold-style-cast -Wrange-loop-analysis -Wredundant-parens -Wreserved-id-macro -Wshadow-field -Wshadow-uncaptured-local -Wshadow -Wstring-plus-int -Wundef -Wunreachable-code-return -Wunreachable-code -Wunused-exception-parameter -Wunused-macros -Wunused-member-function -Wzero-as-null-pointer-constant -Weverything -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c99-extensions -Wno-conversion -Wno-ctad-maybe-unsupported -Wno-deprecated-dynamic-exception-spec -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-nested-anon-types -Wno-packed -Wno-padded -Wno-return-std-move-in-c++11 -Wno-shift-sign-overflow -Wno-sign-conversion -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -O2 -g -DNDEBUG -O3 -fno-pie -D OS_LINUX -nostdinc++ -std=gnu++2a -MD -MT src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -MF src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o.d -o src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -c ../src/AggregateFunctions/AggregateFunctionSumMap.cpp
I guess the problem is CPU time, since looks like memory is fine (I
cannot reproduce it right now locally, maybe because I have clang-11).
2021-09-10 08:52:17 +03:00
alexey-milovidov
5b61e6275b
Merge pull request #28844 from athom/patch-1
...
Update where.md, change typo expluded to excluded
2021-09-10 08:46:27 +03:00
Azat Khuzhin
4d82e8918b
Update comments for ENABLE_CHECK_HEAVY_BUILDS
2021-09-10 08:39:56 +03:00
yeer
ac7330b805
Update where.md
...
type: expluded => excluded
2021-09-10 11:41:46 +08:00
Maksim Kita
21d33c3c2e
ShellCommandSource fix logging
2021-09-10 01:28:22 +03:00
Azat Khuzhin
a8c61f1d63
Build w/ ENABLE_EMBEDDED_COMPILER in shared build on CI
2021-09-10 01:10:57 +03:00
Maksim Kita
b1edf77904
Executable added option send_chunk_header
2021-09-10 01:04:52 +03:00
Maksim Kita
4f4cc9d740
Merge pull request #27994 from lehasm/alexey-sm-DOCSUP-13375-document-CREATE-DROP-FUNCTION
...
DOCSUP-13375: Document the SQL-statements CREATE FUNCTION and DROP FUNCTION
2021-09-10 00:55:43 +03:00
Maksim Kita
25a27978e6
Merge pull request #28818 from ClickHouse/remove_incorrect_code
...
Remove incorrect code and check what will happen
2021-09-10 00:49:36 +03:00
Maksim Kita
31766575e0
Merge pull request #28820 from azat/llvm-shared
...
Fix shared build with ENABLE_EMBEDDED_COMPILER
2021-09-10 00:49:18 +03:00
Maksim Kita
bdbf0c1967
Merge pull request #28821 from azat/compile-short-circuit-eval
...
Fix expressions compilation with short circuit evaluation
2021-09-10 00:49:00 +03:00
alexey-milovidov
2236a5df98
Merge pull request #28085 from myrrc/improvement/merge-tree-part-opt-parse
...
optional<> semantics for parsing MergeTreePartInfo and DetachedPartInfo
2021-09-10 00:04:03 +03:00
Dmitriy
62b15122ff
DOCSUP-13875: Document the replication_wait_for_inactive_replica_timeout setting ( #28464 )
...
* Add documentation of settings
Задокументировал две настройки.
* Fix links
Поправил ссылки.
* Update docs/en/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/alter/index.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/alter/index.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/optimize.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/truncate.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/truncate.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Translate to Russian
Выполнил перевод на русский язык.
* Fix links
Поправил ссылки.
* Update settings.md
Поправил перевод.
* Update docs/en/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/alter/index.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/optimize.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/sql-reference/statements/truncate.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/ru/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/ru/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Update docs/en/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
* Fix errors
Исправил неточности в описании.
* Update optimize.md
Поправил описание запроса OPTIMIZE.
Co-authored-by: Dmitriy <sevirov@yandex-team.ru>
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-09-09 23:33:08 +03:00
alesapin
2359a222ff
Fix more tests
2021-09-09 23:29:42 +03:00
Azat Khuzhin
63e14fc389
Fix expressions compilation with short circuit evaluation
...
Before this patch, you may get the following error:
Column Function is not a contiguous block of memory
Since under short circuit evaluation you may get Function not the
result.
2021-09-09 22:43:11 +03:00