Commit Graph

72617 Commits

Author SHA1 Message Date
Alexey Milovidov
8935a70f60 Fix another misoptimization 2021-09-12 01:41:37 +03:00
Alexey Milovidov
d85588210d Fix bad optimization of ORDER BY in subquery if it contains WITH FILL 2021-09-12 01:24:20 +03:00
Alexey Milovidov
526c7253d0 Fix wrong optimization 2021-09-12 01:15:37 +03:00
Kseniia Sumarokova
6cc4ad97c3
Update WriteBufferToRabbitMQProducer.h 2021-09-12 00:35:18 +03:00
Azat Khuzhin
c99bd06195 copier: fix partition_key alias using WITH
After those changes it is not possible to add an alias for the tuple
element anymore, i.e.:

  $ clickhouse-local -q 'select ((1,2) = (1,2) as foo), foo'
  1       1
  $ clickhouse-upstream local -q 'select ((1,2) = ((1,2) as foo)), foo'
  1       (1,2)

But these can be done using WITH statement.
2021-09-12 00:15:31 +03:00
Anton Popov
02a7edf902 allow to include subccolumns in describe query 2021-09-12 00:07:33 +03:00
Raúl Marín
9fb8eeaea3 Use project types 2021-09-11 22:54:30 +02:00
Anton Popov
a4153e5629
Merge pull request #28853 from ClickHouse/fix_logical_race_on_dictionaries_loading
Fix logical race on dictionaries loading
2021-09-11 23:41:30 +03:00
Raúl Marín
50360aa586 Add extra performance tests for Nullable floating point Sum
The existing ones aren't that representative since there isn't
any NULL values, so the branch predictor is correct 100% of the time
2021-09-11 22:37:45 +02:00
Raúl Marín
940e075941 Remove notNull changes 2021-09-11 22:36:22 +02:00
Azat Khuzhin
748d5a86d2 Add some comments for systemd service. 2021-09-11 23:15:38 +03:00
Azat Khuzhin
5a0e8af483 Use Should-* instead of Required-* for init script 2021-09-11 23:00:07 +03:00
Azat Khuzhin
c7e5cd7cda Fix arrayCumSumNonNegative() for consts 2021-09-11 22:39:30 +03:00
Azat Khuzhin
6403cbee46 Fix arrayDifference() for consts 2021-09-11 22:39:30 +03:00
Azat Khuzhin
3c75844a79 Fix SIGSEGV for arrayCompact() with consts
arrayCompact() implements useDefaultImplementationForConstants() but it
is a no-op for FunctionArrayMapped, fix this.

CI report [1]:

  [1]: https://clickhouse-test-reports.s3.yandex.net/28890/baac2e561cfb05bc174f0bd25571e607d19c7463/fuzzer_asan/report.html#fail1
2021-09-11 22:39:20 +03:00
Azat Khuzhin
2f34ab1717 tests: update 00900_long_parquet_load.reference 2021-09-11 21:59:42 +03:00
Raúl Marín
46eb305eec T != Value 2021-09-11 20:46:16 +02:00
kssenii
453bd27c0a Review fixes 2021-09-11 20:40:18 +03:00
Mike Kot
3401b32e37 Adapted usage of is_decimal concept 2021-09-11 19:35:12 +02:00
mergify[bot]
267e56df8c
Merge branch 'master' into fix-web-disk-2 2021-09-11 15:45:43 +00:00
Sergei Semin
d2c678928c remove cases with unspecified behaviour (out of UInt64 range) 2021-09-11 18:14:00 +03:00
Sergei Semin
b7358636c1 remove duplicated libraries 2021-09-11 18:10:18 +03:00
mergify[bot]
97fbdec71c
Merge branch 'master' into fix-build-clang-13-2 2021-09-11 15:03:52 +00:00
Raúl Marín
a13b6f2a09 WIP: Speed up sum(floating point) 2021-09-11 15:37:52 +02:00
Azat Khuzhin
5f05ac23b0 Add $time dependency for clickhouse-server.init script 2021-09-11 16:08:41 +03:00
Azat Khuzhin
aa5f1d77fa Add After/Wants time-sync.target for the clickhouse-server.service 2021-09-11 16:08:41 +03:00
Azat Khuzhin
baac2e561c Add query to the exception message in case of error during processing INSERT block on client
Since client process the INSERT block itself, and only after, send it
to the client, for example:

    clickhouse-client --stacktrace --input_format_null_as_default=1 --query="INSERT INTO FUNCTION null('k Int, v Tuple(Int,Int)') VALUES ()"
    Code: 62. DB::Exception: Cannot parse expression of type Int32 here: ): While executing ValuesBlockInputFormat: (in query: INSERT INTO FUNCTION null('k Int, v Tuple(Int,Int)') VALUES ()): data for INSERT was parsed from query. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below):
2021-09-11 16:02:01 +03:00
alexey-milovidov
648d561242
Merge pull request #28828 from azat/nested-combinators
Relax nested function for If-combinator check (but forbid nested identical combinators)
2021-09-11 15:53:19 +03:00
Azat Khuzhin
976874d57b tests/01016_input_null_as_default: remove one more tuple nesting
CI report [1]:

    01016_input_null_as_default:                                            [ FAIL ] - having stderror:
    Code: 53. DB::Exception: Bad size of tuple. Expected size: 1, actual size: 2.: While executing ValuesBlockInputFormat: data for INSERT was parsed from stdin. (TYPE_MISMATCH)

  [1]: https://clickhouse-test-reports.s3.yandex.net/28888/5ad9cb9b797abe498ae558f9c233e16cd89e41b4/fast_test.html#fail1
2021-09-11 15:38:30 +03:00
Kseniia Sumarokova
d420de6448
Merge pull request #28827 from azat/client-improvements
client: print full query in case of failures (test hints mismatch or server exception)
2021-09-11 15:33:16 +03:00
Azat Khuzhin
b51c180922 tests/00132_sets: remove one more tuple nesting
CI report [1]:

    2021-09-11 14:25:31 [5656ccb14ed9] 2021.09.11 14:25:31.498685 [ 18561 ] {6096a73f-2066-4c78-9f99-73f013c96deb} <Error> executeQuery: Code: 43. DB::Exception: Invalid types for IN function: Tuple(UInt8, Tuple(UInt8, UInt8)) and Tuple(Tuple(Tuple(UInt8, Tuple(UInt8, UInt8)), Tuple(UInt8, Tuple(UInt8, UInt8)))).: While processing (1 AS number, (2, 3) AS tuple) IN tuple(((1, (2, 3)), (4, (5, 6)))). (ILLEGAL_TYPE_OF_ARGUMENT) (version 21.11.1.1) (from [::1]:32780) (comment: 00132_sets.sql) (in query: SELECT number, tuple FROM (SELECT 1 AS number, (2, 3) AS tuple) WHERE (number, tuple) IN (((1, (2, 3)), (4, (5, 6))));), Stack trace (when copying this message, always include the lines below):

  [1]: https://clickhouse-test-reports.s3.yandex.net/28888/5ad9cb9b797abe498ae558f9c233e16cd89e41b4/fast_test.html#fail1
2021-09-11 15:29:43 +03:00
Azat Khuzhin
2f1a58f4b0 Fix waiting for mutation with mutations_sync=2
CI: https://clickhouse-test-reports.s3.yandex.net/28827/c6b9816d45e3e8255eb1aee9b9059dfe7397fdab/functional_stateless_tests_(thread).html#fail1
2021-09-11 15:23:00 +03:00
Azat Khuzhin
5ad9cb9b79 Add test_storage_mysql::test_mysql_in with multiple columns in IN 2021-09-11 14:15:27 +03:00
Azat Khuzhin
f40ab891eb Support tuple with zero/one arguments in transformQueryForExternalDatabase() 2021-09-11 14:15:27 +03:00
Azat Khuzhin
e4e584a3de Do not transform (k,v) IN ((K, V)) -> (k, v) IN (K, V) during parsing
Before this patch the query `(k,v) IN ((1, 2))` was transformed `(k,v)
IN (1,2)` by ClickHouse, and this produce the following error while
sending such query to MySQL:

    DB::Exception: mysqlxx::BadQuery: Operand should contain 2 column(s)

(There is a workaround though, add another value for IN section, i.e.
`(k,v) IN ((1,2),(0,0))`, but this is icky)

v0: (k,v) IN ((1,2)) -> (k,v) IN tuple((1,2))
    But these is not supported by transformQueryForExternalDatabase()

v1: (k,v) IN ((1,2)) -> (k,v) IN ((1,2))
    But now you will get an error for the following query:

        SELECT ('foo', 'bar') IN (('foo', 'bar'))

    Now you will get:

        Logical error: 'Trying to get name of not a column: ExpressionList'.

    (from two places, first is DB::ActionsMatcher::visit, easy to fix by
     inherit ASTExpressionList from ASTWithAlias, but the second is
     evaluteConstantExpression(), which  requires additional support for
     these case while creating a set in DB::createBlockForSet())

v2: (k,v) IN ((1,2)) -> (k,v) IN tuple((1,2))
    So instead I prefer these, and add support for this case in
    transformQueryForExternalDatabase() in the next patch.
2021-09-11 14:15:27 +03:00
Azat Khuzhin
0c2f612a86 gtest_transform_query_for_external_database: add test with multiple columns in IN 2021-09-11 14:15:27 +03:00
Azat Khuzhin
ef161f10f7 gtest_transform_query_for_external_database: print input query 2021-09-11 14:15:27 +03:00
Azat Khuzhin
c77c4e22ae Add MySQL query into after transformation in case of BadQuery
Since it may differs from the user input
2021-09-11 14:06:19 +03:00
Pavel Kruglov
9c756fc4b2 Fix bug with LowCardinality in short-curcuit function evaluation 2021-09-11 12:56:25 +03:00
Maksim Kita
48e8e24552
Merge pull request #28882 from ClickHouse/async-metric-skip-some-exceptions
Skip "no data" exception when reading thermal sensors
2021-09-11 11:36:23 +03:00
Maksim Kita
aef6112af4
Merge pull request #28881 from azat/mysql-sync-thread-joinable
Join MaterializedMySQLSyncThread only if it is joinable
2021-09-11 11:35:43 +03:00
Vladimir C
aee967de3d
Merge pull request #28194 from ka1bi4/romanzhukov-DOCSUP-12543-bin-unbin-functions 2021-09-11 10:47:17 +03:00
Vladimir C
a581603964
Apply suggestions from code review 2021-09-11 10:46:55 +03:00
Artur
c77a65f9d5 avoid error with order 2021-09-11 07:28:54 +00:00
Alexey Milovidov
7e084304fb Skip "no data" exception when reading thermal sensors 2021-09-11 10:03:06 +03:00
Azat Khuzhin
ff17de531c Join MaterializedMySQLSyncThread only if it is joinable
It is possible to trigger
MaterializedMySQLSyncThread::stopSynchronization() from the same thread
in case of some exception at startup, when some interpreter holds the
storage refcnt, and later MaterializedMySQLSyncThread will try to join
itself from the main thread.

Here is a stack trace for example:

<details>

    4  0x000000000f7ae45c in Poco::Event::wait (this=0x7f1b90c38170) at ../contrib/poco/Foundation/include/Poco/Event.h:97
    5  ThreadFromGlobalPool::join (this=0x7f1b90c23120) at ../src/Common/ThreadPool.h:210
    6  DB::MaterializeMySQLSyncThread::stopSynchronization (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:229
    7  DB::MaterializeMySQLSyncThread::~MaterializeMySQLSyncThread (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:85
    8  0x000000000f816dc8 in DB::DatabaseMaterializeMySQL<DB::DatabaseAtomic>::~DatabaseMaterializeMySQL (this=0x7f1b8024c918) at ../src/Databases/MySQL/DatabaseMaterializeMySQL.h:21
    11 std::__1::shared_ptr<DB::IDatabase>::~shared_ptr (this=<optimized out>) at ../contrib/libcxx/include/memory:3212
    12 0x000000000f8726a6 in DB::InterpreterCreateQuery::createTable (this=<optimized out>, create=...) at ../src/Interpreters/InterpreterCreateQuery.cpp:952
    13 0x000000000f87735c in DB::InterpreterCreateQuery::execute (this=0x7f1aeef59860) at ../src/Interpreters/InterpreterCreateQuery.cpp:1225
    14 0x000000000fe22253 in DB::executeQueryImpl (begin=<optimized out>, end=<optimized out>, context=..., internal=true, stage=DB::QueryProcessingStage::Complete, has_query_tail=<optimized out>, istr=<optimized out>) at ../src/Interpreters/executeQuery.cpp:574
    15 0x000000000fe208e3 in DB::executeQuery (query=..., context=..., internal=<optimized out>, stage=DB::QueryProcessingStage::FetchColumns, may_have_embedded_data=<optimized out>) at ../src/Interpreters/executeQuery.cpp:933
    16 0x000000000faafcde in DB::MySQLInterpreter::InterpreterMySQLDDLQuery<DB::MySQLInterpreter::InterpreterCreateImpl>::execute (this=<optimized out>) at ../src/Interpreters/MySQL/InterpretersMySQLDDLQuery.h:75
    17 0x000000000faade78 in DB::InterpreterExternalDDLQuery::execute (this=<optimized out>) at ../src/Interpreters/InterpreterExternalDDLQuery.cpp:64
    18 0x000000000fe22253 in DB::executeQueryImpl (begin=<optimized out>, end=<optimized out>, context=..., internal=true, stage=DB::QueryProcessingStage::Complete, has_query_tail=<optimized out>, istr=<optimized out>) at ../src/Interpreters/executeQuery.cpp:574
    19 0x000000000fe208e3 in DB::executeQuery (query=..., context=..., internal=<optimized out>, stage=DB::QueryProcessingStage::FetchColumns, may_have_embedded_data=<optimized out>) at ../src/Interpreters/executeQuery.cpp:933
    20 0x000000000f7ba406 in DB::tryToExecuteQuery (query_to_execute=..., query_context=..., database=..., comment=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:69
    21 0x000000000f7d4b88 in DB::dumpDataForTables() (connection=..., query_prefix=..., database_name=..., mysql_database_name=..., context=..., is_cancelled=..., need_dumping_tables=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:326
    22 DB::MaterializeMySQLSyncThread::prepareSynchronized(DB::MaterializeMetadata&)::$_1::operator()() const (this=<optimized out>) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:391
    29 DB::commitMetadata() (function=..., persistent_tmp_path=..., persistent_path=...) at ../src/Databases/MySQL/MaterializeMetadata.cpp:197
    30 0x000000000f80a000 in DB::MaterializeMetadata::transaction(DB::MySQLReplication::Position const&, std::__1::function<void ()> const&) (this=0x7f1b6375c3d8, position=..., fun=...) at ../src/Databases/MySQL/MaterializeMetadata.cpp:230
    31 0x000000000f7b2231 in DB::MaterializeMySQLSyncThread::prepareSynchronized (this=0x7f1b8024ca68, metadata=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:388
    32 0x000000000f7b169c in DB::MaterializeMySQLSyncThread::synchronization (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:180
    33 0x000000000f7d4074 in DB::MaterializeMySQLSyncThread::startSynchronization()::$_0::operator()() const (this=<optimized out>) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:236

</details>
2021-09-11 09:27:12 +03:00
Azat Khuzhin
c6b9816d45 tests: mark 00763_lock_buffer as long
https://clickhouse-test-reports.s3.yandex.net/28827/58266e4ed6db77109e6237ecd2612d34c0b19516/functional_stateless_tests_flaky_check_(address).html#fail1
2021-09-11 09:15:38 +03:00
Azat Khuzhin
f7540dad56 tests: update 01305_replica_create_drop_zookeeper after client query printing 2021-09-11 09:13:23 +03:00
alexey-milovidov
28911a959f
Merge pull request #28431 from azat/thread-pool-joinable
[RFC] Do not allow ThreadFromGlobalPool::join() from the spawned/occupated thread
2021-09-11 05:44:56 +03:00
alexey-milovidov
d50335041d
Merge pull request #28874 from azat/heavy-build-heavy-system
Relax ENABLE_CHECK_HEAVY_BUILDS restrictions on systems under pressure (increase RLIMIT_CPU)
2021-09-11 05:29:32 +03:00