Commit Graph

10704 Commits

Author SHA1 Message Date
Maksim Kita
3f6050cee9 Fixed error code include 2021-02-22 23:32:43 +03:00
Maksim Kita
8520fc3e4c Replace null fields in tuple during parsing with default values 2021-02-22 23:32:43 +03:00
alexey-milovidov
cb8cfdba77
Merge branch 'master' into fffix 2021-02-22 23:08:44 +03:00
Azat Khuzhin
89c311981d Fix buffer size for trace collection
[1]: https://clickhouse-test-reports.s3.yandex.net/0/a860e3e7582f489a640c42db3bd4bd320f97a5bc/stress_test_(debug).html#fail1
2021-02-22 22:45:04 +03:00
jennyma
6433ab3da6 fix default_replica_path and default_replica_name values are useless on Replicated(*)MergeTree engine when the engine needs specify other parameters 2021-02-22 18:34:01 +08:00
alexey-milovidov
073ee6c2c3
Merge pull request #21053 from ClickHouse/more-consistent-datetime64-parsing
More consistent DateTime64 parsing
2021-02-22 13:13:21 +03:00
alexey-milovidov
8b82c7ea0c
Merge pull request #21050 from azat/datetime64-float-fix
Fix DateTime64 from Float
2021-02-22 13:12:09 +03:00
alexey-milovidov
50639b7dba
Merge pull request #21052 from ClickHouse/file-engine-settings
Merging #20620
2021-02-22 13:10:31 +03:00
alesapin
1aa0c0bff0 More stable test 2021-02-21 23:36:05 +03:00
Alexey Milovidov
952cee9ec7 Merge remote-tracking branch 'azat/datetime64-float-fix' into more-consistent-datetime64-parsing 2021-02-21 23:24:38 +03:00
Alexey Milovidov
f7b7561552 Merge branch 'master' into more-consistent-datetime64-parsing 2021-02-21 23:24:07 +03:00
Alexey Milovidov
46ac365629 Merge branch 'master' into topvisor/file-engine-settings 2021-02-21 23:10:26 +03:00
Alexey Milovidov
0a62f95bbe Merge branch 'master' into distributed_aggregation_memory_efficient_by_default 2021-02-21 23:06:49 +03:00
Alexey Milovidov
d7f017c4dd Improve performance of trivial count query in presense of "distributed_aggregation_memory_efficient" 2021-02-21 23:06:44 +03:00
Alexey Milovidov
0ab14120ef Improve performance of trivial count query in presense of "distributed_aggregation_memory_efficient" 2021-02-21 23:06:31 +03:00
Azat Khuzhin
521537eed8 Fix DateTime64 from Float 2021-02-21 22:36:40 +03:00
alexey-milovidov
2ac5c4b71b
Merge pull request #20996 from azat/system-stack-trace-diagnostics
Add log message when stacktrace cannot be obtained for thread
2021-02-21 22:18:13 +03:00
alesapin
f8a581daba Remove possibly redundant allocations 2021-02-21 20:42:19 +03:00
alexey-milovidov
125bf1d0d9
Merge pull request #20479 from ucasFL/tr
fix transform with floating point key
2021-02-21 17:37:20 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias
Normalized function names (lower-case case-insensitive names)
2021-02-21 16:23:51 +03:00
Alexey Milovidov
4085782b05 More gentle exception messages 2021-02-21 15:34:48 +03:00
Alexey Milovidov
af2f4ceb0e Add disambiguation while parsing already scaled DateTime64 #13194 2021-02-21 15:34:04 +03:00
Alexey Milovidov
00e6b6232f Revert "optimize aggfunc column data copy (#19407)"
This reverts commit 7e3186c826.
2021-02-21 12:49:50 +03:00
feng lv
5d3b8d99e7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into tr 2021-02-21 07:34:09 +00:00
Alexey Milovidov
b736f4fdee Suppression for PVS-Studio (looks like a false positive) 2021-02-21 10:21:59 +03:00
feng lv
a33183ee1e fix transform with floating point key
better

update test
2021-02-21 05:20:42 +00:00
alesapin
331de3d653 Trying to add tests more stable 2021-02-21 00:08:19 +03:00
alesapin
48d5d75282 Make unit test more stable 2021-02-20 23:47:18 +03:00
Vitaly Baranov
2eecaee08d Better tests for protobuf format. 2021-02-20 23:13:32 +03:00
Alexey Milovidov
64e76a4a8d Minor changes in Decimal 2021-02-20 21:13:36 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated
DatabaseReplicated: continuation of #10485
2021-02-20 19:39:34 +03:00
Anton Popov
48e188681c do not start mutation for alters with wrong type conversion 2021-02-20 19:11:08 +03:00
alesapin
9f520f42c1 Fix style 2021-02-20 18:56:55 +03:00
alesapin
0c2cf3cf30 Calculate checksum with siphash 2021-02-20 18:36:56 +03:00
vdimir
6cc2fb5e9f
Try to fix race in storage join: block parralel inserts 2021-02-20 18:00:59 +03:00
alexey-milovidov
316cdf5039
Merge pull request #20978 from ClickHouse/little-better
Just little better
2021-02-20 17:53:07 +03:00
alexey-milovidov
e336abad1c
Merge pull request #20981 from azat/http-client-reset-uncaught-exception
Fix uncaught exception when HTTP client goes away
2021-02-20 16:30:04 +03:00
Alexander Tokmakov
863c099254 fix 2021-02-20 15:57:07 +03:00
alesapin
e7f792c94d Fix typos 2021-02-20 14:28:39 +03:00
alexey-milovidov
04f5b72391
Merge pull request #20995 from amosbird/ubsanfix2
Suppress signed overflow in AggregateFunctionGroupArrayMoving 2
2021-02-20 14:25:36 +03:00
alesapin
89dd15a91d Checksum for header and logging, better names 2021-02-20 14:04:38 +03:00
Maksim Kita
c5643c47d0
Merge pull request #20970 from kitaisreal/function-sum-map-decimal-fix
Function sumMap decimal fix
2021-02-20 13:43:12 +03:00
kssenii
a38a31c954 Fix typos check 2021-02-20 09:58:24 +00:00
Amos Bird
f37631830f
Comments 2021-02-20 16:45:25 +08:00
Alexey Milovidov
09b4c04c1e Merge branch 'master' into little-better 2021-02-20 11:27:40 +03:00
Nikolai Kochetov
2f67cc02c1
Merge pull request #20881 from azat/async-task-queue-wait-fix
Fix 'Empty task was returned from async task queue' on query cancellation
2021-02-20 10:56:04 +03:00
alexey-milovidov
9c48fcfdef
Merge pull request #20882 from azat/distributed_group_by_no_merge-improvements
Do only merging of sorted blocks on initiator with distributed_group_by_no_merge=2
2021-02-20 10:36:32 +03:00
alesapin
c61769311e Merge branch 'master' into persistent_nukeeper_log_storage 2021-02-20 10:22:22 +03:00
Azat Khuzhin
1ccb333ac5 Fix WriteBufferFromHTTPServerResponse usage in other places (add missing finalize())
Since I saw the following:

    0. DB::WriteBufferFromOStream::nextImpl()
    1. DB::WriteBufferFromHTTPServerResponse::nextImpl()
    2. DB::WriteBufferFromHTTPServerResponse::finalize()
    3. DB::WriteBufferFromHTTPServerResponse::~WriteBufferFromHTTPServerResponse()
    4. DB::StaticRequestHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)
    5. Poco::Net::HTTPServerConnection::run()
    6. Poco::Net::TCPServerConnection::start()
2021-02-20 10:15:38 +03:00
kssenii
f820047cc8 Fix 2021-02-20 06:52:10 +00:00
alexey-milovidov
b4196c8e68
Merge pull request #20168 from ClickHouse/in-memory-compression
Compression for Memory tables
2021-02-20 09:49:37 +03:00
Alexey Milovidov
1eb6f4743d Merge branch 'master' into distributed_aggregation_memory_efficient_by_default 2021-02-20 09:40:31 +03:00
Azat Khuzhin
7c04f15c80 Add log message when stacktrace cannot be obtained for thread
This is to provide better diagnostics for 01051_system_stack_trace
failure [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/20881/866dfaec793f764dc9ba167d3ac9f6521b9b3381/functional_stateless_tests_(release,_wide_parts_enabled).html#fail1
2021-02-20 09:13:00 +03:00
Amos Bird
487fb09ff6
Suppress signed overflow in AggregateFunctionGroupArrayMoving 2 2021-02-20 14:11:01 +08:00
Alexander Tokmakov
2a36d6cb55 review suggestions 2021-02-20 02:41:58 +03:00
M0r64n
2f7d0ba926 Replace direct truncate with O_TRUNC flag 2021-02-20 03:27:23 +04:00
kssenii
21ee685ef0 Fix brotly 2021-02-19 22:18:14 +00:00
Maksim Kita
131fdb4507
Merge pull request #20958 from ClickHouse/fix-eintr-for-epoll-wait
Check for EINTR in epoll_wait
2021-02-20 00:28:19 +03:00
Azat Khuzhin
657bb33e73 Merge remote-tracking branch 'upstream/master' into async-task-queue-wait-fix
Merge upstream/master to debug possible uncaught exceptions with gdb
diagnostics merged.
2021-02-20 00:01:21 +03:00
Alexey Milovidov
0f77b6fd95 Even more better 2021-02-19 22:01:45 +03:00
Azat Khuzhin
fba1c7fcc1 Fix uncaught exception when HTTP client goes away
Even after #20464 it was still possible, for example [1].

    2021.02.19 11:40:21.886191 [ 68373 ] {} <Trace> DynamicQueryHandler: Request URI: /?database=test_ds2d6y&log_comment=/usr/share/clickhouse-test/queries/0_stateless/01302_aggregate_state_exception_memory_leak.sh&enable_http_compression=1&http_zlib_compression_level=1

    <snip>

    2021.02.19 11:41:35.289940 [ 365 ] {} <Fatal> BaseDaemon: (version 21.3.1.6058, build id: 8D46D65205E2C8B7FE408A0B4EC76CA0483F9E92) (from thread 68373) Terminate called for uncaught exception:
    Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 262568, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x15b3c7db in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:56: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8aba66e in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:0: DB::WriteBufferFromOStream::nextImpl() @ 0x8b8c105 in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x8b8c537 in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:44: DB::Write

  [1]: https://clickhouse-test-reports.s3.yandex.net/16481/5d150cce4778dd14f58dcff67435bdec1efa155b/stress_test_(thread).html#fail1

And according to this partial stacktrace it seems that the dtor of
WriteBufferFromOStream was called from
WriteBufferFromHTTPServerResponse, since the class name starts from
DB::Write*

The problem is that if first time WriteBufferFromOStream::next() fails,
it will reset position to make next write no-op, however
WriteBufferFromHTTPServerResponse::next() will set position to available
buffer back, and next() will throw again, but this time it can be from
dtor.
2021-02-19 22:00:57 +03:00
Alexey Milovidov
252bcccdda Just little better 2021-02-19 21:32:39 +03:00
Maksim Kita
fc03c1013c
Fixed style check 2021-02-19 20:42:51 +03:00
alexey-milovidov
2299930c31
Merge pull request #20545 from kssenii/rabbit-format-settings
rabbitmq: add missing format factory settings
2021-02-19 20:24:17 +03:00
alexey-milovidov
91de5628f0
Merge pull request #20866 from CurtizJ/fix-distinct-state
Fix usage of 'distinct' combinator with 'state' combinator
2021-02-19 20:22:53 +03:00
Maksim Kita
0b5213c80d
Added comment 2021-02-19 19:49:19 +03:00
tavplubix
3509fe88d8
Merge pull request #20878 from ClickHouse/fix_18926
Fix COM_FIELD_LIST response in MySQL protocol
2021-02-19 19:25:54 +03:00
Maksim Kita
b84112a603 Function sumMap decimal fix 2021-02-19 19:25:50 +03:00
alesapin
ad374ec095 Rename file 2021-02-19 19:05:26 +03:00
kssenii
30f0e48777 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-postgres-numeric 2021-02-19 14:53:27 +00:00
alesapin
12d05c2792 Better startup 2021-02-19 17:47:53 +03:00
alesapin
068c9cfbf7 Fix logs level 2021-02-19 17:13:29 +03:00
alesapin
5c9420c077 More correct epoll usage 2021-02-19 15:56:51 +03:00
Ivan
414f470c79
Make Poco HTTP Server zero-copy again (#19516)
* Refactoring: part 1

* Refactoring: part 2

* Handle request using ReadBuffer interface

* Struggles with ReadBuffer's

* Fix URI parsing

* Implement parsing of multipart/form-data

* Check HTTP_LENGTH_REQUIRED before eof() or will hang

* Fix HTTPChunkedReadBuffer

* Fix build and style

* Fix test

* Resist double-eof

* Fix arcadian build
2021-02-19 15:51:26 +03:00
Maksim Kita
71c65b5d82
Merge pull request #20611 from kitaisreal/rename-untyped-function-reinterpret-as
Rename untyped function reinterpretAs into reinterpret
2021-02-19 14:35:00 +03:00
alesapin
3d954c4314 Better request/response logic 2021-02-19 14:06:17 +03:00
alesapin
6c9322bb2e Sane constant while reading requests 2021-02-19 12:57:39 +03:00
alesapin
86a74ca6b5 Fix size deserialization 2021-02-19 12:24:50 +03:00
Nikolai Kochetov
ed4697cffc Fix timeout in epoll_wait for RemoteQueryExecutorReadContext 2021-02-19 12:20:24 +03:00
Nikolai Kochetov
d438d7e390 Fix timeout in epoll_wait for PollingQueue 2021-02-19 12:07:34 +03:00
Alexey Milovidov
39f07d62a4 Disable in-memory compression by default 2021-02-19 12:02:18 +03:00
Alexey Milovidov
a9d9a6d56f Merge branch 'master' into in-memory-compression 2021-02-19 12:01:33 +03:00
Nikolai Kochetov
7d1119680e Remove not needed file. 2021-02-19 11:52:33 +03:00
Nikolai Kochetov
b9d6df9618 Check for eintr in epoll_wait 2021-02-19 11:49:41 +03:00
alesapin
b72b13bab0 Better list performance 2021-02-19 10:25:55 +03:00
alesapin
88a6d4e206 Revert "Revert "Better list requests""
This reverts commit 839d6f7072.
2021-02-19 10:05:52 +03:00
Amos Bird
fc185e5fb7
Another try 2021-02-19 11:56:24 +08:00
alesapin
839d6f7072 Revert "Better list requests"
This reverts commit ee98b2a472.
2021-02-19 00:09:41 +03:00
alesapin
ee98b2a472 Better list requests 2021-02-19 00:03:16 +03:00
Azat Khuzhin
9c01869090 Fix 'Empty task was returned from async task queue' on query cancellation 2021-02-18 22:15:21 +03:00
Azat Khuzhin
af660140c3 Do only merging of sorted blocks on initiator with distributed_group_by_no_merge=2
When distributed_group_by_no_merge=2 is used (or when
optimize_distributed_group_by_sharding_key takes place), remote servers
will do full ORDER BY, so initiator can skip this step and do only merge
of ordered blocks.
2021-02-18 21:43:27 +03:00
Azat Khuzhin
b854a7b7f8 Add some details into comment for first_stage/second_stage
Regardless
distributed_group_by_no_merge=2/optimize_distributed_group_by_sharding_key
2021-02-18 21:41:50 +03:00
alesapin
0336764426 Fix tidy one more time 2021-02-18 18:51:16 +03:00
Alexander Tokmakov
8097532cb7 Merge branch 'master' into database_replicated 2021-02-18 17:42:06 +03:00
Vitaly Baranov
556dc81ab9 Fix undefined-behavior in ReservoirSamplerDeterministic.h 2021-02-18 16:32:01 +03:00
Alexander Tokmakov
97f4c457ec fix MySQL COMM_FIELD_LIST response 2021-02-18 16:27:51 +03:00
alesapin
1245710aed Merge branch 'master' into persistent_nukeeper_log_storage 2021-02-18 15:27:01 +03:00
Amos Bird
2c4bc43014
Backward compatible 2021-02-18 20:20:29 +08:00
Amos Bird
f402aa4057
Normalize constant expression 2021-02-18 20:20:28 +08:00
Amos Bird
2dc7ba160a
Better 2021-02-18 20:20:28 +08:00
Amos Bird
77fd060665
Normalize function names 2021-02-18 20:20:28 +08:00
kssenii
2aad067e7c Support conversion for postgres numeric without precision and scale 2021-02-18 12:16:58 +00:00
zlx19950903
c92e613b82
Add a function htmlOrXmlCoarseParse to extract content from html or xml format string. (#19600)
* add html and xml coarse parse

* add test file

* add conditional check: hyperscan

* fix style error

* add conditional check

* bug fix

* delete unit

* typos check fix

* add unit test

* style check fix

* fix build error: case style

* acradis_skip test fix

* LINT error fix

* Remove comments

Co-authored-by: guojiantao <guojiantao15@mails.ucas.ac.cn>
Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
Co-authored-by: Ivan Lezhankin <ilezhankin@yandex-team.ru>
2021-02-18 15:05:55 +03:00
alesapin
904b4754cc Fix tidy 2021-02-18 14:47:37 +03:00
alesapin
7f815325ba More tests for broken changelog read 2021-02-18 14:42:09 +03:00
alesapin
5b597fdf44 Force sync setting and ability to start with broken log 2021-02-18 13:23:48 +03:00
Maksim Kita
4278098f9a Reinterpret function added Decimal, DateTim64 support 2021-02-18 13:09:01 +03:00
Maksim Kita
dcabbc95ed
Merge pull request #20623 from kitaisreal/dictionary-create-source-with-functions-fix
Dictionary create source with functions crash fix
2021-02-18 11:50:20 +03:00
Vitaly Baranov
3cbb325913
Merge pull request #20506 from vitlibar/refactor-protobuf-format-io
Improved serialization in Protobuf format.
2021-02-18 11:31:37 +03:00
Alexey Milovidov
04cb91a0fd Fix error 2021-02-18 06:02:41 +03:00
Alexey Milovidov
5007f7f018 Fix typo 2021-02-18 05:57:15 +03:00
Alexey Milovidov
634be2b933 Fix error 2021-02-18 03:52:09 +03:00
Alexey Milovidov
b7011f4f9c Fix build 2021-02-18 02:52:07 +03:00
Anton Popov
dcba99f4b1 fix usage of 'distinct' combinator with 'state' combinator 2021-02-18 02:19:58 +03:00
Alexey Milovidov
1781a64370 Whitespaces 2021-02-18 02:11:41 +03:00
Alexey Milovidov
9b72255ca4 Implement compression for all columns except LowCardinality 2021-02-18 02:09:46 +03:00
Alexey Milovidov
9069318fa6 Merge branch 'master' into in-memory-compression 2021-02-18 00:48:39 +03:00
alesapin
ff663dc511 Fsync at server shutdown 2021-02-17 23:36:25 +03:00
alesapin
6e244e7bb1 Trying without fsync 2021-02-17 22:32:38 +03:00
alexey-milovidov
80e6ad7aae
Merge pull request #20610 from ucasFL/fix-union-distinct-limit
fix subquery with limit
2021-02-17 21:24:10 +03:00
alexey-milovidov
3891dd6284
Update InterpreterSelectQuery.cpp 2021-02-17 21:23:27 +03:00
Vitaly Baranov
18e036d19b Improved serialization for data types combined of Arrays and Tuples.
Improved matching enum data types to protobuf enum type.
Fixed serialization of the Map data type.
Omitted values are now set by default.
2021-02-17 20:50:09 +03:00
Maksim Kita
50e135db0f
Added comment 2021-02-17 19:24:04 +03:00
Kseniia Sumarokova
20ff183549
Merge branch 'master' into rabbit-format-settings 2021-02-17 19:01:53 +03:00
alesapin
2de201dd5f Merge branch 'fix_fast_test_retry' into persistent_nukeeper_log_storage 2021-02-17 17:51:50 +03:00
Nikita Mikhaylov
d15bac31ff
Merge pull request #20399 from nikitamikhaylov/keen-wolf-filefuncwithreadbuf
Merging #19204
2021-02-17 16:11:22 +03:00
Maksim Kita
b2c09f002f Dictionary create source with functions crash fix 2021-02-17 15:26:00 +03:00
alesapin
0de1535b23
Merge pull request #20596 from kssenii/rabbitmq-add-vhost
Support vhost for rabbitmq engine
2021-02-17 15:18:22 +03:00
alesapin
9396bae2e2 More reliable test keeper tests 2021-02-17 14:53:47 +03:00
Vladimir
2c99a36228
Merge pull request #19685 from hexiaoting/dev_joinon 2021-02-17 13:22:55 +03:00
Maksim Kita
c608fa1e6a Added error reinterpretation tests 2021-02-17 12:53:12 +03:00
Vitaly Baranov
753f5b84d9
Merge pull request #20106 from vitlibar/fix-quota-consuming
Fix quota consuming
2021-02-17 12:52:43 +03:00
alesapin
acf843a01a Slightly more optimal 2021-02-17 12:00:12 +03:00
feng lv
3b40099578 fix subquery with limit 2021-02-17 08:26:52 +00:00
Maksim Kita
5f88f5817f Rename untyped function reinterpretAs into reinterpret 2021-02-17 11:23:24 +03:00
alesapin
af95db2fcf Test log storage instead of changelog 2021-02-17 11:00:17 +03:00
alesapin
dfaa79b88e Add missed file 2021-02-17 10:10:46 +03:00
Alexey Milovidov
fa20016091 Enable distributed_aggregation_memory_efficient by default 2021-02-17 09:53:18 +03:00
alexey-milovidov
6363d44488
Merge pull request #20498 from azat/dist-parse-dirs-sigsegv-fix
Add proper checks while parsing directory names for async INSERT (fixes SIGSEGV)
2021-02-17 09:43:11 +03:00
kssenii
d79f7c04f6 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-format-settings 2021-02-17 06:03:19 +00:00
kssenii
29362bb483 Support vhost 2021-02-16 22:05:14 +00:00
Alexander Tokmakov
1aac7b3471 Merge branch 'master' into database_replicated 2021-02-17 00:39:56 +03:00
Nikita Mikhaylov
5f8a6ab9c1 remove probably useless code 2021-02-16 23:54:05 +03:00
Vitaly Baranov
d357fb9129 Fix reading from the table system.quota_usage. 2021-02-16 23:54:05 +03:00
Vitaly Baranov
298130402e SYSTEM queries now consume quota. 2021-02-16 23:54:05 +03:00
Vitaly Baranov
d8d2bd885c Fix calculation of interval's end in quota consumption. 2021-02-16 23:54:05 +03:00
Vitaly Baranov
f83be158ba SHOW TABLES is now considered as one query in the quota calculations, not two queries. 2021-02-16 23:53:58 +03:00
alesapin
d3e87701d4 Persistent storage 2021-02-16 22:02:18 +03:00
alesapin
b76b8013ba Fix tests 2021-02-16 20:47:12 +03:00
alesapin
b029f3e5cf Border test 2021-02-16 20:32:35 +03:00