Commit Graph

1582 Commits

Author SHA1 Message Date
Nikolai Kochetov
d03bcebc8e Remove debug logging. 2021-07-23 12:05:42 +03:00
Nikolai Kochetov
3c17a62686
Merge pull request #26590 from ClickHouse/remove-some-more-streams
Remove some streams.
2021-07-22 21:28:50 +03:00
Nikolai Kochetov
f56a45155f Merge branch 'master' into remove-more-and--more-streams 2021-07-22 19:10:39 +03:00
Nikolai Kochetov
fd754430eb Remove more streams. 2021-07-22 19:05:52 +03:00
Nikolai Kochetov
3ed3f7a9f7 Fix integration tests. 2021-07-22 13:38:22 +03:00
Nikolai Kochetov
5ffd99dfd4
Merge pull request #25674 from amosbird/distributedreturnconnection
Drain connection asynchronously
2021-07-22 11:04:49 +03:00
Azat Khuzhin
3f0dd40c69 Avoid hanging clickhouse-benchmark if connection fails (i.e. on EMFILE) 2021-07-22 10:08:49 +03:00
Nikolai Kochetov
f38de35b14 Rename some constants. 2021-07-21 19:13:17 +03:00
Maksim Kita
49983883cc Added tests 2021-07-21 16:41:45 +03:00
Nikolai Kochetov
2c90928bc4 Fix tests. 2021-07-21 12:52:28 +03:00
Maksim Kita
63f67f7997 Introduced HereDoc token 2021-07-21 11:59:05 +03:00
Azat Khuzhin
1860808969 Add round-robin support for clickhouse-benchmark 2021-07-20 23:00:04 +03:00
Nikolai Kochetov
179ec05a72 Remove some streams. 2021-07-20 21:18:43 +03:00
Alexey Milovidov
2c3c329514 Fix issue in #26282 2021-07-20 03:15:06 +03:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
Vitaly Baranov
4f1926550b
Merge pull request #26429 from vitlibar/remove-mysql-wire-context
Remove MySQLWireContext
2021-07-19 12:21:24 +03:00
Alexey Milovidov
e87a4cfbda Whitespace 2021-07-17 02:29:57 +03:00
alexey-milovidov
38dfe1fc0a
Merge pull request #26172 from ClickHouse/exception_id
Add error id to exceptions
2021-07-17 02:07:17 +03:00
alexey-milovidov
c9a917a163
Merge pull request #26282 from FArthur-cmd/adding-system-warnings-26039
Show warnings
2021-07-17 01:53:57 +03:00
Vitaly Baranov
0f8b196682 Remove MySQLWireContext. 2021-07-16 22:21:20 +03:00
vdimir
70de8dbf74
Add integrity for loaded scripts in play.html 2021-07-16 14:39:56 +03:00
Artur
c0a34e2842 Adding interactive test for warning messages during connection 2021-07-15 12:53:11 +00:00
Alexey Milovidov
18c985a7d7 Save server address in history URLs in Play UI 2021-07-14 21:16:16 +03:00
Artur
5582c4736e correct code-style 2021-07-13 18:05:16 +00:00
Artur
3b61682265 add new config to skip warnings 2021-07-13 17:50:48 +00:00
Alexander Tokmakov
f0ed6be269 Merge branch 'master' into system_zookeeper_log 2021-07-13 20:49:37 +03:00
Artur
ffb0c832bc correct type mistake 2021-07-13 13:35:08 +00:00
Artur
ed9d3be1cc Add ignoring exception for system.warnings, while connecting 2021-07-13 12:39:49 +00:00
Artur
d377c069cb Adding printing warnings when client connect to server 2021-07-13 12:20:28 +00:00
Artur
1fe4f084f3 Merge branch 'adding-system-warnings-26039' of https://github.com/FArthur-cmd/ClickHouse into adding-system-warnings-26039 2021-07-13 11:14:14 +00:00
Artur
d0b69c08ac show warnings to client, when it is connected 2021-07-13 11:13:42 +00:00
alexey-milovidov
d28b143c49
Merge pull request #26246 from FArthur-cmd/adding-system-warnings-26039
Adding system warnings 26039
2021-07-13 03:44:53 +03:00
Artur
1e90287e9a Adding tests for system.warnings 2021-07-12 10:57:39 +00:00
Alexey Milovidov
3887e82dac Merge branch 'exception_id' of github.com:yandex/ClickHouse into exception_id 2021-07-12 03:18:39 +03:00
Alexey Milovidov
c4f34689ec Merge branch 'master' into exception_id 2021-07-11 22:12:06 +03:00
Vladimir
3157a46596
Fix special build, minor fix in Client.cpp 2021-07-10 13:28:44 +03:00
Alexey Milovidov
c63237f8e4 Merge branch 'master' into exception_id 2021-07-10 11:14:46 +03:00
alexey-milovidov
0686155b0e
Merge pull request #26128 from kssenii/fix-ch-local-progress
Fix progress for clickhouse-local
2021-07-10 11:10:46 +03:00
Alexey Milovidov
1f91c65aca Add error id to exceptions 2021-07-10 05:43:53 +03:00
alexey-milovidov
f43f99f6f5
Merge pull request #26152 from CurtizJ/client-color
Change color in client for double colon
2021-07-10 02:46:46 +03:00
Anton Popov
3588c5ad17 add color for double colon 2021-07-09 23:32:14 +03:00
Artur
8113580237 Add warning in debug mode 2021-07-09 14:40:32 +00:00
Alexander Tokmakov
4165ba2a01 add system.zookeeper_log 2021-07-09 17:05:35 +03:00
kssenii
947bb4a942 Fix progress bar for local 2021-07-09 16:43:00 +03:00
alexey-milovidov
818bbd6539
Update programs/server/play.html
Co-authored-by: Vladimir <vdimir@yandex-team.ru>
2021-07-08 17:06:37 +03:00
Alexey Milovidov
15dbb5a07a Final touch 2021-07-08 06:04:35 +03:00
Alexey Milovidov
15b75be59b Add comment 2021-07-08 05:29:21 +03:00
Alexey Milovidov
e841fae852 Improvement 2021-07-08 05:27:40 +03:00
Alexey Milovidov
aa84d6a913 Render pipelines in Play UI 2021-07-08 05:20:24 +03:00
Alexey Milovidov
d437ec2e1d Merge branch 'master' into system-metrics 2021-07-05 03:03:07 +03:00
alexey-milovidov
fef7c557f6
Merge pull request #24789 from kitaisreal/compile-aggregation-functions
Compile aggregate functions
2021-07-05 02:57:49 +03:00
Alexey Milovidov
c059d0a0ee More metrics 2021-07-05 01:41:09 +03:00
Alexey Milovidov
0bc58b7056 Better color 2021-07-04 06:09:44 +03:00
Alexey Milovidov
1a83c45f24 Automatically create links in Web UI 2021-07-04 06:04:42 +03:00
Alexey Milovidov
7a993404b4 Whitespace 2021-07-02 02:30:18 +03:00
Maksim Kita
da8c957167 Aggregator added CompiledExpressionCache 2021-07-01 22:56:36 +03:00
alesapin
cdc95fa98a
Merge pull request #25428 from ClickHouse/zookeeper_snapshots
Tool for conversion of ZooKeeper data into clickhouse-keeper snapshot
2021-06-29 12:58:45 +03:00
alexey-milovidov
e67a54bec3
Merge pull request #25770 from azat/fuzzer-deep-fix
Ignore TOO_DEEP_RECURSION server exception during fuzzing
2021-06-29 01:27:21 +03:00
Kseniia Sumarokova
9d02af7d8b
Merge pull request #25676 from sand6255/MaterializeMySQL-Support-Enum-Data-Type
MaterializeMySQL: support ENUM data type
2021-06-28 19:15:00 +03:00
mergify[bot]
04814114e7
Merge branch 'master' into zookeeper_snapshots 2021-06-28 14:17:32 +00:00
Azat Khuzhin
170301c531 Ignore TOO_DEEP_RECURSION server exception during fuzzing 2021-06-28 10:37:49 +03:00
Storozhuk Kostiantyn
4a3145f586 Materialize my sql support enum data type
* Implemented Enum for MaterializeMySQL
2021-06-25 16:42:25 +08:00
Azat Khuzhin
7739fcc295 clickhouse-client: fix NULL dereference for --param w/o value 2021-06-24 00:08:18 +03:00
alexey-milovidov
7a0cddcb4c
Merge pull request #25569 from oxidecomputer/master
Resolves actual port bound when user requests any available
2021-06-23 22:47:01 +03:00
Kseniia Sumarokova
2e422c0e5c
Merge pull request #25551 from kssenii/add-test
Add test for progress bar
2021-06-23 12:33:06 +03:00
mergify[bot]
fddf47a900
Merge branch 'master' into zookeeper_snapshots 2021-06-23 08:36:14 +00:00
Alexey Milovidov
dbb2532e88 Minor change 2021-06-23 02:02:57 +03:00
kssenii
e0f330331b Fix exception 2021-06-22 19:15:34 +03:00
alesapin
6b815b4f17 Merge branch 'master' into zookeeper_snapshots 2021-06-22 13:50:27 +03:00
Kseniia Sumarokova
41c65a9965
Merge pull request #25528 from kssenii/something
Fix convertion of datetime with timezone for mysql, odbc, etc..
2021-06-21 22:55:22 +03:00
Benjamin Naecker
3275a53723 Resolves the actual port a server binds, in the case the user requests
any available port from the OS.
2021-06-21 12:36:32 -07:00
alesapin
743600a359 Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-21 13:46:22 +03:00
kssenii
c0732ddc12 Fix datetime with timezone 2021-06-21 08:22:12 +00:00
alexey-milovidov
7cd86086c2
Merge pull request #25521 from azat/fuzzer-catch-TOO_DEEP_RECURSION
Catch TOO_DEEP_RECURSION in fuzzer for formatted query too
2021-06-21 02:03:55 +03:00
Azat Khuzhin
fa29fe6ba9 Catch TOO_DEEP_RECURSION for formatted query too 2021-06-21 00:39:59 +03:00
Anton Popov
d8b6f15ef4
Merge pull request #23027 from azat/distributed-push-down-limit
Add ability to push down LIMIT for distributed queries
2021-06-20 23:08:50 +03:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652)
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
2021-06-20 11:24:43 +03:00
kssenii
babb11a887 Some fixes 2021-06-19 19:18:13 +00:00
Azat Khuzhin
2bc9664b52 Catch "Maximum parse depth" error in fuzzer
This exception should not fail the fuzzer check like here [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/25494/c6339b4c08ad8f50bf34831367de9a1ea90b1e09/fuzzer_msan/report.html#fail1
2021-06-19 21:34:37 +03:00
kssenii
2031607642 Reimplement 2021-06-19 11:52:28 +00:00
alexey-milovidov
d1348f2275
Merge pull request #25419 from nikitamikhaylov/copier-sharding-key
Fix segfault when sharding_key is absent in config
2021-06-19 01:23:44 +03:00
kssenii
f33387837e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-06-18 14:27:48 +00:00
kssenii
b30cb3f7da Test 2021-06-18 14:14:29 +00:00
kssenii
7cc6588f96 Tool to put files on server 2021-06-18 14:11:32 +00:00
kssenii
bdc8fe1ca0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into progress-bar 2021-06-18 09:33:29 +03:00
alesapin
dbe4ba8c2c Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-17 19:51:11 +03:00
alesapin
1a6abb4db4 Better 2021-06-17 19:32:50 +03:00
Nikita Mikhaylov
0d0ea91a0f fix 2021-06-17 15:58:05 +00:00
alexey-milovidov
04be5437d9
Merge pull request #25296 from abyss7/http-issues
Add settings for HTTP header limitations
2021-06-17 01:50:48 +03:00
alexey-milovidov
43cee9018e
Merge pull request #25281 from ClickHouse/client-update-prompt-on-reconnect
Update prompt in client when reconnecting
2021-06-17 01:40:08 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Kseniia Sumarokova
b4840bc4a9
Merge pull request #25280 from kssenii/bridges-constraints
Privilege drop and resource constraints for bridge processes
2021-06-16 22:13:16 +03:00
Ivan Lezhankin
ba08a580f8 Add test 2021-06-16 17:33:14 +03:00
mergify[bot]
b179184b3b
Merge branch 'master' into fix-install-toctou 2021-06-16 05:57:49 +00:00
Ivan Lezhankin
b182d87d9c Add settings for HTTP header limitations 2021-06-15 17:33:46 +03:00
Nikita Mikhaylov
a52bba91b7
Merge pull request #16401 from abyss7/ast-table-identifier-2
ASTTableIdentifier part #2: Introduce ASTTableIdentifier
2021-06-15 13:51:30 +03:00
Alexey Milovidov
e3653e70da Update prompt in client when reconnecting 2021-06-15 04:43:35 +03:00
kssenii
b152d7589f Bridge contsraints 2021-06-15 01:01:10 +00:00
Alexey Milovidov
22bd65996c Fix TOCTOU in Install 2021-06-15 00:58:29 +03:00
alexey-milovidov
a2c6b98b94
Merge pull request #25146 from oxidecomputer/master
Adds a better way to include binary resources
2021-06-13 16:13:34 +03:00
mergify[bot]
9cfedd9c64
Merge branch 'master' into progress-bar 2021-06-12 01:15:23 +00:00
alexey-milovidov
3792d20061
Merge pull request #24863 from azat/echo-hint
clickhouse-client: echo hint improvements
2021-06-12 03:23:54 +03:00
alexey-milovidov
b64950b193
Merge pull request #25045 from kssenii/odbc-fixes
odbc fix
2021-06-12 03:13:39 +03:00
Ivan
a7fa4e641c
Merge branch 'master' into ast-table-identifier-2 2021-06-10 16:43:22 +03:00
Alexander Kuzmenkov
c0e6aa97a6
Merge pull request #24734 from ClickHouse/aku/frame-expressions
support expressions in window frame
2021-06-10 00:25:32 +03:00
Benjamin Naecker
c3506bf16d Adds a better way to include binary resources
- Uses a small assembly file to include binary resources, rather than
  objcopy
- Updates `base/common/getResource.cpp` for this new method of inclusion
- Removes linux-only guards in CMake files, as this solution is
  cross-platform.

The resulting binary resources are available in the ClickHouse server
binary on Linux, macOS, and illumos platforms. FreeBSD has not been
tested, but will likely work as well.
2021-06-09 14:03:30 -07:00
Nikita Mikhaylov
82b8d45cd7
Merge pull request #23518 from nikitamikhaylov/copier-stuck
Bugfixes and improvements of `clickhouse-copier`
2021-06-09 11:36:42 +03:00
mergify[bot]
3a47feedb0
Merge branch 'master' into progress-bar 2021-06-09 08:04:48 +00:00
Azat Khuzhin
18e8f0eb5e Add ability to push down LIMIT for distributed queries
This way the remote nodes will not need to send all the rows, so this
will decrease network io and also this will make queries w/
optimize_aggregation_in_order=1/LIMIT X and w/o ORDER BY faster since it
initiator will not need to read all the rows, only first X (but note
that for this you need to your data to be sharded correctly or you may
get inaccurate results).

Note, that having lots of processing stages will increase the complexity
of interpreter (it is already not that clean and simple right now).

Although using separate QueryProcessingStage looks pretty natural.

Another option is to make WithMergeableStateAfterAggregation always, but
in this case you will not be able to disable only this optimization,
i.e. if there will be some issue with it.

v2: fix OFFSET
v3: convert 01814_distributed_push_down_limit test to .sh and add retries
v4: add test with OFFSET
v5: add new query stage into the bash completion
v6/tests: use LIMIT O,L syntax over LIMIT L OFFSET O since it is broken in ANTLR parser
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_(antlr_debug).html#fail1
v7/tests: set use_hedged_requests to 0, to avoid excessive log entries on retries
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_flaky_check_(address).html#fail1
2021-06-09 02:29:50 +03:00
kssenii
bd39c9fdd1 Better way 2021-06-07 18:56:46 +00:00
Ivan
579250168f
Merge branch 'master' into ast-table-identifier-2 2021-06-07 20:30:39 +03:00
kssenii
ba56465a99 Fix 2021-06-07 12:15:24 +00:00
Azat Khuzhin
8ec874dc33 Add writeRetry helper 2021-06-07 09:38:15 +03:00
alexey-milovidov
be66ecacef
Merge pull request #24918 from cn-ds/patch-1
Improve documentation about <remote_url_allow_hosts>
2021-06-07 03:01:17 +03:00
mergify[bot]
d8ed474062
Merge branch 'master' into copier-stuck 2021-06-05 01:40:23 +00:00
Nikita Mikhaylov
f6932026d5 delete partition from original table in case of errors while moving 2021-06-04 14:21:47 +00:00
Nikita Mikhaylov
68c92cd93e style 2021-06-03 23:58:47 +00:00
Nikita Mikhaylov
b978405f4a fix 2021-06-03 15:50:44 +00:00
Nikita Mikhaylov
6c583b0e36 Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-stuck 2021-06-03 15:05:23 +00:00
Amos Bird
9dfa4c06bd
Try reusing built sets during multi-pass analysis 2021-06-03 22:06:05 +08:00
cn-ds
aa98d6be3a
Improve documentation about <remote_url_allow_hosts>
Adding an example in the comments about the xml tag to use to specify allowed host

https://github.com/ClickHouse/ClickHouse/issues/24836
2021-06-03 14:21:42 +02:00
alesapin
4a1bf51ad1
Merge branch 'master' into limited_fetches 2021-06-02 23:01:47 +03:00
Nikita Mikhaylov
2248c77643 done 2021-06-02 17:18:54 +03:00
Ivan Lezhankin
ae38b4a3f1 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2021-06-02 16:07:59 +03:00
Nikita Mikhaylov
4382aac25d support for ReplicatedTable creation without arguments 2021-06-02 15:46:37 +03:00
Alexander Kuzmenkov
1d304d86a4 a lucky charm to ward off clang-tidy 2021-06-02 15:18:03 +03:00
Alexander Kuzmenkov
ae79bbea7d Merge remote-tracking branch 'origin/master' into HEAD 2021-06-02 15:06:22 +03:00
Alexander Kuzmenkov
78fe7891b4 better messages in fuzzer status 2021-06-02 15:06:12 +03:00
Azat Khuzhin
9882d160d8 clichouse-client: fix readability-identifier-naming 2021-06-02 07:38:56 +03:00
alesapin
feed2636f4 Add exponential smoothing to Throttler 2021-06-02 01:07:29 +03:00
Nikita Mikhaylov
312bb96eeb Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-stuck 2021-06-02 01:04:47 +03:00
Azat Khuzhin
ffacb7dcdf Add description for test hints 2021-06-02 01:04:06 +03:00
Azat Khuzhin
69cf881947 clickhouse-client: add echoOn/echoOff hints 2021-06-02 01:04:06 +03:00
Azat Khuzhin
1c595c127f clickhouse-client: echo queries only after "{ echo }" hint
Before this patch clickhouse-client interprets the whole queries and if
"{ echo }" found, it starts echoing queries, but this will make it
impossible to skip some of lines.
2021-06-02 01:04:06 +03:00
Nikita Mikhaylov
f57863b974 fix tests 2021-06-02 01:03:08 +03:00
kssenii
de277f8ac4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into progress-bar 2021-06-01 07:58:19 +00:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs
Poco::File to std::filesystem
2021-05-31 23:17:02 +03:00
mergify[bot]
08eca33d81
Merge branch 'master' into progress-bar 2021-05-31 19:54:20 +00:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Nikita Mikhaylov
33dcebbb24 style 2021-05-31 17:41:35 +03:00
Nikita Mikhaylov
01511d0cbe better 2021-05-31 17:40:07 +03:00
Nikita Mikhaylov
1c548058e6 add status json 2021-05-31 17:38:55 +03:00
Nikita Mikhaylov
ac072243d6 style + enable trivial test 2021-05-31 17:38:55 +03:00
Nikita Mikhaylov
6d19dea761 better 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
e9f66a0045 delete alter table optimize 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
fa22b41d5a added allow_to_drop_target_partitions with test 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
9022646633 support copying of tables with data skipping indices 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
24af47063e allow to copy materialized and alias 2021-05-31 17:37:10 +03:00
Nikita Mikhaylov
c41cc36046 support TTL on columns 2021-05-31 17:37:10 +03:00
Nikita Mikhaylov
90ab394769 better 2021-05-31 17:37:10 +03:00