Commit Graph

1486 Commits

Author SHA1 Message Date
Alexey Milovidov
3887e82dac Merge branch 'exception_id' of github.com:yandex/ClickHouse into exception_id 2021-07-12 03:18:39 +03:00
Alexey Milovidov
c4f34689ec Merge branch 'master' into exception_id 2021-07-11 22:12:06 +03:00
Vladimir
3157a46596
Fix special build, minor fix in Client.cpp 2021-07-10 13:28:44 +03:00
Alexey Milovidov
c63237f8e4 Merge branch 'master' into exception_id 2021-07-10 11:14:46 +03:00
alexey-milovidov
0686155b0e
Merge pull request #26128 from kssenii/fix-ch-local-progress
Fix progress for clickhouse-local
2021-07-10 11:10:46 +03:00
Alexey Milovidov
1f91c65aca Add error id to exceptions 2021-07-10 05:43:53 +03:00
alexey-milovidov
f43f99f6f5
Merge pull request #26152 from CurtizJ/client-color
Change color in client for double colon
2021-07-10 02:46:46 +03:00
Anton Popov
3588c5ad17 add color for double colon 2021-07-09 23:32:14 +03:00
Artur
8113580237 Add warning in debug mode 2021-07-09 14:40:32 +00:00
Alexander Tokmakov
4165ba2a01 add system.zookeeper_log 2021-07-09 17:05:35 +03:00
kssenii
947bb4a942 Fix progress bar for local 2021-07-09 16:43:00 +03:00
alexey-milovidov
818bbd6539
Update programs/server/play.html
Co-authored-by: Vladimir <vdimir@yandex-team.ru>
2021-07-08 17:06:37 +03:00
Alexey Milovidov
15dbb5a07a Final touch 2021-07-08 06:04:35 +03:00
Alexey Milovidov
15b75be59b Add comment 2021-07-08 05:29:21 +03:00
Alexey Milovidov
e841fae852 Improvement 2021-07-08 05:27:40 +03:00
Alexey Milovidov
aa84d6a913 Render pipelines in Play UI 2021-07-08 05:20:24 +03:00
Alexey Milovidov
d437ec2e1d Merge branch 'master' into system-metrics 2021-07-05 03:03:07 +03:00
alexey-milovidov
fef7c557f6
Merge pull request #24789 from kitaisreal/compile-aggregation-functions
Compile aggregate functions
2021-07-05 02:57:49 +03:00
Alexey Milovidov
c059d0a0ee More metrics 2021-07-05 01:41:09 +03:00
Alexey Milovidov
0bc58b7056 Better color 2021-07-04 06:09:44 +03:00
Alexey Milovidov
1a83c45f24 Automatically create links in Web UI 2021-07-04 06:04:42 +03:00
Alexey Milovidov
7a993404b4 Whitespace 2021-07-02 02:30:18 +03:00
Maksim Kita
da8c957167 Aggregator added CompiledExpressionCache 2021-07-01 22:56:36 +03:00
alesapin
cdc95fa98a
Merge pull request #25428 from ClickHouse/zookeeper_snapshots
Tool for conversion of ZooKeeper data into clickhouse-keeper snapshot
2021-06-29 12:58:45 +03:00
alexey-milovidov
e67a54bec3
Merge pull request #25770 from azat/fuzzer-deep-fix
Ignore TOO_DEEP_RECURSION server exception during fuzzing
2021-06-29 01:27:21 +03:00
Kseniia Sumarokova
9d02af7d8b
Merge pull request #25676 from sand6255/MaterializeMySQL-Support-Enum-Data-Type
MaterializeMySQL: support ENUM data type
2021-06-28 19:15:00 +03:00
mergify[bot]
04814114e7
Merge branch 'master' into zookeeper_snapshots 2021-06-28 14:17:32 +00:00
Azat Khuzhin
170301c531 Ignore TOO_DEEP_RECURSION server exception during fuzzing 2021-06-28 10:37:49 +03:00
Storozhuk Kostiantyn
4a3145f586 Materialize my sql support enum data type
* Implemented Enum for MaterializeMySQL
2021-06-25 16:42:25 +08:00
Azat Khuzhin
7739fcc295 clickhouse-client: fix NULL dereference for --param w/o value 2021-06-24 00:08:18 +03:00
alexey-milovidov
7a0cddcb4c
Merge pull request #25569 from oxidecomputer/master
Resolves actual port bound when user requests any available
2021-06-23 22:47:01 +03:00
Kseniia Sumarokova
2e422c0e5c
Merge pull request #25551 from kssenii/add-test
Add test for progress bar
2021-06-23 12:33:06 +03:00
mergify[bot]
fddf47a900
Merge branch 'master' into zookeeper_snapshots 2021-06-23 08:36:14 +00:00
Alexey Milovidov
dbb2532e88 Minor change 2021-06-23 02:02:57 +03:00
kssenii
e0f330331b Fix exception 2021-06-22 19:15:34 +03:00
alesapin
6b815b4f17 Merge branch 'master' into zookeeper_snapshots 2021-06-22 13:50:27 +03:00
Kseniia Sumarokova
41c65a9965
Merge pull request #25528 from kssenii/something
Fix convertion of datetime with timezone for mysql, odbc, etc..
2021-06-21 22:55:22 +03:00
Benjamin Naecker
3275a53723 Resolves the actual port a server binds, in the case the user requests
any available port from the OS.
2021-06-21 12:36:32 -07:00
alesapin
743600a359 Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-21 13:46:22 +03:00
kssenii
c0732ddc12 Fix datetime with timezone 2021-06-21 08:22:12 +00:00
alexey-milovidov
7cd86086c2
Merge pull request #25521 from azat/fuzzer-catch-TOO_DEEP_RECURSION
Catch TOO_DEEP_RECURSION in fuzzer for formatted query too
2021-06-21 02:03:55 +03:00
Azat Khuzhin
fa29fe6ba9 Catch TOO_DEEP_RECURSION for formatted query too 2021-06-21 00:39:59 +03:00
Anton Popov
d8b6f15ef4
Merge pull request #23027 from azat/distributed-push-down-limit
Add ability to push down LIMIT for distributed queries
2021-06-20 23:08:50 +03:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652)
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
2021-06-20 11:24:43 +03:00
Azat Khuzhin
2bc9664b52 Catch "Maximum parse depth" error in fuzzer
This exception should not fail the fuzzer check like here [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/25494/c6339b4c08ad8f50bf34831367de9a1ea90b1e09/fuzzer_msan/report.html#fail1
2021-06-19 21:34:37 +03:00
alexey-milovidov
d1348f2275
Merge pull request #25419 from nikitamikhaylov/copier-sharding-key
Fix segfault when sharding_key is absent in config
2021-06-19 01:23:44 +03:00
kssenii
bdc8fe1ca0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into progress-bar 2021-06-18 09:33:29 +03:00
alesapin
dbe4ba8c2c Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-17 19:51:11 +03:00
alesapin
1a6abb4db4 Better 2021-06-17 19:32:50 +03:00
Nikita Mikhaylov
0d0ea91a0f fix 2021-06-17 15:58:05 +00:00
alexey-milovidov
04be5437d9
Merge pull request #25296 from abyss7/http-issues
Add settings for HTTP header limitations
2021-06-17 01:50:48 +03:00
alexey-milovidov
43cee9018e
Merge pull request #25281 from ClickHouse/client-update-prompt-on-reconnect
Update prompt in client when reconnecting
2021-06-17 01:40:08 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Kseniia Sumarokova
b4840bc4a9
Merge pull request #25280 from kssenii/bridges-constraints
Privilege drop and resource constraints for bridge processes
2021-06-16 22:13:16 +03:00
Ivan Lezhankin
ba08a580f8 Add test 2021-06-16 17:33:14 +03:00
mergify[bot]
b179184b3b
Merge branch 'master' into fix-install-toctou 2021-06-16 05:57:49 +00:00
Ivan Lezhankin
b182d87d9c Add settings for HTTP header limitations 2021-06-15 17:33:46 +03:00
Nikita Mikhaylov
a52bba91b7
Merge pull request #16401 from abyss7/ast-table-identifier-2
ASTTableIdentifier part #2: Introduce ASTTableIdentifier
2021-06-15 13:51:30 +03:00
Alexey Milovidov
e3653e70da Update prompt in client when reconnecting 2021-06-15 04:43:35 +03:00
kssenii
b152d7589f Bridge contsraints 2021-06-15 01:01:10 +00:00
Alexey Milovidov
22bd65996c Fix TOCTOU in Install 2021-06-15 00:58:29 +03:00
alexey-milovidov
a2c6b98b94
Merge pull request #25146 from oxidecomputer/master
Adds a better way to include binary resources
2021-06-13 16:13:34 +03:00
mergify[bot]
9cfedd9c64
Merge branch 'master' into progress-bar 2021-06-12 01:15:23 +00:00
alexey-milovidov
3792d20061
Merge pull request #24863 from azat/echo-hint
clickhouse-client: echo hint improvements
2021-06-12 03:23:54 +03:00
alexey-milovidov
b64950b193
Merge pull request #25045 from kssenii/odbc-fixes
odbc fix
2021-06-12 03:13:39 +03:00
Ivan
a7fa4e641c
Merge branch 'master' into ast-table-identifier-2 2021-06-10 16:43:22 +03:00
Alexander Kuzmenkov
c0e6aa97a6
Merge pull request #24734 from ClickHouse/aku/frame-expressions
support expressions in window frame
2021-06-10 00:25:32 +03:00
Benjamin Naecker
c3506bf16d Adds a better way to include binary resources
- Uses a small assembly file to include binary resources, rather than
  objcopy
- Updates `base/common/getResource.cpp` for this new method of inclusion
- Removes linux-only guards in CMake files, as this solution is
  cross-platform.

The resulting binary resources are available in the ClickHouse server
binary on Linux, macOS, and illumos platforms. FreeBSD has not been
tested, but will likely work as well.
2021-06-09 14:03:30 -07:00
Nikita Mikhaylov
82b8d45cd7
Merge pull request #23518 from nikitamikhaylov/copier-stuck
Bugfixes and improvements of `clickhouse-copier`
2021-06-09 11:36:42 +03:00
mergify[bot]
3a47feedb0
Merge branch 'master' into progress-bar 2021-06-09 08:04:48 +00:00
Azat Khuzhin
18e8f0eb5e Add ability to push down LIMIT for distributed queries
This way the remote nodes will not need to send all the rows, so this
will decrease network io and also this will make queries w/
optimize_aggregation_in_order=1/LIMIT X and w/o ORDER BY faster since it
initiator will not need to read all the rows, only first X (but note
that for this you need to your data to be sharded correctly or you may
get inaccurate results).

Note, that having lots of processing stages will increase the complexity
of interpreter (it is already not that clean and simple right now).

Although using separate QueryProcessingStage looks pretty natural.

Another option is to make WithMergeableStateAfterAggregation always, but
in this case you will not be able to disable only this optimization,
i.e. if there will be some issue with it.

v2: fix OFFSET
v3: convert 01814_distributed_push_down_limit test to .sh and add retries
v4: add test with OFFSET
v5: add new query stage into the bash completion
v6/tests: use LIMIT O,L syntax over LIMIT L OFFSET O since it is broken in ANTLR parser
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_(antlr_debug).html#fail1
v7/tests: set use_hedged_requests to 0, to avoid excessive log entries on retries
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_flaky_check_(address).html#fail1
2021-06-09 02:29:50 +03:00
kssenii
bd39c9fdd1 Better way 2021-06-07 18:56:46 +00:00
Ivan
579250168f
Merge branch 'master' into ast-table-identifier-2 2021-06-07 20:30:39 +03:00
kssenii
ba56465a99 Fix 2021-06-07 12:15:24 +00:00
Azat Khuzhin
8ec874dc33 Add writeRetry helper 2021-06-07 09:38:15 +03:00
alexey-milovidov
be66ecacef
Merge pull request #24918 from cn-ds/patch-1
Improve documentation about <remote_url_allow_hosts>
2021-06-07 03:01:17 +03:00
mergify[bot]
d8ed474062
Merge branch 'master' into copier-stuck 2021-06-05 01:40:23 +00:00
Nikita Mikhaylov
f6932026d5 delete partition from original table in case of errors while moving 2021-06-04 14:21:47 +00:00
Nikita Mikhaylov
68c92cd93e style 2021-06-03 23:58:47 +00:00
Nikita Mikhaylov
b978405f4a fix 2021-06-03 15:50:44 +00:00
Nikita Mikhaylov
6c583b0e36 Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-stuck 2021-06-03 15:05:23 +00:00
Amos Bird
9dfa4c06bd
Try reusing built sets during multi-pass analysis 2021-06-03 22:06:05 +08:00
cn-ds
aa98d6be3a
Improve documentation about <remote_url_allow_hosts>
Adding an example in the comments about the xml tag to use to specify allowed host

https://github.com/ClickHouse/ClickHouse/issues/24836
2021-06-03 14:21:42 +02:00
alesapin
4a1bf51ad1
Merge branch 'master' into limited_fetches 2021-06-02 23:01:47 +03:00
Nikita Mikhaylov
2248c77643 done 2021-06-02 17:18:54 +03:00
Ivan Lezhankin
ae38b4a3f1 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2021-06-02 16:07:59 +03:00
Nikita Mikhaylov
4382aac25d support for ReplicatedTable creation without arguments 2021-06-02 15:46:37 +03:00
Alexander Kuzmenkov
1d304d86a4 a lucky charm to ward off clang-tidy 2021-06-02 15:18:03 +03:00
Alexander Kuzmenkov
ae79bbea7d Merge remote-tracking branch 'origin/master' into HEAD 2021-06-02 15:06:22 +03:00
Alexander Kuzmenkov
78fe7891b4 better messages in fuzzer status 2021-06-02 15:06:12 +03:00
Azat Khuzhin
9882d160d8 clichouse-client: fix readability-identifier-naming 2021-06-02 07:38:56 +03:00
alesapin
feed2636f4 Add exponential smoothing to Throttler 2021-06-02 01:07:29 +03:00
Nikita Mikhaylov
312bb96eeb Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-stuck 2021-06-02 01:04:47 +03:00
Azat Khuzhin
ffacb7dcdf Add description for test hints 2021-06-02 01:04:06 +03:00
Azat Khuzhin
69cf881947 clickhouse-client: add echoOn/echoOff hints 2021-06-02 01:04:06 +03:00
Azat Khuzhin
1c595c127f clickhouse-client: echo queries only after "{ echo }" hint
Before this patch clickhouse-client interprets the whole queries and if
"{ echo }" found, it starts echoing queries, but this will make it
impossible to skip some of lines.
2021-06-02 01:04:06 +03:00
Nikita Mikhaylov
f57863b974 fix tests 2021-06-02 01:03:08 +03:00
kssenii
de277f8ac4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into progress-bar 2021-06-01 07:58:19 +00:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs
Poco::File to std::filesystem
2021-05-31 23:17:02 +03:00
mergify[bot]
08eca33d81
Merge branch 'master' into progress-bar 2021-05-31 19:54:20 +00:00