Commit Graph

14643 Commits

Author SHA1 Message Date
tavplubix
a6a43ce3ee
Revert "Implement h3ToGeo function" 2021-06-22 17:25:21 +03:00
Anton Popov
ac25b55e87
Merge pull request #25546 from CurtizJ/fix-index-array-join
Fix usage of index with array columns and ARRAY JOIN
2021-06-22 16:23:00 +03:00
Nikita Mikhaylov
2ec3bc2d96
Merge pull request #25520 from azat/not-parser-fix
Fix NOT parsing
2021-06-22 16:19:39 +03:00
Nikita Mikhaylov
19b3659517
Merge pull request #24867 from bharatnc/ncb/h3-indexing-funcs
Implement h3ToGeo function
2021-06-22 15:59:09 +03:00
alesapin
a8ce3bed2b
Merge pull request #25421 from ClickHouse/update_buffer_size_in_nuraft
Update NuRaft buffer to 64 bit size
2021-06-22 10:55:31 +03:00
Kseniia Sumarokova
41c65a9965
Merge pull request #25528 from kssenii/something
Fix convertion of datetime with timezone for mysql, odbc, etc..
2021-06-21 22:55:22 +03:00
Anton Popov
ffa56bde24 fix usage of index with array columns and ARRAY JOIN 2021-06-21 15:34:05 +03:00
kssenii
7d3dee172c FIx progress 2021-06-21 12:19:25 +00:00
kssenii
fb9c92cb6c Fix 2021-06-21 10:32:40 +00:00
alesapin
2f24ad6165 Merge branch 'fix_some_tests' into update_buffer_size_in_nuraft 2021-06-21 13:20:27 +03:00
alexey-milovidov
8ba6a5393f
Merge pull request #25509 from azat/fix-altstack
Fix alternative stack for SIGSEGV handling
2021-06-21 12:10:00 +03:00
alexey-milovidov
d2c86e52e6
Update checkStackSize.cpp 2021-06-21 12:09:14 +03:00
alexey-milovidov
f37faf02fa
Update checkStackSize.cpp 2021-06-21 12:08:30 +03:00
alexey-milovidov
9a4125dd7c
Update ThreadStatus.cpp 2021-06-21 12:06:41 +03:00
kssenii
c0732ddc12 Fix datetime with timezone 2021-06-21 08:22:12 +00:00
Kruglov Pavel
a54cbef5b7
Merge pull request #24341 from Avogar/arrow
Support complex types in Arrow/Parquet/ORC
2021-06-21 11:05:57 +03:00
Azat Khuzhin
fabff86788 Fix NOT parsing 2021-06-20 23:27:24 +03:00
Anton Popov
d8b6f15ef4
Merge pull request #23027 from azat/distributed-push-down-limit
Add ability to push down LIMIT for distributed queries
2021-06-20 23:08:50 +03:00
Azat Khuzhin
12eadc2b5e Add a guard page for alternative signal handling stack 2021-06-20 20:08:50 +03:00
Azat Khuzhin
4dcbf3a5c5 Replace __pthread_get_minstack() with a const 2021-06-20 14:03:04 +03:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652)
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
2021-06-20 11:24:43 +03:00
Azat Khuzhin
9a3a1397e6 Fix alternative stack for SIGSEGV handling
Previous stack size of 4096 wasn't enough, and this lead to that on
SIGSEGV signal handler may overwrite some TLS data (like in the
following example [1]) and in real binary "current_thread" was
overwritten, and this leads to lack of query-id on SIGSEGV, like here
[2].

  [1]: https://gist.github.com/azat/2ee360fd4f2828d363b0926431afacc6
  [2]: https://clickhouse-test-reports.s3.yandex.net/24411/79563953201ce6249a8fd49e22be3902ca4ee43a/fuzzer_ubsan/report.html#fail1

Fix this by using __pthread_get_minstack() that takes TLS block into
account. Yes it is private, but let's try (like rust tried).

This patch will also decrease TLS, since now the per-thread stack will
be allocated in the heap, not on the stack of each thread.

Refs: https://sourceware.org/legacy-ml/libc-alpha/2012-06/msg00335.html
Refs: https://maskray.me/blog/2021-02-14-all-about-thread-local-storage
Refs: https://sourceware.org/bugzilla/show_bug.cgi?id=11787
2021-06-19 22:47:01 +03:00
Azat Khuzhin
57960746e2 Add getStackSize() helper 2021-06-19 20:09:01 +03:00
Azat Khuzhin
c6339b4c08 Stable NOT chain formatting 2021-06-19 08:03:36 +03:00
Vladimir
1c91a8d2c2
Merge pull request #25454 from vdimir/issue-23351-v2 2021-06-18 21:56:46 +03:00
Kseniia Sumarokova
4fe722d1a4
Merge pull request #25431 from vitlibar/no-underscore_before-preprocessed_configs
Fix underscore before preprocessed configs.
2021-06-18 17:53:03 +03:00
Kseniia Sumarokova
5974f96f0c
Merge pull request #23656 from kssenii/progress-bar
Progress indication for File table engine in clickhouse-local and insert from file in clickhouse-client
2021-06-18 17:46:54 +03:00
mergify[bot]
c723dd7d40
Merge branch 'master' into arrow 2021-06-18 12:18:57 +00:00
vdimir
0ebca58e86
Check column in removeColumnNullability 2021-06-18 14:06:55 +03:00
Vladimir
4de0d1a593
Merge pull request #24885 from Jokser/disk-s3-read-error-fix 2021-06-18 13:38:59 +03:00
Vladimir
6af272891a
Merge pull request #24870 from azat/mv-dist-join-fix 2021-06-18 11:18:43 +03:00
vdimir
b25f6ba5b6
Remove assumeMutable from removeColumnNullability 2021-06-18 11:13:53 +03:00
Vladimir
fab7c9c7f6
Merge pull request #24061 from vdimir/issue-23351 2021-06-18 11:07:28 +03:00
Vladimir
51e5a019ed
Merge pull request #25410 from vdimir/issue-21730 2021-06-18 11:00:26 +03:00
kssenii
f7029b168b Generate ya make 2021-06-18 10:56:24 +03:00
kssenii
bdc8fe1ca0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into progress-bar 2021-06-18 09:33:29 +03:00
Nikolai Kochetov
b2620e1def
Merge pull request #24405 from amosbird/replacemorealias
Rewrite more alias columns
2021-06-18 09:10:31 +03:00
Vitaly Baranov
fc3d77bd66 Fix underscore before preprocessed configs. 2021-06-18 06:14:11 +03:00
Alexey Milovidov
3e71181451 Whitespaces 2021-06-18 01:42:33 +03:00
Maksim Kita
b34b66c55d
Merge pull request #25394 from ClickHouse/function-bit-position-to-array-merge
Merging #23843
2021-06-18 01:31:33 +03:00
Maksim Kita
2c4c2680f7
Merge pull request #25430 from kitaisreal/database-mysql-rename
DatabaseMySQL rename
2021-06-18 01:30:52 +03:00
Kseniia Sumarokova
47cb288834
Update src/Common/ProgressIndication.h
Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
2021-06-18 00:24:28 +03:00
Nikolai Kochetov
669b8a8b96
Merge pull request #25370 from ClickHouse/disable-query_plan_filter_push_down-because-of-25368
Fix lost WHERE condition in expression-push-down optimization of query plan
2021-06-17 23:07:32 +03:00
Nikolai Kochetov
5227a76e3a Add comment. 2021-06-17 23:05:39 +03:00
Maksim Kita
ca672b0a0d Function bitPositionsToArray added support for big integers 2021-06-17 22:32:39 +03:00
Pavel Kovalenko
ce8e2f7f28 Avoid double initializing ReadBufferFromS3 2021-06-17 22:31:50 +03:00
Maksim Kita
cf77e9f2c6 DatabaseMySQL rename 2021-06-17 22:26:34 +03:00
Vladimir
9e5ef95677
Merge pull request #25288 from amosbird/fixjoingetornull 2021-06-17 21:54:21 +03:00
alexey-milovidov
35411d1085
Merge pull request #24427 from azat/formatting-fixes
Fix formatting of negative values
2021-06-17 21:21:47 +03:00
alexey-milovidov
496aff2118
Merge pull request #25300 from ClickHouse/simpla-agg-func-for-summing-mt
Support simple aggregate functions for SummingMergeTree.
2021-06-17 20:18:51 +03:00