thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-25 00:52:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,745 Commits 9,166 Branches 1,781 Tags 2.1 GiB
45879472d3
Commit Graph

6903 Commits

Author SHA1 Message Date
Denis Zhuravlev
c0eefb644e test for #21413 2021-03-04 16:58:12 -04:00
Azat Khuzhin
205ebc7cff Tune idle_connection_timeout to avoid reconnects 
There may be 1 hour between two different distributed queries in tests
(see [1] and [2]), if this will happen then the connection may be closed
already due to idle_connection_timeout.

  [1]: https://clickhouse-test-reports.s3.yandex.net/21432/572d5862961ffda88125679e3a5ea684c316b223/functional_stateless_tests_(debug).html#fail1
  [2]: https://clickhouse-test-reports.s3.yandex.net/21432/572d5862961ffda88125679e3a5ea684c316b223/functional_stateless_tests_(antlr_debug).html#fail1

Let's tune it to avoid such issues and override other settings in tests.

Another option is to execute SYSTEM RELOAD CONFIG before each test...
 2021-03-04 22:48:09 +03:00
Azat Khuzhin
7794ac66d4 Fix flaky tests with distributed queries 
There was few attempts to fix this [1] and [2], but it still pops up
sometimes, for example here [3].

  [1]: 009f57fc2f / #11211
  [2]: cdb6bed8b9 / #14198
  [3]: https://clickhouse-test-reports.s3.yandex.net/21318/38be9ff43ac4c46ce6e803fc125d910bde1d4c71/functional_stateless_tests_(release,_databasereplicated).html#fail1

Let's use more generic approach, and do not hide any errors (and see
does increasing timeout will be enough).
 2021-03-04 22:48:04 +03:00
tavplubix
9244bc43ee
Merge pull request #21021 from zhang2014/fix/materialize_mysql_integration_test 
Try fix MaterializeMySQL integration test
 2021-03-04 22:22:20 +03:00
Denis Zhuravlev
102a0b2c5d test for #21413 2021-03-04 10:46:01 -04:00
filimonov
bd8c3eb117
avoid race in librdkafka 
see https://github.com/edenhill/librdkafka/issues/3279
 2021-03-04 15:01:10 +01:00
alesapin
39de525c74 Simplify config 2021-03-04 16:49:22 +03:00
alesapin
9e7a848169 Simplify config 2021-03-04 16:48:06 +03:00
alesapin
ad77de2a60 Enable snapshots in dangerous tests 2021-03-04 16:45:44 +03:00
alesapin
aa754a3ca8 Test restore from snapshot 2021-03-04 16:40:43 +03:00
alesapin
ff67860770 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-04 14:30:51 +03:00
Ivan
218542589a
Merge pull request #19673 from azat/distributed-bytes_to_throw_insert 
Add ability to throttle INSERT into Distributed
 2021-03-04 13:28:23 +03:00
Nikolai Kochetov
a195e783d2 Try fix perftests. 2021-03-04 13:25:49 +03:00
Nikolai Kochetov
b985e33294
Merge pull request #21434 from azat/dist-query-SIGSEGV-on-exception-fix 
Fix SIGSEGV for distributed queries on failures
 2021-03-04 11:32:17 +03:00
alesapin
7556a3d723 Enable snapshots 2021-03-04 11:00:26 +03:00
Azat Khuzhin
e858dae187 Fix SIGSEGV for distributed queries on failures 
Since after pull(), cancel() will be called, and this will lead to
SIGSEGV, since there is no exception, but has_exception was not reseted
in pull()
 2021-03-04 00:42:49 +03:00
Azat Khuzhin
e7948819f9 Fix ParsingException::displayText() 
Before it was silently try-catched for messages with additional {}, and
it is very easy to trigger, i.e.:

    SELECT toDateTime(format('{}-{}-01 00:00:00', '2021', '1'))

Will print:

    Code: 41. DB::Exception: Received from localhost:9000. DB::Exception: Cannot parse datetime 2021-1-01 00:00:00{}: Cannot parse DateTime from String: while executing 'FUNCTION toDateTime(format('{}-{}-01 00:00:00', '2021', '1') :: 3) -> toDateTime(format('{}-{}-01 00:00:00', '2021', '1')) DateTime : 2'.
 2021-03-03 23:44:51 +03:00
Azat Khuzhin
a57c646467 Fix 01293_system_distribution_queue flakiness 2021-03-03 23:30:24 +03:00
Azat Khuzhin
9a4f469a38 Fix 00753_distributed_system_columns_and_system_tables flakiness 2021-03-03 23:30:24 +03:00
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced 
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
 2021-03-03 23:30:23 +03:00
Azat Khuzhin
cabe4ca1bb tests: split 00753_system_columns_and_system_tables (to disable Distributed part for arcadia) 2021-03-03 23:30:03 +03:00
Azat Khuzhin
3102835d56 tests: Use total_bytes>0 for Distributed engine in 00753_system_columns_and_system_tables 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b43046ba06 Distributed: More accurate distribution_queue counters 
So now system.distribution_queue will show accurate statistics, so tests
does not requires sleep anymore.

But note that with too much distributed pending this will iterate over
all directories.
 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT 
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
 2021-03-03 23:30:00 +03:00
Azat Khuzhin
ce09b7ff89 Distributed: Implement totalBytes() (system.tables.total_bytes) 2021-03-03 23:29:11 +03:00
alesapin
76e10a4fb1 Missed files for tests 2021-03-03 23:05:33 +03:00
alesapin
0f733828c2 Missed config 2021-03-03 19:44:06 +03:00
alesapin
eb5e416fd5 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-03 19:19:02 +03:00
alesapin
7e28bfeb9e Fix log store start 2021-03-03 18:37:31 +03:00
Alexey Milovidov
e8df9971f1 Fix Arcadia 2021-03-03 18:12:39 +03:00
alesapin
869818d4ff Better test 2021-03-03 16:18:28 +03:00
Alexander Kuzmenkov
f169be740a
Merge pull request #21381 from ClickHouse/aku/perf-negative 
fix a rare false negative in perf tests
 2021-03-03 15:55:14 +03:00
alesapin
186b39f0d5 Followup fix 2021-03-03 15:31:21 +03:00
alesapin
37fc5faa6f Fix nasty serialization bug 2021-03-03 15:21:21 +03:00
alesapin
e615299ae6 Fix some bugs and add snapshots validation 2021-03-03 14:10:24 +03:00
Nikolai Kochetov
cb122160ae
Merge pull request #19291 from Avogar/hedged-requests 
Implementation of HedgedRequests
 2021-03-03 12:58:37 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key 
Fix alter modify query for partition key and other metadata fields
 2021-03-03 10:15:11 +03:00
alexey-milovidov
36c5fd840b
Merge pull request #21392 from ClickHouse/add-test-14740 
Add a test for #14740
 2021-03-03 06:33:18 +03:00
Alexey Milovidov
71b52542d9 Add to skip list for parallel runs 2021-03-03 04:37:11 +03:00
Alexey Milovidov
8f45ed5117 Add a test for #15469 2021-03-03 04:35:43 +03:00
alexey-milovidov
9092bc3b85
Merge pull request #21387 from den-crane/tests/issue21369 
test for issue#21369
 2021-03-03 03:05:01 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid 
Forbid to drop a column if it's referenced by materialized view
 2021-03-03 02:55:06 +03:00
Alexey Milovidov
a93e29ef4a Add a test for #14740 2021-03-03 02:00:19 +03:00
Pavel Kruglov
d1524b749e Fix tests 2021-03-03 01:24:41 +03:00
Denis Zhuravlev
1278b5c71a test for issue#21369 2021-03-02 16:29:11 -04:00
Nikolai Kochetov
8e6fa404c4
Merge pull request #21246 from ucasFL/union-distinct-improve 
Improve Normalization of ASTSelectWithUnionQuery
 2021-03-02 21:37:07 +03:00
Alexander Kuzmenkov
08148e062f Fix a rare false negative in perf tests 2021-03-02 19:21:30 +03:00
Pavel Kruglov
4269eaa252 Change tests 2021-03-02 17:05:33 +03:00
Pavel Kruglov
4166ae1f45 Restart timeout when we make progress 2021-03-02 15:40:24 +03:00
Nikolai Kochetov
ffbea93b10
Merge pull request #19576 from traceon/row-policy-with-prewhere 
Allow row policies with PREWHERE
 2021-03-02 13:51:37 +03:00
alesapin
06678d650d Merge branch 'master' into fix_alter_partition_key 2021-03-02 13:43:41 +03:00
alesapin
9ebf1b4fad Get rid of separate minmax index fields 2021-03-02 13:33:54 +03:00
tavplubix
ee22eeea6c
Merge pull request #21357 from ClickHouse/fix-21140 
Fix inconsistent column type for Values input format.
 2021-03-02 13:12:12 +03:00
Pavel Kruglov
153bfbfc28 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-03-02 11:59:32 +03:00
tavplubix
d305b23338
Merge pull request #21079 from azat/distributed_ddl_pool_size-zk-fix 
Fix various issues in DDLWorker (SIGSEGV and others)
 2021-03-02 11:59:30 +03:00
feng lv
a26c9e64a9 fix 
fix
 2021-03-02 03:20:03 +00:00
Nikolai Kochetov
f82e00ff3c Fix inconsistent column type for Values input format. 2021-03-01 23:10:50 +03:00
Alexander Kuzmenkov
977b98ad69
Merge pull request #21094 from den-crane/tests/window_functions 
test for window functions
 2021-03-01 19:00:45 +03:00
Denis Glazachev
14513e520e Fix privileges 2021-03-01 18:03:29 +04:00
Alexander Kuzmenkov
366fba4b04
Update 01592_long_window_functions1.sql 2021-03-01 16:06:44 +03:00
Anton Popov
c98e6bb1ad
Merge pull request #20550 from ClickHouse/fix-number-of-threads-in-subqueries 
Fix number of threads in subqueries
 2021-03-01 15:49:38 +03:00
alesapin
9c8afbeb53 Fix alter modify query for partition key and other metadata fields 2021-03-01 12:59:19 +03:00
Nikolai Kochetov
976dbe8077
Merge pull request #20341 from ClickHouse/filter-push-down 
Filter push down
 2021-03-01 12:35:06 +03:00
vdimir
b7c7c97d10
Merge branch 'master' into join-cast-types-v2 2021-03-01 11:42:21 +03:00
Nikolai Kochetov
eb42896e26 Merge branch 'master' into ucasFL-union-distinct-improve 2021-03-01 10:43:39 +03:00
alexey-milovidov
3bd180c416
Merge pull request #20149 from keen-wolf/timezone_about 
fix toMinute function to handle special timezone
 2021-02-28 21:53:51 +03:00
Denis Glazachev
eb80c189e4 Merge branch 'master' into row-policy-with-prewhere 2021-02-28 20:14:31 +04:00
alexey-milovidov
467fcbec06
Merge pull request #21292 from ClickHouse/extract-text-from-html 
Rewrite extractTextFromHTML function
 2021-02-28 16:36:18 +03:00
feng lv
51021c1164 forbid to drop a column if it's referenced by materialized view 2021-02-28 05:24:39 +00:00
Azat Khuzhin
d42d4cfd6b Improve test_distributed_ddl_parallel to cover more cases 
Refs: #21264
 2021-02-28 05:45:40 +03:00
Azat Khuzhin
a795db57f1 tests: decrease distributed_ddl_task_timeout to 60 in test_distributed_ddl_parallel 2021-02-28 05:45:11 +03:00
Azat Khuzhin
06e8bb955d tests: catch exceptions in threads in test_distributed_ddl_parallel 
Otherwise the test will not report an error.
 2021-02-28 05:45:11 +03:00
Azat Khuzhin
b899d8d919 tests: regression for test for invalid reference to zookeeper for distributed_ddl.pool_size > 1 2021-02-28 05:45:11 +03:00
Alexey Milovidov
107d8ec811 Merge branch 'master' into extract-text-from-html 2021-02-28 04:03:32 +03:00
alexey-milovidov
9f0400f015
Merge pull request #21171 from azat/tests-bump-zookeeper 
Bump zookeeper version to 3.6.2 in tests
 2021-02-28 02:35:58 +03:00
alexey-milovidov
b8fba768e5
Merge pull request #21264 from ClickHouse/fix_zookeeper_update 
Fix several bugs with ZooKeeper client
 2021-02-28 01:57:04 +03:00
alexey-milovidov
470bc2b01b
Merge pull request #21279 from kssenii/add-tests 
Add tests for zstd and zlib http compression
 2021-02-28 01:33:21 +03:00
Alexey Milovidov
0ab4afeeed Tests and documentation 2021-02-28 00:55:56 +03:00
Alexey Milovidov
2ac673b12a Update logic and tests 2021-02-27 22:33:55 +03:00
Alexey Milovidov
fd1cf49e92 Rewrite extractTextFromHTML function 2021-02-27 22:22:38 +03:00
kssenii
4f61ef3099 Tests for zstd and zlib 2021-02-27 14:12:45 +00:00
alexey-milovidov
56af09891b
Merge pull request #20991 from kssenii/brotli 
Fix Brotli http compression issue
 2021-02-27 16:39:49 +03:00
Maksim Kita
9996f68160
Merge pull request #20990 from kitaisreal/constraints-complex-type-support 
Constraints complex types support
 2021-02-27 13:16:29 +03:00
Vladimir
df728dc4ba
Merge pull request #21248 from vdimir/fix-join-totals-nullable 
Convert columns to nullable for totals in join
 2021-02-27 11:31:02 +03:00
feng lv
4ac5b62f19 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct-improve 2021-02-27 08:02:41 +00:00
Pavel Kruglov
197058dc12 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-27 00:36:33 +03:00
Azat Khuzhin
bc1a316b28 Bump zookeeper version to 3.6.2 in tests 
3.4.12 was released 1.05.2018
 2021-02-26 23:41:43 +03:00
alesapin
a48bb7e3a0 Fix redundant zookeeper reconnect and possibility of multiple zookeeper session for a single ClickHouse server 2021-02-26 22:05:25 +03:00
alexey-milovidov
5ecd9a9577
Merge pull request #21111 from filimonov/better-kafka-tests 
Better kafka tests
 2021-02-26 21:30:35 +03:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
Pavel Kruglov
91f6f9227b Small fixes and tests updates 2021-02-26 18:55:58 +03:00
Maksim Kita
2679613503
Added test to skip list DatabaseReplicated 2021-02-26 18:37:57 +03:00
kssenii
991718d459 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into brotli 2021-02-26 14:31:24 +00:00
tavplubix
09f62447af
Merge pull request #21060 from mxzlxy/fix-bug-for-replicated-engine-parameter 
fix default_replica_path and default_replica_name values are useless …
 2021-02-26 17:18:56 +03:00
vdimir
932286df1f
Convert columns to nullable for totals in join 2021-02-26 16:32:34 +03:00
feng lv
4c30c10092 add test 
fix
 2021-02-26 12:14:57 +00:00
Vladimir
78024ae2d6
Merge pull request #21009 from vdimir/fix-race-storage-join 2021-02-26 14:26:10 +03:00
Maksim Kita
c51773d2b3 Updated test 2021-02-26 12:11:30 +03:00
Nikolai Kochetov
f63d7b45ce Merge branch 'master' into fix-number-of-threads-in-subqueries 2021-02-26 11:15:03 +03:00