Commit Graph

3115 Commits

Author SHA1 Message Date
Vitaly Baranov
1dda596689 Fix releasing query ID and session ID at the end of query processing. 2021-10-10 17:34:45 +03:00
tavplubix
1aeb5d55dd
Grep server log even if it contains binary data (#29903)
* grep server log even if it contains binary data

* Update cluster.py

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-10-10 12:55:37 +03:00
Azat Khuzhin
c66942bba5 Fix test_input_format_parallel_parsing_memory_tracking::test_memory_tracking_total
Server has pretty low memory limit 3GB, so let's use local to generate
the data.
2021-10-09 16:15:42 +03:00
Vitaly Baranov
fe12404cb0 Fix tests test_log_family_s3, test_log_family_hdfs. 2021-10-09 15:14:23 +03:00
Azat Khuzhin
b8bde2d451 Fix test_backup_restore after #29649
CI: https://clickhouse-test-reports.s3.yandex.net/29856/42ca2b4bb241827edf69bbd6938d6b19c31935f1/integration_tests_(asan).html#fail1
Cc: @CurtizJ
2021-10-09 15:14:20 +03:00
kssenii
e42a687b80 Fix 2021-10-08 23:52:15 +03:00
tavplubix
f0e3122507
Merge pull request #29804 from ClickHouse/improvement_for_28373
Improvement for #28373
2021-10-08 14:30:00 +03:00
Alexander Tokmakov
90cc63aecd fix tests 2021-10-07 22:05:51 +03:00
kssenii
d6e1d0503d Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-07 16:21:00 +03:00
Haavard Kvaalen
c41923c595 MaterializedMySQL: Update GTID set at end of transaction
We would update the set of seen GTIDs as soon as we saw a GTID_EVENT,
which arrives before a transaction.  This would mostly work fine, but
if we lost the connection to MySQL in the middle of a large transaction
we would persist that the transaction had been processed as soon as the
transaction had started.  When the connection was reestablished, we
would not process the transaction again, which meant that we only
applied parts of it.

Fix this by updating the seen GTIDs at the end of the transaction
instead.
2021-10-07 14:09:00 +02:00
Alexander Tokmakov
0db1e3614c fix test 2021-10-07 13:21:42 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491)
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-10-07 11:26:08 +03:00
Alexander Tokmakov
c0ee6d46f7 resolve dependency of StorageDictionary 2021-10-06 15:39:14 +03:00
alesapin
47de753750 Merge branch 'master' into nvartolomei-nv/parts-uuid-move-shard-kill 2021-10-06 11:31:05 +03:00
mergify[bot]
6eec0d76a7
Merge branch 'master' into update-minio 2021-10-06 05:45:18 +00:00
Kseniia Sumarokova
3663d23e4d
Merge pull request #29731 from kssenii/exists-subquery
Support EXISTS(subquery)
2021-10-05 22:26:19 +03:00
kssenii
51f78e4446 Merge branch 'update-minio' of github.com:kssenii/ClickHouse into update-minio 2021-10-05 13:20:07 +00:00
kssenii
1c37b0df53 Ping tests 2021-10-05 13:19:43 +00:00
mergify[bot]
3b1f49aac7
Merge branch 'master' into nv/parts-uuid-move-shard-kill 2021-10-05 12:36:02 +00:00
Vitaly Baranov
8a01b32cba
Merge pull request #28637 from vitlibar/fix-materialized-column-as-sharding-key
Fix materialized column as sharding key
2021-10-05 10:53:24 +03:00
Kseniia Sumarokova
d235520b3b
Merge pull request #29706 from kssenii/fix-flacky-test
Fix flacky test
2021-10-04 17:43:51 +03:00
Nikolai Kochetov
e2d07fbdf5
Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-04 17:35:32 +03:00
Nikolai Kochetov
6169b180cb Fix backward compatibility after #21196. 2021-10-04 17:27:44 +03:00
kssenii
f1ea186bff Exists 2021-10-04 13:12:30 +00:00
Kseniia Sumarokova
a8f872f63e
Update test.py 2021-10-04 11:38:38 +03:00
Vitaly Baranov
217659c1c6 Add test. 2021-10-04 10:56:38 +03:00
kssenii
a3ee621d5e Fix 2021-10-04 07:28:58 +00:00
kssenii
0cb7193fde Fix url with failover case 2021-10-03 13:58:03 +00:00
Maksim Kita
c966806205
Merge pull request #29529 from kitaisreal/external-xml-loaders-reload-with-server-configuration-support
ExternalXMLLoaders reload with server configuration support
2021-10-02 16:26:28 +03:00
alexey-milovidov
86d1ce63cf
Merge pull request #29649 from CurtizJ/merging-29376
Merging #29376
2021-10-02 12:02:42 +03:00
alexey-milovidov
9025f522ec
Merge pull request #29603 from qoega/improve-integration-2
Improve some more integration tests
2021-10-02 06:09:24 +03:00
Anton Popov
d404efbf7e add test for freeze in-memory parts 2021-10-01 22:05:36 +03:00
Nikolai Kochetov
754f7aafeb
Merge pull request #29608 from ClickHouse/fix-many-signals-in-integration-tests
Do not send many signals at server restart (integration tests).
2021-10-01 21:09:00 +03:00
Vitaly Baranov
27f6d5864d
Merge pull request #29060 from azat/inter-server-secret-auth-fix
Do not allow to reuse previous credentials in case of inter-server secret
2021-10-01 20:44:48 +03:00
Maksim Kita
4a480699bd Added tests 2021-10-01 16:44:09 +03:00
Vitaly Baranov
b2d13d0b19 Add test for expanding macros in RabbitMQ settings. 2021-10-01 12:58:54 +03:00
Nikolai Kochetov
291a8332b9 Do not send many signals at server restart (integration tests). 2021-10-01 12:52:24 +03:00
alexey-milovidov
b29e877f26
Merge pull request #29182 from ClickHouse/yandex-to-clickhouse-in-configs
Replace `<yandex>` to `<clickhouse>` in configs and more.
2021-10-01 12:24:12 +03:00
Yatsishin Ilya
ada6c50aaa Improve some tests 2021-10-01 08:44:50 +03:00
alexey-milovidov
7a3f85cda1
Update test.py 2021-10-01 05:04:12 +03:00
Alexey Milovidov
32c0b22a6a Skip test for executable table function under MSan 2021-10-01 05:02:57 +03:00
Alexey Milovidov
4271f8c738 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-10-01 01:33:58 +03:00
Azat Khuzhin
1af02f02bd Add a test for INSERT w/o user in interserver mode
v2: ensure that the test fails with the version w/o fix
v3: force connect by modifying config and reload it
v4: add comments
2021-10-01 01:13:08 +03:00
Azat Khuzhin
2822f71eec tests/integration/helpers: add query_id for query() 2021-10-01 01:13:08 +03:00
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings (#29140) 2021-10-01 00:26:24 +03:00
Alexey Milovidov
551c52c236 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-09-30 17:45:14 +03:00
Azat Khuzhin
98c311205f Cover IS NULL/IS NOT NULL for MySQL in integration tests 2021-09-29 21:45:09 +03:00
Alexey Milovidov
964e7efde5 Compatibility 2021-09-29 02:41:12 +03:00
alexey-milovidov
1e6f9ac635
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 20:26:44 +03:00
Ilya Yatsishin
e66e081119
Merge pull request #28522 from qoega/improve-integration-1
Improve integration tests: work with iptables and services listing
2021-09-28 16:35:38 +03:00
Alexey Milovidov
c7b7c8339f Compatibility 2021-09-28 05:00:06 +03:00
alexey-milovidov
995913e89c
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 02:27:36 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views
Rewrite PushingToViews
2021-09-27 21:19:11 +03:00
Maksim Kita
f7aed5aab8
Merge pull request #29393 from azat/rocksdb-fix-data-race
rocksdb: fix race condition during multiple DB opening
2021-09-27 11:32:59 +03:00
Kevin Michel
045d4f2d32
Don't replicate DROP/DETACH/DROP DETACHED on Replicated database 2021-09-27 10:10:46 +02:00
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
Nikolai Kochetov
841ef13dee
Merge pull request #29338 from ClickHouse/less-sleeps-in-integratin-tests
Less sleeps in integration tests.
2021-09-27 10:35:31 +03:00
Yatsishin Ilya
650de10dea Merge remote-tracking branch 'origin/master' into improve-integration-1 2021-09-27 09:47:49 +03:00
alexey-milovidov
3609d4adad
Merge pull request #29164 from azat/test_send_crash_reports-fix
test_send_crash_reports: suppress Fatal messages check (due to explicit SIGSEGV)
2021-09-27 03:54:10 +03:00
Alexey Milovidov
4c0feaf343 Fix compatibility 2021-09-27 02:11:10 +03:00
Nikolai Kochetov
3c9e6a04bf A little bit better 2021-09-26 18:06:32 +03:00
Azat Khuzhin
ad1e9e42fc Restore test_rocksdb_options::test_table_invalid_column_family_options test
This reverts commit e40a13bd1f.
2021-09-26 08:24:22 +03:00
Azat Khuzhin
d9e3419197 Restore test_rocksdb_options::test_invalid_column_family_options test
This reverts commit 1881c67eff.
2021-09-26 08:24:18 +03:00
Alexey Milovidov
ddd96f0106 Fix compatibility 2021-09-26 03:19:50 +03:00
Alexey Milovidov
fb7ce28198 Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-26 02:10:11 +03:00
alexey-milovidov
0bd962630e
Merge pull request #29341 from ClickHouse/fix-rocksdb-test
Fix integration test for RocksDB
2021-09-26 01:06:55 +03:00
Alexey Milovidov
f4ad62f661 Fix whitespace 2021-09-25 20:07:48 +03:00
alexey-milovidov
e40a13bd1f
Remove trash 2021-09-25 19:47:29 +03:00
Alexey Milovidov
cba6e39fe5 Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
Alexey Milovidov
85cdd2cf75 Fix mistake 2021-09-25 07:00:59 +03:00
Alexey Milovidov
4f7830ae3a Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 06:58:30 +03:00
alexey-milovidov
1881c67eff
Remove trash 2021-09-25 04:44:06 +03:00
Anton Popov
7628273df4
Merge pull request #29266 from CurtizJ/normalize-asts
Normalize ASTs in ColumnsDescription
2021-09-25 04:01:54 +03:00
Alexey Milovidov
2b2bec3679 Make some changes to Python code that I don't understand 2021-09-25 02:04:53 +03:00
Alexey Milovidov
82d57c6e47 Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 01:38:39 +03:00
Nikolai Kochetov
48bdabbc26 Less sleeps in integration tests. 2021-09-24 18:25:19 +03:00
alesapin
3614fb9e37
Merge pull request #29223 from ClickHouse/compressed_logs
Compressed logs for keeper
2021-09-24 16:03:26 +03:00
Nikolai Kochetov
193f566568 Fix exception message. 2021-09-24 14:54:39 +03:00
Yatsishin Ilya
e3610dec92 do not lose docker.log if dependencies failed to start before clickhouse up called 2021-09-24 13:46:55 +03:00
Nikolai Kochetov
24dac111b7 Try fix kafka test. 2021-09-24 13:30:57 +03:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
Yatsishin Ilya
bbac70693e Merge remote-tracking branch 'origin' into improve-integration-1 2021-09-23 17:35:06 +03:00
Yatsishin Ilya
ad4f7498ad add comment 2021-09-23 17:35:04 +03:00
Nikita Mikhaylov
e8ab9feb94
Merge pull request #29265 from ClickHouse/fix_yaml_test
Fix YAML integration tests
2021-09-23 15:12:38 +03:00
alesapin
941724bc11 Fix test rocksdb 2021-09-23 12:19:30 +03:00
alesapin
d1bb8d666e Move all yaml tests to clickhouse config 2021-09-23 11:08:40 +03:00
mergify[bot]
d69fc35dd9
Merge branch 'master' into compressed_logs 2021-09-22 19:43:16 +00:00
kssenii
1b9565dfa8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-09-22 22:08:44 +03:00
Anton Popov
8943389d03 normalize ASTs in ColumnsDescription 2021-09-22 18:55:20 +03:00
kssenii
a84fc77b52 Fixes 2021-09-22 18:10:25 +03:00
alesapin
3bea8d5201 Fix config root name 2021-09-22 18:00:08 +03:00
kssenii
b51840a205 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-22 11:45:22 +03:00
Alexey Milovidov
e9e77b4403 .tech -> .com 2021-09-22 03:22:57 +03:00
Alexey Milovidov
04962e61ee Fix integration tests 2021-09-22 02:08:09 +03:00
alexey-milovidov
0880ae2883
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-21 20:17:28 +03:00
Kevin Michel
008c3c812b
Allow non-replicated ALTER TABLE FETCH/ATTACH in Replicated databases
`ALTER TABLE ... FETCH` and `ALTER TABLE ... ATTACH` queries were
disabled in the Replicated database engine, because it could cause
accidental duplication of data.

This enables these queries but without replicating them.

In the case of `FETCH`, the part will only be fetched on the server
where the query is issued.

Similarly, in the case of `ATTACH`, the attached part only needs to
be available on the server where the query is issued.

If the table itself is using one of the Replicated MergeTree engines,
the attached data is then replicated by the table engine itself,
without intervention of the database engine.

This change is meant to help with live backup/restore when using the
Replicated database engine, using FREEZE for backup and ATTACH for
restore.
2021-09-21 17:45:23 +02:00
Yatsishin Ilya
c1f24cdd5e ro 2021-09-21 18:31:32 +03:00
Yatsishin Ilya
2f24a93dc1 Merge remote-tracking branch 'origin' into improve-integration-1 2021-09-21 18:11:31 +03:00
alesapin
ac7579f187 Compressed logs for keeper 2021-09-21 17:29:05 +03:00
kssenii
e301457e91 Fix prefetch mistake 2021-09-21 15:40:48 +03:00
mergify[bot]
a8215b0436
Merge branch 'master' into test_send_crash_reports-fix 2021-09-21 10:21:34 +00:00
kssenii
0104cb1c8e Better 2021-09-21 01:51:17 +03:00
Anton Popov
55cf857aba
Merge pull request #28707 from azat/max_suspicious_broken_parts_bytes
Implement max_suspicious_broken_parts_bytes
2021-09-20 19:55:07 +03:00
tavplubix
922cf7ee20
Merge pull request #28373 from ClickHouse/tables_topsort
Resolve table dependencies on metadata loading
2021-09-20 14:46:47 +03:00
alesapin
a249dcc5f5
Merge pull request #28301 from kssenii/materialized-postgresql
MaterializedPostgreSQL: allow dynamically adding/deleting tables, altering settings
2021-09-20 14:16:29 +03:00
Yatsishin Ilya
cfa513c490 Merge remote-tracking branch 'origin' into improve-integration-1 2021-09-20 10:29:31 +03:00
Alexey Milovidov
147ef79670 Change website URL, part 1 2021-09-20 01:56:28 +03:00
Alexey Milovidov
f0925b3cf6 Changed substitution path name 2021-09-20 01:50:09 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
Azat Khuzhin
f242b5e48c test_send_crash_reports: suppress Fatal messages check (due to explicit SIGSEGV)
CI: https://clickhouse-test-reports.s3.yandex.net/0/20d8523a2ebe22c93266e56fc097e273e4c8c68b/integration_tests_(asan).html#fail1
2021-09-19 00:15:35 +03:00
Azat Khuzhin
8f6cb9bd71 test_send_crash_reports: tiny pylint fixes 2021-09-19 00:15:34 +03:00
alexey-milovidov
20d8523a2e
Merge pull request #28888 from azat/mysql-in-fix
Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )
2021-09-18 15:53:40 +03:00
Azat Khuzhin
a613382d48 Cover max_suspicious_broken_parts/max_suspicious_broken_parts_bytes 2021-09-17 22:06:32 +03:00
Nicolae Vartolomei
9a02061d9c
Rewrite part movement between shards logic and add kill support
See https://github.com/ClickHouse/ClickHouse/pull/24585#issuecomment-857735081
for extra context about the current implementation.
2021-09-17 18:11:32 +01:00
mergify[bot]
c23f1fc160
Merge branch 'master' into fix_part_checker 2021-09-17 07:23:12 +00:00
Nikita Mikhaylov
c52b8ec083
Introduced MergeTask and MutateTask (#25165)
Introduced MergeTask and MutateTask
2021-09-17 00:19:58 +03:00
Kseniia Sumarokova
e9d44d0848
Merge pull request #28797 from kssenii/rabbit-fix
Rewrite StorageRabbitMQ a little
2021-09-16 23:58:51 +03:00
kssenii
432ca1ed93 Fix tests 2021-09-16 10:46:43 +00:00
kssenii
1650c92407 Better 2021-09-16 11:38:26 +03:00
Nikita Mikhaylov
6bf5b4cf2d
Merge pull request #28953 from vitlibar/allow-change-server-side-compression-from-client
gRPC: Allow change server-side compression from client.
2021-09-16 00:55:06 +03:00
alexey-milovidov
b6b0e442fe
Merge pull request #28723 from azat/test_distributed_load_balancing-fix
Increase number of iterations in test_distributed_load_balancing (fixes flakiness)
2021-09-16 00:25:54 +03:00
kssenii
84c8757049 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-15 12:28:31 +03:00
kssenii
710c8d12dd Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql 2021-09-15 12:28:09 +03:00
mergify[bot]
4993c7c527
Merge branch 'master' into fix_part_checker 2021-09-14 14:13:52 +00:00
Kseniia Sumarokova
d3e604e14b
Merge pull request #28603 from kssenii/fix-web-disk-2
Some fixes for static files disk
2021-09-14 08:24:24 +03:00
Alexander Tokmakov
8ac19caca9 Merge branch 'master' into tables_topsort 2021-09-13 16:58:39 +03:00
mergify[bot]
c54434084f
Merge branch 'master' into fix_part_checker 2021-09-13 13:19:34 +00:00
Filatenkov Artur
c23fe5baf6
Improve codec for encr 19896 (#27476)
* change syntax of encrypted command

* commit all encrypted changes

* correct encryption

* correct config for test

* add tests and correct code style and typos

* correct test

* fix unbundled build

* add log warning messages

* improve code according to review comments

* correct nonce

* correct errors found by fuzzing

* improve codec AES_128_GCM_SIV. Add AES_256_GCM_SIV. Add sections for last in tests. Improve documentation

* Update CompressionCodecEncrypted.h

* Update 01683_codec_encrypted.sql

* correct compression factory after changes in master

* correct behavior with wrong key in data

* correct fuzzer

* add connection for fuzzer with fix for compression_encrypted

* refactor code

* add load from config with throwing errors on server start

* fix typos and check style

* Update Server.cpp

* correct loading and reading

* refactor code. fix uninitialized value

* refactor code

* move defines from server to cpp file

* correct build

* remove repeated code

* correct namespace

* fix code style
2021-09-13 11:25:36 +03:00
alesapin
91e10adca5 Merge branch 'master' into fix_part_checker 2021-09-13 10:19:04 +03:00
Vitaly Baranov
113ddceabb gRPC: Allow change server-side compression from client. 2021-09-13 01:07:28 +03:00
mergify[bot]
267e56df8c
Merge branch 'master' into fix-web-disk-2 2021-09-11 15:45:43 +00:00
Azat Khuzhin
5ad9cb9b79 Add test_storage_mysql::test_mysql_in with multiple columns in IN 2021-09-11 14:15:27 +03:00
kssenii
ba1534f1df Fix checks 2021-09-10 14:16:01 +03:00
Alexander Tokmakov
3f8fb1e562 Merge branch 'master' into tables_topsort 2021-09-10 13:16:41 +03:00
alesapin
6243b64ff5
Merge pull request #28656 from ClickHouse/move_docker_images_to_separate_repo
Move docker images to a separate repo
2021-09-10 10:24:21 +03:00
alesapin
2359a222ff Fix more tests 2021-09-09 23:29:42 +03:00
alesapin
099c71f70b Add test 2021-09-09 19:45:42 +03:00
kssenii
0c013ed91b Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2 2021-09-09 15:26:36 +03:00
kssenii
904cf74159 Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql 2021-09-09 15:26:20 +03:00
kssenii
00be148fce Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-09 15:26:09 +03:00
tavplubix
341a6c51d6
Merging #24866 (#28691)
* Add StorageSystemISTables.cpp/.h

* Another attempt

* Columns and Views

* Add information schema db and fix information schema 'tables' table

* fix build

* remove copy-paste, add views to system tables

* add test

* fix

* fix_tests

Co-authored-by: Damir Petrov <petrovdamir2235@gmail.com>
Co-authored-by: Damir Petrov <0442a403@verstehen.sas.yp-c.yandex.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-09-09 12:37:51 +03:00
kssenii
e9721804b6 Fix tests 2021-09-09 12:34:47 +03:00
kssenii
f26da04cdf Fix checks 2021-09-09 09:53:58 +03:00
kssenii
bed2688dad Review fixes 2021-09-09 01:25:08 +03:00
kssenii
797f858bc6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql 2021-09-08 22:52:48 +03:00
kssenii
41bc68e82e Tests, url table function 2021-09-08 22:28:22 +03:00
kssenii
6108a4139c Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2 2021-09-08 20:54:07 +03:00
kssenii
ab6937ee45 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-08 20:43:37 +03:00
Nikita Mikhaylov
2b59fbacd3
Update tests/integration/test_MemoryTracking/test.py 2021-09-08 16:14:00 +03:00
mergify[bot]
9f0c50335f
Merge branch 'master' into tables_topsort 2021-09-08 10:55:11 +00:00