thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-01 20:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
61,827 Commits 9,193 Branches 1,782 Tags 2.2 GiB
5b8a3831e6
Commit Graph

1863 Commits

Author SHA1 Message Date
Pavel Kruglov
0e1d67ad9a Add LOG_DEBUG to debug test_distributed_load_balancing 2021-02-16 10:56:45 +03:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
kssenii
2a887b9772 Add missing format factory settings 2021-02-15 22:06:10 +00:00
Mike Kot
8182482cbd Add test stub 2021-02-15 21:06:20 +03:00
Azat Khuzhin
8d11d09615 Add a test for malformed directores for Distributed async INSERT 2021-02-15 21:00:50 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
Pavel Kruglov
df09a5cac9 Refactor 3.0 and fix tests 2021-02-15 16:21:36 +03:00
Alexander Tokmakov
9c7cf9e92e remove some debug code 2021-02-15 13:26:34 +03:00
alesapin
9686649b02 Fix non-zero session reconnect in integration test 2021-02-15 12:42:50 +03:00
Alexander Tokmakov
320fd6b264 startup without zk 2021-02-15 03:04:46 +03:00
Alexey Milovidov
b13d1f3142 Fix integration test 2021-02-14 04:34:42 +03:00
Denis Glazachev
99a04b6c64 Merge branch 'master' into row-policy-with-prewhere 
* master: (759 commits)
  Suppress UBSan report in Decimal comparison
  Suppress UBSan report in Decimal comparison
  Fix UBSan report in arrayDifference
  Update README.md
  Non significant change in AggregationCommon
  Print stack trace on SIGTRAP
  Fix dependent test
  Fix tests for better parallel run
  Add test for already working code
  Revert "Fix access control manager destruction order"
  Update index.md
  Update index.md
  Update index.md
  Bit more complicated example for isIPv4String - ru
  Bit more complicated example for isIPv4String
  cleanup
  Replace database with ordinary
  Added comments
  Split tests to make them stable
  Fixes
  ...

# Conflicts:
#	src/Storages/MergeTree/MergeTreeRangeReader.cpp
 2021-02-14 02:24:00 +04:00
Alexander Tokmakov
96cc01926c Merge branch 'master' into better_ddl_queue_cleanup 2021-02-14 00:22:09 +03:00
alesapin
f80137626a
Merge pull request #19580 from ClickHouse/in_memory_raft 
In memory coordination inside ClickHouse
 2021-02-13 10:19:23 +03:00
alexey-milovidov
2fe2190a20
Merge pull request #19721 from azat/buffer-profile 
Add separate config directive for Buffer profile
 2021-02-13 02:43:41 +03:00
alexey-milovidov
371e84957a
Merge pull request #20303 from azat/constant-folding-fixes 
Do not allow early constant folding of explicitly forbidden functions
 2021-02-13 02:20:06 +03:00
Alexander Tokmakov
184ec67dac better ddl queue cleanup 2021-02-12 22:23:50 +03:00
Alexander Tokmakov
f64f9b672b fix 2021-02-12 19:22:01 +03:00
alesapin
6aecb62416 Replace database with ordinary 2021-02-12 12:17:10 +03:00
alesapin
c883b7d154 Split tests to make them stable 2021-02-12 11:50:20 +03:00
alexey-milovidov
4e32dbd024
Merge pull request #19317 from abyss7/dwarf-folly 
Print inline frames for fatal stacktraces
 2021-02-12 02:38:50 +03:00
Alexander Tokmakov
c9cf63e958 fix 2021-02-12 01:23:40 +03:00
alesapin
df181b534e Better connection reset 2021-02-11 23:29:01 +03:00
alesapin
5f32f65211 Merge branch 'master' into in_memory_raft 2021-02-11 23:09:29 +03:00
Azat Khuzhin
3993ad6f01 Fix test_system_merges by using mutations_sync=1 
After early_constant_folding started to ignore not only ignore(), but
all functions with isSuitableForConstantFolding() == false, there became
more sleep(2) calls for this test:
- MergeTreeDataSelectExecutor::readFromParts -> DB::KeyCondition::KeyCondition
- MergeTreeDataMergerMutator::mutatePartToTemporaryPart -> DB::isStorageTouchedByMutations -> FilterTransform::transform
- MergeTreeDataMergerMutator::mutatePartToTemporaryPart -> DB::MergeTreeDataMergerMutator::mutateAllPartColumns -> FilterTransform::transform

While before it was optimized to 0 during WHERE analysis.
 2021-02-11 22:46:56 +03:00
alesapin
99a471e047 Add ability to start as follower 2021-02-11 13:25:10 +03:00
alesapin
0acd018361 Fix typo 2021-02-11 12:58:02 +03:00
alesapin
74630acff5 More debug in test 2021-02-11 12:49:49 +03:00
alesapin
b49b7f859d Simplify startup with fixed config 2021-02-11 12:17:57 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Azat Khuzhin
e2d5972eca Cover buffer_profile config directive 2021-02-10 22:44:26 +03:00
Ivan
bc58f4827f
Increase timeout in tests 2021-02-10 17:45:45 +03:00
alesapin
c95140d906 Better startup and non-verbose logging by default 2021-02-10 12:28:53 +03:00
Ilya Yatsishin
1fdc59b453
Merge pull request #20248 from qoega/integration-2 2021-02-10 08:46:20 +03:00
alesapin
7848f0202c One more test 2021-02-10 00:02:34 +03:00
alesapin
51c221f993 Fix outdated session kill 2021-02-09 21:29:06 +03:00
Yatsishin Ilya
3ba6629d7b add stable sort for tests 2021-02-09 18:48:33 +03:00
Yatsishin Ilya
eac13d9d9d Add retries to test_access_control_on_cluster 2021-02-09 18:37:33 +03:00
Alexander Tokmakov
2c6a0e74fb better replica creation 2021-02-09 18:14:20 +03:00
alesapin
b130fbfd78 Add coordination settings 2021-02-09 17:47:18 +03:00
alesapin
9667bdcbd2 Merge branch 'master' into in_memory_raft 2021-02-09 16:06:31 +03:00
Pavel Kruglov
46840b0a4f Add __init__.py 2021-02-09 00:47:13 +03:00
Nicolae Vartolomei
84489b8243 Improve replicated fetches timeouts test and make it 3x faster 2021-02-08 21:19:32 +00:00
Alexander Tokmakov
5e0cbfe972 Merge branch 'master' into database_replicated 2021-02-08 23:01:12 +03:00
Nicolae Vartolomei
6181daf92d Workaround for drop not finishing if it is started while table is readonly 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
f39eef9227 Reformat test and drop tables at the end to reduce the size of test results archive 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
165ba59a15 Increase network delay and add more info to logs for debugging 
CI behaviour is different from my test environment
 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
75fca08b2a Update test_replicated_fetches_network_partition with new settings 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
b153e8c190 Add support for custom fetchPart timeouts 2021-02-08 19:44:02 +00:00
Nicolae Vartolomei
813092ff55 Test replicated fetches_network partition 2021-02-08 19:44:02 +00:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
Ivan Lezhankin
00bb0e6b35 Skip send_crash_reports test with TSAN 2021-02-08 18:46:48 +03:00
Ivan Lezhankin
a287443438 Merge remote-tracking branch 'upstream/master' into dwarf-folly 2021-02-08 18:36:41 +03:00
alesapin
2413d6bd38 Test multinode watches 2021-02-08 16:26:06 +03:00
Pavel Kruglov
12b925dd7a Small update 2021-02-08 14:06:45 +03:00
Anton Ivashkin
1b6ce056a5 Merge master 2021-02-08 12:34:20 +03:00
alesapin
d5ee7d33e9 Merge branch 'master' into in_memory_raft 2021-02-08 10:30:31 +03:00
Vladimir Chebotarev
098a6d6a05 Added test for S3 table function. 2021-02-08 09:22:00 +03:00
JackyWoo
77c0f0a0e7 add quota type QUERY_SELECTS and QUERY_INSERTS 2021-02-07 10:57:18 +08:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
tavplubix
1c5d6bbe44
Merge pull request #20051 from stigsb/materializemysql-system-tables-fix 
Show details of MaterializeMySQL tables in `system.tables`
 2021-02-06 14:50:11 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID 
CLICKHOUSE-606: query deduplication based on parts' UUID
 2021-02-05 22:47:34 +03:00
Anton Ivashkin
e64c63c611 Merge master 2021-02-05 20:10:06 +03:00
Stig Bakken
3fc1641d91 Show details of MaterializeMySQL tables in system.tables 2021-02-05 21:37:29 +08:00
tavplubix
f59dc7f40d
Merge pull request #20066 from havardk/multi-table-update 
Fix handling of TABLE_MAP_EVENT from MySQL
 2021-02-05 13:11:59 +03:00
vdimir
30f98795bc
Merge remote-tracking branch 'upstream/master' into disk-s3-backup-restore-metadata 2021-02-04 18:38:28 +03:00
alesapin
933105a667 Fix session timeout 2021-02-04 16:31:38 +03:00
alesapin
d85e9b496c Fix gcc-10 build 2021-02-04 16:22:30 +03:00
Haavard Kvaalen
e6f1ce48fe Fix handling of TABLE_MAP_EVENT from MySQL 
The MySQL replication code assumed that row update events would be
preceded by a single TABLE_MAP_EVENT.  However, if a single SQL
statement modifies rows in multiple tables, MySQL will first send
table map events for all involved tables, and then row update events.

Depending on circumstances, this could lead to an exception when the row
update was processed, the update could be incorrectly dropped, or the
update could be applied to the wrong table.
 2021-02-04 12:49:10 +01:00
alesapin
9c04d912ec Merge branch 'master' into in_memory_raft 2021-02-04 11:50:32 +03:00
alesapin
e7a83868dd Fix build 2021-02-04 11:28:11 +03:00
Vladimir Chebotarev
f106d58c05 Minor fix. 2021-02-04 10:41:47 +03:00
Vladimir Chebotarev
a597ed0ff0 Fixed open behavior of remote host filter in case when there is remote_url_allow_hosts section in configuration but no entries there. 2021-02-04 09:34:31 +03:00
alesapin
1ff87ac6f9 Add background session lifetime control 2021-02-03 23:32:15 +03:00
alexey-milovidov
402c018d1e
Merge pull request #19947 from azat/dist-INSERT-block-structure-mismatch 
Fix "Block structure mismatch" for INSERT into Distributed
 2021-02-03 21:27:45 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Pavel Kovalenko
b7151aa754 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskDecorator.h
#	src/Disks/IDisk.h
#	src/Disks/S3/DiskS3.cpp
 2021-02-03 14:22:18 +03:00
alexey-milovidov
830b0eb36e
Merge pull request #19974 from qoega/kerberized-hdfs-port-clash 
Fixed port clash from test_storage_kerberized_hdfs test.
 2021-02-03 10:16:14 +03:00
alesapin
6743dd4656 Same for the first test 2021-02-02 22:56:07 +03:00
Alexander Tokmakov
6456ccf0da better test 2021-02-02 22:39:04 +03:00
alesapin
1e0a528bac Fix possible test flakyness 2021-02-02 22:33:57 +03:00
Azat Khuzhin
ac510f9770 test_read_temporary_tables_on_failure: increase timeout to avoid false-positive 
In [1]:

    start_cluster = <helpers.cluster.ClickHouseCluster object at 0x7f2d46895dd8>

        def test_different_versions(start_cluster):
            with pytest.raises(QueryTimeoutExceedException):
                node.query("SELECT sleep(3)", timeout=1)
            with pytest.raises(QueryRuntimeException):
    >           node.query("SELECT 1", settings={'max_concurrent_queries_for_user': 1})
    E           Failed: DID NOT RAISE <class 'helpers.client.QueryRuntimeException'>

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/b68508002d134ead05bf2ca0e22a13a34a5c55c6/integration_tests_(thread).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
33628e1bd9 test_materialize_mysql_database: add retries for test_select_without_columns_8_0 
Here is an example:

    2021.01.31 14:04:43.615618 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:43.616453 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_1_1_0 to 0_1_1_0.
    ...
    2021.01.31 14:04:43.786823 [ 12 ] {4a0d1f80-4a96-4809-a84e-d55d7cd25cdd} <Debug> executeQuery: (from 172.18.0.1:45162, using production parser) SELECT count(_version) FROM db.t FORMAT TSV
    ...
    2021.01.31 14:04:44.566037 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:44.566943 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_2_2_0 to 0_2_2_0.
    ...
    2021.01.31 14:04:52.343911 [ 54 ] {} <Error> MaterializeMySQLSyncThread: Code: 1002, e.displayText() = DB::Exception: Master maybe lost, Stack trace (when copying this message, always include the lines below):

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/978f23e3437d0d0eb64727e87ffe1b80289c9fc2/integration_tests_(release).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
3d3d6777d3 test_dictionaries_dependency_xml: decrease dictionaries lifetime 
Since ExternalLoader::PeriodicUpdater::check_period_sec = 5, and so if
it will be scheduled too early the reload will be skipped, and indeed
this is what you can see in logs [1], the reload is done each 10
seconds, not 5:

    2021.01.31 14:20:22.590999 [ 48 ] {} <Trace> ExternalDictionariesLoader: Supposed update time for 'dep_x' is 2021-01-31 14:20:27 (loaded, lifetime [5, 5], no errors)
    2021.01.31 14:20:22.591016 [ 48 ] {} <Trace> ExternalDictionariesLoader: Next update time for 'dep_x' was set to 2021-01-31 14:20:27
    ...
    2021.01.31 14:20:32.164882 [ 50 ] {} <Trace> ExternalDictionariesLoader: Start loading object 'dep_x'

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/37797fdf5b30dc97147e73b3ac8ca9025b80aaed/integration_tests_(release).html#fail1
 2021-02-02 22:04:32 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Pavel Kruglov
dd9af192c5 Update test 2021-02-02 18:18:05 +03:00
alesapin
9979181c02 Merge branch 'master' into in_memory_raft 2021-02-02 17:05:20 +03:00
alesapin
94201ebf04 More info in test 2021-02-02 15:41:09 +03:00
Pavel Kruglov
cc14cb11f9 Update test 2021-02-02 15:20:28 +03:00
Pavel Kruglov
60a92e9a99 Fix build, add comments, update tests 2021-02-02 15:14:31 +03:00
Yatsishin Ilya
19d6e867c6 HDFS test had a clash with 9000 ClickHouse port. 2021-02-02 12:08:47 +03:00
tavplubix
dbcddbcebd
Merge pull request #19770 from stigsb/materializemysql_system_parts 
Show MaterializeMySQL tables in system.parts
 2021-02-02 11:43:42 +03:00
alesapin
d265e3b419 Less timeouts 2021-02-02 11:02:25 +03:00
alesapin
a9c50caa9a
Merge pull request #19942 from ClickHouse/fix_test_keeper_b2b_test 
Fix test keeper integration tests
 2021-02-02 10:17:27 +03:00
Pavel Kruglov
7d9eb966f0 Fix 2021-02-01 20:09:55 +03:00
alesapin
e6fd6abe6f Fix test keeper integration tests 2021-02-01 18:07:34 +03:00
alesapin
365bf65f5a Fix install script 2021-02-01 17:14:59 +03:00
alesapin
2aa8a6304b
Merge pull request #15450 from CurtizJ/fix-ttl-group-by 
Fix some cases of TTL expressions
 2021-02-01 16:48:07 +03:00
alesapin
0aca40d8cb Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
Stig Bakken
bedc472bb0 Show MaterializeMySQL tables in system.parts 2021-02-01 16:59:10 +08:00
alesapin
67412bd529 Don't accept connections if we don't see leader + race fix from upstream 2021-02-01 10:51:10 +03:00
alesapin
d9598c47b4
Merge pull request #19702 from ClickHouse/fix_rare_bug_after_part_corruption 
Fix incorrect virtual_parts after part corruption
 2021-02-01 10:09:48 +03:00
Denis Glazachev
6aa86846ac Removing obsoleted test 2021-01-31 05:41:28 +04:00
Alexey Milovidov
7b8589dbde Merge branch 'master' into utf8mb4-in-mysql-client 2021-01-31 01:37:27 +03:00
Alexey Milovidov
282409e8a9 Add a patch from Stig Bakken 2021-01-31 01:37:12 +03:00
Ilya Yatsishin
1684be296c
Merge pull request #19833 from filimonov/fixes-int-tests 2021-01-30 22:51:52 +03:00
Ilya Yatsishin
5afcc927ac
Merge pull request #19706 from qoega/easier-integration-no-project-dir 2021-01-30 17:22:00 +03:00
Mikhail Filimonov
39f75ccd19
Revert changes in test_multiple_disks 2021-01-30 11:26:31 +01:00
Alexander Kuzmenkov
98e88d7305
Merge pull request #19793 from excitoon-favorites/fixcompressions3 
Fixed table function S3 `auto` compression mode
 2021-01-29 23:42:05 +03:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
Mikhail Filimonov
ab8db8a239
Try to make test_dir.tar smaller 2021-01-29 17:57:52 +01:00
Yatsishin Ilya
ac168fe4e3 add sleep 2021-01-29 19:16:10 +03:00
Yatsishin Ilya
01c9b90144 fix project name for clickhouse instance 2021-01-29 19:16:10 +03:00
Yatsishin Ilya
5f6a4ad60d Remove --project-directory for docker-compose. Fix logs formatting from docker container. 2021-01-29 19:16:10 +03:00
alesapin
e4f9a4ea65 Merge branch 'master' into in_memory_raft 2021-01-29 15:56:54 +03:00
alesapin
bcc3c4193e
Merge pull request #19697 from qoega/easier-macro-xml-generate 
No more dicttoxml with excessive logs in integration tests
 2021-01-29 12:00:20 +03:00
Vladimir Chebotarev
d729aacb09 Fixed table function S3 auto compression mode. 2021-01-29 07:54:52 +03:00
alesapin
2881c830e3 Merge branch 'master' into fix_rare_bug_after_part_corruption 2021-01-28 23:16:52 +03:00
alexey-milovidov
aeaa8d2439
Merge pull request #19759 from ClickHouse/fix_test_test_postgres_odbc 
Make integration odbc tests idempotent
 2021-01-28 20:18:16 +03:00
Yatsishin Ilya
70963e4396 Fixed test_kafka_flush_by_time test. 2021-01-28 18:03:23 +03:00
Anton Popov
7db0766192 fix flaky checks 2021-01-28 17:40:50 +03:00
alesapin
1510e3147d Different ports for nodes 2021-01-28 16:08:07 +03:00
alesapin
8d0d2ca8e0 Add some partition tests 2021-01-28 15:07:26 +03:00
alesapin
18f1fd0caa Make integration odbc tests idempotent 2021-01-28 11:40:12 +03:00
Anton Popov
e5125b8c73 add comments and test for compatibility 2021-01-28 02:39:15 +03:00
alesapin
c74631c650 Fix test and add logical error 2021-01-27 21:54:05 +03:00
alesapin
a33963e211 Better raft server startup 2021-01-27 20:54:25 +03:00
alesapin
dd54e5afe8 Merge branch 'master' into in_memory_raft 2021-01-27 15:32:16 +03:00
alesapin
4a6aef5421 Merge branch 'debug/test_concurrent_ttl_merges' of https://github.com/qoega/ClickHouse into fix_flaky_test_multiple_ttl 2021-01-27 14:51:55 +03:00
alesapin
01c8b9e1b1 Fix rare bug when some replicated operations (like mutation) cannot process some parts after data corruption 2021-01-27 13:07:18 +03:00
Yatsishin Ilya
d21d838604 No more dicttoxml with excessive logs 2021-01-27 12:50:11 +03:00
vdimir
fc8b8d23b2
Merge pull request #18812 from excitoon-favorites/betters3endpoints 
Added prefix-based S3 endpoint settings
 2021-01-26 20:47:57 +03:00
Yatsishin Ilya
76adc85c75 fix 2021-01-26 19:33:37 +03:00
Yatsishin Ilya
403e74d941 Add more debuginfo for test_concurrent_ttl_merges test 2021-01-26 18:12:08 +03:00
alesapin
6441d3200b Merge branch 'master' into in_memory_raft 2021-01-26 14:14:39 +03:00
bo zeng
c2d079d40e
fix issue #18086 (#18124) 
* fix issue #18086

* Update convertMySQLDataType.cpp

* add test case for type mapping for mysql

* remove type mapping test case from materialize_with_ddl.py, will add case in mysql engine.

* add test on types and values for mysql database engine.

* typo T_T

* add missing variables

* update the assert code to align with new test data.

* better code for testing, and remove null test since it is not supported.

* remove edge value from test since it always fail. should fix it in clickhouse-client.

* remove some testcases currently not support.

* correct test case

* correct test case. maybe I need to build up a test environment in my local pc...

* correct test data

* Update convertMySQLDataType.cpp

Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
 2021-01-26 13:37:57 +03:00
Alexander Tokmakov
3146a1a954 fix 2021-01-25 21:59:23 +03:00
alesapin
97b9dba460 Multinode config 2021-01-25 17:10:18 +03:00
Alexander Tokmakov
3bd4d97353 Merge branch 'master' into database_replicated 2021-01-25 14:19:04 +03:00
tavplubix
4be33e81ee
Merge pull request #19344 from zhang2014/patch-2 
ISSUES-18684 fix MaterializeMySQL integration test failure
 2021-01-25 12:46:53 +03:00
Vladimir Chebotarev
8ba8e8efd5 Added prefix-based S3 endpoint settings. 2021-01-25 06:40:57 +03:00
alexey-milovidov
be2d1deac2
Merge pull request #19431 from ClickHouse/integration_tests_forbid_check_call 
Integration tests: print stderr of failed subprocess
 2021-01-23 02:28:42 +03:00
Alexander Tokmakov
14e842c197 print stderr of failed subprocess 2021-01-22 17:27:23 +03:00
Ivan Lezhankin
c965e66a3b Increase timeout for crash report 2021-01-22 15:02:41 +03:00
Winter Zhang
7f68afa362
ISSUES-18684 fix MaterializeMySQL integration failure 2021-01-21 13:11:07 +08:00
Anton Ivashkin
357d98eb36 Merge master 2021-01-20 12:23:03 +03:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
Alexander Tokmakov
e2467427d1 Merge branch 'master' into update_cassandra 2021-01-19 14:10:15 +03:00
Alexander Tokmakov
7f97a11c84 Merge branch 'master' into database_replicated 2021-01-18 17:09:39 +03:00
Pavel Kovalenko
1e3a059f64 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskCacheWrapper.cpp
#	src/Disks/S3/DiskS3.cpp
 2021-01-18 13:39:49 +03:00
Pavel Kovalenko
2dda8ed1e0 Ability to backup-restore metadata files for DiskS3 (minor fixes) 2021-01-18 13:37:09 +03:00
alexey-milovidov
d2366b5c6c
Merge pull request #18233 from vitlibar/fix-access-rights-for-select-count 
Fix access rights for SELECT count()
 2021-01-17 00:12:08 +03:00
alexey-milovidov
7de745ce77
Merge pull request #18554 from kssenii/pg2ch 
Add PostgreSQL table function, dictionary source, database engine
 2021-01-16 23:55:05 +03:00
Azat Khuzhin
7135faf672 Cover distributed send with different headers 2021-01-16 00:45:11 +03:00
Alexey Milovidov
b293d56a63 Merge branch 'master' into kssenii-pg2ch 2021-01-15 22:32:03 +03:00
alexey-milovidov
b97beea22a
Merge pull request #19101 from ClickHouse/check_compression_codec_read 
Fix compression codec read for empty files
 2021-01-15 20:55:58 +03:00
Alexey Milovidov
a19e7edd14 Merge branch 'master' into kssenii-pg2ch 2021-01-15 17:33:19 +03:00
alexey-milovidov
ca825f36f1
Merge pull request #18853 from azat/dist-send-checksums 
Improve checksum checks for async INSERT into Distributed on the sender
 2021-01-15 17:12:54 +03:00
alesapin
d601faa669
Merge pull request #18935 from fastio/bugfix_attach_partition_does_not_reset_mutation 
Bugfix: attach partition should reset the mutation
 2021-01-15 12:21:16 +03:00
alesapin
0662d6bd7d Fix compression codec read for empty files 2021-01-15 12:04:23 +03:00
Alexander Tokmakov
f2184eea5e try update cassandra driver library 2021-01-14 21:05:08 +03:00
Pavel Kovalenko
53389f79c0 Ability to backup-restore metadata files for DiskS3 (minor fixes) 2021-01-13 15:05:32 +03:00
alesapin
7650968b1e
Fix flaky test 2021-01-13 10:19:43 +03:00
alesapin
73e536a074
Merge pull request #18928 from ClickHouse/more_checks_in_writer_wide 
More checks in merge tree writer wide
 2021-01-13 09:59:45 +03:00
Pavel Kovalenko
b09862b7b9 Ability to backup-restore metadata files for DiskS3 (fixes and tests) 2021-01-12 20:18:40 +03:00
alesapin
f3e55183ad Better test and check 2021-01-12 11:46:31 +03:00
fastio
b43bd4d900 test 2021-01-12 12:54:59 +08:00
Pavel Kovalenko
0164965bc0 Fix get_process_pid 2021-01-11 20:40:11 +03:00
Pavel Kovalenko
0856b2c514 Ability to backup-restore metadata files for DiskS3 (fixes and tests) 2021-01-11 20:37:08 +03:00
tavplubix
213c653ef8
Merge pull request #17656 from bharatnc/ncb/ddl-worker-queue-table 
add  system.distributed_ddl_queue table
 2021-01-11 14:59:26 +03:00
Azat Khuzhin
22c07e7342 Update test_check_table expectations 
Since now CompressedReadBufferBase read checksum and header at once it
will expect 25 (16+9) bytes not 16 (only checksum)
 2021-01-11 08:05:56 +03:00
Azat Khuzhin
f09a53103f Add a test for corruption during distributed send 2021-01-10 18:17:14 +03:00
kssenii
fc9de76f7d Fix incorrect getCreateTableQueryImpl with multidim arrays 2021-01-10 09:00:35 +00:00
kssenii
0088d66d59 Hopefully better array conversions for insert query 2021-01-09 22:42:42 +00:00
Azat Khuzhin
471deab63a Rename fsync_tmp_directory to fsync_directories for Distributed engine 2021-01-09 17:51:30 +03:00
Azat Khuzhin
9cdb1d4ae6 Simple coverage of fsync_after_insert/fsync_tmp_directory with batching 2021-01-09 16:31:52 +03:00
alexey-milovidov
6dcb887eed
Merge pull request #18857 from amosbird/fixmaxpartitionremotesetting 
Correctly override default settings remotely
 2021-01-08 22:25:43 +03:00
Amos Bird
a3d19fa64d
Correctly override default settings remotely 2021-01-08 12:28:09 +08:00
bharatnc
24d67082a9 few more changes and simplification to integration test 2021-01-07 11:10:41 -08:00
bharatnc
57126d1901 Add values column and fix tests 2021-01-07 11:10:41 -08:00
bharatnc
f6d33c49bb DDLWorkerQueueTable - fix integration test 2021-01-07 11:10:41 -08:00
bharatnc
b008c49bd9 DDlWorkerQueueTable - add integration test 2021-01-07 11:10:40 -08:00
Amos Bird
63f0348291
Fix test 2021-01-07 11:31:23 +08:00
alexey-milovidov
72b142a00a
Merge branch 'master' into pg2ch 2021-01-06 23:18:59 +03:00
Amos Bird
a32f6dcf1b
Apply alter settings on cluster to all replicas 2021-01-06 20:08:46 +08:00
John Skopis
9bcc5a6175 Support interserver credential rotation 
Restarting a server instance to change the interserver password results
in many replicas being out of sync until all clusters are using the new
credential.

This commit adds dynamic credential loading for both the client
(Replicated* tables) and server (InterserverIOHTTPHandler).

This commit also adds the ability to rotate credentials, i.e. accept more
than one credential during a credential change.

state0 (no auth):

    <interserver_http_credentials />

state1 (auth+allow_empty migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>222</password>
        <allow_empty>true</allow_empty>
    </interserver_http_credentials>

state2 (auth+new admin password migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>333</password>
        <users>
            <admin>222</admin>
        </users>
    </interserver_http_credentials>
 2021-01-04 09:59:09 +00:00
Alexey Milovidov
c7b678b752 Fix integration test 2021-01-04 02:20:12 +03:00
Nikolai Kochetov
3a1ec56486 Revert "Merge pull request #18411 from ClickHouse/try-fix-max_result_rows" 
This reverts commit 9a2b163db7, reversing
changes made to 5e97df7c29.
 2021-01-03 21:51:57 +03:00
Nikita Mikhaylov
f27aa148e0
Merge branch 'master' into parallel-parsing-input-format 2020-12-31 06:16:43 +04:00
Nikita Mikhaylov
99b4d2f9ba
Merge pull request #18636 from ClickHouse/revert-15316-copier-segfault 
Revert "Add some extra tests to copier"
 2020-12-31 03:43:51 +04:00
alexey-milovidov
9a2b163db7
Merge pull request #18411 from ClickHouse/try-fix-max_result_rows 
Use Port::Data instead of Chunk in LazyOutputFormat.
 2020-12-31 02:24:10 +03:00
Nikita Mikhailov
b6a27de508 Merge branch 'parallel-parsing-input-format' of github.com:nikitamikhaylov/ClickHouse into parallel-parsing-input-format 2020-12-30 18:23:05 +03:00
Nikita Mikhailov
8e31ce4a5c Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-30 18:22:11 +03:00
Nikita Mikhaylov
797c969c4d
Revert "Add some extra tests to copier" 2020-12-30 17:25:55 +03:00
Nikita Mikhaylov
736cc6959a
remove kafka flaky fix 2020-12-30 17:24:39 +03:00
alexey-milovidov
1bcdf37c36
Merge pull request #18614 from CurtizJ/fix-empty-parts 
Fix removing of empty parts in tables with old syntax
 2020-12-30 17:20:31 +03:00
Nikita Mikhailov
cc794b7a83 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-30 16:55:15 +03:00
Vitaly Baranov
235ea220ec SELECT count() FROM table now requires at least one column to be accessible in the table. 2020-12-30 15:00:08 +03:00
Vitaly Baranov
460006b502
Merge pull request #18232 from vitlibar/fix-access-rights-for-select-join 
Fix access rights for SELECT JOIN.
 2020-12-30 14:42:16 +03:00
Nikita Mikhailov
2b90b4e01d Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-29 22:47:06 +03:00
filimonov
e81a99156f
Attempt to prevent test flap 
That extra SELECT is executed synchronously, after that all further actions will have a more stable timing.
 2020-12-29 18:51:00 +01:00
Anton Popov
6336fbf1df fix removing of empty parts in tables with old syntax 2020-12-29 20:16:57 +03:00
kssenii
c1eced8993 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg2ch 2020-12-29 15:09:53 +00:00
Alexander Tokmakov
ecc0339472 add test 2020-12-28 23:21:57 +03:00
Nikita Mikhailov
c3288c3fbf Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-28 15:09:37 +03:00
kssenii
8f8920a7ee Add table cache, better drop table 2020-12-27 15:52:15 +00:00
kssenii
00a37404ca Add postgresql database engine 2020-12-27 12:26:29 +00:00
kssenii
d95e8e2d74 Better 2020-12-27 12:18:59 +00:00
kssenii
375e8e9736 Add postgres dictionary source 2020-12-27 12:18:09 +00:00
kssenii
9cb2c75464 Add tests for storage 2020-12-27 12:17:56 +00:00
Vitaly Baranov
9271b3d627 SELECT JOIN now requires the SELECT privilege on each of the joined tables. 2020-12-26 18:35:32 +03:00
alesapin
c9bbde649c
Merge pull request #18312 from ClickHouse/simplier_coverage 
Make single image for building coverage report
 2020-12-25 10:18:39 +03:00
Nikita Mikhaylov
7d8a307bb2
Merge pull request #15316 from nikitamikhaylov/copier-segfault 
Add some extra tests to copier
 2020-12-25 03:08:12 +03:00
alesapin
303c4f98cc Add ability to collect coverage from integration tests 2020-12-24 18:51:53 +03:00
Nikita Mikhailov
63dc8c5462 better 2020-12-24 17:24:51 +03:00
Nikolai Kochetov
34cd4fea0e Fixing integrational test. 2020-12-24 12:05:14 +03:00
alexey-milovidov
c4bbb95b0f
Merge pull request #18405 from ClickHouse/add_test_for_compact_parts 
Add test for already working code
 2020-12-24 01:24:09 +03:00
Nikita Mikhailov
7ac7d991ad replace word anime with word big 2020-12-23 22:02:53 +03:00
Nikita Mikhailov
a35ef5ace3 better 2020-12-23 21:59:27 +03:00
alesapin
e3d9f4739d
Update test.py 2020-12-23 15:06:21 +03:00
alesapin
28aef2ddd4
Update test.py 2020-12-23 15:05:55 +03:00
alesapin
af53559b4f
Update test.py 2020-12-23 15:05:25 +03:00
alesapin
5fc3da74d0
Update test.py 2020-12-23 15:04:29 +03:00
alesapin
54455b4740 Add test for already working code 2020-12-23 14:53:49 +03:00
Nikita Mikhailov
dcfbe782c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-23 05:20:22 +03:00
Nikita Mikhaylov
1e12290e2d
Update task_many_to_one.xml 2020-12-23 04:26:23 +03:00
nikitamikhaylov
ffce2a490b done 2020-12-23 01:46:40 +03:00
Amos Bird
c974fb8948
Remove obtrusive tests 2020-12-22 18:42:18 +08:00
alexey-milovidov
345b572c2e
Merge branch 'master' into tavplubix-patch-1 2020-12-20 09:42:59 +03:00
alexey-milovidov
7ec772f480
Merge pull request #18129 from ClickHouse/boringssl2 
Port to BoringSSL (part 2)
 2020-12-18 18:00:10 +03:00
alexey-milovidov
0d9519a0a2
Merge pull request #17567 from ClickHouse/insert_quorum_parallel-default 
Insert quorum parallel by default
 2020-12-18 08:08:26 +03:00
Alexey Milovidov
6c62301894 Merge branch 'master' of github.com:yandex/ClickHouse into boringssl2 2020-12-18 07:59:18 +03:00
Alexey Milovidov
56054b90f6 Change format of dhparam in integration test 2020-12-18 06:35:20 +03:00
Nikita Mikhailov
6a55e018e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-18 02:25:32 +03:00
tavplubix
8bc9d8ed7e
Merge pull request #18198 from ClickHouse/materialize_mysql_optimize_on_insert 
Enable optimize_on_insert for MaterializeMySQL
 2020-12-18 01:50:32 +03:00
alesapin
799997db35
Merge pull request #16621 from arenadata/ADQM-148 
HDFS configuration and Kerberos Authenticaiton
 2020-12-17 23:16:58 +03:00
Vitaly Baranov
b9c537ac37
Merge pull request #18104 from vitlibar/fix-access-rights-for-merge-table-function-2 
Fix access rights required for the merge() table function #2.
 2020-12-17 22:59:33 +03:00
alesapin
82ed528d98
Merge pull request #18191 from ClickHouse/hard_watches_test 
Test watches for in-memory zookeeper
 2020-12-17 22:51:52 +03:00
Alexey Milovidov
e4433157e7 Merge branch 'master' into insert_quorum_parallel-default 2020-12-17 22:42:32 +03:00
Alexey Milovidov
c2b0f9d530 Update integration test 2020-12-17 22:42:15 +03:00
Nikita Mikhailov
fbf2ac35e8 fix tsan 2020-12-17 18:14:09 +03:00
Alexander Tokmakov
52ef4289fe enable optimize_on_insert for MaterializeMySQL 2020-12-17 16:05:38 +03:00
alesapin
fa266790b4 Better test 2020-12-17 14:29:29 +03:00
tavplubix
7e73b6acd8
Merge pull request #17642 from filimonov/detach-permanently 
DETACH TABLE PERMANENTLY
 2020-12-17 14:26:51 +03:00
alesapin
b6bd84df84 One more test for watches 2020-12-17 14:25:59 +03:00
Vitaly Baranov
da18af96a3 Fix access rights required for the merge() table function. 2020-12-17 12:29:07 +03:00
alesapin
66228f4054
Merge pull request #18065 from ClickHouse/fix_watch_race_testkeeper 
Remove dead list watches from TestKeeperStorage
 2020-12-17 10:15:30 +03:00
alesapin
422467628a Merge branch 'master' into fix_watch_race_testkeeper 2020-12-16 19:45:57 +03:00
Anton Ivashkin
0f0500ca0c Merge master 2020-12-16 18:31:13 +03:00
Mikhail Filimonov
f14cafee51
Fix tests 2020-12-16 16:06:43 +01:00
Pervakov Grigorii
4edbb42ba1 Use IDisk in Set and Join storages 2020-12-16 14:58:44 +03:00
Ilya Golshtein
fc0a1af214 doc minor changes, cleanup, krb5-user as a recommended package 2020-12-16 14:48:33 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00
Ilya Golshtein
d1d657335b USE_INTERNAL_HDFS3_LIBRARY + bash lint 2020-12-16 14:48:33 +03:00
Ilya Golshtein
d9166a0645 HADOOP_SECURE_DN_USER way, kinit thread, docker capabilities 2020-12-16 14:48:33 +03:00
Ilya Golshtein
eb10948ca2 kerberized_hdfs cluster started 2020-12-16 14:48:33 +03:00
Ilya Golshtein
185ac744b1 kerberized hdfs compiled 2020-12-16 14:48:33 +03:00
Alexey Milovidov
62ce33153a Added valuable comment 2020-12-16 06:34:25 +03:00
alesapin
dff71850a8 Merge branch 'master' into fix_watch_race_testkeeper 2020-12-15 19:04:24 +03:00
alexey-milovidov
22aba554c0
Revert "Fix access rights required for the merge() table function." 2020-12-15 18:07:20 +03:00
Nikolai Kochetov
6defcbb662
Merge branch 'master' into optimize-data-on-insert 2020-12-15 16:50:42 +03:00
alesapin
9d17f01dc9 Merge branch 'master' into fix_watch_race_testkeeper 2020-12-15 13:41:04 +03:00
Vitaly Baranov
5a8c750b0b
Merge pull request #17983 from vitlibar/fix-access-rights-for-merge-table-function 
Fix access rights required for the merge() table function.
 2020-12-15 09:17:52 +03:00
alexey-milovidov
363ff79cca
Merge pull request #17934 from Jokser/s3-adaptive-single-part-upload 
Adaptive choose of single/multi part upload in WriteBufferFromS3
 2020-12-14 23:00:43 +03:00
alesapin
73780838a1 Remove dead list watches 2020-12-14 19:01:29 +03:00
tavplubix
dd2ae6926d
Merge pull request #14849 from ClickHouse/allow_atomic_database_inside_materialize_mysql 
Allow Atomic database inside MaterializeMySQL
 2020-12-14 16:27:13 +03:00
alexey-milovidov
d55c5bec94
Merge pull request #18053 from azat/librdkafka-snappy-UB-fix 
Fix possible corruption in librdkafka snappy decompression
 2020-12-14 13:59:11 +03:00
Vitaly Baranov
a98e2311f8
Merge pull request #17908 from vitlibar/fix-checking-introspection-grants 
Fix checking introspection grants
 2020-12-14 12:10:17 +03:00
Pavel Kovalenko
346c241c20 Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload 
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
 2020-12-14 11:00:30 +03:00
Azat Khuzhin
2e569899d1 Add a test for kafka with snappy compression method (regression for UB in snappy) 
I've checked it manually and it works with clang build and does not with
gcc (there are NULL bytes in the middle of value).
 2020-12-14 00:38:07 +03:00
Vitaly Baranov
9310437920 Fix access rights required for the merge() table function. 2020-12-13 23:34:53 +03:00
Alexander Tokmakov
7f7eed4031 fix test 2020-12-13 17:57:15 +03:00
Alexander Tokmakov
0c80fce7f0 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-13 17:56:50 +03:00
Pavel Kovalenko
c2c497e290 Use s3 disk explicitly for log tables in test_log_family_s3. 2020-12-12 16:44:19 +03:00
alexey-milovidov
656a9222ac
Merge pull request #18000 from ClickHouse/filimonov-kafka-flaky-test 
kafka test_premature_flush_on_eof flap
 2020-12-12 02:29:26 +03:00
Pavel Kovalenko
59a5e63417 Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload 
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
#	src/Disks/S3/DiskS3.h
#	src/Disks/S3/registerDiskS3.cpp
#	src/IO/WriteBufferFromS3.cpp
#	src/IO/WriteBufferFromS3.h
 2020-12-11 18:28:41 +03:00
Alexander Tokmakov
ed355f8663 fix 2020-12-11 16:50:45 +03:00
Vitaly Baranov
1b3893bcab User with allow_ddl=0 cannot do DDL but can grant DDL. 2020-12-11 16:38:49 +03:00
Vitaly Baranov
648be453a4 User with allow_introspection_functions=0 cannot call introspection functions but can grant INTROSPECTION. 2020-12-11 16:38:49 +03:00
Alexander Tokmakov
58a31e4882 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-11 16:14:03 +03:00
filimonov
8e2ca9518b
Update test.py 
adjusting timeouts a bit, in the good hope that it will prevent flakiness of the test.
 2020-12-11 10:13:03 +01:00
Kruglov Pavel
9a785dbbf5
Merge branch 'master' into disk-s3-send-metadata 2020-12-10 20:17:41 +03:00
Vladimir Chebotarev
39d867e56d
Added proper authentication for S3 client (#16856) 2020-12-10 12:19:42 +03:00
alesapin
850f903d40 Merge branch 'master' into complete_zk_api 2020-12-09 18:18:38 +03:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
Pavel Kovalenko
dfbb31174e Ability to set metadata when put S3 object 2020-12-08 21:31:57 +03:00
alesapin
8b0fe1404a Tiny sleeps 2020-12-08 17:50:10 +03:00
alesapin
3d0dbbe411 Fix build 2020-12-08 17:49:18 +03:00
alesapin
91e3162e09 More correct session close 2020-12-08 17:28:51 +03:00
Ivan
0ba712a2d3
Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-08 16:45:51 +03:00
alesapin
f4231d552e Banch of fixes 2020-12-08 16:28:39 +03:00
Kruglov Pavel
e19eb6f17a
Merge branch 'master' into optimize-data-on-insert 2020-12-08 15:57:46 +03:00
Vitaly Baranov
ad01e877be Mark grpc protocol's tests as non-flaky. 2020-12-07 23:40:02 +03:00
Vitaly Baranov
559afe028f Retrieve logs from grpc library and pass them to our logging system. 2020-12-07 23:39:55 +03:00
Vitaly Baranov
b8faa1663f Fix configs in tests. 2020-12-07 22:02:18 +03:00
alesapin
b5d2559626
Merge pull request #17709 from kssenii/better-startup 
rabbitmq: fix startup with no connection
 2020-12-07 19:08:18 +03:00
alesapin
710550f8e5 Kill container with retries 2020-12-07 12:25:27 +03:00
Azat Khuzhin
ea64ec390f Fix flaky test_ttl_move 
- replace sleep() with waiting for the message in log
- due to previous paragaph, use unique table names
- increase sleep timeout in some places
 2020-12-04 21:27:50 +03:00
Pavel Kruglov
905ba78adc Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-12-04 18:56:46 +03:00
alesapin
7c2a04d77d
Merge pull request #17671 from ClickHouse/update_integration_tests_image 
Update docker image in integration tests runner
 2020-12-04 15:51:08 +03:00
kssenii
ebab21178e Fix, add test 2020-12-02 18:45:30 +00:00
Alexander Tokmakov
a0ef3a113a Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-02 18:29:02 +03:00
tavplubix
7e2f7ba623
trigger CI 2020-12-02 18:03:15 +03:00
alesapin
5065d9c64b Merge branch 'master' into update_integration_tests_image 2020-12-02 13:58:24 +03:00
TCeason
08d36f862d
Merge branch 'master' into add_mysql_killed_test_for_materializemysql 2020-12-02 13:25:07 +08:00
alesapin
51f49e3d47 Check output of cluster start cmd 2020-12-01 23:23:32 +03:00
alexey-milovidov
de77715cb6
Merge pull request #17662 from vitlibar/mark-grpc-protocol-tests-as-flaky 
Mark grpc protocol's tests as flaky.
 2020-12-01 22:05:17 +03:00
Alexander Tokmakov
19c8399eb0 Merge branch 'master' into database_replicated 2020-12-01 17:33:07 +03:00
tavplubix
0b5261b1af
Merge pull request #17622 from TCeason/add_clickhouse_killed_test_for_materializemysql 
add integration test: ClickHouse killed while insert for MaterializeMySQL ENGINE
 2020-12-01 12:52:52 +03:00
Vitaly Baranov
7f093c4f9a Mark grpc protocol's tests as flaky. 2020-12-01 12:38:51 +03:00
Alexander Tokmakov
ab197a49c8 better code, fixes 2020-11-30 23:12:43 +03:00
taichong
5b9d9376c3 add integration test: ClickHouse killed while insert for MaterializeMySQL ENGINE 2020-11-30 17:33:31 +08:00
taichong
6d8cf30f5d add integration test: MySQL killed while insert for MaterializeMySQL ENGINE 2020-11-30 16:31:09 +08:00
Alexey Milovidov
1711d428eb Fix GRPC tests 2020-11-30 09:34:46 +03:00
alesapin
1d179dac22
Merge pull request #17486 from ClickHouse/fix_more_flaky_tests 
Fix more flaky integration tests
 2020-11-28 10:43:29 +03:00
tavplubix
aaaa7ad59f
Merge pull request #16806 from TCeason/add_test_for_materialize_mysql 
add network partition integration test for MaterializeMySQL
 2020-11-27 20:39:06 +03:00
Alexander Tokmakov
9e3fd3c170 Merge branch 'master' into database_replicated 2020-11-27 17:08:34 +03:00
Alexander Tokmakov
f1a52a609b separate DatabaseReplicatedDDLWorker 2020-11-27 17:04:03 +03:00
Nikita Mikhaylov
0b6f5c75b9
Merge pull request #17435 from vitlibar/grpc-protocol-2 
Implement GRPC protocol (corrections)
 2020-11-27 15:19:50 +03:00
alesapin
17fa29f44f Less pulls of network image 2020-11-27 14:38:04 +03:00
alesapin
f12ba9f2c1 Fix test 2020-11-27 14:26:42 +03:00
alesapin
50cf7fda39 Less flaky test_realod_zookeeper 2020-11-27 14:05:59 +03:00
alexey-milovidov
dfae1efbbd
Merge pull request #17070 from fastio/master 
Support multiple ZooKeeper clusters
 2020-11-27 10:38:01 +03:00
taichong
c8f7a56a25 fix CI timeout err 2020-11-27 11:34:49 +08:00
Nikita Mikhaylov
084c75fa6e
Merge pull request #17309 from nikitamikhaylov/merging-sequential-consistency 
Merging #16309
 2020-11-26 15:26:05 +03:00
Vitaly Baranov
1fc43b3c93 Remove trailing whitespaces in config.xml 2020-11-26 10:27:04 +03:00
Vitaly Baranov
49cf980761 Use port 9100 for grpc by default. 2020-11-26 10:14:27 +03:00
Nikita Mikhaylov
27acf6462f
Merge pull request #17248 from kaka11chen/15235 
Fix #15235. When clickhouse-copier handle non-partitioned table, throws segfault error.
 2020-11-25 14:31:59 +03:00
alesapin
3e1b2f515b
Merge pull request #17311 from ClickHouse/fix_integration_tests 
Fix some flaky tests
 2020-11-25 10:09:27 +03:00
chenqi
302cd55f45 Fix #15235. When clickhouse-copier handle non-partitioned table, throw segfault error. 2020-11-25 11:52:00 +08:00
Vitaly Baranov
4b3488934c Disable running test test_odbc_interaction.test_bridge_dies_with_parent with asan. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
3c85165e60 Redirect stderr, stdout for clickhouse-odbc-bridge to files. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
950bccb130 Add test for authentication. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
a0e384b0c0 Added support for external tables. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
23842e7ac6 Added support for input() function. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
0e3a8840b5 Support cancellation of executing query via gRPC. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
88f1eeec2f Add tests for parallel executing of queries. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
eab3006a4d Support SSL and compression when using gRPC protocol. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
b0cb3eb306 Extend the protocol with streaming and nonstreaming functions. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
b51e14253d Added support for sessions in gRPC protocol. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
9285f7edc1 Send profile info to client. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
98e2cc4117 Fix sending progress. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
4f0405af93 Send logs via gRPC protocol too. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
218d9ea3e8 Added input_data_delimiter to protocol. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
797c84889f Use ReadBuffer to join input data from multiple query infos. 
That fixes inserting of a row splitted between two query infos.
 2020-11-24 17:55:01 +03:00
Vitaly Baranov
63c8d8124a Added test for inserting to columns with defaults. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
de4586739a Fix using output format. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
8146093794 Provide more information about errors. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
a327f24e3c Refine the protocol. Code cleanup in tests. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
db9f762e73 Fix tests. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
13f2352c57 Move files to right places. 2020-11-24 17:53:34 +03:00
mnkonkova
6cd1557d67 Implement GRPC protocol. 2020-11-24 17:53:34 +03:00
Alexander Tokmakov
e82c63e05c fixes 2020-11-24 15:28:54 +03:00
Alexander Tokmakov
dce0bf8efc Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-11-24 15:28:31 +03:00
Nikita Mikhaylov
6787f98476
Merge pull request #14958 from nikitamikhaylov/refactor_cache_dictionary 
Refactor cache dictionary
 2020-11-24 14:55:02 +03:00
nikitamikhaylov
5be300ff22 Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency 2020-11-24 14:46:57 +03:00
Alexander Tokmakov
5ccf975a4a Merge branch 'master' into database_replicated 2020-11-24 14:38:02 +03:00
Alexander Tokmakov
dad21ee684 maintain metadata in zk 2020-11-24 13:24:39 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects 
S3 max redirects
 2020-11-24 12:13:10 +03:00
alesapin
b05385cec0 Remove code from test which copy-pasted from another test 2020-11-24 11:17:37 +03:00
alesapin
cb234e28ea Fix garbage test 2020-11-24 11:09:10 +03:00
alesapin
5477984856 Longer integration tests for rabbitMQ 2020-11-24 10:33:35 +03:00
taichong
b5440882cd remove kill server while insert 2020-11-24 10:11:04 +08:00
alesapin
db1aef8994 More retries 2020-11-23 23:44:46 +03:00
alesapin
3d177d414a Merge branch 'master' into fix_integration_tests 2020-11-23 23:27:46 +03:00
alexey-milovidov
ac710ff0b7
Merge pull request #17299 from Jokser/flaky-s3-test-custom-auth-headers 
Fixed flaky test_storage_s3::test_custom_auth_headers
 2020-11-23 20:33:48 +03:00
alesapin
1c5da1c178 Better timeouts 2020-11-23 19:39:57 +03:00
alesapin
c9b382b4af Fix flaky integration tests 2020-11-23 17:40:32 +03:00
nikitamikhaylov
065cd00257 better 2020-11-23 17:24:32 +03:00
nikitamikhaylov
68bef22fda Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency 2020-11-23 16:28:35 +03:00
tavplubix
7af89cba8a
Merge pull request #16715 from zhang2014/fix/ISSUES-16605 
ISSUES-16605 fix without returning affected rows when insert select query in MySQL handler
 2020-11-23 15:54:43 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Pavel Kovalenko
5f98d38689 Fixed flaky test_storage_s3::test_custom_auth_headers 2020-11-23 13:19:43 +03:00
taichong
a6c2a90756 modify integration test materialize mysql database 
1. fix asan test err

2. reset pymysql conn if ping err

3. finish mysql & ck killed test

4. modify check_query
 2020-11-23 17:55:07 +08:00
taichong
f99d6ea799 add ck & mysql killed while insert 2020-11-23 15:47:44 +08:00
taichong
122740767f add integration test err 2020-11-23 15:47:44 +08:00
taichong
714767f491 make it normal 2020-11-23 15:47:44 +08:00
taichong
5a50535abc add mysql kill sync id test 2020-11-23 15:47:44 +08:00
taichong
81bdf50085 add some except error test 2020-11-23 15:47:44 +08:00
taichong
e691b4a0d4 add network partition integration test for MaterializeMySQL 2020-11-23 15:47:44 +08:00
taichong
e8628f61e0 add mysql kill sync id test 2020-11-23 15:47:44 +08:00
taichong
3287f691ee add some except error test 2020-11-23 15:47:44 +08:00
taichong
5c3d4b2245 add network partition integration test for MaterializeMySQL 2020-11-23 15:47:44 +08:00
alesapin
9c8b0da382
Merge pull request #16033 from nvartolomei/nv/parts-uuid 
Add unique identifiers IMergeTreeDataPart structure
 2020-11-22 16:13:19 +03:00
zhang2014
e1e5eede86 ISSUES-16605 try fix integration failure 2020-11-22 20:13:56 +08:00
zhang2014
5b3154b298 ISSUES-16605 try fix integration test failure 2020-11-22 20:13:56 +08:00
zhang2014
a6dbba5932 ISSUES-16605 add integration test 2020-11-22 20:13:56 +08:00
Nicolae Vartolomei
7c8bc1c04e Use JSON metadata in WAL 2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
94293ca3ce Assign UUIDs to parts only when configured to do so 
Avoid breaking backwards compatibility by default for now.
 2020-11-20 13:49:17 +00:00
Pavel Kruglov
e5588b94aa Fix style and tests 2020-11-20 13:38:53 +03:00
tavplubix
3e335c1e95
Merge pull request #17091 from zhang2014/fix/binlog_checksum 
Fix MaterializeMySQL SYNC failure when modify binlog_checksum
 2020-11-20 13:25:14 +03:00
Anton Ivashkin
a7d4f4be65 Add 's3_max_redirects' test 2020-11-20 11:18:44 +03:00
Pavel Kruglov
608722b6ab Fix integration tests 2020-11-20 02:59:58 +03:00
Nicolae Vartolomei
425dc4b11b Add unique identifiers IMergeTreeDataPart structure 
For now uuids are not generated at all, they are present only if the
part is updated manually (as you can see in the integration test).

The only place where they can be seen today by an end user is in
`system.parts` table. I was looking for hiding this column behind an
option but couldn't find an easy way to do that.

Likely this is also required for WAL, but need to think how not to break
compatibility.

Relates to #13574, https://github.com/ClickHouse/ClickHouse/issues/13574

Next 1: In the upcoming PR the plan is to integrate de-duplication based on
these fingerprints in the query pipeline.

Next 2: We'll enable automatic generation of uuids and come up with a
way for conditionally sending uuids when processing distributed queries
only when part movement is in progress.
 2020-11-19 13:14:25 +00:00
tavplubix
8b6fbea199
trigger CI 2020-11-19 12:47:24 +03:00
Peng Jian
3c86c8b3c9 fix test cases 2020-11-19 15:44:47 +08:00
Peng Jian
d8ae52118b add test 2020-11-19 15:44:47 +08:00
zhang2014
9df9691ccb Try fix integration test 2020-11-19 13:01:23 +08:00
nikitamikhaylov
13e711b27e rename back config 2020-11-18 16:38:14 +03:00
nikitamikhaylov
70a08608f8 add reload dictionary 2020-11-17 20:58:31 +03:00
nikitamikhaylov
663730c758 try to fix flaky test 2020-11-17 20:33:07 +03:00
nikitamikhaylov
9c6b896928 style + flaky test fix 2020-11-17 17:36:04 +03:00
nikitamikhaylov
31b1f644c1 better 2020-11-17 16:59:13 +03:00
nikitamikhaylov
fbd0d14dd6 save 2020-11-17 16:59:12 +03:00
nikitamikhaylov
9b46b53fa3 better defaults 2020-11-17 16:59:12 +03:00
nikitamikhaylov
028fc00166 fix tests 2020-11-17 16:59:12 +03:00
zhang2014
b4fc2ecc00 Try fix integration test 2020-11-17 09:41:35 +08:00
alesapin
5dd32819f5 Merge branch 'master' into fix_ddl_worker_non_leader 2020-11-16 22:34:16 +03:00
Winter Zhang
f6a205d64b
Merge branch 'master' into fix/binlog_checksum 2020-11-16 21:53:22 +08:00
zhang2014
6ef93ac73c Try fix MaterializeMySQL SYNC with modify binlog_checksum 2020-11-16 21:46:36 +08:00
alesapin
6759932c61 Remove accident changes 2020-11-16 15:49:08 +03:00
alesapin
f839742288 Add a test 2020-11-16 15:47:02 +03:00
TCeason
45bacd6f74 Merge branch 'master' into add_sync_user_privs_exception 2020-11-16 12:59:44 +08:00
ubuntu
f7b60c17f5 make test stable 2020-11-13 20:38:28 +08:00
alesapin
8d3858fc22
Merge pull request #16895 from CurtizJ/remove-empty-parts 
Remove empty parts after they were pruned by TTL [2]
 2020-11-13 10:00:18 +03:00
taichong
829037b487 Merge branch 'add_sync_user_privs_exception' of https://github.com/TCeason/ClickHouse into add_sync_user_privs_exception 2020-11-13 13:26:19 +08:00
Alexander Tokmakov
0731bfb7be Merge branch 'master' into database_replicated 2020-11-12 21:04:18 +03:00
Anton Popov
4cecd89e84 fix dropping of empty parts 2020-11-12 20:37:54 +03:00
Alexander Tokmakov
2a6c0b9180 try reuse DDLWorker in DatabaseReplicated 2020-11-12 20:35:29 +03:00
taichong
fefa2a2548 add log std out 2020-11-12 22:24:22 +08:00
Anton Popov
0aa35d53c7 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-12 14:31:58 +03:00
alesapin
a8d466a645 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-12 12:51:02 +03:00
taichong
091df85cdc modify test db name 2020-11-12 09:49:12 +08:00
TCeason
3912ed4965 Merge branch 'master' into add_sync_user_privs_exception 2020-11-12 09:34:08 +08:00
taichong
e2a9fe44b0 modify test error 2020-11-12 08:17:13 +08:00
Anton Popov
2497598a49 Remove empty parts after they were pruned by TTL 2020-11-11 19:19:43 +03:00
Pavel Kovalenko
e311759c5d
Abort multipart upload if no data was written to WriteBufferFromS3. (#16840) 2020-11-11 15:15:16 +03:00
taichong
f4f4379168 modify test case add precheck 2020-11-11 19:36:09 +08:00
taichong
6c8b5b573b modify test for Integration Tests Flaky Check 2020-11-11 19:36:09 +08:00
taichong
e5da2d4086 print mysql result for test 2020-11-11 19:35:39 +08:00
taichong
64acfea79c throw exception when mysql sync user privs error. 2020-11-11 19:32:28 +08:00
taichong
5c1a01897a Revise the code according to the review 2020-11-11 19:27:37 +08:00
taichong
aecb9716e1 print mysql result for test 2020-11-11 19:27:37 +08:00
taichong
2f6510d75b try to fix ci failed 2020-11-11 19:27:37 +08:00
taichong
d907afa77e throw exception when mysql sync user privs error. 2020-11-11 19:27:37 +08:00
alesapin
637e3dc2c2
Merge pull request #16627 from amosbird/zkc 
reload auxiliary zookeepers configuration
 2020-11-11 10:00:22 +03:00
alesapin
2e357516a6 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-10 13:53:08 +03:00
Vladimir Chebotarev
059357d51e
ALTER UPDATE/DELETE ... IN PARTITION with partition pruning in ReplicatedMergeTree (#13403) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-11-10 13:23:46 +03:00
Amos Bird
aa8e6db786
reload auxiliary zookeepers configuration 2020-11-10 15:56:37 +08:00
alesapin
a056379ec1 Remove duplicate code 2020-11-09 12:25:50 +03:00
alesapin
cedaa938f3 Add simple integration test 2020-11-09 12:10:32 +03:00
alexey-milovidov
7acfa7d407
Merge pull request #16728 from azat/enable-use_compact_format_in_distributed_parts_names-by-default 
Enable use_compact_format_in_distributed_parts_names by default
 2020-11-07 13:40:54 +03:00
alexey-milovidov
712a35bb7b
Merge pull request #16762 from filimonov/kafka-protobuf-issue12615 
Test for the issue #12615
 2020-11-07 13:40:19 +03:00
Azat Khuzhin
ac8df08186 Set use_compact_format_in_distributed_parts_names=0 for some integration tests 
To avoid depends from default.
 2020-11-06 21:02:44 +03:00
Mikhail Filimonov
b072bc7d2b
Test for the issue #12615 2020-11-06 17:54:44 +01:00
alexey-milovidov
654d127ce6
Merge pull request #16630 from sundy-li/patch-6 
better change the config
 2020-11-04 11:36:07 +03:00
alexey-milovidov
09805399d9
Merge pull request #16560 from azat/test_input_format_parallel_parsing_memory_tracking-flackiness 
Fix test_input_format_parallel_parsing_memory_tracking flackiness
 2020-11-04 01:37:25 +03:00
sundy-li
dea4bd483e remove unused codes2 2020-11-03 19:40:45 +08:00
sundy-li
9cfeb9fd32 remove unused codes 2020-11-03 18:13:48 +08:00
sundy-li
72bde1c688 better change the config 2020-11-03 17:32:31 +08:00