thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-27 10:02:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
60,513 Commits 9,196 Branches 1,782 Tags 2.1 GiB
03ce320926
Commit Graph

1709 Commits

Author SHA1 Message Date
Azat Khuzhin
a795db57f1 tests: decrease distributed_ddl_task_timeout to 60 in test_distributed_ddl_parallel 2021-02-28 05:45:11 +03:00
Azat Khuzhin
06e8bb955d tests: catch exceptions in threads in test_distributed_ddl_parallel 
Otherwise the test will not report an error.
 2021-02-28 05:45:11 +03:00
Azat Khuzhin
b899d8d919 tests: regression for test for invalid reference to zookeeper for distributed_ddl.pool_size > 1 2021-02-28 05:45:11 +03:00
alexey-milovidov
9f0400f015
Merge pull request #21171 from azat/tests-bump-zookeeper 
Bump zookeeper version to 3.6.2 in tests
 2021-02-28 02:35:58 +03:00
alexey-milovidov
b8fba768e5
Merge pull request #21264 from ClickHouse/fix_zookeeper_update 
Fix several bugs with ZooKeeper client
 2021-02-28 01:57:04 +03:00
Pavel Kruglov
197058dc12 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-27 00:36:33 +03:00
Azat Khuzhin
bc1a316b28 Bump zookeeper version to 3.6.2 in tests 
3.4.12 was released 1.05.2018
 2021-02-26 23:41:43 +03:00
alesapin
a48bb7e3a0 Fix redundant zookeeper reconnect and possibility of multiple zookeeper session for a single ClickHouse server 2021-02-26 22:05:25 +03:00
alexey-milovidov
5ecd9a9577
Merge pull request #21111 from filimonov/better-kafka-tests 
Better kafka tests
 2021-02-26 21:30:35 +03:00
Pavel Kruglov
91f6f9227b Small fixes and tests updates 2021-02-26 18:55:58 +03:00
tavplubix
09f62447af
Merge pull request #21060 from mxzlxy/fix-bug-for-replicated-engine-parameter 
fix default_replica_path and default_replica_name values are useless …
 2021-02-26 17:18:56 +03:00
Anton Ivashkin
5b267b7eec Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-02-26 13:15:18 +03:00
Anton Ivashkin
c891cf4557 Fixes by review response 2021-02-26 12:48:57 +03:00
Mikhail Filimonov
9e3547681b
try to address python kafka client disconnects better 2021-02-26 07:17:31 +01:00
Mikhail Filimonov
60ede36068
simple fix for CI failure 2021-02-26 06:41:11 +01:00
Mikhail Filimonov
c4470ff283
proper placement for test 2021-02-26 06:38:14 +01:00
Alexander Tokmakov
d46fee0f04 Merge branch 'master' into stress_test_results 2021-02-26 00:24:22 +03:00
Alexander Tokmakov
f42beb4e6c add test 2021-02-25 20:09:39 +03:00
alexey-milovidov
6a5ab40988
Merge pull request #21119 from fastio/bugfix_drop_metadata_from_auxiliary 
fix the metadata leak when the Replicated*MergeTree with custom ZooKeeper cluster is dropped
 2021-02-25 19:24:58 +03:00
Alexander Tokmakov
fbbb0c4b69 fix 2021-02-25 19:11:43 +03:00
Mikhail Filimonov
2bcfff257c
Remove most of sleeps in tests 2021-02-25 17:02:59 +01:00
Mikhail Filimonov
6c6eaf2a60
Fixes 2021-02-25 17:02:57 +01:00
Mikhail Filimonov
a4c9e62d6d
Integration tests: introduce wait_for_log_line. 
It uses tail -f executed in container, this way i don't need to pull
the file many times, rememeber positions etc.
 2021-02-25 17:02:56 +01:00
Mikhail Filimonov
32fa3dbc99
Integration tests: add IPC_LOCK and SYS_NICE caps 
It is needed to make integration tests work with a installed binary (/usr/bin/clickhouse) mounted in.
Otherwise the binary mounted in have the capability, while the container is not allowing that
and you get cryptic message:
standard_init_linux.go:211: exec user process caused "operation not permitted"
 2021-02-25 17:02:54 +01:00
Pavel Kovalenko
cf8b105d42 Merge remote-tracking branch 'origin/master' into unfreeze-partitions 2021-02-25 15:24:43 +03:00
alesapin
8f81dce32f
Merge pull request #20585 from ClickHouse/persistent_nukeeper_log_storage 
Persistent coordination log storage
 2021-02-25 10:06:03 +03:00
Alexander Tokmakov
65384f43d8 fix 2021-02-25 10:02:45 +03:00
fastio
24321c28f6 add test case 2021-02-25 14:19:58 +08:00
Alexander Tokmakov
efd50a819b fix 2021-02-25 02:09:09 +03:00
Alexander Tokmakov
3632c1d879 add flacky check runner 2021-02-24 19:41:44 +03:00
Pavel Kovalenko
078dfce038 Add ALTER TABLE UNFREEZE command. 2021-02-24 17:28:30 +03:00
Alexander Tokmakov
f41e68d118 fix style check 2021-02-23 23:15:11 +03:00
Alexander Tokmakov
7a6c9ceeef fix 2021-02-23 15:25:28 +03:00
Alexander Tokmakov
7bcec2947f fix 2021-02-23 13:48:06 +03:00
Kseniia Sumarokova
b2850ec94f
Merge branch 'master' into fix-postgres-numeric 2021-02-23 11:40:28 +03:00
alesapin
ddc9fb151e Fix tests 2021-02-23 11:37:27 +03:00
Alexander Tokmakov
d5ea345aa9 add integration tests runner 2021-02-23 10:26:23 +03:00
alesapin
5b888e1193 Merge branch 'master' into persistent_nukeeper_log_storage 2021-02-22 23:52:26 +03:00
alesapin
74a07e406b Even more stable 2021-02-22 10:33:27 +03:00
Winter Zhang
521891edec
Update materialize_with_ddl.py 2021-02-22 12:47:38 +08:00
alexey-milovidov
e1c4ba0169
Merge pull request #21016 from azat/integration-test_materialize_mysql_database 
Preserve mysql logs in test_materialize_mysql_database
 2021-02-21 22:28:19 +03:00
Kruglov Pavel
815b4ff0d9
Merge branch 'master' into hedged-requests 2021-02-21 17:10:24 +03:00
Pavel Kruglov
dd5185c779 Refactor 4.0 2021-02-21 17:03:24 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
Denis Glazachev
c1158b50f4 Merge branch 'master' into row-policy-with-prewhere 
* master: (155 commits)
  Update version_date.tsv after release 20.8.13.15
  Update version_date.tsv after release 20.12.7.3
  Update version_date.tsv after release 21.1.5.4
  Update version_date.tsv after release 21.2.4.6
  fix
  Add test to skip list
  Fix WriteBufferFromHTTPServerResponse usage in other places (add missing finalize())
  Fix WriteBufferFromHTTPServerResponse usage in odbc-bridge
  Update config.xml
  Suppress signed overflow in AggregateFunctionGroupArrayMoving 2
  Update BaseDaemon.cpp
  review suggestions
  Fix bash syntax in 01731_async_task_queue_wait
  Do not use view() in 01731_async_task_queue_wait to fix ANTLR parser
  Increase buffer for uncaught exception / std::terminate
  Even more better
  Fix uncaught exception when HTTP client goes away
  test for decimal ( p , s) in dictionaries
  Just little better
  Fixed style check
  ...
 2021-02-21 13:25:47 +04:00
Denis Glazachev
4f57b68563 Fix the test 2021-02-21 13:22:41 +04:00
Denis Glazachev
dda1eb1f00 Fix the test 2021-02-21 13:17:58 +04:00
zhang2014
4529cdac81 try fix MaterializeMySQL integration test 2021-02-21 14:38:29 +08:00
Azat Khuzhin
d343060dd5 tests/integration: add start_and_wait() in test_materialize_mysql_database 2021-02-20 22:09:01 +03:00
Azat Khuzhin
80f448a466 tests/integration: preserve mysql docker logs in test_materialize_mysql_database 2021-02-20 22:09:01 +03:00
Denis Glazachev
ce6d2ff724 Fix the test 2021-02-20 22:14:12 +04:00
tavplubix
103d569885
Merge branch 'master' into fix_force_drop_materialized_view 2021-02-20 20:41:03 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated 
DatabaseReplicated: continuation of #10485
 2021-02-20 19:39:34 +03:00
alesapin
c61769311e Merge branch 'master' into persistent_nukeeper_log_storage 2021-02-20 10:22:22 +03:00
kssenii
f820047cc8 Fix 2021-02-20 06:52:10 +00:00
alexey-milovidov
b4196c8e68
Merge pull request #20168 from ClickHouse/in-memory-compression 
Compression for Memory tables
 2021-02-20 09:49:37 +03:00
Denis Glazachev
8361904b4d Add cases for throwIf() 2021-02-20 10:47:29 +04:00
Denis Glazachev
2ab643170d Add 'SETTINGS optimize_move_to_prewhere = 0' in WHERE-only SELECTs 2021-02-20 00:45:19 +04:00
tavplubix
057c87f322
Update test.py 2021-02-19 22:32:50 +03:00
alesapin
7ee72dfd0c Missed tests 2021-02-19 22:24:20 +03:00
alexey-milovidov
2299930c31
Merge pull request #20545 from kssenii/rabbit-format-settings 
rabbitmq: add missing format factory settings
 2021-02-19 20:24:17 +03:00
Alexander Tokmakov
5cfd687dfb fix 2021-02-19 19:22:47 +03:00
Denis Glazachev
8cc72b8fe0 Merge branch 'master' into row-policy-with-prewhere 
* master: (160 commits)
  Make Poco HTTP Server zero-copy again (#19516)
  Fixed documentation
  ccache 4.2+ does not requires any quirks for SOURCE_DATE_EPOCH
  Add a function `htmlOrXmlCoarseParse` to extract content from html or xml format string. (#19600)
  Reinterpret function added Decimal, DateTim64 support
  Add test
  Update InterpreterSelectQuery.cpp
  Improved serialization for data types combined of Arrays and Tuples. Improved matching enum data types to protobuf enum type. Fixed serialization of the Map data type. Omitted values are now set by default.
  Log stdout and stderr when failed to start docker in integration tests.
  Added comment
  Don't backport base commit of branch in the same branch (#20628)
  Fix fasttest retry for failed tests
  Dictionary create source with functions crash fix
  Added error reinterpretation tests
  Update run.sh
  Updated documentation
  fix subquery with limit
  Rename untyped function reinterpretAs into reinterpret
  ignore data store files
  Support vhost
  ...
 2021-02-19 19:49:37 +04:00
Denis Glazachev
bfccfd9eec Fix tests 
Add tests
WIP: Failures are legit!
 2021-02-19 19:32:35 +04:00
kssenii
30f0e48777 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-postgres-numeric 2021-02-19 14:53:27 +00:00
Alexey Milovidov
a9d9a6d56f Merge branch 'master' into in-memory-compression 2021-02-19 12:01:33 +03:00
Alexey Milovidov
8e8a81883e Merge branch 'master' into in-memory-compression 2021-02-19 11:59:11 +03:00
Denis Glazachev
879d9206c1 Add tests for row-level security with prewhere 2021-02-19 02:47:28 +04:00
Alexander Tokmakov
8097532cb7 Merge branch 'master' into database_replicated 2021-02-18 17:42:06 +03:00
alesapin
1245710aed Merge branch 'master' into persistent_nukeeper_log_storage 2021-02-18 15:27:01 +03:00
Amos Bird
77fd060665
Normalize function names 2021-02-18 20:20:28 +08:00
kssenii
2aad067e7c Support conversion for postgres numeric without precision and scale 2021-02-18 12:16:58 +00:00
alesapin
7f815325ba More tests for broken changelog read 2021-02-18 14:42:09 +03:00
Kruglov Pavel
142a541361
Merge branch 'master' into hedged-requests 2021-02-18 14:23:00 +03:00
Kruglov Pavel
419962dbfb
Merge pull request #20712 from kssenii/add-test 
Add test for already fixed odbc Postgres date type conversion
 2021-02-18 13:43:17 +03:00
alesapin
5b597fdf44 Force sync setting and ability to start with broken log 2021-02-18 13:23:48 +03:00
Ilya Yatsishin
bc1f564d55
Merge pull request #20631 from vitlibar/verbose-message-on-fail-to-start-docker 
Log stdout and stderr when failed to start docker in integration tests
 2021-02-18 07:54:58 +03:00
Alexey Milovidov
9069318fa6 Merge branch 'master' into in-memory-compression 2021-02-18 00:48:39 +03:00
kssenii
732a1d9ceb Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-test 2021-02-17 21:29:04 +00:00
kssenii
62486d6e06 Add test 2021-02-17 19:02:57 +00:00
Pavel Kruglov
e19f62f645 Merge branch 'hedged-requests' of github.com:Avogar/ClickHouse into hedged-requests 2021-02-17 20:44:32 +03:00
Pavel Kruglov
bc6fe4f28c Minor fixes 2021-02-17 20:44:11 +03:00
Kruglov Pavel
794c3449ca
Merge branch 'master' into hedged-requests 2021-02-17 20:36:36 +03:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
Vitaly Baranov
c704a8cc45 Log stdout and stderr when failed to start docker in integration tests. 2021-02-17 20:07:03 +03:00
Kseniia Sumarokova
20ff183549
Merge branch 'master' into rabbit-format-settings 2021-02-17 19:01:53 +03:00
alesapin
2de201dd5f Merge branch 'fix_fast_test_retry' into persistent_nukeeper_log_storage 2021-02-17 17:51:50 +03:00
Alexander Tokmakov
13ae988eff make force_drop_table work with materialized view 2021-02-17 17:23:07 +03:00
alesapin
499c100b12 Better test 2021-02-17 16:41:43 +03:00
alesapin
e82bd824d7 Fix restart replica in test 2021-02-17 16:07:59 +03:00
alesapin
9396bae2e2 More reliable test keeper tests 2021-02-17 14:53:47 +03:00
Vitaly Baranov
753f5b84d9
Merge pull request #20106 from vitlibar/fix-quota-consuming 
Fix quota consuming
 2021-02-17 12:52:43 +03:00
alexey-milovidov
6363d44488
Merge pull request #20498 from azat/dist-parse-dirs-sigsegv-fix 
Add proper checks while parsing directory names for async INSERT (fixes SIGSEGV)
 2021-02-17 09:43:11 +03:00
kssenii
d79f7c04f6 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-format-settings 2021-02-17 06:03:19 +00:00
Vitaly Baranov
298130402e SYSTEM queries now consume quota. 2021-02-16 23:54:05 +03:00
Vitaly Baranov
f83be158ba SHOW TABLES is now considered as one query in the quota calculations, not two queries. 2021-02-16 23:53:58 +03:00
alesapin
8717dbd0e2 Missed configs 2021-02-16 22:29:09 +03:00
tavplubix
68b427a99d
Merge pull request #20448 from ClickHouse/better_ddl_queue_cleanup 
Better distributed DDL queue cleanup
 2021-02-16 19:29:50 +03:00
Pavel Kruglov
0e1d67ad9a Add LOG_DEBUG to debug test_distributed_load_balancing 2021-02-16 10:56:45 +03:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
kssenii
2a887b9772 Add missing format factory settings 2021-02-15 22:06:10 +00:00
Azat Khuzhin
8d11d09615 Add a test for malformed directores for Distributed async INSERT 2021-02-15 21:00:50 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
Pavel Kruglov
df09a5cac9 Refactor 3.0 and fix tests 2021-02-15 16:21:36 +03:00
Alexander Tokmakov
9c7cf9e92e remove some debug code 2021-02-15 13:26:34 +03:00
alesapin
9686649b02 Fix non-zero session reconnect in integration test 2021-02-15 12:42:50 +03:00
Alexander Tokmakov
320fd6b264 startup without zk 2021-02-15 03:04:46 +03:00
Alexey Milovidov
b13d1f3142 Fix integration test 2021-02-14 04:34:42 +03:00
Denis Glazachev
99a04b6c64 Merge branch 'master' into row-policy-with-prewhere 
* master: (759 commits)
  Suppress UBSan report in Decimal comparison
  Suppress UBSan report in Decimal comparison
  Fix UBSan report in arrayDifference
  Update README.md
  Non significant change in AggregationCommon
  Print stack trace on SIGTRAP
  Fix dependent test
  Fix tests for better parallel run
  Add test for already working code
  Revert "Fix access control manager destruction order"
  Update index.md
  Update index.md
  Update index.md
  Bit more complicated example for isIPv4String - ru
  Bit more complicated example for isIPv4String
  cleanup
  Replace database with ordinary
  Added comments
  Split tests to make them stable
  Fixes
  ...

# Conflicts:
#	src/Storages/MergeTree/MergeTreeRangeReader.cpp
 2021-02-14 02:24:00 +04:00
Alexander Tokmakov
96cc01926c Merge branch 'master' into better_ddl_queue_cleanup 2021-02-14 00:22:09 +03:00
alesapin
f80137626a
Merge pull request #19580 from ClickHouse/in_memory_raft 
In memory coordination inside ClickHouse
 2021-02-13 10:19:23 +03:00
alexey-milovidov
2fe2190a20
Merge pull request #19721 from azat/buffer-profile 
Add separate config directive for Buffer profile
 2021-02-13 02:43:41 +03:00
alexey-milovidov
371e84957a
Merge pull request #20303 from azat/constant-folding-fixes 
Do not allow early constant folding of explicitly forbidden functions
 2021-02-13 02:20:06 +03:00
Alexander Tokmakov
184ec67dac better ddl queue cleanup 2021-02-12 22:23:50 +03:00
Alexander Tokmakov
f64f9b672b fix 2021-02-12 19:22:01 +03:00
alesapin
6aecb62416 Replace database with ordinary 2021-02-12 12:17:10 +03:00
alesapin
c883b7d154 Split tests to make them stable 2021-02-12 11:50:20 +03:00
alexey-milovidov
4e32dbd024
Merge pull request #19317 from abyss7/dwarf-folly 
Print inline frames for fatal stacktraces
 2021-02-12 02:38:50 +03:00
Alexander Tokmakov
c9cf63e958 fix 2021-02-12 01:23:40 +03:00
alesapin
df181b534e Better connection reset 2021-02-11 23:29:01 +03:00
alesapin
5f32f65211 Merge branch 'master' into in_memory_raft 2021-02-11 23:09:29 +03:00
Azat Khuzhin
3993ad6f01 Fix test_system_merges by using mutations_sync=1 
After early_constant_folding started to ignore not only ignore(), but
all functions with isSuitableForConstantFolding() == false, there became
more sleep(2) calls for this test:
- MergeTreeDataSelectExecutor::readFromParts -> DB::KeyCondition::KeyCondition
- MergeTreeDataMergerMutator::mutatePartToTemporaryPart -> DB::isStorageTouchedByMutations -> FilterTransform::transform
- MergeTreeDataMergerMutator::mutatePartToTemporaryPart -> DB::MergeTreeDataMergerMutator::mutateAllPartColumns -> FilterTransform::transform

While before it was optimized to 0 during WHERE analysis.
 2021-02-11 22:46:56 +03:00
alesapin
99a471e047 Add ability to start as follower 2021-02-11 13:25:10 +03:00
alesapin
0acd018361 Fix typo 2021-02-11 12:58:02 +03:00
alesapin
74630acff5 More debug in test 2021-02-11 12:49:49 +03:00
alesapin
b49b7f859d Simplify startup with fixed config 2021-02-11 12:17:57 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Azat Khuzhin
e2d5972eca Cover buffer_profile config directive 2021-02-10 22:44:26 +03:00
Ivan
bc58f4827f
Increase timeout in tests 2021-02-10 17:45:45 +03:00
alesapin
c95140d906 Better startup and non-verbose logging by default 2021-02-10 12:28:53 +03:00
Ilya Yatsishin
1fdc59b453
Merge pull request #20248 from qoega/integration-2 2021-02-10 08:46:20 +03:00
alesapin
7848f0202c One more test 2021-02-10 00:02:34 +03:00
alesapin
51c221f993 Fix outdated session kill 2021-02-09 21:29:06 +03:00
Yatsishin Ilya
3ba6629d7b add stable sort for tests 2021-02-09 18:48:33 +03:00
Yatsishin Ilya
eac13d9d9d Add retries to test_access_control_on_cluster 2021-02-09 18:37:33 +03:00
Alexander Tokmakov
2c6a0e74fb better replica creation 2021-02-09 18:14:20 +03:00
alesapin
b130fbfd78 Add coordination settings 2021-02-09 17:47:18 +03:00
alesapin
9667bdcbd2 Merge branch 'master' into in_memory_raft 2021-02-09 16:06:31 +03:00
Pavel Kruglov
46840b0a4f Add __init__.py 2021-02-09 00:47:13 +03:00
Nicolae Vartolomei
84489b8243 Improve replicated fetches timeouts test and make it 3x faster 2021-02-08 21:19:32 +00:00
Alexander Tokmakov
5e0cbfe972 Merge branch 'master' into database_replicated 2021-02-08 23:01:12 +03:00
Nicolae Vartolomei
6181daf92d Workaround for drop not finishing if it is started while table is readonly 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
f39eef9227 Reformat test and drop tables at the end to reduce the size of test results archive 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
165ba59a15 Increase network delay and add more info to logs for debugging 
CI behaviour is different from my test environment
 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
75fca08b2a Update test_replicated_fetches_network_partition with new settings 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
b153e8c190 Add support for custom fetchPart timeouts 2021-02-08 19:44:02 +00:00
Nicolae Vartolomei
813092ff55 Test replicated fetches_network partition 2021-02-08 19:44:02 +00:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
Ivan Lezhankin
00bb0e6b35 Skip send_crash_reports test with TSAN 2021-02-08 18:46:48 +03:00
Ivan Lezhankin
a287443438 Merge remote-tracking branch 'upstream/master' into dwarf-folly 2021-02-08 18:36:41 +03:00
alesapin
2413d6bd38 Test multinode watches 2021-02-08 16:26:06 +03:00
Pavel Kruglov
12b925dd7a Small update 2021-02-08 14:06:45 +03:00
Anton Ivashkin
1b6ce056a5 Merge master 2021-02-08 12:34:20 +03:00
alesapin
d5ee7d33e9 Merge branch 'master' into in_memory_raft 2021-02-08 10:30:31 +03:00
Vladimir Chebotarev
098a6d6a05 Added test for S3 table function. 2021-02-08 09:22:00 +03:00
JackyWoo
77c0f0a0e7 add quota type QUERY_SELECTS and QUERY_INSERTS 2021-02-07 10:57:18 +08:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
tavplubix
1c5d6bbe44
Merge pull request #20051 from stigsb/materializemysql-system-tables-fix 
Show details of MaterializeMySQL tables in `system.tables`
 2021-02-06 14:50:11 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID 
CLICKHOUSE-606: query deduplication based on parts' UUID
 2021-02-05 22:47:34 +03:00
Anton Ivashkin
e64c63c611 Merge master 2021-02-05 20:10:06 +03:00
Stig Bakken
3fc1641d91 Show details of MaterializeMySQL tables in system.tables 2021-02-05 21:37:29 +08:00
tavplubix
f59dc7f40d
Merge pull request #20066 from havardk/multi-table-update 
Fix handling of TABLE_MAP_EVENT from MySQL
 2021-02-05 13:11:59 +03:00
vdimir
30f98795bc
Merge remote-tracking branch 'upstream/master' into disk-s3-backup-restore-metadata 2021-02-04 18:38:28 +03:00
alesapin
933105a667 Fix session timeout 2021-02-04 16:31:38 +03:00
alesapin
d85e9b496c Fix gcc-10 build 2021-02-04 16:22:30 +03:00
Haavard Kvaalen
e6f1ce48fe Fix handling of TABLE_MAP_EVENT from MySQL 
The MySQL replication code assumed that row update events would be
preceded by a single TABLE_MAP_EVENT.  However, if a single SQL
statement modifies rows in multiple tables, MySQL will first send
table map events for all involved tables, and then row update events.

Depending on circumstances, this could lead to an exception when the row
update was processed, the update could be incorrectly dropped, or the
update could be applied to the wrong table.
 2021-02-04 12:49:10 +01:00
alesapin
9c04d912ec Merge branch 'master' into in_memory_raft 2021-02-04 11:50:32 +03:00
alesapin
e7a83868dd Fix build 2021-02-04 11:28:11 +03:00
Vladimir Chebotarev
f106d58c05 Minor fix. 2021-02-04 10:41:47 +03:00
Vladimir Chebotarev
a597ed0ff0 Fixed open behavior of remote host filter in case when there is remote_url_allow_hosts section in configuration but no entries there. 2021-02-04 09:34:31 +03:00
alesapin
1ff87ac6f9 Add background session lifetime control 2021-02-03 23:32:15 +03:00
alexey-milovidov
402c018d1e
Merge pull request #19947 from azat/dist-INSERT-block-structure-mismatch 
Fix "Block structure mismatch" for INSERT into Distributed
 2021-02-03 21:27:45 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Pavel Kovalenko
b7151aa754 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskDecorator.h
#	src/Disks/IDisk.h
#	src/Disks/S3/DiskS3.cpp
 2021-02-03 14:22:18 +03:00
alexey-milovidov
830b0eb36e
Merge pull request #19974 from qoega/kerberized-hdfs-port-clash 
Fixed port clash from test_storage_kerberized_hdfs test.
 2021-02-03 10:16:14 +03:00
alesapin
6743dd4656 Same for the first test 2021-02-02 22:56:07 +03:00
Alexander Tokmakov
6456ccf0da better test 2021-02-02 22:39:04 +03:00
alesapin
1e0a528bac Fix possible test flakyness 2021-02-02 22:33:57 +03:00
Azat Khuzhin
ac510f9770 test_read_temporary_tables_on_failure: increase timeout to avoid false-positive 
In [1]:

    start_cluster = <helpers.cluster.ClickHouseCluster object at 0x7f2d46895dd8>

        def test_different_versions(start_cluster):
            with pytest.raises(QueryTimeoutExceedException):
                node.query("SELECT sleep(3)", timeout=1)
            with pytest.raises(QueryRuntimeException):
    >           node.query("SELECT 1", settings={'max_concurrent_queries_for_user': 1})
    E           Failed: DID NOT RAISE <class 'helpers.client.QueryRuntimeException'>

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/b68508002d134ead05bf2ca0e22a13a34a5c55c6/integration_tests_(thread).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
33628e1bd9 test_materialize_mysql_database: add retries for test_select_without_columns_8_0 
Here is an example:

    2021.01.31 14:04:43.615618 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:43.616453 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_1_1_0 to 0_1_1_0.
    ...
    2021.01.31 14:04:43.786823 [ 12 ] {4a0d1f80-4a96-4809-a84e-d55d7cd25cdd} <Debug> executeQuery: (from 172.18.0.1:45162, using production parser) SELECT count(_version) FROM db.t FORMAT TSV
    ...
    2021.01.31 14:04:44.566037 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:44.566943 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_2_2_0 to 0_2_2_0.
    ...
    2021.01.31 14:04:52.343911 [ 54 ] {} <Error> MaterializeMySQLSyncThread: Code: 1002, e.displayText() = DB::Exception: Master maybe lost, Stack trace (when copying this message, always include the lines below):

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/978f23e3437d0d0eb64727e87ffe1b80289c9fc2/integration_tests_(release).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
3d3d6777d3 test_dictionaries_dependency_xml: decrease dictionaries lifetime 
Since ExternalLoader::PeriodicUpdater::check_period_sec = 5, and so if
it will be scheduled too early the reload will be skipped, and indeed
this is what you can see in logs [1], the reload is done each 10
seconds, not 5:

    2021.01.31 14:20:22.590999 [ 48 ] {} <Trace> ExternalDictionariesLoader: Supposed update time for 'dep_x' is 2021-01-31 14:20:27 (loaded, lifetime [5, 5], no errors)
    2021.01.31 14:20:22.591016 [ 48 ] {} <Trace> ExternalDictionariesLoader: Next update time for 'dep_x' was set to 2021-01-31 14:20:27
    ...
    2021.01.31 14:20:32.164882 [ 50 ] {} <Trace> ExternalDictionariesLoader: Start loading object 'dep_x'

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/37797fdf5b30dc97147e73b3ac8ca9025b80aaed/integration_tests_(release).html#fail1
 2021-02-02 22:04:32 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Pavel Kruglov
dd9af192c5 Update test 2021-02-02 18:18:05 +03:00
alesapin
9979181c02 Merge branch 'master' into in_memory_raft 2021-02-02 17:05:20 +03:00
alesapin
94201ebf04 More info in test 2021-02-02 15:41:09 +03:00
Pavel Kruglov
cc14cb11f9 Update test 2021-02-02 15:20:28 +03:00
Pavel Kruglov
60a92e9a99 Fix build, add comments, update tests 2021-02-02 15:14:31 +03:00
Yatsishin Ilya
19d6e867c6 HDFS test had a clash with 9000 ClickHouse port. 2021-02-02 12:08:47 +03:00
tavplubix
dbcddbcebd
Merge pull request #19770 from stigsb/materializemysql_system_parts 
Show MaterializeMySQL tables in system.parts
 2021-02-02 11:43:42 +03:00
alesapin
d265e3b419 Less timeouts 2021-02-02 11:02:25 +03:00
alesapin
a9c50caa9a
Merge pull request #19942 from ClickHouse/fix_test_keeper_b2b_test 
Fix test keeper integration tests
 2021-02-02 10:17:27 +03:00
Pavel Kruglov
7d9eb966f0 Fix 2021-02-01 20:09:55 +03:00
alesapin
e6fd6abe6f Fix test keeper integration tests 2021-02-01 18:07:34 +03:00
alesapin
365bf65f5a Fix install script 2021-02-01 17:14:59 +03:00
alesapin
2aa8a6304b
Merge pull request #15450 from CurtizJ/fix-ttl-group-by 
Fix some cases of TTL expressions
 2021-02-01 16:48:07 +03:00
alesapin
0aca40d8cb Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
Stig Bakken
bedc472bb0 Show MaterializeMySQL tables in system.parts 2021-02-01 16:59:10 +08:00
alesapin
67412bd529 Don't accept connections if we don't see leader + race fix from upstream 2021-02-01 10:51:10 +03:00
alesapin
d9598c47b4
Merge pull request #19702 from ClickHouse/fix_rare_bug_after_part_corruption 
Fix incorrect virtual_parts after part corruption
 2021-02-01 10:09:48 +03:00
Denis Glazachev
6aa86846ac Removing obsoleted test 2021-01-31 05:41:28 +04:00