Mike Kot
c947280dfc
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-04-01 21:38:51 +03:00
Mike Kot
60e5946449
Removed needless logging, fixed data part initialization while checking
...
in the detached folder
2021-04-01 20:11:14 +03:00
kssenii
99f0b08ac3
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres
2021-04-01 13:48:33 +00:00
Alexey Milovidov
39c3efdb00
Fix build
2021-04-01 16:38:04 +03:00
Alexander Kuzmenkov
c7bf01fa33
typo
2021-04-01 16:08:18 +03:00
Alexander Kuzmenkov
ba64bc8557
quick fix for inaccessible DatabaseReplicated
2021-04-01 15:21:29 +03:00
kssenii
8300a74f5a
Fix
2021-04-01 12:09:44 +00:00
vdimir
3f464595eb
Accurate removing 'join' part for queries to 'Merge' engine
2021-04-01 14:31:57 +03:00
vdimir
fff8043e55
Revert "Fix joined table access with Merge engine and aggregation"
...
This reverts commit 5476e68d6c
2021-04-01 14:31:46 +03:00
kssenii
522d170588
Add default port
2021-04-01 11:30:20 +00:00
vdimir
5476e68d6c
Fix joined table access with Merge engine and aggregation
2021-04-01 14:26:28 +03:00
kssenii
747967a9bb
Better
2021-04-01 10:27:24 +00:00
alesapin
7c8f54e694
More changes
2021-04-01 11:07:56 +03:00
Alexey Milovidov
7db8fcdebd
Fix buffer overflow in TokenExtractor #19233
2021-04-01 05:37:19 +03:00
alexey-milovidov
bbd41d64c2
Update DataPartsExchange.cpp
2021-04-01 02:59:47 +03:00
alexey-milovidov
538be4086b
Recursive removal is extremely dangerous.
2021-04-01 02:59:23 +03:00
alesapin
c15d7e009d
Some initial code
2021-03-31 18:20:30 +03:00
kssenii
4e849f3b77
Less logging
2021-03-31 14:06:24 +00:00
kssenii
ba5ddd5c32
Do not parse replicas in PoolWithFailover
2021-03-31 14:03:52 +00:00
Nicolae Vartolomei
5cd1dc485f
Delete temporary fetch directory if it already exists
...
It could be left there after a network timeout, a system crash, or
checksum issues.
The downside of this PR is that it would be harder to debug checksum
issues if replication will continuously delete and try to fetch the same
part.
2021-03-31 14:14:58 +01:00
Maksim Kita
55984e849c
Merge branch 'master' into hierarchy-dictionaries-updated
2021-03-31 15:55:07 +03:00
tavplubix
6c1e2a0d87
Merge pull request #21690 from ClickHouse/test_multiple_nodes
...
Run three nodes with Replicated database and NuKeeper in CI
2021-03-31 12:31:13 +03:00
Alexey Milovidov
e9bfc7456d
Merge branch 'master' into virtualutil
2021-03-31 08:19:43 +03:00
alexey-milovidov
a21fe0b492
Merge pull request #22378 from ianton-ru/s3_zero_copy_replication
...
Fix S3 Zero-Copy replication for hybrid storage
2021-03-31 04:15:47 +03:00
alexey-milovidov
59c158300a
Merge pull request #22365 from ClickHouse/aku/processes-database
...
add current database to system.processes
2021-03-31 04:14:17 +03:00
alexey-milovidov
5c15b8a896
Merge branch 'master' into virtualutil
2021-03-30 23:59:44 +03:00
Anton Ivashkin
f87435213e
Fix S3 Zero-Copy replication for hybrid storage
2021-03-30 20:38:04 +03:00
kssenii
ce05087b1b
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres
2021-03-30 17:27:10 +00:00
Alexander Tokmakov
4f896d2b07
fix tests
2021-03-30 19:46:23 +03:00
Alexander Kuzmenkov
0252f60c23
add current database to system.processes
2021-03-30 16:36:08 +03:00
Vladimir
5ae9f2c5bb
Merge pull request #21329 from depressed-pho/isIPAddressContainedIn
2021-03-30 13:26:49 +03:00
alesapin
2f891521db
Merge pull request #22291 from ClickHouse/no_logical_error_on_mutation
...
Relax covering part check on mutation
2021-03-30 13:25:27 +03:00
Nikolai Kochetov
78e27dc1b6
Merge pull request #22270 from liuneng1994/fix-query-failed
...
fix constant condition conflict with column name and add stateless test
2021-03-30 09:37:43 +03:00
alexey-milovidov
b7b55ce61d
Merge pull request #20088 from nvartolomei/nv/replicated-fetches-timeouts
...
Replicated fetches timeouts
2021-03-30 03:46:09 +03:00
Alexey Milovidov
230b2930b7
Merge branch 'master' into virtualutil
2021-03-30 00:43:00 +03:00
tavplubix
3c0f5a57ec
Merge pull request #21535 from ClickHouse/distributed_ddl_improvements
...
Distributed DDL improvements
2021-03-29 22:40:11 +03:00
alexey-milovidov
9d84f3113a
Merge pull request #21512 from azat/optimize_skip_unused_shards_limit
...
Add optimize_skip_unused_shards_limit
2021-03-29 21:13:18 +03:00
alexey-milovidov
1512ff832e
Merge branch 'master' into nv/replicated-fetches-timeouts
2021-03-29 19:25:13 +03:00
Anton Popov
ea82e7725f
Merge pull request #21562 from CurtizJ/serialization-refactoring-4
...
Refactoring of data types serialization
2021-03-29 16:36:44 +03:00
Alexander Kuzmenkov
0171ab95fa
Merge pull request #21640 from vdimir/fix-join-where-database-engine
...
Remove unknown columns from joined table in where for queries to external database engines
2021-03-29 16:14:10 +03:00
alesapin
f439e17ece
Fix build
2021-03-29 15:41:26 +03:00
alesapin
2c06eb8382
Relax covering part check on mutation
2021-03-29 14:59:28 +03:00
Neng Liu
0d78427fbf
fix query failed and add stateless test
2021-03-29 16:12:02 +08:00
Vladimir
0e9926e6c6
Update src/Storages/transformQueryForExternalDatabase.cpp
...
Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
2021-03-29 09:34:43 +03:00
alexey-milovidov
7ab2890c4d
Merge pull request #22206 from ClickHouse/mmap-cache
...
Add cache for mmap IO
2021-03-29 05:17:55 +03:00
alexey-milovidov
c08289517a
Merge pull request #21766 from amosbird/indexfix
...
Fix scalar subquery index analysis
2021-03-29 02:03:27 +03:00
Alexey Milovidov
a0a3380d91
Remove useless headers
2021-03-28 22:46:45 +03:00
Alexey Milovidov
2a8ac01cdb
Rename as suggested by Kita
2021-03-28 22:24:28 +03:00
kssenii
1bd1a705ce
Fix clang tidy
2021-03-28 12:45:49 +00:00
kssenii
0a66f11f69
Fix style, fix build
2021-03-28 10:27:37 +00:00
Alexey Milovidov
6f9f7d118d
Fixes
2021-03-28 04:27:26 +03:00
Alexey Milovidov
50f712e198
Integrate mmap cache to the infrastructure
2021-03-28 04:10:30 +03:00
kssenii
c17748a0ba
Add proper settings
2021-03-27 21:57:35 +00:00
kssenii
ef537b802f
Better comments
2021-03-27 21:10:44 +00:00
kssenii
22b515fbc9
Add namespace, simplify names
2021-03-27 20:14:02 +00:00
kssenii
f40c582e7a
Replicas/shards for postgres storage
2021-03-27 20:11:48 +00:00
kssenii
f141f027f4
Make common class for external storages
2021-03-27 17:12:47 +00:00
kssenii
95e8a8b9f0
Support shards
2021-03-27 14:40:07 +00:00
kssenii
ae868208c2
Use pool with failover in mysql storage
2021-03-27 14:39:45 +00:00
alesapin
3b119355d2
Merge pull request #22118 from ClickHouse/try_to_fix992
...
Trying to fix my favorite test
2021-03-27 10:17:14 +03:00
Maksim Kita
9f2f0d1095
Refactored hierarchy dictionaries interface
2021-03-26 18:42:32 +03:00
Alexander Kuzmenkov
a455762933
Merge pull request #22117 from ucasFL/useless-code
...
remove useless code
2021-03-26 17:18:14 +03:00
alesapin
35d1443a9c
Don't wait when table shutdown called
2021-03-26 15:54:08 +03:00
tavplubix
ae62bee939
Merge pull request #21913 from ClickHouse/fix_context_ref
...
Fix possibly dangling reference to Context
2021-03-26 10:39:13 +03:00
PHO
1868ca34b8
Fix build on GCC where std::isinf() cannot be called without the namespace prefix
2021-03-26 12:11:22 +09:00
Alexey Milovidov
177a017c5c
Minor modification #22115
2021-03-25 20:51:32 +03:00
alexey-milovidov
f895bc895c
Merge pull request #22011 from ClickHouse/min_max_time_system_parts_datetime64
...
Expose DateTime64 minmax part index in system.parts and system.parts_columns
2021-03-25 16:02:33 +03:00
Anton Popov
6a15431be7
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-25 15:57:35 +03:00
Anton Popov
f58d470dd6
Merge pull request #21830 from foolchi/optimize_select_with_final
...
optimize select final with prewhere primary key
2021-03-25 15:32:13 +03:00
feng lv
b7622868fc
remove useless code
2021-03-25 11:10:41 +00:00
Nikolai Kochetov
0e2f52518f
Extract converting from UnionStep.
2021-03-25 12:57:14 +03:00
kssenii
bc36baef26
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-25 07:48:07 +00:00
Nikolai Kochetov
977ebe8b44
Merge pull request #22058 from azat/system.errors-trace
...
Convert system.errors.stack_trace from String into Array(UInt64)
2021-03-25 10:23:05 +03:00
Mike Kot
285af08949
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-03-24 22:34:20 +03:00
Mike Kot
9f25ee3719
Replaced removing checksums.txt with version
...
with the explicit data loading and verification.
If the function fails, the exception will re-throw upper,
cancelling the fetch in the handling of the replicated log entry,
but this event will also wake the PartCheckThread, which will
issue a re-fetch.
2021-03-24 22:30:16 +03:00
kssenii
d9326cc229
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-24 18:28:55 +00:00
alexey-milovidov
ffa6585170
Merge pull request #21837 from Jokser/disk-cache-cleanup-directory-before-move
...
Cleanup destination directory during DiskCacheWrapper::moveFile()
2021-03-24 21:19:48 +03:00
fuqi
487d6bdcd9
add test case
...
fix order key check
2021-03-24 16:36:45 +08:00
Azat Khuzhin
7c07b43597
Convert system.errors.stack_trace from String into Array(UInt64)
...
This should decrease overhead for the errors collecting.
2021-03-24 06:52:48 +03:00
alexey-milovidov
612d4fb073
Update IMergeTreeDataPart.cpp
2021-03-24 02:03:14 +03:00
Anton Popov
8a56d04714
Merge branch 'merge_tree_part_block_number' of git://github.com/ahrefs/ClickHouse into merge_21760
2021-03-23 16:47:14 +03:00
Alexander Tokmakov
c938f4f2fe
fix
2021-03-23 14:29:29 +03:00
alesapin
36898bdc4a
Merge pull request #21142 from Jokser/unfreeze-partitions
...
Add ALTER TABLE UNFREEZE command.
2021-03-23 14:05:38 +03:00
Pavel Kovalenko
a92cf30b67
Code review fixes.
2021-03-23 13:33:07 +03:00
Alexey Milovidov
8d0210b510
Expose DateTime64 minmax part index in system.parts and system.parts_columns #18244
2021-03-23 01:16:41 +03:00
Alexey Milovidov
ce67afd9e3
Fix bad code #18244
2021-03-23 00:49:18 +03:00
Alexander Kuzmenkov
2f5dbf57b6
Merge pull request #21839 from kssenii/add-postgres-connection-pool
...
Add connection pool for postgres engine
2021-03-22 19:49:51 +03:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements
2021-03-22 18:13:19 +03:00
Mike Kot
da67e06aa0
Added another test case to handle missing part data
2021-03-22 17:52:21 +03:00
Mike Kot
c55a73b752
Added the solution to handle the corruption case
...
When the part data (e.g. data.bin) is corrupted, but the checksums.txt
is present -- explicitly deleting the checksums.txt.
Removed the extra logging, changes some exceptions message.
2021-03-22 17:23:43 +03:00
Mike Kot
5789507e8b
Investigating, why the checksums may match when they shouldn't.
2021-03-22 17:23:43 +03:00
kssenii
8a650c4408
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-22 13:38:25 +00:00
Maksim Kita
8be263062d
Merge pull request #21910 from kitaisreal/table-function-dictionary
...
Added table function dictionary
2021-03-22 16:08:56 +03:00
tavplubix
0e5362a02c
Merge pull request #21903 from ClickHouse/tavplubix-patch-3
...
Show path to data directory of EmbeddedRocksDB
2021-03-22 12:46:53 +03:00
Nikolai Kochetov
730785f4c2
Merge pull request #21459 from ClickHouse/refactor-actions-dag
...
Refactor actions dag
2021-03-22 11:58:16 +03:00
kssenii
99295c050a
Better logging
2021-03-20 15:41:15 +00:00
kssenii
40e50e6b02
Add unhexN method
2021-03-20 15:12:46 +00:00
Maksim Kita
849a980644
Updated ExternalDictionariesLoader interface with context parameter
2021-03-20 18:02:09 +03:00
kssenii
ed158e2b77
Proper setting
2021-03-20 14:42:03 +00:00
kssenii
878292d341
Remove redundant rethrow
2021-03-20 14:26:36 +00:00
kssenii
8062a81ddd
Separate common for materialize* storages
2021-03-20 14:13:57 +00:00
kssenii
4c37ff3fbc
Less lambdas
2021-03-20 13:17:30 +00:00
kssenii
60fbeb8354
Better replica consumer code
2021-03-20 12:53:12 +00:00
kssenii
8091c8724d
Fix metadata file
2021-03-20 12:15:32 +00:00
kssenii
caffccd73e
Fixes
2021-03-20 11:53:41 +00:00
Kseniia Sumarokova
4e63b8e5dd
Update src/Storages/PostgreSQL/PostgreSQLReplicationHandler.cpp
...
Co-authored-by: tavplubix <tavplubix@gmail.com>
2021-03-20 13:50:16 +03:00
kssenii
6bb81630e5
Small fixes
2021-03-20 10:32:11 +00:00
kssenii
619f0f51fd
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica
2021-03-20 09:47:05 +00:00
kssenii
698b1b043c
Fix typos check
2021-03-19 21:54:59 +00:00
kssenii
3b32c8fe9a
Block on empty connection pool
2021-03-19 21:00:45 +00:00
kssenii
e066feb81c
Slightly better
2021-03-19 18:50:41 +00:00
kssenii
50743b19c4
Add one more test
2021-03-19 16:12:33 +00:00
kssenii
f374a71165
Fix
2021-03-19 16:07:53 +00:00
Alexander Tokmakov
1845df25f3
fix possibly dangling reference to Context
2021-03-19 17:02:48 +03:00
Maksim Kita
32831e37ba
Added table function dictionary
2021-03-19 15:47:27 +03:00
Maksim Kita
a53c90e509
Upated system.dictionaries table
2021-03-19 12:43:57 +03:00
kssenii
a0f717619c
Remove initial filling
2021-03-19 08:33:04 +00:00
kssenii
9057aad798
Better version
2021-03-19 08:11:36 +00:00
tavplubix
fc0b87c5c4
Update StorageEmbeddedRocksDB.h
2021-03-19 10:24:27 +03:00
kssenii
f1ef87d966
Fix
2021-03-18 20:04:54 +00:00
Nikolai Kochetov
c3c393a7aa
Merge branch 'master' into refactor-actions-dag
2021-03-18 14:33:07 +03:00
kssenii
7847b78af9
Fix style, build
2021-03-17 20:49:56 +00:00
fuqi
cfa92f0045
fix code style
2021-03-18 00:25:43 +08:00
fuqi
b8edc12812
fix code style
2021-03-17 23:56:55 +08:00
Mike Kot
e309dd275a
Trying to investigate the errors after attaching the part.
...
+ A new entry is added to the part_log.
2021-03-17 18:49:04 +03:00
kssenii
3903d59d30
Better
2021-03-17 14:34:04 +00:00
fuqi
e116e8d5e8
fix bug
2021-03-17 22:19:10 +08:00
kssenii
ae64a24844
Add connection pool
2021-03-17 13:55:47 +00:00
Mike Kot
831b90f272
Replaced the iteration algorithm in part finder
...
Now, instead of iterating through the directories, we iterate though
directories of on of the table disks (which doesn't give us a
substantial boost but is a bit neater to read).
- Updated the system.replication_queue command types.
- Fixed the part ptr being empty (added the checksum loading and
initialization).
- Removed extra logging.
2021-03-17 16:55:26 +03:00
Mike Kot
2ccdb7ef5c
Multiple small code and test updates
...
- Updated the docs to make everything clear.
- Multiple small logger fixes.
- Changed the attach_part command -- now it's after check for the
covering parts -- motivation is to do less work with the checksums
fetching.
- Better logging in the integration test.
2021-03-17 16:52:35 +03:00
kssenii
34dea63248
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-17 13:32:43 +00:00
Nikita Mikhaylov
f51b41b7a4
Merge pull request #21710 from kssenii/replica-priorities
...
Support replicas priority for postgres dictionary source
2021-03-17 16:02:00 +03:00
Kseniia Sumarokova
f8fc4281f2
Update PostgreSQLReplicaConnection.h
2021-03-17 15:56:47 +03:00
fuqi
9ab713c2e1
optimize select final with prewhere primary key
2021-03-17 18:57:58 +08:00
kssenii
87c740730b
Rename to MaterializePostgreSQL
2021-03-17 10:52:12 +00:00
fuqi
5e20ea2c33
optimize select final with prewhere primary key
2021-03-17 18:49:24 +08:00
Pavel Kovalenko
5627ae9067
Make MergeTreeData::getPartitionMatcher const
2021-03-17 12:12:55 +03:00
Nikolai Kochetov
3a3f800f6e
Merge pull request #21579 from azat/dist-query-zero-shards-fix
...
Fix optimize_skip_unused_shards for zero shards case
2021-03-17 11:54:08 +03:00
Nikolai Kochetov
4f1f344d4d
Merge pull request #21529 from azat/system.errors-improvements
...
Add last_error_time/last_error_message/last_error_stacktrace/remote columns for system.errors
2021-03-17 11:51:29 +03:00
alexey-milovidov
d02726bcac
Merge pull request #9404 from Enmk/DateTime64_extended_range
...
Date time64 extended range
2021-03-17 11:06:03 +03:00
tavplubix
a093980767
Merge pull request #21711 from kssenii/non-default-schema
...
Support non-default table schema for postgres storage/table-function
2021-03-17 02:10:32 +03:00
Azat Khuzhin
9dee842b60
Distinguish remote and local error info
2021-03-16 22:31:49 +03:00
Azat Khuzhin
a337691b06
Fix modernize-use-bool-literals clang-tidy warning in StorageSystemErrors
2021-03-16 22:31:49 +03:00
Azat Khuzhin
efdd04c958
Drop last_ prefix for ErrorCodes::ValuePair::error_time_ms
2021-03-16 22:31:49 +03:00
Azat Khuzhin
44c9dc753d
Add system.errors.last_error_stacktrace column
2021-03-16 22:31:49 +03:00
Azat Khuzhin
775f8f7682
Add system.errors.last_error_message column
2021-03-16 22:31:49 +03:00
Azat Khuzhin
c8852331a2
Add system.errors.last_error_time column
2021-03-16 22:31:49 +03:00
Azat Khuzhin
0d01eaf94f
Guard ErrorCodes with mutex over atomic
2021-03-16 22:31:49 +03:00
Azat Khuzhin
259e5ba88e
Separate accounting of remote exceptions in system.errors
2021-03-16 22:31:46 +03:00
tavplubix
848bb59175
Merge pull request #21728 from ClickHouse/fix_alter_decimal
...
Fix alter modify for decimal columns when type size doesn't change
2021-03-16 20:01:35 +03:00
