Antonio Andelic
83a3243758
Merge pull request #47065 from ClickHouse/debug-keeper
...
Fix Keeper with `force_sync = false`
2023-03-10 09:27:13 +01:00
Antonio Andelic
5bc21538e5
Enable use_environment_credentials by default
2023-03-09 10:31:55 +00:00
Antonio Andelic
0ad436aa2c
Revert debug changes
2023-03-09 08:07:59 +00:00
Antonio Andelic
b4061d8aa4
Add logs
2023-03-08 13:06:10 +00:00
Mike Kot
9920a52c51
use std::lerp, constexpr hex.h
2023-03-07 22:50:17 +00:00
Antonio Andelic
dd4db9132a
Merge branch 'master' into debug-keeper
2023-03-07 13:15:05 +01:00
Antonio Andelic
c38d820d0f
Add 4LW for cleaning resource
2023-03-05 16:57:18 +00:00
Antonio Andelic
cd4100cb19
Merge branch 'master' into fix-data-size
2023-03-03 15:33:43 +01:00
Antonio Andelic
fdcb5b784f
Merge branch 'master' into debug-keeper
2023-03-03 13:45:13 +01:00
Robert Schulze
740aeaba1f
Apply some CTAD
2023-03-02 13:36:47 +00:00
Antonio Andelic
6ebee202bd
Flush buffer when no sync
2023-03-01 14:47:15 +00:00
Antonio Andelic
c797122dbe
Better
2023-03-01 08:45:50 +00:00
Antonio Andelic
6e28406751
Merge branch 'master' into fix-data-size
2023-02-28 20:09:29 +00:00
Alexey Milovidov
d8cda3dbb8
Remove PVS-Studio
2023-02-19 23:30:05 +01:00
Alexander Tokmakov
e96340e183
Merge pull request #46410 from ClickHouse/fix_noisy_log_messages
...
Fix noisy log messages
2023-02-15 01:22:42 +03:00
Alexander Tokmakov
1bf3735ca9
fix noisy log messages
2023-02-14 15:33:27 +01:00
Sema Checherinda
9bef1bec28
Merge pull request #46274 from aiven-sal/aiven-sal/keeper
...
coordination: do not allow election_timeout_lower_bound_ms > election_timeout_upper_bound_ms
2023-02-14 12:44:04 +01:00
Sema Checherinda
5882f3878f
except zero values election_timeout
2023-02-13 12:22:08 +01:00
Salvatore Mesoraca
fff2c09b87
coordination: do not allow election_timeout_lower_bound_ms > election_timeout_upper_bound_ms
2023-02-13 10:09:45 +01:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias
2023-02-09 16:07:45 +01:00
Azat Khuzhin
8cc41b7f41
Check return value of ::close()
...
Note, that according close(2), EINTR should not be retriable for close()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-07 11:28:22 +01:00
Antonio Andelic
b11458f142
Fix erase
2023-02-07 07:51:22 +00:00
Antonio Andelic
9cde46acff
Correctly calculate size on update
2023-02-06 16:28:05 +00:00
Antonio Andelic
aca4f08bf5
Correctly calculate size after snapshot load
2023-02-06 14:59:23 +00:00
Antonio Andelic
8929838fcc
Merge branch 'master' into add-support-for-settings-alias
2023-02-03 17:47:00 +00:00
Antonio Andelic
d5117f2aa6
Define S3 client with bucket and endpoint resolution ( #45783 )
...
* Update aws
* Define S3 client with bucket and endpoint resolution
* Add defines for ErrorCodes
* Use S3Client everywhere
* Remove unused errorcode
* Add DROP S3 CLIENT CACHE query
* Add a comment
* Fix style
* Update aws
* Update reference files
* Add missing include
* Fix unit test
* Remove unneeded declarations
* Correctly use RetryStrategy
* Rename S3Client to Client
* Fix retry count
* fix clang-tidy warnings
2023-02-03 14:30:52 +01:00
LiuYangkuan
f29700bd2f
use IDisk to do IO in Keeper's snapshots and logs
2023-02-02 19:47:30 +08:00
Antonio Andelic
f613bfd8d2
Merge branch 'master' into add-support-for-settings-alias
2023-02-02 11:15:39 +00:00
Antonio Andelic
85cfee4bb9
Better alias definition
2023-02-01 13:54:03 +00:00
Antonio Andelic
88c1744975
Better
2023-02-01 14:25:58 +01:00
Antonio Andelic
5de6f86c27
Fix clang-tidy error
2023-02-01 12:28:44 +00:00
Antonio Andelic
ec04cf6876
Merge branch 'master' into add-support-for-settings-alias
2023-02-01 09:01:01 +00:00
Antonio Andelic
ece92fa866
Update src/Coordination/Changelog.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-01-31 17:11:18 +01:00
Antonio Andelic
8457d3bf3d
Apply comments
2023-01-31 14:18:54 +00:00
Antonio Andelic
1786237be6
Merge branch 'master' into keeper-use-fallocate
2023-01-31 13:53:47 +00:00
Vitaly Baranov
f9282cfcac
Make version_id an optional parameter again.
2023-01-27 15:10:19 +01:00
Vitaly Baranov
aea9ccdb60
Pass request settings to S3::getObjectInfo().
2023-01-27 15:10:09 +01:00
Vitaly Baranov
a8304525ed
Move getObjectInfo() to a separate header.
2023-01-27 15:09:38 +01:00
Antonio Andelic
714fad1529
Add support for settings alias
2023-01-26 14:06:46 +00:00
Antonio Andelic
ebfe7f9991
Fix style
2023-01-26 11:18:07 +00:00
Antonio Andelic
1186718658
Add overallocate size setting
2023-01-26 09:44:37 +00:00
Antonio Andelic
7bfdfa4bf7
Preallocate more space
2023-01-25 16:26:14 +00:00
Antonio Andelic
7421d8f872
Merge branch 'master' into keeper-use-fallocate
2023-01-25 11:36:47 +00:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages ( #45449 )
...
* save format string for NetException
* format exceptions
* format exceptions 2
* format exceptions 3
* format exceptions 4
* format exceptions 5
* format exceptions 6
* fix
* format exceptions 7
* format exceptions 8
* Update MergeTreeIndexGin.cpp
* Update AggregateFunctionMap.cpp
* Update AggregateFunctionMap.cpp
* fix
2023-01-24 00:13:58 +03:00
Alexander Tokmakov
870cfcc36a
less fmt::runtime usages
2023-01-17 00:11:59 +01:00
Antonio Andelic
6a0e481ed7
Merge branch 'master' into keeper-use-fallocate
2023-01-12 13:57:05 +00:00
Antonio Andelic
30510c5eb6
Merge pull request #44937 from ClickHouse/fix-changelog-deadlock
...
Fix deadlock in Keeper's changelog
2023-01-10 08:59:51 +01:00
Antonio Andelic
22679ddf11
Add comment
2023-01-09 14:26:20 +01:00
Antonio Andelic
a39f437306
Typo fix
2023-01-09 13:34:48 +01:00
Antonio Andelic
27b2bae640
make it better
2023-01-09 12:01:22 +00:00
Antonio Andelic
4fcbe14435
Small fixes for Coordination unit tests
2023-01-08 13:44:32 +00:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2
...
Replace old named collections code with new (from #43147 ) part 2
2023-01-06 13:06:26 +01:00
Antonio Andelic
9b3314a931
Fix deadlock in Changelog flush
2023-01-05 14:51:05 +00:00
Antonio Andelic
136436958b
Merge pull request #44908 from ClickHouse/fix-keeper-snapshot-on-exit
...
Avoid Keeper crash on shutdown (fix `test_keeper_snapshot_on_exit`)
2023-01-05 08:59:58 +01:00
Antonio Andelic
7c2cdfa65a
Don't throw exception if queue is finished
2023-01-04 15:46:38 +00:00
kssenii
67509aa2d5
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
2023-01-03 16:41:30 +01:00
Smita Kulkarni
8a5fd7f96d
Added comments and addressed review comments - Allow macro substitution in endpoint
2023-01-03 11:44:26 +01:00
Antonio Andelic
f2653c016a
Manually set deleted flag
2023-01-02 14:36:58 +00:00
Antonio Andelic
f32fab5025
Check if file exists before rename
2023-01-02 13:48:13 +00:00
Antonio Andelic
2e93640614
Fix ASAN error
2023-01-02 13:37:40 +00:00
Antonio Andelic
b845f880fb
Polish
2023-01-02 10:42:15 +00:00
Antonio Andelic
14355d8009
Merge branch 'master' into keeper-use-fallocate
2023-01-02 08:22:27 +00:00
Antonio Andelic
55548a6678
style fix
2023-01-02 08:22:00 +00:00
SmitaRKulkarni
f9a3aeb172
Merge branch 'master' into 40951_Allow_macro_substitution_in_endpoint
2022-12-30 10:43:14 +01:00
Alexey Milovidov
f0fa0977ed
Fix wrong log messages
2022-12-26 18:36:07 +01:00
Smita Kulkarni
3d62bc2635
Allow macro substitution in endpoint of disks
...
Implementation:
* Updated to substitute macros in endpoint for all disks (s3, hfs, web).
Testing:
* Added an integration test test/integration/test_endpoint_macro_substitution.
2022-12-23 15:09:24 +01:00
Antonio Andelic
64ade259cf
Fix tests
2022-12-22 10:20:55 +00:00
Antonio Andelic
88d41dea93
Merge branch 'master' into keeper-use-fallocate
2022-12-21 08:59:09 +00:00
Antonio Andelic
9711355cc9
Fix tests
2022-12-21 08:43:55 +00:00
kssenii
6bd4f8c029
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
2022-12-20 21:17:28 +01:00
Antonio Andelic
6d83221d4e
Remove unused error code
2022-12-19 12:20:26 +00:00
Antonio Andelic
c2c115d2ba
Add setting for max_log_file_size
2022-12-19 11:51:05 +00:00
Antonio Andelic
f241701c2a
Add checks when fallocate is not supported
2022-12-19 11:48:54 +00:00
Antonio Andelic
9a8b248735
Merge branch 'master' into keeper-use-fallocate
2022-12-19 09:32:05 +00:00
Nikolai Kochetov
62ff98344e
Validate s3 part upload settings.
2022-12-17 14:09:53 +00:00
kssenii
30547d2dcd
Replace old named collections code for url
2022-12-17 00:24:05 +01:00
Antonio Andelic
ee900ca7df
Add truncate
2022-12-15 13:09:52 +00:00
Antonio Andelic
bf6f163952
Merge branch 'master' into keeper-use-fallocate
2022-12-15 09:05:40 +00:00
Antonio Andelic
d454e00016
Changelog refactor
2022-12-15 08:12:02 +00:00
Anton Popov
0c87031e80
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-13 16:33:21 +00:00
Antonio Andelic
662c1f956c
Changelog refactoring
2022-12-13 14:00:27 +00:00
Antonio Andelic
898cf867af
Try with fallocate
2022-12-13 11:08:59 +00:00
Vitaly Baranov
b91af1b650
Fix initialization of s3 request settings.
2022-12-10 05:43:51 +01:00
Antonio Andelic
e1a6a1182e
Throw when port already in use ( #43984 )
2022-12-07 13:27:49 +01:00
Antonio Andelic
200141da13
Use logger in tests
2022-12-02 08:16:18 +00:00
Antonio Andelic
7cd76df918
Merge branch 'master' into parallel-log-appending
2022-12-01 14:44:36 +00:00
Antonio Andelic
554ff1db6f
Small changes
2022-12-01 12:54:29 +00:00
Antonio Andelic
940fdd4abd
Remove comment
2022-12-01 12:24:27 +01:00
Antonio Andelic
25ae52111e
Protect writer with mutex
2022-12-01 11:18:23 +00:00
Antonio Andelic
c12c0cc8af
Merge branch 'master' into parallel-log-appending
2022-12-01 08:53:52 +00:00
Antonio Andelic
2f501d5327
Merge branch 'master' into keeper-prometheus
2022-11-30 13:24:37 +01:00
Antonio Andelic
fbda86ddc8
PR fixes
2022-11-30 13:24:08 +01:00
Antonio Andelic
7f897f1a20
Refactor
2022-11-29 13:00:13 +00:00
Antonio Andelic
c8b2cb5650
Update src/Coordination/CoordinationSettings.h
...
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
2022-11-29 10:08:25 +01:00
Antonio Andelic
678958603b
Merge branch 'master' into keeper-prometheus
2022-11-28 09:48:57 +01:00
Antonio Andelic
fe44ac0c86
Merge branch 'master' into parallel-log-appending
2022-11-28 08:45:20 +00:00
Antonio Andelic
e91314d57f
Use initialized flag
2022-11-28 08:44:46 +00:00
Antonio Andelic
cade21b76e
Fix 4LW test
2022-11-26 19:24:10 +00:00
Antonio Andelic
76eca8bdfb
Merge branch 'master' into more-aggresive-batching-keeper
2022-11-26 18:28:11 +00:00
Antonio Andelic
cd471e9c61
Add setting for quick batch size
2022-11-26 17:33:40 +00:00
Anton Popov
65a78bcd91
improve performance of storage S3
2022-11-26 15:24:01 +00:00
Antonio Andelic
7437c719a7
Try batching requests more aggresively
2022-11-25 17:25:28 +00:00
Antonio Andelic
38d99c241f
Fix tests
2022-11-25 13:53:03 +00:00
Antonio Andelic
53f708366c
Fix data race in Keeper snapshot
2022-11-25 12:50:42 +00:00
Antonio Andelic
ce532cd12d
Fix circular deps
2022-11-24 13:43:05 +00:00
Antonio Andelic
a6f38cb1cc
Merge branch 'master' into keeper-prometheus
2022-11-24 12:12:08 +01:00
Antonio Andelic
0cc53995df
Merge branch 'master' into parallel-log-appending
2022-11-22 15:05:08 +01:00
Kseniia Sumarokova
5c90d5aa7e
Merge pull request #43253 from xiedeyantu/fix-s3-glob
...
fix s3 support question mark wildcard
2022-11-22 14:26:56 +01:00
Antonio Andelic
4827bcb6bb
Merge branch 'master' into parallel-log-appending
2022-11-22 14:14:17 +01:00
Antonio Andelic
55ef96dc11
Fix unit tests
2022-11-22 11:37:59 +00:00
Antonio Andelic
aeb0d39b50
Fix shutdown
2022-11-21 15:01:18 +00:00
Antonio Andelic
c9472d67b6
Refactor changelog
2022-11-21 12:22:36 +00:00
Antonio Andelic
ff681d5faa
Merge branch 'master' into parallel-log-appending
2022-11-21 11:59:16 +00:00
Antonio Andelic
3cb202a63b
Merge pull request #43026 from JackyWoo/keeper_manually_assign_leader
...
Keeper support manually assigning leader
2022-11-21 09:30:00 +01:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus
2022-11-21 09:10:10 +01:00
xiedeyantu
c258d3ac8b
fix s3 support question mark wildcard
2022-11-18 12:11:22 +08:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling""
2022-11-17 17:35:04 +01:00
Alexander Tokmakov
9011a18234
Revert "S3 request per second rate throttling"
2022-11-16 22:33:48 +03:00
Sergei Trifonov
159743edd6
Merge pull request #43014 from ClickHouse/disk-s3-throttler
...
S3 request per second rate throttling
2022-11-16 18:51:06 +01:00
JackyWoo
f8d71f60a2
Merge branch 'master' into keeper_manually_assign_leader
2022-11-16 09:34:40 +08:00
Antonio Andelic
96838f5be7
Merge branch 'master' into keeper-prometheus
2022-11-15 12:00:02 +00:00
Antonio Andelic
d08dce5ec7
Test parallel append
2022-11-14 13:17:05 +00:00
Raúl Marín
54db7c6520
Enforce checking read output
2022-11-11 10:56:18 +01:00
Antonio Andelic
94aab4276c
Bring back heavy metrics
2022-11-10 12:56:27 +01:00
Antonio Andelic
8aeb406d82
Fix standalone build
2022-11-10 10:49:00 +01:00
Antonio Andelic
36d763011b
Compile standalone keeper
2022-11-10 09:23:52 +01:00
Antonio Andelic
1ed3930809
Add more metrics for Keeper
2022-11-09 15:51:41 +01:00
Antonio Andelic
c92ae5a385
Add suport for AsyncMetrics in standalone Keeper
2022-11-09 13:46:03 +01:00
Sergei Trifonov
8eedd1e046
Merge branch 'master' into disk-s3-throttler
2022-11-08 15:00:56 +01:00
serxa
6d5d9ff421
rename ReadWriteSettings -> RequestSettings
2022-11-08 13:48:23 +00:00
JackyWoo
9060141f16
revert last change
2022-11-08 18:44:43 +08:00
JackyWoo
e865842098
Add rqld to white list
2022-11-08 16:03:13 +08:00
JackyWoo
eeedd7d9ba
Keeper support manually assigning leader
2022-11-08 12:01:53 +08:00
serxa
2daec0b45e
S3 request per second rate throttling + refactoring
2022-11-07 18:05:40 +00:00
Antonio Andelic
9ac829d4c4
Merge pull request #41766 from JackyWoo/manual_snapshot_for_keeper
...
Manually snapshot creation for keeper
2022-11-07 09:18:58 +01:00
JackyWoo
73dfe4acd7
Add to default white list
2022-11-05 17:56:55 +08:00
Antonio Andelic
25b19e6ba0
Merge branch 'master' into manual_snapshot_for_keeper
2022-11-03 08:55:01 +01:00
Antonio Andelic
664690f9a9
Add comments
2022-11-03 07:38:43 +00:00
Antonio Andelic
a6c4204b0c
Don't iterate all cached nodes on commit
2022-11-03 07:36:25 +00:00
JackyWoo
c7a0ebeb05
little fix
2022-10-25 17:46:24 +08:00
JackyWoo
9a36a509fe
fix test
2022-10-25 17:15:49 +08:00
JackyWoo
b5d1c4e657
replace snpd with lgif
2022-10-24 20:08:58 +08:00
JackyWoo
2f30c817bf
little fix
2022-10-24 17:23:47 +08:00
JackyWoo
39c88c74e8
check whether last manually created snapshot is done
2022-10-22 22:31:17 +08:00
JackyWoo
bf291790b7
Merge branch 'master' into manual_snapshot_for_keeper
2022-10-22 16:57:50 +08:00
Azat Khuzhin
905a95e166
Review fixes
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 22:40:13 +02:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Antonio Andelic
540728ab96
Simplify logic a bit
2022-10-18 10:58:25 +00:00
Antonio Andelic
1275c464aa
Fix unit test build
2022-10-18 10:44:22 +00:00
Antonio Andelic
8157fd0595
fix build
2022-10-17 14:16:56 +00:00
Antonio Andelic
238f86d892
Small refactoring
2022-10-17 11:06:33 +00:00
Antonio Andelic
7c7395575f
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-10-17 11:02:36 +00:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup
...
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake
Logical merge conflicts:
- Functions/tryDecrypt.cpp
2022-10-06 08:43:25 +00:00
Antonio Andelic
94f1fe39bb
Add support for exists in multiread
2022-09-29 10:39:33 +00:00
Antonio Andelic
3109ce51c6
Merge branch 'master' into keeper-multiread
2022-09-29 06:50:49 +00:00
Antonio Andelic
349bd7fd9a
Dont fail MultiRead on first failed op
2022-09-29 06:50:28 +00:00
Robert Schulze
f24fab7747
Fix some #include atrocities
2022-09-28 13:49:28 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h
...
Less duplication, less confusion ...
2022-09-28 13:31:57 +00:00
Robert Schulze
6d70b4a1f6
Generate config_version.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Antonio Andelic
d0457addbd
Support filtered list
2022-09-27 15:02:26 +00:00
JackyWoo
d1c85f68e8
manually snapshot creation for keeper
2022-09-26 18:29:15 +08:00
Antonio Andelic
937d534cd4
Add support for simple list
2022-09-26 08:05:02 +00:00
Antonio Andelic
f833366555
Merge branch 'master' into keeper-multiread
2022-09-26 07:16:45 +00:00
Antonio Andelic
9da433a904
Fix unused
2022-09-22 14:33:35 +00:00
Antonio Andelic
44a3d6babe
Address PR comments
2022-09-22 13:03:27 +00:00
Antonio Andelic
6798b500e9
Wait on startup for Keeper
2022-09-21 15:12:16 +00:00
Antonio Andelic
eefbbf53e8
Extract S3 logic
2022-09-21 11:53:54 +00:00
Antonio Andelic
fc6c1faa57
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-09-21 06:57:58 +00:00
Antonio Andelic
1763af37b9
Merge pull request #41403 from ClickHouse/keeper-fix-shutdown-without-server
...
Fix Keeper segfault during shutdown when RAFT server failed to start
2022-09-19 12:03:58 +02:00
alesapin
b6d61318d0
One more validation for broken config in keeper
2022-09-18 00:04:16 +02:00
Alexander Tokmakov
42dab6dc02
Revert "Merge pull request #38200 from ClickHouse/keeper-linearizable-reads"
...
This reverts commit 222b964070da01982652
2022-09-17 19:22:58 +02:00
Antonio Andelic
81f7cf3ca6
Add support for MultiRead in Keeper
2022-09-16 12:00:35 +00:00
Antonio Andelic
acd9cb089e
Fix segfault during shutdown when RAFT server failed to start
2022-09-16 10:03:21 +00:00
Antonio Andelic
5662e0050c
Define client for multi read
2022-09-16 09:55:02 +00:00
Antonio Andelic
296fe01fb8
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-09-16 08:00:15 +00:00
Antonio Andelic
39307f9dba
Extract into function
2022-09-15 16:14:53 +00:00
Antonio Andelic
0dbd0c1f72
Merge branch 'master' into keeper-linearizable-reads
2022-09-15 16:03:18 +00:00
Antonio Andelic
ad7864cb5f
enable S3 if client is built
2022-09-15 14:06:22 +00:00
Antonio Andelic
d3d1676352
Fix config update detection
2022-09-15 13:48:39 +00:00
Antonio Andelic
d4e4ac3801
Lock file during S3 snapshot upload
2022-09-15 09:11:55 +00:00
Antonio Andelic
9fdd2f2e61
Basic implementation for S3 snapshot upload
2022-09-15 09:11:55 +00:00
Antonio Andelic
3a9996f04c
Add bg thread for snapshot upload to S3
2022-09-15 09:11:55 +00:00
Antonio Andelic
36f46891dd
Merge pull request #41215 from ClickHouse/keeper-close-connection-on-shutdown
...
Close sessions on Keeper shutdown
2022-09-13 19:09:34 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage
...
KeeperMap storage engine
2022-09-13 14:15:09 +02:00
alesapin
69d08b1007
Merge pull request #41075 from ClickHouse/keeper-fix-possible-segfault
...
Fix possible segfault during Keeper shutdown
2022-09-13 12:13:24 +02:00
Antonio Andelic
a6b5ffec5d
Polishing
2022-09-13 09:51:46 +00:00
Antonio Andelic
b2cc6a8cc6
Use promise/future
2022-09-12 18:19:41 +00:00
Antonio Andelic
c2dfabe51c
Use shared_ptr for Event
2022-09-12 12:25:39 +00:00
Antonio Andelic
e985ee3354
Close sessions on Keeper shutdown
2022-09-12 12:22:48 +00:00
Antonio Andelic
32483aeec8
Merge branch 'master' into keeper-storage
2022-09-09 12:24:01 +00:00
Antonio Andelic
951ee44229
Process read requests with snapshot install
2022-09-07 11:51:56 +00:00
Antonio Andelic
a377a5f1e9
Change assert to LOGICAL_ERROR
2022-09-07 09:24:50 +00:00
Antonio Andelic
4cdf18cc81
Shutdown storage after RAFT
2022-09-07 09:20:47 +00:00
Antonio Andelic
c6b17bd7f9
Don't preprocess if storage finalized
2022-09-07 09:14:03 +00:00
Antonio Andelic
f0ba3ee927
Fix typo
2022-09-05 09:51:02 +02:00
Antonio Andelic
110ad53300
Merge branch 'master' into keeper-linearizable-reads
2022-09-05 07:12:45 +00:00
Antonio Andelic
b816d46520
Merge branch 'master' into fix-keeper-system-path-check
2022-09-02 13:42:16 +02:00
Antonio Andelic
9ca20b97c2
Merge branch 'master' into keeper-storage
2022-09-02 12:13:07 +02:00
Antonio Andelic
e64436fef3
Fix typos with new codespell
2022-09-02 08:54:48 +00:00
Antonio Andelic
ad2196155c
Add test for system node modification
2022-09-02 08:14:06 +00:00
Antonio Andelic
b7eebfc626
Correctly check if the node is using system path
2022-09-02 07:47:12 +00:00
Antonio Andelic
7164e59b54
Merge branch 'master' into keeper-linearizable-reads
2022-09-01 08:52:31 +00:00
Antonio Andelic
fa4c7259f9
Fix typos
2022-09-01 08:46:28 +00:00
Antonio Andelic
c7d1dd942c
Address PR comments
2022-09-01 07:56:06 +00:00
Antonio Andelic
21cdb9a500
Don't remove from cache if it wasn't added during rollback
2022-08-30 08:14:53 +00:00
Antonio Andelic
2416ddfae7
Add extra message about startup preprocessing
2022-08-30 08:14:31 +00:00
Antonio Andelic
57471d11db
Fix uncommitted auth
2022-08-29 12:55:30 +00:00
Antonio Andelic
50af0b1938
Add bunch of comments for request processing
2022-08-29 10:50:18 +00:00
Antonio Andelic
b4af21b6fb
Small fixes
2022-08-29 07:06:51 +00:00
Antonio Andelic
bff2b232ef
Cache deltas for path
2022-08-26 14:03:38 +00:00
Antonio Andelic
26d68dce23
Fix style
2022-08-26 10:55:07 +00:00
Antonio Andelic
dae65178b7
Improve preprocessing performance
2022-08-26 10:46:14 +00:00
Antonio Andelic
5d45f2faa9
Add comments
2022-08-23 08:22:03 +00:00
Antonio Andelic
46882791cd
Fix build
2022-08-23 07:29:47 +00:00
Antonio Andelic
444ac9af6c
Merge branch 'master' into keeper-linearizable-reads
2022-08-23 07:03:35 +00:00
Antonio Andelic
99d23cb67e
Merge branch 'master' into keeper-linearizable-reads
2022-08-23 07:03:20 +00:00
Antonio Andelic
a87a762b0c
Merge pull request #39973 from ClickHouse/keeper-listen-host
...
Support `interserver_listen_host` in Keeper
2022-08-23 09:03:08 +02:00
Alexey Milovidov
74e1f4dc61
Fix clang-tidy
2022-08-20 17:09:20 +02:00
Antonio Andelic
9396834762
Addressed comments
2022-08-17 09:29:08 +00:00
Antonio Andelic
24ee3e022d
Merge branch 'master' into keeper-listen-host
2022-08-09 09:39:27 +02:00
Alexey Milovidov
48bed932ec
Merge pull request #39931 from HarryLeeIBM/hlee-s390x-coordination
...
Fix Endian issue in KeeperSnapshotManager for s390x
2022-08-09 04:24:32 +03:00
HarryLeeIBM
42323cfb92
Use constexpr char array as magic number
2022-08-08 10:59:09 -07:00
Antonio Andelic
6b935a252f
Support multiple listen hosts
2022-08-08 09:08:19 +00:00
Alexey Milovidov
adc880f362
Merge pull request #39867 from ClickHouse/keeper-block-memory-tracker-commit
...
Block memory tracker in Keeper during commit
2022-08-06 02:02:29 +03:00
HarryLeeIBM
cbd01a3a73
Fix Endian issue in KeeperSnapshotManager
2022-08-05 11:07:42 -07:00
Antonio Andelic
07a85513e8
Merge branch 'master' into keeper-create-snapshot-on-exit
2022-08-04 13:12:26 +00:00
Antonio Andelic
10d7259c2b
Add log for snapshot on exit
2022-08-04 13:12:24 +00:00
Antonio Andelic
0a7d2e7b8a
Use LockMemoryExceptionThread
2022-08-04 13:03:05 +00:00
Antonio Andelic
278921be3b
Fix style
2022-08-04 07:25:28 +00:00
Antonio Andelic
fa98338ce1
Add comment for memory tracker blocking
2022-08-04 07:13:42 +00:00
Antonio Andelic
56ca93ca3f
Block memory tracker in Keeper during commit
2022-08-03 14:14:57 +00:00
Antonio Andelic
05467e315f
Merge pull request #39698 from ClickHouse/update-digest-version
...
Update Keeper version for digest
2022-08-03 13:13:00 +02:00
Antonio Andelic
b3b3c371f0
Update KeeperStorage.h
2022-08-02 09:20:02 +02:00
Antonio Andelic
7cdad88327
Create snapshot on shutdown
2022-07-30 11:34:17 +00:00
Antonio Andelic
21becb3030
Update version for digest
2022-07-29 07:36:19 +00:00
Antonio Andelic
e3cb65a99a
Rollback on failed preprocess
2022-07-29 06:51:53 +00:00
Antonio Andelic
c0a5d45258
Rollback on failed PreAppend
2022-07-28 14:06:45 +00:00
Antonio Andelic
1090d6bca7
Rollback request if storing log fails
2022-07-28 12:55:29 +00:00
Antonio Andelic
e62526720f
Address PR comments
2022-07-27 07:51:30 +00:00
Antonio Andelic
04b03b6a90
Don't allow to overwrite on startup
2022-07-25 12:38:48 +00:00
Antonio Andelic
afb6cb6824
Add KeeperContext
2022-07-23 14:45:31 +00:00
Antonio Andelic
6aff87d4b5
Ignore system paths from snapshots and logstore
2022-07-23 13:46:59 +00:00
Antonio Andelic
46a6fbd7aa
Small polish
2022-07-22 11:03:07 +00:00
Antonio Andelic
3040ff0959
Merge branch 'master' into keeper-version-check
2022-07-22 08:07:55 +00:00
Antonio Andelic
75476d5110
Store only root system node
2022-07-22 08:07:38 +00:00
Antonio Andelic
fdd79e62da
Merge pull request #39393 from ClickHouse/keeper-disable-digest-default
...
Disable real-time digest in Keeper by default
2022-07-21 15:15:22 +02:00
Antonio Andelic
7d30ab80c4
Fix compatibility
2022-07-21 09:31:06 +00:00
Alexey Milovidov
19c4c0bd6e
Update CoordinationSettings.cpp
2022-07-21 06:41:42 +03:00
Nikolai Kochetov
91043351aa
Fixing build.
2022-07-20 20:30:16 +00:00
Antonio Andelic
a4450225c7
Disable digest by default
2022-07-19 17:14:44 +00:00
Antonio Andelic
a3e00faaf9
Fix unit tests
2022-07-19 13:14:36 +00:00
Antonio Andelic
ce570b6ee3
Add logs and 4LW for api version
2022-07-19 09:02:57 +00:00
Antonio Andelic
e6ded88ea3
Small refactoring
2022-07-19 08:51:12 +00:00
Antonio Andelic
cfc741030f
Revert to the version with path
2022-07-18 14:30:15 +00:00
Antonio Andelic
48db20d5a1
Add 4LW for api version
2022-07-18 12:19:16 +00:00
Antonio Andelic
b8ffb151dc
Some small polishing
2022-07-18 08:52:52 +00:00
Antonio Andelic
7d7c4ce4cd
Add ApiVersion request
2022-07-18 08:44:38 +00:00
Antonio Andelic
73e0c35ab0
Use 4LW for api version
2022-07-15 10:13:39 +00:00
Antonio Andelic
d81758898a
Remove extra newline
2022-07-13 15:26:25 +00:00
Antonio Andelic
f05f22aa13
Merge branch 'master' into keeper-version-check
2022-07-13 15:25:48 +00:00
Antonio Andelic
3b5bdd1e2a
Add test for current API
2022-07-13 12:41:16 +00:00
Antonio Andelic
c296e84cab
fix unit tests in debug mode
2022-07-13 12:33:18 +00:00
Antonio Andelic
838a11a85a
Rename RaftAppendResult
2022-07-13 11:35:06 +00:00
Antonio Andelic
2634f80956
Remove trailing whitespaces
2022-07-13 11:06:58 +00:00
Antonio Andelic
b49ac48bfd
Merge branch 'master' into keeper-linearizable-reads
2022-07-13 09:30:21 +00:00
Nikita Mikhaylov
1f0280bcde
Fix path retrieval for Keeper's state ( #39148 )
2022-07-13 11:25:34 +02:00
Antonio Andelic
ee4828b084
Fix list watches
2022-07-13 09:23:05 +00:00
Antonio Andelic
8bd86c39f6
Add valid values for read_mode setting
2022-07-12 08:00:31 +00:00
Antonio Andelic
fd1e5d43a0
fix unit tests in debug mode
2022-07-12 07:52:50 +00:00
Antonio Andelic
5f661e3e2b
Merge branch 'master' into keeper-linearizable-reads
2022-07-12 07:33:34 +00:00
Antonio Andelic
0555477ec1
Remove debug logs
2022-07-12 07:05:29 +00:00
Antonio Andelic
6dcdf0fcb1
Throw CORRUPTED_DATA in debug mode for invalid checksum
2022-07-12 06:41:58 +00:00
Antonio Andelic
1f5b3b9922
Add restrictions for modification of internal paths
2022-07-11 13:06:24 +00:00
Antonio Andelic
568ba5d3cc
Address PR comments
2022-07-11 12:56:09 +00:00
Antonio Andelic
efc7f5c534
Merge branch 'master' into keeper-persist-state
2022-07-11 12:43:06 +00:00
Antonio Andelic
41dd4e13f4
Merge branch 'master' into keeper-version-check
2022-07-11 09:38:20 +00:00
Antonio Andelic
e4f7635df7
Format
2022-07-10 19:10:27 +00:00
Antonio Andelic
183f5cc940
Add checksum to the state file
2022-07-10 15:01:38 +00:00
Antonio Andelic
2b26865f90
Unit test fixes
2022-07-10 14:40:08 +00:00
Antonio Andelic
4622084d2a
Add test for persisted state
2022-07-10 09:47:04 +00:00
Antonio Andelic
f60074e9ec
Make state durable
2022-07-10 09:33:49 +00:00
Antonio Andelic
37f799550b
Merge pull request #38072 from lingpeng0314/master
...
Add Keeper related monitoring data
2022-07-10 09:39:57 +02:00
Alexander Tokmakov
7630fc76ae
avoid weird exception
2022-07-07 17:19:05 +02:00
Antonio Andelic
e96af48e28
Polish
2022-07-07 11:35:02 +00:00
alesapin
bd8a3ee841
Merge pull request #38338 from ClickHouse/zookeeper-add-extra-list-argument
...
Extend ZooKeeper list request with support for filtering persistent or ephemeral nodes only
2022-07-05 12:53:51 +02:00
Antonio Andelic
db94955319
Merge pull request #38556 from ClickHouse/keeper-dont-rollback-session-id
...
Don't rollback SessionID request in Keeper
2022-07-05 10:36:56 +02:00
alesapin
dde76824ca
Fix ub
2022-07-02 16:02:42 +02:00
Antonio Andelic
8c77632881
Fix style
2022-07-01 14:58:19 +00:00
Antonio Andelic
97e32e6ad9
Use commit_ext in unit tests
2022-07-01 14:55:26 +00:00
alesapin
919baf331e
Review fixes
2022-07-01 15:57:24 +02:00
Antonio Andelic
5a97c0fc8b
Finalize requests in separate thread
2022-07-01 13:19:28 +00:00
Antonio Andelic
ae0fe1ac85
Add setting for read mode
2022-07-01 09:30:59 +00:00
alesapin
66705eacb0
Add test for keeper mntr command
2022-07-01 00:07:22 +02:00
Antonio Andelic
7333e36639
Merge branch 'master' into keeper-linearizable-reads
2022-06-29 13:59:31 +00:00
Antonio Andelic
2de659715b
Don't rollback SessionID request
2022-06-29 10:21:37 +00:00
Antonio Andelic
7a8fbbf132
Merge branch 'master' into zookeeper-add-extra-list-argument
2022-06-29 08:31:03 +00:00
Antonio Andelic
1d26446f84
Define new list request type
2022-06-29 08:30:39 +00:00
Antonio Andelic
77144322d7
Add extra checks for read result
2022-06-29 07:39:39 +00:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to
...
Replace where possible by std::lock_guard which is more light-weight.
2022-06-28 19:19:06 +00:00
Antonio Andelic
73000a042d
Don't reconnect manually
2022-06-28 13:44:30 +00:00
Antonio Andelic
6d7050cb42
Additional refactoring for request processing
2022-06-28 09:54:16 +00:00
Antonio Andelic
23c705222f
Process read requests in bg thread
2022-06-28 08:48:19 +00:00
Antonio Andelic
a6c628cab4
Small fixes for queues
2022-06-27 14:34:27 +00:00
Antonio Andelic
dc5cc33769
Process outside of lock
2022-06-27 14:22:53 +00:00
Antonio Andelic
53e906fdd5
Improve batching by processing single request from each session
2022-06-27 12:05:27 +00:00
Antonio Andelic
8a3531c2db
Refactor batching requests logic
2022-06-27 07:30:59 +00:00
Antonio Andelic
0d906e311d
PR review fixes
2022-06-27 06:53:09 +00:00
Antonio Andelic
76c4fd9c8a
Some fixes for batch reads
2022-06-24 14:42:24 +00:00
Antonio Andelic
ecd545fe4f
Batch read requests
2022-06-24 11:36:12 +00:00
Antonio Andelic
91637bf79a
Merge branch 'master' into keeper-linearizable-reads
2022-06-23 14:07:00 +00:00
Antonio Andelic
2e71ae97a1
Add unit test for different list request types
2022-06-23 10:28:12 +00:00
Antonio Andelic
ae3d9fd962
Expose list request type in internal client
2022-06-23 10:28:12 +00:00
Antonio Andelic
568c957eb2
Fix TestKeeper
2022-06-23 10:28:12 +00:00
Antonio Andelic
3a71b63b5d
Add list request type
2022-06-23 10:28:12 +00:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa
2022-06-20 16:39:32 +02:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA)
...
- TSA is a static analyzer build by Google which finds race conditions
and deadlocks at compile time.
- It works by associating a shared member variable with a
synchronization primitive that protects it. The compiler can then
check at each access if proper locking happened before. A good
introduction are [0] and [1].
- TSA requires some help by the programmer via annotations. Luckily,
LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
std::shared_mutex and std::scoped_lock. This commit enables them
(--> contrib/libcxx-cmake/CMakeLists.txt).
- Further, this commit adds convenience macros for the low-level
annotations for use in ClickHouse (--> base/defines.h). For
demonstration, they are leveraged in a few places.
- As we compile with "-Wall -Wextra -Weverything", the required compiler
flag "-Wthread-safety-analysis" was already enabled. Negative checks
are an experimental feature of TSA and disabled
(--> cmake/warnings.cmake). Compile times did not increase noticeably.
- TSA is used in a few places with simple locking. I tried TSA also
where locking is more complex. The problem was usually that it is
unclear which data is protected by which lock :-(. But there was
definitely some weird code where locking looked broken. So there is
some potential to find bugs.
*** Limitations of TSA besides the ones listed in [1]:
- The programmer needs to know which lock protects which piece of shared
data. This is not always easy for large classes.
- Two synchronization primitives used in ClickHouse are not annotated in
libcxx:
(1) std::unique_lock: A releaseable lock handle often together with
std::condition_variable, e.g. in solve producer-consumer problems.
(2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
considered a design flaw + typically it is slower than a standard
mutex. In this commit, one std::recursive_mutex was converted to
std::mutex and annotated with TSA.
- For free-standing functions (e.g. helper functions) which are passed
shared data members, it can be tricky to specify the associated lock.
This is because the annotations use the normal C++ rules for symbol
resolution.
[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
lingpeng0314
f06b19bdbf
Add keeper monitoring data
2022-06-20 11:32:18 +08:00
Antonio Andelic
d56894e04e
Don't rollback session_id
2022-06-18 19:16:17 +00:00
Antonio Andelic
d2f3d581eb
Use fast lin reads in jepsen
2022-06-18 18:43:02 +00:00
Antonio Andelic
9fe06019c0
Some small fixes
2022-06-17 18:42:46 +00:00
Antonio Andelic
125e1bd182
Initial version for linearizable reads
2022-06-17 18:19:09 +00:00
Antonio Andelic
ac0b7ab20b
Fix backwards compatibility with older snapshots
2022-06-15 13:46:27 +00:00
Antonio Andelic
7e1f64002d
Address review comments
2022-06-15 12:48:30 +00:00
Antonio Andelic
6e55593398
Fix remove preprocess
2022-06-14 13:23:46 +00:00
Antonio Andelic
b7bd5a8eb1
Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations
2022-06-14 12:51:35 +00:00
Maksim Kita
da8b1b1eba
Merge pull request #38025 from kitaisreal/use-base-sort-instead-of-standard
...
Use pdqsort instead of standard sort
2022-06-14 12:13:35 +02:00
Antonio Andelic
5d4b289550
Update stats in update delta
2022-06-14 09:14:53 +00:00
Antonio Andelic
bf27fe284c
Use std::list for deltas
2022-06-14 07:37:02 +00:00
Antonio Andelic
7e99e9fe43
Fix child num assert
2022-06-14 07:17:11 +00:00
Antonio Andelic
e5504f1b33
Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations
2022-06-13 15:43:10 +00:00
Maksim Kita
98a89b50ff
Use pdqsort instead of standard sort
2022-06-13 15:31:08 +02:00
mergify[bot]
2bf9818af0
Merge branch 'master' into keeper-broken-to-detached
2022-06-13 10:04:54 +00:00
alesapin
aff6b7a8d6
Fix keeper storage
2022-06-12 00:01:41 +02:00
alesapin
aa1fb9b2e7
Remove trash
2022-06-10 15:27:27 +02:00
alesapin
4c574f30af
Add comment
2022-06-10 15:12:14 +02:00
alesapin
682eb6bcc9
Add checks for numChildren
2022-06-10 15:10:45 +02:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros
...
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.
__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
2022-06-10 10:22:31 +02:00
Antonio Andelic
2c37fe3d7b
Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations
2022-06-06 12:39:55 +00:00
Antonio Andelic
3de1d10d93
move comment
2022-06-06 12:37:03 +00:00
Antonio Andelic
6bb2d897b4
Merge branch 'master' into keeper-broken-to-detached
2022-06-06 12:19:34 +00:00
Antonio Andelic
fbdc499533
Extract more logic into a function
2022-06-06 12:19:13 +00:00
Michael Lex
041b8f6a22
Fix bug in clickhouse-keeper: dead_session_check_period was passed as micros instead of millis. ( #37824 )
2022-06-03 15:35:45 +02:00
mergify[bot]
d115ddb40c
Merge branch 'master' into keeper-broken-to-detached
2022-06-02 07:39:13 +00:00
Alexey Milovidov
b5f48a7d3f
Merge branch 'master' of github.com:ClickHouse/ClickHouse into llvm-14
2022-06-01 22:09:58 +02:00
mergify[bot]
4aefcf29e3
Merge branch 'revert-37534-revert-37036-keeper-preprocess-operations' into keeper-real-time-digest
2022-06-01 10:17:49 +00:00
mergify[bot]
cbb26d1677
Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations
2022-06-01 10:16:41 +00:00
Alexey Milovidov
4bb04f913f
Fix clang-tidy-14
2022-05-31 17:20:07 +02:00
Azat Khuzhin
d86181d3cd
keeper: store only unique session IDs for watches
...
This should speed up keeper, especially in case of incorrect usage (like
the case that had been fixed in #37640 ), especially in case on non
release build.
And also this should fix SIGKILL in stress tests.
You will find some details for one of such SIGKILL in `<details>` tag [1]:
<details>
$ pigz -cd clickhouse-server.stress.log.gz | tail
2022.05.27 16:17:24.882971 [ 637 ] {} <Trace> BackgroundSchedulePool/BgSchPool: Waiting for threads to finish.
2022.05.27 16:17:24.896749 [ 637 ] {} <Debug> MemoryTracker: Peak memory usage (for query): 4.09 MiB.
2022.05.27 16:17:24.907163 [ 637 ] {} <Debug> Application: Shut down storages.
2022.05.27 16:17:24.907233 [ 637 ] {} <Debug> Application: Waiting for current connections to servers for tables to finish.
2022.05.27 16:17:24.934335 [ 637 ] {} <Information> Application: Closed all listening sockets. Waiting for 1 outstanding connections.
2022.05.27 16:17:29.843491 [ 637 ] {} <Information> Application: Closed connections to servers for tables. But 1 remain. Probably some tables of other users cannot finish their connections after context shutdown.
2022.05.27 16:17:29.843632 [ 637 ] {} <Debug> KeeperDispatcher: Shutting down storage dispatcher
2022.05.27 16:17:34.612616 [ 688 ] {} <Test> virtual Coordination::ZooKeeperRequest::~ZooKeeperRequest(): Processing of request xid=2147483647 took 10000 ms
2022.05.27 16:17:54.612109 [ 3176 ] {} <Debug> KeeperTCPHandler: Session #12 expired
2022.05.27 16:19:59.823038 [ 635 ] {} <Fatal> Application: Child process was terminated by signal 9 (KILL). If it is not done by 'forcestop' command or manually, the possible cause is OOM Killer (see 'dmesg' and look at the '/var/log/kern.log' for the details).
Thread 26 (Thread 0x7f1c7703f700 (LWP 708)):
0 0x000000000b074b2a in __tsan::MemoryAccessImpl(__tsan::ThreadState*, unsigned long, int, bool, bool, unsigned long long*, __tsan::Shadow) ()
1 0x000000000b08630c in __tsan::MemoryAccessRange(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, bool) ()
2 0x000000000b01ff03 in memmove ()
3 0x000000001bbc8996 in std::__1::__move<long, long> (__first=0xb8600000d83304, __last=<optimized out>, __result=0x7f1c021cd000) at ../contrib/libcxx/include/__algorithm/move.h:57
4 std::__1::move<long*, long*> (__first=0xb8600000d83304, __last=<optimized out>, __result=0x7f1c021cd000) at ../contrib/libcxx/include/__algorithm/move.h:70
5 std::__1::vector<long, std::__1::allocator<long> >::erase (this=0x7b1400584c48, __position=...) at ../contrib/libcxx/include/vector:1608
6 DB::KeeperStorage::clearDeadWatches (this=0x7b5800001ad8, this@entry=0x7b5800001800, session_id=session_id@entry=12) at ../src/Coordination/KeeperStorage.cpp:1228
7 0x000000001bbc5c55 in DB::KeeperStorage::processRequest (this=0x7b5800001800, zk_request=..., session_id=12, time=1, new_last_zxid=..., check_acl=true) at ../src/Coordination/KeeperStorage.cpp:1122
8 0x000000001bba06a3 in DB::KeeperStateMachine::commit (this=<optimized out>, log_idx=3549, data=...) at ../src/Coordination/KeeperStateMachine.cpp:143
9 0x000000001bba6193 in nuraft::state_machine::commit_ext (this=0x7b4c00001f98, params=...) at ../contrib/NuRaft/include/libnuraft/state_machine.hxx:75
10 0x00000000202c5a55 in nuraft::raft_server::commit_app_log (this=this@entry=0x7b6c00002a18, idx_to_commit=idx_to_commit@entry=3549, le=..., need_to_handle_commit_elem=true, initial_commit_exec=false) at ../contrib/NuRaft/src/handle_commit.cxx:311
11 0x00000000202c4f98 in nuraft::raft_server::commit_in_bg_exec (this=<optimized out>, this@entry=0x7b6c00002a18, timeout_ms=timeout_ms@entry=0, initial_commit_exec=false) at ../contrib/NuRaft/src/handle_commit.cxx:241
12 0x00000000202c4613 in nuraft::raft_server::commit_in_bg (this=this@entry=0x7b6c00002a18) at ../contrib/NuRaft/src/handle_commit.cxx:149
...
Thread 28 (Thread 0x7f1c7603d700 (LWP 710)):
0 0x00007f1d22a6d110 in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0
1 0x00007f1d22a650a3 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
2 0x000000000b0337b0 in pthread_mutex_lock ()
3 0x00000000221884da in std::__1::__libcpp_mutex_lock (__m=0x7b4c00002088) at ../contrib/libcxx/include/__threading_support:303
4 std::__1::mutex::lock (this=0x7b4c00002088) at ../contrib/libcxx/src/mutex.cpp:33
5 0x000000001bba4188 in std::__1::lock_guard<std::__1::mutex>::lock_guard (__m=..., this=<optimized out>) at ../contrib/libcxx/include/__mutex_base:91
6 DB::KeeperStateMachine::getDeadSessions (this=0x7b4c00001f98) at ../src/Coordination/KeeperStateMachine.cpp:360
7 0x000000001bb79b4b in DB::KeeperServer::getDeadSessions (this=0x7b4400012700) at ../src/Coordination/KeeperServer.cpp:572
8 0x000000001bb64d1a in DB::KeeperDispatcher::sessionCleanerTask (this=<optimized out>, this@entry=0x7b640001c218) at ../src/Coordination/KeeperDispatcher.cpp:399
...
Thread 1 (Thread 0x7f1d227148c0 (LWP 637)):
0 0x00007f1d22a69376 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
1 0x000000000b0895e0 in __tsan::call_pthread_cancel_with_cleanup(int (*)(void*), void (*)(void*), void*) ()
2 0x000000000b017091 in pthread_cond_wait ()
3 0x0000000020569d98 in Poco::EventImpl::waitImpl (this=0x7b2000008798) at ../contrib/poco/Foundation/src/Event_POSIX.cpp:106
4 0x000000001bb636cf in Poco::Event::wait (this=0x7b2000008798) at ../contrib/poco/Foundation/include/Poco/Event.h:97
5 ThreadFromGlobalPool::join (this=<optimized out>) at ../src/Common/ThreadPool.h:217
6 DB::KeeperDispatcher::shutdown (this=0x7b640001c218) at ../src/Coordination/KeeperDispatcher.cpp:322
7 0x0000000019ca8bfc in DB::Context::shutdownKeeperDispatcher (this=<optimized out>) at ../src/Interpreters/Context.cpp:2111
8 0x000000000b0a979b in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_9::operator()() const (this=0x7ffcde44f0a0) at ../programs/server/Server.cpp:1407
</details>
[1]: https://s3.amazonaws.com/clickhouse-test-reports/37593/2613149f6bf4f242bbbf2c3c8539b5176fd77286/stress_test__thread__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-30 11:50:48 +03:00
Antonio Andelic
6a984cfa6f
Update Changelog.cpp
2022-05-27 09:45:09 +02:00
Antonio Andelic
ea1f04fed4
Test broken logs folder
2022-05-27 07:42:09 +00:00
Antonio Andelic
1ad5fe5ced
Move broken logs to detached folder
2022-05-26 13:37:20 +00:00
Antonio Andelic
d9dd11bb66
Revert "Revert "Add support for preprocessing ZooKeeper operations in clickhouse-keeper""
2022-05-26 08:14:55 +02:00
Antonio Andelic
6a962549d5
Revert "Add support for preprocessing ZooKeeper operations in clickhouse-keeper"
2022-05-25 16:45:32 +02:00
Antonio Andelic
9e5a635da2
Fix rollback
2022-05-25 14:21:26 +00:00
Antonio Andelic
fe72cd7478
Merge branch 'master' into keeper-real-time-digest
2022-05-25 07:41:31 +00:00
Antonio Andelic
61e38b9e82
Allow comitting of logs without digest
2022-05-25 07:41:00 +00:00
Antonio Andelic
c55f4e470a
Small fix
2022-05-24 13:11:11 +00:00
Antonio Andelic
85e7118300
Fix integrations test
2022-05-24 09:10:45 +00:00
Antonio Andelic
e91e7fdba7
Fix style
2022-05-23 15:33:29 +00:00
Antonio Andelic
c268296fc6
Fix single node force recovery and add tests
2022-05-23 09:53:46 +00:00
Antonio Andelic
2b8f71b4a8
Fix style
2022-05-23 07:55:23 +00:00
Antonio Andelic
ea1cbff0d8
Fix unit tests
2022-05-23 07:51:26 +00:00
Antonio Andelic
f379f225fa
Add ephemerals on preprocess
2022-05-23 06:48:39 +00:00
Antonio Andelic
d8c247cfdc
Add error
2022-05-20 10:14:51 +00:00
Antonio Andelic
25ab817c2b
Revert formatting
2022-05-20 10:12:09 +00:00
Antonio Andelic
c133f815bd
Merge branch 'keeper-preprocess-operations' into keeper-real-time-digest
2022-05-20 07:57:47 +00:00
Antonio Andelic
f5759e2d7e
Cache nodes latest state
2022-05-19 12:30:57 +00:00
Antonio Andelic
6fba1c96ed
Define small test for digest check
2022-05-17 13:53:12 +00:00
Antonio Andelic
c5e4598447
Calculate digest on preprocess
2022-05-17 10:23:51 +00:00
Antonio Andelic
639ceb84b1
Add comment and fix typo
2022-05-17 10:15:12 +00:00
alesapin
f958203b6c
Update src/Coordination/KeeperStorage.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-05-17 11:55:54 +02:00
alesapin
e75aa445a6
Update src/Coordination/KeeperStorage.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-05-17 11:36:12 +02:00
Antonio Andelic
fc5a79f186
Polishing changes
2022-05-17 08:16:55 +00:00
Antonio Andelic
b88f3fcfb1
Merge branch 'keeper-preprocess-operations' into keeper-real-time-digest
2022-05-17 06:52:30 +00:00
Antonio Andelic
573e42d0dd
Address PR comments
2022-05-17 06:45:51 +00:00
Antonio Andelic
2205c01697
Add to snapshot and config for digest
2022-05-16 13:36:31 +00:00
Antonio Andelic
99b7e23812
Add version for digest
2022-05-16 13:36:31 +00:00
Antonio Andelic
2ec61a8d7f
Initial implementation of digest
2022-05-16 13:36:31 +00:00
Antonio Andelic
6bd0b0043d
Add zxid to snapshot
2022-05-16 13:35:54 +00:00
Antonio Andelic
0dccafda2e
Fetch zxid independently of log idx
2022-05-16 13:35:54 +00:00
Antonio Andelic
adb8ac4fda
Change log level
2022-05-16 13:15:31 +00:00
Antonio Andelic
f3646cea41
Rename preprocess test
2022-05-11 12:17:29 +00:00
Antonio Andelic
65af47a90b
Merge branch 'master' into keeper-preprocess-operations
2022-05-11 12:11:17 +00:00
Antonio Andelic
e6d187001c
Fix unit tests and modify messages
2022-05-11 12:10:17 +00:00
Antonio Andelic
b2aa1802cd
Add unit test for basic CRUD with preprocessing
2022-05-11 09:08:55 +00:00
Antonio Andelic
0fb11ab3ff
Rename uncommitted state
2022-05-11 09:08:39 +00:00
Antonio Andelic
15672a8374
Fix integration tests
2022-05-11 07:53:32 +00:00
Antonio Andelic
79080ff0cd
Fix style
2022-05-10 13:43:45 +00:00
Antonio Andelic
8610d74177
init cversion
2022-05-10 13:31:39 +00:00
Antonio Andelic
88007809c2
Add rollback support
2022-05-10 13:04:35 +00:00
Antonio Andelic
02319f92a3
Define auth for sessions with preprocess
2022-05-10 12:53:18 +00:00
Antonio Andelic
f8d0aa4bc1
Fix update delta
2022-05-10 09:53:15 +00:00
Antonio Andelic
7eeb463fdb
Fix local
2022-05-10 07:00:38 +00:00
Antonio Andelic
a988cfa27b
Small fixes
2022-05-09 13:23:31 +00:00
Antonio Andelic
a89b57aeb6
Fix commit
2022-05-09 10:19:20 +00:00
Antonio Andelic
f5bdef4435
Remove indentation
2022-05-09 09:57:06 +00:00
Antonio Andelic
7c7bac180f
Rename variables with short names
2022-05-09 09:52:43 +00:00
Antonio Andelic
246a5043b4
Rewrite commit with single lambda
2022-05-09 09:52:39 +00:00
Antonio Andelic
285bb21b91
Define close session
2022-05-09 09:35:16 +00:00
Antonio Andelic
9796527890
Support Auth check
2022-05-09 09:16:05 +00:00
Antonio Andelic
ad7226e151
Add processLocal for read requests
2022-05-09 08:32:25 +00:00
Antonio Andelic
7f6fa9fe83
Fix splitting impl of basic operations
2022-05-09 07:02:42 +00:00
Antonio Andelic
cff68aa31f
Precommit implementation
2022-05-09 07:02:42 +00:00
Antonio Andelic
94ba3d129c
Define preprocess with updated nodes
2022-05-09 07:02:42 +00:00
Antonio Andelic
c4cf1c863b
Add precommit initial
2022-05-09 07:02:42 +00:00
alesapin
f4cdd86ae8
Force keeper snapshot equality on different replicas
2022-05-05 19:23:01 +02:00
alesapin
b76406fc8f
Merge pull request #36910 from ClickHouse/add_test
...
Fix bug in keeper which could lead to corrupted compressed logs
2022-05-04 23:44:45 +02:00
alesapin
bc50ddc37a
Fix typos in text
2022-05-04 18:43:50 +02:00
alesapin
c6556da5fc
Fix bug which can lead to corrupted logs in compressed format
2022-05-04 18:08:00 +02:00
alesapin
0dffd7c44a
Fix coordination
2022-05-03 20:20:07 +02:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available""
2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available"
2022-05-03 00:36:50 +02:00
Dmitry Novik
71b6f89166
Merge pull request #35637 from ClickHouse/memory-overcommit-free
...
Memory overcommit: continue query execution if memory is available
2022-05-02 19:00:18 +02:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase
...
Reorganize source files so that base won't depend on Common
2022-04-30 09:40:58 +03:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free
2022-04-29 19:31:59 +00:00
alesapin
61dfbd4a83
Merge pull request #36758 from ClickHouse/warn_user_if_setting_is_strange
...
Add small script for keeper check
2022-04-29 12:10:36 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common
2022-04-29 10:26:35 +08:00
alesapin
6e742be0e9
Update KeeperServer.cpp
2022-04-28 19:09:46 +02:00
alesapin
db9cb4cf09
Merge pull request #36736 from ClickHouse/keeper-id-guards
...
Add an extra check for RAFT config change
2022-04-28 18:22:39 +02:00
alesapin
3107b2bcd6
Add small script for stupid keeper check
2022-04-28 18:13:54 +02:00
Antonio Andelic
44ff5f1697
Add some checks for endpoint change
2022-04-28 08:54:05 +00:00
Antonio Andelic
9646487c09
Address PR review comments
2022-04-26 07:32:02 +00:00
Antonio Andelic
cf022542bd
Merge branch 'master' into keeper-recovery-mode
2022-04-26 06:50:02 +00:00
alesapin
1fffa56f63
Merge branch 'master' into memory-overcommit-free
2022-04-25 20:21:08 +02:00
mergify[bot]
eefc302779
Merge branch 'master' into fix-keeper-hostname-check
2022-04-22 14:07:53 +00:00
Antonio Andelic
4e389d8df1
Rename to hostname, add tests
2022-04-22 08:16:14 +00:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes
...
Clang tidy fixes
2022-04-21 16:11:27 +02:00
Antonio Andelic
6ac3784af7
Add config example
2022-04-21 13:49:36 +00:00
Antonio Andelic
1778ddda97
Simplify code
2022-04-21 13:32:12 +00:00
Antonio Andelic
792b06cc66
Add config for host check
2022-04-21 13:32:12 +00:00
Antonio Andelic
b2770fbe1c
Add setting for disabling checks
2022-04-21 13:32:12 +00:00
Antonio Andelic
03779fd463
Merge branch 'master' into keeper-recovery-mode
2022-04-21 12:34:18 +00:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings
...
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
2022-04-20 10:29:05 +02:00
Antonio Andelic
bb0d941add
Run tests on a single cluster
2022-04-20 08:06:17 +00:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains
...
Activate clang-tidy warning "readability-container-contains"
2022-04-20 08:15:46 +02:00
Antonio Andelic
e129f8ce00
Small fix
2022-04-19 12:39:34 +00:00
Antonio Andelic
70f3d3a863
Merge branch 'master' into keeper-recovery-mode
2022-04-19 12:32:48 +00:00
Antonio Andelic
3e77340a81
Allow clickhouse-keeper to serve 4-letter commands before quorum ( #35992 )
...
* Use async start for clichkouse-keeper
* Return non active server for 4lw commands if quorum not achieved
2022-04-19 14:03:00 +02:00
Antonio Andelic
73db184ad6
Ignore requests on leader while in recovery mode
2022-04-19 10:23:54 +00:00
Antonio Andelic
608c0996d0
Fix tests
2022-04-19 08:59:44 +00:00
Antonio Andelic
6c3bf0a5d3
Format files
2022-04-19 08:08:36 +00:00
Antonio Andelic
272965fc44
Refactoring
2022-04-19 08:08:35 +00:00
Antonio Andelic
4ecb66c1ad
Support recovery with four letter commands
2022-04-19 08:08:35 +00:00
Antonio Andelic
ff2ebe113c
WIP
2022-04-19 08:08:35 +00:00
Antonio Andelic
dbd88a5acb
Use 1 leader quorum for recovery
2022-04-19 08:08:35 +00:00
Antonio Andelic
0e1ba927bd
Add argument for force recovery
2022-04-19 08:08:35 +00:00
Antonio Andelic
bb4bc17af1
Force recover from configuration
2022-04-19 08:08:35 +00:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
...
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexey Milovidov
f86808914c
Fix strange trash in Keeper
2022-04-18 04:44:30 +02:00
alesapin
c0317c818c
Fix bug in keeper
2022-04-14 17:33:46 +02:00
zhanglistar
0ae820550d
fix ut error sometimes
2022-04-12 14:30:53 +08:00
Antonio Andelic
bf1f34ddb3
Fix unit tests
2022-04-06 07:43:48 +00:00
Antonio Andelic
d296eeee2d
Small changes for Keeper
2022-04-05 13:56:28 +00:00
alesapin
d0bafe93d6
Fix name
2022-03-19 21:02:11 +01:00
alesapin
bf5b3a856d
Rename some variables in keeper
2022-03-17 11:55:15 +01:00
zhangyuli1
3cba1177ee
remove wchc from four_letter_word_white_list
2022-03-16 17:40:49 +08:00
Maksim Kita
65c52298b6
Fix clang-tidy warnings in Compression, Coordination, Core folders
2022-03-14 18:17:35 +00:00
alesapin
71ecd6be74
Merge pull request #35004 from ClickHouse/more_keeper_sanity_checks
...
More keeper sanity checks
2022-03-07 19:53:59 +01:00
alesapin
0c93c9e21d
Merge pull request #35010 from bigo-sg/keeperstatopt
...
Keeper atomic stat
2022-03-07 12:34:28 +01:00
alesapin
e81bbfb5b5
Review fixes
2022-03-07 12:13:37 +01:00
alesapin
4d86bf6fdf
Merge branch 'master' into more_keeper_sanity_checks
2022-03-07 11:41:04 +01:00
zhanglistar
9efc8a1d38
Fix min/max stat
2022-03-07 11:11:14 +08:00
alesapin
5db9018ae0
Fix unitialized variable
2022-03-04 13:26:56 +01:00
alesapin
2ab920a404
Review fixes
2022-03-04 13:14:38 +01:00
alesapin
c33808d3db
Fix read old records from logs
2022-03-03 15:18:03 +01:00
alesapin
f24fa16184
Fix address check
2022-03-03 11:29:43 +01:00
zhanglistar
9ee0d2c8a0
keeper atomic stat
2022-03-03 14:30:22 +08:00
alesapin
04d4c52e01
Add sanity checks for keeper
2022-03-02 20:37:59 +01:00
alesapin
74ff3ee4bc
Add sanity check for localhost
2022-03-02 20:02:02 +01:00
Maksim Kita
b1a956c5f1
clang-tidy check performance-move-const-arg fix
2022-03-02 18:15:27 +00:00
alesapin
576ff0b40d
Merge pull request #34715 from azat/fix-opened-file-cache-race
...
Workaround for a bug in NuRaft library
2022-03-02 13:58:43 +01:00
Azat Khuzhin
54ceadd4b3
Avoid busy polling in keeper while searching for changelog files to delete
...
Fixes : #34534
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-27 18:27:49 +03:00
alesapin
fd95d7a498
Merge pull request #34486 from bigo-sg/keeperVersion
...
Keeper: add version to SnapshotableHashTable
2022-02-23 13:35:49 +03:00
alesapin
9d21911ca7
Merge pull request #34534 from bigo-sg/changelogAsyncDel
...
Keeper changelog async delete useless logs
2022-02-23 10:30:23 +03:00
zhanglistar
0ca819197d
delete useless header in changelog
2022-02-22 19:13:55 +08:00
zhanglistar
1291483094
changelog modify log
2022-02-22 19:09:45 +08:00
zhanglistar
96fe231c35
Changelog delete useless sleep in dctor
2022-02-22 19:07:33 +08:00
zhanglistar
6db770f841
Changelog add some logs and adjust thread exiting logic
2022-02-22 18:55:31 +08:00
zhangxiao871
a06e4b991e
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into yandex-master
2022-02-22 11:10:47 +08:00
zhangxiao871
ad124a4d10
compatible.
2022-02-22 11:10:14 +08:00
zhanglistar
0299fd296d
Use ConcurrentBoundedQueue instead of boost spsc queue.
2022-02-22 09:34:33 +08:00
Azat Khuzhin
0025110a16
Replace exit() with abort() in case of NuRaft errors
...
CI founds [1]:
WARNING: ThreadSanitizer: data race (pid=99)
Write of size 8 at 0x7b14002192b0 by thread T27:
12 std::__1::map<>
13 DB::OpenedFileCache::~OpenedFileCache() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:27:7 (clickhouse+0xac66de6)
14 cxa_at_exit_wrapper(void*) crtstuff.c (clickhouse+0xaa3646f)
15 decltype(*(std::__1::forward<nuraft::raft_server*&>(fp0)).*fp()) std::__1::__invoke<void ()(), nuraft::raft_server*&, void>()
Previous read of size 8 at 0x7b14002192b0 by thread T37 (mutexes: write M732116415018761216):
4 DB::OpenedFileCache::get() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:47:37 (clickhouse+0xac66784)
Thread T27 'nuraft_commit' (tid=193, running) created by main thread at:
...
Thread T37 'MergeMutate' (tid=204, running) created by main thread at:
...
But it also reports that the mutex was already destroyed:
Mutex M732116415018761216 is already destroyed.
The problem is that [nuraft can call `exit()`](1707a7572a/src/handle_commit.cxx (L157-L158)https://s3.amazonaws.com/clickhouse-test-reports/33057/5a8cf3ac98808dadf125068a33ed9c622998a484/fuzzer_astfuzzertsan,actions//report.html
Let's replace exit() with abort() to avoid this.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-19 13:05:08 +03:00
alesapin
efb9a6d0fa
Merge pull request #34584 from bigo-sg/keerpSnapMemOpt
...
Keeper memory optimization by not holding snapshot in memory
2022-02-18 10:34:31 +03:00
zhanglistar
677b34d41d
Merge branch 'ClickHouse:master' into keeperVersion
2022-02-18 14:07:30 +08:00
zhanglistar
03c648e317
Update src/Coordination/KeeperStateMachine.cpp
...
Co-authored-by: alesapin <alesapin@gmail.com>
2022-02-17 18:15:25 +08:00
zhanglistar
cd0f7d5b16
Fix buffer not alloced bug
2022-02-17 16:33:35 +08:00
zhanglistar
7868153098
no std::cerr in KeeperSnapshotManager.h
2022-02-17 10:49:33 +08:00
zhanglistar
f2b2723a3d
add buffer alloc and errno
2022-02-17 10:34:22 +08:00
zhanglistar
213a3481b5
move mmap file2buffer to function
2022-02-15 20:25:19 +08:00
alesapin
c2bdcbbe77
Merge pull request #34587 from bigo-sg/changelogNoclone
...
Keeper change log no need to clone log entry
2022-02-15 14:06:28 +03:00
zhanglistar
64d40fad2b
Merge branch 'master' into keeperVersion
2022-02-15 09:22:54 +08:00
Maksim Kita
db245cfbc9
Merge pull request #34523 from bigo-sg/nolockappendentries
...
Keeper no lock on append_entries
2022-02-14 21:28:10 +01:00
zhanglistar
b790903a61
keeper Changelog no need clone log entry
2022-02-14 22:43:37 +08:00
zhanglistar
ceeed2e928
keeper no memory snapshot
2022-02-14 21:34:55 +08:00
zhanglistar
edcea7dc31
keeper snapshot memory opt.
2022-02-14 19:43:08 +08:00
alexey-milovidov
ea71dc9d11
Merge pull request #34510 from kitaisreal/table-functions-insert-partition-by-refactoring
...
Improve performance of insert into table functions URL, S3, File, HDFS
2022-02-12 10:14:00 +03:00
zhanglistar
1dad40e25f
Delete useless header
2022-02-12 00:04:08 +08:00
zhanglistar
5906b36095
Keeper changelog async delete useless logs
2022-02-11 23:58:15 +08:00
zhanglistar
e07c5751fa
No lock on append_entries
2022-02-11 17:18:55 +08:00
