thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 09:52:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
140,106 Commits 9,324 Branches 1,783 Tags 2.1 GiB
642da7934a
Commit Graph

1285 Commits

Author SHA1 Message Date
Antonio Andelic
9d218a8c7b Better 2023-10-16 08:50:53 +00:00
Antonio Andelic
2d851ec163 Better 2023-10-16 08:22:39 +00:00
Antonio Andelic
58120b3f47 Polish 2023-10-16 08:17:23 +00:00
Antonio Andelic
606f742254 Merge branch 'master' into keeper-better-preprocessing 2023-10-16 06:57:19 +00:00
Antonio Andelic
c69838e052
Merge pull request #55591 from ClickHouse/fix-keeper-context 
Apply Context changes to standalone Keeper
 2023-10-16 08:41:17 +02:00
alesapin
3b02748cb6 Fix some typos 2023-10-15 15:43:02 +02:00
Antonio Andelic
67068849da
Fix comment 2023-10-13 15:59:37 +02:00
Azat Khuzhin
720ea8441c Fix global context for tests with --gtest_filter 
If you run tests that requires context, but do not initialize it, then
it will SIGSEGV, because Context is not initialized.

Fix this by using google test envrionment, and instead of gtest_main
implement own main function that will initialize it.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-13 13:44:30 +02:00
Antonio Andelic
48933f52fe Fix Keeper Context 2023-10-13 11:30:37 +00:00
Antonio Andelic
34d8bb18cf Better preprocessing in Keeper 2023-09-29 14:34:17 +00:00
Antonio Andelic
01f64be558 Set path in response 2023-09-26 07:41:05 +00:00
Antonio Andelic
176baa6695 Merge branch 'master' into keeper-operation-create-if-not-exists 2023-09-22 13:47:30 +00:00
Antonio Andelic
b65b1ee97b Merge branch 'master' into keeper-operation-create-if-not-exists 2023-09-20 13:56:36 +00:00
Antonio Andelic
88ab5514a0 Merge branch 'master' into keeper-reduce-batch-size 2023-09-20 13:56:06 +00:00
Sema Checherinda
e7550523c8
Merge pull request #54651 from CheSema/limit_backoff_timeout 
limit the delay before next try in S3
 2023-09-18 19:41:26 +02:00
Antonio Andelic
7f3279fa21 Revert batch size to 100 for Keeper 2023-09-18 11:18:39 +00:00
Antonio Andelic
69333912d5 Merge branch 'master' into keeper-operation-create-if-not-exists 2023-09-18 10:57:11 +00:00
Robert Schulze
7d04cf6b58
Merge pull request #54646 from rschu1ze/bump-abseil 
Bump abseil
 2023-09-18 09:44:35 +02:00
Robert Schulze
5f46106984
Try to fix sizes in unit tests, pt. III 2023-09-17 14:41:51 +00:00
Robert Schulze
066bdf57c4
Try to fix sizes in unit test, pt. II 2023-09-15 14:53:05 +00:00
Robert Schulze
603b3c4b26
Try to fix sizes in unit test 2023-09-15 09:32:43 +00:00
Sema Checherinda
d9e15c00c9 limit the delay before next try in S3 2023-09-14 19:45:07 +02:00
Antonio Andelic
ae88bd3da5
Merge pull request #54572 from ClickHouse/fix-install-backup 
Fix snapshot install in Keeper
 2023-09-14 14:44:45 +02:00
Antonio Andelic
1c18f6c21d Use log idx for applyUncommittedState 2023-09-13 10:13:02 +00:00
Antonio Andelic
479ae52f9c Merge branch 'master' into keeper-batch-flushes 2023-09-11 11:36:23 +00:00
serxa
6a2534b555 fix keeper build #2 2023-09-10 17:43:04 +00:00
serxa
d326554caa fix keeper build 2023-09-10 14:55:40 +00:00
Antonio Andelic
4a1e2959fa Apply PR comments 2023-09-08 12:52:35 +00:00
Antonio Andelic
5f1a75ceae Fix tests 2023-09-01 12:13:33 +00:00
Antonio Andelic
7739dd7157 Fix integration tests 2023-08-31 14:52:42 +00:00
Antonio Andelic
dd824fe558 Merge branch 'master' into keeper-batch-flushes 2023-08-31 14:35:36 +00:00
Antonio Andelic
4137cb916d Merge branch 'master' into keeper-batch-flushes 2023-08-28 13:14:06 +00:00
alesapin
0e601b856f
Merge pull request #53751 from Algunenano/clang18 
Support clang-18 (Wmissing-field-initializers)
 2023-08-25 23:27:06 +02:00
Pengyuan Bian
c82cc148e9 Add configuration for reconnect limit in raft limit. 2023-08-25 00:38:49 +00:00
Raúl Marín
6490213207 Merge remote-tracking branch 'blessed/master' into clang18 2023-08-24 17:35:26 +02:00
Arthur Passos
2bade7db08
Add global proxy setting (#51749) 
* initial impl

* fix env ut

* move ut directory

* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider

* minor adjustment

* add a few tests, still incomplete

* add proxy support for url table function

* use proxy for select from url as well

* remove optional from return type, just returns empty config

* fix style

* style

* black

* ohg boy

* rm in progress file

* god pls don't let me kill anyone

* ...

* add use_aws guards

* remove hard coded s3 proxy resolver

* add concurrency-mt-unsafe

* aa

* black

* add logging back

* revert change

* imrpove code a bit

* helper functions and separate tests

* for some reason, this env test is not working..

* formatting

* :)

* clangtidy

* lint

* revert some stupid things

* small test adjusmtments

* simplify tests

* rename test

* remove extra line

* freaking style change

* simplify a bit

* fix segfault & remove an extra call

* tightly couple proxy provider with context..

* remove useless include

* rename config prefix parameter

* simplify provider a bit

* organize provider a bit

* add a few comments

* comment out proxy env tests

* fix nullptr in unit tests

* make sure old storage proxy config is properly covered without global context instance

* move a few functions from class to anonymous namespace

* fix no fallback for specific storage conf

* change API to accept http method instead of bool

* implement http/https distinction in listresolver, any still not implemented

* implement http/https distinction in remote resolver

* progress on code, improve tests and add url function working test

* use protcol instead of method for http and https

* small fix

* few more adjustments

* fix style

* black

* move enum to proxyconfiguration

* wip

* fix build

* fix ut

* delete atomicroundrobin class

* remove stale include

* add some tests.. need to spend some more time on the design..

* change design a bit

* progress

* use existing context for tests

* rename aux function and fix ut

* ..

* rename test

* try to simplify tests a bit

* simplify tests a bit more

* attempt to fix tests, accept more than one remote resolver

* use proper log id

* try waiting for resolver

* proper wait logic

* black

* empty

* address a few comments

* refactor tests

* remove old tests

* baclk

* use RAII to set/unset env

* black

* clang tidy

* fix env proxy not respecting any

* use log trace

* fix wrong logic in getRemoteREsolver

* fix wrong logic in getRemoteREsolver

* fix test

* remove unwanted code

* remove ClientConfigurationperRequest and auxilary classes

* remove unwanted code

* remove adapter test

* few adjustments and add test for s3 storage conf  with new proxy settings

* black

* use chassert for context

* Add getenv comment
 2023-08-24 16:07:26 +03:00
Raúl Marín
93dac0c880 Support clang-18 (Wmissing-field-initializers) 2023-08-23 15:53:45 +02:00
Antonio Andelic
625be86dc0 Fix tests 2023-08-23 09:00:57 +00:00
Antonio Andelic
59557ddd7c Merge branch 'master' into keeper-batch-flushes 2023-08-23 08:11:43 +00:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache 
Better usage of ccache
 2023-08-11 04:43:45 +03:00
Alexey Milovidov
ae6e59c4c5
Merge pull request #53099 from ClickHouse/remove-wikistat-loader 
Remove old util
 2023-08-10 17:01:57 +03:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
Alexey Milovidov
aa757490bd Ditch tons of garbage 2023-08-09 02:19:02 +02:00
Alexey Milovidov
1ab99e5fdd Replace old docs with symlinks 2023-08-08 22:38:52 +02:00
Antonio Andelic
1dbe007ee5 Add coordination setting for async replication 2023-08-08 13:12:07 +00:00
Antonio Andelic
4b3e399f7d Wait for read requests 2023-08-07 13:04:38 +00:00
Antonio Andelic
c0ff486ada Merge branch 'master' into keeper-batch-flushes 2023-08-07 06:20:41 +00:00
Antonio Andelic
5cb856d167 Try batching multiple flush requests in Keeper 2023-08-04 11:53:22 +00:00
Vitaly Baranov
552331fd54 Use the same default paths for clickhouse_keeper (symlink) as for clickhouse_keeper (executable), 
i.e. "/var/lib/clickhouse-keeper/.."
 2023-08-04 13:38:21 +02:00
Anton Popov
92cb5e9ae3
Merge pull request #52687 from alkorgun/fix-named-collections-on-cluster-23.7 
Fix named collections on cluster 23.7
 2023-08-03 14:50:28 +02:00
Antonio Andelic
a6a495a767
Merge branch 'master' into fix-keeper-reductant-lock 2023-07-31 19:53:25 +02:00
Antonio Andelic
a473dc4b51 Wait for response 2023-07-31 09:24:06 +00:00
Илья Коргун
bcc41fbc2f Fix reference to a local binding declared in an enclosing function 2023-07-27 10:46:35 +03:00
Alexey Milovidov
21382afa2b Check for punctuation 2023-07-25 06:10:04 +02:00
Antonio Andelic
7b03f3277e
Try fix session on close test 2023-07-24 17:59:50 +02:00
Antonio Andelic
73155cf562
Merge branch 'master' into fix-keeper-reductant-lock 2023-07-24 10:17:35 +02:00
Antonio Andelic
10c2d5dade
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-21 13:24:18 +02:00
Antonio Andelic
4a47040763
Merge pull request #52358 from ClickHouse/fix-keeper-apply-snapshot 
Fix `apply_snapshot` in Keeper
 2023-07-21 12:52:12 +02:00
Antonio Andelic
f997adfe27 Retry if sessions not closed because missing leader 2023-07-20 14:02:55 +00:00
Antonio Andelic
a24bf14450 Use correct ZXID 2023-07-20 10:44:44 +00:00
Antonio Andelic
0c86df519f Fix unspported disks in Keeper 2023-07-19 12:41:25 +00:00
Antonio Andelic
61dc020b4d
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-17 12:50:17 +02:00
Antonio Andelic
9677cc5e2b
Apply suggestions from code review 2023-07-17 12:50:02 +02:00
frinkr
e1abdd0419
atomic exchange simplified 2023-07-17 10:51:24 +08:00
frinkr
6dceedd721
Merge branch 'ClickHouse:master' into fix-keeper-reductant-lock 2023-07-17 10:21:05 +08:00
Azat Khuzhin
6abdde12ca Print Zxid in keeper stat command in hex (so as ZooKeeper) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-14 21:17:09 +02:00
Alexey Milovidov
b1d1918607
Merge branch 'master' into retry 2023-07-08 03:08:48 +03:00
Alexey Milovidov
685f2949b7
Revert "Fix build" 2023-07-07 02:53:13 +03:00
Alexey Milovidov
c47b32b17a Fix build 2023-07-07 01:52:18 +02:00
Alexey Milovidov
48eb30de51 Fix build 2023-07-07 01:42:29 +02:00
Mike Kot
5302b478a4 proper reconfig batch handling 2023-07-07 00:20:54 +03:00
Mike Kot
bafcc3afdc remove reconfig in process flag as it's useless 2023-07-07 00:20:54 +03:00
Mike Kot
c46b125d0a review fixes 2023-07-07 00:20:54 +03:00
Mike Kot
2f0cd05497 handle leader removal corner cases 2023-07-07 00:20:54 +03:00
Mike Kot
1bef6fc76c process reconfig in keeper dispatcher 2023-07-07 00:20:54 +03:00
Mike Kot
297d566600 acl check 2023-07-07 00:20:54 +03:00
Mike Kot
b4f750ed66 review fixes 2023-07-07 00:20:54 +03:00
Mike Kot
8b6376005a "reconfig" support for CH Keeper 2023-07-07 00:20:54 +03:00
frinkr
36fbc22b16
Merge branch 'master' into fix-keeper-reductant-lock 2023-07-04 18:04:08 +08:00
Jiang Yuqing
f443a00d14
KeeperDispatcher: remove reductant lock as the ConcurrentBoundedQueue is thread-safe 2023-07-04 17:58:04 +08:00
Antonio Andelic
2058b1346e Merge branch 'master' into keeper-with-disks 2023-06-30 13:22:33 +00:00
Antonio Andelic
eff1cc0e5d
Remove unused errorcode 2023-06-28 10:20:33 +02:00
Antonio Andelic
be852d554b Use magic_enum for values 2023-06-28 07:45:20 +00:00
Antonio Andelic
efbe01b6bb Merge branch 'master' into keeper-add-api-version-config 2023-06-27 10:11:04 +00:00
Alexey Milovidov
7407330130
Merge branch 'master' into retry 2023-06-23 08:18:18 +03:00
Michael Kolupaev
4a570a05c9 Decrease default timeouts for S3 and HTTP requests 2023-06-21 18:08:50 +00:00
Sema Checherinda
1cb02e2710 do call finalize for all buffers 2023-06-16 16:38:18 +02:00
Antonio Andelic
4ac090b122 Fix tests 2023-06-13 11:01:44 +00:00
Antonio Andelic
6f1c486537
Fix build 2023-06-13 10:25:53 +02:00
Antonio Andelic
25f08f8d19
Remove duplicate include 2023-06-12 15:35:44 +02:00
Antonio Andelic
cb3ac81110
Merge branch 'master' into keeper-with-disks 2023-06-12 15:14:24 +02:00
Antonio Andelic
a7e6264d56 Add backward compatibility 2023-06-12 13:12:05 +00:00
Antonio Andelic
b634012feb Add feature flags for Keeper API 2023-06-12 10:57:03 +00:00
Antonio Andelic
91a4b6e331 Merge branch 'master' into keeper-add-api-version-config 2023-06-12 07:10:58 +00:00
Antonio Andelic
b8b70d78e1 Build fix 2023-06-09 13:34:27 +00:00
Antonio Andelic
07582d56f3 Add config for pinning api version 2023-06-09 13:18:13 +00:00
Antonio Andelic
ab8365630b Reject logs containing unknown operation 2023-06-09 10:10:40 +00:00
Antonio Andelic
f4d182ed67
Merge branch 'master' into keeper-operation-create-if-not-exists 2023-06-06 09:19:35 +02:00
Antonio Andelic
ece96f54e9 Fix tests 2023-06-05 09:47:58 +00:00
Antonio Andelic
c4873027c3 Merge branch 'master' into keeper-with-disks 2023-06-05 07:08:33 +00:00
Alexey Milovidov
bed19aa792
Merge pull request #50483 from ClickHouse/fix-keeper-snapshot-install 
Keeper fix: apply uncommitted state after snapshot install
 2023-06-05 04:39:33 +03:00
Antonio Andelic
47966c4e81 Finalize bufferg 2023-06-02 14:10:52 +00:00
Antonio Andelic
336c9d7136 Apply uncommitted state after snapshot deser 2023-06-02 13:31:14 +00:00
Antonio Andelic
ffd4f7f196 Add better support for state disks 2023-06-01 14:39:01 +00:00
Antonio Andelic
e8442b7a88 Add finalize calls 2023-06-01 12:54:00 +00:00
frinkr
c9d0d217f5 fix Keeper deadlock on exception when preprocessing requests 2023-06-01 09:57:42 +08:00
Konstantin Bogdanov
b61ffe3ff7
Merge checks 2023-06-01 03:22:40 +02:00
Konstantin Bogdanov
3a3539b996
Style fix 2023-06-01 03:22:40 +02:00
Konstantin Bogdanov
203276dfcc
Use storage.uncommited_state 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
965f7850f8
Cleanup 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
f84fdb7f10
Increment Keeper API version 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
b53f36369e
Remove new request object 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
0d6561ee77
Prototype 2023-06-01 03:22:23 +02:00
Antonio Andelic
21aba94909 Add support for latest snapshot disk 2023-05-30 13:22:40 +00:00
Antonio Andelic
67746bfff2 Merge branch 'master' into keeper-with-disks 2023-05-30 09:50:12 +00:00
Raúl Marín
1ae1f1273a Avoid clang 15 crash 2023-05-29 11:40:45 +02:00
Antonio Andelic
0740bfbe4b Better disks moving 2023-05-26 14:31:09 +00:00
Antonio Andelic
a9569478e6
fix typo 2023-05-26 13:09:09 +02:00
Antonio Andelic
8a2a63a7bd Merge branch 'master' into keeper-with-disks 2023-05-26 10:48:41 +00:00
Antonio Andelic
3d98e591ba Handle backwards compatibility 2023-05-26 10:46:18 +00:00
Antonio Andelic
767193caf5 Merge branch 'master' into keeper-log-improvements 2023-05-26 07:35:36 +00:00
Antonio Andelic
6a8a21e09a Add test for s3 disk restore 2023-05-25 16:01:40 +00:00
Antonio Andelic
bb77441acb Add support for old storage disks 2023-05-25 13:31:11 +00:00
Antonio Andelic
f519aa4613 Calculate size 2023-05-25 11:23:03 +00:00
Antonio Andelic
4483602c50 Remove double initialization 2023-05-25 08:52:14 +00:00
Antonio Andelic
7e68f61df1 Fix snapshot recovery 2023-05-25 08:43:11 +00:00
Antonio Andelic
092cf99147 Fix build 2023-05-24 13:45:14 +00:00
Antonio Andelic
fa3fb49d33 Merge branch 'master' into keeper-with-disks 2023-05-24 09:04:47 +00:00
Antonio Andelic
5db2160762 Fix build 2023-05-24 09:04:12 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
kssenii
295fe3b228 Merge remote-tracking branch 'upstream/master' into abstract-async-prefetched-buffer 2023-05-22 20:01:38 +02:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Antonio Andelic
bde2cf96b1 Better 2023-05-22 12:24:16 +00:00
alesapin
cc3897a84a
Merge pull request #50033 from kssenii/disk-object-storage-minor-changes 
Get rid of indirect write buffer in object storages
 2023-05-22 14:08:17 +02:00
Antonio Andelic
dc31396035 Merge branch 'master' into keeper-with-disks 2023-05-22 08:10:24 +00:00
kssenii
3e42ee7f2b Get rid of finalize callback in object storages 2023-05-19 17:29:37 +02:00
Antonio Andelic
9c3b17fa18
Remove whitespace 2023-05-19 13:00:51 +02:00
Antonio Andelic
e46476dba2
Update src/Coordination/Changelog.cpp 
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2023-05-19 12:44:20 +02:00
Antonio Andelic
6e468b29e8 Check return value of ftruncate 2023-05-19 10:15:06 +00:00
Antonio Andelic
bf6afd27a6 build with disks 2023-05-18 07:39:22 +00:00
Antonio Andelic
465123fa1b Merge branch 'master' into keeper-with-disks 2023-05-16 08:07:06 +00:00
Azat Khuzhin
665545ec45 Fix "reference to local binding" after fixes for clang-17 
Follow-up for: #49851 (cc @alexey-milovidov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-15 12:45:20 +02:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Antonio Andelic
bb8b9dc9c3 Merge branch 'master' into keeper-log-improvements 2023-05-05 11:57:14 +00:00
alesapin
9456eb59fd
Merge pull request #48466 from ClickHouse/keeper-correctly-handle-snapshots 
Correctly handle concurrent snapshots in Keeper
 2023-05-04 19:16:00 +02:00
johanngan
731823b873 Add support for SSE-KMS configuration with S3 
https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html

Similar to the server_side_encryption_customer_key_base64 option for
configuring SSE-C with S3, add the following settings to configure
SSE-KMS on a per-endpoint/disk basis:
  - server_side_encryption_kms_key_id
  - server_side_encryption_kms_encryption_context
  - server_side_encryption_kms_bucket_key_enabled
 2023-05-03 21:35:38 -05:00
Antonio Andelic
b8b7a362b2
Merge branch 'master' into keeper-correctly-handle-snapshots 2023-05-03 13:51:09 +02:00
Antonio Andelic
78ecfac8d5 Merge branch 'master' into keeper-log-improvements 2023-05-03 08:31:21 +00:00
Antonio Andelic
dce61d7321 Merge branch 'master' into keeper-log-improvements 2023-05-02 11:14:48 +00:00
Antonio Andelic
f695040a19
Merge pull request #48897 from ClickHouse/keeper-check-not-exists 
Add `CheckNotExists` request to Keeper
 2023-05-02 13:14:22 +02:00
Raúl Marín
f0e045bb3d Merge remote-tracking branch 'blessed/master' into arenita 2023-04-24 10:42:56 +02:00
Alexey Milovidov
530b764953 Fix IBM 2023-04-21 12:38:45 +02:00
Antonio Andelic
d500a5637d Cache large requests 2023-04-21 09:43:04 +00:00
Antonio Andelic
b52646c555 Eliminate double log serializing 2023-04-21 08:20:35 +00:00
Antonio Andelic
58e9b56fcb Fix CheckNotExists 2023-04-19 09:06:20 +00:00
Antonio Andelic
fb16623d48 Add CheckNotExists request to Keeper 2023-04-18 14:45:24 +00:00
Raúl Marín
2b70e08f23 Don't count unreserved bytes in Arenas as read_bytes 2023-04-13 12:43:24 +02:00
Antonio Andelic
b30aff06d8
Merge pull request #48595 from aalexfvk/alexfvk/fix_create_not_auth_problem 
Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes.
 2023-04-11 14:41:37 +02:00
Aleksei Filatov
dd2364361c
Update src/Coordination/tests/gtest_coordination.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-04-11 10:45:30 +03:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 17:59:30 +02:00
Aleksei Filatov
e8bf96d126 Add unit tests 2023-04-10 13:02:43 +03:00
Aleksei Filatov
000c2b5b91 Correct FixupACL for auth scheme 2023-04-10 13:02:31 +03:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
Antonio Andelic
88f77c686d Correctly handle concurrent snapshots 2023-04-06 15:40:05 +00:00
Antonio Andelic
e982f2a67a Merge branch 'master' into enable-env-credentials-default 2023-03-31 09:11:01 +00:00
Antonio Andelic
44e95aa65f Merge branch 'master' into nosign-keyword-for-s3 2023-03-29 11:10:03 +00:00
Antonio Andelic
160aa186bb Add support for NOSIGN keyword and no_sign_request config 2023-03-28 07:05:35 +00:00
Antonio Andelic
79e22d13fb small fix 2023-03-27 07:41:31 +00:00
Antonio Andelic
f355bc5342 Merge branch 'master' into more-batching-keeper 2023-03-27 07:28:17 +00:00
Antonio Andelic
6c3765c8b6 Small fix 2023-03-24 20:40:45 +00:00
Antonio Andelic
f353561204 Avoid breaking batches with read requests 2023-03-24 14:37:40 +00:00
Alexander Gololobov
afe3af230f Limit keeper request batching by size in bytes 2023-03-24 14:41:09 +01:00
Antonio Andelic
51416da818 Merge branch 'master' into enable-env-credentials-default 2023-03-13 10:20:21 +00:00
Antonio Andelic
a170a909a4 Add expiration window for S3 credentials 2023-03-10 10:06:32 +00:00
Antonio Andelic
83a3243758
Merge pull request #47065 from ClickHouse/debug-keeper 
Fix Keeper with `force_sync = false`
 2023-03-10 09:27:13 +01:00
Antonio Andelic
5bc21538e5 Enable use_environment_credentials by default 2023-03-09 10:31:55 +00:00
Antonio Andelic
0ad436aa2c Revert debug changes 2023-03-09 08:07:59 +00:00
Antonio Andelic
b4061d8aa4 Add logs 2023-03-08 13:06:10 +00:00
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
Antonio Andelic
dd4db9132a
Merge branch 'master' into debug-keeper 2023-03-07 13:15:05 +01:00
Antonio Andelic
c38d820d0f Add 4LW for cleaning resource 2023-03-05 16:57:18 +00:00
Antonio Andelic
cd4100cb19
Merge branch 'master' into fix-data-size 2023-03-03 15:33:43 +01:00
Antonio Andelic
fdcb5b784f
Merge branch 'master' into debug-keeper 2023-03-03 13:45:13 +01:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Antonio Andelic
6ebee202bd Flush buffer when no sync 2023-03-01 14:47:15 +00:00
Antonio Andelic
c797122dbe Better 2023-03-01 08:45:50 +00:00
Antonio Andelic
6e28406751 Merge branch 'master' into fix-data-size 2023-02-28 20:09:29 +00:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Alexander Tokmakov
e96340e183
Merge pull request #46410 from ClickHouse/fix_noisy_log_messages 
Fix noisy log messages
 2023-02-15 01:22:42 +03:00
Alexander Tokmakov
1bf3735ca9 fix noisy log messages 2023-02-14 15:33:27 +01:00
Sema Checherinda
9bef1bec28
Merge pull request #46274 from aiven-sal/aiven-sal/keeper 
coordination: do not allow election_timeout_lower_bound_ms > election_timeout_upper_bound_ms
 2023-02-14 12:44:04 +01:00
Sema Checherinda
5882f3878f
except zero values election_timeout 2023-02-13 12:22:08 +01:00
Salvatore Mesoraca
fff2c09b87
coordination: do not allow election_timeout_lower_bound_ms > election_timeout_upper_bound_ms 2023-02-13 10:09:45 +01:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias 2023-02-09 16:07:45 +01:00
Azat Khuzhin
8cc41b7f41 Check return value of ::close() 
Note, that according close(2), EINTR should not be retriable for close()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-07 11:28:22 +01:00
Antonio Andelic
b11458f142 Fix erase 2023-02-07 07:51:22 +00:00
Antonio Andelic
9cde46acff Correctly calculate size on update 2023-02-06 16:28:05 +00:00
Antonio Andelic
aca4f08bf5 Correctly calculate size after snapshot load 2023-02-06 14:59:23 +00:00
Antonio Andelic
8929838fcc Merge branch 'master' into add-support-for-settings-alias 2023-02-03 17:47:00 +00:00
Antonio Andelic
d5117f2aa6
Define S3 client with bucket and endpoint resolution (#45783) 
* Update aws

* Define S3 client with bucket and endpoint resolution

* Add defines for ErrorCodes

* Use S3Client everywhere

* Remove unused errorcode

* Add DROP S3 CLIENT CACHE query

* Add a comment

* Fix style

* Update aws

* Update reference files

* Add missing include

* Fix unit test

* Remove unneeded declarations

* Correctly use RetryStrategy

* Rename S3Client to Client

* Fix retry count

* fix clang-tidy warnings
 2023-02-03 14:30:52 +01:00
LiuYangkuan
f29700bd2f use IDisk to do IO in Keeper's snapshots and logs 2023-02-02 19:47:30 +08:00
Antonio Andelic
f613bfd8d2 Merge branch 'master' into add-support-for-settings-alias 2023-02-02 11:15:39 +00:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Antonio Andelic
88c1744975
Better 2023-02-01 14:25:58 +01:00
Antonio Andelic
5de6f86c27 Fix clang-tidy error 2023-02-01 12:28:44 +00:00
Antonio Andelic
ec04cf6876 Merge branch 'master' into add-support-for-settings-alias 2023-02-01 09:01:01 +00:00
Antonio Andelic
ece92fa866
Update src/Coordination/Changelog.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-01-31 17:11:18 +01:00
Antonio Andelic
8457d3bf3d Apply comments 2023-01-31 14:18:54 +00:00
Antonio Andelic
1786237be6 Merge branch 'master' into keeper-use-fallocate 2023-01-31 13:53:47 +00:00
Vitaly Baranov
f9282cfcac Make version_id an optional parameter again. 2023-01-27 15:10:19 +01:00
Vitaly Baranov
aea9ccdb60 Pass request settings to S3::getObjectInfo(). 2023-01-27 15:10:09 +01:00
Vitaly Baranov
a8304525ed Move getObjectInfo() to a separate header. 2023-01-27 15:09:38 +01:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
Antonio Andelic
ebfe7f9991 Fix style 2023-01-26 11:18:07 +00:00
Antonio Andelic
1186718658 Add overallocate size setting 2023-01-26 09:44:37 +00:00
Antonio Andelic
7bfdfa4bf7 Preallocate more space 2023-01-25 16:26:14 +00:00
Antonio Andelic
7421d8f872 Merge branch 'master' into keeper-use-fallocate 2023-01-25 11:36:47 +00:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Antonio Andelic
6a0e481ed7 Merge branch 'master' into keeper-use-fallocate 2023-01-12 13:57:05 +00:00
Antonio Andelic
30510c5eb6
Merge pull request #44937 from ClickHouse/fix-changelog-deadlock 
Fix deadlock in Keeper's changelog
 2023-01-10 08:59:51 +01:00
Antonio Andelic
22679ddf11
Add comment 2023-01-09 14:26:20 +01:00
Antonio Andelic
a39f437306
Typo fix 2023-01-09 13:34:48 +01:00
Antonio Andelic
27b2bae640 make it better 2023-01-09 12:01:22 +00:00
Antonio Andelic
4fcbe14435 Small fixes for Coordination unit tests 2023-01-08 13:44:32 +00:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
Antonio Andelic
9b3314a931 Fix deadlock in Changelog flush 2023-01-05 14:51:05 +00:00
Antonio Andelic
136436958b
Merge pull request #44908 from ClickHouse/fix-keeper-snapshot-on-exit 
Avoid Keeper crash on shutdown (fix `test_keeper_snapshot_on_exit`)
 2023-01-05 08:59:58 +01:00
Antonio Andelic
7c2cdfa65a Don't throw exception if queue is finished 2023-01-04 15:46:38 +00:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
Smita Kulkarni
8a5fd7f96d Added comments and addressed review comments - Allow macro substitution in endpoint 2023-01-03 11:44:26 +01:00
Antonio Andelic
f2653c016a Manually set deleted flag 2023-01-02 14:36:58 +00:00
Antonio Andelic
f32fab5025 Check if file exists before rename 2023-01-02 13:48:13 +00:00
Antonio Andelic
2e93640614 Fix ASAN error 2023-01-02 13:37:40 +00:00
Antonio Andelic
b845f880fb Polish 2023-01-02 10:42:15 +00:00
Antonio Andelic
14355d8009 Merge branch 'master' into keeper-use-fallocate 2023-01-02 08:22:27 +00:00
Antonio Andelic
55548a6678 style fix 2023-01-02 08:22:00 +00:00
SmitaRKulkarni
f9a3aeb172
Merge branch 'master' into 40951_Allow_macro_substitution_in_endpoint 2022-12-30 10:43:14 +01:00
Alexey Milovidov
f0fa0977ed Fix wrong log messages 2022-12-26 18:36:07 +01:00
Smita Kulkarni
3d62bc2635 Allow macro substitution in endpoint of disks 
Implementation:
* Updated to substitute macros in endpoint for all disks (s3, hfs, web).
Testing:
* Added an integration test test/integration/test_endpoint_macro_substitution.
 2022-12-23 15:09:24 +01:00
Antonio Andelic
64ade259cf Fix tests 2022-12-22 10:20:55 +00:00
Antonio Andelic
88d41dea93 Merge branch 'master' into keeper-use-fallocate 2022-12-21 08:59:09 +00:00
Antonio Andelic
9711355cc9 Fix tests 2022-12-21 08:43:55 +00:00
kssenii
6bd4f8c029 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-20 21:17:28 +01:00
Antonio Andelic
6d83221d4e Remove unused error code 2022-12-19 12:20:26 +00:00
Antonio Andelic
c2c115d2ba Add setting for max_log_file_size 2022-12-19 11:51:05 +00:00
Antonio Andelic
f241701c2a Add checks when fallocate is not supported 2022-12-19 11:48:54 +00:00
Antonio Andelic
9a8b248735 Merge branch 'master' into keeper-use-fallocate 2022-12-19 09:32:05 +00:00
Nikolai Kochetov
62ff98344e Validate s3 part upload settings. 2022-12-17 14:09:53 +00:00