thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 09:22:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
131,930 Commits 9,297 Branches 1,782 Tags 2.1 GiB
d5f86f671d
Commit Graph

3259 Commits

Author SHA1 Message Date
Sema Checherinda
4a1e207e7a review notes 2023-11-16 12:31:00 +01:00
Jianfei Hu
69f214cdbc fix comments. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-16 08:04:57 +00:00
Jianfei Hu
ea92dbb1c7 fix build for non USE_S3 case 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-15 20:53:35 +00:00
Jianfei Hu
d0398e3c1d remove variant header 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-15 18:47:28 +00:00
Jianfei Hu
d862dfdf9c fix comments 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-15 18:38:23 +00:00
Sema Checherinda
6e3e6383ba perf check 2 2023-11-15 19:00:27 +01:00
avogar
38f200d969 Fix Date text parsing in optimistic path 
1
 2023-11-14 18:58:00 +00:00
Jianfei Hu
9df2775f08 reduce timeout and setTimeout earlier. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-14 17:58:16 +00:00
Sema Checherinda
3075bd9745 track clickhouse high level retries 2023-11-14 11:34:12 +01:00
Sema Checherinda
8d36fd6e54 get rid off of client_with_long_timeout_ptr 2023-11-14 11:34:12 +01:00
Sema Checherinda
27fb25d056 alter the naming, fix client_with_long_timeout in s3 storage 2023-11-14 11:34:12 +01:00
Sema Checherinda
be01a5cd3e turn off agressive timeouts for heavy requests 2023-11-14 11:34:12 +01:00
Sema Checherinda
770a762317 aggressive timeout 2023-11-14 11:34:11 +01:00
Jianfei Hu
554d907189 Fix the keeper_server availability zone configuration. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-13 23:42:51 +00:00
Jianfei Hu
d3517140d9 Revert "Revert "Add /keeper/availability-zone node to allow server load balancing within AZ."" 
This reverts commit e5aad4a6fd.
 2023-11-13 19:03:27 +00:00
Alexey Milovidov
c81cc3f953
Merge pull request #56660 from ClickHouse/better-ccache-2 
Better use of build cache
 2023-11-13 19:28:33 +01:00
Alexey Milovidov
82c461031e Fix build 2023-11-13 10:09:23 +01:00
Alexey Milovidov
58319f08be Merge branch 'master' into system-symbols 2023-11-11 04:12:37 +01:00
Alexey Milovidov
e5aad4a6fd
Revert "Add /keeper/availability-zone node to allow server load balancing within AZ." 2023-11-11 05:43:58 +03:00
Alexey Milovidov
cc39484f7f Merge branch 'master' into system-symbols 2023-11-09 23:05:55 +01:00
Sema Checherinda
7f34e0b24e recheck 404 response for complete multipart upload 2023-11-08 20:51:49 +01:00
Alexey Milovidov
68d3f60ef7 Merge branch 'master' into system-symbols 2023-11-08 17:07:11 +01:00
kssenii
dbea50738b Merge remote-tracking branch 'origin/master' into fs-cache-improvement 2023-11-08 11:53:54 +01:00
Antonio Andelic
788c66027c
Merge pull request #56104 from incfly/keeper-az 
Add /keeper/availability-zone node to allow server load balancing within AZ.
 2023-11-08 09:46:24 +01:00
Nikita Taranov
c52571223e better 2023-11-07 22:04:13 +01:00
Alexey Milovidov
ca83da14f2 Merge branch 'master' of github.com:ClickHouse/ClickHouse into system-symbols 2023-11-07 19:43:19 +01:00
Sema Checherinda
02df5ac59c
Merge pull request #54340 from KevinyhZou/bug_fix_csv_parse_crlf 
Fix CSV read while the end of line is CR(\r)
 2023-11-07 12:51:05 +01:00
Jianfei Hu
033bb19198 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into keeper-az 2023-11-07 02:50:54 +00:00
kevinyhzou
d1743e08f7 ci fix 2023-11-07 09:45:46 +08:00
Alexander Tokmakov
fbdfa9a44d
Merge pull request #56316 from ClickHouse/change_some_exception_codes 
Change some exception codes
 2023-11-06 19:17:02 +01:00
Raúl Marín
653a51a862
Merge pull request #56286 from bigo-sg/ch_56285 
Fix inconsistency of "cast('0' as DateTime64(3))" and "cast('0' as Nullable(DateTime64(3)))"
 2023-11-06 10:30:51 +01:00
Alexey Milovidov
184e6f840e Remove useless header files 2023-11-06 03:49:55 +01:00
Arthur Passos
b6e205dcdf
Add ClickHouse setting to disable tunneling for HTTPS requests over HTTP proxy (#55033) 
* initial commit. integ tests passing, need to re-run unit & my own personal tests

* partial refactoring to remove Protocol::ANY

* improve naming

* remove all usages of ProxyConfiguration::Protocol::ANY

* fix ut

* blabla

* support url functions as well

* support for HTTPS requests over HTTP proxy with tunneling off

* remove gtestabc

* fix silly mistake

* ...

* remove usages of httpclientsession::proxyconfig in src/

* got you

* remove stale comment

* it seems like I need reasonable defaults

* fix ut

* add some comments

* remove no longer needed header

* matrix out

* add https over http proxy with no tunneling

* soem docs

* partial refactoring

* rename to use_tunneling_for_https_requests_over_http_proxy

* improve docs

* use shorter version

* remove useless test

* rename the setting

* update

* fix typo

* fix setting docs typo

* move ); up

* move ) up
 2023-11-04 13:47:52 -04:00
Yakov Olkhovskiy
d9fe33ae7b fix clang tidy 2023-11-04 03:06:45 +00:00
Nikita Taranov
7046832056 change signatures 2023-11-03 22:50:03 +01:00
Alexander Tokmakov
289fcccbfc change some exception codes 2023-11-03 20:19:24 +01:00
Jianfei Hu
b426cfe462 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into keeper-az 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-03 16:42:29 +00:00
Jianfei Hu
bdf49a69f4 throw exception 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-03 16:35:06 +00:00
taiyang-li
3c70b9f65d add uts 2023-11-03 15:32:37 +08:00
taiyang-li
93be383117 fix issue https://github.com/ClickHouse/ClickHouse/issues/56285 2023-11-03 12:41:00 +08:00
Jianfei Hu
136efba321 integration test working and std::exception_ptr. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-03 02:24:42 +00:00
Jianfei Hu
e29b7640c4 gcp metadata use dns. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-01 17:53:34 +00:00
Jianfei Hu
b22773ccd1
Apply suggestions from code review 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2023-11-01 10:30:29 -07:00
Kruglov Pavel
08971270aa
Merge pull request #56135 from Avogar/fix-json-exception-in-http 
Fix wrong query result when http_write_exception_in_output_format=1
 2023-11-01 14:36:28 +01:00
Kseniia Sumarokova
e5efc42633
Merge branch 'master' into fs-cache-improvement 2023-11-01 10:47:36 +01:00
Jianfei Hu
6f94770a8e more fix -DUSE_AWS_S3=0 case. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-01 00:51:45 +00:00
Nikita Taranov
1d58cedce5 better 2023-10-31 22:48:20 +01:00
Jianfei Hu
045d67ffc1 more style fix. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>

space before catch.

Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 21:41:39 +00:00
Jianfei Hu
3fff813659 style fix and revert src/Coordination/KeeperSnapshotManagerS3.h 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 18:48:59 +00:00
Jianfei Hu
f902243c05 comments use static. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 17:43:40 +00:00
Jianfei Hu
cfa1b96d23 remove comments. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 17:43:24 +00:00
Jianfei Hu
e231d063fb fix the exception handlig. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 05:16:14 +00:00
Jianfei Hu
573054d755 add new code and rework. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 04:43:44 +00:00
Jianfei Hu
6d72873bdb use poco better works. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 02:34:50 +00:00
Yakov Olkhovskiy
9f4be7e847 fix 2023-10-30 23:31:15 +00:00
Kruglov Pavel
4effc676f9
Merge pull request #56046 from Avogar/cr-in-unquoted-csv-string 
Allow unquoted strings with CR in CSV format
 2023-10-30 20:46:20 +01:00
avogar
c7857646bc Fix wrong query result when http_write_exception_in_output_format=1 2023-10-30 18:09:29 +00:00
Yakov Olkhovskiy
e1d0994c0a fix 2023-10-30 05:37:47 +00:00
Yakov Olkhovskiy
d72069dea4 compressors to not produce output for no data, HTTP buffer not to add Content-Encoding if no data 2023-10-29 18:21:21 +00:00
Jianfei Hu
dec8efc39e make az as attribute of keepersnapshotmangers3. 
empty config not working.
https://pastila.nl/?00048e4f/d56af90d13f2d871e30dfb1db9660266#yff6d8HVQm0blQUUiZg8SQ==

Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-29 02:04:38 +00:00
Yakov Olkhovskiy
7887e1e14e fix 2023-10-28 23:26:38 +00:00
Jianfei Hu
0b301f73c1 initial add keeper az info in /keeper/availabilty-zone 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-28 21:54:59 +00:00
Nikita Taranov
213ce63ffb
Merge branch 'master' into optimize_reading3 2023-10-28 00:04:20 +02:00
Kruglov Pavel
f1f1846572
Merge pull request #55861 from bigo-sg/ch_55858 
Throw exception when parsing illegal string as float if precise_float_parsing is true
 2023-10-27 14:24:55 +02:00
李扬
308b2942d7
Update readFloatText.h 2023-10-27 11:42:33 +08:00
Yakov Olkhovskiy
0cf851316c use CH Buffer for HTTP out stream, add metrics for interfaces 2023-10-27 02:38:36 +00:00
avogar
9d207bf027 Allow unquoted strings with CR in CSV format 2023-10-26 13:50:54 +00:00
taiyang-li
d5feaa72c1 improve thrown exception message 2023-10-25 20:16:18 +08:00
Nikita Taranov
a86bda5208 use local thread pool reader for cache segments 2023-10-24 14:06:13 +02:00
taiyang-li
2a894335b7 commit again 2023-10-24 16:16:05 +08:00
zvonand
7d54c1fb00 check EOF before reading buffer 2023-10-20 22:27:02 +02:00
zvonand
e5bfc7d3fc Improved DateTime64 parsing 
Allow to parse unambigous short DateTime64
Throw error on invalid DT64 representation (as for DateTime)
 2023-10-20 22:27:02 +02:00
Nikita Taranov
9b13c796e3 set ignore metric in all readers 2023-10-20 22:24:56 +02:00
Nikita Taranov
2a6a97c1fa read sync from cache 2023-10-20 22:24:56 +02:00
taiyang-li
6e3f35fc94 fix issue https://github.com/ClickHouse/ClickHouse/issues/55858 2023-10-20 16:18:39 +08:00
Aleksei Filatov
f56804e6e7 Use pool for proxified S3 http sessions 2023-10-17 12:15:43 +03:00
Michael Kolupaev
ce7eca0615
DWARF input format (#55450) 
* Add ReadBufferFromFileBase::isRegularLocalFile()

* DWARF input format

* Review comments

* Changed things around ENABLE_EMBEDDED_COMPILER build setting

* Added 'ranges' column

* no-msan no-ubsan
 2023-10-16 17:00:07 -07:00
kssenii
43c5e173de Merge remote-tracking branch 'upstream/master' into fs-cache-improvement 2023-10-16 13:06:28 +02:00
alesapin
3b02748cb6 Fix some typos 2023-10-15 15:43:02 +02:00
Alexey Milovidov
9892990f8e
Merge pull request #55262 from azat/compressed-streams-memory-leak 
Avoid possible memory leaks in compressors in case of missing buffer finalization
 2023-10-14 00:22:42 +02:00
Alexey Milovidov
5ad055df98
Merge pull request #55372 from azat/web-disk-fix-race 
Fix data-race in web disk
 2023-10-11 01:03:48 +02:00
Azat Khuzhin
59569b62b9 Add a comment about thread-safety of getFileOffsetOfBufferEnd() 2023-10-10 21:12:31 +02:00
Alexey Milovidov
972f3678fa Review #51946 and partially revert it 2023-10-08 18:00:59 +02:00
Alexey Milovidov
c7ae1afb8f Review #51946 and partially revert it 2023-10-08 15:17:41 +02:00
Azat Khuzhin
28f17bec96 Avoid possible memory leaks in case of missing buffer finalization 
Sometimes it is still possible, the pattern could looks like this:

    try
    {
        writer->finalize();
        writer->flush();
        write_buf->finalize();
    }
    catch (...)
    {
        /// Stop ParallelFormattingOutputFormat correctly.
        release();
        throw;
    }

Here, write_buf will not be finalized, in case of exception during
writer finalize()/flush().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 08:51:45 +02:00
kssenii
a72541d93f Improvement for big reads 2023-09-29 19:03:00 +02:00
Alexey Milovidov
8f9a227de1
Merge pull request #54931 from yariks5s/s3_style_url 
s3-style URL fix
 2023-09-29 07:12:01 +03:00
avogar
e8ea41e2d4 Add setting to print pretty deep nested Tuples 2023-09-28 22:17:44 +00:00
yariks5s
61e7054f1f fixed map issue 2023-09-28 17:16:37 +00:00
yariks5s
e4e23d73ea fixed tests + added map as a parameter 2023-09-28 17:04:01 +00:00
Sema Checherinda
f73eef9ed8
Merge pull request #54697 from CheSema/s3_retry_cancelation_point 
add cancelation point to s3 retries
 2023-09-28 19:01:09 +02:00
yariks5s
ad94e08b0b made using macroses 2023-09-28 15:17:15 +00:00
Nikolay Degterinsky
9ebecb5499
Merge pull request #54639 from evillique/timestamp-function 
Add `timestamp` function for compatibility with MySQL
 2023-09-28 16:44:16 +02:00
Sema Checherinda
75e5c80d58 fix typo 2023-09-28 13:37:51 +02:00
Sema Checherinda
29f9673a5f
Update src/IO/S3/Client.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2023-09-28 12:41:16 +02:00
yariks5s
1dd305107d added default values 2023-09-28 09:55:42 +00:00
yariks5s
d7a2d2fa5b test fix 2023-09-27 23:50:53 +00:00
yariks5s
b3e965b724 small fix 2023-09-27 22:41:35 +00:00
yariks5s
583d4ea8b7 removed chassert 2023-09-27 22:41:02 +00:00
Sema Checherinda
31ea4052d0 cut the build dependency between commin_io and other libraries 2023-09-28 00:15:36 +02:00
Sema Checherinda
5fdca302e6 fix build, resolve process_list_elem race 2023-09-28 00:15:36 +02:00
Sema Checherinda
b9a2c82910 do not share http_client, make code simple 2023-09-28 00:15:26 +02:00
Sema Checherinda
e629a17218 add cancelation point to s3 retries 2023-09-28 00:15:06 +02:00
Robert Schulze
675819be6b
Minor fixes 2023-09-27 21:15:20 +00:00
Nikolay Degterinsky
c1f59eccd5 Add fraction part to the time argument, add docs, better test 2023-09-27 20:17:32 +00:00
Yarik Briukhovetskyi
9c337c7687
Update URI.cpp 2023-09-27 20:55:29 +02:00
yariks5s
adf1af63a5 added tests, moved domains to config 2023-09-27 18:53:05 +00:00
robot-clickhouse-ci-1
7a825c1417
Merge pull request #54427 from Avogar/json-object-as-tuple-inference 
Add new features to schema inference for JSON formats
 2023-09-27 20:12:45 +02:00
Robert Schulze
cde10fe7b5
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks 2023-09-26 18:59:41 +00:00
avogar
6b3dbc4403 Apply suggestions 2023-09-26 16:41:35 +00:00
Vitaly Baranov
fe008c23c4
Merge pull request #54900 from vitlibar/retry-backup-s3-operations-after-conection-reset 
Retry backup S3 operations after connection reset failure
 2023-09-26 18:36:10 +02:00
Antonio Andelic
535b9337f4
Merge pull request #55012 from ClickHouse/lower-s3-credential-logs 
Lower log levels for `SSOCredentialsProvider`
 2023-09-26 16:08:00 +02:00
yariks5s
4939e59cfe Merge branch 'master' into s3_style_url 2023-09-26 13:24:00 +00:00
Kruglov Pavel
bea80ab5b7
Merge branch 'master' into json-object-as-tuple-inference 2023-09-26 15:23:08 +02:00
Kruglov Pavel
69a17bbef6
Merge pull request #52853 from Avogar/http-valid-json-on-exception 
Output valid JSON/XML on excetpion during HTTP query execution
 2023-09-26 14:25:55 +02:00
Robert Schulze
9fff447716
Re-enable clang-tidy checks 2023-09-26 09:34:12 +00:00
Antonio Andelic
ef6c7fe45d Lower log levels for SSOCredentialsProvider 2023-09-26 08:30:34 +00:00
avogar
9e75825515 Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-object-as-tuple-inference 2023-09-25 17:24:36 +00:00
avogar
42ca897f2d Better schema inference for JSON formats 2023-09-25 15:42:59 +00:00
serxa
44acb53ad3 Merge branch 'master' into io-scheduling-throttler 2023-09-23 10:51:18 +00:00
Vitaly Baranov
1e567d5008 Retry backup s3 operations after ConnectionResetException. 2023-09-23 01:56:28 +02:00
yariks5s
e1f40adfe2 fix typo 2023-09-22 16:21:08 +00:00
yariks5s
dce8908641 fixed url convers. 2023-09-22 16:13:29 +00:00
Robert Schulze
877e4f3aab
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 20:21:12 +00:00
Michael Kolupaev
9af9b4a085
Enable connection pooling for s3 table function (#54812) 
Enable connection pooling for s3 table function
 2023-09-21 09:27:20 -07:00
Robert Schulze
5209bd2d51
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 14:45:55 +00:00
Robert Schulze
f5137dd0b4
More clang-tidy fixes 2023-09-21 14:40:57 +00:00
Vitaly Baranov
bf4662aca7
Merge pull request #53336 from azat/backups/s3-throttling 
Fix throttling of BACKUPs from/to S3 (in case native copy was not used) and in some other places as well
 2023-09-21 10:42:28 +02:00
Azat Khuzhin
c706101891 Fix throttling of BACKUPs from/to S3 (in case native copy was not used) 
In some cases native copy is not possible, and such requests should be
throttled.

v0: copyS3FileNativeWithFallback
v2: revert v0 and pass write_settings
v3: pass read_settings to copyFile()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-20 18:28:43 +02:00
Duc Canh Le
2a5b4ee29a Merge branch 'master' into fix_http_timeout 
To trigger CI pipeline
 2023-09-20 07:22:02 +00:00
Kruglov Pavel
e163670357
Merge branch 'master' into http-valid-json-on-exception 2023-09-19 13:42:53 +02:00
serxa
14569b0f7b fix start time in test 2023-09-19 10:13:05 +00:00
serxa
b154f71a57 Merge branch 'io-scheduling-throttler' of github.com:ClickHouse/ClickHouse into io-scheduling-throttler 2023-09-19 10:10:35 +00:00
serxa
311db94640 default max_burst equals to max_speed as in Throttler 2023-09-19 10:10:21 +00:00
serxa
d97b4f0685 better semantics for locking 2023-09-19 10:09:16 +00:00
serxa
71655cda8e add more tests 2023-09-19 10:08:09 +00:00
Sergei Trifonov
b2ed38742c
Update src/IO/ISchedulerNode.h 
Co-authored-by: vdimir <vdimir@clickhouse.com>
 2023-09-19 11:32:05 +02:00
serxa
c5f3170a82 update actual amount of tokens in bucket 2023-09-18 18:12:55 +00:00
Sema Checherinda
e7550523c8
Merge pull request #54651 from CheSema/limit_backoff_timeout 
limit the delay before next try in S3
 2023-09-18 19:41:26 +02:00
serxa
1787803157 fix scheduler node registration 2023-09-18 17:25:16 +00:00
serxa
c3f8ee589e fix uninitialized value 2023-09-18 17:24:58 +00:00
serxa
e94908ce2e add unit tests + fix bugs 2023-09-18 17:14:06 +00:00
Robert Schulze
f5e8028bb1
Merge pull request #54642 from rschu1ze/broken-re2st 
Remove broken lockless variant of re2
 2023-09-17 15:30:57 +02:00
Sema Checherinda
7ce3a587ca fix build 2023-09-15 20:52:21 +02:00
Kruglov Pavel
dbd24b240c
Merge branch 'master' into http-valid-json-on-exception 2023-09-15 14:55:31 +02:00
Sema Checherinda
0edfa22181 attempts count is retries plus 1 2023-09-15 13:13:14 +02:00
Sema Checherinda
d9e15c00c9 limit the delay before next try in S3 2023-09-14 19:45:07 +02:00
Robert Schulze
7b378dbad3
Remove broken lockless variant of re2 2023-09-14 16:40:42 +00:00
serxa
8b00e29ce9 fix style 2023-09-14 10:40:41 +00:00
serxa
2ab922777a fix typos 2023-09-14 09:21:23 +00:00