Sema Checherinda
ff01ca80b4
reduce retires time for queries, increase retries count for backups
2024-05-15 16:37:42 +02:00
Han Fei
171bf4f201
Merge branch 'master' into hanfei/fix-intersection
2024-05-15 13:59:52 +02:00
kssenii
5ede316ec5
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-05-15 12:52:25 +02:00
Antonio Andelic
9744feb95e
Fix deadlock in parallel read buffer
2024-05-15 11:02:10 +02:00
Yakov Olkhovskiy
88a833335f
fix
2024-05-14 20:25:26 +00:00
Yakov Olkhovskiy
dfdf31f1b6
host-net conversion
2024-05-14 18:09:11 +00:00
Yakov Olkhovskiy
daf8277e55
fix
2024-05-14 16:00:58 +00:00
Yakov Olkhovskiy
05e823a1e9
add chunked wrapper to native protocol
2024-05-14 15:37:20 +00:00
avogar
a7e87e22ad
Merge branch 'master' of github.com:ClickHouse/ClickHouse into 56257_parse_crlf_with_TSV_files
2024-05-14 11:56:43 +00:00
kssenii
9ce465bb88
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-05-13 16:14:58 +02:00
Antonio Andelic
449aacb3d0
Correct
2024-05-13 15:08:52 +02:00
Antonio Andelic
2b38b8a97b
Correct byte count
2024-05-13 14:49:26 +02:00
Anton Popov
47d0eadb6f
Merge remote-tracking branch 'upstream/master' into HEAD
2024-05-13 12:12:31 +00:00
divanik
faef8807d2
Resolve conflicts
2024-05-13 12:04:06 +00:00
divanik
d468a0a577
Add archives reading support to s3
2024-05-13 12:00:15 +00:00
Antonio Andelic
e65dd3d939
Cancel S3 reads properly when parallel reads are used
2024-05-13 12:36:29 +02:00
Alexey Milovidov
23cf6d0d8b
Useless changes
2024-05-11 22:51:57 +02:00
Anton Popov
9ed3acce82
refactoring near azure blob storage
2024-05-10 18:22:23 +00:00
Alexey Milovidov
224e1940ae
Useless changes
2024-05-09 03:58:34 +02:00
Alexey Milovidov
0d14a2c67e
Useless changes
2024-05-09 03:11:02 +02:00
Alexey Milovidov
1d69518c4d
Useless changes
2024-05-09 02:07:04 +02:00
Alexey Milovidov
95f12ef274
Useless changes
2024-05-09 01:08:33 +02:00
Han Fei
116055b532
refine load part logic
2024-05-08 15:04:16 +02:00
Han Fei
746e96eddd
Merge branch 'master' into hanfei/fix-intersection
2024-05-07 23:54:54 +02:00
Han Fei
13b312092c
rewrite load part logic
2024-05-07 23:54:49 +02:00
kssenii
cd093ca793
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-05-07 12:50:48 +02:00
Shaun Struwig
3316df88aa
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-05-05 05:30:40 +02:00
Shaun Struwig
2b4def9ebb
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-05-05 05:30:26 +02:00
kssenii
e9d9f2bc25
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-05-03 13:55:12 +02:00
Sema Checherinda
083bcfc1ce
distinct message for s3 error 'no key' for cases disk and storage
2024-05-02 13:08:25 +02:00
Shaun Struwig
40f33b6236
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-04-29 10:25:47 +02:00
kssenii
7a416f2e76
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-28 12:19:32 +02:00
Azat Khuzhin
d278866d52
Add copy_data_failpoint failpoint
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-04-28 07:34:07 +02:00
Shaun Struwig
a69658b1dd
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-04-27 21:40:58 +02:00
kssenii
91014456b5
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-26 19:24:07 +02:00
kssenii
18e4c0f1da
Fix remaining integration test
2024-04-26 13:35:18 +02:00
Kruglov Pavel
52e3c3aa4e
Merge branch 'master' into 56257_parse_crlf_with_TSV_files
2024-04-24 16:20:19 +01:00
Smita Kulkarni
065eceecbf
Merge branch 'master' into Azure_write_buffer_parallel_upload
2024-04-19 17:37:15 +02:00
Robert Schulze
7dc05d092b
Merge remote-tracking branch 'rschu1ze/master' into be-less-boring-32
2024-04-15 16:35:10 +00:00
kssenii
27672e1dfc
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-15 11:28:14 +01:00
Alexander Tokmakov
d510c3ac55
Merge pull request #62266 from ClickHouse/cannot_allocate_thread
...
Add fault injection for "Cannot allocate thread"
2024-04-12 14:40:42 +00:00
Robert Schulze
dbc04c84dc
Merge pull request #62177 from rschu1ze/double-conversion
...
Bump `double-conversion` submodule
2024-04-12 13:28:33 +00:00
Smita Kulkarni
b8c77a4453
Support parallel write buffer for AzureBlobStorage
2024-04-11 10:13:41 +02:00
Alexander Tokmakov
87476b8b98
Merge branch 'master' into tavplubix-patch-10
2024-04-10 23:37:22 +02:00
Alexander Tokmakov
d8e97b51bf
Merge branch 'master' into cannot_allocate_thread
2024-04-10 21:21:42 +02:00
Anton Popov
4a8c1bf6a3
Merge pull request #62306 from CurtizJ/azure-retryable-errors
...
Better handling of errors from azure storage
2024-04-10 13:38:33 +00:00
Nikita Taranov
a2147b8ded
Merge branch 'master' into keep_alive_max_reqs
2024-04-09 21:32:36 +02:00
Shaun Struwig
d5fe715b3b
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-04-09 18:11:06 +02:00
Robert Schulze
22e164c65d
Merge pull request #62375 from rschu1ze/static-instance2
...
Don't access static members through instance, pt. II
2024-04-09 14:37:24 +00:00
Nikita Taranov
d594cb2e41
Merge pull request #62208 from arthurpassos/s3_aws_private_link_style
...
Add support for S3 access through aws private link interface
2024-04-09 12:32:38 +00:00
Robert Schulze
4035334634
Merge remote-tracking branch 'rschu1ze/master' into static-instance2
2024-04-09 11:45:27 +00:00
Sema Checherinda
04ea8a85f2
Merge pull request #62249 from ClickHouse/chesema-keep-alive-send-and-receive
...
several fixes for client's keep alive connections
2024-04-09 10:27:28 +00:00
Alexander Gololobov
b3dd5f519d
Merge pull request #62348 from ClickHouse/http_log_levels
...
Reduce log levels for ReadWriteBufferFromHTTP retries
2024-04-09 07:02:31 +00:00
Shaun Struwig
971263c4e6
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-04-08 21:52:47 +02:00
Shaun Struwig
dde99cbe8c
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-04-08 21:51:57 +02:00
Robert Schulze
2418d673f1
Suppress false positives in leaksan
2024-04-08 11:03:43 +00:00
Robert Schulze
b8b74ff815
Suppress msan false positives
2024-04-08 11:03:35 +00:00
Robert Schulze
97281203b8
Don't access static members through instance, pt. II
...
clang-tidy rightfully complains
(-readability-static-accessed-through-instance) that static member
functions are accessed through non-static members
2024-04-07 11:09:35 +00:00
Robert Schulze
cd5a06892e
Merge remote-tracking branch 'rschu1ze/master' into double-conversion
2024-04-07 10:27:55 +00:00
Anton Popov
fa01f26def
add test
2024-04-05 17:39:12 +00:00
Arthur Passos
14eb498477
Merge branch 'ClickHouse:master' into s3_aws_private_link_style
2024-04-05 12:34:23 -03:00
Arthur Passos
7d50bb8c4b
fix ut once again
2024-04-05 12:33:43 -03:00
Alexander Gololobov
b2bcfaf344
Reduce log levels for ReadWriteBufferFromHTTP retries
2024-04-05 17:18:22 +02:00
Raúl Marín
7df11b1852
Merge pull request #62264 from Algunenano/nested_includes_Ñ
...
Remove more nested includes
2024-04-05 17:00:28 +02:00
kssenii
357e5e3ae6
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-05 16:15:56 +02:00
Sema Checherinda
dddb0d9f4a
fix http_keep_alive_max_requests set up
2024-04-05 03:02:45 +02:00
Sema Checherinda
8b75b6ae77
Merge branch 'master' into chesema-keep-alive-send-and-receive
2024-04-05 01:04:22 +02:00
Sema Checherinda
ae3a199939
support max requests for keep alive
2024-04-05 00:37:29 +02:00
Sema Checherinda
dd852da339
add more debug logs
2024-04-05 00:30:03 +02:00
Alexander Tokmakov
5db9fbed52
cancel tasks on exception
2024-04-04 22:32:57 +02:00
Alexander Tokmakov
724f9b8a3e
Update ReadBufferFromPocoSocket.cpp
2024-04-04 20:14:33 +02:00
Robert Schulze
7ce2a47ba5
Merge pull request #62265 from rschu1ze/static-instance
...
Don't access static members through instance
2024-04-04 14:36:03 +02:00
Yarik Briukhovetskyi
edacb991e0
Merge pull request #62257 from yariks5s/fix_retention_docs
...
Fix s3-style link mapper for gcs
2024-04-04 14:30:33 +02:00
Shaun Struwig
05b2cfb563
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-04-04 14:26:42 +02:00
Arthur Passos
b7c0501ac2
fix ut
2024-04-04 09:26:37 -03:00
kssenii
be30806dbe
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-04 13:20:12 +02:00
Arthur Passos
c6aed8b793
add optional port to regex
2024-04-03 19:19:54 -03:00
Raúl Marín
2c2f2b9526
Missing include
2024-04-03 22:59:59 +02:00
Robert Schulze
de2a0be025
Don't access static members through instance
...
- clang-tidy rightfully complains (-readability-static-accessed-through-instance)
- not going to enable the warning for now to avoid breaking the build
2024-04-03 18:50:33 +00:00
Arthur Passos
be55c3533c
default to path style
2024-04-03 15:48:30 -03:00
Raúl Marín
76b6c125ff
Remove boost dep in wide integers when possible
2024-04-03 20:36:29 +02:00
yariks5s
ca27cf3fde
fix mapper for gcs
2024-04-03 16:38:41 +00:00
Kruglov Pavel
05db73f518
Merge branch 'master' into 56257_parse_crlf_with_TSV_files
2024-04-03 17:17:44 +02:00
Arthur Passos
394a4f39ec
remove unnecessary matching character
2024-04-03 10:37:39 -03:00
Sema Checherinda
8e6cbc8b31
several fixes for client's keep alive connections
2024-04-03 15:13:59 +02:00
Arthur Passos
f6a240b7be
add some unit tests
2024-04-03 10:11:56 -03:00
Arthur Passos
15dd5ce5f6
use raw literal string
2024-04-03 08:39:16 -03:00
Sema Checherinda
19e85c58cf
Merge pull request #62068 from ClickHouse/chesema-fix-url-storage
...
ReadWriteBufferFromHTTP set right header host when redirected
2024-04-03 12:52:49 +02:00
Arthur Passos
c1fd9262a7
change exception placeholder
2024-04-02 18:42:57 -03:00
Arthur Passos
0ad3ed5ae3
Add support for S3 access through aws private link interface
2024-04-02 18:38:58 -03:00
Robert Schulze
7d87adc91a
Upgrade double-conversion to 3.3.0
2024-04-02 13:51:36 +00:00
Raúl Marín
c35a436435
Remove nested dependency on DateLutImpl
2024-04-02 14:45:48 +02:00
Raúl Marín
40a22ffa63
Remove a few nested include dependencies
2024-04-02 13:43:14 +02:00
Sema Checherinda
9a9f0161bb
add test
2024-04-01 01:33:40 +02:00
Sema Checherinda
1ea058a702
ReadWriteBufferFromHTTP set right header host when redireced
2024-03-28 19:33:23 +01:00
Kruglov Pavel
9b5b44dd5f
Merge pull request #61889 from Avogar/allow-to-save-bad-json-escape-sequences
...
Add a setting to allow saving bad escape sequences in JSON input formats
2024-03-28 14:34:02 +01:00
kssenii
ca1c119115
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-27 22:32:44 +01:00
kssenii
e0c02bf3af
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-27 19:27:07 +01:00
Shaun Struwig
0e76731c6a
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-03-27 03:06:51 +01:00
Michael Kolupaev
899b856ad6
Add profile events for number of HTTP requests and bytes read
2024-03-26 09:14:55 +00:00
Michael Kolupaev
1fb902c5b7
Fix sending a new request every 2 MB
2024-03-26 09:14:55 +00:00
avogar
dc87c483dd
Add a setting to allow saving bad escape sequences in JSON input formats
2024-03-25 21:58:53 +00:00
Alexey Milovidov
21a0d17637
Merge branch 'master' into geodistance-double-precision
2024-03-25 17:47:42 +03:00
kssenii
46941e7243
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-25 15:10:11 +01:00
Alexey Milovidov
65650809b4
Double precision of geoDistance if the arguments are Float64
2024-03-24 23:11:35 +01:00
Smita Kulkarni
0d54dbf460
Merge branch 'master' into Azure_write_buffer
2024-03-24 20:09:47 +01:00
Alexey Milovidov
8ddfe9a48b
Merge pull request #61665 from ClickHouse/build-improvements-magic-enum-template-instantiation
...
Build improvements by removing magic enum from header and apply some explicit template instantiation
2024-03-24 19:31:46 +03:00
alesapin
a41c005a6e
Merge branch 'master' into Azure_write_buffer
2024-03-24 15:04:18 +01:00
Alexey Milovidov
056c8ce021
Merge pull request #59965 from nickitat/express_support
...
S3Express support
2024-03-23 20:52:44 +03:00
Smita Kulkarni
19e7022a87
Merge branch 'master' into Azure_write_buffer
2024-03-23 11:48:10 +01:00
Alexey Milovidov
3200aa3bcb
Merge pull request #61622 from YenchangChan/copy_s3_file
...
Remove duplicate code in copyS3File.cpp
2024-03-22 23:24:50 +03:00
Nikita Taranov
146d760338
rm more
2024-03-22 17:38:06 +00:00
Nikita Taranov
d7b34a80bb
stash
2024-03-22 16:09:14 +00:00
Smita Kulkarni
929173cce2
Fix S3 buffer allocation
2024-03-21 19:18:16 +01:00
Smita Kulkarni
71eee6a300
Merge branch 'master' into Azure_write_buffer
2024-03-21 16:22:28 +01:00
Alexey Milovidov
43b50c0df1
Merge pull request #61694 from evillique/fix-msan-issue-2
...
Fix use-of-uninitialized-value in parseDateTimeBestEffort
2024-03-21 18:16:11 +03:00
Smita Kulkarni
6c5e881da7
Merge branch 'master' into Azure_write_buffer
2024-03-21 11:12:19 +01:00
Smita Kulkarni
e0d14a1eaf
Updated lambda and name of BufferAllocationPolicy
2024-03-21 11:10:54 +01:00
Nikolay Degterinsky
f162ea8341
Fix use-of-uninitialized-value in parseDateTimeBestEffort
2024-03-21 09:51:10 +00:00
János Benjamin Antal
0b54b34b5b
Avoid instantiation of CacheBase's ctor in header file
2024-03-20 17:00:36 +00:00
János Benjamin Antal
9e8c731e1e
Remove some inclusion of magic enum
2024-03-20 17:00:36 +00:00
Smita Kulkarni
2c0e266cfe
Addressed review comments
2024-03-20 10:58:46 +01:00
YenchangChan
e7aa23d332
Remove duplicate code in copyS3File.cpp
2024-03-20 17:02:28 +08:00
kssenii
0122c3e883
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-19 19:31:02 +01:00
Raúl Marín
de855ca917
Reduce header dependencies
2024-03-19 17:04:29 +01:00
Smita Kulkarni
45e15d16e5
Merge branch 'master' into Azure_write_buffer
2024-03-19 09:31:11 +01:00
Smita Kulkarni
0530055842
Updated names in BufferAllocationPolicy
2024-03-18 15:59:05 +01:00
Robert Schulze
c17536c3b3
Enable clang-tidy in headers
2024-03-18 08:00:09 +00:00
avogar
feda83a7c8
Merge branch 'master' of github.com:ClickHouse/ClickHouse into 56257_parse_crlf_with_TSV_files
2024-03-14 17:44:38 +00:00
Robert Schulze
06d917fcf0
Merge pull request #61150 from rschu1ze/clang-tidy-list
...
Some clang-tidy fixes
2024-03-14 14:54:41 +01:00
Robert Schulze
3486b4a6e3
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-list
2024-03-14 10:44:20 +00:00
Kseniia Sumarokova
bc9dd7bb0b
Merge branch 'master' into revive-cache-contention-fix
2024-03-13 16:52:00 +01:00
kssenii
3931351ec4
Pass timeout through setting
2024-03-13 12:36:47 +01:00
kssenii
296f7a1da2
Fix
2024-03-12 19:02:58 +01:00
Kseniia Sumarokova
f7ac982a6b
Merge branch 'master' into use-iobject-storage-for-table-engines-1
2024-03-12 15:18:19 +01:00
Robert Schulze
1b1685e3f1
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-list
2024-03-12 13:02:59 +00:00
Shaun Struwig
f251a6d262
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-03-11 18:52:28 +01:00
Kseniia Sumarokova
38ba6990bb
Merge branch 'master' into use-iobject-storage-for-table-engines-1
2024-03-11 18:47:56 +01:00
Kseniia Sumarokova
4b94bcd54e
Update ReadSettings.h
2024-03-11 18:41:17 +01:00
kssenii
bfc4f5e1d3
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-11 14:57:15 +01:00
Robert Schulze
e5e84419af
Fix clang-tidy-s
2024-03-11 10:58:54 +00:00
Kseniia Sumarokova
3832b0e126
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-11 11:30:03 +01:00
kssenii
70272d4174
Minor
2024-03-11 10:55:01 +01:00
Smita Kulkarni
c724d36220
Merge branch 'master' into Azure_write_buffer
2024-03-11 09:27:24 +01:00
Sema Checherinda
77c5de700f
fix how web disk read empty directories
2024-03-10 10:00:18 +01:00
Sema Checherinda
5c41727725
http connections pools
2024-03-10 10:00:18 +01:00
Shaun Struwig
b8ff9b8aca
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-03-08 22:13:26 +01:00
Smita Kulkarni
10b5ce8ab3
Updated BufferAllocationPolicy
2024-03-07 10:26:50 +01:00
Michael Kolupaev
fb1a684d98
Fix race in PageCache ( #60878 )
...
* Fix race in PageCache
* Transportation security administration
* Fix test flakiness, remove accidentally left over debug logging
2024-03-06 08:46:15 -08:00
kssenii
0c293bce11
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-06 12:06:19 +08:00
Blargian
2ad8ab2a57
Fix linker errors
2024-03-05 19:13:20 +01:00
Shaun Struwig
beb0d08bdb
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-03-05 14:09:01 +01:00
Kseniia Sumarokova
dbbe6b9f69
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-05 11:13:32 +08:00
Sema Checherinda
71029373f2
Merge pull request #60748 from Beetelbrox/fjurado/dont-add-metadata-headers-to-unsupported-multipart-steps
...
Prevent setting custom metadata headers on unsupported multipart upload operations
2024-03-04 17:04:33 +01:00
Alexey Milovidov
de6c5d3a40
Merge pull request #60552 from ClickHouse/pc2
...
Userspace page cache again
2024-03-04 17:47:35 +03:00
beetelbrox
e8f4e4eb77
Fix formatting
2024-03-04 12:54:55 +01:00
beetelbrox
229945cdd2
Fix whitespace
2024-03-04 10:32:45 +01:00
beetelbrox
53e0d01d5e
Do not send metadata headers on unsupported multipartoperations
2024-03-04 10:28:27 +01:00
Alexey Milovidov
da2481c5c8
Merge remote-tracking branch 'origin/master' into remove-old-code-5
2024-03-04 00:12:03 +01:00
Alexey Milovidov
cbf5443585
Remove old code
2024-03-04 00:11:55 +01:00
Alexey Milovidov
d15f7b2158
Fix questionable behavior in the parseDateTimeBestEffort function
2024-03-03 01:42:59 +01:00
Shaun Struwig
b54286bed4
Merge branch 'master' into 56257_parse_crlf_with_TSV_files
2024-03-01 18:21:05 +01:00
Igor Nikonov
a28385966c
Merge pull request #60513 from ClickHouse/cleanup-random-progress-bar
...
Simple cleanup while fixing progress bar
2024-03-01 16:51:46 +01:00
Robert Schulze
3a5c9b627b
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-src
2024-03-01 10:31:55 +00:00
Smita Kulkarni
0f2d47e5a4
Renamed WriteBufferFromS3TaskTracker to ThreadPoolTaskTracker
2024-03-01 10:52:44 +01:00
Kseniia Sumarokova
5ac53c9578
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-01 11:05:47 +08:00
Robert Schulze
4ee1aa8c7c
Fixing more headers
2024-02-29 15:40:30 +00:00
Nikita Taranov
f9d1c5769e
Merge branch 'master' into express_support
2024-02-29 16:26:43 +01:00
Smita Kulkarni
5641fd8ba9
Fix build after merge
2024-02-29 16:13:05 +01:00
Joshua Hildred
58a8658528
Use std::array and std::any_of
2024-02-29 05:20:53 -08:00
Joshua Hildred
fe67293533
Add support for lzma
2024-02-29 05:20:53 -08:00
Joshua Hildred
ca9a6f809f
add support for .tar.zst and tar.xz
2024-02-29 05:20:53 -08:00
Joshua Hildred
e6134bbb7b
Address review comments
2024-02-29 05:20:52 -08:00
josh-hildred
f34ef716f0
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
1ce6256479
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
0fe15f0a22
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
5963b67dff
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
6b32271b88
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
f145b494c0
Update src/IO/Archives/LibArchiveWriter.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
84a3a05eb4
Update src/IO/Archives/LibArchiveReader.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
Joshua Hildred
f7ab6b9e37
Move tar compression extension definitions
2024-02-29 05:20:52 -08:00
Joshua Hildred
c02502b6f1
Removed unused error code
2024-02-29 05:20:52 -08:00
Joshua Hildred
92b446a552
Add support for tar archive compression
2024-02-29 05:20:52 -08:00
Joshua Hildred
c3118ca742
Address review comments
2024-02-29 05:20:52 -08:00
josh-hildred
714fdeb906
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
84814f2a0c
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
f94b5e95b2
Update src/IO/Archives/LibArchiveReader.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
e085b0fcf0
Update src/IO/Archives/IArchiveWriter.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
Joshua Hildred
5fdeebf785
Fix parameter naming
2024-02-29 05:20:51 -08:00
Joshua Hildred
2e2b89f9aa
Fix whitespace
2024-02-29 05:20:51 -08:00
Joshua Hildred
ef7a66b542
Update error messages and error handling
2024-02-29 05:20:51 -08:00
Joshua Hildred
d0776e5d2a
Fixes tidy issue and error formating for test
2024-02-29 05:20:51 -08:00
Joshua Hildred
dc74cebe4e
Fix formating
2024-02-29 05:20:51 -08:00
Joshua Hildred
c53021f55e
Fix formating
2024-02-29 05:20:51 -08:00
Joshua Hildred
48ad506fe3
Cleaned up the uses of the libarchive structs
2024-02-29 05:20:51 -08:00
josh-hildred
4c1ac01e25
Update src/IO/Archives/LibArchiveReader.cpp
...
Co-authored-by: ltrk2 <107155950+ltrk2@users.noreply.github.com>
2024-02-29 05:20:51 -08:00
Joshua Hildred
ac6d74334c
Address review comments
2024-02-29 05:20:51 -08:00
Joshua Hildred
e1fc5c8e48
Add archive unit tests
2024-02-29 05:20:51 -08:00
Joshua Hildred
3c77237dc4
getAllFiles now uses correct StreamInfo
2024-02-29 05:20:51 -08:00
Joshua Hildred
f9c0c99f39
Fix issue with LibArchiveWriter setCompression method
2024-02-29 05:20:51 -08:00
Joshua Hildred
1eb4cdf0f8
Add unit test coverage for ArchiveReaders
2024-02-29 05:20:51 -08:00
Joshua Hildred
eb4ec0e871
Add support for reading and writing backups as a
...
tar archive using libarchive.
2024-02-29 05:20:51 -08:00
Igor Nikonov
a8ee493b67
Merge remote-tracking branch 'origin/master' into cleanup-random-progress-bar
2024-02-29 13:09:20 +00:00
Robert Schulze
ed655815fe
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-src
2024-02-29 12:58:53 +00:00
Smita Kulkarni
3c00d19261
Merge branch 'master' into Azure_write_buffer
2024-02-29 13:39:31 +01:00
Michael Kolupaev
42437a2ae1
Userspace page cache ( #53770 )
...
* Userspace page cache
* Maybe it'll build this time, who knows.
* 'auto' went out of fashion, I guess
* Documentation, tsan workaround, metric 'UnreclaimableRss', disable page cache in the test that uses DatabaseOrdinary
* Moved CachedInMemoryReadBufferFromFile to object store level, changed settings, addressed other comments.
* Fix
* Another fix
* Fix restricted seek, fix ppc64le build
* Don't allow page cache with file cache
* Adjust tests a little
* Fix clang-tidy
* Conflicts
* Comments
* Maybe unbroke AsynchronousBoundedReadBuffer
* SettingsChangesHistory.h
* Fix warning in test
2024-02-29 12:28:32 +00:00
Alexander Tokmakov
15e1191f74
Revert "Userspace page cache ( #53770 )"
...
This reverts commit 7b55c61551
2024-02-29 13:20:17 +01:00
Igor Nikonov
59c3bb768f
Remove header
2024-02-29 12:19:27 +00:00
Michael Kolupaev
7b55c61551
Userspace page cache ( #53770 )
...
* Userspace page cache
* Maybe it'll build this time, who knows.
* 'auto' went out of fashion, I guess
* Documentation, tsan workaround, metric 'UnreclaimableRss', disable page cache in the test that uses DatabaseOrdinary
* Moved CachedInMemoryReadBufferFromFile to object store level, changed settings, addressed other comments.
* Fix
* Another fix
* Fix restricted seek, fix ppc64le build
* Don't allow page cache with file cache
* Adjust tests a little
* Fix clang-tidy
* Conflicts
* Comments
* Maybe unbroke AsynchronousBoundedReadBuffer
* SettingsChangesHistory.h
* Fix warning in test
2024-02-29 11:27:32 +00:00
Nikita Taranov
37917a3ed3
better
2024-02-29 01:42:32 +01:00
Nikita Taranov
974ba7364f
better
2024-02-29 00:55:17 +01:00
Robert Schulze
a1c164a758
Fix a bunch of clang-tidy warnings in headers
2024-02-28 23:43:12 +00:00
Alexey Milovidov
e1e66dc2b8
Merge pull request #60095 from ClickHouse/fix-inconsistent-formatting
...
Fix inconsistent formatting of queries
2024-02-29 00:41:40 +03:00
Raúl Marín
b12245fdc5
Decrease logging level for http retriable errors to Info
2024-02-28 19:47:42 +01:00
Alexey Milovidov
8cf619bc03
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-28 00:30:44 +01:00
Robert Schulze
e185805c1c
Minor: Replace boost::algorithm::ends_with() by std::string::ends_with()
2024-02-27 15:28:02 +00:00
Robert Schulze
466cf93ee2
Merge pull request #60450 from rschu1ze/starts_with
...
Minor: Replace `boost::algorithm::starts_with()` by `std::string::starts_with()`
2024-02-27 16:21:02 +01:00
Kseniia Sumarokova
5771e739f0
Update ReadSettings.h
2024-02-27 23:11:29 +08:00
Robert Schulze
62c12cbfca
Replace boost::algorithm::starts_with() by std::string::starts_with()
2024-02-27 10:42:47 +00:00
Alexey Milovidov
a3f1411679
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-26 19:01:11 +01:00
Antonio Andelic
ffe1872e23
Merge pull request #60203 from ClickHouse/limit_libarchive_format
...
Limit libarchive format to what we use
2024-02-26 09:25:19 +01:00
Alexey Milovidov
bf75d2878b
Merge branch 'master' into fix-inconsistent-formatting
2024-02-26 00:09:02 +01:00
Nikita Taranov
835b47519a
impl
2024-02-25 00:33:50 +01:00
Alexey Milovidov
9e32d0f5b2
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-23 15:05:46 +01:00
kssenii
f1e95fb78b
Add a way to force read-through cache for merges
2024-02-22 15:40:00 +01:00
Nikita Taranov
b44e4c6d78
Merge branch 'master' into fix_head_bucket_after_update
2024-02-22 14:32:00 +01:00
kssenii
f23ddec69f
Fix unit tests build
2024-02-22 11:50:36 +01:00
Smita Kulkarni
3eb1f24770
Merge branch 'master' into Azure_write_buffer
2024-02-22 11:31:47 +01:00
Blargian
e60ead336a
remove readEscapedStringInto function
2024-02-21 18:14:22 +01:00
Blargian
6b5953859e
Check for deserializeTextEscaped in other serializations, fix review changes
2024-02-21 18:00:05 +01:00
Maksim Kita
25bfcdd21f
Fixed code review issues
2024-02-21 11:41:25 +03:00
Maksim Kita
c0e1f50168
Copy S3 file GCP fallback to buffer copy
2024-02-21 11:41:25 +03:00
Alexey Milovidov
71c267e0b0
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-21 07:56:51 +01:00
santrancisco
be33d52016
Fix styling
2024-02-21 17:26:56 +11:00
Nikita Taranov
8401cd8c41
impl
2024-02-21 00:47:54 +01:00
Shaun Struwig
4a5761ce1f
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-02-20 22:18:01 +01:00
kssenii
bdd5eeb2eb
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-02-20 16:59:33 +01:00
Antonio Andelic
34e8e673e9
Merge pull request #60144 from ClickHouse/per-user-s3-settings
...
Support specifying users for S3 settings
2024-02-20 13:20:07 +01:00
Alexey Milovidov
ead2e7bc41
Revert "Restriction for the access key id for s3."
2024-02-20 12:32:25 +01:00
Alexey Milovidov
209acaa8b2
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-20 09:49:05 +01:00
santrancisco
c8a3a0bb11
Replacing libarchive filter_all and format_all code with specific compression method we want to support
2024-02-20 16:48:53 +11:00
kssenii
e23b929e18
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-02-19 19:55:47 +01:00
kssenii
0552f44f70
Fixes after merge with master, move some part of code to object storage
2024-02-19 16:43:49 +01:00
Antonio Andelic
175a1db787
Support specifying users for s3 settings
2024-02-19 16:43:27 +01:00
Vitaly Baranov
f10a8868e1
Merge pull request #60097 from vitlibar/move-thread-pool-callback-runner-to-common
...
Move threadPoolCallbackRunner to the "Common" folder
2024-02-19 14:10:36 +01:00
Kruglov Pavel
5fd2582e83
Merge pull request #59500 from Avogar/exponent-floats-inference
...
Don't infer floats in exponential notation by default
2024-02-19 13:51:07 +01:00
Blargian
d2fb1e4ff8
Add FormSchemaReader
2024-02-19 07:58:22 +01:00
Vitaly Baranov
3323d5ce81
Move threadPoolCallbackRunner to the "Common" folder.
2024-02-17 21:59:51 +01:00
Alexey Milovidov
69dfd721ce
Fix inconsistent formatting of INSERT queries
2024-02-17 10:11:59 +01:00
Alexey Milovidov
2f315e0eb5
Style check for abbreviations
2024-02-17 02:14:15 +01:00
Shaun Struwig
1d440f0399
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-02-16 19:48:33 +01:00
kssenii
1cccb3ccdf
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-02-16 15:21:10 +01:00
Kruglov Pavel
4d6f167e0c
Merge pull request #59092 from Avogar/auto-format-detection
...
Try to detect file format automatically during schema inference if it's unknown
2024-02-16 14:32:18 +01:00
Smita Kulkarni
62dfa4c65d
Merge branch 'master' into Azure_write_buffer
2024-02-16 11:19:44 +01:00
Alexey Milovidov
bbd7acd7f9
Merge branch 'master' into exponent-floats-inference
2024-02-15 01:46:51 +01:00
Shaun Struwig
525a5188e4
Merge branch 'master' into 56257_parse_crlf_with_TSV_files
2024-02-14 17:40:08 +01:00
Blargian
ba0dc7bc54
fix failing style check and tests
2024-02-14 17:38:11 +01:00
Dmitry Novik
433d835df6
Merge pull request #59920 from azat/http-retries-logs
...
Decrease logging level for http retriable errors to Warning (and fix 00157_cache_dictionary flakiness)
2024-02-14 17:11:28 +01:00
Blargian
6b4d42689c
add SchemaReader for Form format
2024-02-14 17:03:11 +01:00
Blargian
80b2276599
fix style check
2024-02-14 10:38:20 +01:00
Shaun Struwig
27a8bcc438
Update ReadHelpers.cpp to fix failing style check
2024-02-13 20:16:37 +01:00
Kruglov Pavel
1dbfeafb42
Merge branch 'master' into auto-format-detection
2024-02-13 19:08:33 +01:00
Kruglov Pavel
61983a7ec8
Merge branch 'master' into exponent-floats-inference
2024-02-13 19:04:07 +01:00
Smita Kulkarni
d12ecdc5f0
Asynchronous WriteBuffer for AzureBlobStorage
2024-02-13 12:35:17 +01:00
Nikita Mikhaylov
4b85f6b39a
Revert "Less error prone interface of read buffers" ( #59911 )
2024-02-13 12:02:46 +01:00
Azat Khuzhin
4228eaf44a
Decrease logging level for http retriable errors to Warning
...
Messages like this:
2024-02-12 21:25:24 [5c66f008cd40] 2024.02.12 14:25:01.596769 [ 2530 ] {e71fcb6d-356d-4962-95bd-ef8b8c504e11} <Error> ReadWriteBufferFromHTTP: HTTP request to `https://clickhouse-datasets-web.s3.us-east-1.amazonaws.com/store/78e/78ebf6a1-d987-4579-b3ec-00c1a087b1f3/201403_1_1_2/UserAgent.bin ` failed at try 1/10 with bytes read: 311149/378695. Error: DB::HTTPException: Received error from remote server /store/78e/78ebf6a1-d987-4579-b3ec-00c1a087b1f3/201403_1_1_2/UserAgent.bin. HTTP status code: 500 Internal Server Error, body: <?xml version="1.0" encoding="UTF-8"?>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-13 08:31:28 +01:00
Alexey Milovidov
a7c30098b6
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-02-09 22:52:49 +01:00
kssenii
3cca841038
Unite s3/hdfs/azure storage implementations into a single class on top of IObjectStorage
2024-02-09 21:32:04 +01:00
Vitaly Baranov
09e727d48b
Merge pull request #56988 from ClickHouse/Azure_backup
...
Backup & Restore support for AzureBlobStorage
2024-02-09 14:45:35 +01:00
Blargian
0ff4202452
fix up test and modify skipRowEndDelimiter so that crlf works with .tsv
2024-02-08 07:08:17 +01:00
Smita Kulkarni
e594608c42
Merge branch 'master' into Azure_backup
2024-02-07 13:48:46 +01:00
Alexey Milovidov
d65a56d967
Merge branch 'master' into enable-coverage-for-debug-build
2024-02-07 04:58:21 +01:00
Smita Kulkarni
fd2fdcdb4b
Fixed unwanted dereferencing
2024-02-05 11:23:37 +01:00
Smita Kulkarni
f036948f91
Fix clang tidy build
2024-02-05 10:09:48 +01:00
Smita Kulkarni
23b9f43d4f
Fix style
2024-02-04 16:18:17 +01:00
Blargian
ab384f8652
add support_crlf for TSV format
2024-02-04 15:29:57 +01:00
Smita Kulkarni
b0994c5fa7
Addressed comments, added test for named collection
2024-02-04 11:28:20 +01:00
avogar
ff21aa9a19
Don't infer floats in exponential notation by default
2024-02-01 19:47:05 +00:00
Konstantin Bogdanov
5969668503
Concurrent table creation in DatabaseReplicated
2024-02-01 09:51:52 +01:00
Dmitry Novik
5b630ed745
Merge pull request #59399 from azat/build/format-fwd-decl
...
Forward declaration for PeekableReadBuffer
2024-02-01 09:15:07 +01:00
Kruglov Pavel
4987708583
Merge pull request #59298 from Avogar/fix-getting-filename-from-read-buffers
...
Fix getting filename from read buffer wrappers
2024-01-31 14:34:07 +01:00
Azat Khuzhin
010f1c6cb7
Forward declaration for PeekableReadBuffer
...
ReadHelpers.h is very common header and is the root cause of "recompile
everything".
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-30 19:24:19 +01:00
avogar
377937d415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into auto-format-detection
2024-01-29 15:45:18 +00:00
Alexey Milovidov
cb69a1b25d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-01-29 15:35:04 +01:00
Smita Kulkarni
3183026521
Merge branch 'master' into Azure_backup
2024-01-29 11:08:13 +01:00
Smita Kulkarni
a22b68f46f
Added setting azure_max_single_part_copy_size
2024-01-29 10:49:36 +01:00
Alexey Milovidov
4a94ae5a1b
Merge branch 'master' into enable-coverage-for-debug-build
2024-01-28 22:24:26 +01:00
Smita Kulkarni
d264a5a148
Update client & settings to std::shared_ptr<const>
2024-01-28 12:06:52 +01:00
avogar
00c140b7bb
Fix getting filename from read buffer wrappers
2024-01-27 21:13:08 +00:00
avogar
5833641fa5
Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type
2024-01-26 16:54:02 +00:00
Kruglov Pavel
46a6b84a5a
Merge branch 'master' into auto-format-detection
2024-01-25 22:11:07 +01:00
Maksim Kita
2a327107b6
Updated implementation
2024-01-25 14:31:49 +03:00
Smita Kulkarni
09f1e2840c
Simplified calculatePartSize and upload task
2024-01-25 10:06:05 +01:00
Smita Kulkarni
f551081dd4
Addressed review comments
2024-01-24 21:10:50 +01:00
Kruglov Pavel
6c0615d493
Merge pull request #59099 from skyoct/fix/fix_mult_redirect
...
Disable "Too many redirects" error retry
2024-01-24 12:22:01 +01:00
Kruglov Pavel
718064bb43
Minor change to restart CI
2024-01-24 12:19:10 +01:00
Smita Kulkarni
4d92eba316
Merge branch 'master' into Azure_backup
2024-01-24 11:13:57 +01:00
skyoct
95829e074e
Opt code
2024-01-24 03:49:21 +00:00
avogar
617cc514b7
Try to detect file format automatically during schema inference if it's unknown
2024-01-23 18:59:39 +00:00
Anton Popov
4292ee7d51
Merge pull request #58886 from CurtizJ/better-read-buffers
...
Less error prone interface of read buffers
2024-01-23 14:59:27 +01:00
Alexander Gololobov
9a0e5d0aac
Merge pull request #58914 from ClickHouse/ot_spans_in_insert
...
Opentelemetry spans to analyze CPU and S3 bottlenecks on inserts
2024-01-23 13:53:13 +01:00
Smita Kulkarni
2e7ce5b0e2
Updated settings ptr and fetching of client from Disk & ObjectStorage
2024-01-22 16:24:43 +01:00
Anton Popov
e1e7a9cd45
Merge branch 'master' into better-read-buffers
2024-01-22 15:50:48 +01:00
Nikita Taranov
4d6ef8ed86
Merge pull request #56961 from nickitat/some_questionable_code
...
Bypass `Poco::BasicBufferedStreamBuf` in `ReadBufferFromIStream`
2024-01-22 14:08:09 +01:00
Sema Checherinda
9adab57771
Correct initialization of ConnectionTimeouts ( #59000 )
2024-01-22 12:03:36 +01:00
skyoct
fdf8008804
Fix redirect retry
2024-01-22 10:27:35 +00:00
Vitaly Baranov
9377f4276b
Merge pull request #58962 from vitlibar/fix-memory-dealloc-in-copy-data-to-s3
...
Fix memory management in copyDataToS3File
2024-01-20 23:35:42 +01:00
Alexey Milovidov
dc1dac3ba4
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-01-20 21:41:07 +01:00
Alexey Milovidov
735708e9bb
Merge branch 'master' into enable-coverage-for-debug-build
2024-01-20 21:40:53 +01:00
Sema Checherinda
1bbb5f7f51
Merge pull request #59015 from ClickHouse/chesema-adaptive-timeouts-gcp
...
preserve 'amz-sdk-invocation-id' and 'amz-sdk-request' headers with gcp
2024-01-20 17:48:20 +01:00
serxa
172d68ceca
Merge branch 'master' into cpu-scheduler
2024-01-20 12:21:21 +00:00
serxa
824deab761
split resource scheduler off IO/ into Common/Scheduler/
2024-01-20 12:06:26 +00:00
Alexey Milovidov
8acc981e8c
Merge pull request #59008 from ClickHouse/fix-log-level-for-http
...
Update log level for http buffer
2024-01-20 04:02:42 +01:00
Kruglov Pavel
20379d34d5
Merge branch 'master' into variant-data-type
2024-01-19 23:17:42 +01:00
Sema Checherinda
539451bbe3
preserve 'amz-sdk-invocation-id' and 'amz-sdk-request' headers with gcp
2024-01-19 21:07:47 +01:00
kssenii
4c8ced6af8
Fix log level
2024-01-19 18:04:58 +01:00
Alexey Milovidov
6a75641048
Merge pull request #58966 from ClickHouse/arseek
...
Fix seek in ReadBufferFromZipArchive
2024-01-19 18:04:03 +01:00
Vitaly Baranov
19bc0b744f
Add comments and const after review.
2024-01-19 12:54:55 +01:00
Alexey Milovidov
dcdbaf9150
Merge pull request #58900 from MikhailBurdukov/restriction_for_s3_access_key_id
...
Restriction for the access key id for s3.
2024-01-19 04:13:17 +01:00
Alexey Milovidov
370815803e
Update Client.cpp
2024-01-19 06:12:44 +03:00
Alexey Milovidov
7c0b167e2d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-01-19 03:59:27 +01:00
Michael Kolupaev
3a19ea6e6c
Fix seek in ReadBufferFromZipArchive
2024-01-19 01:46:07 +00:00
Vitaly Baranov
24ca8fbbec
Improve logs.
2024-01-18 21:15:39 +01:00
Vitaly Baranov
0caafe4810
Fix memory deallocation in copyDataToS3File()
...
- ReadBufferFromS3 is created now only when it's necessary and destroyed after that.
2024-01-18 20:48:56 +01:00
Nikita Taranov
b5e333df96
remove test
2024-01-18 19:56:02 +01:00
Kruglov Pavel
5444cde408
Merge branch 'master' into variant-data-type
2024-01-18 18:31:27 +01:00
MikhailBurdukov
b3d74fc907
style
2024-01-18 15:39:25 +00:00
MikhailBurdukov
eac310fcb2
Review
2024-01-18 15:19:11 +00:00
Smita Kulkarni
e71977b724
Merge branch 'master' into Azure_backup
2024-01-18 08:29:01 +01:00
Yakov Olkhovskiy
490bb809c3
Merge pull request #58846 from ClickHouse/fix-compress-destructor
...
Fix HTTP compressors finalization
2024-01-17 17:49:12 -05:00
Alexey Milovidov
c098f41dbc
Merge branch 'master' into enable-coverage-for-debug-build
2024-01-17 22:35:51 +01:00
Alexey Milovidov
f499cf3fd5
Update Client.cpp
2024-01-17 22:30:18 +03:00
Alexander Gololobov
0a42dbafe8
Add attributes
2024-01-17 17:46:15 +01:00
Nikita Taranov
1db805dfa4
impl
2024-01-17 16:24:24 +01:00
Alexander Gololobov
7dda18c209
Spans in mergeBlock and finalizeImpl
2024-01-17 16:07:53 +01:00
Anton Popov
245e1cbd5c
Merge remote-tracking branch 'upstream/master' into better-read-buffers
2024-01-17 13:46:48 +00:00
Nikita Taranov
5485d2a2ec
impl
2024-01-17 14:38:06 +01:00
Nikita Taranov
bbf070caa6
impl
2024-01-17 14:08:02 +01:00
Yakov Olkhovskiy
d5d060a17e
fix memory leak in bzip2
2024-01-17 12:53:37 +00:00
Kruglov Pavel
6d064512e1
Merge pull request #58614 from Blargian/58363_disable_ansi_pretty_automatically
...
58363 Automatically disable ANSI escape sequences in Pretty formats if the output is not a terminal
2024-01-17 13:45:41 +01:00
Nikita Taranov
bb124b544c
Merge pull request #58077 from nickitat/revert_rest
...
Remove unneeded parameter `use_external_buffer` from `AsynchronousReadBuffer*`
2024-01-17 13:45:07 +01:00
MikhailBurdukov
e2729ed4c9
Restriction for the access key id for s3.
2024-01-17 10:43:58 +00:00
Yakov Olkhovskiy
e6b3f69137
style fix
2024-01-17 02:58:18 +00:00
Yakov Olkhovskiy
e06e4695e1
refactoring bzip2 buffer and fix possible data loss in internal buffer
2024-01-17 01:55:38 +00:00
Anton Popov
299293c088
better interface in read buffers
2024-01-16 23:26:20 +00:00
Yakov Olkhovskiy
92d42567fa
separate explicit finalization for buffers
2024-01-16 22:15:35 +00:00
Yakov Olkhovskiy
fcd4297d20
fix compressors finalization
2024-01-16 00:22:17 +00:00
xleoken
6f7e7c95ba
Ecloud EOS Storage support
2024-01-15 11:07:43 -05:00
Alexey Milovidov
e4cd02ea39
Fix typo
2024-01-15 00:36:24 +01:00
Alexey Milovidov
d112492c56
Remove some code
2024-01-13 03:48:04 +01:00
Blargian
aa8876a611
#58363 - Changes based on review of draft PR - changed output_format_pretty_color to use UInt64Auto. Added isWritingToTerminal function to IO/WriteHelpers.h and updated test
2024-01-12 12:31:57 +02:00
Ilya Yatsishin
904c02d955
Fix
2024-01-12 12:43:11 +04:00
Alexey Milovidov
df04476037
Update src/IO/TimeoutSetter.cpp
...
Co-authored-by: Konstantin Bogdanov <thevar1able@users.noreply.github.com>
2024-01-12 07:41:24 +03:00
Alexey Milovidov
6c604ea1a7
Fix Mac OS X
2024-01-12 04:00:36 +01:00
Yakov Olkhovskiy
4fc11e189c
Merge branch 'master' into revert-58450-revert-56064-feature-server-iface-metrics
2024-01-10 12:03:47 -05:00
Yakov Olkhovskiy
45d5e29ba0
master merge adjustments
2024-01-10 16:55:09 +00:00
Robert Schulze
52d75ab68f
Merge pull request #58458 from rschu1ze/replace-std_regexp-by-re2
...
Replace `std::regex` by re2
2024-01-10 17:46:54 +01:00
Vitaly Baranov
dbb19e72b3
Merge pull request #58629 from vitlibar/remove-finalize-from-destructor-of-write-buffer-from-encrypted-file
...
Remove finalize() from ~WriteBufferFromEncryptedFile
2024-01-10 16:36:11 +01:00
Yakov Olkhovskiy
2e124b19b8
Merge branch 'master' into revert-58450-revert-56064-feature-server-iface-metrics
2024-01-10 09:05:18 -05:00
Robert Schulze
499227b9cf
Merge remote-tracking branch 'rschu1ze/master' into replace-std_regexp-by-re2
2024-01-10 10:02:53 +00:00
pufit
6cf55b82f4
Merge pull request #58539 from canhld94/file_custom_compress_level
...
Allow explicitly set compression level in output format
2024-01-09 13:43:38 -05:00
Vitaly Baranov
01dfe282ea
Remove finalize() from ~WriteBufferFromEncryptedFile.
2024-01-09 17:22:17 +01:00
Sema Checherinda
7c7e72c4b7
Merge pull request #58573 from ClickHouse/chesema-s3-client-creation
...
fix and test that S3Clients are reused
2024-01-09 13:28:20 +01:00
Kruglov Pavel
b947609b8e
Merge branch 'master' into variant-data-type
2024-01-08 15:04:51 +01:00
Robert Schulze
f553b55e3a
Merge remote-tracking branch 'rschu1ze/master' into regex-std-re2
2024-01-07 22:31:35 +00:00
Robert Schulze
8e804487f3
Some fixups
2024-01-07 22:28:08 +00:00
Sema Checherinda
6f626d8294
fix auth_settings.hasUpdates function
2024-01-07 23:00:26 +01:00
Smita Kulkarni
fd92c1961e
Fix clang tidy build
2024-01-07 16:33:48 +01:00
Yakov Olkhovskiy
45ac4d2b67
Merge remote-tracking branch 'origin/master' into revert-58450-revert-56064-feature-server-iface-metrics
2024-01-07 15:23:46 +00:00
Sema Checherinda
d5f86f671d
fix and test that S3Clients reused
2024-01-07 02:19:06 +01:00
avogar
7e5ba62017
Allow to read Bool values into String in JSON input formats
2024-01-05 20:33:30 +00:00
Smita Kulkarni
b250acff78
Fixed style check
2024-01-05 11:26:32 +01:00
Duc Canh Le
29c4d96b48
fix build and add a tests
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-05 06:54:56 +00:00
Smita Kulkarni
c14605caa7
Added flag use_native_copy and updated to use StartCopyFromUri for native copy with large files
2024-01-04 18:27:54 +01:00
Smita Kulkarni
91bad5bc39
Updated to use MultiVersion for BlobContainerClient in Backups and updated to get client from disk
2024-01-04 16:06:36 +01:00
Duc Canh Le
2e14cfb526
add settings for output compression level and window size
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-04 08:16:00 +00:00
Yakov Olkhovskiy
85f03478ef
Revert "Revert "Use CH Buffer for HTTP out stream, add metrics for interfaces""
2024-01-03 11:47:15 -05:00
Kruglov Pavel
4d8cf71ba7
Merge branch 'master' into variant-data-type
2024-01-03 15:21:23 +01:00
Kruglov Pavel
570d1c013b
Merge pull request #57053 from Avogar/better-parsing-exceptions
...
Better exception messages in input formats
2024-01-03 14:47:51 +01:00
Smita Kulkarni
df221f7db6
Renamed Bucket-Key to Container-Blob
2024-01-03 11:35:06 +01:00
Raúl Marín
d491758939
Revert "Use CH Buffer for HTTP out stream, add metrics for interfaces"
2024-01-03 10:42:15 +01:00
Smita Kulkarni
4122de9721
Updated tests and added settings
2024-01-02 20:19:01 +01:00
Yakov Olkhovskiy
5633fb8145
Merge pull request #56064 from ClickHouse/feature-server-iface-metrics
...
Use CH Buffer for HTTP out stream, add metrics for interfaces
2024-01-02 10:17:52 -05:00
Kruglov Pavel
e69bda7f08
Merge pull request #57438 from Avogar/fix-resetting-buffers
...
Fix working with read buffers in StreamingFormatExecutor
2024-01-02 15:34:35 +01:00
Alexey Milovidov
2f3e17c737
Remove commented code
2023-12-30 18:39:12 +01:00
Alexey Milovidov
a1bd76388e
Remove commented code
2023-12-30 18:38:41 +01:00
Alexey Milovidov
aad63c8273
Merge branch 'master' into feature-server-iface-metrics
2023-12-30 15:44:42 +01:00
Alexey Milovidov
8365722c95
Merge pull request #58336 from amosbird/macro2constexpr
...
Organize symbols from src/* into DB namespace
2023-12-30 12:40:36 +01:00
Amos Bird
907e844635
Fix examples
2023-12-30 10:48:19 +08:00
Alexey Milovidov
aa462fa7cd
Merge branch 'master' into feature-server-iface-metrics
2023-12-29 22:37:52 +01:00
Amos Bird
18395a387c
Fix another unitest
2023-12-30 01:14:21 +08:00
Kruglov Pavel
c03e36e012
Merge branch 'master' into better-parsing-exceptions
2023-12-29 18:07:32 +01:00
Azat Khuzhin
8c54380d80
Avoid sending ComposeObject requests after upload to GCS
...
This should not be required anymore, but leave it as an option, since
likely this is required for old files.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Azat Khuzhin
f4a7789cd4
Convert various S3::Client settings into separate ClientSettings struct
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Azat Khuzhin
2b2d10689b
Make S3::Client::ComposeObject private
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
avogar
c6085b3eb9
Fix working with read buffers in StreamingFormatExecutor
2023-12-28 18:47:57 +00:00
Smita Kulkarni
bf60a06a11
Merge branch 'master' into Azure_backup
2023-12-28 11:51:49 +01:00
Azat Khuzhin
c6774cfe42
Add missing includes for ZipArchiveWriter
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-27 17:06:17 +01:00
Azat Khuzhin
cf1dae5321
Add missing includes into BitHelpers.h
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-27 17:06:17 +01:00
Azat Khuzhin
b9233f6d4f
Move Allocator code into module part
...
This should reduce amount of code that should be recompiled on
Exception.h changes (and everything else that had been included there).
This will actually not help a lot, because it is also included into
PODArray.h and ThreadPool.h at least... Sigh.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-27 15:42:08 +01:00
Smita Kulkarni
d46d914521
Updated thread name
2023-12-27 10:28:52 +01:00
Yarik Briukhovetskyi
41c275274b
Merge pull request #58148 from yariks5s/fix_s3_regions
...
S3-links region independency
2023-12-26 12:59:00 +01:00
Alexey Milovidov
817d0c6553
Merge pull request #58202 from vitlibar/explicit-finalize-in-zip-archive-writer
...
Explicit finalize in ZipArchiveWriter
2023-12-24 03:30:21 +01:00
Vitaly Baranov
e1a136b791
Explicit finalize() function in ZipArchiveWriter.
...
Simplify too complicated code in ZipArchiveWriter.
2023-12-24 00:33:59 +01:00
Alexey Milovidov
a400657155
Merge pull request #58165 from azat/s3-passthrough-exception
...
Pass through exceptions for reading from S3
2023-12-23 11:35:23 +01:00
Azat Khuzhin
896cbb6a75
Retry on poco exceptions while reading from S3 as well
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-23 09:07:43 +01:00
Alexey Milovidov
ffca9bc564
Fix #58171
2023-12-23 02:04:35 +01:00
Azat Khuzhin
17e21aa21e
Pass throught exceptions for reading from S3
...
Reading from S3 has retries on almost any exception, except for
CANNOT_ALLOCATE_MEMORY, but the problem is that this exception may not
be even delivered to ReadBufferFromS3::processException(), due to
std::istream will hide it (see [1]).
[1]: 1834e42289/libcxx/include/istream (L1059)
2023-12-22 23:29:28 +01:00
yariks5s
50724f1b5d
Merge branch 'fix_s3_regions' of github.com:yariks5s/ClickHouse into fix_s3_regions
2023-12-22 17:28:22 +00:00
yariks5s
a543048b5f
review fix
2023-12-22 17:28:09 +00:00
Yarik Briukhovetskyi
ab6cc0c9f0
Update Client.cpp
2023-12-22 14:59:21 +01:00
yariks5s
823abaa73e
init
2023-12-22 13:27:03 +00:00
Yakov Olkhovskiy
5dc88542cb
Merge branch 'master' into feature-server-iface-metrics
2023-12-20 16:46:29 -05:00
Kruglov Pavel
ca7001d70f
Merge pull request #57850 from shanielh/feature/s3-session-token
...
Feature/s3 session token
2023-12-20 17:22:09 +01:00
Nikita Taranov
354b689800
impl
2023-12-20 16:07:00 +01:00
Kruglov Pavel
087946f385
Merge branch 'master' into variant-data-type
2023-12-20 00:40:18 +01:00
Julia Kartseva
2c32589949
Merge pull request #58019 from jkartseva/fix-build-errno-exception
...
fix freebsd build
2023-12-19 11:16:45 -08:00
Sema Checherinda
802ad4ec47
Merge pull request #57523 from ClickHouse/remove-the-limit-for-connections-per-endpoint
...
Remove the limit for the number of connections per endpoint for background fetches + small refactoring
2023-12-19 19:16:01 +01:00
avogar
319ae440b6
Implement Variant data type
2023-12-19 16:45:15 +00:00
vdimir
252cb8a507
Merge pull request #54602 from fionera/issue-49028
...
Add setting http_make_head_request
2023-12-19 13:33:59 +01:00
Julia Kartseva
31f04b66c5
fix freebsd build
...
https://github.com/ClickHouse/ClickHouse/actions/runs/7256640256/job/19769624432?pr=58015#step:8:5466
```
Dec 19 03:09:41 /build/src/IO/AIO.cpp:140:15: error: use of undeclared identifier 'ErrnoException'; did you mean 'DB::ErrnoException'?
Dec 19 03:09:41 140 | throw ErrnoException(DB::ErrorCodes::CANNOT_IOSETUP, "io_setup failed");
```
2023-12-19 03:47:28 +00:00
Nikita Mikhaylov
6360b76792
Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-the-limit-for-connections-per-endpoint
2023-12-18 21:49:31 +00:00
Raúl Marín
65728f14ce
Review error codes
2023-12-18 10:40:02 +01:00
Shani Elharrar
c696c0bfe7
S3Common.AuthSettings: Allow passing SESSION_TOKEN to AWSCredentials
...
This sets the infrastructure of loading session_token and passing it directly
to all AWSCredentials instances that are created using the AuthSettings.
The default SESSION_TOKEN is set to an empty string as documented in AWS SDK
reference: https://sdk.amazonaws.com/cpp/api/0.12.9/d4/d27/class_aws_1_1_auth_1_1_a_w_s_credentials.html
2023-12-17 10:29:15 +02:00
Yakov Olkhovskiy
42a770f209
optimize event update
2023-12-15 19:51:52 +00:00
Yakov Olkhovskiy
001a38048f
use ProfileEvents instead of CurrentMetrics
2023-12-15 19:17:42 +00:00
Raúl Marín
b269f87f4c
Better text_log with ErrnoException
2023-12-15 19:27:56 +01:00
Yakov Olkhovskiy
febdd542f8
Merge remote-tracking branch 'origin/master' into feature-server-iface-metrics
2023-12-14 18:59:44 +00:00
Nikita Mikhaylov
a0af0392cd
Random changes in random files ( #57642 )
2023-12-14 12:47:11 +01:00
Nikita Mikhaylov
8372c70958
Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-the-limit-for-connections-per-endpoint
2023-12-13 18:29:56 +00:00
Kruglov Pavel
ee39dca8c7
Merge branch 'master' into better-parsing-exceptions
2023-12-12 17:10:38 +01:00
Kruglov Pavel
909b1fb36b
Merge pull request #57004 from Avogar/template-deadlock
...
Fix possible deadlock in Template format during sync after error
2023-12-11 19:02:40 +01:00
Igor Nikonov
87e47bdbee
Merge pull request #57467 from ClickHouse/rm-select
...
Remove select() usage
2023-12-09 22:27:28 +01:00
Alexey Milovidov
0ee13fde16
Fix "logs and exception messages formatting", part 3
2023-12-07 22:01:53 +01:00
Igor Nikonov
d196973ab3
Merge remote-tracking branch 'origin/master' into rm-select
2023-12-05 12:59:18 +00:00
Nikita Mikhaylov
04d167c6d9
Better
2023-12-05 13:34:37 +01:00
Alexey Milovidov
10d65a1ade
Merge pull request #55559 from azat/s3-fix-excessive-reads
...
Add ability to disable checksums for S3 to avoid excessive input file read
2023-12-05 06:34:21 +01:00
kssenii
4a28f10c3d
Minor cache changes
2023-12-04 19:02:37 +01:00
vdimir
a4ae90de0d
Merge pull request #57275 from ClickHouse/vdimir/merge_task_tmp_data
...
Background merges correctly use temporary data storage in the cache
2023-12-04 14:52:20 +01:00
Igor Nikonov
434c2113d2
Fix style
2023-12-04 10:51:17 +00:00
Igor Nikonov
73ff984755
Remove select() usage
2023-12-04 09:33:34 +00:00
robot-ch-test-poll
1b49463bd2
Merge pull request #55841 from nickitat/optimize_reading3
...
Optimize reading from cache
2023-12-01 17:36:57 +01:00
Yakov Olkhovskiy
416d28687e
Merge remote-tracking branch 'origin/master' into feature-server-iface-metrics
2023-11-30 23:00:04 +00:00
Nikolai Kochetov
823ba2db46
Merge pull request #57075 from yariks5s/s3_links_fix
...
S3-style links bug fix
2023-11-29 17:41:08 +01:00
vdimir
b5babe1692
MergeTask uses temporary data storage
2023-11-29 16:18:32 +00:00
Nikita Taranov
52f644c0df
Merge branch 'master' into optimize_reading3
2023-11-28 16:36:38 +01:00
Antonio Andelic
6e8e4a6ca5
Lower level for annoying log
2023-11-28 12:41:35 +00:00
Yakov Olkhovskiy
d8e41d8886
fix after merge
2023-11-28 03:28:32 +00:00
Yakov Olkhovskiy
c0a9b82c3c
fix
2023-11-27 22:06:01 +00:00
Yakov Olkhovskiy
52ef3fe34b
style fix
2023-11-27 21:32:50 +00:00
