Michael Kolupaev
1fb902c5b7
Fix sending a new request every 2 MB
2024-03-26 09:14:55 +00:00
avogar
dc87c483dd
Add a setting to allow saving bad escape sequences in JSON input formats
2024-03-25 21:58:53 +00:00
Alexey Milovidov
21a0d17637
Merge branch 'master' into geodistance-double-precision
2024-03-25 17:47:42 +03:00
kssenii
46941e7243
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-25 15:10:11 +01:00
Alexey Milovidov
65650809b4
Double precision of geoDistance if the arguments are Float64
2024-03-24 23:11:35 +01:00
Smita Kulkarni
0d54dbf460
Merge branch 'master' into Azure_write_buffer
2024-03-24 20:09:47 +01:00
Alexey Milovidov
8ddfe9a48b
Merge pull request #61665 from ClickHouse/build-improvements-magic-enum-template-instantiation
...
Build improvements by removing magic enum from header and apply some explicit template instantiation
2024-03-24 19:31:46 +03:00
alesapin
a41c005a6e
Merge branch 'master' into Azure_write_buffer
2024-03-24 15:04:18 +01:00
Alexey Milovidov
056c8ce021
Merge pull request #59965 from nickitat/express_support
...
S3Express support
2024-03-23 20:52:44 +03:00
Smita Kulkarni
19e7022a87
Merge branch 'master' into Azure_write_buffer
2024-03-23 11:48:10 +01:00
Alexey Milovidov
3200aa3bcb
Merge pull request #61622 from YenchangChan/copy_s3_file
...
Remove duplicate code in copyS3File.cpp
2024-03-22 23:24:50 +03:00
Smita Kulkarni
929173cce2
Fix S3 buffer allocation
2024-03-21 19:18:16 +01:00
Smita Kulkarni
71eee6a300
Merge branch 'master' into Azure_write_buffer
2024-03-21 16:22:28 +01:00
Alexey Milovidov
43b50c0df1
Merge pull request #61694 from evillique/fix-msan-issue-2
...
Fix use-of-uninitialized-value in parseDateTimeBestEffort
2024-03-21 18:16:11 +03:00
Smita Kulkarni
6c5e881da7
Merge branch 'master' into Azure_write_buffer
2024-03-21 11:12:19 +01:00
Smita Kulkarni
e0d14a1eaf
Updated lambda and name of BufferAllocationPolicy
2024-03-21 11:10:54 +01:00
Nikolay Degterinsky
f162ea8341
Fix use-of-uninitialized-value in parseDateTimeBestEffort
2024-03-21 09:51:10 +00:00
János Benjamin Antal
0b54b34b5b
Avoid instantiation of CacheBase's ctor in header file
2024-03-20 17:00:36 +00:00
János Benjamin Antal
9e8c731e1e
Remove some inclusion of magic enum
2024-03-20 17:00:36 +00:00
Smita Kulkarni
2c0e266cfe
Addressed review comments
2024-03-20 10:58:46 +01:00
YenchangChan
e7aa23d332
Remove duplicate code in copyS3File.cpp
2024-03-20 17:02:28 +08:00
kssenii
0122c3e883
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-19 19:31:02 +01:00
Raúl Marín
de855ca917
Reduce header dependencies
2024-03-19 17:04:29 +01:00
Smita Kulkarni
45e15d16e5
Merge branch 'master' into Azure_write_buffer
2024-03-19 09:31:11 +01:00
Smita Kulkarni
0530055842
Updated names in BufferAllocationPolicy
2024-03-18 15:59:05 +01:00
Robert Schulze
c17536c3b3
Enable clang-tidy in headers
2024-03-18 08:00:09 +00:00
avogar
feda83a7c8
Merge branch 'master' of github.com:ClickHouse/ClickHouse into 56257_parse_crlf_with_TSV_files
2024-03-14 17:44:38 +00:00
Robert Schulze
06d917fcf0
Merge pull request #61150 from rschu1ze/clang-tidy-list
...
Some clang-tidy fixes
2024-03-14 14:54:41 +01:00
Robert Schulze
3486b4a6e3
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-list
2024-03-14 10:44:20 +00:00
Kseniia Sumarokova
bc9dd7bb0b
Merge branch 'master' into revive-cache-contention-fix
2024-03-13 16:52:00 +01:00
kssenii
3931351ec4
Pass timeout through setting
2024-03-13 12:36:47 +01:00
kssenii
296f7a1da2
Fix
2024-03-12 19:02:58 +01:00
Kseniia Sumarokova
f7ac982a6b
Merge branch 'master' into use-iobject-storage-for-table-engines-1
2024-03-12 15:18:19 +01:00
Robert Schulze
1b1685e3f1
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-list
2024-03-12 13:02:59 +00:00
Shaun Struwig
f251a6d262
Merge branch 'ClickHouse:master' into 59557_form_input_format
2024-03-11 18:52:28 +01:00
Kseniia Sumarokova
38ba6990bb
Merge branch 'master' into use-iobject-storage-for-table-engines-1
2024-03-11 18:47:56 +01:00
Kseniia Sumarokova
4b94bcd54e
Update ReadSettings.h
2024-03-11 18:41:17 +01:00
kssenii
bfc4f5e1d3
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-11 14:57:15 +01:00
Robert Schulze
e5e84419af
Fix clang-tidy-s
2024-03-11 10:58:54 +00:00
Kseniia Sumarokova
3832b0e126
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-11 11:30:03 +01:00
kssenii
70272d4174
Minor
2024-03-11 10:55:01 +01:00
Smita Kulkarni
c724d36220
Merge branch 'master' into Azure_write_buffer
2024-03-11 09:27:24 +01:00
Sema Checherinda
77c5de700f
fix how web disk read empty directories
2024-03-10 10:00:18 +01:00
Sema Checherinda
5c41727725
http connections pools
2024-03-10 10:00:18 +01:00
Shaun Struwig
b8ff9b8aca
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-03-08 22:13:26 +01:00
Smita Kulkarni
10b5ce8ab3
Updated BufferAllocationPolicy
2024-03-07 10:26:50 +01:00
Michael Kolupaev
fb1a684d98
Fix race in PageCache ( #60878 )
...
* Fix race in PageCache
* Transportation security administration
* Fix test flakiness, remove accidentally left over debug logging
2024-03-06 08:46:15 -08:00
kssenii
0c293bce11
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-06 12:06:19 +08:00
Blargian
2ad8ab2a57
Fix linker errors
2024-03-05 19:13:20 +01:00
Shaun Struwig
beb0d08bdb
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-03-05 14:09:01 +01:00
Kseniia Sumarokova
dbbe6b9f69
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-05 11:13:32 +08:00
Sema Checherinda
71029373f2
Merge pull request #60748 from Beetelbrox/fjurado/dont-add-metadata-headers-to-unsupported-multipart-steps
...
Prevent setting custom metadata headers on unsupported multipart upload operations
2024-03-04 17:04:33 +01:00
Alexey Milovidov
de6c5d3a40
Merge pull request #60552 from ClickHouse/pc2
...
Userspace page cache again
2024-03-04 17:47:35 +03:00
beetelbrox
e8f4e4eb77
Fix formatting
2024-03-04 12:54:55 +01:00
beetelbrox
229945cdd2
Fix whitespace
2024-03-04 10:32:45 +01:00
beetelbrox
53e0d01d5e
Do not send metadata headers on unsupported multipartoperations
2024-03-04 10:28:27 +01:00
Alexey Milovidov
da2481c5c8
Merge remote-tracking branch 'origin/master' into remove-old-code-5
2024-03-04 00:12:03 +01:00
Alexey Milovidov
cbf5443585
Remove old code
2024-03-04 00:11:55 +01:00
Alexey Milovidov
d15f7b2158
Fix questionable behavior in the parseDateTimeBestEffort function
2024-03-03 01:42:59 +01:00
Shaun Struwig
b54286bed4
Merge branch 'master' into 56257_parse_crlf_with_TSV_files
2024-03-01 18:21:05 +01:00
Igor Nikonov
a28385966c
Merge pull request #60513 from ClickHouse/cleanup-random-progress-bar
...
Simple cleanup while fixing progress bar
2024-03-01 16:51:46 +01:00
Robert Schulze
3a5c9b627b
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-src
2024-03-01 10:31:55 +00:00
Smita Kulkarni
0f2d47e5a4
Renamed WriteBufferFromS3TaskTracker to ThreadPoolTaskTracker
2024-03-01 10:52:44 +01:00
Kseniia Sumarokova
5ac53c9578
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-01 11:05:47 +08:00
Robert Schulze
4ee1aa8c7c
Fixing more headers
2024-02-29 15:40:30 +00:00
Nikita Taranov
f9d1c5769e
Merge branch 'master' into express_support
2024-02-29 16:26:43 +01:00
Smita Kulkarni
5641fd8ba9
Fix build after merge
2024-02-29 16:13:05 +01:00
Joshua Hildred
58a8658528
Use std::array and std::any_of
2024-02-29 05:20:53 -08:00
Joshua Hildred
fe67293533
Add support for lzma
2024-02-29 05:20:53 -08:00
Joshua Hildred
ca9a6f809f
add support for .tar.zst and tar.xz
2024-02-29 05:20:53 -08:00
Joshua Hildred
e6134bbb7b
Address review comments
2024-02-29 05:20:52 -08:00
josh-hildred
f34ef716f0
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
1ce6256479
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
0fe15f0a22
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
5963b67dff
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
6b32271b88
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
f145b494c0
Update src/IO/Archives/LibArchiveWriter.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
84a3a05eb4
Update src/IO/Archives/LibArchiveReader.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
Joshua Hildred
f7ab6b9e37
Move tar compression extension definitions
2024-02-29 05:20:52 -08:00
Joshua Hildred
c02502b6f1
Removed unused error code
2024-02-29 05:20:52 -08:00
Joshua Hildred
92b446a552
Add support for tar archive compression
2024-02-29 05:20:52 -08:00
Joshua Hildred
c3118ca742
Address review comments
2024-02-29 05:20:52 -08:00
josh-hildred
714fdeb906
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
84814f2a0c
Update src/IO/Archives/LibArchiveWriter.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
f94b5e95b2
Update src/IO/Archives/LibArchiveReader.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
josh-hildred
e085b0fcf0
Update src/IO/Archives/IArchiveWriter.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2024-02-29 05:20:52 -08:00
Joshua Hildred
5fdeebf785
Fix parameter naming
2024-02-29 05:20:51 -08:00
Joshua Hildred
2e2b89f9aa
Fix whitespace
2024-02-29 05:20:51 -08:00
Joshua Hildred
ef7a66b542
Update error messages and error handling
2024-02-29 05:20:51 -08:00
Joshua Hildred
d0776e5d2a
Fixes tidy issue and error formating for test
2024-02-29 05:20:51 -08:00
Joshua Hildred
dc74cebe4e
Fix formating
2024-02-29 05:20:51 -08:00
Joshua Hildred
c53021f55e
Fix formating
2024-02-29 05:20:51 -08:00
Joshua Hildred
48ad506fe3
Cleaned up the uses of the libarchive structs
2024-02-29 05:20:51 -08:00
josh-hildred
4c1ac01e25
Update src/IO/Archives/LibArchiveReader.cpp
...
Co-authored-by: ltrk2 <107155950+ltrk2@users.noreply.github.com>
2024-02-29 05:20:51 -08:00
Joshua Hildred
ac6d74334c
Address review comments
2024-02-29 05:20:51 -08:00
Joshua Hildred
e1fc5c8e48
Add archive unit tests
2024-02-29 05:20:51 -08:00
Joshua Hildred
3c77237dc4
getAllFiles now uses correct StreamInfo
2024-02-29 05:20:51 -08:00
Joshua Hildred
f9c0c99f39
Fix issue with LibArchiveWriter setCompression method
2024-02-29 05:20:51 -08:00
Joshua Hildred
1eb4cdf0f8
Add unit test coverage for ArchiveReaders
2024-02-29 05:20:51 -08:00
Joshua Hildred
eb4ec0e871
Add support for reading and writing backups as a
...
tar archive using libarchive.
2024-02-29 05:20:51 -08:00
Igor Nikonov
a8ee493b67
Merge remote-tracking branch 'origin/master' into cleanup-random-progress-bar
2024-02-29 13:09:20 +00:00
Robert Schulze
ed655815fe
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-src
2024-02-29 12:58:53 +00:00
Smita Kulkarni
3c00d19261
Merge branch 'master' into Azure_write_buffer
2024-02-29 13:39:31 +01:00
Michael Kolupaev
42437a2ae1
Userspace page cache ( #53770 )
...
* Userspace page cache
* Maybe it'll build this time, who knows.
* 'auto' went out of fashion, I guess
* Documentation, tsan workaround, metric 'UnreclaimableRss', disable page cache in the test that uses DatabaseOrdinary
* Moved CachedInMemoryReadBufferFromFile to object store level, changed settings, addressed other comments.
* Fix
* Another fix
* Fix restricted seek, fix ppc64le build
* Don't allow page cache with file cache
* Adjust tests a little
* Fix clang-tidy
* Conflicts
* Comments
* Maybe unbroke AsynchronousBoundedReadBuffer
* SettingsChangesHistory.h
* Fix warning in test
2024-02-29 12:28:32 +00:00
Alexander Tokmakov
15e1191f74
Revert "Userspace page cache ( #53770 )"
...
This reverts commit 7b55c61551
2024-02-29 13:20:17 +01:00
Igor Nikonov
59c3bb768f
Remove header
2024-02-29 12:19:27 +00:00
Michael Kolupaev
7b55c61551
Userspace page cache ( #53770 )
...
* Userspace page cache
* Maybe it'll build this time, who knows.
* 'auto' went out of fashion, I guess
* Documentation, tsan workaround, metric 'UnreclaimableRss', disable page cache in the test that uses DatabaseOrdinary
* Moved CachedInMemoryReadBufferFromFile to object store level, changed settings, addressed other comments.
* Fix
* Another fix
* Fix restricted seek, fix ppc64le build
* Don't allow page cache with file cache
* Adjust tests a little
* Fix clang-tidy
* Conflicts
* Comments
* Maybe unbroke AsynchronousBoundedReadBuffer
* SettingsChangesHistory.h
* Fix warning in test
2024-02-29 11:27:32 +00:00
Nikita Taranov
37917a3ed3
better
2024-02-29 01:42:32 +01:00
Nikita Taranov
974ba7364f
better
2024-02-29 00:55:17 +01:00
Robert Schulze
a1c164a758
Fix a bunch of clang-tidy warnings in headers
2024-02-28 23:43:12 +00:00
Alexey Milovidov
e1e66dc2b8
Merge pull request #60095 from ClickHouse/fix-inconsistent-formatting
...
Fix inconsistent formatting of queries
2024-02-29 00:41:40 +03:00
Raúl Marín
b12245fdc5
Decrease logging level for http retriable errors to Info
2024-02-28 19:47:42 +01:00
Alexey Milovidov
8cf619bc03
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-28 00:30:44 +01:00
Robert Schulze
e185805c1c
Minor: Replace boost::algorithm::ends_with() by std::string::ends_with()
2024-02-27 15:28:02 +00:00
Robert Schulze
466cf93ee2
Merge pull request #60450 from rschu1ze/starts_with
...
Minor: Replace `boost::algorithm::starts_with()` by `std::string::starts_with()`
2024-02-27 16:21:02 +01:00
Kseniia Sumarokova
5771e739f0
Update ReadSettings.h
2024-02-27 23:11:29 +08:00
Robert Schulze
62c12cbfca
Replace boost::algorithm::starts_with() by std::string::starts_with()
2024-02-27 10:42:47 +00:00
Alexey Milovidov
a3f1411679
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-26 19:01:11 +01:00
Antonio Andelic
ffe1872e23
Merge pull request #60203 from ClickHouse/limit_libarchive_format
...
Limit libarchive format to what we use
2024-02-26 09:25:19 +01:00
Alexey Milovidov
bf75d2878b
Merge branch 'master' into fix-inconsistent-formatting
2024-02-26 00:09:02 +01:00
Nikita Taranov
835b47519a
impl
2024-02-25 00:33:50 +01:00
Alexey Milovidov
9e32d0f5b2
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-23 15:05:46 +01:00
kssenii
f1e95fb78b
Add a way to force read-through cache for merges
2024-02-22 15:40:00 +01:00
Nikita Taranov
b44e4c6d78
Merge branch 'master' into fix_head_bucket_after_update
2024-02-22 14:32:00 +01:00
kssenii
f23ddec69f
Fix unit tests build
2024-02-22 11:50:36 +01:00
Smita Kulkarni
3eb1f24770
Merge branch 'master' into Azure_write_buffer
2024-02-22 11:31:47 +01:00
Blargian
e60ead336a
remove readEscapedStringInto function
2024-02-21 18:14:22 +01:00
Blargian
6b5953859e
Check for deserializeTextEscaped in other serializations, fix review changes
2024-02-21 18:00:05 +01:00
Maksim Kita
25bfcdd21f
Fixed code review issues
2024-02-21 11:41:25 +03:00
Maksim Kita
c0e1f50168
Copy S3 file GCP fallback to buffer copy
2024-02-21 11:41:25 +03:00
Alexey Milovidov
71c267e0b0
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-21 07:56:51 +01:00
santrancisco
be33d52016
Fix styling
2024-02-21 17:26:56 +11:00
Nikita Taranov
8401cd8c41
impl
2024-02-21 00:47:54 +01:00
Shaun Struwig
4a5761ce1f
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-02-20 22:18:01 +01:00
kssenii
bdd5eeb2eb
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-02-20 16:59:33 +01:00
Antonio Andelic
34e8e673e9
Merge pull request #60144 from ClickHouse/per-user-s3-settings
...
Support specifying users for S3 settings
2024-02-20 13:20:07 +01:00
Alexey Milovidov
ead2e7bc41
Revert "Restriction for the access key id for s3."
2024-02-20 12:32:25 +01:00
Alexey Milovidov
209acaa8b2
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-inconsistent-formatting
2024-02-20 09:49:05 +01:00
santrancisco
c8a3a0bb11
Replacing libarchive filter_all and format_all code with specific compression method we want to support
2024-02-20 16:48:53 +11:00
kssenii
e23b929e18
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-02-19 19:55:47 +01:00
kssenii
0552f44f70
Fixes after merge with master, move some part of code to object storage
2024-02-19 16:43:49 +01:00
Antonio Andelic
175a1db787
Support specifying users for s3 settings
2024-02-19 16:43:27 +01:00
Vitaly Baranov
f10a8868e1
Merge pull request #60097 from vitlibar/move-thread-pool-callback-runner-to-common
...
Move threadPoolCallbackRunner to the "Common" folder
2024-02-19 14:10:36 +01:00
Kruglov Pavel
5fd2582e83
Merge pull request #59500 from Avogar/exponent-floats-inference
...
Don't infer floats in exponential notation by default
2024-02-19 13:51:07 +01:00
Blargian
d2fb1e4ff8
Add FormSchemaReader
2024-02-19 07:58:22 +01:00
Vitaly Baranov
3323d5ce81
Move threadPoolCallbackRunner to the "Common" folder.
2024-02-17 21:59:51 +01:00
Alexey Milovidov
69dfd721ce
Fix inconsistent formatting of INSERT queries
2024-02-17 10:11:59 +01:00
Alexey Milovidov
2f315e0eb5
Style check for abbreviations
2024-02-17 02:14:15 +01:00
Shaun Struwig
1d440f0399
Merge branch 'ClickHouse:master' into 56257_parse_crlf_with_TSV_files
2024-02-16 19:48:33 +01:00
kssenii
1cccb3ccdf
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-02-16 15:21:10 +01:00
Kruglov Pavel
4d6f167e0c
Merge pull request #59092 from Avogar/auto-format-detection
...
Try to detect file format automatically during schema inference if it's unknown
2024-02-16 14:32:18 +01:00
Smita Kulkarni
62dfa4c65d
Merge branch 'master' into Azure_write_buffer
2024-02-16 11:19:44 +01:00
Alexey Milovidov
bbd7acd7f9
Merge branch 'master' into exponent-floats-inference
2024-02-15 01:46:51 +01:00
Shaun Struwig
525a5188e4
Merge branch 'master' into 56257_parse_crlf_with_TSV_files
2024-02-14 17:40:08 +01:00
Blargian
ba0dc7bc54
fix failing style check and tests
2024-02-14 17:38:11 +01:00
Dmitry Novik
433d835df6
Merge pull request #59920 from azat/http-retries-logs
...
Decrease logging level for http retriable errors to Warning (and fix 00157_cache_dictionary flakiness)
2024-02-14 17:11:28 +01:00
Blargian
6b4d42689c
add SchemaReader for Form format
2024-02-14 17:03:11 +01:00
Blargian
80b2276599
fix style check
2024-02-14 10:38:20 +01:00
Shaun Struwig
27a8bcc438
Update ReadHelpers.cpp to fix failing style check
2024-02-13 20:16:37 +01:00
Kruglov Pavel
1dbfeafb42
Merge branch 'master' into auto-format-detection
2024-02-13 19:08:33 +01:00
Kruglov Pavel
61983a7ec8
Merge branch 'master' into exponent-floats-inference
2024-02-13 19:04:07 +01:00
Smita Kulkarni
d12ecdc5f0
Asynchronous WriteBuffer for AzureBlobStorage
2024-02-13 12:35:17 +01:00
Nikita Mikhaylov
4b85f6b39a
Revert "Less error prone interface of read buffers" ( #59911 )
2024-02-13 12:02:46 +01:00
Azat Khuzhin
4228eaf44a
Decrease logging level for http retriable errors to Warning
...
Messages like this:
2024-02-12 21:25:24 [5c66f008cd40] 2024.02.12 14:25:01.596769 [ 2530 ] {e71fcb6d-356d-4962-95bd-ef8b8c504e11} <Error> ReadWriteBufferFromHTTP: HTTP request to `https://clickhouse-datasets-web.s3.us-east-1.amazonaws.com/store/78e/78ebf6a1-d987-4579-b3ec-00c1a087b1f3/201403_1_1_2/UserAgent.bin ` failed at try 1/10 with bytes read: 311149/378695. Error: DB::HTTPException: Received error from remote server /store/78e/78ebf6a1-d987-4579-b3ec-00c1a087b1f3/201403_1_1_2/UserAgent.bin. HTTP status code: 500 Internal Server Error, body: <?xml version="1.0" encoding="UTF-8"?>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-13 08:31:28 +01:00
Alexey Milovidov
a7c30098b6
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-02-09 22:52:49 +01:00
kssenii
3cca841038
Unite s3/hdfs/azure storage implementations into a single class on top of IObjectStorage
2024-02-09 21:32:04 +01:00
Vitaly Baranov
09e727d48b
Merge pull request #56988 from ClickHouse/Azure_backup
...
Backup & Restore support for AzureBlobStorage
2024-02-09 14:45:35 +01:00
Blargian
0ff4202452
fix up test and modify skipRowEndDelimiter so that crlf works with .tsv
2024-02-08 07:08:17 +01:00
Smita Kulkarni
e594608c42
Merge branch 'master' into Azure_backup
2024-02-07 13:48:46 +01:00
Alexey Milovidov
d65a56d967
Merge branch 'master' into enable-coverage-for-debug-build
2024-02-07 04:58:21 +01:00
Smita Kulkarni
fd2fdcdb4b
Fixed unwanted dereferencing
2024-02-05 11:23:37 +01:00
Smita Kulkarni
f036948f91
Fix clang tidy build
2024-02-05 10:09:48 +01:00
Smita Kulkarni
23b9f43d4f
Fix style
2024-02-04 16:18:17 +01:00
Blargian
ab384f8652
add support_crlf for TSV format
2024-02-04 15:29:57 +01:00
Smita Kulkarni
b0994c5fa7
Addressed comments, added test for named collection
2024-02-04 11:28:20 +01:00
avogar
ff21aa9a19
Don't infer floats in exponential notation by default
2024-02-01 19:47:05 +00:00
Konstantin Bogdanov
5969668503
Concurrent table creation in DatabaseReplicated
2024-02-01 09:51:52 +01:00
Dmitry Novik
5b630ed745
Merge pull request #59399 from azat/build/format-fwd-decl
...
Forward declaration for PeekableReadBuffer
2024-02-01 09:15:07 +01:00
Kruglov Pavel
4987708583
Merge pull request #59298 from Avogar/fix-getting-filename-from-read-buffers
...
Fix getting filename from read buffer wrappers
2024-01-31 14:34:07 +01:00
Azat Khuzhin
010f1c6cb7
Forward declaration for PeekableReadBuffer
...
ReadHelpers.h is very common header and is the root cause of "recompile
everything".
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-30 19:24:19 +01:00
avogar
377937d415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into auto-format-detection
2024-01-29 15:45:18 +00:00
Alexey Milovidov
cb69a1b25d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-01-29 15:35:04 +01:00
Smita Kulkarni
3183026521
Merge branch 'master' into Azure_backup
2024-01-29 11:08:13 +01:00
Smita Kulkarni
a22b68f46f
Added setting azure_max_single_part_copy_size
2024-01-29 10:49:36 +01:00
Alexey Milovidov
4a94ae5a1b
Merge branch 'master' into enable-coverage-for-debug-build
2024-01-28 22:24:26 +01:00
Smita Kulkarni
d264a5a148
Update client & settings to std::shared_ptr<const>
2024-01-28 12:06:52 +01:00
avogar
00c140b7bb
Fix getting filename from read buffer wrappers
2024-01-27 21:13:08 +00:00
avogar
5833641fa5
Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type
2024-01-26 16:54:02 +00:00
Kruglov Pavel
46a6b84a5a
Merge branch 'master' into auto-format-detection
2024-01-25 22:11:07 +01:00
Maksim Kita
2a327107b6
Updated implementation
2024-01-25 14:31:49 +03:00
Smita Kulkarni
09f1e2840c
Simplified calculatePartSize and upload task
2024-01-25 10:06:05 +01:00
Smita Kulkarni
f551081dd4
Addressed review comments
2024-01-24 21:10:50 +01:00
Kruglov Pavel
6c0615d493
Merge pull request #59099 from skyoct/fix/fix_mult_redirect
...
Disable "Too many redirects" error retry
2024-01-24 12:22:01 +01:00
Kruglov Pavel
718064bb43
Minor change to restart CI
2024-01-24 12:19:10 +01:00
Smita Kulkarni
4d92eba316
Merge branch 'master' into Azure_backup
2024-01-24 11:13:57 +01:00
skyoct
95829e074e
Opt code
2024-01-24 03:49:21 +00:00
avogar
617cc514b7
Try to detect file format automatically during schema inference if it's unknown
2024-01-23 18:59:39 +00:00
Anton Popov
4292ee7d51
Merge pull request #58886 from CurtizJ/better-read-buffers
...
Less error prone interface of read buffers
2024-01-23 14:59:27 +01:00
Alexander Gololobov
9a0e5d0aac
Merge pull request #58914 from ClickHouse/ot_spans_in_insert
...
Opentelemetry spans to analyze CPU and S3 bottlenecks on inserts
2024-01-23 13:53:13 +01:00
Smita Kulkarni
2e7ce5b0e2
Updated settings ptr and fetching of client from Disk & ObjectStorage
2024-01-22 16:24:43 +01:00
Anton Popov
e1e7a9cd45
Merge branch 'master' into better-read-buffers
2024-01-22 15:50:48 +01:00
Nikita Taranov
4d6ef8ed86
Merge pull request #56961 from nickitat/some_questionable_code
...
Bypass `Poco::BasicBufferedStreamBuf` in `ReadBufferFromIStream`
2024-01-22 14:08:09 +01:00
Sema Checherinda
9adab57771
Correct initialization of ConnectionTimeouts ( #59000 )
2024-01-22 12:03:36 +01:00
skyoct
fdf8008804
Fix redirect retry
2024-01-22 10:27:35 +00:00
Vitaly Baranov
9377f4276b
Merge pull request #58962 from vitlibar/fix-memory-dealloc-in-copy-data-to-s3
...
Fix memory management in copyDataToS3File
2024-01-20 23:35:42 +01:00
Alexey Milovidov
dc1dac3ba4
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-01-20 21:41:07 +01:00
Alexey Milovidov
735708e9bb
Merge branch 'master' into enable-coverage-for-debug-build
2024-01-20 21:40:53 +01:00
Sema Checherinda
1bbb5f7f51
Merge pull request #59015 from ClickHouse/chesema-adaptive-timeouts-gcp
...
preserve 'amz-sdk-invocation-id' and 'amz-sdk-request' headers with gcp
2024-01-20 17:48:20 +01:00
serxa
172d68ceca
Merge branch 'master' into cpu-scheduler
2024-01-20 12:21:21 +00:00
serxa
824deab761
split resource scheduler off IO/ into Common/Scheduler/
2024-01-20 12:06:26 +00:00
Alexey Milovidov
8acc981e8c
Merge pull request #59008 from ClickHouse/fix-log-level-for-http
...
Update log level for http buffer
2024-01-20 04:02:42 +01:00
Kruglov Pavel
20379d34d5
Merge branch 'master' into variant-data-type
2024-01-19 23:17:42 +01:00
Sema Checherinda
539451bbe3
preserve 'amz-sdk-invocation-id' and 'amz-sdk-request' headers with gcp
2024-01-19 21:07:47 +01:00
kssenii
4c8ced6af8
Fix log level
2024-01-19 18:04:58 +01:00
Alexey Milovidov
6a75641048
Merge pull request #58966 from ClickHouse/arseek
...
Fix seek in ReadBufferFromZipArchive
2024-01-19 18:04:03 +01:00
Vitaly Baranov
19bc0b744f
Add comments and const after review.
2024-01-19 12:54:55 +01:00
Alexey Milovidov
dcdbaf9150
Merge pull request #58900 from MikhailBurdukov/restriction_for_s3_access_key_id
...
Restriction for the access key id for s3.
2024-01-19 04:13:17 +01:00
Alexey Milovidov
370815803e
Update Client.cpp
2024-01-19 06:12:44 +03:00
Alexey Milovidov
7c0b167e2d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into enable-coverage-for-debug-build
2024-01-19 03:59:27 +01:00
Michael Kolupaev
3a19ea6e6c
Fix seek in ReadBufferFromZipArchive
2024-01-19 01:46:07 +00:00
Vitaly Baranov
24ca8fbbec
Improve logs.
2024-01-18 21:15:39 +01:00
Vitaly Baranov
0caafe4810
Fix memory deallocation in copyDataToS3File()
...
- ReadBufferFromS3 is created now only when it's necessary and destroyed after that.
2024-01-18 20:48:56 +01:00
Nikita Taranov
b5e333df96
remove test
2024-01-18 19:56:02 +01:00
Kruglov Pavel
5444cde408
Merge branch 'master' into variant-data-type
2024-01-18 18:31:27 +01:00
MikhailBurdukov
b3d74fc907
style
2024-01-18 15:39:25 +00:00
MikhailBurdukov
eac310fcb2
Review
2024-01-18 15:19:11 +00:00
Smita Kulkarni
e71977b724
Merge branch 'master' into Azure_backup
2024-01-18 08:29:01 +01:00
Yakov Olkhovskiy
490bb809c3
Merge pull request #58846 from ClickHouse/fix-compress-destructor
...
Fix HTTP compressors finalization
2024-01-17 17:49:12 -05:00
Alexey Milovidov
c098f41dbc
Merge branch 'master' into enable-coverage-for-debug-build
2024-01-17 22:35:51 +01:00
Alexey Milovidov
f499cf3fd5
Update Client.cpp
2024-01-17 22:30:18 +03:00
Alexander Gololobov
0a42dbafe8
Add attributes
2024-01-17 17:46:15 +01:00
Nikita Taranov
1db805dfa4
impl
2024-01-17 16:24:24 +01:00
Alexander Gololobov
7dda18c209
Spans in mergeBlock and finalizeImpl
2024-01-17 16:07:53 +01:00
Anton Popov
245e1cbd5c
Merge remote-tracking branch 'upstream/master' into better-read-buffers
2024-01-17 13:46:48 +00:00
Nikita Taranov
5485d2a2ec
impl
2024-01-17 14:38:06 +01:00
Nikita Taranov
bbf070caa6
impl
2024-01-17 14:08:02 +01:00
Yakov Olkhovskiy
d5d060a17e
fix memory leak in bzip2
2024-01-17 12:53:37 +00:00
Kruglov Pavel
6d064512e1
Merge pull request #58614 from Blargian/58363_disable_ansi_pretty_automatically
...
58363 Automatically disable ANSI escape sequences in Pretty formats if the output is not a terminal
2024-01-17 13:45:41 +01:00
Nikita Taranov
bb124b544c
Merge pull request #58077 from nickitat/revert_rest
...
Remove unneeded parameter `use_external_buffer` from `AsynchronousReadBuffer*`
2024-01-17 13:45:07 +01:00
MikhailBurdukov
e2729ed4c9
Restriction for the access key id for s3.
2024-01-17 10:43:58 +00:00
Yakov Olkhovskiy
e6b3f69137
style fix
2024-01-17 02:58:18 +00:00
Yakov Olkhovskiy
e06e4695e1
refactoring bzip2 buffer and fix possible data loss in internal buffer
2024-01-17 01:55:38 +00:00
Anton Popov
299293c088
better interface in read buffers
2024-01-16 23:26:20 +00:00
Yakov Olkhovskiy
92d42567fa
separate explicit finalization for buffers
2024-01-16 22:15:35 +00:00
Yakov Olkhovskiy
fcd4297d20
fix compressors finalization
2024-01-16 00:22:17 +00:00
xleoken
6f7e7c95ba
Ecloud EOS Storage support
2024-01-15 11:07:43 -05:00
Alexey Milovidov
e4cd02ea39
Fix typo
2024-01-15 00:36:24 +01:00
Alexey Milovidov
d112492c56
Remove some code
2024-01-13 03:48:04 +01:00
Blargian
aa8876a611
#58363 - Changes based on review of draft PR - changed output_format_pretty_color to use UInt64Auto. Added isWritingToTerminal function to IO/WriteHelpers.h and updated test
2024-01-12 12:31:57 +02:00
Ilya Yatsishin
904c02d955
Fix
2024-01-12 12:43:11 +04:00
Alexey Milovidov
df04476037
Update src/IO/TimeoutSetter.cpp
...
Co-authored-by: Konstantin Bogdanov <thevar1able@users.noreply.github.com>
2024-01-12 07:41:24 +03:00
Alexey Milovidov
6c604ea1a7
Fix Mac OS X
2024-01-12 04:00:36 +01:00
Yakov Olkhovskiy
4fc11e189c
Merge branch 'master' into revert-58450-revert-56064-feature-server-iface-metrics
2024-01-10 12:03:47 -05:00
Yakov Olkhovskiy
45d5e29ba0
master merge adjustments
2024-01-10 16:55:09 +00:00
Robert Schulze
52d75ab68f
Merge pull request #58458 from rschu1ze/replace-std_regexp-by-re2
...
Replace `std::regex` by re2
2024-01-10 17:46:54 +01:00
Vitaly Baranov
dbb19e72b3
Merge pull request #58629 from vitlibar/remove-finalize-from-destructor-of-write-buffer-from-encrypted-file
...
Remove finalize() from ~WriteBufferFromEncryptedFile
2024-01-10 16:36:11 +01:00
Yakov Olkhovskiy
2e124b19b8
Merge branch 'master' into revert-58450-revert-56064-feature-server-iface-metrics
2024-01-10 09:05:18 -05:00
Robert Schulze
499227b9cf
Merge remote-tracking branch 'rschu1ze/master' into replace-std_regexp-by-re2
2024-01-10 10:02:53 +00:00
pufit
6cf55b82f4
Merge pull request #58539 from canhld94/file_custom_compress_level
...
Allow explicitly set compression level in output format
2024-01-09 13:43:38 -05:00
Vitaly Baranov
01dfe282ea
Remove finalize() from ~WriteBufferFromEncryptedFile.
2024-01-09 17:22:17 +01:00
Sema Checherinda
7c7e72c4b7
Merge pull request #58573 from ClickHouse/chesema-s3-client-creation
...
fix and test that S3Clients are reused
2024-01-09 13:28:20 +01:00
Kruglov Pavel
b947609b8e
Merge branch 'master' into variant-data-type
2024-01-08 15:04:51 +01:00
Robert Schulze
f553b55e3a
Merge remote-tracking branch 'rschu1ze/master' into regex-std-re2
2024-01-07 22:31:35 +00:00
Robert Schulze
8e804487f3
Some fixups
2024-01-07 22:28:08 +00:00
Sema Checherinda
6f626d8294
fix auth_settings.hasUpdates function
2024-01-07 23:00:26 +01:00
Smita Kulkarni
fd92c1961e
Fix clang tidy build
2024-01-07 16:33:48 +01:00
Yakov Olkhovskiy
45ac4d2b67
Merge remote-tracking branch 'origin/master' into revert-58450-revert-56064-feature-server-iface-metrics
2024-01-07 15:23:46 +00:00
Sema Checherinda
d5f86f671d
fix and test that S3Clients reused
2024-01-07 02:19:06 +01:00
avogar
7e5ba62017
Allow to read Bool values into String in JSON input formats
2024-01-05 20:33:30 +00:00
Smita Kulkarni
b250acff78
Fixed style check
2024-01-05 11:26:32 +01:00
Duc Canh Le
29c4d96b48
fix build and add a tests
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-05 06:54:56 +00:00
Smita Kulkarni
c14605caa7
Added flag use_native_copy and updated to use StartCopyFromUri for native copy with large files
2024-01-04 18:27:54 +01:00
Smita Kulkarni
91bad5bc39
Updated to use MultiVersion for BlobContainerClient in Backups and updated to get client from disk
2024-01-04 16:06:36 +01:00
Duc Canh Le
2e14cfb526
add settings for output compression level and window size
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-04 08:16:00 +00:00
Yakov Olkhovskiy
85f03478ef
Revert "Revert "Use CH Buffer for HTTP out stream, add metrics for interfaces""
2024-01-03 11:47:15 -05:00
Kruglov Pavel
4d8cf71ba7
Merge branch 'master' into variant-data-type
2024-01-03 15:21:23 +01:00
Kruglov Pavel
570d1c013b
Merge pull request #57053 from Avogar/better-parsing-exceptions
...
Better exception messages in input formats
2024-01-03 14:47:51 +01:00
Smita Kulkarni
df221f7db6
Renamed Bucket-Key to Container-Blob
2024-01-03 11:35:06 +01:00
Raúl Marín
d491758939
Revert "Use CH Buffer for HTTP out stream, add metrics for interfaces"
2024-01-03 10:42:15 +01:00
Smita Kulkarni
4122de9721
Updated tests and added settings
2024-01-02 20:19:01 +01:00
Yakov Olkhovskiy
5633fb8145
Merge pull request #56064 from ClickHouse/feature-server-iface-metrics
...
Use CH Buffer for HTTP out stream, add metrics for interfaces
2024-01-02 10:17:52 -05:00
Kruglov Pavel
e69bda7f08
Merge pull request #57438 from Avogar/fix-resetting-buffers
...
Fix working with read buffers in StreamingFormatExecutor
2024-01-02 15:34:35 +01:00
Alexey Milovidov
2f3e17c737
Remove commented code
2023-12-30 18:39:12 +01:00
Alexey Milovidov
a1bd76388e
Remove commented code
2023-12-30 18:38:41 +01:00
Alexey Milovidov
aad63c8273
Merge branch 'master' into feature-server-iface-metrics
2023-12-30 15:44:42 +01:00
Alexey Milovidov
8365722c95
Merge pull request #58336 from amosbird/macro2constexpr
...
Organize symbols from src/* into DB namespace
2023-12-30 12:40:36 +01:00
Amos Bird
907e844635
Fix examples
2023-12-30 10:48:19 +08:00
Alexey Milovidov
aa462fa7cd
Merge branch 'master' into feature-server-iface-metrics
2023-12-29 22:37:52 +01:00
Amos Bird
18395a387c
Fix another unitest
2023-12-30 01:14:21 +08:00
Kruglov Pavel
c03e36e012
Merge branch 'master' into better-parsing-exceptions
2023-12-29 18:07:32 +01:00
Azat Khuzhin
8c54380d80
Avoid sending ComposeObject requests after upload to GCS
...
This should not be required anymore, but leave it as an option, since
likely this is required for old files.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Azat Khuzhin
f4a7789cd4
Convert various S3::Client settings into separate ClientSettings struct
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Azat Khuzhin
2b2d10689b
Make S3::Client::ComposeObject private
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
avogar
c6085b3eb9
Fix working with read buffers in StreamingFormatExecutor
2023-12-28 18:47:57 +00:00
Smita Kulkarni
bf60a06a11
Merge branch 'master' into Azure_backup
2023-12-28 11:51:49 +01:00
Azat Khuzhin
c6774cfe42
Add missing includes for ZipArchiveWriter
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-27 17:06:17 +01:00
Azat Khuzhin
cf1dae5321
Add missing includes into BitHelpers.h
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-27 17:06:17 +01:00
Azat Khuzhin
b9233f6d4f
Move Allocator code into module part
...
This should reduce amount of code that should be recompiled on
Exception.h changes (and everything else that had been included there).
This will actually not help a lot, because it is also included into
PODArray.h and ThreadPool.h at least... Sigh.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-27 15:42:08 +01:00
Smita Kulkarni
d46d914521
Updated thread name
2023-12-27 10:28:52 +01:00
Yarik Briukhovetskyi
41c275274b
Merge pull request #58148 from yariks5s/fix_s3_regions
...
S3-links region independency
2023-12-26 12:59:00 +01:00
