Smita Kulkarni
c7967fb721
Add an option to INTO OUTFILE to also print details to standard output.
...
Implementation:
- Added a bool to ASTQueryWithOutput & patched the usage in ClientBase.
- Added a new buffer TeeWriteBuffer which extends from WriteBufferFromFile (used to write data to the file) and has WriteBufferFromFileDescriptor (used to write data to stdout). The WriteBufferFromFileDescriptor uses the same buffer as TeeWriteBuffer.
- Added a new bool select_into_outfile_and_stdout in ClientBase to enable/disable progress rendering.
Testing:
- Added a test tests/queries/0_stateless/02346_into_outfile_and_stdout.sh
Documentation:
- Updated the english documentation for the new option in SELECT.
2022-07-09 10:12:46 +02:00
Nikolay Degterinsky
0c783800c8
Better
2022-07-07 01:47:33 +00:00
Kseniia Sumarokova
b3fd8e3bd6
Merge pull request #38802 from kssenii/fix-s3-seekable-with-parallel
...
Fix s3 seekable reads with parallel read buffer
2022-07-05 11:23:16 +02:00
kssenii
f5bf744028
Fix
2022-07-04 16:05:08 +02:00
kssenii
a772a09149
Fix
2022-07-04 00:14:45 +02:00
Robert Schulze
bb441faa42
Merge pull request #38491 from ClickHouse/clang-tidy-in-branches
...
Fix some clang-tidy warnings in headers
2022-06-29 10:22:04 +02:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to
...
Replace where possible by std::lock_guard which is more light-weight.
2022-06-28 19:19:06 +00:00
Robert Schulze
3e528c0385
Fix duplicate include
2022-06-28 08:13:51 +00:00
Robert Schulze
5e28580f11
Fix some clang-tidy warnings in headers
2022-06-27 23:49:07 +02:00
Alexander Tokmakov
74f38710a8
Merge branch 'master' into cleanup_garbage_in_store_dir
2022-06-23 21:43:28 +02:00
alesapin
0f8eed98c4
Merge branch 'master' into better-support-gcp
2022-06-22 12:46:01 +02:00
Alexander Tokmakov
ba0fcec993
add background cleanup of store/ subdirs
2022-06-21 12:35:47 +02:00
Kseniia Sumarokova
2d039b739c
Update ReadBufferFromFileDescriptor.cpp
2022-06-20 19:13:20 +02:00
kssenii
cea81822bd
Fix
2022-06-20 17:19:36 +02:00
kssenii
5dd1bb2fd8
improvements for getFileSize
2022-06-20 15:22:56 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6
...
Backup Improvements 6
2022-06-20 14:13:30 +02:00
Alexey Milovidov
7700c26076
Merge pull request #38196 from ClickHouse/revert-38194-revert-37015-zstd_window_log_max
...
Revert "Revert "Add a setting to use more memory for zstd decompression""
2022-06-19 19:58:03 +03:00
Alexey Milovidov
04b7a34870
Maybe fix error
2022-06-19 05:21:59 +02:00
Alexey Milovidov
73709b0488
Revert "Revert "Add a setting to use more memory for zstd decompression""
2022-06-18 15:55:35 +03:00
alesapin
16e8b85fbf
Revert "Add a setting to use more memory for zstd decompression"
2022-06-18 14:08:14 +02:00
Alexey Milovidov
e20259e9ca
Merge pull request #37015 from wuxiaobai24/zstd_window_log_max
...
Add a setting to use more memory for zstd decompression
2022-06-18 04:19:27 +03:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache
...
Extract some diff from pr #36171
2022-06-17 12:07:31 +02:00
mergify[bot]
296905771f
Merge branch 'master' into better-support-gcp
2022-06-17 01:19:11 +00:00
kssenii
797482f132
Better
2022-06-16 16:40:29 +02:00
mergify[bot]
7f24574609
Merge branch 'master' into better-support-gcp
2022-06-15 23:18:20 +00:00
Vitaly Baranov
e891eba80e
Finalize write buffers used in backups.
2022-06-15 20:26:27 +02:00
Daniel Kutenin
a769dea8ef
Merge branch 'ClickHouse:master' into master
2022-06-15 19:12:10 +01:00
Danila Kutenin
048f56bf4d
Fix some tests and comments
2022-06-15 14:40:21 +00:00
kssenii
500f49972b
Extract diff from PR 36171
2022-06-15 16:40:18 +02:00
Danila Kutenin
08e3f77a9c
Optimize most important parts with NEON SIMD
...
First part, updated most UTF8, hashing, memory and codecs. Except
utf8lower and upper, maybe a little later.
That includes huge amount of research with movemask dealing. Exact
details and blog post TBD.
2022-06-15 13:19:29 +00:00
mergify[bot]
12bd2c0e3f
Merge branch 'master' into dt64-neg-subseconds
2022-06-15 02:24:17 +00:00
Anton Popov
a59be0fd5d
better support of GCP storage
2022-06-15 00:23:45 +00:00
alesapin
eae10aab6d
Merge pull request #37964 from excitoon-favorites/betterurlhandlingforversionid
...
Better parsing of `versionId` in `S3::URI::URI`
2022-06-12 13:12:02 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size
...
Support to set max cache size for per query in local cache
2022-06-10 16:17:18 +02:00
Kseniia Sumarokova
0ae2168fb6
Merge pull request #36328 from bigo-sg/async_hdfs_read_buffer
...
Apply read_method 'threadpool' for StorageHive
2022-06-10 15:04:21 +02:00
Robert Schulze
5f5732a2c4
Merge pull request #37969 from ClickHouse/consistent-macro-usage
...
More consistent use of platform macros
2022-06-10 14:10:01 +02:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros
...
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.
__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
2022-06-10 10:22:31 +02:00
Vladimir Chebotarev
512f16f5fa
Better parsing of versionId in S3::URI::URI.
2022-06-10 10:06:31 +03:00
Vladimir Chebotarev
3b344a3d26
Got rid of S3AuthSigner ( #37769 )
2022-06-09 14:44:50 +02:00
KinderRiven
6883ecda96
fix
2022-06-08 14:19:50 +08:00
Azat Khuzhin
71285edfbd
Fix reading of empty S3 files
...
It is possible for ReadBufferFromS3::nextImpl() called even after eof(),
at least once, and in this case, if the file was empty, then local
working_buffer will be null, while impl.working_buffer will be empty,
but not null, and so local position() after impl->position() =
position() will be incorrect.
I found this with test_storage_s3/test.py::test_empty_file in debug
build, assertion catched this, so maybe it worth get back debug
integration build...
v2: fix test_log_family_s3 failures
https://s3.amazonaws.com/clickhouse-test-reports/37801/b5e6e2ddae94d6a7eac551309cb67003dff97df1/integration_tests__asan__actions__[2/3].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 20:41:26 +03:00
taiyang-li
73a484256e
Merge branch 'master' into async_hdfs_read_buffer
2022-06-07 12:16:46 +08:00
KinderRiven
3b2a427002
fix
2022-06-07 11:50:56 +08:00
taiyang-li
b36d9f8143
refactor readinto
2022-06-06 12:58:22 +08:00
KinderRiven
13e74bef5e
support skip_download_if_exceeds_query_cache
2022-06-06 02:36:23 +08:00
KinderRiven
3ffb0b3549
support_max_query_cache_size
2022-06-05 16:21:36 +08:00
Alexander Tokmakov
084dd6f3c2
Merge pull request #37767 from excitoon-favorites/removeperrequestconfigurationfromawssdkcpp
...
Moved `ClientConfigurationPerRequest` to ClickHouse
2022-06-03 17:42:50 +03:00
zvonand
e6498a3235
update comment
2022-06-03 14:52:34 +03:00
taiyang-li
f202c35311
Merge branch 'master' into async_hdfs_read_buffer
2022-06-03 17:52:09 +08:00
Kruglov Pavel
37a4d11c64
Merge pull request #37645 from azat/fix-write
...
Fix possible "Cannot write to finalized buffer"
2022-06-02 14:01:30 +02:00
Antonio Andelic
46abbac352
Merge pull request #37726 from ClickHouse/fix-parallel-download-exception
...
Initialize `ParallelReadBuffer` after construction
2022-06-02 12:35:10 +02:00
Vladimir Chebotarev
a857bc2ccf
Update S3Common.cpp
2022-06-02 08:46:41 +03:00
Vladimir Chebotarev
5fcf840156
Typo.
2022-06-02 08:43:44 +03:00
Vladimir Chebotarev
d5022a0c01
Moved ClientConfigurationPerRequest from ClickHouse/aws-sdk-cpp#1 and ClickHouse/aws-sdk-cpp#2 to ClickHouse.
2022-06-02 06:07:01 +03:00
Azat Khuzhin
62d78d8f20
Fix WriteBufferFromS3 is_finalized check in case of exception
...
WriteBufferFromS3::is_finalized is not set if finalizeImpl() throws,
while WriteBuffer::finalized correctly set even in case of exception, so
it should be used instead.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-01 21:50:30 +03:00
Antonio Andelic
08c20be4d0
Cleaner exception handling in ParallelReadBuffer
2022-06-01 11:51:01 +00:00
Antonio Andelic
f49dd19e7a
Revert "Initialize ParallelReadBuffer after construction"
...
This reverts commit 31e1e67836
2022-06-01 11:43:58 +00:00
Antonio Andelic
31e1e67836
Initialize ParallelReadBuffer after construction
2022-06-01 06:25:32 +00:00
zvonand
8e28ba2583
fix DT64 reader
2022-05-31 18:15:51 +03:00
taiyang-li
dbb8a09825
merge master and solve conflict
2022-05-30 10:47:04 +08:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage
...
Separate object storage operations from disks
2022-05-27 13:59:50 +02:00
Alexey Milovidov
8ba865bb60
Merge pull request #37344 from excitoon-favorites/fixs3colonandequalssign
...
Fixed error with symbols in key name in S3
2022-05-27 00:58:35 +03:00
alesapin
6f5c86e55e
Merge branch 'master' into i_object_storage
2022-05-25 20:49:01 +02:00
taiyang-li
1d9f65a7d4
Merge branch 'master' into async_hdfs_read_buffer
2022-05-25 17:10:22 +08:00
Alexey Milovidov
516fba27dc
Merge branch 'master' into allow-setuid-inside-clickhouse
2022-05-24 23:31:14 +02:00
Kruglov Pavel
6c9a524f6b
Merge pull request #37192 from Avogar/formats-with-names
...
Improve performance and memory usage for select of subset of columns for some formats
2022-05-24 13:28:14 +02:00
李扬
e33cfc889c
Fix bug of datetime64 parsed from string '1969-12-31 23:59:59.123' ( #37039 )
2022-05-24 10:47:17 +02:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage
2022-05-23 20:29:49 +00:00
alesapin
1e1e6d4fa0
Review fixes
2022-05-23 16:24:06 +02:00
alesapin
216184dfd3
Rename one more file
2022-05-23 13:20:32 +02:00
taiyang-li
14f84f02d5
Merge branch 'master' into async_hdfs_read_buffer
2022-05-23 18:36:21 +08:00
Kruglov Pavel
f539fb835d
Merge branch 'master' into formats-with-names
2022-05-23 12:14:20 +02:00
Alexey Milovidov
c541d30a38
Fix error
2022-05-23 02:48:05 +02:00
Alexey Milovidov
cc985d9b92
Fix error
2022-05-23 02:30:33 +02:00
avogar
a4cf07708c
Fix comments
2022-05-20 14:57:27 +00:00
avogar
566d1b15fd
Merge branch 'master' of github.com:ClickHouse/ClickHouse into formats-with-names
2022-05-20 13:54:52 +00:00
avogar
a6a430c5ee
Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-columns-formats
2022-05-20 11:08:30 +00:00
alesapin
654b27e307
Merge branch 'master' into i_object_storage
2022-05-20 11:56:13 +02:00
Vladimir Chebotarev
cdbe6672cb
Added one more symbol and proper comment.
2022-05-20 11:35:18 +03:00
alesapin
c87c3fcfd9
Move Azure blob storage
2022-05-19 19:25:45 +02:00
Vladimir Chebotarev
1ad19051e5
Fixed error with symbols in key name in S3.
2022-05-19 11:01:05 +03:00
Nikolai Kochetov
64ecb3941c
Merge pull request #37259 from ClickHouse/clangtidies2
...
Activate more clangtidies
2022-05-18 13:01:40 +02:00
alesapin
e99c90118e
Merge branch 'master' into i_object_storage
2022-05-18 12:36:22 +02:00
Kseniia Sumarokova
38c87ab9cd
Merge pull request #36493 from kssenii/fix-encrypted-buffer
...
Fix offset update ReadBufferFromEncryptedFile
2022-05-18 09:46:17 +02:00
Robert Schulze
0c55ac76d2
A few clangtidy updates
...
Enable:
- bugprone-lambda-function-name: "Checks for attempts to get the name of
a function from within a lambda expression. The name of a lambda is
always something like operator(), which is almost never what was
intended."
- bugprone-unhandled-self-assignment: "Finds user-defined copy
assignment operators which do not protect the code against
self-assignment either by checking self-assignment explicitly or using
the copy-and-swap or the copy-and-move method.""
- hicpp-invalid-access-moved: "Warns if an object is used after it has
been moved."
- hicpp-use-noexcept: "This check replaces deprecated dynamic exception
specifications with the appropriate noexcept specification (introduced
in C++11)"
- hicpp-use-override: "Adds override (introduced in C++11) to overridden
virtual functions and removes virtual from those functions as it is
not required."
- performance-type-promotion-in-math-fn: "Finds calls to C math library
functions (from math.h or, in C++, cmath) with implicit float to
double promotions."
Split up:
- cppcoreguidelines-*. Some of them may be useful (haven't checked in
detail), therefore allow to toggle them individually.
Disable:
- linuxkernel-*. Obvious.
2022-05-17 20:56:57 +02:00
kssenii
b53ca506e6
Add unit test
2022-05-17 14:55:42 +02:00
alesapin
4d4634b12d
Merge pull request #37164 from kssenii/ficx-async-reads-assertions
...
fix async reads from remote fs internal setting not being always turned on
2022-05-17 11:29:14 +02:00
taiyang-li
14ab7eb5a3
merge master and solve conflict
2022-05-17 16:28:08 +08:00
alesapin
ceb7249916
Fix stupid buge
2022-05-16 19:57:48 +02:00
alesapin
ce6d5f7dcc
Merge branch 'master' into i_object_storage
2022-05-16 17:42:22 +02:00
kssenii
f7493965b7
Remove additional check for merge tree
2022-05-16 11:19:57 +02:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies
2022-05-16 10:12:50 +02:00
alesapin
4f1d4dde22
Disable s3 logging by default
2022-05-16 00:16:00 +02:00
Kseniia Sumarokova
f4a8a0cefe
Merge pull request #37200 from kssenii/add-s3-requests-metric
...
Add S3Requests metric
2022-05-14 10:08:21 +02:00
alesapin
cc304a3178
Merge branch 'master' into i_object_storage
2022-05-13 23:44:00 +02:00
avogar
68bb07d166
Better naming
2022-05-13 18:39:19 +00:00
kssenii
5dd3ad08a1
Add S3Requests metric
2022-05-13 18:12:46 +02:00
Kruglov Pavel
4549d2f40f
Fix build
2022-05-13 17:41:26 +02:00
kssenii
d8ec49d9b3
Fix
2022-05-13 17:17:13 +02:00
avogar
febabb2787
Fix build
2022-05-13 15:08:02 +00:00
