thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
98,159 Commits 9,319 Branches 1,783 Tags 2.1 GiB
021b882455
Commit Graph

2106 Commits

Author SHA1 Message Date
Smita Kulkarni
c7967fb721 Add an option to INTO OUTFILE to also print details to standard output. 
Implementation:
- Added a bool to ASTQueryWithOutput & patched the usage in ClientBase.
- Added a new buffer TeeWriteBuffer which extends from WriteBufferFromFile (used to write data to the file) and has WriteBufferFromFileDescriptor (used to write data to stdout). The WriteBufferFromFileDescriptor uses the same buffer as TeeWriteBuffer.
- Added a new bool select_into_outfile_and_stdout in ClientBase to enable/disable progress rendering.

Testing:
 - Added a test tests/queries/0_stateless/02346_into_outfile_and_stdout.sh

Documentation:
- Updated the english documentation for the new option in SELECT.
 2022-07-09 10:12:46 +02:00
Nikolay Degterinsky
0c783800c8 Better 2022-07-07 01:47:33 +00:00
kssenii
e55169ebad Fix 2022-07-05 20:15:35 +02:00
Kseniia Sumarokova
b3fd8e3bd6
Merge pull request #38802 from kssenii/fix-s3-seekable-with-parallel 
Fix s3 seekable reads with parallel read buffer
 2022-07-05 11:23:16 +02:00
kssenii
b52084265c Merge master 2022-07-04 21:37:43 +02:00
kssenii
f5bf744028 Fix 2022-07-04 16:05:08 +02:00
kssenii
a772a09149 Fix 2022-07-04 00:14:45 +02:00
Robert Schulze
bb441faa42
Merge pull request #38491 from ClickHouse/clang-tidy-in-branches 
Fix some clang-tidy warnings in headers
 2022-06-29 10:22:04 +02:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to 
Replace where possible by std::lock_guard which is more light-weight.
 2022-06-28 19:19:06 +00:00
Robert Schulze
3e528c0385
Fix duplicate include 2022-06-28 08:13:51 +00:00
Robert Schulze
5e28580f11
Fix some clang-tidy warnings in headers 2022-06-27 23:49:07 +02:00
Kruglov Pavel
86e8f31ad4
Merge branch 'master' into schema-inference-cache 2022-06-24 16:10:25 +02:00
Alexander Tokmakov
cfe0e434e5
Revert "Add support for io_uring read method" 2022-06-24 12:19:41 +03:00
Alexey Milovidov
812ab9bd6b
Merge pull request #36103 from sauliusvl/uring 
Add support for io_uring read method
 2022-06-24 00:34:29 +03:00
Alexander Tokmakov
74f38710a8 Merge branch 'master' into cleanup_garbage_in_store_dir 2022-06-23 21:43:28 +02:00
kssenii
f7b329ee57 Merge master 2022-06-23 14:56:48 +02:00
alesapin
0f8eed98c4
Merge branch 'master' into better-support-gcp 2022-06-22 12:46:01 +02:00
kssenii
4178abc0a5 Merge master 2022-06-21 22:38:51 +02:00
avogar
c14364e3d9 Check last modification time for URL function too 2022-06-21 17:18:14 +00:00
avogar
d37ad2e6de Implement cache for schema inference for file/s3/hdfs/url 2022-06-21 13:02:48 +00:00
Alexander Tokmakov
ba0fcec993 add background cleanup of store/ subdirs 2022-06-21 12:35:47 +02:00
Saulius Valatka
0dc7854af1 add initial io_uring support 2022-06-21 09:59:46 +03:00
Kseniia Sumarokova
2d039b739c
Update ReadBufferFromFileDescriptor.cpp 2022-06-20 19:13:20 +02:00
kssenii
cea81822bd Fix 2022-06-20 17:19:36 +02:00
kssenii
5dd1bb2fd8 improvements for getFileSize 2022-06-20 15:22:56 +02:00
kssenii
90be49faa4 Merge master 2022-06-20 14:26:56 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6 
Backup Improvements 6
 2022-06-20 14:13:30 +02:00
Alexey Milovidov
7700c26076
Merge pull request #38196 from ClickHouse/revert-38194-revert-37015-zstd_window_log_max 
Revert "Revert "Add a setting to use more memory for zstd decompression""
 2022-06-19 19:58:03 +03:00
Alexey Milovidov
04b7a34870 Maybe fix error 2022-06-19 05:21:59 +02:00
Alexey Milovidov
73709b0488
Revert "Revert "Add a setting to use more memory for zstd decompression"" 2022-06-18 15:55:35 +03:00
alesapin
16e8b85fbf
Revert "Add a setting to use more memory for zstd decompression" 2022-06-18 14:08:14 +02:00
Alexey Milovidov
e20259e9ca
Merge pull request #37015 from wuxiaobai24/zstd_window_log_max 
Add a setting to use more memory for zstd decompression
 2022-06-18 04:19:27 +03:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache 
Extract some diff from pr #36171
 2022-06-17 12:07:31 +02:00
mergify[bot]
296905771f
Merge branch 'master' into better-support-gcp 2022-06-17 01:19:11 +00:00
kssenii
797482f132 Better 2022-06-16 16:40:29 +02:00
mergify[bot]
7f24574609
Merge branch 'master' into better-support-gcp 2022-06-15 23:18:20 +00:00
Vitaly Baranov
e891eba80e Finalize write buffers used in backups. 2022-06-15 20:26:27 +02:00
Daniel Kutenin
a769dea8ef
Merge branch 'ClickHouse:master' into master 2022-06-15 19:12:10 +01:00
Danila Kutenin
048f56bf4d Fix some tests and comments 2022-06-15 14:40:21 +00:00
kssenii
500f49972b Extract diff from PR 36171 2022-06-15 16:40:18 +02:00
Danila Kutenin
08e3f77a9c Optimize most important parts with NEON SIMD 
First part, updated most UTF8, hashing, memory and codecs. Except
utf8lower and upper, maybe a little later.

That includes huge amount of research with movemask dealing. Exact
details and blog post TBD.
 2022-06-15 13:19:29 +00:00
mergify[bot]
12bd2c0e3f
Merge branch 'master' into dt64-neg-subseconds 2022-06-15 02:24:17 +00:00
Anton Popov
a59be0fd5d better support of GCP storage 2022-06-15 00:23:45 +00:00
kssenii
4813b90281 Merge master 2022-06-13 21:32:00 +02:00
kssenii
82629cc2be Continue 2022-06-13 16:32:07 +02:00
alesapin
eae10aab6d
Merge pull request #37964 from excitoon-favorites/betterurlhandlingforversionid 
Better parsing of `versionId` in `S3::URI::URI`
 2022-06-12 13:12:02 +02:00
kssenii
aaf9a46f07 Merge master 2022-06-11 15:56:05 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size 
Support to set max cache size for per query in local cache
 2022-06-10 16:17:18 +02:00
Kseniia Sumarokova
0ae2168fb6
Merge pull request #36328 from bigo-sg/async_hdfs_read_buffer 
Apply read_method 'threadpool' for StorageHive
 2022-06-10 15:04:21 +02:00
Robert Schulze
5f5732a2c4
Merge pull request #37969 from ClickHouse/consistent-macro-usage 
More consistent use of platform macros
 2022-06-10 14:10:01 +02:00
kssenii
0fd2c4b2c0 Continuation 2022-06-10 12:33:29 +02:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros 
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
 2022-06-10 10:22:31 +02:00
Vladimir Chebotarev
512f16f5fa Better parsing of versionId in S3::URI::URI. 2022-06-10 10:06:31 +03:00
Vladimir Chebotarev
3b344a3d26
Got rid of S3AuthSigner (#37769) 2022-06-09 14:44:50 +02:00
KinderRiven
6883ecda96 fix 2022-06-08 14:19:50 +08:00
Azat Khuzhin
71285edfbd Fix reading of empty S3 files 
It is possible for ReadBufferFromS3::nextImpl() called even after eof(),
at least once, and in this case, if the file was empty, then local
working_buffer will be null, while impl.working_buffer will be empty,
but not null, and so local position() after impl->position() =
position() will be incorrect.

I found this with test_storage_s3/test.py::test_empty_file in debug
build, assertion catched this, so maybe it worth get back debug
integration build...

v2: fix test_log_family_s3 failures
    https://s3.amazonaws.com/clickhouse-test-reports/37801/b5e6e2ddae94d6a7eac551309cb67003dff97df1/integration_tests__asan__actions__[2/3].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
taiyang-li
73a484256e Merge branch 'master' into async_hdfs_read_buffer 2022-06-07 12:16:46 +08:00
KinderRiven
3b2a427002 fix 2022-06-07 11:50:56 +08:00
taiyang-li
b36d9f8143 refactor readinto 2022-06-06 12:58:22 +08:00
KinderRiven
13e74bef5e support skip_download_if_exceeds_query_cache 2022-06-06 02:36:23 +08:00
KinderRiven
3ffb0b3549 support_max_query_cache_size 2022-06-05 16:21:36 +08:00
kssenii
f41621736d Fixes 2022-06-03 17:03:05 +02:00
Alexander Tokmakov
084dd6f3c2
Merge pull request #37767 from excitoon-favorites/removeperrequestconfigurationfromawssdkcpp 
Moved `ClientConfigurationPerRequest` to ClickHouse
 2022-06-03 17:42:50 +03:00
zvonand
e6498a3235 update comment 2022-06-03 14:52:34 +03:00
taiyang-li
f202c35311 Merge branch 'master' into async_hdfs_read_buffer 2022-06-03 17:52:09 +08:00
kssenii
3d85391195 Better 2022-06-02 18:00:13 +02:00
kssenii
92a799a0c4 Add ObjectStorageProxy, CachedObjectStorage 2022-06-02 16:45:21 +02:00
Kruglov Pavel
37a4d11c64
Merge pull request #37645 from azat/fix-write 
Fix possible "Cannot write to finalized buffer"
 2022-06-02 14:01:30 +02:00
Antonio Andelic
46abbac352
Merge pull request #37726 from ClickHouse/fix-parallel-download-exception 
Initialize `ParallelReadBuffer` after construction
 2022-06-02 12:35:10 +02:00
Vladimir Chebotarev
a857bc2ccf
Update S3Common.cpp 2022-06-02 08:46:41 +03:00
Vladimir Chebotarev
5fcf840156
Typo. 2022-06-02 08:43:44 +03:00
Vladimir Chebotarev
d5022a0c01 Moved ClientConfigurationPerRequest from ClickHouse/aws-sdk-cpp#1 and ClickHouse/aws-sdk-cpp#2 to ClickHouse. 2022-06-02 06:07:01 +03:00
Azat Khuzhin
62d78d8f20 Fix WriteBufferFromS3 is_finalized check in case of exception 
WriteBufferFromS3::is_finalized is not set if finalizeImpl() throws,
while WriteBuffer::finalized correctly set even in case of exception, so
it should be used instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-01 21:50:30 +03:00
Antonio Andelic
08c20be4d0 Cleaner exception handling in ParallelReadBuffer 2022-06-01 11:51:01 +00:00
Antonio Andelic
f49dd19e7a Revert "Initialize ParallelReadBuffer after construction" 
This reverts commit 31e1e67836.
 2022-06-01 11:43:58 +00:00
Antonio Andelic
31e1e67836 Initialize ParallelReadBuffer after construction 2022-06-01 06:25:32 +00:00
zvonand
8e28ba2583 fix DT64 reader 2022-05-31 18:15:51 +03:00
kssenii
372709db53 Add test 2022-05-30 21:32:49 +02:00
kssenii
bfbc0e3184 Better, important fixes 2022-05-30 20:35:31 +02:00
taiyang-li
dbb8a09825 merge master and solve conflict 2022-05-30 10:47:04 +08:00
kssenii
25b1ed526d Merge master 2022-05-27 16:50:45 +02:00
kssenii
b78455168e Continuation 2022-05-27 16:25:33 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Alexey Milovidov
8ba865bb60
Merge pull request #37344 from excitoon-favorites/fixs3colonandequalssign 
Fixed error with symbols in key name in S3
 2022-05-27 00:58:35 +03:00
kssenii
cbb93fa0ce Continuation 2022-05-26 18:54:59 +02:00
kssenii
b89a4edc04 Merge master 2022-05-25 20:55:04 +02:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
kssenii
0dbc09ef17 Continuation 2022-05-25 20:28:46 +02:00
kssenii
0556237b68 Continuation 2022-05-25 16:49:40 +02:00
taiyang-li
1d9f65a7d4 Merge branch 'master' into async_hdfs_read_buffer 2022-05-25 17:10:22 +08:00
Alexey Milovidov
516fba27dc Merge branch 'master' into allow-setuid-inside-clickhouse 2022-05-24 23:31:14 +02:00
kssenii
35d2dec8d5 Fix 2022-05-24 16:15:55 +02:00
Kruglov Pavel
6c9a524f6b
Merge pull request #37192 from Avogar/formats-with-names 
Improve performance and memory usage for select of subset of columns for some formats
 2022-05-24 13:28:14 +02:00
李扬
e33cfc889c
Fix bug of datetime64 parsed from string '1969-12-31 23:59:59.123' (#37039) 2022-05-24 10:47:17 +02:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage 2022-05-23 20:29:49 +00:00
alesapin
1e1e6d4fa0 Review fixes 2022-05-23 16:24:06 +02:00
alesapin
216184dfd3 Rename one more file 2022-05-23 13:20:32 +02:00
taiyang-li
14f84f02d5 Merge branch 'master' into async_hdfs_read_buffer 2022-05-23 18:36:21 +08:00
Kruglov Pavel
f539fb835d
Merge branch 'master' into formats-with-names 2022-05-23 12:14:20 +02:00
Alexey Milovidov
c541d30a38 Fix error 2022-05-23 02:48:05 +02:00
Alexey Milovidov
cc985d9b92 Fix error 2022-05-23 02:30:33 +02:00
kssenii
9aa20193d5 Merge master 2022-05-22 00:56:20 +02:00
kssenii
ffd8d9723a Continuation 2022-05-22 00:21:40 +02:00
avogar
a4cf07708c Fix comments 2022-05-20 14:57:27 +00:00
avogar
566d1b15fd Merge branch 'master' of github.com:ClickHouse/ClickHouse into formats-with-names 2022-05-20 13:54:52 +00:00
avogar
a6a430c5ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-columns-formats 2022-05-20 11:08:30 +00:00
alesapin
654b27e307 Merge branch 'master' into i_object_storage 2022-05-20 11:56:13 +02:00
Vladimir Chebotarev
cdbe6672cb Added one more symbol and proper comment. 2022-05-20 11:35:18 +03:00
alesapin
c87c3fcfd9 Move Azure blob storage 2022-05-19 19:25:45 +02:00
Vladimir Chebotarev
1ad19051e5 Fixed error with symbols in key name in S3. 2022-05-19 11:01:05 +03:00
Nikolai Kochetov
64ecb3941c
Merge pull request #37259 from ClickHouse/clangtidies2 
Activate more clangtidies
 2022-05-18 13:01:40 +02:00
alesapin
e99c90118e Merge branch 'master' into i_object_storage 2022-05-18 12:36:22 +02:00
Kseniia Sumarokova
38c87ab9cd
Merge pull request #36493 from kssenii/fix-encrypted-buffer 
Fix offset update ReadBufferFromEncryptedFile
 2022-05-18 09:46:17 +02:00
Robert Schulze
0c55ac76d2
A few clangtidy updates 
Enable:

- bugprone-lambda-function-name: "Checks for attempts to get the name of
  a function from within a lambda expression. The name of a lambda is
  always something like operator(), which is almost never what was
  intended."

- bugprone-unhandled-self-assignment: "Finds user-defined copy
  assignment operators which do not protect the code against
  self-assignment either by checking self-assignment explicitly or using
  the copy-and-swap or the copy-and-move method.""

- hicpp-invalid-access-moved: "Warns if an object is used after it has
  been moved."

- hicpp-use-noexcept: "This check replaces deprecated dynamic exception
  specifications with the appropriate noexcept specification (introduced
  in C++11)"

- hicpp-use-override: "Adds override (introduced in C++11) to overridden
  virtual functions and removes virtual from those functions as it is
  not required."

- performance-type-promotion-in-math-fn: "Finds calls to C math library
  functions (from math.h or, in C++, cmath) with implicit float to
  double promotions."

Split up:

- cppcoreguidelines-*. Some of them may be useful (haven't checked in
  detail), therefore allow to toggle them individually.

Disable:

- linuxkernel-*. Obvious.
 2022-05-17 20:56:57 +02:00
kssenii
e4e6e44e86 Merge master 2022-05-17 15:47:33 +02:00
kssenii
b53ca506e6 Add unit test 2022-05-17 14:55:42 +02:00
alesapin
4d4634b12d
Merge pull request #37164 from kssenii/ficx-async-reads-assertions 
fix async reads from remote fs internal setting not being always turned on
 2022-05-17 11:29:14 +02:00
taiyang-li
14ab7eb5a3 merge master and solve conflict 2022-05-17 16:28:08 +08:00
kssenii
6a51dea791 Merge master 2022-05-16 22:13:22 +02:00
alesapin
ceb7249916 Fix stupid buge 2022-05-16 19:57:48 +02:00
alesapin
ce6d5f7dcc Merge branch 'master' into i_object_storage 2022-05-16 17:42:22 +02:00
kssenii
f7493965b7 Remove additional check for merge tree 2022-05-16 11:19:57 +02:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
alesapin
4f1d4dde22 Disable s3 logging by default 2022-05-16 00:16:00 +02:00
kssenii
646cf4cc95 Better 2022-05-14 14:26:04 +02:00
Kseniia Sumarokova
f4a8a0cefe
Merge pull request #37200 from kssenii/add-s3-requests-metric 
Add S3Requests metric
 2022-05-14 10:08:21 +02:00
alesapin
cc304a3178 Merge branch 'master' into i_object_storage 2022-05-13 23:44:00 +02:00
avogar
68bb07d166 Better naming 2022-05-13 18:39:19 +00:00
kssenii
58faaf0ea3 Merge master 2022-05-13 18:51:07 +02:00
kssenii
5dd3ad08a1 Add S3Requests metric 2022-05-13 18:12:46 +02:00
Kruglov Pavel
4549d2f40f
Fix build 2022-05-13 17:41:26 +02:00
kssenii
d8ec49d9b3 Fix 2022-05-13 17:17:13 +02:00
avogar
febabb2787 Fix build 2022-05-13 15:08:02 +00:00
kssenii
0870b9376c Fix 2022-05-13 16:30:59 +02:00
avogar
cef13c2c02 Allow to skip unknown columns in Native format 2022-05-13 14:27:15 +00:00
avogar
b17fec659a Improve performance and memory usage for select of subset of columns for some formats 2022-05-13 13:51:28 +00:00
Nikolai Kochetov
b2c363a38a
Merge pull request #37157 from ClickHouse/try-enable-cache-for-https-session 
Enable DNS cache for HTTPSClientSession.
 2022-05-13 15:42:30 +02:00
wuxiaobai24
f3e573bfde Merge remote-tracking branch 'upstream/master' into zstd_window_log_max 2022-05-13 20:42:54 +08:00
mergify[bot]
4a661b6e78
Merge branch 'master' into json-columns-formats 2022-05-13 11:32:03 +00:00
wuxiaobai24
ab5636a46a fix 2022-05-13 17:52:39 +08:00
Kseniia Sumarokova
320efb2221
Merge branch 'master' into better-read-from-cache 2022-05-12 23:35:39 +02:00
Kseniia Sumarokova
3fd9517ff8
Merge pull request #37130 from ClickHouse/kssenii-patch-3 
Update default remote fs read method in ReadSettings
 2022-05-12 23:34:28 +02:00
Nikolai Kochetov
f8bd4fa0f0 Remove old comment. 2022-05-12 15:24:36 +00:00
Nikolai Kochetov
77fb1b84ac Enable DNS cache for HTTPSClientSession. 2022-05-12 15:21:29 +00:00
kssenii
483e056bcc Merge master 2022-05-12 14:58:24 +02:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
Kseniia Sumarokova
2a4f763a5e
Merge pull request #37075 from kssenii/fix-progress-for-insert-select 
Fix progress indication for `INSERT SELECT` in clickhouse-local and for file progress in client
 2022-05-12 12:05:53 +02:00
alesapin
5dcd25be23 Initial implementation 2022-05-12 00:04:54 +02:00
Kseniia Sumarokova
cf96f3ce3f
Update Progress.cpp 2022-05-11 22:52:30 +02:00
Kseniia Sumarokova
ec4955f157
Update ReadSettings.h 2022-05-11 18:32:31 +02:00
kssenii
c84e0d13fa Fix test 2022-05-11 14:36:34 +02:00
kssenii
61f2737e17 Merge master 2022-05-10 19:31:22 +02:00
wuxiaobai24
85356bbf64 fix 2022-05-11 00:53:09 +08:00
kssenii
f4b45a5805 Better profile events 2022-05-10 16:00:53 +02:00
kssenii
0feda67ec4 Fix progress for insert select 2022-05-10 15:04:08 +02:00
avogar
f6b16880bd Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-columns-formats 2022-05-10 12:57:18 +00:00
mergify[bot]
2c911da020
Merge branch 'master' into dont_finalize_s3_buffer 2022-05-10 11:26:51 +00:00
kssenii
2f0f76369a Better 2022-05-09 21:25:26 +02:00
Robert Schulze
1b81bb49b4
Enable clang-tidy modernize-deprecated-headers & hicpp-deprecated-headers 
Official docs:

  Some headers from C library were deprecated in C++ and are no longer
  welcome in C++ codebases. Some have no effect in C++. For more details
  refer to the C++ 14 Standard [depr.c.headers] section. This check
  replaces C standard library headers with their C++ alternatives and
  removes redundant ones.
 2022-05-09 08:23:33 +02:00
wuxiaobai24
58755cb156 add zstd_window_log_max setting 2022-05-08 14:02:40 +08:00
Alexey Milovidov
4eac661588
Merge branch 'master' into master 2022-05-07 13:21:59 +03:00
avogar
62a7ba3f26 Add columnar JSON formats 2022-05-06 16:48:48 +00:00
kssenii
fd1ab053f8 Merge master 2022-05-06 12:50:33 +02:00
mergify[bot]
9ce436733f
Merge branch 'master' into fix-current-size-count-in-cache 2022-05-05 17:33:56 +00:00
kssenii
0117fd40a6 Review fixes 2022-05-05 17:39:36 +02:00
Kseniia Sumarokova
224ab38b65
Merge pull request #36801 from ClickHouse/parallel-url-threads 
Use mutex per worker in ParallelReadBuffer
 2022-05-05 16:36:55 +02:00
Alexander Gololobov
7e3a805ae3
Merge pull request #36572 from ClickHouse/fix-trash-s3-metrics 
Everything related to S3 metrics was not good
 2022-05-05 11:44:02 +02:00
alesapin
b76406fc8f
Merge pull request #36910 from ClickHouse/add_test 
Fix bug in keeper which could lead to corrupted compressed logs
 2022-05-04 23:44:45 +02:00
Alexander Tokmakov
e30365a531
Merge pull request #36872 from ClickHouse/fix_exception_message 
Add extra info when sending exception
 2022-05-04 23:24:47 +03:00
Kruglov Pavel
77e55c344c
Merge pull request #36667 from Avogar/mysqldump-format 
Add MySQLDump input format
 2022-05-04 19:49:48 +02:00
alesapin
63f15368ff Remove unused header 2022-05-04 18:30:23 +02:00
alesapin
c09825919b Better comments 2022-05-04 18:24:28 +02:00
alesapin
c6556da5fc Fix bug which can lead to corrupted logs in compressed format 2022-05-04 18:08:00 +02:00
mergify[bot]
e0fa563092
Merge branch 'master' into fix_exception_message 2022-05-04 15:56:03 +00:00
mergify[bot]
6ecddf4077
Merge branch 'master' into fix-trash-s3-metrics 2022-05-04 10:14:29 +00:00
Antonio Andelic
29e0edd49f Unlock mutex before destroying worker 2022-05-04 06:30:54 +00:00
Antonio Andelic
056fac9aef Merge branch 'master' into parallel-url-threads 2022-05-04 06:26:56 +00:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
kssenii
0266cdf125 Add entries to cache log when cache was not event attempted 2022-05-03 18:29:41 +02:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing 
Parallel parsing of hive text format
 2022-05-03 15:56:28 +02:00
Kruglov Pavel
a902b9d1e7
Try fix builds 2022-05-03 15:56:25 +02:00
Alexander Tokmakov
94af11dd74 add extra info when sending exception 2022-05-03 15:13:47 +02:00
alesapin
991f78b485 Get rid of atomic 2022-05-03 12:17:50 +02:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable 
... IStorage has deleted copy ctor / assignment already
 2022-05-03 09:07:08 +02:00
Antonio Andelic
fd0ba2d10b Protect background exception 2022-05-03 06:34:03 +00:00
Antonio Andelic
bfdb4566c6 Use mutex per worker 2022-05-03 06:18:39 +00:00
alesapin
98e3b16560 Terminate if S3 buffer is not finalized 2022-05-02 22:14:18 +02:00
avogar
533e4adf6c Try to fix build 2022-05-02 12:07:31 +00:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying 
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
 2022-05-02 08:46:52 +02:00
Alexey Milovidov
ef9633499c Merge branch 'master' into fix-trash-s3-metrics 2022-05-02 03:55:10 +02:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase 
Reorganize source files so that base won't depend on Common
 2022-04-30 09:40:58 +03:00
Kseniia Sumarokova
ca994e0861
Merge pull request #36660 from kssenii/fix-stress-test 
Fix stress test after 36639
 2022-04-29 12:56:25 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Kseniia Sumarokova
4102502717
Merge pull request #36619 from ClickHouse/parallel-url-performance-issue 
ParallelReadBuffer small improvements
 2022-04-29 02:55:30 +02:00
mergify[bot]
41aa85fd4c
Merge branch 'master' into fix-stress-test 2022-04-29 00:54:09 +00:00
kssenii
1cbf672237 Another fix 2022-04-28 22:51:57 +02:00
Sergei Trifonov
11f40376ae
Merge pull request #34571 from surahman/CH-31221-AWS-S3-object-version-id 
[CH-31221] AWS S3 Object Version Specification
 2022-04-28 15:45:33 +02:00
Alexander Tokmakov
21a021c210
Merge pull request #36745 from vdimir/fix-build-seekable-read-buff 
Fix build SeekableReadBufferWithSize -> SeekableReadBuffer
 2022-04-28 16:34:24 +03:00
vdimir
7c3cf30163
Fix build SeekableReadBufferWithSize -> SeekableReadBuffer 2022-04-28 15:05:04 +02:00
Anton Kozlov
ca519ca75e
Improve performance of file descriptor cache by narrowing mutex scopes (#36682) 2022-04-28 14:56:53 +02:00
Kseniia Sumarokova
4c371f710e
Merge pull request #36676 from kssenii/refactor-with-size-buffer 
Better version of SeekableReadBufferWithSize
 2022-04-28 13:44:25 +02:00
kssenii
9b38c9f7eb Fix 2022-04-28 13:06:43 +02:00
Antonio Andelic
a693898038 Merge branch 'master' into parallel-url-performance-issue 2022-04-28 07:29:07 +00:00
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3 
Backup for replicated tables and other improvements
 2022-04-28 09:23:53 +02:00
kssenii
37d11cfb1c Merge master 2022-04-28 00:44:15 +02:00
kssenii
67e0cf9ab6 One more fix 2022-04-28 00:37:57 +02:00
Kseniia Sumarokova
39b3c1c6af
Merge pull request #36697 from KinderRiven/fix_bug_for_local_cache 
Fix deadlock for local cache
 2022-04-27 18:11:43 +02:00
KinderRiven
cd540c3c51 fix 2022-04-27 19:36:25 +08:00
KinderRiven
813f1bd001 fix 2022-04-27 19:31:50 +08:00
KinderRiven
ad4fb637f0 fix set better state 2022-04-27 19:17:46 +08:00
KinderRiven
504e2c26db fix bug for deadlock in local cache 2022-04-27 17:34:14 +08:00
Alexey Milovidov
e85c5679e9
Update S3Common.cpp 2022-04-27 03:33:01 +03:00
Alexey Milovidov
3c53cd42bb Remove excessive logging from S3 2022-04-27 02:30:13 +02:00
kssenii
0c43b7b08a Merge master 2022-04-26 16:10:47 +02:00
kssenii
9d364cdce2 Refactor 2022-04-26 15:33:53 +02:00
kssenii
a395532d54 Add support for exact right bounded reads for some other buffers 2022-04-26 12:56:40 +02:00
Sergei Trifonov
c1a0165f7d merge from master and fix conflicts 2022-04-26 12:39:16 +02:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
Vitaly Baranov
3966ee1e30 Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry. 2022-04-25 16:34:33 +02:00
Antonio Andelic
4e3b79ec0c remove arena 2022-04-25 12:23:24 +00:00
Antonio Andelic
3dd38f4917 Use vector for segments 2022-04-25 12:10:41 +00:00
Antonio Andelic
ef00ac12fa Use Memory 2022-04-25 11:39:07 +00:00
taiyang-li
f040c8ae65 merge master and solve conflict 2022-04-24 11:42:36 +08:00
Alexey Milovidov
6de4f2bbff Everything related to S3 metrics was a trash 2022-04-24 03:42:18 +02:00
Kseniia Sumarokova
33bb48106f
Merge pull request #36314 from CurtizJ/print-bad-filenames 
Show names of erroneous files in case of parsing errors while executing table functions
 2022-04-22 13:24:55 +02:00
Sergei Trifonov
b371251e80 Merge branch 'master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-22 08:39:02 +02:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes 
Clang tidy fixes
 2022-04-21 16:11:27 +02:00
kssenii
24e376337d Merge with master 2022-04-21 12:33:56 +02:00
taiyang-li
1ea96a8732 merge master and solve conflict 2022-04-21 17:16:07 +08:00
Sergei Trifonov
34842f209c merge master and fix conflicts 2022-04-20 18:35:56 +02:00
taiyang-li
d533b569ad debugging 2022-04-20 19:58:31 +08:00
李扬
a1e54c3918
Fix wrong result of datetime64 when negative (#35440) 2022-04-20 13:11:31 +02:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
f37812f89a
Merge pull request #36079 from kssenii/change-hashed-path 
Change hashed path for cache
 2022-04-20 10:27:44 +02:00
taiyang-li
fb6a56d4b0 finish debug 2022-04-20 16:24:18 +08:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings 
Better settings configuration for s3
 2022-04-20 10:22:06 +02:00
Anton Popov
bee4ca9b62 add more tests for error diagnostics in files 2022-04-19 15:56:34 +00:00
Anton Popov
3e361c9759 Merge remote-tracking branch 'upstream/master' into HEAD 2022-04-19 14:18:04 +00:00
kssenii
7cfc0ac433 Merge with master 2022-04-19 15:13:45 +02:00
Sergei Trifonov
75c1053ad7 Merge remote-tracking branch 'origin/master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-19 13:17:25 +02:00
mergify[bot]
64d5e0c8a2
Merge branch 'master' into change-hashed-path 2022-04-19 09:33:48 +00:00
kssenii
1f9a597b77 Merge master 2022-04-19 11:31:13 +02:00
taiyang-li
99ad2f5fa6 Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer 2022-04-19 15:49:38 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia 
Remove remaining parts of Arcadia
 2022-04-18 07:02:54 +03:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
taiyang-li
0ad2a76fae Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer 2022-04-16 18:45:39 +08:00
taiyang-li
cd83fd5f8a tobe debug 2022-04-16 18:41:18 +08:00
Alexey Milovidov
33ffdcaa4f Return two changes 2022-04-16 02:18:22 +02:00
Anton Popov
2de6668b3f show names of erroneous files 2022-04-16 00:10:47 +00:00