thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
94,800 Commits 9,315 Branches 1,783 Tags 2.1 GiB
90fa2ed8c1
Commit Graph

1917 Commits

Author SHA1 Message Date
Antonio Andelic
46abbac352
Merge pull request #37726 from ClickHouse/fix-parallel-download-exception 
Initialize `ParallelReadBuffer` after construction
 2022-06-02 12:35:10 +02:00
Vladimir Chebotarev
a857bc2ccf
Update S3Common.cpp 2022-06-02 08:46:41 +03:00
Vladimir Chebotarev
5fcf840156
Typo. 2022-06-02 08:43:44 +03:00
Vladimir Chebotarev
d5022a0c01 Moved ClientConfigurationPerRequest from ClickHouse/aws-sdk-cpp#1 and ClickHouse/aws-sdk-cpp#2 to ClickHouse. 2022-06-02 06:07:01 +03:00
Azat Khuzhin
62d78d8f20 Fix WriteBufferFromS3 is_finalized check in case of exception 
WriteBufferFromS3::is_finalized is not set if finalizeImpl() throws,
while WriteBuffer::finalized correctly set even in case of exception, so
it should be used instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-01 21:50:30 +03:00
Antonio Andelic
08c20be4d0 Cleaner exception handling in ParallelReadBuffer 2022-06-01 11:51:01 +00:00
Antonio Andelic
f49dd19e7a Revert "Initialize ParallelReadBuffer after construction" 
This reverts commit 31e1e67836.
 2022-06-01 11:43:58 +00:00
Antonio Andelic
31e1e67836 Initialize ParallelReadBuffer after construction 2022-06-01 06:25:32 +00:00
zvonand
8e28ba2583 fix DT64 reader 2022-05-31 18:15:51 +03:00
taiyang-li
dbb8a09825 merge master and solve conflict 2022-05-30 10:47:04 +08:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Alexey Milovidov
8ba865bb60
Merge pull request #37344 from excitoon-favorites/fixs3colonandequalssign 
Fixed error with symbols in key name in S3
 2022-05-27 00:58:35 +03:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
taiyang-li
1d9f65a7d4 Merge branch 'master' into async_hdfs_read_buffer 2022-05-25 17:10:22 +08:00
Alexey Milovidov
516fba27dc Merge branch 'master' into allow-setuid-inside-clickhouse 2022-05-24 23:31:14 +02:00
Kruglov Pavel
6c9a524f6b
Merge pull request #37192 from Avogar/formats-with-names 
Improve performance and memory usage for select of subset of columns for some formats
 2022-05-24 13:28:14 +02:00
李扬
e33cfc889c
Fix bug of datetime64 parsed from string '1969-12-31 23:59:59.123' (#37039) 2022-05-24 10:47:17 +02:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage 2022-05-23 20:29:49 +00:00
alesapin
1e1e6d4fa0 Review fixes 2022-05-23 16:24:06 +02:00
alesapin
216184dfd3 Rename one more file 2022-05-23 13:20:32 +02:00
taiyang-li
14f84f02d5 Merge branch 'master' into async_hdfs_read_buffer 2022-05-23 18:36:21 +08:00
Kruglov Pavel
f539fb835d
Merge branch 'master' into formats-with-names 2022-05-23 12:14:20 +02:00
Alexey Milovidov
c541d30a38 Fix error 2022-05-23 02:48:05 +02:00
Alexey Milovidov
cc985d9b92 Fix error 2022-05-23 02:30:33 +02:00
avogar
a4cf07708c Fix comments 2022-05-20 14:57:27 +00:00
avogar
566d1b15fd Merge branch 'master' of github.com:ClickHouse/ClickHouse into formats-with-names 2022-05-20 13:54:52 +00:00
avogar
a6a430c5ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-columns-formats 2022-05-20 11:08:30 +00:00
alesapin
654b27e307 Merge branch 'master' into i_object_storage 2022-05-20 11:56:13 +02:00
Vladimir Chebotarev
cdbe6672cb Added one more symbol and proper comment. 2022-05-20 11:35:18 +03:00
alesapin
c87c3fcfd9 Move Azure blob storage 2022-05-19 19:25:45 +02:00
Vladimir Chebotarev
1ad19051e5 Fixed error with symbols in key name in S3. 2022-05-19 11:01:05 +03:00
Nikolai Kochetov
64ecb3941c
Merge pull request #37259 from ClickHouse/clangtidies2 
Activate more clangtidies
 2022-05-18 13:01:40 +02:00
alesapin
e99c90118e Merge branch 'master' into i_object_storage 2022-05-18 12:36:22 +02:00
Kseniia Sumarokova
38c87ab9cd
Merge pull request #36493 from kssenii/fix-encrypted-buffer 
Fix offset update ReadBufferFromEncryptedFile
 2022-05-18 09:46:17 +02:00
Robert Schulze
0c55ac76d2
A few clangtidy updates 
Enable:

- bugprone-lambda-function-name: "Checks for attempts to get the name of
  a function from within a lambda expression. The name of a lambda is
  always something like operator(), which is almost never what was
  intended."

- bugprone-unhandled-self-assignment: "Finds user-defined copy
  assignment operators which do not protect the code against
  self-assignment either by checking self-assignment explicitly or using
  the copy-and-swap or the copy-and-move method.""

- hicpp-invalid-access-moved: "Warns if an object is used after it has
  been moved."

- hicpp-use-noexcept: "This check replaces deprecated dynamic exception
  specifications with the appropriate noexcept specification (introduced
  in C++11)"

- hicpp-use-override: "Adds override (introduced in C++11) to overridden
  virtual functions and removes virtual from those functions as it is
  not required."

- performance-type-promotion-in-math-fn: "Finds calls to C math library
  functions (from math.h or, in C++, cmath) with implicit float to
  double promotions."

Split up:

- cppcoreguidelines-*. Some of them may be useful (haven't checked in
  detail), therefore allow to toggle them individually.

Disable:

- linuxkernel-*. Obvious.
 2022-05-17 20:56:57 +02:00
kssenii
b53ca506e6 Add unit test 2022-05-17 14:55:42 +02:00
alesapin
4d4634b12d
Merge pull request #37164 from kssenii/ficx-async-reads-assertions 
fix async reads from remote fs internal setting not being always turned on
 2022-05-17 11:29:14 +02:00
taiyang-li
14ab7eb5a3 merge master and solve conflict 2022-05-17 16:28:08 +08:00
alesapin
ceb7249916 Fix stupid buge 2022-05-16 19:57:48 +02:00
alesapin
ce6d5f7dcc Merge branch 'master' into i_object_storage 2022-05-16 17:42:22 +02:00
kssenii
f7493965b7 Remove additional check for merge tree 2022-05-16 11:19:57 +02:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
alesapin
4f1d4dde22 Disable s3 logging by default 2022-05-16 00:16:00 +02:00
Kseniia Sumarokova
f4a8a0cefe
Merge pull request #37200 from kssenii/add-s3-requests-metric 
Add S3Requests metric
 2022-05-14 10:08:21 +02:00
alesapin
cc304a3178 Merge branch 'master' into i_object_storage 2022-05-13 23:44:00 +02:00
avogar
68bb07d166 Better naming 2022-05-13 18:39:19 +00:00
kssenii
5dd3ad08a1 Add S3Requests metric 2022-05-13 18:12:46 +02:00
Kruglov Pavel
4549d2f40f
Fix build 2022-05-13 17:41:26 +02:00
kssenii
d8ec49d9b3 Fix 2022-05-13 17:17:13 +02:00
avogar
febabb2787 Fix build 2022-05-13 15:08:02 +00:00
kssenii
0870b9376c Fix 2022-05-13 16:30:59 +02:00
avogar
cef13c2c02 Allow to skip unknown columns in Native format 2022-05-13 14:27:15 +00:00
avogar
b17fec659a Improve performance and memory usage for select of subset of columns for some formats 2022-05-13 13:51:28 +00:00
Nikolai Kochetov
b2c363a38a
Merge pull request #37157 from ClickHouse/try-enable-cache-for-https-session 
Enable DNS cache for HTTPSClientSession.
 2022-05-13 15:42:30 +02:00
wuxiaobai24
f3e573bfde Merge remote-tracking branch 'upstream/master' into zstd_window_log_max 2022-05-13 20:42:54 +08:00
mergify[bot]
4a661b6e78
Merge branch 'master' into json-columns-formats 2022-05-13 11:32:03 +00:00
wuxiaobai24
ab5636a46a fix 2022-05-13 17:52:39 +08:00
Kseniia Sumarokova
320efb2221
Merge branch 'master' into better-read-from-cache 2022-05-12 23:35:39 +02:00
Kseniia Sumarokova
3fd9517ff8
Merge pull request #37130 from ClickHouse/kssenii-patch-3 
Update default remote fs read method in ReadSettings
 2022-05-12 23:34:28 +02:00
Nikolai Kochetov
f8bd4fa0f0 Remove old comment. 2022-05-12 15:24:36 +00:00
Nikolai Kochetov
77fb1b84ac Enable DNS cache for HTTPSClientSession. 2022-05-12 15:21:29 +00:00
kssenii
483e056bcc Merge master 2022-05-12 14:58:24 +02:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
Kseniia Sumarokova
2a4f763a5e
Merge pull request #37075 from kssenii/fix-progress-for-insert-select 
Fix progress indication for `INSERT SELECT` in clickhouse-local and for file progress in client
 2022-05-12 12:05:53 +02:00
alesapin
5dcd25be23 Initial implementation 2022-05-12 00:04:54 +02:00
Kseniia Sumarokova
cf96f3ce3f
Update Progress.cpp 2022-05-11 22:52:30 +02:00
Kseniia Sumarokova
ec4955f157
Update ReadSettings.h 2022-05-11 18:32:31 +02:00
kssenii
c84e0d13fa Fix test 2022-05-11 14:36:34 +02:00
kssenii
61f2737e17 Merge master 2022-05-10 19:31:22 +02:00
wuxiaobai24
85356bbf64 fix 2022-05-11 00:53:09 +08:00
kssenii
f4b45a5805 Better profile events 2022-05-10 16:00:53 +02:00
kssenii
0feda67ec4 Fix progress for insert select 2022-05-10 15:04:08 +02:00
avogar
f6b16880bd Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-columns-formats 2022-05-10 12:57:18 +00:00
mergify[bot]
2c911da020
Merge branch 'master' into dont_finalize_s3_buffer 2022-05-10 11:26:51 +00:00
kssenii
2f0f76369a Better 2022-05-09 21:25:26 +02:00
Robert Schulze
1b81bb49b4
Enable clang-tidy modernize-deprecated-headers & hicpp-deprecated-headers 
Official docs:

  Some headers from C library were deprecated in C++ and are no longer
  welcome in C++ codebases. Some have no effect in C++. For more details
  refer to the C++ 14 Standard [depr.c.headers] section. This check
  replaces C standard library headers with their C++ alternatives and
  removes redundant ones.
 2022-05-09 08:23:33 +02:00
wuxiaobai24
58755cb156 add zstd_window_log_max setting 2022-05-08 14:02:40 +08:00
Alexey Milovidov
4eac661588
Merge branch 'master' into master 2022-05-07 13:21:59 +03:00
avogar
62a7ba3f26 Add columnar JSON formats 2022-05-06 16:48:48 +00:00
kssenii
fd1ab053f8 Merge master 2022-05-06 12:50:33 +02:00
mergify[bot]
9ce436733f
Merge branch 'master' into fix-current-size-count-in-cache 2022-05-05 17:33:56 +00:00
kssenii
0117fd40a6 Review fixes 2022-05-05 17:39:36 +02:00
Kseniia Sumarokova
224ab38b65
Merge pull request #36801 from ClickHouse/parallel-url-threads 
Use mutex per worker in ParallelReadBuffer
 2022-05-05 16:36:55 +02:00
Alexander Gololobov
7e3a805ae3
Merge pull request #36572 from ClickHouse/fix-trash-s3-metrics 
Everything related to S3 metrics was not good
 2022-05-05 11:44:02 +02:00
alesapin
b76406fc8f
Merge pull request #36910 from ClickHouse/add_test 
Fix bug in keeper which could lead to corrupted compressed logs
 2022-05-04 23:44:45 +02:00
Alexander Tokmakov
e30365a531
Merge pull request #36872 from ClickHouse/fix_exception_message 
Add extra info when sending exception
 2022-05-04 23:24:47 +03:00
Kruglov Pavel
77e55c344c
Merge pull request #36667 from Avogar/mysqldump-format 
Add MySQLDump input format
 2022-05-04 19:49:48 +02:00
alesapin
63f15368ff Remove unused header 2022-05-04 18:30:23 +02:00
alesapin
c09825919b Better comments 2022-05-04 18:24:28 +02:00
alesapin
c6556da5fc Fix bug which can lead to corrupted logs in compressed format 2022-05-04 18:08:00 +02:00
mergify[bot]
e0fa563092
Merge branch 'master' into fix_exception_message 2022-05-04 15:56:03 +00:00
mergify[bot]
6ecddf4077
Merge branch 'master' into fix-trash-s3-metrics 2022-05-04 10:14:29 +00:00
Antonio Andelic
29e0edd49f Unlock mutex before destroying worker 2022-05-04 06:30:54 +00:00
Antonio Andelic
056fac9aef Merge branch 'master' into parallel-url-threads 2022-05-04 06:26:56 +00:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
kssenii
0266cdf125 Add entries to cache log when cache was not event attempted 2022-05-03 18:29:41 +02:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing 
Parallel parsing of hive text format
 2022-05-03 15:56:28 +02:00
Kruglov Pavel
a902b9d1e7
Try fix builds 2022-05-03 15:56:25 +02:00
Alexander Tokmakov
94af11dd74 add extra info when sending exception 2022-05-03 15:13:47 +02:00
alesapin
991f78b485 Get rid of atomic 2022-05-03 12:17:50 +02:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable 
... IStorage has deleted copy ctor / assignment already
 2022-05-03 09:07:08 +02:00
Antonio Andelic
fd0ba2d10b Protect background exception 2022-05-03 06:34:03 +00:00
Antonio Andelic
bfdb4566c6 Use mutex per worker 2022-05-03 06:18:39 +00:00
alesapin
98e3b16560 Terminate if S3 buffer is not finalized 2022-05-02 22:14:18 +02:00
avogar
533e4adf6c Try to fix build 2022-05-02 12:07:31 +00:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying 
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
 2022-05-02 08:46:52 +02:00
Alexey Milovidov
ef9633499c Merge branch 'master' into fix-trash-s3-metrics 2022-05-02 03:55:10 +02:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase 
Reorganize source files so that base won't depend on Common
 2022-04-30 09:40:58 +03:00
Kseniia Sumarokova
ca994e0861
Merge pull request #36660 from kssenii/fix-stress-test 
Fix stress test after 36639
 2022-04-29 12:56:25 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Kseniia Sumarokova
4102502717
Merge pull request #36619 from ClickHouse/parallel-url-performance-issue 
ParallelReadBuffer small improvements
 2022-04-29 02:55:30 +02:00
mergify[bot]
41aa85fd4c
Merge branch 'master' into fix-stress-test 2022-04-29 00:54:09 +00:00
kssenii
1cbf672237 Another fix 2022-04-28 22:51:57 +02:00
Sergei Trifonov
11f40376ae
Merge pull request #34571 from surahman/CH-31221-AWS-S3-object-version-id 
[CH-31221] AWS S3 Object Version Specification
 2022-04-28 15:45:33 +02:00
Alexander Tokmakov
21a021c210
Merge pull request #36745 from vdimir/fix-build-seekable-read-buff 
Fix build SeekableReadBufferWithSize -> SeekableReadBuffer
 2022-04-28 16:34:24 +03:00
vdimir
7c3cf30163
Fix build SeekableReadBufferWithSize -> SeekableReadBuffer 2022-04-28 15:05:04 +02:00
Anton Kozlov
ca519ca75e
Improve performance of file descriptor cache by narrowing mutex scopes (#36682) 2022-04-28 14:56:53 +02:00
Kseniia Sumarokova
4c371f710e
Merge pull request #36676 from kssenii/refactor-with-size-buffer 
Better version of SeekableReadBufferWithSize
 2022-04-28 13:44:25 +02:00
kssenii
9b38c9f7eb Fix 2022-04-28 13:06:43 +02:00
Antonio Andelic
a693898038 Merge branch 'master' into parallel-url-performance-issue 2022-04-28 07:29:07 +00:00
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3 
Backup for replicated tables and other improvements
 2022-04-28 09:23:53 +02:00
kssenii
37d11cfb1c Merge master 2022-04-28 00:44:15 +02:00
kssenii
67e0cf9ab6 One more fix 2022-04-28 00:37:57 +02:00
Kseniia Sumarokova
39b3c1c6af
Merge pull request #36697 from KinderRiven/fix_bug_for_local_cache 
Fix deadlock for local cache
 2022-04-27 18:11:43 +02:00
KinderRiven
cd540c3c51 fix 2022-04-27 19:36:25 +08:00
KinderRiven
813f1bd001 fix 2022-04-27 19:31:50 +08:00
KinderRiven
ad4fb637f0 fix set better state 2022-04-27 19:17:46 +08:00
KinderRiven
504e2c26db fix bug for deadlock in local cache 2022-04-27 17:34:14 +08:00
Alexey Milovidov
e85c5679e9
Update S3Common.cpp 2022-04-27 03:33:01 +03:00
Alexey Milovidov
3c53cd42bb Remove excessive logging from S3 2022-04-27 02:30:13 +02:00
kssenii
9d364cdce2 Refactor 2022-04-26 15:33:53 +02:00
Sergei Trifonov
c1a0165f7d merge from master and fix conflicts 2022-04-26 12:39:16 +02:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
Vitaly Baranov
3966ee1e30 Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry. 2022-04-25 16:34:33 +02:00
Antonio Andelic
4e3b79ec0c remove arena 2022-04-25 12:23:24 +00:00
Antonio Andelic
3dd38f4917 Use vector for segments 2022-04-25 12:10:41 +00:00
Antonio Andelic
ef00ac12fa Use Memory 2022-04-25 11:39:07 +00:00
taiyang-li
f040c8ae65 merge master and solve conflict 2022-04-24 11:42:36 +08:00
Alexey Milovidov
6de4f2bbff Everything related to S3 metrics was a trash 2022-04-24 03:42:18 +02:00
Kseniia Sumarokova
33bb48106f
Merge pull request #36314 from CurtizJ/print-bad-filenames 
Show names of erroneous files in case of parsing errors while executing table functions
 2022-04-22 13:24:55 +02:00
Sergei Trifonov
b371251e80 Merge branch 'master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-22 08:39:02 +02:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes 
Clang tidy fixes
 2022-04-21 16:11:27 +02:00
taiyang-li
1ea96a8732 merge master and solve conflict 2022-04-21 17:16:07 +08:00
Sergei Trifonov
34842f209c merge master and fix conflicts 2022-04-20 18:35:56 +02:00
taiyang-li
d533b569ad debugging 2022-04-20 19:58:31 +08:00
李扬
a1e54c3918
Fix wrong result of datetime64 when negative (#35440) 2022-04-20 13:11:31 +02:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
f37812f89a
Merge pull request #36079 from kssenii/change-hashed-path 
Change hashed path for cache
 2022-04-20 10:27:44 +02:00
taiyang-li
fb6a56d4b0 finish debug 2022-04-20 16:24:18 +08:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings 
Better settings configuration for s3
 2022-04-20 10:22:06 +02:00