Commit Graph

2106 Commits

Author SHA1 Message Date
kssenii
c9b512e33e Fix 2022-09-02 16:15:54 +02:00
kssenii
0db38fcbc9 Fix 2022-09-02 15:32:46 +02:00
alesapin
45fe9c800a
Merge pull request #40898 from georgthegreat/patch-2
Remove remaining usage of // Y_IGNORE magic comment
2022-09-02 12:24:57 +02:00
Robert Schulze
c7c00f9002
Merge pull request #40739 from ClickHouse/clang-tidy-for-headers
Enable clang-tidy for headers
2022-09-02 07:54:50 +02:00
Sergei Trifonov
29e922f05c support all profile events for disk s3 2022-09-02 02:22:52 +02:00
Sergei Trifonov
66b5cf566c fix bugs 2022-09-01 19:02:04 +02:00
Sergei Trifonov
ac9dc53c73 Merge branch 'master' into s3-detailed-metrics 2022-09-01 18:31:02 +02:00
Sergei Trifonov
bcb6475c4a add separate s3 profile events for disk s3 2022-09-01 18:30:55 +02:00
Yuriy Chernyshov
304dc30f0f
Remove remaining usage of // Y_IGNORE magic comment 2022-09-01 18:15:56 +03:00
kssenii
3134bdeb13 Fix 2022-09-01 16:35:28 +02:00
Nikita Mikhaylov
c7b0b87574
Better exception handling in DiskObjectStorageTransaction (#40813)
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-09-01 13:29:59 +02:00
Vitaly Baranov
6dcca686cb
Merge pull request #40827 from vitlibar/fix-incremental-backups-for-log-family
Fix incremental backups for Log family.
2022-09-01 13:13:32 +02:00
kssenii
82879543d3 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-01 13:13:01 +02:00
Kseniia Sumarokova
849262bd99
Merge pull request #40858 from kssenii/move-cache-files
Move cache files from Common
2022-09-01 12:28:00 +02:00
Robert Schulze
de64c6b103
Merge branch 'master' into clang-tidy-for-headers 2022-09-01 10:24:56 +02:00
Kruglov Pavel
b7ea529307
Merge pull request #40746 from Avogar/fix-csv-schema-inference
Fix possible error Attempt to read after eof in CSV schema inference
2022-08-31 18:29:01 +02:00
kssenii
0f747e7987 Move cache files 2022-08-31 17:50:55 +02:00
Vitaly Baranov
007ae0e6cc Fix incremental backups for Log family. 2022-08-31 12:57:28 +02:00
Robert Schulze
cedf75ed5e
Enable clang-tidy for headers
clang-tidy now also checks code in header files. Because the analyzer
finds tons of issues, activate the check only for directory "base/" (see
file ".clang-tidy"). All other directories, in particular "src/" are
left to future work.

While many findings were fixed, some were not (and suppressed instead).
Reasons for this include: a) the file is 1:1 copypaste of a 3rd-party
lib (e.g. pcg_extras.h) and fixing stuff would make upgrades/fixes more
difficult b) a fix would have broken lots of using code
2022-08-31 10:48:15 +00:00
kssenii
a5f69517ae Allow to load marks in threadpool 2022-08-30 20:20:05 +02:00
avogar
85fa372ad7 Fix possible error Attempt to read after eof in CSV schema inference 2022-08-29 11:18:53 +00:00
Azat Khuzhin
9fff08eac7 WriteBufferFromS3: remove unused ThreadGroupStatus
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-29 11:30:17 +02:00
Alexey Milovidov
6e564b18bf
Merge pull request #40600 from FrankChen021/check_url_arg
Validate the CompressionMethod parameter of URL table engine
2022-08-27 19:29:55 +03:00
alesapin
7d1e3ecfa3
Merge pull request #40506 from ClickHouse/fix_bad_exception
Fix lost errors in AWS CPP SDK
2022-08-25 13:07:35 +02:00
Kruglov Pavel
66e677b307
Merge pull request #40546 from Avogar/fix-inferring-datetimes
Fix possible error 'Decimal math overflow' while parsing DateTime64
2022-08-25 11:18:30 +02:00
Frank Chen
4c1fe0d623 validate parameter
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-25 14:49:17 +08:00
Alexey Milovidov
aaafb797c0
Merge pull request #40541 from ClickHouse/paranoid_check_for_s3_uploads
Add paranoid option to check blobs to S3 after upload
2022-08-24 23:51:22 +02:00
Kseniia Sumarokova
4e7578cf3f
Merge pull request #40565 from FrankChen021/url_engine
Fix segment fault when writing data to URL table engine
2022-08-24 20:36:45 +02:00
Alexander Gololobov
6c256cbe4a
Merge pull request #40474 from ClickHouse/allow-to-parse-date-as-datetime
Allow parsing Date as DateTime and DateTime64
2022-08-24 17:23:44 +02:00
alesapin
f2be18b3a6 Remove trailing 2022-08-24 16:57:59 +02:00
alesapin
f5f0934c13
Update PocoHTTPClient.h 2022-08-24 16:15:43 +02:00
alesapin
e3672d1d40 Fix memsan 2022-08-24 15:25:10 +02:00
alesapin
f0b733313f Merge branch 'master' into fix_bad_exception 2022-08-24 14:44:40 +02:00
alesapin
cac3005f4d Merge branch 'master' into paranoid_check_for_s3_uploads 2022-08-24 14:43:11 +02:00
Kruglov Pavel
109f384e5d
Fix typo 2022-08-24 13:40:19 +02:00
Frank Chen
b432b0d100
Merge branch 'master' into url_engine 2022-08-24 17:59:21 +08:00
Frank Chen
335293549b Fix segment fault when sending data to remote HTTP server via URL table engine
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-24 17:38:16 +08:00
avogar
de489ba1f7 Fix possible error 'Decimal math overflow' while parsing DateTime64 2022-08-23 14:42:23 +00:00
alesapin
07286dd042 Add paranoid option to check blobs to S3 after upload 2022-08-23 15:12:09 +02:00
Kseniia Sumarokova
d814a56277
Merge pull request #40420 from kssenii/fix-cache-bug
Fix incorrect fallback to skip cache in case of very high concurrency level
2022-08-23 14:18:38 +02:00
alesapin
a646db4665 Better 2022-08-23 13:56:41 +02:00
alesapin
eef80b0d06 Fix message 2022-08-23 13:23:53 +02:00
alesapin
d97f3b012a Better 2022-08-23 13:20:05 +02:00
alesapin
802d56df49 Merge branch 'master' into fix_bad_exception 2022-08-23 12:22:05 +02:00
Kruglov Pavel
54ed9029b8
Merge pull request #40482 from ClickHouse/Avogar-patch-1
Fix bug in HadoopSnappyReadBuffer
2022-08-23 11:47:59 +02:00
alesapin
61d014f6ec
Merge branch 'master' into fix_bad_exception 2022-08-23 00:35:29 +02:00
alesapin
780f39b108 Fix lost errors in AWS CPP SDK 2022-08-23 00:34:39 +02:00
Kruglov Pavel
3666b25e6b
Fix 2022-08-22 18:48:32 +02:00
Kruglov Pavel
2636c032c4
Use loop instead of recursion 2022-08-22 18:48:02 +02:00
Kseniia Sumarokova
f85e5f8808
Merge branch 'master' into fix-cache-bug 2022-08-22 16:54:07 +02:00
Kruglov Pavel
ae81341ab5
Fix hadoop unit test 2022-08-22 12:42:18 +02:00
Kruglov Pavel
019686f6e1
Fix bug in HadoopSnappyReadBuffer 2022-08-22 12:40:58 +02:00
Alexey Milovidov
75c0ed528d Fix test 2022-08-22 01:21:32 +02:00
Alexey Milovidov
2faa23112c Fix test 2022-08-22 00:39:06 +02:00
kssenii
26384b2543 Fix 2022-08-22 00:02:05 +02:00
Alexey Milovidov
bddf1cc312 Allow to parse Date as DateTime and DateTime64 2022-08-21 23:12:41 +02:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Sema Checherinda
f8480c26e7 do not trigger allocator to return null under sanitizers 2022-08-19 11:22:34 +02:00
Sema Checherinda
128e1fec3d Memory don't do alignment by itself, Allocator does 2022-08-18 16:01:30 +02:00
Sema Checherinda
b101ebdf32 check integer overflow at Memory class 2022-08-18 14:30:52 +02:00
Sema Checherinda
89bada927c fix heap buffer overflow by limiting http chunk size 2022-08-18 14:30:52 +02:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache
Add schema inference cache for s3/hdfs/file/url
2022-08-18 13:07:50 +02:00
kssenii
d63d34f7bb Merge master 2022-08-17 15:49:32 +02:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Maksim Kita
58c9270a19
Merge branch 'master' into write-buffer-from-s3-potential-deadlock-fix 2022-08-15 19:20:55 +02:00
Maksim Kita
ee5e61c9a5 Fixed tests 2022-08-15 11:09:39 +02:00
Alexey Milovidov
914f4b8155
Merge pull request #40179 from HarryLeeIBM/hlee-s390x-file-enc
Fix Endian issue in FileEncryption for s390x
2022-08-13 22:48:48 +03:00
HarryLeeIBM
296835e2d3 Fix Endian issue in FileEncrption for s390x 2022-08-12 18:55:40 -07:00
Alexey Milovidov
6b8f4f1d59 Merge branch 'master' into server-side-time 2022-08-12 21:36:40 +02:00
Alexander Tokmakov
a9c6a883dc
Merge pull request #40158 from ClickHouse/make_tests_less_flaky
Set sync_request_timeout to 10 to avoid reconnections in tests
2022-08-12 22:08:53 +03:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
New caching strategies
2022-08-12 17:23:37 +02:00
Alexey Milovidov
8c3d1f2b5b
Merge branch 'master' into server-side-time 2022-08-12 17:40:02 +03:00
Alexander Tokmakov
fe7e8dbc3c fix my favorite ctor 2022-08-12 16:28:21 +02:00
Alexander Tokmakov
0395dca7be set sync_request_timeout to 10 to avoid reconnections in tests 2022-08-12 15:30:46 +02:00
kssenii
768a207176 Merge master 2022-08-11 22:25:53 +02:00
Maksim Kita
6a7dbc556c WriteBufferFromS3 potential deadlock fix 2022-08-11 18:22:01 +02:00
Kseniia Sumarokova
be69169f97
Merge pull request #38048 from KinderRiven/decoupling_cache_function_and_algorithm
Decoupling local cache function and cache algorithm
2022-08-11 16:49:56 +02:00
Alexey Milovidov
0b24dbaec8 Merge 2022-08-10 19:41:44 +02:00
KinderRiven
9d83b93e88 fix rebase 2022-08-11 00:11:40 +08:00
Alexey Milovidov
c283351a5f Fix error 2022-08-10 09:31:24 +02:00
Alexey Milovidov
5bc85a6178 Do not write elapsed_ns in HTTP headers 2022-08-10 09:09:04 +02:00
Tanya Bragin
eb61db3b67 initial changes to close #37492 2022-08-08 19:45:00 -07:00
alexX512
c9c26d4898 Fix review 2022-08-08 21:34:24 +00:00
Alexey Milovidov
db85ebfaa7 Add server-side time to Progress 2022-08-08 06:55:41 +02:00
alexX512
62f84b88eb Review fixes 2022-08-07 20:02:21 +00:00
alexX512
6bf29cb610 Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
alexX512
1133e42367 Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf 2022-08-07 19:16:40 +00:00
kssenii
7a9b0bc47f Merge master 2022-08-05 01:48:52 +02:00
Kseniia Sumarokova
bd5610b732
Merge branch 'master' into update-azure-lib 2022-08-03 23:00:57 +02:00
Alexey Milovidov
316528817b
Merge pull request #39179 from azat/fsync-profile-events
Add profile events for fsync
2022-08-02 05:44:35 +03:00
Azat Khuzhin
3e627e2861 Add profile events for fsync
The following new provile events had been added:

- FileSync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.
- DirectorySync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.
- FileSyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.
- DirectorySyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.

v2: rewrite test to sh with retries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-31 23:19:30 +03:00
kssenii
e5f4a619ed Merge master 2022-07-31 20:24:40 +03:00
kssenii
687f0fabfc Move some dependencies 2022-07-31 20:17:22 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Alexey Milovidov
22bc89690d
Merge pull request #39222 from azat/fix-http-session
Do not report "Failed communicating with" on and on for parts exchange
2022-07-31 02:24:05 +03:00
Vitaly Baranov
2f3b27d1ac Fix seeking while reading from encrypted disk. 2022-07-29 12:45:29 +02:00
Vitaly Baranov
3b6bacbf0b Add tests. 2022-07-28 22:38:50 +02:00
Vladimir C
ee515b8862
Merge pull request #39656 from HarryLeeIBM/hlee-s390x-bit-helpers
Fix Endian issue in BitHelpers for s390x
2022-07-28 13:31:14 +02:00
Vladimir C
a8be51f4df
Merge pull request #39567 from Algunenano/progress_result_rows_and_bytes
Add result_rows and result_bytes to progress reports
2022-07-27 16:43:55 +02:00