Commit Graph

476 Commits

Author SHA1 Message Date
robot-ch-test-poll1
abc48a0b73
Merge pull request #52627 from ClickHouse/fix_build_with_clang15
Fix build with clang-15
2023-07-27 09:49:29 +02:00
Илья Коргун
5395d34fd7 Fix missing constructor call 2023-07-27 10:32:10 +03:00
Alexander Tokmakov
74f3e76b18 fix build with clang-15 2023-07-26 17:15:28 +02:00
Kruglov Pavel
0d34e97dbe
Merge branch 'master' into formats-with-subcolumns 2023-07-26 13:30:35 +02:00
Kruglov Pavel
fec5675cd4
Merge branch 'master' into better-progress-bar-2 2023-07-24 19:59:38 +02:00
Kseniia Sumarokova
d34f2bed07
Merge pull request #51386 from bigo-sg/fix_hdfs_read_buffer_heap_overflow
Fix heap overflow in read buffer from hdfs when read_until_position is not zero
2023-07-22 15:00:58 +02:00
Kseniia Sumarokova
6ea96f0686
Merge branch 'master' into minor_improve_hdfs_read_buffer 2023-07-22 14:37:56 +02:00
avogar
3682254342 Fix hdfs tests 2023-07-18 13:49:23 +00:00
Kruglov Pavel
64e88cde21
Merge branch 'master' into better-progress-bar-2 2023-07-18 13:37:53 +02:00
Kruglov Pavel
1dd05319b5
Merge branch 'master' into formats-with-subcolumns 2023-07-17 19:13:42 +02:00
李扬
6aec25e681
Merge branch 'master' into fix_hdfs_read_buffer_heap_overflow 2023-07-11 10:03:09 +08:00
Andrey Zvonov
c030e9dbe4
Merge branch 'master' into zvonand-issue-49290 2023-07-10 11:01:26 +02:00
avogar
810d1ee069 Fix tests 2023-07-06 13:48:57 +00:00
avogar
d11cd0dc30 Fix tests 2023-07-05 17:56:03 +00:00
avogar
98aa6b317f Support reading subcolumns from file/s3/hdfs/url/azureBlobStorage table functions 2023-07-04 21:17:26 +00:00
Kseniia Sumarokova
a8e2eb9686
Merge branch 'master' into minor_improve_hdfs_read_buffer 2023-07-01 13:19:21 +02:00
Kseniia Sumarokova
349083e673
Update ReadBufferFromHDFS.cpp 2023-06-30 13:54:57 +02:00
avogar
f88c1fdf44 Fixes 2023-06-29 11:59:09 +00:00
avogar
4eeb431003 Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-progress-bar-2 2023-06-28 18:53:08 +00:00
李扬
0e15266170
Merge branch 'master' into improve_geohash 2023-06-26 14:54:44 +08:00
taiyang-li
769169f820 fix heap overflow in read buffer from hdfs 2023-06-25 18:26:53 +08:00
Andrey Zvonov
0b6688ffe4
Merge branch 'master' into zvonand-issue-49290 2023-06-24 12:36:27 +02:00
avogar
938a9ee3c1 Better 2023-06-23 14:03:59 +00:00
avogar
c679dd400e Make better 2023-06-23 13:43:40 +00:00
avogar
24fab7bfde Remove old includes 2023-06-22 18:48:15 +00:00
avogar
cf082f2f9a Use read_bytes/total_bytes_to_read for progress bar in s3/file/url/... table functions 2023-06-22 17:24:43 +00:00
Andrey Zvonov
18e7b029d5
Merge branch 'master' into zvonand-issue-49290 2023-06-22 17:20:24 +02:00
Sema Checherinda
d0bb985061 fix other classes based on SinkToStorage 2023-06-22 14:33:25 +02:00
Sema Checherinda
95349a405b release buffers with exception context 2023-06-22 13:00:13 +02:00
Sema Checherinda
01de36f1fa
Merge pull request #50395 from CheSema/better-log
require `finalize()` call before d-tor for all writes buffers
2023-06-21 21:12:02 +02:00
avogar
1d80072c21 Fix tests and builds 2023-06-20 13:01:47 +00:00
zvonand
4e61f93fde upd 2023-06-19 23:44:59 +02:00
zvonand
b25555d445 update style 2023-06-17 13:49:15 +02:00
zvonand
1c10578f14 update to master 2023-06-16 21:30:47 +02:00
Sema Checherinda
e8d4dedbcc fix build and tests 2023-06-16 19:52:52 +02:00
avogar
3209ebe34b Improve progress bar for file/s3/hdfs/url table functions. Step 1 2023-06-16 15:51:18 +00:00
avogar
870f3d1270 Fix comments 2023-06-15 12:59:46 +00:00
Andrey Zvonov
2f572b7211
Merge branch 'master' into zvonand-issue-49290 2023-06-14 14:22:32 +02:00
zvonand
1d8013074d fix style & black 2023-06-13 18:47:02 +02:00
avogar
2e1f56ae33 Address comments 2023-06-13 14:43:50 +00:00
Andrey Zvonov
cd1a3916a6 cleanup HDFS 2023-06-12 13:28:17 +02:00
zvonand
7d7bd5bb66 update comment describing workflow 2023-06-12 11:50:23 +02:00
zvonand
2c97a94892 fix hdfs + style update 2023-06-11 01:50:17 +02:00
Kruglov Pavel
bf28074d32
Merge branch 'master' into allow-skip-empty-files 2023-06-08 12:36:18 +02:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
Kruglov Pavel
f99a7366da
Fix tests 2023-06-01 16:56:18 +02:00
Kruglov Pavel
0beca0336d
Merge pull request #49112 from ClickHouse/Avogar-patch-3
Fix possible terminate called for uncaught exception in some places
2023-05-31 16:55:43 +02:00
avogar
d4efbbfbd3 Allow to skip empty files in file/s3/url/hdfs table functions 2023-05-30 19:32:24 +00:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
kevinyhzou
15b847b410 init file_size as 0 2023-05-24 09:49:12 +08:00
Kruglov Pavel
b5cad024e0
Merge branch 'master' into urlCluster 2023-05-22 14:59:34 +02:00
taiyang-li
1f91a75b54 fix bugs of geoHashEncode 2023-05-22 15:52:58 +08:00
kevinyhzou
e6d6d41f22 minor improve read buffer of hdfs 2023-05-22 12:23:31 +08:00
kssenii
3e42ee7f2b Get rid of finalize callback in object storages 2023-05-19 17:29:37 +02:00
avogar
334f062fa0 fix style 2023-05-15 16:39:26 +00:00
avogar
f9e0eb47d7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-15 16:35:24 +00:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
avogar
70a8fd2c50 Fix schema inference with named collection, refactor Cluster table functions 2023-05-12 13:58:45 +00:00
avogar
3ee8de792c Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-11 12:46:20 +00:00
kssenii
ecfbf1e304 Remove dependency from DB::Context in readers 2023-05-02 21:45:27 +02:00
avogar
c503f6532c Add more finalize() to avoid terminate 2023-04-24 15:11:36 +00:00
avogar
2949ceced1 Fix adding structure to cluster table functions, make it better 2023-04-24 13:20:04 +00:00
Alexey Milovidov
31000978f4
Merge branch 'master' into improve_cluster_table_funcion_skip_unavailable_shards 2023-04-23 16:18:48 +03:00
avogar
447189a6ca Better 2023-04-21 17:54:09 +00:00
avogar
944f54aadf Finish urlCluster, refactor code, reduce code duplication 2023-04-21 17:24:37 +00:00
Kruglov Pavel
2ad161d2b7
Merge branch 'master' into non-blocking-connect 2023-04-19 13:39:40 +02:00
Alexey Milovidov
4d74e3da4d
Merge branch 'master' into improve_cluster_table_funcion_skip_unavailable_shards 2023-04-12 22:13:52 +03:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 17:59:30 +02:00
Alexey Milovidov
db61140607
Merge branch 'master' into improve_cluster_table_funcion_skip_unavailable_shards 2023-04-09 08:03:23 +03:00
Kruglov Pavel
3ee12e21fb
Merge branch 'master' into non-blocking-connect 2023-03-23 20:53:44 +01:00
avogar
38e44861ae Fix possible race conditions 2023-03-21 16:01:54 +00:00
Robert Schulze
5b036a1a3b
More preparation for libcxx(abi), llvm, clang-tidy 16 (follow-up to #47722) 2023-03-20 12:55:03 +00:00
Alexander Tokmakov
ed08f8f5c5
Merge branch 'master' into revert_25674 2023-03-12 02:33:25 +03:00
Alexander Tokmakov
7b1b238d0b Revert "Merge pull request #25674 from amosbird/distributedreturnconnection"
This reverts commit 5ffd99dfd4, reversing
changes made to 2796aa333f.
2023-03-11 19:09:47 +01:00
Maksim Kita
0358cb36d8 Fixed tests 2023-03-11 11:51:54 +01:00
zk_kiger
4f05eeeff9
Merge branch 'master' into improve_cluster_table_funcion_skip_unavailable_shards 2023-02-27 17:15:02 +08:00
kigerzhang
f74f3718a0 add skip_unavailable_shards setting for table function cluste 2023-02-23 17:05:51 +08:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Sergei Trifonov
2931c3bbe1
Merge pull request #45711 from ClickHouse/io-scheduler-integration
Integrate IO scheduler with buffers for remote reads and writes
2023-02-11 10:26:58 +01:00
serxa
8c3f85e58d more review fixes 2023-02-10 17:39:28 +00:00
serxa
acabee8554 review fixes 2023-02-09 20:17:46 +00:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Sergei Trifonov
b0fc551c27
Merge branch 'master' into io-scheduler-integration 2023-02-08 18:16:00 +01:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
2023-02-05 20:49:34 +00:00
Raúl Marín
7c31cb7adc Proper includes for ConnectionTimeoutsContext.h 2023-01-31 16:11:32 +01:00
Sergei Trifonov
5c2686fbf0
Merge branch 'master' into io-scheduler-integration 2023-01-27 19:56:23 +01:00
serxa
21e871a0a4 integrate IO scheduler with buffers 2023-01-27 18:47:22 +00:00
Anton Popov
b58b73b0e7
Merge pull request #45529 from CurtizJ/fix-storage-s3-race
Try to fix test `test_storage_s3/test.py::test_wrong_s3_syntax` (race in `StorageS3`)
2023-01-26 14:21:32 +01:00
Kruglov Pavel
cd3d3fdd68
Merge pull request #45483 from Avogar/fix-hdfs-cluster
Fix schema inference from insertion table in hdfsCluster
2023-01-25 20:24:51 +01:00
avogar
07ad750366 Fix tests 2023-01-24 15:08:19 +00:00
Anton Popov
5c0307bc6a fix race in StorageURL and StorageHDFS 2023-01-24 12:34:43 +00:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Alexander Gololobov
91fb1eab8c
Merge pull request #45373 from ClickHouse/throttler-metrics
Add detailed profile events for throttling
2023-01-22 11:20:45 +01:00
avogar
eed1db7e07 Fix schema inference in hdfsCluster 2023-01-20 21:17:35 +00:00
Anton Popov
f40fd7a151
Add checks for compilation of regexps (#45356) 2023-01-17 23:46:04 +01:00
serxa
ce7e22b87b add detailed profile events for throttling 2023-01-17 18:29:24 +00:00
Maksim Kita
fbba28b31e Analyzer aggregation without column fix 2023-01-10 16:49:55 +01:00
Kseniia Sumarokova
b22b51213f
Merge pull request #44868 from kssenii/improve-async-buffer-from-remote
Small perf improvement for async buffer from remote
2023-01-09 13:31:15 +01:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] (#44955)
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)""

This reverts commit b8d9066004.

* Fix build

* Better

* Fix test

* Automatic style fix

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-01-09 13:30:32 +01:00
kssenii
bda8f0d410 Better async buffer from remote 2023-01-03 15:27:37 +01:00