Commit Graph

5875 Commits

Author SHA1 Message Date
Igor Nikonov
7c4d197ca5
Merge branch 'master' into pr-cleanup-remote-source 2023-11-08 19:38:15 +01:00
Yarik Briukhovetskyi
b8ea31db43
Merge pull request #56309 from yariks5s/preferred_optimize_projection_name
New setting preferred_projection_name
2023-11-08 18:06:03 +01:00
Nikolai Kochetov
7eacea4c51
Update src/Processors/QueryPlan/Optimizations/useDataParallelAggregation.cpp
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-11-08 17:28:57 +01:00
taiyang-li
6199bb83a9 Merge branch 'master' into orc_tuple_field_prune 2023-11-08 14:54:12 +08:00
yariks5s
6f6454e696 style check 2023-11-07 18:12:56 +00:00
yariks5s
181231d500 init 2023-11-07 17:56:02 +00:00
Robert Schulze
32a3b1b049
Merge remote-tracking branch 'rschu1ze/master' into idatatype-convenience-functions 2023-11-07 15:17:59 +00:00
Sema Checherinda
02df5ac59c
Merge pull request #54340 from KevinyhZou/bug_fix_csv_parse_crlf
Fix CSV read while the end of line is CR(\r)
2023-11-07 12:51:05 +01:00
Robert Schulze
54b5d573d7
Merge remote-tracking branch 'rschu1ze/master' into idatatype-convenience-functions 2023-11-07 10:18:30 +00:00
yariks5s
2e03462d87 style fix 2023-11-06 22:26:35 +00:00
edef
a52e22a858 Support reading arrow::LargeListArray 2023-11-06 21:17:29 +00:00
yariks5s
faba36225e fixed tests 2023-11-06 19:10:00 +00:00
yariks5s
ec32c23f0c style check 2023-11-06 15:25:14 +00:00
yariks5s
7206419a6b Merge branch 'preferred_optimize_projection_name' of github.com:yariks5s/ClickHouse into preferred_optimize_projection_name 2023-11-06 14:30:33 +00:00
yariks5s
e48e6b6a5a fixed review, changed tests 2023-11-06 14:30:10 +00:00
Robert Schulze
8896134531
Cleanup IDataType convenience functions 2023-11-06 13:25:32 +00:00
Alexander Tokmakov
eba52b318d
Merge pull request #56272 from ClickHouse/rdb_fix_rare_logical_error
Fix rare logical error in Replicated database
2023-11-06 14:14:55 +01:00
kevinyhzou
2a50daf5dd Allow cr at end of csv line 2023-11-06 12:21:42 +08:00
kevinyhzou
ef30e6723d bug fix csv read while end of line is not crlf 2023-11-06 12:21:42 +08:00
Igor Nikonov
9bdcba1b20
Merge branch 'master' into pr-progress-bar 2023-11-05 00:09:46 +01:00
Alexey Milovidov
36d18acf35
Merge pull request #55007 from Alex-Cheng/refactor-MergeJoinTransform
Refactor merge join transform
2023-11-04 17:08:07 +01:00
taiyang-li
8e23dd9090 add some comments 2023-11-04 10:08:26 +08:00
taiyang-li
8a6ae6e150 change as request 2023-11-04 09:41:01 +08:00
Dmitry Novik
23564148f9 Merge remote-tracking branch 'origin/master' into parallel-window 2023-11-03 17:53:51 +00:00
yariks5s
6d9b517b19 fix suggestions 2023-11-03 17:21:37 +00:00
Yarik Briukhovetskyi
92588dfac7
style fix 2023-11-03 18:07:19 +01:00
yariks5s
d95262622b init 2023-11-03 16:47:06 +00:00
Kruglov Pavel
7e4742f68a
Merge pull request #56167 from ClickHouse/revert-56150-revert-55989-lc-as-arrow-dict-fix
Revert "Revert "Fix output/input of Arrow dictionary column""
2023-11-03 16:18:10 +00:00
yariks5s
4235c690d4 init commit 2023-11-03 15:14:25 +00:00
vdimir
86524fce0f
Merge pull request #56246 from ClickHouse/vdimir/analyzer_01019_alter_materialized_view
Analyzer: fix 01019_alter_materialized_view_consistent
2023-11-03 11:04:14 +01:00
taiyang-li
4b7146d47e remove useless codes 2023-11-03 16:03:26 +08:00
Alexander Tokmakov
0ce1560e5d retry if table replica being restarted 2023-11-02 18:30:32 +01:00
Nikolai Kochetov
80d4e9a045 Merge branch 'master' into aggregate-by-partition-analyzer 2023-11-02 11:21:38 +00:00
Alexey Milovidov
1267246785
Merge pull request #55642 from ClickHouse/morton-utils
Indexing by space-filling curves
2023-11-02 12:21:33 +01:00
Nikolai Kochetov
5b6bf58730 Proper process aliases for aggregation-by-partition optimization. 2023-11-02 11:19:37 +00:00
vdimir
5703e0995b
Analyzer: fix 01019_alter_materialized_view_consistent 2023-11-02 10:27:22 +00:00
taiyang-li
819c7e75ff Merge branch 'master' into orc_tuple_field_prune 2023-11-02 17:52:05 +08:00
taiyang-li
6ccde98943 change as request 2023-11-02 11:01:50 +08:00
李扬
424f1bec93
Apply suggestions from code review
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
2023-11-02 10:30:28 +08:00
李扬
c899ff9da2
Apply suggestions from code review
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
2023-11-02 10:30:14 +08:00
Alexey Milovidov
697f2591b9 Merge with master 2023-11-02 00:33:33 +01:00
Kruglov Pavel
aa7c1f63ab
Merge pull request #56172 from Avogar/fix-schema-cache-for-json
Fix schema cache for fallback JSON->JSONEachRow with changed settings
2023-11-01 20:24:34 +01:00
Kruglov Pavel
e6f00d5e1d
Merge pull request #56117 from bigo-sg/fixed_struct_field_prune
Improve parquet struct fields reading
2023-11-01 15:32:50 +01:00
Kruglov Pavel
bf77ce691c
Merge pull request #55982 from yariks5s/npy_input_format
New input format Npy
2023-11-01 14:26:22 +01:00
taiyang-li
dc897215da fix failed uts tests/queries/0_stateless/02312_parquet_orc_arrow_names_tuples.sql 2023-11-01 20:42:07 +08:00
Nikolai Kochetov
4deaf7cefb
Merge pull request #56134 from yariks5s/force_optimize_projection_name
Implementing force_optimize_projection_name
2023-11-01 13:12:10 +01:00
taiyang-li
24c45a4ee0 fix failed uts 2023-11-01 18:47:11 +08:00
taiyang-li
001cbe7912 fix typos 2023-11-01 16:58:25 +08:00
taiyang-li
b276587422 fix failed uts 2023-11-01 15:43:20 +08:00
lgbo-ustc
8334585eaf improve parquet struct field reading 2023-11-01 15:18:39 +08:00
Alex Cheng
9e31858100
Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform 2023-11-01 09:32:28 +08:00
yariks5s
6c4bf59021 fix suggestions and enhance tests 2023-10-31 18:10:55 +00:00
avogar
518e52473d Fix schema cache for fallback JSON->JSONEachRow with changed settings 2023-10-31 14:12:38 +00:00
Kruglov Pavel
4faa3d0294
Revert "Revert "Fix output/input of Arrow dictionary column"" 2023-10-31 12:30:45 +01:00
Nikolai Kochetov
f748f12426
Merge pull request #51746 from ClickHouse/fix-read-in-order-with-array-join
Fix 'Cannot find column' in read-in-order optimization with ARRAY JOIN
2023-10-31 11:51:01 +01:00
Nikolai Kochetov
0523e6cbd0
Merge pull request #53282 from ClickHouse/planner-prepare-filters-for-analysis-2
Support skip_unused_shards in Analyzer
2023-10-31 11:33:59 +01:00
Alexey Milovidov
467b4d85e2
Revert "Fix output/input of Arrow dictionary column" 2023-10-31 09:28:09 +03:00
taiyang-li
c97b2c5be7 fix code style 2023-10-31 12:00:45 +08:00
taiyang-li
e5db57204d fix bugs 2023-10-31 11:57:47 +08:00
taiyang-li
b72341e1a8 Merge branch 'master' into orc_tuple_field_prune 2023-10-31 10:07:43 +08:00
Han Fei
1237cef26d address comments 2023-10-31 00:53:46 +01:00
Kruglov Pavel
4c2a132d96
Merge pull request #55989 from Avogar/lc-as-arrow-dict-fix
Fix output/input of Arrow dictionary column
2023-10-30 20:47:49 +01:00
Kruglov Pavel
4effc676f9
Merge pull request #56046 from Avogar/cr-in-unquoted-csv-string
Allow unquoted strings with CR in CSV format
2023-10-30 20:46:20 +01:00
Nikolai Kochetov
84f6a243b7 Merge branch 'master' into fix-read-in-order-with-array-join 2023-10-30 16:35:31 +00:00
yariks5s
03236c48ed init 2023-10-30 16:21:50 +00:00
yariks5s
9a2d89e3e4 removed getSize() and enhanced docs 2023-10-30 12:42:19 +00:00
taiyang-li
ad67b6c2ea allow tuple field pruning 2023-10-30 19:33:06 +08:00
Nikolai Kochetov
554ceb4e1d Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-10-30 11:56:30 +01:00
Maksim Kita
f8e209ebd2 WindowTransform decrease amount of virtual function calls 2023-10-30 13:50:25 +03:00
Kruglov Pavel
c10a3b3838
Merge branch 'master' into lc-as-arrow-dict-fix 2023-10-30 11:20:57 +01:00
avogar
57bc4854c2 Fix 2023-10-30 10:17:49 +00:00
avogar
d1fcbc6e47 Fix fetching schema from schema registry in AvroConfluent 2023-10-30 10:17:48 +00:00
Han Fei
19cdc6dfe2 Merge branch 'master' into hanfei/statistic 2023-10-30 04:53:18 +01:00
Alexey Milovidov
6ad027ed7a
Update ConstantExpressionTemplate.cpp 2023-10-30 05:32:36 +03:00
yariks5s
e14a7f066a fix typos 2023-10-28 01:46:59 +00:00
yariks5s
894724bfb3 suggested changes 2023-10-28 01:17:25 +00:00
yariks5s
23635352f1 fixed due to review 2023-10-27 15:43:03 +00:00
Kruglov Pavel
bb4b95e891
Merge branch 'master' into schema-inference-union 2023-10-27 14:53:58 +02:00
frinkr
18c50c11b3
Multithreading after window functions (#50771)
* feat: Preserve number of streams after evaluation the window functions to allow parallel stream processing

* fix style

* fix style

* fix style

* setting query_plan_preserve_num_streams_after_window_functions default true

* fix tests by SETTINGS query_plan_preserve_num_streams_after_window_functions=0

* fix test references

* Resize the streams after the last window function, to keep the order between WindowTransforms (and WindowTransform works on single stream anyway).

* feat: Preserve number of streams after evaluation the window functions to allow parallel stream processing

* fix style

* fix style

* fix style

* setting query_plan_preserve_num_streams_after_window_functions default true

* fix tests by SETTINGS query_plan_preserve_num_streams_after_window_functions=0

* fix test references

* Resize the streams after the last window function, to keep the order between WindowTransforms (and WindowTransform works on single stream anyway).

* add perf test

* perf: change the dataset from 50M to 5M

* rename query_plan_preserve_num_streams_after_window_functions -> query_plan_enable_multithreading_after_window_functions

* update test reference

* fix clang-tidy

---------

Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-10-27 12:36:28 +02:00
Kruglov Pavel
570b66f027
Merge branch 'master' into schema-inference-union 2023-10-26 19:26:00 +02:00
avogar
9d207bf027 Allow unquoted strings with CR in CSV format 2023-10-26 13:50:54 +00:00
Duc Canh Le
f7c0914c2b get rid of cloning single-row-chunk for last row
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-26 10:39:21 +00:00
zvonand
0766c73aab Rename date_time_overflow_mode -> date_time_overflow_behavior, moved it to format settings 2023-10-25 23:11:13 +02:00
zvonand
5b86e8c714 updated after review 2023-10-25 23:10:58 +02:00
zvonand
2f3695add8 Introduce setting for dt overflow exception
Added tests and docs
2023-10-25 23:10:24 +02:00
Alexey Milovidov
bb5a60dc19
Merge pull request #55893 from ClickHouse/revert-partial-result-2
Revert "Revert "Revert "Add settings for real-time updates during query execution"""
2023-10-25 22:20:28 +02:00
Igor Nikonov
1a5d485c6c
Merge branch 'master' into pr-progress-bar 2023-10-25 19:53:20 +02:00
yariks5s
2ab1ae42c1 added docs and tests, style check 2023-10-25 10:37:05 +00:00
Igor Nikonov
fafe452f1e
Merge branch 'master' into pr-cleanup-remote-source 2023-10-25 11:48:23 +02:00
robot-ch-test-poll1
ef78889aa2
Merge pull request #55952 from ClickHouse/disable_apply_deleted_mask
Added a setting to allow reading rows marked as deleted
2023-10-25 01:14:07 +02:00
Igor Nikonov
7b7f100250 Merge remote-tracking branch 'origin/master' into pr-progress-bar 2023-10-24 21:02:29 +00:00
Igor Nikonov
c3bd80f1c8 Set progress callback in constructor 2023-10-24 21:02:20 +00:00
Joanna Hulboj
7e3587034d Consume leading zeroes when parsing a number in ConstantExpressionTemplate 2023-10-24 21:08:05 +01:00
avogar
c080ba9d7e Fix output LowCardinality as Arrow dictionary 2023-10-24 19:49:04 +00:00
李扬
465962df7f
Support orc filter push down (file + stripe + rowgroup level) (#55330)
* support orc filter push down

* update orc lib version

* replace setqueryinfo with setkeycondition

* fix issue https://github.com/ClickHouse/ClickHouse/issues/53536

* refactor source with key condition

* fix building error

* remove std::cout

* update orc

* update orc version

* fix bugs

* improve code

* upgrade orc lib

* fix code style

* change as requested

* add performance tests for orc filter push down

* add performance tests for orc filter push down

* fix all bugs

* fix default as null issue

* add uts for null as default issues

* upgrade orc lib

* fix failed orc lib uts and fix typo

* fix failed uts

* fix failed uts

* fix ast fuzzer tests

* fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html

* fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm

* fix wrong performance tests

* disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html

* add some comments

* add some comments

* inline range::equals and range::less

* fix data race of key condition

* trigger ci
2023-10-24 12:08:17 -07:00
yariks5s
4e09fb3e27 made_logic 2023-10-24 14:55:52 +00:00
Igor Nikonov
2c055480d6 Remove unnecessary flag 2023-10-24 14:52:47 +00:00
Kruglov Pavel
9a56ec4d63
Merge pull request #55891 from Avogar/try-fix-orc
Try to fix possible segfault in Native ORC input format
2023-10-24 13:57:43 +02:00
Alexander Gololobov
959b8b64bd Added a setting to allow reading rows marked as deleted 2023-10-23 19:59:17 +02:00
Robert Schulze
bef5cea5a0
Merge remote-tracking branch 'rschu1ze/master' into string-addinterval 2023-10-23 14:08:18 +00:00
Igor Nikonov
0bbf7ba921
Merge branch 'master' into pr-progress-bar 2023-10-23 14:56:59 +02:00
Kruglov Pavel
32da07b4ee
Merge pull request #55695 from Avogar/restrict-iceberg
Fixes and improvements for Iceberg storage
2023-10-23 13:58:23 +02:00
Alex Cheng
49b3136491
Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform 2023-10-23 09:56:22 +08:00
Robert Schulze
c2215cb5f0
Support string arguments 2023-10-22 09:28:45 +00:00
Alexey Milovidov
5217d64551 Remove garbage 2023-10-22 01:53:50 +02:00
Alexey Milovidov
1be4ff229b
Merge pull request #55895 from ClickHouse/fix-window-functions-with-sparse-columns
Fix window functions in case of sparse columns.
2023-10-22 00:02:34 +02:00
János Benjamin Antal
1b0cadebd0 Convert sparse columns to full for window functions 2023-10-21 11:46:25 +00:00
Igor Nikonov
e378d920c4
Merge branch 'master' into pr-progress-bar 2023-10-21 13:16:12 +02:00
Alexey Milovidov
7ec4b99e94 Revert partial result 2023-10-21 03:14:22 +02:00
Kruglov Pavel
6f61ccfe28
Merge branch 'master' into schema-inference-union 2023-10-20 22:54:11 +02:00
avogar
6934e27e8b Add union mode for schema inference to infer union schema of files with different schemas 2023-10-20 20:46:41 +00:00
avogar
8cc0dc17eb Try to fix possible segfault in Native ORC input format 2023-10-20 18:50:48 +00:00
yariks5s
87f26f5132 dealt with 2dim arrays 2023-10-20 17:05:05 +00:00
Anton Popov
5819bcd07a
Support asynchronous inserts for native protocol (#54730)
* support async insert for native protocol

* use separate queue for async inserts via native protocol

* fix test

* better logging for async inserts and more tests

* disable mixed internal and external data in async inserts

* fix tests

* fix quota in async inserts

* disable async insert for secondary query of distributed
2023-10-20 18:39:48 +02:00
Raúl Marín
d08ce13107
Merge pull request #51692 from Algunenano/parallel_replicas_row_estimation
Automatic decision of nº of parallel replicas
2023-10-20 15:50:43 +02:00
Azat Khuzhin
d5c960ceb6 Fix data race in CreatingSetsTransform
The problem is that findOrPromiseToBuild() can also call
shared_future::get():

    (lldb) bt 10
    * thread 194, name = 'MergeMutate', stop reason = step over
      * frame 0: 0x0000564126046bd2 clickhouse`std::rethrow_exception(p=(__ptr_ = 0x00007b5000130c80)) at exception_pointer_cxxabi.ipp:68:39
        frame 1: 0x000056411c4c5ddd clickhouse`std::__1::__assoc_state<std::__1::shared_ptr<DB::Set>>::copy(this=<unavailable>) at future:701:9
        frame 2: 0x000056411c4c0e3f clickhouse`DB::PreparedSetsCache::findOrPromiseToBuild(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) [inlined] std::__1::shared_future<std::__1::shared_ptr<DB::Set>>::get[abi:v15000](this=0x00007b0c00154eb8) const at future:2243:46
        frame 3: 0x000056411c4c0e2e clickhouse`DB::PreparedSetsCache::findOrPromiseToBuild(this=0x00007b1c00077018, key="__set_10438532338982543705_15680061365857338647") at PreparedSets.cpp:349:116

Note, that my test did not find the problem because of #55261 had not
been merged yet.

Follow-up for: #55338
Fixes: #55279
v2: s/createDeprecated/createRuntime/

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-19 12:26:02 +02:00
yariks5s
6dc88a4ca4 new changes 2023-10-18 18:02:05 +00:00
Dmitry Novik
5ca43e51c1 Remove unused code 2023-10-17 20:45:00 +00:00
avogar
323486f9e8 Add tests 2023-10-17 18:10:47 +00:00
Han Fei
4b23142775
Merge branch 'master' into hanfei/statistic 2023-10-17 16:40:31 +02:00
Raúl Marín
3eaf752284 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-17 11:36:39 +02:00
Alex Cheng
ce493b57f0
Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform 2023-10-17 16:42:53 +08:00
Igor Nikonov
4b36761f89
Merge branch 'master' into pr-progress-bar 2023-10-17 10:36:32 +02:00
Michael Kolupaev
ce7eca0615
DWARF input format (#55450)
* Add ReadBufferFromFileBase::isRegularLocalFile()

* DWARF input format

* Review comments

* Changed things around ENABLE_EMBEDDED_COMPILER build setting

* Added 'ranges' column

* no-msan no-ubsan
2023-10-16 17:00:07 -07:00
Jiebin Sun
df17cd467b
Release more num_streams if data is small (#53867)
* Release more num_streams if data is small

Besides the sum_marks and min_marks_for_concurrent_read, we could also involve the
system cores to get the num_streams if the data is small. Increasing the num_streams
and decreasing the min_marks_for_concurrent_read would improve the parallel performance
if the system has plentiful cores.

Test the patch on 2x80 vCPUs system. Q39 of clickbench has got 3.3x performance improvement.
Q36 has got 2.6x performance improvement. The overall geomean has got 9% gain.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

* Release more num_streams if data is small
Change the min marks from 4 to 8 as the profit is small and 8 granules
is the default block size.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

---------

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2023-10-16 18:41:38 +02:00
Raúl Marín
f799f5d7a1 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-16 15:07:11 +02:00
Igor Nikonov
fdd7bfbb83 Merge remote-tracking branch 'origin/master' into pr-progress-bar 2023-10-16 12:59:00 +00:00
alesapin
3b02748cb6 Fix some typos 2023-10-15 15:43:02 +02:00
Duc Canh Le
285ae778e4 Merge branch 'master' into final_no_copy
Fix flaky test 02447_drop_database_replica
2023-10-14 03:34:42 +00:00
Anton Popov
1c9d883d8a Merge remote-tracking branch 'upstream/master' into HEAD 2023-10-13 17:17:01 +00:00
Alexander Tokmakov
e3e105d154
Merge pull request #55527 from azat/values-eof-check-fix
Fix checking of non handled data for Values format
2023-10-13 18:07:02 +02:00
yariks5s
cb08da617f added read and parse impl 2023-10-13 15:16:07 +00:00
Dmitry Novik
d4be706ed4 Merge remote-tracking branch 'origin/master' into parallel-window 2023-10-13 13:20:38 +00:00
Raúl Marín
500dab9569 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-13 10:12:57 +02:00
Duc Canh Le
f7830bc13a some low level optimization
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-13 07:41:49 +00:00
Anton Popov
a01acf5d2a remove projection from StorageSnapshot 2023-10-13 03:18:43 +00:00
Alexey Milovidov
8a1363bcf1
Merge pull request #49486 from bigo-sg/test_hive_null_as_default
Set defaults_for_omitted_fields to true for hive text format
2023-10-13 02:01:09 +02:00
Anton Popov
fb666e137c remove old code of projection analysis 2023-10-12 23:25:20 +00:00
Dmitry Novik
cec02c60e8 Merge remote-tracking branch 'origin/master' into parallel-window 2023-10-12 22:45:30 +00:00
Igor Nikonov
b220df93df Remove unnecessary total rows update in parallel replicas 2023-10-12 21:33:25 +00:00
yariks5s
9ae025d7e6 mid commit 2023-10-12 17:37:59 +00:00
Alexey Milovidov
b53a98217c
Merge pull request #55489 from kitaisreal/external-aggregation-improve-performance
Improve performance of external aggregation with a lot of temporary files
2023-10-12 18:00:41 +02:00
Alexey Milovidov
e48fad0e24
Merge pull request #55495 from kitaisreal/aggregating-transform-init-generate-race-condition-fix
AggregatingTransform initGenerate race condition fix
2023-10-12 17:54:36 +02:00
Dmitry Novik
f31a6f2c0d Support only Full Sort 2023-10-12 14:49:16 +00:00
Han Fei
42dc61abf5 Merge branch 'master' into hanfei/statistic 2023-10-12 16:19:45 +02:00
Azat Khuzhin
2cbb069b68 Add ability to ignore data after semicolon in Values format
This is required for client, to handle comments in multiquery mode.

v0: separate context for input format
v2: cannot use separate context since params and stuff are changed in global context
v3: do not sent this setting to the server (breaks queries for readonly profiles)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-12 14:55:26 +02:00
Antonio Andelic
22f5d6d708 Reset promise_to_build 2023-10-12 10:57:21 +00:00
Azat Khuzhin
f379d9cac5 Fix checking of non handled data for Values format
PeekableReadBuffer::hasUnreadData() does not checks the underlying
buffer, and so it simply ignore some issues, like:

    INSERT INTO test_01179_str values ('foo'); ('bar')

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-12 12:25:08 +02:00
Duc Canh Le
b3f4daf6c8 avoid casting columns
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-12 07:40:41 +00:00
Duc Canh Le
93a4d69c05 combine FINAL selection and WHERE filter
Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-12 04:58:42 +00:00
taiyang-li
7cd24d0af0 solve conflicts 2023-10-12 10:30:05 +08:00
Robert Schulze
7b14392675
Merge pull request #55481 from rschu1ze/better-use-mysql-types-in-show-columns
Make `use_mysql_types_in_show_columns` affect only `SHOW COLUMNS`
2023-10-12 00:36:46 +02:00
Maksim Kita
ae94055a29 Updated implementation 2023-10-11 19:01:00 +03:00
Maksim Kita
f9033bdf31 Improve performance of external aggregation with a lot of temporary files 2023-10-11 19:01:00 +03:00
Igor Nikonov
62060a0603 Merge remote-tracking branch 'origin/master' into pr-coordinator-usage-cleanup 2023-10-11 15:11:15 +00:00
Robert Schulze
97d8e16e8d
Fix MySQL packet row data type
Cf. https://github.com/ClickHouse/ClickHouse/pull/55479#discussion_r1355175581
2023-10-11 15:09:50 +00:00
Igor Nikonov
9d95f4e1b6 Cleanup: parallel replica coordinator usage 2023-10-11 15:04:59 +00:00
robot-ch-test-poll1
80aa9cad51
Merge pull request #55479 from slvrtrn/fix-mysql-datetime-incompatibilities
Fix MySQL text protocol DateTime formatting and LowCardinality(Nullable(T)) types reporting
2023-10-11 16:56:26 +02:00
Raúl Marín
0b9bd809e7 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-11 16:50:22 +02:00
Maksim Kita
0859fc8de9 AggregatingTransform initGenerate race condition fix 2023-10-11 14:28:22 +03:00
Duc Canh Le
dfc8793b5c refactor implementation
Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-11 10:30:38 +00:00
Duc Canh Le
c6aa438bad pass all stateless tests
Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-11 06:51:42 +00:00
slvrtrn
e06d3ca1a5 Fix MySQL text protocol DateTime
Introduce `removeLowCardinalityAndNullable` function
Fix incorrect removeLowCar/removeNullable usages
Add more MySQL text protocol tests
Deprecate old Java client tests
Use JDK 17 for test MySQL Java container
2023-10-10 19:51:09 +02:00
Raúl Marín
95d2063e91 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-10 17:29:45 +02:00
Nikita Mikhaylov
4456fe40f9
Remove the old code for projection analysis (#55112) 2023-10-10 17:13:32 +02:00
Duc Canh Le
867ac957bf wip: pass all ci tests
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-10 11:31:48 +00:00
Alexey Milovidov
e204b1d616
Merge pull request #55419 from evillique/mongo-fix
Fix MongoDB connection issues
2023-10-10 01:30:04 +02:00
Nikolay Degterinsky
153b542b29 Small fix 2023-10-09 15:13:45 +00:00
Nikolay Degterinsky
a1e56a3df1 Fix MongoDB connection issues 2023-10-09 14:46:48 +00:00
Han Fei
fcb19b4f77 Merge branch 'master' into hanfei/statistic 2023-10-09 14:24:58 +02:00
Alexey Milovidov
e0258582ae
Merge pull request #55338 from azat/fix-CreatingSetsTransform-data-race
Fix data-race in CreatingSetsTransform (on errors) due to throwing shared exception
2023-10-09 13:27:10 +02:00
Alexey Milovidov
fac83efdd2
Merge pull request #55335 from azat/fix-sort-sigsegv
Fix SIGSEGV in MergeSortingPartialResultTransform (due to zero chunks after remerge())
2023-10-09 02:48:12 +02:00
Azat Khuzhin
56dbba1f95 Fix SIGSEGV in MergeSortingPartialResultTransform (due to zero chunks after remerge())
It is possible to have non zero input chunks before remerge() and zero
after:

    2023.10.08 10:21:20.944928 [ 4321 ] {427df456-1400-4fbe-8bd7-c4de139f00ca} <Debug> MergeSortingTransform: Re-merging intermediate ORDER BY data (1 blocks with 0 rows) to save memory consumption
    2023.10.08 10:36:16.447001 [ 14466 ] {} <Fatal> BaseDaemon: (version 23.10.1.1, build id: 3E53B6B3A53FD562F44C88703BD88EB713881A44, git hash: 5ddfb170a1096cf88664b1a4b9b7bd2e7ef36c29) (from thread 4321) (query_id: 427df456-1400-4fbe-8bd7-c4de139f00ca) (query: SELECT CounterID FROM test.visits ARRAY JOIN Goals.ID WHERE CounterID = 942285 ORDER BY CounterID

CI: https://s3.amazonaws.com/clickhouse-test-reports/55276/099665478df3e77d1df0332dd705ca7209e903af/stress_test__debug_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 22:21:01 +02:00
Alexey Milovidov
fa54b21424
Merge pull request #55103 from ClickHouse/fix_shell_command_abort
Proper cleanup in case of exception in ctor of ShellCommandSource
2023-10-08 21:36:28 +02:00
Azat Khuzhin
cd3c140dd6 Fix data-race in CreatingSetsTransform (on errors) due to throwing shared exception
WARNING: ThreadSanitizer: data race (pid=3436)
  Read of size 1 at 0x7b50001b07a7 by thread T187:
    0 std::__1::basic_string<>::__is_long[abi:v15000] const build_docker/./contrib/llvm-project/libcxx/include/string:1499:33 (clickhouse-tsan+0x1faa29ff) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    1 std::__1::basic_string<>::size[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/string:968:17 (clickhouse-tsan+0x1faa29ff)
    2 std::__1::basic_string<>::empty[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/string:996:42 (clickhouse-tsan+0x1faa29ff)
    3 Poco::Exception::extendedMessage() build_docker/./base/poco/Foundation/src/Exception.cpp:114:13 (clickhouse-tsan+0x1faa29ff)
    4 DB::Exception::addMessage() <null> (clickhouse-tsan+0x7e0d5f1) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    5 DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:70:23 (clickhouse-tsan+0x1b3f2fed) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    6 DB::ExecutionThreadContext::executeTask() build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:95:9 (clickhouse-tsan+0x1b3f2fed)
    7 DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273:26 (clickhouse-tsan+0x1b3e5a10) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    8 DB::PipelineExecutor::executeSingleThread(unsigned long) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:239:5 (clickhouse-tsan+0x1b3e4bb8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    9 DB::PipelineExecutor::executeImpl(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:399:9 (clickhouse-tsan+0x1b3e4bb8)
    10 DB::PipelineExecutor::execute(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:111:9 (clickhouse-tsan+0x1b3e48be) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    11 DB::CompletedPipelineExecutor::execute() build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:110:18 (clickhouse-tsan+0x1b3e31c8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    12 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:202:14 (clickhouse-tsan+0x19a58396) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)

  Previous write of size 8 at 0x7b50001b07a0 by thread T192:
    0 std::__1::basic_string<>::__set_long_cap[abi:v15000](unsigned long) build_docker/./contrib/llvm-project/libcxx/include/string:1592:37 (clickhouse-tsan+0x238809e2) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    1 std::__1::basic_string<>::__grow_by_and_replace(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, char const*) build_docker/./contrib/llvm-project/libcxx/include/string:2367:5 (clickhouse-tsan+0x238809e2)
    2 std::__1::basic_string<>::append(char const*, unsigned long) build_docker/./contrib/llvm-project/libcxx/include/string:2692:9 (clickhouse-tsan+0x238809e2)
    3 std::__1::basic_string<>::append(char const*) build_docker/./contrib/llvm-project/libcxx/include/string:2849:12 (clickhouse-tsan+0x1faa2a58) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    4 Poco::Exception::extendedMessage() build_docker/./base/poco/Foundation/src/Exception.cpp:114:27 (clickhouse-tsan+0x1faa2a58)
    5 DB::Exception::addMessage() <null> (clickhouse-tsan+0x7e0d5f1) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    6 DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:70:23 (clickhouse-tsan+0x1b3f2fed) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    7 DB::ExecutionThreadContext::executeTask() build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:95:9 (clickhouse-tsan+0x1b3f2fed)
    8 DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273:26 (clickhouse-tsan+0x1b3e5a10) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    9 DB::PipelineExecutor::executeSingleThread(unsigned long) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:239:5 (clickhouse-tsan+0x1b3e4bb8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    10 DB::PipelineExecutor::executeImpl(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:399:9 (clickhouse-tsan+0x1b3e4bb8)
    11 DB::PipelineExecutor::execute(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:111:9 (clickhouse-tsan+0x1b3e48be) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    12 DB::CompletedPipelineExecutor::execute() build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:110:18 (clickhouse-tsan+0x1b3e31c8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    13 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:202:14 (clickhouse-tsan+0x19a58396) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)

  Location is heap block of size 480 at 0x7b50001b0600 allocated by thread T193:
    0 posix_memalign <null> (clickhouse-tsan+0x7d6ac85) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    1 std::__1::__libcpp_aligned_alloc[abi:v15000](unsigned long, unsigned long) build_docker/./contrib/llvm-project/libcxxabi/../libcxx/include/new:328:9 (clickhouse-tsan+0x238b5a05) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    2 __cxxabiv1::__aligned_malloc_with_fallback(unsigned long) build_docker/./contrib/llvm-project/libcxxabi/src/fallback_malloc.cpp:215:20 (clickhouse-tsan+0x238b5a05)
    3 __cxa_allocate_exception build_docker/./contrib/llvm-project/libcxxabi/src/cxa_exception.cpp:190:17 (clickhouse-tsan+0x238b31e5) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    4 std::exception_ptr std::make_exception_ptr[abi:v15000]<DB::Exception>(DB::Exception) build_docker/./contrib/llvm-project/libcxx/include/exception:206:9 (clickhouse-tsan+0x144bfa8a) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    5 DB::CreatingSetsTransform::~CreatingSetsTransform() build_docker/./src/Processors/Transforms/CreatingSetsTransform.cpp:30:45 (clickhouse-tsan+0x1b7603c3) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    ...
    35 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:211:1 (clickhouse-tsan+0x19a58608) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
2023-10-08 17:16:39 +02:00
Azat Khuzhin
099665478d Fix incorrect merging of Nested for SELECT FINAL FROM SummingMergeTree
The problem was the order of the columns, in case of SELECT FINAL it got
"counters_Map.count", "counters_Map.id"

But in case of OPTIMIZE FINAL it got "counters_Map.id",
"counters_Map.count" correctly.

Note, that this bugs exists there from the very recent versions, I've
checked 19.x and it was there.

P.S. there is a workaround for this problem, if you will use one of the
following patterns for key columns:
- *ID
- *Key
- *Type

That way it will be explicitly matched as key and everything will work.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 07:32:47 +02:00
Duc Canh Le
c1b0982a56 finalize implementation
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-06 08:32:46 +00:00
Duc Canh Le
c24c2c712e Merge branch 'master' into final_no_copy
- resolve conflicts

Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-10-06 08:20:37 +00:00
Nikolai Kochetov
3bd71c937c Merge branch 'master' into fix-read-in-order-with-array-join 2023-10-04 09:17:09 +00:00
Nikolai Kochetov
d944b59902 Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-10-03 14:28:16 +00:00
vdimir
3a9abde35d
Merge pull request #54514 from ClickHouse/vdimir/allow_experimental_partial_result
Add setting allow_experimental_partial_result
2023-09-29 10:32:57 +02:00
Alexander Gololobov
d81d5dd75c Cleanup in case of exception in ctor 2023-09-28 16:28:35 +02:00
Nikita Taranov
0e506b618e
impl (#54934) 2023-09-28 14:12:19 +02:00
vdimir
a134569bc3
fix MergeSortingPartialResultTransform::getRealProcessorSnapshot 2023-09-28 09:41:01 +00:00
vdimir
4e86eec7d1
fix Pipe::unitePipes with partial results 2023-09-28 09:41:00 +00:00
vdimir
3f3feea0b7
Add setting allow_experimental_partial_result 2023-09-28 09:40:56 +00:00
robot-clickhouse-ci-1
7a825c1417
Merge pull request #54427 from Avogar/json-object-as-tuple-inference
Add new features to schema inference for JSON formats
2023-09-27 20:12:45 +02:00
Robert Schulze
2d750e9e2f
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks 2023-09-27 12:15:29 +00:00
Antonio Andelic
580aa0031c
Merge pull request #55013 from ClickHouse/another-promise-fix
Set exception for promise in `CreatingSetsTransform` in more cases
2023-09-27 09:01:42 +02:00
Michael Kolupaev
23bc286280
Print more information about one logical error in MergeTreeDataWriter (#55001) 2023-09-26 13:23:26 -07:00
Han Fei
4e7745f757 Merge branch 'master' into hanfei/statistic 2023-09-26 21:43:53 +02:00
Robert Schulze
cde10fe7b5
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks 2023-09-26 18:59:41 +00:00
Kruglov Pavel
bea80ab5b7
Merge branch 'master' into json-object-as-tuple-inference 2023-09-26 15:23:08 +02:00
Kruglov Pavel
69a17bbef6
Merge pull request #52853 from Avogar/http-valid-json-on-exception
Output valid JSON/XML on excetpion during HTTP query execution
2023-09-26 14:25:55 +02:00
Robert Schulze
9fff447716
Re-enable clang-tidy checks 2023-09-26 09:34:12 +00:00
Antonio Andelic
098efb261e Throw proper exception if promise was not set 2023-09-26 09:03:38 +00:00
avogar
9e75825515 Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-object-as-tuple-inference 2023-09-25 17:24:36 +00:00
avogar
42ca897f2d Better schema inference for JSON formats 2023-09-25 15:42:59 +00:00
Dmitry Novik
d4acd9ec61 Set correct stream sorting 2023-09-25 15:34:35 +00:00
Alex Cheng
d5d3cf0e9d Merge remote-tracking branch 'origin/master' into refactor-MergeJoinTransform 2023-09-25 19:41:35 +08:00
Alex Cheng
c7a88c8946 refactor MergeJoinTransform code. 2023-09-25 18:04:25 +08:00
robot-ch-test-poll4
9d61ed4665
Merge pull request #54664 from bigo-sg/arrow_join_max_block_size
Respect max_block_size for array join to avoid possible OOM
2023-09-23 21:30:51 +02:00
robot-ch-test-poll4
ba6f0431a5
Merge pull request #54933 from ClibMouse/feature/big-endian-bson-each-row
Provide support for BSON on BE
2023-09-23 03:00:27 +02:00
robot-clickhouse-ci-2
d98234dc9d
Merge pull request #54803 from Avogar/ephemeral-columns-from-files
Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files
2023-09-22 23:24:42 +02:00
kothiga
3e57b007a8
Use LE version of unalignedStore. 2023-09-22 12:25:17 -07:00
robot-clickhouse-ci-1
9a3d1d6296
Merge pull request #54920 from ClickHouse/set-promise-on-exception
Set exception for promise in `CreatingSetsTransform`
2023-09-22 20:45:09 +02:00
kothiga
80d511093b
Provide support for BSON on BE 2023-09-22 09:21:21 -07:00
Kruglov Pavel
4d675dbad4
Merge pull request #54825 from azat/fix-virtual-columns-filtering
Fix filtering parts with indexHint for non analyzer (resubmit)
2023-09-22 18:20:16 +02:00
Dmitry Novik
3b0579b339 Fix splitting into buckets 2023-09-22 14:44:40 +00:00
Antonio Andelic
a691dea6cf Set exception for promise 2023-09-22 13:13:54 +00:00
Robert Schulze
9a91f97ac4
Merge pull request #53831 from rschu1ze/clang-17
Switch CI to Clang 17
2023-09-22 12:28:35 +02:00
Igor Nikonov
b1cc698477
Merge pull request #54564 from vitlibar/fix-sorting-of-union-of-sorted
Fix sorting of UNION ALL of already sorted results
2023-09-21 22:49:53 +02:00
Robert Schulze
877e4f3aab
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 20:21:12 +00:00
Michael Kolupaev
9af9b4a085
Enable connection pooling for s3 table function (#54812)
Enable connection pooling for s3 table function
2023-09-21 09:27:20 -07:00
Robert Schulze
5209bd2d51
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 14:45:55 +00:00
Robert Schulze
f5137dd0b4
More clang-tidy fixes 2023-09-21 14:40:57 +00:00
avogar
3e08800cb5 Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files 2023-09-20 16:25:55 +00:00
Dmitry Novik
bb6952317c Merge remote-tracking branch 'origin/master' into parallel-window 2023-09-20 15:38:42 +00:00
Igor Nikonov
2d446fc1ca Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas 2023-09-20 13:40:08 +00:00
Kruglov Pavel
49ee14f701
Merge pull request #54809 from ClickHouse/pqmeta
Prevent ParquetMetadata reading 40 MB from each file unnecessarily
2023-09-20 12:53:22 +02:00
SmitaRKulkarni
135df6870c
Merge pull request #54706 from ClickHouse/replacing_merge_tree_with_clean_deleted_rows_disabled
Fix ReplacingMergeTree with vertical merge and cleanup
2023-09-20 12:06:55 +02:00
SmitaRKulkarni
d8adf05de2
Added a new column _block_number (#47532)
Added a new virtual column _block_number which is persisted on merges when allow_experimental_block_number_column is enabled
2023-09-20 11:31:12 +02:00
Azat Khuzhin
d9a634eb0f Fix filtering parts with indexHint for non analyzer
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit ffa82e9297)
2023-09-20 11:29:35 +02:00
Michael Kolupaev
c856ec4087 Prevent ParquetMetadata reading 40 MB from each file unnecessarily 2023-09-19 21:58:50 +00:00
Igor Nikonov
e28f4e819c
Merge branch 'master' into parallel-replicas-not-enough-replicas 2023-09-19 23:05:24 +02:00
Azat Khuzhin
c439c4bca2
Revert "Fix filtering parts with indexHint for non analyzer" 2023-09-19 21:39:21 +02:00
Han Fei
eb86c113e5
Merge pull request #54184 from priera/fix_mv_insertion_when_duplicated_parts
Inserting only non-duplicate chunks in MV
2023-09-19 20:41:45 +02:00
robot-ch-test-poll4
a3dcce21b3
Merge pull request #54405 from Avogar/json-or-jsonl
Parse data in JSON format as JSONEachRow if failed to parse metadata
2023-09-19 20:15:09 +02:00
Kruglov Pavel
d555fdffa5
Merge pull request #54449 from azat/parts-prune-indexHint
Fix filtering parts with indexHint for non analyzer
2023-09-19 19:15:41 +02:00
Han Fei
17243b84ab Merge branch 'master' into hanfei/statistic 2023-09-19 17:09:17 +02:00
Smita Kulkarni
e2887ad27b Merge branch 'master' into replacing_merge_tree_with_clean_deleted_rows_disabled 2023-09-19 15:58:41 +02:00
avogar
f974970c3c Apply suggestion 2023-09-19 11:53:40 +00:00
avogar
5bd2e9f610 Fix tests 2023-09-19 11:53:40 +00:00
avogar
8c29408f5e Parse data in JSON format as JSONEachRow if failed to parse metadata 2023-09-19 11:53:40 +00:00
Kruglov Pavel
e163670357
Merge branch 'master' into http-valid-json-on-exception 2023-09-19 13:42:53 +02:00
Kruglov Pavel
3c83e43351
Remove debug logging 2023-09-19 13:38:43 +02:00
priera
b92e37d2df Fixing PR comments 2023-09-19 12:15:01 +02:00
priera
663ce74609 fixed clang-tidy error 2023-09-19 12:15:00 +02:00
priera
b30e33580b fix test 2023-09-19 12:15:00 +02:00
priera
6d27ea2742 fixed style 2023-09-19 12:15:00 +02:00
priera
94d737264c inserting non-duplicate chunks 2023-09-19 12:15:00 +02:00
Smita Kulkarni
142a4631f1 Updated to clear current_row_sources when cleanedup rows are added 2023-09-19 11:22:10 +02:00
Igor Nikonov
047d214436 Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas 2023-09-18 15:29:56 +00:00
Igor Nikonov
e1019ba3c4 Disabling parallel replicas per shard will be done separately 2023-09-18 15:27:55 +00:00
Kruglov Pavel
9c888ea42b
Merge pull request #53549 from Avogar/group-by-constant-keys
Optimize group by constant keys
2023-09-18 12:12:40 +02:00
taiyang-li
39ca44b51a Respect max_block_size for array join to avoid possible OOM 2023-09-18 11:04:38 +08:00
Robert Schulze
f5e8028bb1
Merge pull request #54642 from rschu1ze/broken-re2st
Remove broken lockless variant of re2
2023-09-17 15:30:57 +02:00
Amos Bird
0518b64b58
Fix nullable primary key in final (#54164)
* Fix nullable primary key in final

* Real fix

* Address reviews
2023-09-15 22:44:13 +02:00
Smita Kulkarni
8ef910e1fa Updated to check if cleanup is enabled before adding to cleanedup_rows_count 2023-09-15 22:34:09 +02:00
Kruglov Pavel
2075f9c667
Merge branch 'master' into group-by-constant-keys 2023-09-15 15:10:08 +02:00
avogar
35d975bfea Add comment in ParallelInputFormat, remove unneded include 2023-09-15 13:07:04 +00:00
Kruglov Pavel
dbd24b240c
Merge branch 'master' into http-valid-json-on-exception 2023-09-15 14:55:31 +02:00
Sema Checherinda
45cf793544
Merge pull request #53919 from canhld94/optimize_all_lonely_parts
Do not implicitly read pk and version columns in lonely parts if unnecessary
2023-09-15 13:11:00 +02:00
Robert Schulze
7b378dbad3
Remove broken lockless variant of re2 2023-09-14 16:40:42 +00:00
Vitaly Baranov
9a0e1ef592 Fix sorting of UNION ALL of already sorted results. 2023-09-14 15:04:37 +02:00
Robert Schulze
a9ae813db0
Merge pull request #54115 from slvrtrn/simplified-prepared-statements-for-mysql
Implement the MySQL binary protocol implementation for initial support of Tableau Online
2023-09-14 12:27:11 +02:00
slvrtrn
c0961d9378 Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-09-13 19:33:11 +02:00
Sema Checherinda
8a9b544a97
Merge branch 'master' into optimize_all_lonely_parts 2023-09-13 16:07:19 +02:00
Arthur Passos
da8caeffd2 Merge branch 'master' into arrow_parquet_account_for_monotonically_increasing_offsets_across_batches 2023-09-12 17:50:36 -03:00
Igor Nikonov
7b3f32b95a
Merge pull request #54520 from ClickHouse/pr-cleanup
Parallel replicas: cleanup unused params
2023-09-12 19:48:18 +02:00
slvrtrn
dddea9219a Address the review comments 2023-09-12 18:39:03 +02:00
Anton Popov
3e242f8ba3
Merge pull request #54351 from Avogar/null-as-default-nested
Support NULL as default for nested types Array/Tuple/Map for input formats
2023-09-12 14:58:21 +02:00
Igor Nikonov
1287f68745 Handle clusterAllReplicas/remote cases to avoid unnecessary logging 2023-09-12 12:52:29 +00:00
robot-clickhouse
63243fbc03
Merge pull request #54480 from amosbird/fix_54406
Fix aggregate projections with normalized states
2023-09-12 13:43:41 +02:00
slvrtrn
611a75a87f Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-09-12 10:38:44 +02:00
Igor Nikonov
d5ea047ab8 Parallel replicas: cleanup unused params 2023-09-11 21:52:40 +00:00
Igor Nikonov
2293923f66 Disable parallel replicas on shards with not enough nodes 2023-09-11 21:46:46 +00:00
Amos Bird
667426f1f2
DataTypeAggregateFunction::strictEquals 2023-09-12 03:54:19 +08:00
avogar
803d8dcf85 Support NULL as default for nested types Array/Tuple/Map for input formats 2023-09-11 18:18:33 +00:00
Nikolai Kochetov
903c966cc8
Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-09-11 16:14:03 +02:00
Nikolai Kochetov
59108f3d96
Merge pull request #54476 from ClickHouse/revert-54470-revert-48607-master
Revert "Revert "Add settings for real-time updates during query execution""
2023-09-11 14:59:13 +02:00
Amos Bird
fb0f9ff565
Fix aggregate projections with normalized states 2023-09-10 03:21:22 +08:00
Alexey Milovidov
c0a43df749
Merge pull request #54435 from Alex-Cheng/small_finetune_merge_join
Small fine-tune for using ColumnNullable pointer
2023-09-09 19:28:28 +03:00
alexX512
7f77869216 Increase amount of rows in requests for partial result tests
(cherry picked from commit 54bc3fea9a)
2023-09-09 10:33:05 +00:00
Nikolai Kochetov
9b936c44db
Revert "Revert "Add settings for real-time updates during query execution"" 2023-09-09 12:29:39 +02:00
Alexey Milovidov
03a755732a
Revert "Add settings for real-time updates during query execution" 2023-09-09 03:10:23 +03:00
Azat Khuzhin
ffa82e9297 Fix filtering parts with indexHint for non analyzer
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-08 18:17:06 +02:00
Anton Popov
1aa34c16e3 Merge remote-tracking branch 'upstream/master' into HEAD 2023-09-08 14:00:45 +00:00
Nikolai Kochetov
0095124791
Merge pull request #48607 from alexX512/master
Add settings for real-time updates during query execution
2023-09-08 09:05:33 +02:00
robot-ch-test-poll2
f28ad1e136
Merge pull request #54368 from ClickHouse/fix_replacing_final_error
Fix logical error in vertical merge + replacing merge tree + optimize cleanup
2023-09-07 15:45:43 +02:00
Alex Cheng
75dc6d9708 small fine-tune for using ColumnNullable pointer 2023-09-07 20:41:32 +08:00
Arthur Passos
f34d40cde3 docs 2023-09-06 17:26:34 -03:00
Arthur Passos
a8027be612 fix 2023-09-06 17:25:27 -03:00
robot-ch-test-poll3
7f756ed613
Merge pull request #54354 from ClickHouse/parallel-replicas-cleanup
Parallel replicas: remove unused code
2023-09-06 19:38:14 +02:00
Arthur Passos
ce673d6bea arrow-parquet account for monotonically increasing offsets across multiple batches 2023-09-06 13:59:39 -03:00
Alexander Sapin
4368c00052 Fix logical error in vertical merge + replacing merge tree + optimize cleanup 2023-09-06 18:43:56 +02:00
Kruglov Pavel
5b29640b22
Merge branch 'master' into group-by-constant-keys 2023-09-06 18:06:51 +02:00
Duc Canh Le
638f2e4c87 wip: final without copy row-by-row
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-09-06 16:04:54 +00:00
robot-ch-test-poll4
8c71582316
Merge pull request #54185 from ClickHouse/analyzer-partition-id-pruning
Fix partition id pruning for analyzer.
2023-09-06 16:53:35 +02:00
Igor Nikonov
c51cfef065 Remove unused code 2023-09-06 12:53:58 +00:00
Igor Nikonov
54c5158291
Merge pull request #54315 from ClickHouse/add-logs-parallel-replica-over-distributed
Add logs for parallel replica over distributed
2023-09-06 14:23:45 +02:00
Anton Popov
401496aa1f Apply suggestions from code review
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-09-06 10:27:46 +00:00
robot-clickhouse-ci-1
02339a1f22
Merge pull request #54326 from ClickHouse/fix_segfault_system_zookeeper
Fix segfault in system.zookeeper
2023-09-06 02:20:50 +02:00
Alexander Tokmakov
a8489c21da fix segfault in system.zookeeper 2023-09-05 20:23:16 +02:00
Igor Nikonov
47d572d8bc Log cluster_for_parallel_replicas on initiator 2023-09-05 13:17:10 +00:00
slvrtrn
6e0a254368 Update tests, update imports, remove requirements 2023-09-04 21:50:06 +02:00
slvrtrn
bb0eff9669 Revert format changes 2023-09-04 21:15:26 +02:00
irenjj
c9261bbf18 Merge remote-tracking branch 'upstream/master' into feat_markdown 2023-09-04 22:35:53 +08:00
Han Fei
f60dad0598 Merge branch 'master' into hanfei/statistic 2023-09-04 16:17:07 +02:00
Raúl Marín
e192d4c624
Fix summary reporting with parallel replicas with LIMIT (#53050) 2023-09-04 15:53:06 +02:00
Alexey Milovidov
b660ac9bf1 Merge #54236 2023-09-04 03:57:39 +02:00
Alexey Milovidov
476e15ce3d
Merge pull request #54236 from YinZheng-Sun/Fix
remove semicolon
2023-09-04 04:54:30 +03:00
Yinzheng-Sun
74ec546052 remove semicolon 2023-09-03 21:52:39 +08:00
Nikolai Kochetov
63c681daf4 Try fix tests. 2023-09-01 18:20:56 +00:00
Nikolai Kochetov
4cbfca78e4 Fixing 01748_partition_id_pruning 2023-09-01 14:29:31 +00:00
alexX512
36b8932b37 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-31 21:40:50 +00:00
Nikolai Kochetov
52a7206002 Fix rows_before_limit_at_least for DelayedSource. 2023-08-31 12:53:58 +00:00
Robert Schulze
aefb543734
Merge remote-tracking branch 'rschu1ze/master' into feat_markdown 2023-08-31 11:32:44 +00:00
slvrtrn
8378441248 Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-08-30 19:00:51 +02:00
slvrtrn
f4dd6be8a8 WIP prepared statements 2023-08-30 18:59:39 +02:00
Duc Canh Le
11c94246c7 do not implicitly read pk + version and compute sort expression for lonely parts
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-08-30 05:05:57 +00:00
Sergei Trifonov
802579f3f1
Merge pull request #49618 from ClickHouse/concurrency-control-controllable
Make concurrency control controllable
2023-08-29 19:44:51 +02:00
Anton Popov
02391eafe1 refactoring of reading from MergeTree tables 2023-08-29 13:40:12 +00:00
alexX512
2670af53df Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-29 13:37:58 +00:00
alexX512
70f926a75a Style fix 2023-08-29 13:37:46 +00:00
alexX512
238f6e8327 fix style 2023-08-29 13:15:38 +00:00
Kseniia Sumarokova
463a197da4
Merge pull request #53847 from kssenii/fix-segfault-in-pg
Fix possible segfault while using PostgreSQL engine
2023-08-29 14:45:46 +02:00
alexX512
b3ba00f443 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-29 10:59:29 +00:00
alexX512
47a908dfa8 Add support for nested supqueries for AggregationPartialResult transform and more comments 2023-08-29 10:45:34 +00:00
Duc Canh Le
e9b0d3e4a2 do not implicitly read pk and version columns in lonely parts if nnot necessary 2023-08-29 09:41:18 +00:00
Raúl Marín
45e8185f9b Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-08-29 11:17:34 +02:00
Alexey Milovidov
b7d8ebf5e2
Merge pull request #53511 from Enmk/fix-ReplacingMergeTree-do_not_merge_across_partitions_select_final
Fixed SELECTing from ReplacingMergeTree with do_not_merge_across_partitions_select_final
2023-08-28 16:07:06 +03:00
vdimir
b92c436c29
Merge pull request #53699 from ClickHouse/vdimir/filter_pushdown_const_bug 2023-08-28 11:45:05 +02:00
irenjj
51aa89eed8 Add a setting to automatically escape special characters in Markdown. 2023-08-28 00:10:33 +08:00
alesapin
0e601b856f
Merge pull request #53751 from Algunenano/clang18
Support clang-18 (Wmissing-field-initializers)
2023-08-25 23:27:06 +02:00
slvrtrn
055d2e3c3d Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-08-25 21:27:47 +02:00
slvrtrn
734ffd916c WIP prepared statements 2023-08-25 20:31:21 +02:00
kssenii
4af1bf60ed Fix 2023-08-25 19:24:31 +02:00
kssenii
49d33774f9 Fix 2023-08-25 17:47:56 +02:00
Vasily Nemkov
f896c6349c Merge commit upstream/master into fix-ReplacingMergeTree-do_not_merge_across_partitions_select_final 2023-08-25 16:02:54 +02:00
Raúl Marín
6490213207 Merge remote-tracking branch 'blessed/master' into clang18 2023-08-24 17:35:26 +02:00
Kruglov Pavel
44db5fa992
Merge branch 'master' into cache-count 2023-08-24 17:21:18 +02:00
avogar
c4f8113a78 Fix tests 2023-08-24 15:05:45 +00:00
Kruglov Pavel
cd3433f95a
Merge pull request #53407 from Avogar/tsv-single-carriage-return
Fix processing single carriage return in TSV file segmentation engine
2023-08-24 16:06:05 +02:00
Han Fei
10c622fa08 Merge branch 'master' into hanfei/statistic 2023-08-24 15:56:06 +02:00
Nikolai Kochetov
fc90a1a0bd Fix some skip_unused_shards tests. 2023-08-24 11:07:17 +00:00
Kruglov Pavel
f7e1abd774
Merge branch 'master' into cache-count 2023-08-23 22:31:49 +02:00
avogar
93ba73715a Merge branch 'master' of github.com:ClickHouse/ClickHouse into group-by-constant-keys 2023-08-23 18:47:42 +00:00
Raúl Marín
93dac0c880 Support clang-18 (Wmissing-field-initializers) 2023-08-23 15:53:45 +02:00
avogar
68e3af56d4 Address comments 2023-08-23 13:19:15 +00:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count 2023-08-23 15:18:02 +02:00
Kruglov Pavel
fee323b122
Update src/Processors/Formats/IRowInputFormat.h
Co-authored-by: Michael Kolupaev <michael.kolupaev@clickhouse.com>
2023-08-23 12:15:47 +02:00
Kruglov Pavel
e193aec583
Merge branch 'master' into fast-count-from-files 2023-08-23 12:15:34 +02:00
Alexey Milovidov
67c3b7fa65
Merge pull request #53711 from ClickHouse/sorted-distinct-sparse-columns
Fix: sorted distinct with sparse columns
2023-08-23 11:34:01 +03:00
Igor Nikonov
22895c1d0d Fix: sorted distinct with sparse columns 2023-08-22 23:33:30 +00:00
avogar
cd228754c8 Apply suggestions 2023-08-22 17:32:35 +00:00
Kruglov Pavel
6e3f2fedbd
Apply suggestions from code review
Co-authored-by: Dmitry Novik <mrnovikd@gmail.com>
2023-08-22 19:05:06 +02:00
vdimir
b23773380c
Fix filter pushdown for full_sorting_merge join 2023-08-22 15:06:50 +00:00
avogar
6119246948 Better 2023-08-22 14:51:46 +00:00
robot-clickhouse-ci-1
698ceee1a9
Merge pull request #53631 from amosbird/fix_53630
Fix data race of shell command
2023-08-22 15:12:26 +02:00
Kruglov Pavel
67c5c0203b
Merge branch 'master' into fast-count-from-files 2023-08-22 15:03:48 +02:00
avogar
7f9e81d504 Clean up 2023-08-22 12:55:00 +00:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count 2023-08-22 14:42:22 +02:00
avogar
b4145aeddc Cache number of rows in files for count in file/s3/url/hdfs/azure functions 2023-08-22 11:59:59 +00:00
Antonio Andelic
e0af6e5879
Merge pull request #53644 from ClickHouse/fix-delayed-source-with-totals
Correctly handle totals and extremes with `DelayedSource`
2023-08-22 12:24:16 +02:00
Robert Schulze
acaa2eee58
Merge pull request #53556 from ClibMouse/feature/big-endian-UUID-correction
Change Big Endian-UUID to work the same as Little Endian-UUID
2023-08-22 11:57:44 +02:00
Alexey Milovidov
9874556d85
Merge pull request #53655 from ClibMouse/more-fix-big-endian
State of State and avg aggregation function fix for big endian
2023-08-22 11:49:54 +03:00
Amos Bird
f13fd5b5dc
Wait for some pooled commands 2023-08-22 16:26:35 +08:00
Amos Bird
bd9f526f93
Fix data race of shell command 2023-08-22 14:50:56 +08:00
Michael Kolupaev
d752611c43 Performance test 2023-08-21 14:15:52 -07:00
Michael Kolupaev
2f4d433e69 Parquet filter pushdown 2023-08-21 14:15:52 -07:00
Michael Kolupaev
6009e1b293
Merge pull request #53324 from bigo-sg/ch_gluten_2583
Implement native orc input format without arrow to improve performance
2023-08-21 13:44:57 -07:00
Alexey Milovidov
d7180c7d5c
Merge pull request #53645 from ClickHouse/prepared-set-cache-in-mutation-pipeline-stuck
Prepared set cache in mutation pipeline stuck
2023-08-21 22:21:45 +03:00
Suzy Wang
7cfc2a0872 update api name 2023-08-21 12:18:32 -07:00
Suzy Wang
6edf8f78cc more fixes for big endian 2023-08-21 12:13:56 -07:00
Nikolai Kochetov
f0b68393a8
Merge pull request #53162 from ClickHouse/not-ready-set-bug
Not-ready Set
2023-08-21 21:09:15 +02:00
Kruglov Pavel
fed253a671
Fix build 2023-08-21 16:46:23 +02:00
Antonio Andelic
c4bf151d83 Correctly handle totals and extremes with DelayedSource 2023-08-21 14:07:59 +00:00
Nikolai Kochetov
2d932f0953 Remove obsolete code. 2023-08-21 14:04:07 +00:00
Kruglov Pavel
098818b234
Fix build 2023-08-21 15:45:45 +02:00
Nikolai Kochetov
e9149981ea Fix check for delayed ports. 2023-08-21 13:08:32 +00:00
Kruglov Pavel
88aee95122
Merge branch 'master' into fast-count-from-files 2023-08-21 14:46:33 +02:00
avogar
e590f15d3d Fix for carriage return in tsv 2023-08-21 12:46:03 +00:00
avogar
584bd57f5c Clean up 2023-08-21 12:43:11 +00:00
avogar
47304bf7aa Optimize count from files in most input formats 2023-08-21 12:30:52 +00:00
Kruglov Pavel
c68456a20a
Merge pull request #52692 from Avogar/variable-number-of-volumns-more-formats
Allow variable number of columns in more formats, make it work with schema inference
2023-08-21 13:28:35 +02:00
avogar
4c12914406 Fix single carriage return processing in TSV file segmentation engine 2023-08-21 11:26:18 +00:00
Nikolai Kochetov
388e2a09d6 Merge branch 'master' into not-ready-set-bug 2023-08-21 10:59:51 +00:00
taiyang-li
e5a399d0e9 change as requested 2023-08-21 14:51:25 +08:00
taiyang-li
f2e4315d59 fix building 2023-08-21 14:11:48 +08:00
taiyang-li
3cd9eb9b84 fix style 2023-08-21 12:12:31 +08:00
taiyang-li
3781245027 change as request 2023-08-21 12:10:39 +08:00
taiyang-li
f723e8d43a change as request 2023-08-21 12:09:02 +08:00
Igor Nikonov
a69ea02418 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-19 21:43:25 +00:00
taiyang-li
9866d2727e change as request 2023-08-19 17:31:19 +08:00
Alexey Milovidov
c42c4046fe
Merge pull request #53573 from amosbird/fix_53454
Proper destruction of task in ShellCommandSource
2023-08-19 12:19:29 +03:00
Amos Bird
052281896a
Proper destruction of task 2023-08-19 13:56:57 +08:00
Michael Kolupaev
a1522e22ea
Merge pull request #53281 from Avogar/batch-small-parquet-row-groups
Optimize reading small row groups by batching them together in Parquet
2023-08-18 17:15:42 -07:00
Igor Nikonov
aebda10f7c Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-18 23:00:58 +00:00
kothiga
f33c585bc5
Addressing feedback. 2023-08-18 13:50:31 -07:00
Austin Kothig
6b42975d33
Change BE-UUID to work the same as LE-UUID. Included high and low getters to provide cleaner code when accessing undertype. 2023-08-18 08:19:46 -07:00
avogar
7fa72f5cdf Optimize group by constant keys 2023-08-18 12:47:26 +00:00
Kruglov Pavel
3bae63e48e
Fix special build 2023-08-18 13:56:32 +02:00
taiyang-li
4f9429d2e4 fix ut tests/queries/0_stateless/00900_null_array_orc_load.sh 2023-08-18 17:58:24 +08:00
Vasily Nemkov
55168dc075 Merge remote-tracking branch 'upstream/master' into fix-ReplacingMergeTree-do_not_merge_across_partitions_select_final 2023-08-18 11:41:56 +02:00
Amos Bird
1c300a31b1
Fix build 2023-08-18 15:38:48 +08:00
Amos Bird
b2dd055cdf
Use poll instead 2023-08-18 15:38:48 +08:00
Amos Bird
c43bf153f5
Refactor 2023-08-18 15:38:46 +08:00
Amos Bird
dd0c71b32a
Add error_exit_reaction 2023-08-18 15:38:46 +08:00
Amos Bird
940ba60ae0
Fix non-linux build 2023-08-18 15:38:45 +08:00
Amos Bird
476f3cedc1
Various reactions when executable stderr has data 2023-08-18 15:38:45 +08:00
taiyang-li
15720d9cef fix ut tests/queries/0_stateless/02518_parquet_arrow_orc_boolean_value.sh 2023-08-18 15:10:25 +08:00
taiyang-li
8e0d5b7ee0 fix bugs 2023-08-18 13:31:26 +08:00
Igor Nikonov
06c98956b3 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 20:42:50 +00:00
Igor Nikonov
9f771b10e9 Cleanup 2023-08-17 20:42:22 +00:00
Alexander Tokmakov
89c365803f
Merge pull request #53424 from ClickHouse/exception_message_patterns6
Less exceptions with runtime format string
2023-08-17 17:58:04 +03:00
Igor Nikonov
3e9ae1751e Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 14:45:37 +00:00
avogar
bca91548ad Add setting input_format_parquet_local_file_min_bytes_for_seek 2023-08-17 12:28:01 +00:00
Robert Schulze
ec7daed52f
Merge pull request #50276 from ClibMouse/feature/mergetree-checksum-big-endian-support
MergeTree & SipHash checksum big-endian support
2023-08-17 13:27:18 +02:00
Nikolai Kochetov
8dbe82c192
Merge pull request #53489 from ClickHouse/fix-wrong-column-order-in-parallel-final
Fix wrong columns order for queries with parallel FINAL.
2023-08-17 13:22:13 +02:00
Vasily Nemkov
322db80cd9 Fixed SELECTing from ReplacingMergeTree with do_not_merge_across_partitions_select_final 2023-08-17 12:35:55 +02:00
李扬
cfea6db997
Update native_orc.cpp 2023-08-17 16:57:41 +08:00
taiyang-li
c4777397f7 fix integration test test_kafka_formats 2023-08-17 14:42:51 +08:00
Igor Nikonov
2e4d346e44 Fixes 2023-08-16 23:31:20 +00:00
Igor Nikonov
21ef1f1d1c Remove unnecessary Scalars usage for cluster_for_parallel_replicas 2023-08-16 23:24:01 +00:00
Nikolai Kochetov
3a6a3d491e Fix typo 2023-08-16 20:40:28 +00:00
Nikolai Kochetov
91c241532f Add a check for header structure. 2023-08-16 20:26:23 +00:00
Nikolai Kochetov
30ccfa1638 Fix wrong columns order for queries with parallel FINAL. 2023-08-16 20:23:10 +00:00
taiyang-li
8ba1da755d update example 2023-08-16 11:18:44 +08:00
Alexander Tokmakov
ba44d7260e fix 2023-08-16 00:20:28 +02:00
Michael Kolupaev
cbe4c8adc2 Fix more functions with 'Context has expired' error 2023-08-15 20:37:10 +00:00
Igor Nikonov
9825f8c76e Remove debug logging 2023-08-15 17:03:32 +00:00
taiyang-li
c2017e0ea3 update orc version 2023-08-15 14:49:45 +08:00
taiyang-li
724af579d7 revert file 2023-08-15 12:17:49 +08:00
taiyang-li
bfa9d361cc fix tests 2023-08-15 12:16:22 +08:00
Igor Nikonov
9f7a25eead Fix style once more 2023-08-14 23:27:36 +00:00
Igor Nikonov
e82b6b02ca Fix style 2023-08-14 23:15:27 +00:00
Igor Nikonov
c94994afcf Propagate setting cluster_for_parallel_replicas to shards 2023-08-14 22:50:46 +00:00
ltrk2
2846ea49b4
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-14 13:02:30 -04:00
Igor Nikonov
709287fbdc Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-14 14:37:43 +00:00
taiyang-li
bbe5caa9dd fix building 2023-08-14 19:55:45 +08:00
李扬
613bef89f4
Merge branch 'master' into ch_gluten_2583 2023-08-14 17:44:15 +08:00
taiyang-li
34c3162c7c revert old orc files 2023-08-14 17:42:42 +08:00
taiyang-li
832768745b add examples 2023-08-14 17:27:01 +08:00
taiyang-li
8ee1912de6 remove old orc files 2023-08-14 17:26:24 +08:00
taiyang-li
57bef64fbc add new native orc files 2023-08-14 17:25:58 +08:00
Alexey Milovidov
b0687e650a
Merge pull request #53209 from Avogar/format-one
Add input format One
2023-08-13 21:32:11 +03:00
Alexey Milovidov
5b243d2990
Merge pull request #53006 from kitaisreal/explain-actions-for-join-step
EXPLAIN actions for JOIN step
2023-08-12 02:51:07 +03:00
Sergei Trifonov
771710b377
Merge branch 'master' into concurrency-control-controllable 2023-08-11 16:50:13 +02:00
avogar
7e863a2726 Address comments 2023-08-11 13:17:49 +00:00
avogar
3ad7e57059 Optimize reading small row groups by batching them together in Parquet 2023-08-11 13:17:45 +00:00
Kruglov Pavel
b6b0e9c6bc
Merge branch 'master' into variable-number-of-volumns-more-formats 2023-08-11 14:02:21 +02:00
Kruglov Pavel
00865a7dad
Merge branch 'master' into format-one 2023-08-11 13:54:58 +02:00
ltrk2
1dc1b54c68
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-11 07:48:49 -04:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache
Better usage of ccache
2023-08-11 04:43:45 +03:00
Alexey Milovidov
fd7b92e90a
Merge pull request #53135 from ClickHouse/file_diagnostics_while_reading_header
Add diagnostic info about file name during schema inference
2023-08-10 21:56:12 +03:00
Alexey Milovidov
8fcfdd2bfd
Merge branch 'master' into explain-actions-for-join-step 2023-08-10 17:33:25 +03:00
ltrk2
a2054c04dd
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-10 10:21:34 -04:00
Alexey Milovidov
bb57caa83e
Merge pull request #53180 from ClickHouse/ditch-tons-of-garbage
Ditch tons of garbage
2023-08-10 16:59:51 +03:00
avogar
078683e226 Fix tests 2023-08-10 13:07:06 +00:00