thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-04 21:42:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
134,787 Commits 9,185 Branches 1,782 Tags 2.3 GiB
118263d8e8
Commit Graph

5875 Commits

Author SHA1 Message Date
Igor Nikonov
0bbf7ba921
Merge branch 'master' into pr-progress-bar 2023-10-23 14:56:59 +02:00
Kruglov Pavel
32da07b4ee
Merge pull request #55695 from Avogar/restrict-iceberg 
Fixes and improvements for Iceberg storage
 2023-10-23 13:58:23 +02:00
Alex Cheng
49b3136491
Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform 2023-10-23 09:56:22 +08:00
Robert Schulze
c2215cb5f0
Support string arguments 2023-10-22 09:28:45 +00:00
Alexey Milovidov
5217d64551 Remove garbage 2023-10-22 01:53:50 +02:00
Alexey Milovidov
1be4ff229b
Merge pull request #55895 from ClickHouse/fix-window-functions-with-sparse-columns 
Fix window functions in case of sparse columns.
 2023-10-22 00:02:34 +02:00
János Benjamin Antal
1b0cadebd0 Convert sparse columns to full for window functions 2023-10-21 11:46:25 +00:00
Igor Nikonov
e378d920c4
Merge branch 'master' into pr-progress-bar 2023-10-21 13:16:12 +02:00
Alexey Milovidov
7ec4b99e94 Revert partial result 2023-10-21 03:14:22 +02:00
Kruglov Pavel
6f61ccfe28
Merge branch 'master' into schema-inference-union 2023-10-20 22:54:11 +02:00
avogar
6934e27e8b Add union mode for schema inference to infer union schema of files with different schemas 2023-10-20 20:46:41 +00:00
avogar
8cc0dc17eb Try to fix possible segfault in Native ORC input format 2023-10-20 18:50:48 +00:00
yariks5s
87f26f5132 dealt with 2dim arrays 2023-10-20 17:05:05 +00:00
Anton Popov
5819bcd07a
Support asynchronous inserts for native protocol (#54730) 
* support async insert for native protocol

* use separate queue for async inserts via native protocol

* fix test

* better logging for async inserts and more tests

* disable mixed internal and external data in async inserts

* fix tests

* fix quota in async inserts

* disable async insert for secondary query of distributed
 2023-10-20 18:39:48 +02:00
Raúl Marín
d08ce13107
Merge pull request #51692 from Algunenano/parallel_replicas_row_estimation 
Automatic decision of nº of parallel replicas
 2023-10-20 15:50:43 +02:00
Azat Khuzhin
d5c960ceb6 Fix data race in CreatingSetsTransform 
The problem is that findOrPromiseToBuild() can also call
shared_future::get():

    (lldb) bt 10
    * thread 194, name = 'MergeMutate', stop reason = step over
      * frame 0: 0x0000564126046bd2 clickhouse`std::rethrow_exception(p=(__ptr_ = 0x00007b5000130c80)) at exception_pointer_cxxabi.ipp:68:39
        frame 1: 0x000056411c4c5ddd clickhouse`std::__1::__assoc_state<std::__1::shared_ptr<DB::Set>>::copy(this=<unavailable>) at future:701:9
        frame 2: 0x000056411c4c0e3f clickhouse`DB::PreparedSetsCache::findOrPromiseToBuild(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) [inlined] std::__1::shared_future<std::__1::shared_ptr<DB::Set>>::get[abi:v15000](this=0x00007b0c00154eb8) const at future:2243:46
        frame 3: 0x000056411c4c0e2e clickhouse`DB::PreparedSetsCache::findOrPromiseToBuild(this=0x00007b1c00077018, key="__set_10438532338982543705_15680061365857338647") at PreparedSets.cpp:349:116

Note, that my test did not find the problem because of #55261 had not
been merged yet.

Follow-up for: #55338
Fixes: #55279
v2: s/createDeprecated/createRuntime/

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-19 12:26:02 +02:00
yariks5s
6dc88a4ca4 new changes 2023-10-18 18:02:05 +00:00
Dmitry Novik
5ca43e51c1 Remove unused code 2023-10-17 20:45:00 +00:00
avogar
323486f9e8 Add tests 2023-10-17 18:10:47 +00:00
Han Fei
4b23142775
Merge branch 'master' into hanfei/statistic 2023-10-17 16:40:31 +02:00
Raúl Marín
3eaf752284 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-17 11:36:39 +02:00
Alex Cheng
ce493b57f0
Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform 2023-10-17 16:42:53 +08:00
Igor Nikonov
4b36761f89
Merge branch 'master' into pr-progress-bar 2023-10-17 10:36:32 +02:00
Michael Kolupaev
ce7eca0615
DWARF input format (#55450) 
* Add ReadBufferFromFileBase::isRegularLocalFile()

* DWARF input format

* Review comments

* Changed things around ENABLE_EMBEDDED_COMPILER build setting

* Added 'ranges' column

* no-msan no-ubsan
 2023-10-16 17:00:07 -07:00
Jiebin Sun
df17cd467b
Release more num_streams if data is small (#53867) 
* Release more num_streams if data is small

Besides the sum_marks and min_marks_for_concurrent_read, we could also involve the
system cores to get the num_streams if the data is small. Increasing the num_streams
and decreasing the min_marks_for_concurrent_read would improve the parallel performance
if the system has plentiful cores.

Test the patch on 2x80 vCPUs system. Q39 of clickbench has got 3.3x performance improvement.
Q36 has got 2.6x performance improvement. The overall geomean has got 9% gain.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

* Release more num_streams if data is small
Change the min marks from 4 to 8 as the profit is small and 8 granules
is the default block size.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

---------

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-10-16 18:41:38 +02:00
Raúl Marín
f799f5d7a1 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-16 15:07:11 +02:00
Igor Nikonov
fdd7bfbb83 Merge remote-tracking branch 'origin/master' into pr-progress-bar 2023-10-16 12:59:00 +00:00
alesapin
3b02748cb6 Fix some typos 2023-10-15 15:43:02 +02:00
Duc Canh Le
285ae778e4 Merge branch 'master' into final_no_copy 
Fix flaky test 02447_drop_database_replica
 2023-10-14 03:34:42 +00:00
Anton Popov
1c9d883d8a Merge remote-tracking branch 'upstream/master' into HEAD 2023-10-13 17:17:01 +00:00
Alexander Tokmakov
e3e105d154
Merge pull request #55527 from azat/values-eof-check-fix 
Fix checking of non handled data for Values format
 2023-10-13 18:07:02 +02:00
yariks5s
cb08da617f added read and parse impl 2023-10-13 15:16:07 +00:00
Dmitry Novik
d4be706ed4 Merge remote-tracking branch 'origin/master' into parallel-window 2023-10-13 13:20:38 +00:00
Raúl Marín
500dab9569 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-13 10:12:57 +02:00
Duc Canh Le
f7830bc13a some low level optimization 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-13 07:41:49 +00:00
Anton Popov
a01acf5d2a remove projection from StorageSnapshot 2023-10-13 03:18:43 +00:00
Alexey Milovidov
8a1363bcf1
Merge pull request #49486 from bigo-sg/test_hive_null_as_default 
Set defaults_for_omitted_fields to true for hive text format
 2023-10-13 02:01:09 +02:00
Anton Popov
fb666e137c remove old code of projection analysis 2023-10-12 23:25:20 +00:00
Dmitry Novik
cec02c60e8 Merge remote-tracking branch 'origin/master' into parallel-window 2023-10-12 22:45:30 +00:00
Igor Nikonov
b220df93df Remove unnecessary total rows update in parallel replicas 2023-10-12 21:33:25 +00:00
yariks5s
9ae025d7e6 mid commit 2023-10-12 17:37:59 +00:00
Alexey Milovidov
b53a98217c
Merge pull request #55489 from kitaisreal/external-aggregation-improve-performance 
Improve performance of external aggregation with a lot of temporary files
 2023-10-12 18:00:41 +02:00
Alexey Milovidov
e48fad0e24
Merge pull request #55495 from kitaisreal/aggregating-transform-init-generate-race-condition-fix 
AggregatingTransform initGenerate race condition fix
 2023-10-12 17:54:36 +02:00
Dmitry Novik
f31a6f2c0d Support only Full Sort 2023-10-12 14:49:16 +00:00
Han Fei
42dc61abf5 Merge branch 'master' into hanfei/statistic 2023-10-12 16:19:45 +02:00
Azat Khuzhin
2cbb069b68 Add ability to ignore data after semicolon in Values format 
This is required for client, to handle comments in multiquery mode.

v0: separate context for input format
v2: cannot use separate context since params and stuff are changed in global context
v3: do not sent this setting to the server (breaks queries for readonly profiles)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-12 14:55:26 +02:00
Antonio Andelic
22f5d6d708 Reset promise_to_build 2023-10-12 10:57:21 +00:00
Azat Khuzhin
f379d9cac5 Fix checking of non handled data for Values format 
PeekableReadBuffer::hasUnreadData() does not checks the underlying
buffer, and so it simply ignore some issues, like:

    INSERT INTO test_01179_str values ('foo'); ('bar')

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-12 12:25:08 +02:00
Duc Canh Le
b3f4daf6c8 avoid casting columns 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-12 07:40:41 +00:00
Duc Canh Le
93a4d69c05 combine FINAL selection and WHERE filter 
Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-12 04:58:42 +00:00
taiyang-li
7cd24d0af0 solve conflicts 2023-10-12 10:30:05 +08:00
Robert Schulze
7b14392675
Merge pull request #55481 from rschu1ze/better-use-mysql-types-in-show-columns 
Make `use_mysql_types_in_show_columns` affect only `SHOW COLUMNS`
 2023-10-12 00:36:46 +02:00
Maksim Kita
ae94055a29 Updated implementation 2023-10-11 19:01:00 +03:00
Maksim Kita
f9033bdf31 Improve performance of external aggregation with a lot of temporary files 2023-10-11 19:01:00 +03:00
Igor Nikonov
62060a0603 Merge remote-tracking branch 'origin/master' into pr-coordinator-usage-cleanup 2023-10-11 15:11:15 +00:00
Robert Schulze
97d8e16e8d
Fix MySQL packet row data type 
Cf. https://github.com/ClickHouse/ClickHouse/pull/55479#discussion_r1355175581
 2023-10-11 15:09:50 +00:00
Igor Nikonov
9d95f4e1b6 Cleanup: parallel replica coordinator usage 2023-10-11 15:04:59 +00:00
robot-ch-test-poll1
80aa9cad51
Merge pull request #55479 from slvrtrn/fix-mysql-datetime-incompatibilities 
Fix MySQL text protocol DateTime formatting and LowCardinality(Nullable(T)) types reporting
 2023-10-11 16:56:26 +02:00
Raúl Marín
0b9bd809e7 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-11 16:50:22 +02:00
Maksim Kita
0859fc8de9 AggregatingTransform initGenerate race condition fix 2023-10-11 14:28:22 +03:00
Duc Canh Le
dfc8793b5c refactor implementation 
Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-11 10:30:38 +00:00
Duc Canh Le
c6aa438bad pass all stateless tests 
Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-11 06:51:42 +00:00
slvrtrn
e06d3ca1a5 Fix MySQL text protocol DateTime 
Introduce `removeLowCardinalityAndNullable` function
Fix incorrect removeLowCar/removeNullable usages
Add more MySQL text protocol tests
Deprecate old Java client tests
Use JDK 17 for test MySQL Java container
 2023-10-10 19:51:09 +02:00
Raúl Marín
95d2063e91 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-10 17:29:45 +02:00
Nikita Mikhaylov
4456fe40f9
Remove the old code for projection analysis (#55112) 2023-10-10 17:13:32 +02:00
Duc Canh Le
867ac957bf wip: pass all ci tests 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-10 11:31:48 +00:00
Alexey Milovidov
e204b1d616
Merge pull request #55419 from evillique/mongo-fix 
Fix MongoDB connection issues
 2023-10-10 01:30:04 +02:00
Nikolay Degterinsky
153b542b29 Small fix 2023-10-09 15:13:45 +00:00
Nikolay Degterinsky
a1e56a3df1 Fix MongoDB connection issues 2023-10-09 14:46:48 +00:00
Han Fei
fcb19b4f77 Merge branch 'master' into hanfei/statistic 2023-10-09 14:24:58 +02:00
Alexey Milovidov
e0258582ae
Merge pull request #55338 from azat/fix-CreatingSetsTransform-data-race 
Fix data-race in CreatingSetsTransform (on errors) due to throwing shared exception
 2023-10-09 13:27:10 +02:00
Alexey Milovidov
fac83efdd2
Merge pull request #55335 from azat/fix-sort-sigsegv 
Fix SIGSEGV in MergeSortingPartialResultTransform (due to zero chunks after remerge())
 2023-10-09 02:48:12 +02:00
Azat Khuzhin
56dbba1f95 Fix SIGSEGV in MergeSortingPartialResultTransform (due to zero chunks after remerge()) 
It is possible to have non zero input chunks before remerge() and zero
after:

    2023.10.08 10:21:20.944928 [ 4321 ] {427df456-1400-4fbe-8bd7-c4de139f00ca} <Debug> MergeSortingTransform: Re-merging intermediate ORDER BY data (1 blocks with 0 rows) to save memory consumption
    2023.10.08 10:36:16.447001 [ 14466 ] {} <Fatal> BaseDaemon: (version 23.10.1.1, build id: 3E53B6B3A53FD562F44C88703BD88EB713881A44, git hash: 5ddfb170a1096cf88664b1a4b9b7bd2e7ef36c29) (from thread 4321) (query_id: 427df456-1400-4fbe-8bd7-c4de139f00ca) (query: SELECT CounterID FROM test.visits ARRAY JOIN Goals.ID WHERE CounterID = 942285 ORDER BY CounterID

CI: https://s3.amazonaws.com/clickhouse-test-reports/55276/099665478df3e77d1df0332dd705ca7209e903af/stress_test__debug_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 22:21:01 +02:00
Alexey Milovidov
fa54b21424
Merge pull request #55103 from ClickHouse/fix_shell_command_abort 
Proper cleanup in case of exception in ctor of ShellCommandSource
 2023-10-08 21:36:28 +02:00
Azat Khuzhin
cd3c140dd6 Fix data-race in CreatingSetsTransform (on errors) due to throwing shared exception 
WARNING: ThreadSanitizer: data race (pid=3436)
  Read of size 1 at 0x7b50001b07a7 by thread T187:
    0 std::__1::basic_string<>::__is_long[abi:v15000] const build_docker/./contrib/llvm-project/libcxx/include/string:1499:33 (clickhouse-tsan+0x1faa29ff) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    1 std::__1::basic_string<>::size[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/string:968:17 (clickhouse-tsan+0x1faa29ff)
    2 std::__1::basic_string<>::empty[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/string:996:42 (clickhouse-tsan+0x1faa29ff)
    3 Poco::Exception::extendedMessage() build_docker/./base/poco/Foundation/src/Exception.cpp:114:13 (clickhouse-tsan+0x1faa29ff)
    4 DB::Exception::addMessage() <null> (clickhouse-tsan+0x7e0d5f1) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    5 DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:70:23 (clickhouse-tsan+0x1b3f2fed) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    6 DB::ExecutionThreadContext::executeTask() build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:95:9 (clickhouse-tsan+0x1b3f2fed)
    7 DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273:26 (clickhouse-tsan+0x1b3e5a10) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    8 DB::PipelineExecutor::executeSingleThread(unsigned long) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:239:5 (clickhouse-tsan+0x1b3e4bb8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    9 DB::PipelineExecutor::executeImpl(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:399:9 (clickhouse-tsan+0x1b3e4bb8)
    10 DB::PipelineExecutor::execute(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:111:9 (clickhouse-tsan+0x1b3e48be) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    11 DB::CompletedPipelineExecutor::execute() build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:110:18 (clickhouse-tsan+0x1b3e31c8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    12 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:202:14 (clickhouse-tsan+0x19a58396) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)

  Previous write of size 8 at 0x7b50001b07a0 by thread T192:
    0 std::__1::basic_string<>::__set_long_cap[abi:v15000](unsigned long) build_docker/./contrib/llvm-project/libcxx/include/string:1592:37 (clickhouse-tsan+0x238809e2) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    1 std::__1::basic_string<>::__grow_by_and_replace(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, char const*) build_docker/./contrib/llvm-project/libcxx/include/string:2367:5 (clickhouse-tsan+0x238809e2)
    2 std::__1::basic_string<>::append(char const*, unsigned long) build_docker/./contrib/llvm-project/libcxx/include/string:2692:9 (clickhouse-tsan+0x238809e2)
    3 std::__1::basic_string<>::append(char const*) build_docker/./contrib/llvm-project/libcxx/include/string:2849:12 (clickhouse-tsan+0x1faa2a58) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    4 Poco::Exception::extendedMessage() build_docker/./base/poco/Foundation/src/Exception.cpp:114:27 (clickhouse-tsan+0x1faa2a58)
    5 DB::Exception::addMessage() <null> (clickhouse-tsan+0x7e0d5f1) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    6 DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:70:23 (clickhouse-tsan+0x1b3f2fed) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    7 DB::ExecutionThreadContext::executeTask() build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:95:9 (clickhouse-tsan+0x1b3f2fed)
    8 DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273:26 (clickhouse-tsan+0x1b3e5a10) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    9 DB::PipelineExecutor::executeSingleThread(unsigned long) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:239:5 (clickhouse-tsan+0x1b3e4bb8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    10 DB::PipelineExecutor::executeImpl(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:399:9 (clickhouse-tsan+0x1b3e4bb8)
    11 DB::PipelineExecutor::execute(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:111:9 (clickhouse-tsan+0x1b3e48be) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    12 DB::CompletedPipelineExecutor::execute() build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:110:18 (clickhouse-tsan+0x1b3e31c8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    13 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:202:14 (clickhouse-tsan+0x19a58396) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)

  Location is heap block of size 480 at 0x7b50001b0600 allocated by thread T193:
    0 posix_memalign <null> (clickhouse-tsan+0x7d6ac85) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    1 std::__1::__libcpp_aligned_alloc[abi:v15000](unsigned long, unsigned long) build_docker/./contrib/llvm-project/libcxxabi/../libcxx/include/new:328:9 (clickhouse-tsan+0x238b5a05) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    2 __cxxabiv1::__aligned_malloc_with_fallback(unsigned long) build_docker/./contrib/llvm-project/libcxxabi/src/fallback_malloc.cpp:215:20 (clickhouse-tsan+0x238b5a05)
    3 __cxa_allocate_exception build_docker/./contrib/llvm-project/libcxxabi/src/cxa_exception.cpp:190:17 (clickhouse-tsan+0x238b31e5) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    4 std::exception_ptr std::make_exception_ptr[abi:v15000]<DB::Exception>(DB::Exception) build_docker/./contrib/llvm-project/libcxx/include/exception:206:9 (clickhouse-tsan+0x144bfa8a) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    5 DB::CreatingSetsTransform::~CreatingSetsTransform() build_docker/./src/Processors/Transforms/CreatingSetsTransform.cpp:30:45 (clickhouse-tsan+0x1b7603c3) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
    ...
    35 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:211:1 (clickhouse-tsan+0x19a58608) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)
 2023-10-08 17:16:39 +02:00
Azat Khuzhin
099665478d Fix incorrect merging of Nested for SELECT FINAL FROM SummingMergeTree 
The problem was the order of the columns, in case of SELECT FINAL it got
"counters_Map.count", "counters_Map.id"

But in case of OPTIMIZE FINAL it got "counters_Map.id",
"counters_Map.count" correctly.

Note, that this bugs exists there from the very recent versions, I've
checked 19.x and it was there.

P.S. there is a workaround for this problem, if you will use one of the
following patterns for key columns:
- *ID
- *Key
- *Type

That way it will be explicitly matched as key and everything will work.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:32:47 +02:00
Duc Canh Le
c1b0982a56 finalize implementation 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-06 08:32:46 +00:00
Duc Canh Le
c24c2c712e Merge branch 'master' into final_no_copy 
- resolve conflicts

Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-06 08:20:37 +00:00
Nikolai Kochetov
3bd71c937c Merge branch 'master' into fix-read-in-order-with-array-join 2023-10-04 09:17:09 +00:00
Nikolai Kochetov
d944b59902 Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-10-03 14:28:16 +00:00
vdimir
3a9abde35d
Merge pull request #54514 from ClickHouse/vdimir/allow_experimental_partial_result 
Add setting allow_experimental_partial_result
 2023-09-29 10:32:57 +02:00
Alexander Gololobov
d81d5dd75c Cleanup in case of exception in ctor 2023-09-28 16:28:35 +02:00
Nikita Taranov
0e506b618e
impl (#54934) 2023-09-28 14:12:19 +02:00
vdimir
a134569bc3
fix MergeSortingPartialResultTransform::getRealProcessorSnapshot 2023-09-28 09:41:01 +00:00
vdimir
4e86eec7d1
fix Pipe::unitePipes with partial results 2023-09-28 09:41:00 +00:00
vdimir
3f3feea0b7
Add setting allow_experimental_partial_result 2023-09-28 09:40:56 +00:00
robot-clickhouse-ci-1
7a825c1417
Merge pull request #54427 from Avogar/json-object-as-tuple-inference 
Add new features to schema inference for JSON formats
 2023-09-27 20:12:45 +02:00
Robert Schulze
2d750e9e2f
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks 2023-09-27 12:15:29 +00:00
Antonio Andelic
580aa0031c
Merge pull request #55013 from ClickHouse/another-promise-fix 
Set exception for promise in `CreatingSetsTransform` in more cases
 2023-09-27 09:01:42 +02:00
Michael Kolupaev
23bc286280
Print more information about one logical error in MergeTreeDataWriter (#55001) 2023-09-26 13:23:26 -07:00
Han Fei
4e7745f757 Merge branch 'master' into hanfei/statistic 2023-09-26 21:43:53 +02:00
Robert Schulze
cde10fe7b5
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks 2023-09-26 18:59:41 +00:00
Kruglov Pavel
bea80ab5b7
Merge branch 'master' into json-object-as-tuple-inference 2023-09-26 15:23:08 +02:00
Kruglov Pavel
69a17bbef6
Merge pull request #52853 from Avogar/http-valid-json-on-exception 
Output valid JSON/XML on excetpion during HTTP query execution
 2023-09-26 14:25:55 +02:00
Robert Schulze
9fff447716
Re-enable clang-tidy checks 2023-09-26 09:34:12 +00:00
Antonio Andelic
098efb261e Throw proper exception if promise was not set 2023-09-26 09:03:38 +00:00
avogar
9e75825515 Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-object-as-tuple-inference 2023-09-25 17:24:36 +00:00
avogar
42ca897f2d Better schema inference for JSON formats 2023-09-25 15:42:59 +00:00
Dmitry Novik
d4acd9ec61 Set correct stream sorting 2023-09-25 15:34:35 +00:00
Alex Cheng
d5d3cf0e9d Merge remote-tracking branch 'origin/master' into refactor-MergeJoinTransform 2023-09-25 19:41:35 +08:00
Alex Cheng
c7a88c8946 refactor MergeJoinTransform code. 2023-09-25 18:04:25 +08:00
robot-ch-test-poll4
9d61ed4665
Merge pull request #54664 from bigo-sg/arrow_join_max_block_size 
Respect max_block_size for array join to avoid possible OOM
 2023-09-23 21:30:51 +02:00
robot-ch-test-poll4
ba6f0431a5
Merge pull request #54933 from ClibMouse/feature/big-endian-bson-each-row 
Provide support for BSON on BE
 2023-09-23 03:00:27 +02:00
robot-clickhouse-ci-2
d98234dc9d
Merge pull request #54803 from Avogar/ephemeral-columns-from-files 
Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files
 2023-09-22 23:24:42 +02:00
kothiga
3e57b007a8
Use LE version of unalignedStore. 2023-09-22 12:25:17 -07:00
robot-clickhouse-ci-1
9a3d1d6296
Merge pull request #54920 from ClickHouse/set-promise-on-exception 
Set exception for promise in `CreatingSetsTransform`
 2023-09-22 20:45:09 +02:00
kothiga
80d511093b
Provide support for BSON on BE 2023-09-22 09:21:21 -07:00
Kruglov Pavel
4d675dbad4
Merge pull request #54825 from azat/fix-virtual-columns-filtering 
Fix filtering parts with indexHint for non analyzer (resubmit)
 2023-09-22 18:20:16 +02:00
Dmitry Novik
3b0579b339 Fix splitting into buckets 2023-09-22 14:44:40 +00:00
Antonio Andelic
a691dea6cf Set exception for promise 2023-09-22 13:13:54 +00:00
Robert Schulze
9a91f97ac4
Merge pull request #53831 from rschu1ze/clang-17 
Switch CI to Clang 17
 2023-09-22 12:28:35 +02:00
Igor Nikonov
b1cc698477
Merge pull request #54564 from vitlibar/fix-sorting-of-union-of-sorted 
Fix sorting of UNION ALL of already sorted results
 2023-09-21 22:49:53 +02:00
Robert Schulze
877e4f3aab
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 20:21:12 +00:00
Michael Kolupaev
9af9b4a085
Enable connection pooling for s3 table function (#54812) 
Enable connection pooling for s3 table function
 2023-09-21 09:27:20 -07:00
Robert Schulze
5209bd2d51
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-21 14:45:55 +00:00
Robert Schulze
f5137dd0b4
More clang-tidy fixes 2023-09-21 14:40:57 +00:00
avogar
3e08800cb5 Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files 2023-09-20 16:25:55 +00:00
Dmitry Novik
bb6952317c Merge remote-tracking branch 'origin/master' into parallel-window 2023-09-20 15:38:42 +00:00
Igor Nikonov
2d446fc1ca Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas 2023-09-20 13:40:08 +00:00
Kruglov Pavel
49ee14f701
Merge pull request #54809 from ClickHouse/pqmeta 
Prevent ParquetMetadata reading 40 MB from each file unnecessarily
 2023-09-20 12:53:22 +02:00
SmitaRKulkarni
135df6870c
Merge pull request #54706 from ClickHouse/replacing_merge_tree_with_clean_deleted_rows_disabled 
Fix ReplacingMergeTree with vertical merge and cleanup
 2023-09-20 12:06:55 +02:00
SmitaRKulkarni
d8adf05de2
Added a new column _block_number (#47532) 
Added a new virtual column _block_number which is persisted on merges when allow_experimental_block_number_column is enabled
 2023-09-20 11:31:12 +02:00
Azat Khuzhin
d9a634eb0f Fix filtering parts with indexHint for non analyzer 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit ffa82e9297)
 2023-09-20 11:29:35 +02:00
Michael Kolupaev
c856ec4087 Prevent ParquetMetadata reading 40 MB from each file unnecessarily 2023-09-19 21:58:50 +00:00
Igor Nikonov
e28f4e819c
Merge branch 'master' into parallel-replicas-not-enough-replicas 2023-09-19 23:05:24 +02:00
Azat Khuzhin
c439c4bca2
Revert "Fix filtering parts with indexHint for non analyzer" 2023-09-19 21:39:21 +02:00
Han Fei
eb86c113e5
Merge pull request #54184 from priera/fix_mv_insertion_when_duplicated_parts 
Inserting only non-duplicate chunks in MV
 2023-09-19 20:41:45 +02:00
robot-ch-test-poll4
a3dcce21b3
Merge pull request #54405 from Avogar/json-or-jsonl 
Parse data in JSON format as JSONEachRow if failed to parse metadata
 2023-09-19 20:15:09 +02:00
Kruglov Pavel
d555fdffa5
Merge pull request #54449 from azat/parts-prune-indexHint 
Fix filtering parts with indexHint for non analyzer
 2023-09-19 19:15:41 +02:00
Han Fei
17243b84ab Merge branch 'master' into hanfei/statistic 2023-09-19 17:09:17 +02:00
Smita Kulkarni
e2887ad27b Merge branch 'master' into replacing_merge_tree_with_clean_deleted_rows_disabled 2023-09-19 15:58:41 +02:00
avogar
f974970c3c Apply suggestion 2023-09-19 11:53:40 +00:00
avogar
5bd2e9f610 Fix tests 2023-09-19 11:53:40 +00:00
avogar
8c29408f5e Parse data in JSON format as JSONEachRow if failed to parse metadata 2023-09-19 11:53:40 +00:00
Kruglov Pavel
e163670357
Merge branch 'master' into http-valid-json-on-exception 2023-09-19 13:42:53 +02:00
Kruglov Pavel
3c83e43351
Remove debug logging 2023-09-19 13:38:43 +02:00
priera
b92e37d2df Fixing PR comments 2023-09-19 12:15:01 +02:00
priera
663ce74609 fixed clang-tidy error 2023-09-19 12:15:00 +02:00
priera
b30e33580b fix test 2023-09-19 12:15:00 +02:00
priera
6d27ea2742 fixed style 2023-09-19 12:15:00 +02:00
priera
94d737264c inserting non-duplicate chunks 2023-09-19 12:15:00 +02:00
Smita Kulkarni
142a4631f1 Updated to clear current_row_sources when cleanedup rows are added 2023-09-19 11:22:10 +02:00
Igor Nikonov
047d214436 Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas 2023-09-18 15:29:56 +00:00
Igor Nikonov
e1019ba3c4 Disabling parallel replicas per shard will be done separately 2023-09-18 15:27:55 +00:00
Kruglov Pavel
9c888ea42b
Merge pull request #53549 from Avogar/group-by-constant-keys 
Optimize group by constant keys
 2023-09-18 12:12:40 +02:00
taiyang-li
39ca44b51a Respect max_block_size for array join to avoid possible OOM 2023-09-18 11:04:38 +08:00
Robert Schulze
f5e8028bb1
Merge pull request #54642 from rschu1ze/broken-re2st 
Remove broken lockless variant of re2
 2023-09-17 15:30:57 +02:00
Amos Bird
0518b64b58
Fix nullable primary key in final (#54164) 
* Fix nullable primary key in final

* Real fix

* Address reviews
 2023-09-15 22:44:13 +02:00
Smita Kulkarni
8ef910e1fa Updated to check if cleanup is enabled before adding to cleanedup_rows_count 2023-09-15 22:34:09 +02:00
Kruglov Pavel
2075f9c667
Merge branch 'master' into group-by-constant-keys 2023-09-15 15:10:08 +02:00
avogar
35d975bfea Add comment in ParallelInputFormat, remove unneded include 2023-09-15 13:07:04 +00:00
Kruglov Pavel
dbd24b240c
Merge branch 'master' into http-valid-json-on-exception 2023-09-15 14:55:31 +02:00
Sema Checherinda
45cf793544
Merge pull request #53919 from canhld94/optimize_all_lonely_parts 
Do not implicitly read pk and version columns in lonely parts if unnecessary
 2023-09-15 13:11:00 +02:00
Robert Schulze
7b378dbad3
Remove broken lockless variant of re2 2023-09-14 16:40:42 +00:00
Vitaly Baranov
9a0e1ef592 Fix sorting of UNION ALL of already sorted results. 2023-09-14 15:04:37 +02:00
Robert Schulze
a9ae813db0
Merge pull request #54115 from slvrtrn/simplified-prepared-statements-for-mysql 
Implement the MySQL binary protocol implementation for initial support of Tableau Online
 2023-09-14 12:27:11 +02:00
slvrtrn
c0961d9378 Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-09-13 19:33:11 +02:00
Sema Checherinda
8a9b544a97
Merge branch 'master' into optimize_all_lonely_parts 2023-09-13 16:07:19 +02:00
Arthur Passos
da8caeffd2 Merge branch 'master' into arrow_parquet_account_for_monotonically_increasing_offsets_across_batches 2023-09-12 17:50:36 -03:00
Igor Nikonov
7b3f32b95a
Merge pull request #54520 from ClickHouse/pr-cleanup 
Parallel replicas: cleanup unused params
 2023-09-12 19:48:18 +02:00
slvrtrn
dddea9219a Address the review comments 2023-09-12 18:39:03 +02:00
Anton Popov
3e242f8ba3
Merge pull request #54351 from Avogar/null-as-default-nested 
Support NULL as default for nested types Array/Tuple/Map for input formats
 2023-09-12 14:58:21 +02:00
Igor Nikonov
1287f68745 Handle clusterAllReplicas/remote cases to avoid unnecessary logging 2023-09-12 12:52:29 +00:00
robot-clickhouse
63243fbc03
Merge pull request #54480 from amosbird/fix_54406 
Fix aggregate projections with normalized states
 2023-09-12 13:43:41 +02:00
slvrtrn
611a75a87f Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-09-12 10:38:44 +02:00
Igor Nikonov
d5ea047ab8 Parallel replicas: cleanup unused params 2023-09-11 21:52:40 +00:00
Igor Nikonov
2293923f66 Disable parallel replicas on shards with not enough nodes 2023-09-11 21:46:46 +00:00
Amos Bird
667426f1f2
DataTypeAggregateFunction::strictEquals 2023-09-12 03:54:19 +08:00
avogar
803d8dcf85 Support NULL as default for nested types Array/Tuple/Map for input formats 2023-09-11 18:18:33 +00:00
Nikolai Kochetov
903c966cc8
Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-09-11 16:14:03 +02:00
Nikolai Kochetov
59108f3d96
Merge pull request #54476 from ClickHouse/revert-54470-revert-48607-master 
Revert "Revert "Add settings for real-time updates during query execution""
 2023-09-11 14:59:13 +02:00
Amos Bird
fb0f9ff565
Fix aggregate projections with normalized states 2023-09-10 03:21:22 +08:00
Alexey Milovidov
c0a43df749
Merge pull request #54435 from Alex-Cheng/small_finetune_merge_join 
Small fine-tune for using ColumnNullable pointer
 2023-09-09 19:28:28 +03:00
alexX512
7f77869216 Increase amount of rows in requests for partial result tests 
(cherry picked from commit 54bc3fea9a)
 2023-09-09 10:33:05 +00:00
Nikolai Kochetov
9b936c44db
Revert "Revert "Add settings for real-time updates during query execution"" 2023-09-09 12:29:39 +02:00
Alexey Milovidov
03a755732a
Revert "Add settings for real-time updates during query execution" 2023-09-09 03:10:23 +03:00
Azat Khuzhin
ffa82e9297 Fix filtering parts with indexHint for non analyzer 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-08 18:17:06 +02:00
Anton Popov
1aa34c16e3 Merge remote-tracking branch 'upstream/master' into HEAD 2023-09-08 14:00:45 +00:00
Nikolai Kochetov
0095124791
Merge pull request #48607 from alexX512/master 
Add settings for real-time updates during query execution
 2023-09-08 09:05:33 +02:00
robot-ch-test-poll2
f28ad1e136
Merge pull request #54368 from ClickHouse/fix_replacing_final_error 
Fix logical error in vertical merge + replacing merge tree + optimize cleanup
 2023-09-07 15:45:43 +02:00
Alex Cheng
75dc6d9708 small fine-tune for using ColumnNullable pointer 2023-09-07 20:41:32 +08:00
Arthur Passos
f34d40cde3 docs 2023-09-06 17:26:34 -03:00
Arthur Passos
a8027be612 fix 2023-09-06 17:25:27 -03:00
robot-ch-test-poll3
7f756ed613
Merge pull request #54354 from ClickHouse/parallel-replicas-cleanup 
Parallel replicas: remove unused code
 2023-09-06 19:38:14 +02:00
Arthur Passos
ce673d6bea arrow-parquet account for monotonically increasing offsets across multiple batches 2023-09-06 13:59:39 -03:00
Alexander Sapin
4368c00052 Fix logical error in vertical merge + replacing merge tree + optimize cleanup 2023-09-06 18:43:56 +02:00
Kruglov Pavel
5b29640b22
Merge branch 'master' into group-by-constant-keys 2023-09-06 18:06:51 +02:00
Duc Canh Le
638f2e4c87 wip: final without copy row-by-row 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-09-06 16:04:54 +00:00
robot-ch-test-poll4
8c71582316
Merge pull request #54185 from ClickHouse/analyzer-partition-id-pruning 
Fix partition id pruning for analyzer.
 2023-09-06 16:53:35 +02:00
Igor Nikonov
c51cfef065 Remove unused code 2023-09-06 12:53:58 +00:00
Igor Nikonov
54c5158291
Merge pull request #54315 from ClickHouse/add-logs-parallel-replica-over-distributed 
Add logs for parallel replica over distributed
 2023-09-06 14:23:45 +02:00
Anton Popov
401496aa1f Apply suggestions from code review 
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-09-06 10:27:46 +00:00
robot-clickhouse-ci-1
02339a1f22
Merge pull request #54326 from ClickHouse/fix_segfault_system_zookeeper 
Fix segfault in system.zookeeper
 2023-09-06 02:20:50 +02:00
Alexander Tokmakov
a8489c21da fix segfault in system.zookeeper 2023-09-05 20:23:16 +02:00
Igor Nikonov
47d572d8bc Log cluster_for_parallel_replicas on initiator 2023-09-05 13:17:10 +00:00
slvrtrn
6e0a254368 Update tests, update imports, remove requirements 2023-09-04 21:50:06 +02:00
slvrtrn
bb0eff9669 Revert format changes 2023-09-04 21:15:26 +02:00
irenjj
c9261bbf18 Merge remote-tracking branch 'upstream/master' into feat_markdown 2023-09-04 22:35:53 +08:00
Han Fei
f60dad0598 Merge branch 'master' into hanfei/statistic 2023-09-04 16:17:07 +02:00
Raúl Marín
e192d4c624
Fix summary reporting with parallel replicas with LIMIT (#53050) 2023-09-04 15:53:06 +02:00
Alexey Milovidov
b660ac9bf1 Merge #54236 2023-09-04 03:57:39 +02:00
Alexey Milovidov
476e15ce3d
Merge pull request #54236 from YinZheng-Sun/Fix 
remove semicolon
 2023-09-04 04:54:30 +03:00
Yinzheng-Sun
74ec546052 remove semicolon 2023-09-03 21:52:39 +08:00
Nikolai Kochetov
63c681daf4 Try fix tests. 2023-09-01 18:20:56 +00:00
Nikolai Kochetov
4cbfca78e4 Fixing 01748_partition_id_pruning 2023-09-01 14:29:31 +00:00
alexX512
36b8932b37 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-31 21:40:50 +00:00
Nikolai Kochetov
52a7206002 Fix rows_before_limit_at_least for DelayedSource. 2023-08-31 12:53:58 +00:00
Robert Schulze
aefb543734
Merge remote-tracking branch 'rschu1ze/master' into feat_markdown 2023-08-31 11:32:44 +00:00
slvrtrn
8378441248 Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-08-30 19:00:51 +02:00
slvrtrn
f4dd6be8a8 WIP prepared statements 2023-08-30 18:59:39 +02:00
Duc Canh Le
11c94246c7 do not implicitly read pk + version and compute sort expression for lonely parts 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-08-30 05:05:57 +00:00
Sergei Trifonov
802579f3f1
Merge pull request #49618 from ClickHouse/concurrency-control-controllable 
Make concurrency control controllable
 2023-08-29 19:44:51 +02:00
Anton Popov
02391eafe1 refactoring of reading from MergeTree tables 2023-08-29 13:40:12 +00:00
alexX512
2670af53df Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-29 13:37:58 +00:00
alexX512
70f926a75a Style fix 2023-08-29 13:37:46 +00:00
alexX512
238f6e8327 fix style 2023-08-29 13:15:38 +00:00
Kseniia Sumarokova
463a197da4
Merge pull request #53847 from kssenii/fix-segfault-in-pg 
Fix possible segfault while using PostgreSQL engine
 2023-08-29 14:45:46 +02:00
alexX512
b3ba00f443 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-29 10:59:29 +00:00
alexX512
47a908dfa8 Add support for nested supqueries for AggregationPartialResult transform and more comments 2023-08-29 10:45:34 +00:00
Duc Canh Le
e9b0d3e4a2 do not implicitly read pk and version columns in lonely parts if nnot necessary 2023-08-29 09:41:18 +00:00
Raúl Marín
45e8185f9b Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-08-29 11:17:34 +02:00
Alexey Milovidov
b7d8ebf5e2
Merge pull request #53511 from Enmk/fix-ReplacingMergeTree-do_not_merge_across_partitions_select_final 
Fixed SELECTing from ReplacingMergeTree with do_not_merge_across_partitions_select_final
 2023-08-28 16:07:06 +03:00
vdimir
b92c436c29
Merge pull request #53699 from ClickHouse/vdimir/filter_pushdown_const_bug 2023-08-28 11:45:05 +02:00
irenjj
51aa89eed8 Add a setting to automatically escape special characters in Markdown. 2023-08-28 00:10:33 +08:00
alesapin
0e601b856f
Merge pull request #53751 from Algunenano/clang18 
Support clang-18 (Wmissing-field-initializers)
 2023-08-25 23:27:06 +02:00
slvrtrn
055d2e3c3d Merge remote-tracking branch 'origin' into simplified-prepared-statements-for-mysql 2023-08-25 21:27:47 +02:00
slvrtrn
734ffd916c WIP prepared statements 2023-08-25 20:31:21 +02:00
kssenii
4af1bf60ed Fix 2023-08-25 19:24:31 +02:00
kssenii
49d33774f9 Fix 2023-08-25 17:47:56 +02:00
Vasily Nemkov
f896c6349c Merge commit upstream/master into fix-ReplacingMergeTree-do_not_merge_across_partitions_select_final 2023-08-25 16:02:54 +02:00
Raúl Marín
6490213207 Merge remote-tracking branch 'blessed/master' into clang18 2023-08-24 17:35:26 +02:00
Kruglov Pavel
44db5fa992
Merge branch 'master' into cache-count 2023-08-24 17:21:18 +02:00
avogar
c4f8113a78 Fix tests 2023-08-24 15:05:45 +00:00
Kruglov Pavel
cd3433f95a
Merge pull request #53407 from Avogar/tsv-single-carriage-return 
Fix processing single carriage return in TSV file segmentation engine
 2023-08-24 16:06:05 +02:00
Han Fei
10c622fa08 Merge branch 'master' into hanfei/statistic 2023-08-24 15:56:06 +02:00
Nikolai Kochetov
fc90a1a0bd Fix some skip_unused_shards tests. 2023-08-24 11:07:17 +00:00
Kruglov Pavel
f7e1abd774
Merge branch 'master' into cache-count 2023-08-23 22:31:49 +02:00
avogar
93ba73715a Merge branch 'master' of github.com:ClickHouse/ClickHouse into group-by-constant-keys 2023-08-23 18:47:42 +00:00
Raúl Marín
93dac0c880 Support clang-18 (Wmissing-field-initializers) 2023-08-23 15:53:45 +02:00
avogar
68e3af56d4 Address comments 2023-08-23 13:19:15 +00:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count 2023-08-23 15:18:02 +02:00
Kruglov Pavel
fee323b122
Update src/Processors/Formats/IRowInputFormat.h 
Co-authored-by: Michael Kolupaev <michael.kolupaev@clickhouse.com>
 2023-08-23 12:15:47 +02:00
Kruglov Pavel
e193aec583
Merge branch 'master' into fast-count-from-files 2023-08-23 12:15:34 +02:00
Alexey Milovidov
67c3b7fa65
Merge pull request #53711 from ClickHouse/sorted-distinct-sparse-columns 
Fix: sorted distinct with sparse columns
 2023-08-23 11:34:01 +03:00
Igor Nikonov
22895c1d0d Fix: sorted distinct with sparse columns 2023-08-22 23:33:30 +00:00
avogar
cd228754c8 Apply suggestions 2023-08-22 17:32:35 +00:00
Kruglov Pavel
6e3f2fedbd
Apply suggestions from code review 
Co-authored-by: Dmitry Novik <mrnovikd@gmail.com>
 2023-08-22 19:05:06 +02:00
vdimir
b23773380c
Fix filter pushdown for full_sorting_merge join 2023-08-22 15:06:50 +00:00
avogar
6119246948 Better 2023-08-22 14:51:46 +00:00
robot-clickhouse-ci-1
698ceee1a9
Merge pull request #53631 from amosbird/fix_53630 
Fix data race of shell command
 2023-08-22 15:12:26 +02:00
Kruglov Pavel
67c5c0203b
Merge branch 'master' into fast-count-from-files 2023-08-22 15:03:48 +02:00
avogar
7f9e81d504 Clean up 2023-08-22 12:55:00 +00:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count 2023-08-22 14:42:22 +02:00
avogar
b4145aeddc Cache number of rows in files for count in file/s3/url/hdfs/azure functions 2023-08-22 11:59:59 +00:00
Antonio Andelic
e0af6e5879
Merge pull request #53644 from ClickHouse/fix-delayed-source-with-totals 
Correctly handle totals and extremes with `DelayedSource`
 2023-08-22 12:24:16 +02:00
Robert Schulze
acaa2eee58
Merge pull request #53556 from ClibMouse/feature/big-endian-UUID-correction 
Change Big Endian-UUID to work the same as Little Endian-UUID
 2023-08-22 11:57:44 +02:00
Alexey Milovidov
9874556d85
Merge pull request #53655 from ClibMouse/more-fix-big-endian 
State of State and avg aggregation function fix for big endian
 2023-08-22 11:49:54 +03:00
Amos Bird
f13fd5b5dc
Wait for some pooled commands 2023-08-22 16:26:35 +08:00
Amos Bird
bd9f526f93
Fix data race of shell command 2023-08-22 14:50:56 +08:00
Michael Kolupaev
d752611c43 Performance test 2023-08-21 14:15:52 -07:00
Michael Kolupaev
2f4d433e69 Parquet filter pushdown 2023-08-21 14:15:52 -07:00
Michael Kolupaev
6009e1b293
Merge pull request #53324 from bigo-sg/ch_gluten_2583 
Implement native orc input format without arrow to improve performance
 2023-08-21 13:44:57 -07:00
Alexey Milovidov
d7180c7d5c
Merge pull request #53645 from ClickHouse/prepared-set-cache-in-mutation-pipeline-stuck 
Prepared set cache in mutation pipeline stuck
 2023-08-21 22:21:45 +03:00
Suzy Wang
7cfc2a0872 update api name 2023-08-21 12:18:32 -07:00
Suzy Wang
6edf8f78cc more fixes for big endian 2023-08-21 12:13:56 -07:00
Nikolai Kochetov
f0b68393a8
Merge pull request #53162 from ClickHouse/not-ready-set-bug 
Not-ready Set
 2023-08-21 21:09:15 +02:00
Kruglov Pavel
fed253a671
Fix build 2023-08-21 16:46:23 +02:00
Antonio Andelic
c4bf151d83 Correctly handle totals and extremes with DelayedSource 2023-08-21 14:07:59 +00:00
Nikolai Kochetov
2d932f0953 Remove obsolete code. 2023-08-21 14:04:07 +00:00
Kruglov Pavel
098818b234
Fix build 2023-08-21 15:45:45 +02:00
Nikolai Kochetov
e9149981ea Fix check for delayed ports. 2023-08-21 13:08:32 +00:00
Kruglov Pavel
88aee95122
Merge branch 'master' into fast-count-from-files 2023-08-21 14:46:33 +02:00
avogar
e590f15d3d Fix for carriage return in tsv 2023-08-21 12:46:03 +00:00
avogar
584bd57f5c Clean up 2023-08-21 12:43:11 +00:00
avogar
47304bf7aa Optimize count from files in most input formats 2023-08-21 12:30:52 +00:00
Kruglov Pavel
c68456a20a
Merge pull request #52692 from Avogar/variable-number-of-volumns-more-formats 
Allow variable number of columns in more formats, make it work with schema inference
 2023-08-21 13:28:35 +02:00
avogar
4c12914406 Fix single carriage return processing in TSV file segmentation engine 2023-08-21 11:26:18 +00:00
Nikolai Kochetov
388e2a09d6 Merge branch 'master' into not-ready-set-bug 2023-08-21 10:59:51 +00:00
taiyang-li
e5a399d0e9 change as requested 2023-08-21 14:51:25 +08:00
taiyang-li
f2e4315d59 fix building 2023-08-21 14:11:48 +08:00
taiyang-li
3cd9eb9b84 fix style 2023-08-21 12:12:31 +08:00
taiyang-li
3781245027 change as request 2023-08-21 12:10:39 +08:00
taiyang-li
f723e8d43a change as request 2023-08-21 12:09:02 +08:00
Igor Nikonov
a69ea02418 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-19 21:43:25 +00:00
taiyang-li
9866d2727e change as request 2023-08-19 17:31:19 +08:00
Alexey Milovidov
c42c4046fe
Merge pull request #53573 from amosbird/fix_53454 
Proper destruction of task in ShellCommandSource
 2023-08-19 12:19:29 +03:00
Amos Bird
052281896a
Proper destruction of task 2023-08-19 13:56:57 +08:00
Michael Kolupaev
a1522e22ea
Merge pull request #53281 from Avogar/batch-small-parquet-row-groups 
Optimize reading small row groups by batching them together in Parquet
 2023-08-18 17:15:42 -07:00
Igor Nikonov
aebda10f7c Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-18 23:00:58 +00:00
kothiga
f33c585bc5
Addressing feedback. 2023-08-18 13:50:31 -07:00
Austin Kothig
6b42975d33
Change BE-UUID to work the same as LE-UUID. Included high and low getters to provide cleaner code when accessing undertype. 2023-08-18 08:19:46 -07:00
avogar
7fa72f5cdf Optimize group by constant keys 2023-08-18 12:47:26 +00:00
Kruglov Pavel
3bae63e48e
Fix special build 2023-08-18 13:56:32 +02:00
taiyang-li
4f9429d2e4 fix ut tests/queries/0_stateless/00900_null_array_orc_load.sh 2023-08-18 17:58:24 +08:00
Vasily Nemkov
55168dc075 Merge remote-tracking branch 'upstream/master' into fix-ReplacingMergeTree-do_not_merge_across_partitions_select_final 2023-08-18 11:41:56 +02:00
Amos Bird
1c300a31b1
Fix build 2023-08-18 15:38:48 +08:00
Amos Bird
b2dd055cdf
Use poll instead 2023-08-18 15:38:48 +08:00
Amos Bird
c43bf153f5
Refactor 2023-08-18 15:38:46 +08:00
Amos Bird
dd0c71b32a
Add error_exit_reaction 2023-08-18 15:38:46 +08:00
Amos Bird
940ba60ae0
Fix non-linux build 2023-08-18 15:38:45 +08:00
Amos Bird
476f3cedc1
Various reactions when executable stderr has data 2023-08-18 15:38:45 +08:00
taiyang-li
15720d9cef fix ut tests/queries/0_stateless/02518_parquet_arrow_orc_boolean_value.sh 2023-08-18 15:10:25 +08:00
taiyang-li
8e0d5b7ee0 fix bugs 2023-08-18 13:31:26 +08:00
Igor Nikonov
06c98956b3 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 20:42:50 +00:00
Igor Nikonov
9f771b10e9 Cleanup 2023-08-17 20:42:22 +00:00
Alexander Tokmakov
89c365803f
Merge pull request #53424 from ClickHouse/exception_message_patterns6 
Less exceptions with runtime format string
 2023-08-17 17:58:04 +03:00
Igor Nikonov
3e9ae1751e Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 14:45:37 +00:00
avogar
bca91548ad Add setting input_format_parquet_local_file_min_bytes_for_seek 2023-08-17 12:28:01 +00:00
Robert Schulze
ec7daed52f
Merge pull request #50276 from ClibMouse/feature/mergetree-checksum-big-endian-support 
MergeTree & SipHash checksum big-endian support
 2023-08-17 13:27:18 +02:00
Nikolai Kochetov
8dbe82c192
Merge pull request #53489 from ClickHouse/fix-wrong-column-order-in-parallel-final 
Fix wrong columns order for queries with parallel FINAL.
 2023-08-17 13:22:13 +02:00
Vasily Nemkov
322db80cd9 Fixed SELECTing from ReplacingMergeTree with do_not_merge_across_partitions_select_final 2023-08-17 12:35:55 +02:00
李扬
cfea6db997
Update native_orc.cpp 2023-08-17 16:57:41 +08:00
taiyang-li
c4777397f7 fix integration test test_kafka_formats 2023-08-17 14:42:51 +08:00
Igor Nikonov
2e4d346e44 Fixes 2023-08-16 23:31:20 +00:00
Igor Nikonov
21ef1f1d1c Remove unnecessary Scalars usage for cluster_for_parallel_replicas 2023-08-16 23:24:01 +00:00
Nikolai Kochetov
3a6a3d491e Fix typo 2023-08-16 20:40:28 +00:00
Nikolai Kochetov
91c241532f Add a check for header structure. 2023-08-16 20:26:23 +00:00
Nikolai Kochetov
30ccfa1638 Fix wrong columns order for queries with parallel FINAL. 2023-08-16 20:23:10 +00:00
taiyang-li
8ba1da755d update example 2023-08-16 11:18:44 +08:00
Alexander Tokmakov
ba44d7260e fix 2023-08-16 00:20:28 +02:00
Michael Kolupaev
cbe4c8adc2 Fix more functions with 'Context has expired' error 2023-08-15 20:37:10 +00:00
Igor Nikonov
9825f8c76e Remove debug logging 2023-08-15 17:03:32 +00:00
taiyang-li
c2017e0ea3 update orc version 2023-08-15 14:49:45 +08:00
taiyang-li
724af579d7 revert file 2023-08-15 12:17:49 +08:00
taiyang-li
bfa9d361cc fix tests 2023-08-15 12:16:22 +08:00
Igor Nikonov
9f7a25eead Fix style once more 2023-08-14 23:27:36 +00:00
Igor Nikonov
e82b6b02ca Fix style 2023-08-14 23:15:27 +00:00
Igor Nikonov
c94994afcf Propagate setting cluster_for_parallel_replicas to shards 2023-08-14 22:50:46 +00:00
ltrk2
2846ea49b4
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-14 13:02:30 -04:00
Igor Nikonov
709287fbdc Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-14 14:37:43 +00:00
taiyang-li
bbe5caa9dd fix building 2023-08-14 19:55:45 +08:00
李扬
613bef89f4
Merge branch 'master' into ch_gluten_2583 2023-08-14 17:44:15 +08:00
taiyang-li
34c3162c7c revert old orc files 2023-08-14 17:42:42 +08:00
taiyang-li
832768745b add examples 2023-08-14 17:27:01 +08:00
taiyang-li
8ee1912de6 remove old orc files 2023-08-14 17:26:24 +08:00
taiyang-li
57bef64fbc add new native orc files 2023-08-14 17:25:58 +08:00
Alexey Milovidov
b0687e650a
Merge pull request #53209 from Avogar/format-one 
Add input format One
 2023-08-13 21:32:11 +03:00
Alexey Milovidov
5b243d2990
Merge pull request #53006 from kitaisreal/explain-actions-for-join-step 
EXPLAIN actions for JOIN step
 2023-08-12 02:51:07 +03:00
Sergei Trifonov
771710b377
Merge branch 'master' into concurrency-control-controllable 2023-08-11 16:50:13 +02:00
avogar
7e863a2726 Address comments 2023-08-11 13:17:49 +00:00
avogar
3ad7e57059 Optimize reading small row groups by batching them together in Parquet 2023-08-11 13:17:45 +00:00
Kruglov Pavel
b6b0e9c6bc
Merge branch 'master' into variable-number-of-volumns-more-formats 2023-08-11 14:02:21 +02:00
Kruglov Pavel
00865a7dad
Merge branch 'master' into format-one 2023-08-11 13:54:58 +02:00
ltrk2
1dc1b54c68
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-11 07:48:49 -04:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache 
Better usage of ccache
 2023-08-11 04:43:45 +03:00
Alexey Milovidov
fd7b92e90a
Merge pull request #53135 from ClickHouse/file_diagnostics_while_reading_header 
Add diagnostic info about file name during schema inference
 2023-08-10 21:56:12 +03:00
Alexey Milovidov
8fcfdd2bfd
Merge branch 'master' into explain-actions-for-join-step 2023-08-10 17:33:25 +03:00
ltrk2
a2054c04dd
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-10 10:21:34 -04:00
Alexey Milovidov
bb57caa83e
Merge pull request #53180 from ClickHouse/ditch-tons-of-garbage 
Ditch tons of garbage
 2023-08-10 16:59:51 +03:00
avogar
078683e226 Fix tests 2023-08-10 13:07:06 +00:00
Kruglov Pavel
6600f87f86
Merge branch 'master' into http-valid-json-on-exception 2023-08-10 13:53:32 +02:00
avogar
82aff97dd0 Add comment, more test 2023-08-10 11:51:36 +00:00
Igor Nikonov
0282b6961a Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-10 11:47:18 +00:00
serxa
de655457be fix race 2023-08-10 11:37:43 +00:00
Kruglov Pavel
bb38918a26
Apply suggestions from code review 
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
 2023-08-10 13:21:11 +02:00
Nikolai Kochetov
f27def1361
Merge branch 'master' into fix-read-in-order-with-array-join 2023-08-10 11:49:18 +02:00
Han Fei
2b1f2f1de2 Merge branch 'master' into hanfei/statistic 2023-08-10 00:52:24 +02:00
Han Fei
f5c9d278ad use statistic to order prewhere conditions better 2023-08-09 22:57:49 +02:00
Kruglov Pavel
33a39900ad
Merge branch 'master' into variable-number-of-volumns-more-formats 2023-08-09 19:51:17 +02:00
ltrk2
139e9433a8
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-09 09:48:00 -04:00
Nikolai Kochetov
a2de881975 Fix check for finished outputs in DelayedPortsProcessor. Do not make input ports needed too early in Resize. 2023-08-09 13:35:39 +00:00
Kruglov Pavel
70659e9721
Fix style 2023-08-09 15:07:49 +02:00
avogar
01a7c7560f Add input format One 2023-08-09 11:25:32 +00:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
Alexey Milovidov
aa757490bd Ditch tons of garbage 2023-08-09 02:19:02 +02:00
Alexey Milovidov
5561e3e198 Remove garbage and speed up Debug and Tidy builds 2023-08-09 01:44:39 +02:00
Alexey Milovidov
6804377b72 Merge branch 'master' into file_diagnostics_while_reading_header 2023-08-08 22:03:30 +02:00
Alexander Tokmakov
52bc5aab94
Merge branch 'master' into fix-full-sorting-desc 2023-08-08 12:40:10 +03:00
Alexey Milovidov
5dd99db369 Add diagnostic info about file name during schema inference 2023-08-08 03:55:06 +02:00
Igor Nikonov
fd69b18184
Merge branch 'master' into fix-parallel-replicas-multiply-result 2023-08-07 21:14:33 +02:00
alexX512
520a3c6eeb Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-07 16:52:51 +00:00
alexX512
51372d841d Style fix 2023-08-07 16:52:38 +00:00
alexX512
080b4badbd Support partial result for aggregating transform during query execution 2023-08-07 15:58:14 +00:00
Sergei Trifonov
dcac12c8f9
Merge branch 'master' into concurrency-control-controllable 2023-08-07 13:43:02 +02:00
vdimir
490dfc93c5
Merge pull request #53000 from ClickHouse/vdimir/join_filter_set_sparse 2023-08-07 12:02:56 +02:00
Maksim Kita
9a70b0d417 Fixed tests 2023-08-07 11:06:25 +03:00
Igor Nikonov
cc2cf611a3 fix typo/style 2023-08-06 09:37:41 +00:00
Tanay Tummalapalli
fd58809207 Fix description for filtering sets in full_sorting_merge join 
The description of the `ReadHeadBalancedProcessor` and
`FilterBySetOnTheFlyTransform` was the same as the
`CreatingSetsOnTheFlyTransform` in an `EXPLAIN PIPELINE`
when using `full_sorting_merge`.
 2023-08-06 07:53:28 +05:30
Igor Nikonov
22b73c95a7 Fix comment 2023-08-05 23:31:27 +00:00
Igor Nikonov
6f04dc42d3
Merge branch 'master' into fix-parallel-replicas-multiply-result 2023-08-06 01:23:38 +02:00
Igor Nikonov
a8b782ca5a Parallel reading from replicas over disributed with several shards 2023-08-05 23:19:09 +00:00
Igor Nikonov
cca6f2f297 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-05 00:06:58 +00:00
Igor Nikonov
2bda97d8f7 Potential fix for custom key tests 2023-08-05 00:06:54 +00:00
Alexey Milovidov
e89bd2932c
Merge branch 'master' into vdimir/join_filter_set_sparse 2023-08-05 00:49:52 +03:00
Michael Kolupaev
4ed86fea2f Fix Parquet stats for Float32 and Float64 2023-08-04 21:01:07 +00:00
Kruglov Pavel
b1fe9ce8f7
Merge branch 'master' into formats-with-subcolumns 2023-08-04 13:01:12 +02:00
Maksim Kita
a0fa3cc73b EXPLAIN actions for JOIN step 2023-08-03 18:55:15 +03:00
Igor Nikonov
e91b972486 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-03 15:46:33 +00:00
Igor Nikonov
6a0a0ff30d One line fix for the specific query 2023-08-03 15:40:24 +00:00
vdimir
13d5ef9164
Convert sparse to full in CreateSetAndFilterOnTheFlyStep 2023-08-03 14:43:03 +00:00
ltrk2
a29b36612b
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-03 09:41:25 -04:00
Alexey Milovidov
86f9c120d7 Merge branch 'master' into more-checks-in-thread-status 2023-08-03 14:34:31 +02:00
Alexey Milovidov
b740922c1e
Merge branch 'master' into more-checks-in-thread-status 2023-08-03 00:10:59 +03:00
alexX512
0d84226914 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-08-02 19:18:59 +00:00
alexX512
ee01d64350 Style fix 2023-08-02 18:42:57 +00:00
alexX512
06e187e58a Split mergeSortingTransform and mergeSortingPartialResultTransform into 2 files 2023-08-02 18:15:25 +00:00
Igor Nikonov
44b9ed438f
Merge branch 'master' into fix-window-func-range-check 2023-08-02 19:24:55 +02:00
ltrk2
27a2d4d1c7
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-02 11:36:43 -04:00
Anton Popov
ff137773e7
Merge branch 'master' into formats-with-subcolumns 2023-08-02 15:24:56 +02:00
Nikita Taranov
2cbe79b529
Fix memory consumption when max_block_size is huge in Aggregator + more memory checks (#51566) 
* impl

* remove checks from without_key methods

* maybe will improve smth

* add test

* Update 02797_aggregator_huge_mem_usage_bug.sql

* Update 02797_aggregator_huge_mem_usage_bug.sql

---------

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2023-08-02 15:11:52 +02:00
alexX512
9ac2d12f3b Add test for partial sorting result 2023-08-02 13:07:02 +00:00
Kruglov Pavel
81866bcc9c
Fix special build 2023-08-02 12:35:58 +02:00
avogar
d12e96177a Fix tests 2023-08-01 16:17:03 +00:00
Kruglov Pavel
23aab71d7c
Merge branch 'master' into http-valid-json-on-exception 2023-08-01 16:47:31 +02:00
Kruglov Pavel
8f6526a930
Merge branch 'master' into structure-to-schema 2023-08-01 16:22:14 +02:00
Sergei Trifonov
01196ac41f
Merge branch 'master' into concurrency-control-controllable 2023-08-01 15:40:50 +02:00
alexX512
8e3296a44a Add support of local progress in scheduler 2023-08-01 13:31:18 +00:00
Ilya Yatsishin
e2a021aaf4
Merge pull request #52636 from ClickHouse/distinct-in-order-sqlancer-crashes 
Suspicious DISTINCT crashes from sqlancer
 2023-08-01 14:03:22 +02:00
avogar
fa905ebd27 Clean up 2023-08-01 10:14:09 +00:00
avogar
a71cd56a90 Output valid JSON/XML on excetpion during HTTP query execution 2023-08-01 10:06:56 +00:00
Anton Popov
8447a93931 fix sorting of sparse columns with large limit 2023-07-31 17:40:22 +00:00
ltrk2
e869adf645 Improve function naming 2023-07-31 06:48:50 -07:00
Igor Nikonov
bdf47428f2 Merge remote-tracking branch 'origin/master' into fix-window-func-range-check 2023-07-30 22:40:58 +00:00
Igor Nikonov
ff9f28cd5c
Merge branch 'master' into distinct-in-order-sqlancer-crashes 2023-07-30 21:25:22 +02:00
Alexey Milovidov
caa4590361 Merge branch 'master' into check-for-hiding-cyrillic-characters 2023-07-30 02:16:56 +02:00
Igor Nikonov
918571a5ee Fix: check correctly frame bounds for RANGE 2023-07-29 22:04:21 +00:00
ltrk2
6c9a1b14ef
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-07-28 16:18:46 -04:00
avogar
c3c64a7dd5 Fix 2023-07-28 11:40:05 +00:00
Kruglov Pavel
3e1c409e60
Merge branch 'master' into structure-to-schema 2023-07-28 11:32:16 +02:00
Igor Nikonov
407fb58d4b Fix style check 2023-07-27 20:53:36 +00:00
avogar
6d77d52dfe Allow variable number of columns in TSV/CuatomSeprarated/JSONCompactEachRow, make schema inference work with variable number of columns 2023-07-27 18:02:29 +00:00
Alexey Milovidov
a49c04efda
Merge branch 'master' into distinct-in-order-sqlancer-crashes 2023-07-27 12:12:06 +03:00
Antonio Andelic
f61f36800c Fix style 2023-07-27 08:48:23 +00:00
Alexey Milovidov
65ffe91bf2 Fix double whitespace 2023-07-27 07:13:26 +02:00
Alexey Milovidov
6aab4cc835 Check for unexpected cyrillic 2023-07-27 05:25:40 +02:00
Kruglov Pavel
fab77783f1
Merge pull request #49367 from ClickHouse/enc 
Partially reimplement Parquet encoder to make it faster and parallelizable
 2023-07-27 00:48:54 +02:00
Igor Nikonov
1ead0d7dac Merge remote-tracking branch 'origin/master' into distinct-in-order-sqlancer-crashes 2023-07-26 22:05:10 +00:00
Igor Nikonov
83fd3d4c89 Additional checks related to DistinctSortedChunkTransform 
In response to suspicious crash reports from sqlancer for DISTINCT queries
 2023-07-26 21:59:13 +00:00
Nikolai Kochetov
a4a8c73108
Merge pull request #48669 from ClickHouse/prewhere-optimization-update-sort-desc 
Update sort desc in ReadFromMergeTree after applying PREWHERE info
 2023-07-26 15:47:18 +02:00
Nikolai Kochetov
2283c6c6ac
Merge branch 'master' into fix-read-in-order-with-array-join 2023-07-26 15:37:24 +02:00
robot-ch-test-poll4
110500049a
Merge pull request #50532 from nickitat/more_pushdown_for_right_side_of_join 
Push down to right side of a join in more cases
 2023-07-26 14:43:57 +02:00
Kruglov Pavel
0d34e97dbe
Merge branch 'master' into formats-with-subcolumns 2023-07-26 13:30:35 +02:00
Kruglov Pavel
15cc046883
Merge branch 'master' into better-progress-bar-2 2023-07-26 13:12:24 +02:00
Raúl Marín
974f72800f Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-07-26 10:35:40 +02:00
Nikita Mikhaylov
1dc9ca2c24
Merge pull request #52549 from ClickHouse/whitespace-before-comma 
Check for punctuation
 2023-07-25 20:30:36 +02:00
Nikolai Kochetov
c960b3b533
Merge pull request #52533 from ClickHouse/Avogar-patch-2 
Cancel execution in PipelineExecutor in case of exception in graph->updateNode
 2023-07-25 12:39:22 +02:00
Michael Kolupaev
5ee71bd643 Work around the clang bug 2023-07-25 10:26:26 +00:00
Michael Kolupaev
dfdf5de972 Fixes 2023-07-25 10:16:28 +00:00
Michael Kolupaev
db5cb96050 Start over when falling back to non-dictionary encoding 2023-07-25 10:16:28 +00:00
Michael Kolupaev
8184a289e5 Partially reimplement Parquet encoder to make it faster and parallelizable 2023-07-25 10:16:28 +00:00
alexX512
1fe2f052e3 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-07-25 09:36:44 +00:00
Alexey Milovidov
21382afa2b Check for punctuation 2023-07-25 06:10:04 +02:00
Alexey Milovidov
ecdafeaf83
Merge pull request #52175 from bigo-sg/comment_improve_ch_to_arrow 
Add comments for https://github.com/ClickHouse/ClickHouse/pull/52112
 2023-07-25 06:47:48 +03:00
robot-ch-test-poll1
05d4932842
Merge pull request #52137 from ucasfl/avro-fix 
Avro input format support Union with single type
 2023-07-25 04:48:39 +02:00
Alexey Milovidov
5166944e71
Merge branch 'master' into more-checks-in-thread-status 2023-07-25 05:45:52 +03:00
Nikita Taranov
d0894532fe fix 2023-07-24 20:34:02 +02:00
Nikita Taranov
ab086f15d0 try to push down more 2023-07-24 20:34:01 +02:00
Kruglov Pavel
fec5675cd4
Merge branch 'master' into better-progress-bar-2 2023-07-24 19:59:38 +02:00
Kruglov Pavel
d2d100b68a
Cancel execution in PipelineExecutor in case of exception in graph->updateNode 2023-07-24 17:05:57 +02:00
flynn
5d318ef72a
Merge branch 'master' into avro-fix 2023-07-24 22:13:18 +08:00
Azat Khuzhin
b222476090 Add ability to pass table for connections checks per-shard to ReadFromRemote 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-24 05:50:44 +02:00
Alexey Milovidov
2ee149a100
Merge branch 'master' into fix_52405 2023-07-22 20:41:01 +03:00
Alexey Milovidov
758ec9fa92
Merge pull request #52297 from amosbird/fix_52055 
Don't use minmax_count projections when counting nullable columns
 2023-07-22 20:39:12 +03:00
Alexey Milovidov
04462333d2
Merge pull request #52327 from amosbird/fix_50183 
Add back missing projection QueryAccessInfo.
 2023-07-22 17:50:03 +03:00
Amos Bird
05bdcdf13c
Merge branch 'master' into fix_52055 2023-07-22 12:27:21 +08:00
李扬
e8fe485330
Merge branch 'master' into comment_improve_ch_to_arrow 2023-07-22 11:43:32 +08:00
Alexey Milovidov
e7967e5aab
Merge branch 'master' into fix_52405 2023-07-22 06:09:25 +03:00
Alexey Milovidov
63c5dac972
Merge branch 'master' into more-checks-in-thread-status 2023-07-22 04:58:28 +03:00
Alexey Milovidov
9ae685975e
Merge branch 'master' into avro-fix 2023-07-22 04:56:48 +03:00
Alexey Milovidov
d0cb6596e1
Merge pull request #52259 from Chen768959/fix-50582 
fix issue#50582 Not found column NULL in block
 2023-07-22 01:05:59 +03:00
Igor Nikonov
8c8e097b72
Merge pull request #51013 from bigo-sg/grace_hash_join_full_right_202306 
Grace hash join support full & right join
 2023-07-21 22:22:06 +02:00
alexX512
c403f56e09 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-07-21 17:56:53 +00:00
Nikolai Kochetov
01cb502af3
Merge pull request #52151 from amosbird/fix_52075 
Fix incorrect projection analysis when aggregation expression contains monotonic functions
 2023-07-21 18:30:27 +02:00
Amos Bird
b459da7d8a
Merge branch 'master' into fix_52055 2023-07-21 22:02:04 +08:00
Amos Bird
c080e9b450
Fix normal projection with merge table 2023-07-21 21:48:49 +08:00
robot-clickhouse-ci-2
0db9c79886
Merge pull request #52332 from rschu1ze/better-formatsettings 
Minor: Less awkward IAST::FormatSettings
 2023-07-21 14:49:58 +02:00
Alexander Tokmakov
b45c2c939b
disable expression templates for time intervals (#52335) 2023-07-21 15:17:07 +03:00
ltrk2
90a2c460c6
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-07-21 08:07:18 -04:00
Nikolai Kochetov
0778635996
Merge branch 'master' into fix-read-in-order-with-array-join 2023-07-21 13:37:45 +02:00
Amos Bird
2cad81731b
Try to fix style issues 2023-07-21 16:47:23 +08:00
Amos Bird
8187118232
Better code 2023-07-21 16:47:23 +08:00
Amos Bird
53d77e6b13
Add back missing projection QueryAccessInfo. 2023-07-21 16:47:22 +08:00
Amos Bird
68e7583dbf
reorganize tests and add some comments 2023-07-21 14:54:51 +08:00
Nikolai Kochetov
d7bb006c23
Fix monotonic chain for read-in-order as well. 2023-07-21 14:54:49 +08:00
Amos Bird
696818b340
Don't check monotonicity when analyze projections 2023-07-21 14:54:48 +08:00
Alexander Gololobov
06c1e686a4
Merge pull request #46365 from ClickHouse/split_prewhere_dag_test_enable 
Multi-stage PREWHERE enabled by default
 2023-07-21 08:13:55 +02:00
lgbo-ustc
24371c33bf remove DelayedBlocksTask::finish 2023-07-21 09:24:16 +08:00
Han Fei
4b0be1e535
Merge pull request #52308 from amosbird/fix_48823 
Fix projection analysis with primary key analysis
 2023-07-20 18:25:21 +02:00
Nikita Mikhaylov
668062dc29
Merge branch 'master' into no-keep-context-lock-while-calculating-access 2023-07-20 14:42:13 +02:00
chen768959
f2d184cf1b Consistent style for if statements. 2023-07-20 19:11:08 +08:00
chen768959
84f6a7336c Prevent going beyond the index of const_columns_to_remove. 2023-07-20 19:03:42 +08:00
Kruglov Pavel
342400d0b3
Merge branch 'master' into revert-52322-revert-51716-bug_fix_csv_field_type_not_match 2023-07-20 12:39:38 +02:00
Robert Schulze
bd761c365a
Make serializeAST() more regular 2023-07-20 10:39:26 +00:00
Robert Schulze
25ddcc256b
Make IAST::FormatSettings more regular, pt. III 2023-07-20 10:34:05 +00:00
lgbo-ustc
91dc6a35e1 update 2023-07-20 18:22:01 +08:00
lgbo-ustc
0ba97eeea5 wip: grace hash join support full & right join 2023-07-20 18:22:01 +08:00
chen768959
067e3caa2c Remove constants from description_sorted_. 2023-07-20 18:13:19 +08:00
Nikolay Degterinsky
209429d0e3
Merge pull request #49664 from ilejn/test_for_basic_auth_registry 
Basic auth to fetch Avro schema in Kafka
 2023-07-20 10:58:11 +02:00
李扬
68bf4c3590
Merge branch 'master' into comment_improve_ch_to_arrow 2023-07-20 10:10:47 +08:00
ltrk2
a753c3c6ad
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-07-19 16:22:58 -04:00
Kruglov Pavel
0fca64ced4
Merge pull request #51695 from Avogar/row-binary-with-defaults 
Add RowBinaryWithDefaults format
 2023-07-19 22:10:30 +02:00
ltrk2
ba4072f049 Adapt changes around SipHash 2023-07-19 10:01:58 -07:00
ltrk2
51e2c58a53 Implement endianness-independent SipHash and MergeTree checksum serialization 2023-07-19 10:01:55 -07:00
Han Fei
6bbdf702ae
Merge branch 'master' into hanfei/support-insert-token 2023-07-19 17:05:12 +02:00
Kruglov Pavel
f0026af189
Revert "Revert "Improve CSVInputFormat to check and set default value to column if deserialize failed"" 2023-07-19 14:51:11 +02:00
Kruglov Pavel
7b3564f96a
Revert "Improve CSVInputFormat to check and set default value to column if deserialize failed" 2023-07-19 14:44:59 +02:00
robot-ch-test-poll4
63d0616a22
Merge pull request #51716 from KevinyhZou/bug_fix_csv_field_type_not_match 
Improve CSVInputFormat to check and set default value to column if deserialize failed
 2023-07-19 14:41:05 +02:00
Igor Nikonov
a48960b132
Merge branch 'master' into prewhere-optimization-update-sort-desc 2023-07-19 14:37:22 +02:00
kevinyhzou
dcf7ba2534 remove unuseful code 2023-07-19 19:36:19 +08:00
Amos Bird
2ebbbf0000
Also need to fix aggregate projections 2023-07-19 19:13:45 +08:00
kevinyhzou
95424177d5 review fix 2023-07-19 18:26:54 +08:00
Ilya Golshtein
c1c5ffa309 test_for_basic_auth_registry - cpp code small improvement 2023-07-19 08:32:45 +00:00
dheerajathrey
8e1de7897a indentation fix 2023-07-19 08:32:44 +00:00
dheerajathrey
1564eace38 enable url-encoded basic auth to fetch avro schema in kafka 2023-07-19 08:32:44 +00:00
Chen768959
549026f0ae
fix style error 
fix Trailing whitespaces
 2023-07-19 16:11:14 +08:00
Amos Bird
629e0e0269
Fix projection analysis with primary key analysis 2023-07-19 16:06:21 +08:00
Han Fei
e8be6ec23f
Merge branch 'master' into hanfei/support-insert-token 2023-07-19 10:06:03 +02:00
Amos Bird
c3b8978023
Don't use minmax_count projections when counting nullable columns 2023-07-19 11:56:28 +08:00
Alexey Milovidov
0789f388c3
Update ArrowFieldIndexUtil.h 2023-07-19 02:45:56 +03:00
Alexey Milovidov
6d915042a2 Fix ugly code 2023-07-19 01:44:20 +02:00
Chen768959
9b7ecbaa27
fix issue#50582 
The buildInputOrderInfo() method in optimizeReadInOrder.cpp adds constant columns to order_key_prefix_descr. However, since header_without_constants does not contain constant columns, the constant columns in description_sorted_ also need to be removed in advance.
 2023-07-18 22:00:47 +08:00
Nikolai Kochetov
dff58f54b9
Merge branch 'master' into fix-read-in-order-with-array-join 2023-07-18 15:52:40 +02:00
avogar
67f340b501 Merge branch 'master' of github.com:ClickHouse/ClickHouse into structure-to-schema 2023-07-18 13:52:15 +00:00
Kruglov Pavel
64e88cde21
Merge branch 'master' into better-progress-bar-2 2023-07-18 13:37:53 +02:00
Kruglov Pavel
6985bf0cdb
Merge branch 'master' into row-binary-with-defaults 2023-07-18 13:36:56 +02:00
Nikolai Kochetov
d69df6975f
Merge branch 'master' into add_optimize_use_implicit_projections 2023-07-18 13:17:12 +02:00
Amos Bird
d8dd480cd7
Disable implicit projections by default 2023-07-18 09:50:41 +08:00
Kruglov Pavel
1e616e17ab
Merge branch 'master' into row-binary-with-defaults 2023-07-17 19:13:57 +02:00
Kruglov Pavel
1dd05319b5
Merge branch 'master' into formats-with-subcolumns 2023-07-17 19:13:42 +02:00
Igor Nikonov
778efcb957 Use row_level_filter to restore original header as well 2023-07-17 13:50:36 +00:00
Vitaly Baranov
815a3857de Remove non-const function Context::getClientInfo(). 2023-07-17 15:02:07 +02:00
Han Fei
68751b88f3 change comments 2023-07-17 15:01:48 +02:00
Han Fei
b75c4ae2e5 Merge branch 'master' into hanfei/support-insert-token 2023-07-17 14:57:48 +02:00
kevinyhzou
355faa4251 ci fix 2023-07-17 20:08:32 +08:00
flynn
d6709ded53
Merge branch 'master' into avro-fix 2023-07-17 14:51:34 +08:00
taiyang-li
8ea335aca7 update style 2023-07-17 10:43:13 +08:00
taiyang-li
7716479a37 add comments for https://github.com/ClickHouse/ClickHouse/pull/52112 2023-07-17 10:33:38 +08:00
Alexey Milovidov
3aae818a77
Merge branch 'master' into fix-the-same-names-for-lift-up-functions-before-sorting 2023-07-17 03:21:33 +03:00
Alexey Milovidov
4e727cdb2a
Update liftUpFunctions.cpp 2023-07-17 03:21:24 +03:00
Igor Nikonov
7adc2f5715
Merge branch 'master' into prewhere-optimization-update-sort-desc 2023-07-16 18:17:29 +02:00
Alexander Gololobov
3e4182fc4f Include source columns for sampling into required columns list 2023-07-16 08:24:12 +02:00
Alexey Milovidov
9a421369d7
Merge pull request #49709 from amosbird/fix_49695 
Fix projection with optimize_aggregators_of_group_by_keys
 2023-07-16 04:10:54 +03:00
flynn
386adfad33 Avro input format support Union with single type 2023-07-15 16:21:58 +00:00
Han Fei
7761e5ef9d support async_deduplicate_token for async insert 2023-07-15 14:42:08 +02:00
Igor Nikonov
9738d248c3 Revert unnecessary changes 2023-07-14 12:17:48 +00:00
Igor Nikonov
4750bfd120 Merge remote-tracking branch 'origin/master' into prewhere-optimization-update-sort-desc 2023-07-14 11:58:17 +00:00
Igor Nikonov
c6c6389f3f Add queries with explicitly enabled analyzer 2023-07-14 11:57:18 +00:00
taiyang-li
8ea3bf4ade improve ch to arrow 2023-07-14 16:09:22 +08:00
kevinyhzou
c6b8097090 rebase main 2023-07-14 11:24:38 +08:00
kevinyhzou
b2665031dc review fix 2023-07-13 20:27:14 +08:00
kevinyhzou
ba57c84db3 bug fix csv input field type mismatch 2023-07-13 20:24:10 +08:00
Amos Bird
52246ab5b5
Ignore setting constraints for projection ASTs 2023-07-12 10:34:36 +08:00
Raúl Marín
f5e0c1bd25 Interpreter: Control number of parallel replicas based on row estimation 2023-07-11 13:57:51 +02:00
xiao
bc16531876
Update CompletedPipelineExecutor.cpp 2023-07-11 17:01:56 +08:00
Dmitry Kardymon
385a210fee Merge remote-tracking branch 'origin/master' into ADQM-870 2023-07-10 13:19:21 +00:00
Alexey Milovidov
3d4800995f
Merge pull request #49732 from nickitat/impr_prefetch 
Improve reading with prefetch
 2023-07-09 06:10:58 +03:00
Alexey Milovidov
3a48a7b872
Merge pull request #51492 from nickitat/fix_final_header_mismatch 
Fix Block structure mismatch in Pipe::unitePipes for FINAL
 2023-07-08 08:05:20 +03:00
Igor Nikonov
39a440fa0e Build sort description based on original header 2023-07-07 22:08:31 +00:00
Igor Nikonov
90f6c2322c Merge remote-tracking branch 'origin/master' into prewhere-optimization-update-sort-desc 2023-07-07 12:20:01 +00:00
Igor Nikonov
3dd9c09579 Just save 2023-07-07 11:25:48 +00:00