Commit Graph

218 Commits

Author SHA1 Message Date
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] (#44955)
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)""

This reverts commit b8d9066004.

* Fix build

* Better

* Fix test

* Automatic style fix

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-01-09 13:30:32 +01:00
SmitaRKulkarni
16fca48f5b
Disable functions in readonly for inserts (#44290)
Implementation:
* Added a new parameter to ITableFunction::execute to check if query is insert query.
* Used the new parameter in InterpreterInsertQuery::getTable function when calling ITableFunction::execute .
Testing:
* Added test case for INSERT in 02473_functions_in_readonly_mode.sh.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-01-03 15:25:04 +01:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Azat Khuzhin
5166a1d58e Fix written_bytes/written_rows in INSERT SELECT intermediate progress packets
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-05 19:12:18 +02:00
Nikolai Kochetov
00965ce17a Fixing race. 2022-09-22 17:57:04 +00:00
Nikolai Kochetov
446453bdf5 Fix a destruction order for views ThreadStatus 2022-09-21 18:37:40 +00:00
Duc Canh Le
8590cc46c4 implement intersect + except distinct 2022-08-30 18:09:01 +08:00
Alexander Tokmakov
f9f85a0e8b Revert "Parallel distributed insert select from *Cluster table functions (#39107)"
This reverts commit d3cc234986.
2022-08-24 15:17:15 +03:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions (#39107) 2022-08-15 12:41:17 +02:00
Azat Khuzhin
dd3515da98 Fix parallel_view_processing with optimize_trivial_insert_select=1 2022-07-03 15:14:05 +03:00
Azat Khuzhin
4ae7db8369 Fix max_insert_threads while pushing to views
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-03 15:14:05 +03:00
Dmitry Novik
376412e417 Small refactoring 2022-06-16 15:41:04 +00:00
Nikolai Kochetov
85f4295370 Cleanup 2022-06-02 10:34:40 +00:00
Nikolai Kochetov
147a819221 Refactor a little bit more. 2022-05-31 14:43:38 +00:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Nikolai Kochetov
9756b759c6 Move some resources 2022-05-23 13:46:57 +00:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
Robert Schulze
b475fbc9a7
Merge pull request #37300 from ClickHouse/cmake-cleanup-pt3
Various cmake cleanups
2022-05-20 10:02:36 +02:00
Robert Schulze
43945cea1b
Fixing some warnings 2022-05-16 20:59:27 +02:00
Vxider
f2ef27e5d6 rename windowview header 2022-05-14 09:21:54 +00:00
mergify[bot]
7ffad69019
Merge branch 'master' into wv-populate 2022-05-12 11:52:04 +00:00
Vxider
e4f43d0b01 update getSampleBlock for windowview 2022-05-11 08:47:08 +00:00
Anton Popov
aec30c4076
Merge pull request #37053 from CurtizJ/remove-streams-comments
Remove last mentions of data streams
2022-05-10 13:38:13 +02:00
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
Nikolai Kochetov
a34411f51e Disable trivial_insert_select if query has aggregate functions.
(cherry picked from commit 70ae043455)
2022-05-09 15:56:39 +02:00
Anton Popov
1fc51e09ff fix insertion to column of type Object from multiple files via table function 2022-04-28 18:51:13 +00:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
2022-04-20 10:29:05 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
Anton Popov
caacc7d385 add quota for written bytes 2022-03-29 18:21:29 +00:00
Anton Popov
67195bfdd5 support schema inference for type Object in format JSONEachRow 2022-03-25 21:51:53 +00:00
Alexander Tokmakov
8acfb8d27f Merge branch 'master' into mvcc_prototype 2022-03-07 17:40:15 +01:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Anton Popov
82d24f06eb
Merge pull request #34866 from CurtizJ/async-insert-table-function
Fix async inserts to table functions
2022-03-01 19:45:22 +03:00
Anton Popov
8fa112b841 add comments 2022-02-28 17:09:46 +03:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Alexander Tokmakov
dae044f86b Merge branch 'master' into mvcc_prototype 2022-02-17 13:49:37 +03:00
Nikolai Kochetov
1276c50271 Fix build. 2022-02-15 20:15:07 +00:00
Nikolai Kochetov
b2b29b7ee8 Do not use ReduceToPhysicalTransform. 2022-02-15 12:23:22 +00:00
Yakov Olkhovskiy
579fe6c97a major rework, transform added to the insert pipe 2022-02-13 17:42:59 +00:00
Yakov Olkhovskiy
b9777be271 insertion with ephemerals requires columns list, insertion without columns list omits ephemeral 2022-02-09 01:11:29 +00:00
Alexander Tokmakov
5fad3fdffc throw exception on non-transactional queries 2022-02-01 01:27:55 +03:00
Azat Khuzhin
5ed7440381 Fix possible use-after-free for INSERT into MV with concurrent DROP
ASan founds [1]:

    ==553==
    ERROR: AddressSanitizer: heap-use-after-free on address 0x61e004694080 at pc 0x000029150af2 bp 0x7f70b3f8ef10 sp 0x7f70b3f8ef08
    READ of size 8 at 0x61e004694080 thread T477 (QueryPipelineEx)
        0 0x29150af1 in DB::MergeTreeDataWriter::writeTempPart() >
        1 0x293b8e43 in DB::MergeTreeSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeSink.cpp:27:65
        2 0x29dac73b in DB::SinkToStorage::onConsume(DB::Chunk) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5
        3 0x29c72dd2 in DB::ExceptionKeepingTransform::work()::$_1::operator()() const obj-x86_64-linux-gnu/../src/Processors/Transforms/ExceptionKeepingTransform.cpp:151:51

    0x61e004694080 is located 2048 bytes inside of 2480-byte region [0x61e004693880,0x61e004694230)
    freed by thread T199 (BgSchPool) here:
        ...
        4 0x26220f20 in DB::DatabaseCatalog::TableMarkedAsDropped::~TableMarkedAsDropped() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.h:248:12
        5 0x26220f20 in DB::DatabaseCatalog::dropTableDataTask() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.cpp:908:1

      [1]: https://s3.amazonaws.com/clickhouse-test-reports/33201/4f04d6af61eabf4899eb8188150dc862aaab80fc/stress_test__address__actions_.html

There was a fix in #32572, but it was not complete (yes it reduced the
race window a lot, but not completely), since the inner table still can
go away after the INSERT chain was built, to fix this obtain the
reference earlier.

Follow-up for: #32572 (cc @tavplubix)
2022-01-04 13:34:01 +03:00
Anton Popov
a6e90a046d add more info to query log in async inserts 2021-12-28 17:53:25 +03:00
Maksim Kita
51477adf1b Updated additional cases 2021-12-20 15:55:07 +03:00
alesapin
6bd7e425c6
Merge pull request #22535 from CurtizJ/sparse-serialization
Sparse serialization and ColumnSparse
2021-12-17 15:26:17 +03:00
Anton Popov
16312e7e4a Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-14 18:58:17 +03:00