thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
79,881 Commits 9,338 Branches 1,783 Tags 2.1 GiB
df0af75864
Commit Graph

21096 Commits

Author SHA1 Message Date
Anton Popov
bbb981655d
Merge pull request #32695 from CurtizJ/fix-add-ttl 
Fix `MATERIALIZE TTL` with `TTL WHERE` and `TTL GROUP BY`
 2021-12-14 14:01:15 +03:00
nautaa
0da3881ff4 flush all InMemoryDataParts when wal is not enabled 2021-12-14 16:31:17 +08:00
Azat Khuzhin
33ffd343f3 Add ability to drain connections synchronously 
This is mostly for testing, to avoid hiding errors.
 2021-12-14 10:18:57 +03:00
Azat Khuzhin
510eab25a1 Add timeout into "Timeout exceeded while reading from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
eb9968babf Add timeout into "Read timeout while draining from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
5c198525d2 Disconnect connection on unknown packet. 2021-12-14 10:18:57 +03:00
Azat Khuzhin
a7dc6f309f Remove SparseHashMap.h 
It was added only for arcadia build, and used only in one place, no need
to have a separate typedef for it.
 2021-12-14 10:07:14 +03:00
Azat Khuzhin
e16891713d Fix sparse_hashed dict performance with sequential keys (wrong hash function) 
In #27152 the hash function for sparse_hash_map had been changed to
std::hash<> switch it back to DefaultHash<> (ClickHouse builtin), since
std::hash<> for numeric keys returns itself and this does not works
great with sparse_hash_map.

I've tried the example from #32480 and using some hash fixes the
performance of sparse_hashed layout.

Fixes: #32480

v2: Add comments for SparseHashMap
 2021-12-14 10:07:14 +03:00
mergify[bot]
a5599d9fe2
Merge branch 'master' into watch-events-supports 2021-12-14 06:15:58 +00:00
gulige
2f8998880d
to clear up compiler warning about "C-style cast". 
use std::npos to replace (size_t)-1
 2021-12-14 10:41:51 +08:00
mergify[bot]
96f6915233
Merge branch 'master' into wip_mv_into_join 2021-12-14 01:31:47 +00:00
mergify[bot]
e64718ae01
Merge branch 'master' into strange-code-tcphandler 2021-12-14 01:28:47 +00:00
Kseniia Sumarokova
c9ec664094
Update MySQLHelpers.h 2021-12-14 01:25:53 +03:00
mergify[bot]
2b181d7107
Merge branch 'master' into mongo-extend-auth-2 2021-12-13 22:21:10 +00:00
kssenii
f0d0714e47 Pass timeouts for mysql 2021-12-13 22:12:33 +00:00
Kseniia Sumarokova
dadaeabda7
Merge pull request #32670 from CurtizJ/fix-reading-in-order 
Fix reading in order of sorting key from `Distributed` and `Merge` tables
 2021-12-14 00:42:48 +03:00
Alexey Milovidov
6879f03cb6 Remove garbage 2021-12-13 21:12:47 +03:00
mergify[bot]
827dbbe406
Merge branch 'master' into strange-code-tcphandler 2021-12-13 18:09:00 +00:00
alexey-milovidov
730c16bd0c
Merge pull request #32649 from vdimir/issue_32504 
Reset pipeline in BlockIO::onFinish
 2021-12-13 21:05:33 +03:00
Kseniia Sumarokova
a9ca84ff7d
Merge pull request #32325 from stigsb/table-overrides 
Add TABLE OVERRIDE feature for MaterializedMySQL databases
 2021-12-13 20:39:42 +03:00
Kseniia Sumarokova
9eb2a4fe90
Merge pull request #32643 from nautaa/genarateRandom 
support Date32 for genarateRandom engine
 2021-12-13 20:38:41 +03:00
alexey-milovidov
fd9d40925a
Merge pull request #32640 from ClickHouse/fix-function-file 
Fix surprisingly bad code in function `file`
 2021-12-13 19:34:36 +03:00
kssenii
ddbeadd104 Done 2021-12-13 16:09:18 +00:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency 
Azure Blob Storage Disks
 2021-12-13 18:28:48 +03:00
Anton Popov
0b68980977 fix MATERIALIZE TTL with WHERE and GROUP BY 2021-12-13 17:21:05 +03:00
tavplubix
60fae3af76
Follow-up to #32140 (#32667) 
* Update LeaderElection.h

* Update LeaderElection.h
 2021-12-13 16:41:46 +03:00
Kruglov Pavel
52328f6abc
Merge pull request #32490 from ClickHouse/quote-more-identifiers 
Fix issue with ambiguous query formatting in distributed queries.
 2021-12-13 16:41:01 +03:00
Raúl Marín
589b5a1fdc Merge remote-tracking branch 'blessed/master' into wip_mv_into_join 2021-12-13 14:05:46 +01:00
Anton Popov
745fd4fc16 fix reading in order from Distributed and Merge tables 2021-12-13 15:35:25 +03:00
Anton Popov
a241103714
Merge pull request #32359 from CurtizJ/fix-non-materialized-skip-index 
Fix usage of non-materialized skip indexes
 2021-12-13 14:43:58 +03:00
mergify[bot]
fa8f9671f4
Merge branch 'master' into fix-async-inserts 2021-12-13 11:13:20 +00:00
Raúl Marín
b25cfa0b4d InterpreterInsertQuery::buildChain: Make sure to keep the context alive 2021-12-13 11:50:46 +01:00
Raúl Marín
c9e9bd78b2 Remove old unused file 2021-12-13 11:14:10 +01:00
vdimir
bda4ecbe90
Catch exceprion in ~ZlibDeflatingWriteBuffer 2021-12-13 12:29:35 +03:00
vdimir
7236313435
Reset pipeline in BlockIO::onFinish 2021-12-13 12:29:13 +03:00
mergify[bot]
77be64f1f4
Merge branch 'master' into genarateRandom 2021-12-13 09:18:52 +00:00
mergify[bot]
34417017e4
Merge branch 'master' into master 2021-12-13 09:18:49 +00:00
mergify[bot]
f5e2c38452
Merge branch 'master' into fix_groupBitmapAnd_function_distributed_table 2021-12-13 09:18:45 +00:00
Nikolai Kochetov
22e6fc1685
Merge pull request #32067 from amosbird/projection-fix23 
Fix detaching parts with projections
 2021-12-13 12:00:17 +03:00
nautaa
7cb3d24847 support Date32 for genarateRandom engine 2021-12-13 15:52:12 +08:00
mergify[bot]
47b18c959f
Merge branch 'master' into fix-non-materialized-skip-index 2021-12-13 07:03:02 +00:00
gulige
2f7edf82ab
avoid inconsistent state in Block 
In the member function insert(position, elem) to class Block, if function checkColumnStructure<void> throws an exception for columns with same name but unidentical structure, it will result in the mismatch between index_by_name and data. However we could not presume upon that caller will not use this block object anymore. So it seems better to guarantee consistency innerly.
 2021-12-13 13:53:12 +08:00
Alexey Milovidov
71926a3a97 Fix surprisingly bad code in function "file" 2021-12-13 07:57:54 +03:00
Vxider
f6dceb53a5 small update 2021-12-13 02:38:16 +00:00
Vxider
caad9f6c79 small update 2021-12-12 17:30:44 +00:00
mergify[bot]
dd4eaad2f3
Merge branch 'master' into watch-events-supports 2021-12-12 15:22:53 +00:00
tavplubix
be35ab171b
Merge pull request #32572 from azat/DROP-INSERT-UAF-fix 
Fix table lifetime in case of parallel DROP TABLE and INSERT
 2021-12-12 17:05:07 +03:00
alexey-milovidov
0486c5f341
Merge pull request #32220 from azat/mt-fix-minmax-for-empty-part 
[RFC] Fix minmax partition index during merge in case of empty part in future parts
 2021-12-12 14:45:10 +03:00
cmsxbc
e4b405716f fix multiply row policies on same column 2021-12-12 09:54:07 +00:00
mergify[bot]
72606749ab
Merge branch 'master' into watch-events-supports 2021-12-12 09:51:52 +00:00
Kseniia Sumarokova
5dc7d7e3ef
Merge pull request #32584 from kssenii/rabbitmq-fix 
rabbitmq fix
 2021-12-12 12:34:09 +03:00
alexey-milovidov
6779196c6b
Merge pull request #32587 from ClickHouse/reinterpret-perf 
Slight performance improvement of "reinterpret" function
 2021-12-12 12:14:43 +03:00
alexey-milovidov
09a6e826e7
Merge pull request #32591 from ClickHouse/fix-indecent-error-message 
Fix indecent error message
 2021-12-12 12:12:38 +03:00
Vxider
7eb18ea21e watch events support for window view 2021-12-12 07:45:55 +00:00
alexey-milovidov
26d606c158
Merge pull request #32456 from ClickHouse/fix-29890 
Fix arraySlice with null args.
 2021-12-12 06:38:17 +03:00
alexey-milovidov
71df622b1f
Merge pull request #31888 from ClickHouse/fix-31114 
Remove LowCardinality from window function args.
 2021-12-12 06:37:55 +03:00
Alexey Milovidov
2c68192bb6 Fix strange code in TCPHandler 2021-12-12 05:35:33 +03:00
Alexey Milovidov
453135eb63 Fix indecent error message 2021-12-12 05:24:47 +03:00
alexey-milovidov
151834d743
Merge pull request #32417 from azat/local-atomic-db-fix 
clickhouse-local: fix CREATE DATABASE with Atomic engine
 2021-12-12 04:33:59 +03:00
alexey-milovidov
33674cbe58
Merge pull request #32535 from azat/avg-ubsan 
Suppress UBSan errors for avg() function
 2021-12-12 04:32:44 +03:00
alexey-milovidov
b8a15b666d
Merge pull request #32571 from azat/mv-non-phys-table 
Fix LOGICAL_ERROR for MATERIALIZED VIEW over table functions (i.e. numbers())
 2021-12-12 03:55:51 +03:00
Alexey Milovidov
58ffe89643 Merge branch 'master' into quote-more-identifiers 2021-12-12 03:33:20 +03:00
Alexey Milovidov
3b0f026827 Slight performance improvement of "reinterpret" function 2021-12-12 03:29:26 +03:00
alexey-milovidov
eaf1d764a7
Merge pull request #32578 from bharatnc/ncb/fixes 
remove unused headers in test
 2021-12-12 01:05:12 +03:00
alexey-milovidov
a1cf07fdd3
Merge pull request #32580 from azat/remove-arcadia 
Remove arcadia build support
 2021-12-12 01:03:42 +03:00
kssenii
89db2c57f4 Fix 2021-12-12 00:19:06 +03:00
Vitaly Baranov
8f9b0b4ed0
Merge pull request #32575 from vitlibar/improve-quota-end-of-interval-calculation 
Improve quota's end-of-interval calculations.
 2021-12-11 22:16:04 +03:00
Azat Khuzhin
57c027be0e Remove arcadia build support 2021-12-11 21:25:23 +03:00
Vitaly Baranov
463ce1fcee
Merge pull request #27822 from filimonov/kafka_protobuf_issue26643 
Test for issue #26643
 2021-12-11 20:31:22 +03:00
alexey-milovidov
0bed774d83
Merge pull request #32574 from azat/remove-readline 
[RFC] Remove readline support
 2021-12-11 20:15:03 +03:00
bharatnc
a3dfb89927 remove unused headers in test 2021-12-11 08:30:22 -08:00
zhanglistar
2af221df18 1. update node size in set processor. 2021-12-11 22:24:59 +08:00
Vitaly Baranov
eea269a829 Improve quota's end-of-interval calculations. 2021-12-11 13:56:38 +03:00
Azat Khuzhin
4e4837758a Remove readline support 
- it was not nested for a long time
- replxx is an upstream way for completion
 2021-12-11 11:29:59 +03:00
Vitaly Baranov
abe9dd3368
Merge pull request #32531 from vitlibar/fix-nested-array-sizes-for-missing-columns 
Improve handling nested structures with missing columns while reading protobuf
 2021-12-11 11:08:34 +03:00
mergify[bot]
1e28f7340e
Merge branch 'master' into kafka_protobuf_issue26643 2021-12-11 08:05:20 +00:00
Azat Khuzhin
ca9c5dc4eb Fix LOGICAL_ERROR for MATERIALIZED VIEW over table functions (i.e. numbers()) 
Replace LOGICAL_ERROR with QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW
in this case.
 2021-12-11 11:04:47 +03:00
Azat Khuzhin
9a71598973 Fix table lifetime in case of parallel DROP TABLE and INSERT 
Stress tests founds [1]:

==527==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x37078ffd in unsigned long std::__1::__cxx_atomic_fetch_add<unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order) obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1050:12
    1 0x37078ffd in std::__1::__atomic_base<unsigned long, true>::fetch_add(unsigned long, std::__1::memory_order) obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1719:17
    2 0x37078ffd in std::__1::__atomic_base<unsigned long, true>::operator++() obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1756:57
    3 0x37078ffd in SimpleIncrement::get() obj-x86_64-linux-gnu/../src/Common/SimpleIncrement.h:20:16
    4 0x37078ffd in DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeDataWriter.cpp:276:46
    5 0x373c446c in DB::MergeTreeSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeSink.cpp:27:65

  Uninitialized value was created by a heap deallocation
    6 0x32d481e8 in DB::DatabaseCatalog::TableMarkedAsDropped::~TableMarkedAsDropped() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.h:248:12
    7 0x32d3c134 in DB::DatabaseCatalog::dropTableDataTask() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.cpp:908:1

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32534/fa6090f588dbf4cbb5f28bd2210847b070bb8218/stress_test__memory__actions_.html

The query was CREATE MATERIALIZED VIEW ... POPULATE AS SELECT ... from
00040_aggregating_materialized_view test.
 2021-12-11 11:00:31 +03:00
Vitaly Baranov
94e2e3625b
Merge pull request #31656 from sunny19930321/fix-use-quota-bug 
Fix use quota bug
 2021-12-11 11:00:23 +03:00
Kseniia Sumarokova
9ede6beca7
Merge pull request #32534 from azat/local-file-table-fix 
Fix processing initial table (--table/stdin) in clickhouse-local
 2021-12-11 10:45:19 +03:00
Kseniia Sumarokova
7e0bc290da
Merge pull request #32460 from kssenii/mongo-extend-auth 
Allow empty credentials for mongo engine
 2021-12-11 10:42:15 +03:00
alexey-milovidov
c90e588e24
Merge pull request #32506 from ClickHouse/fix-has-column-in-table-subquery 
Fix queries with hasColumnInTable constant condition and non existing column
 2021-12-11 02:50:06 +03:00
alexey-milovidov
17e5f5ccfe
Merge pull request #32508 from vdimir/issue_32458 
Handle const column in JoinCommon::removeColumnNullability
 2021-12-11 01:21:10 +03:00
Anton Popov
37837f3881
Merge pull request #32453 from kssenii/disable-optimizations-for-window-functions 
Disable some optimizations for window functions
 2021-12-11 00:37:24 +03:00
Jakub Kuklis
578aec97e9 Removing TODO and NOTE marks used mainly as hooks for review in the PR 2021-12-10 22:04:38 +01:00
Jakub Kuklis
a3bc6d2fb7 Style corrections 2021-12-10 22:04:38 +01:00
Jakub Kuklis
85807adec4 Fixing variable shadow error 2021-12-10 22:04:38 +01:00
Jakub Kuklis
74e26354cc Committing missing files 2021-12-10 22:04:38 +01:00
Jakub Kuklis
41a88e49c9 Abstracting metadata creation for remote disks registrations, putting Blob Storage auth in a separate file 2021-12-10 22:04:38 +01:00
Jakub Kuklis
e58f74d8dc Connecting to Azurite and invoking simple INSERTs works in the integration test 2021-12-10 22:04:38 +01:00
Jakub Kuklis
8afe69ff93 Comming back to StorageSharedKeyCredential for the integration test, get 400 Bad requests from Azurite when trying to create a container now 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f180a7ba5f Solved Invalid storage account error with connection strings 2021-12-10 22:04:38 +01:00
Jakub Kuklis
008141de36 Enable authentication via StorageSharedKeyCredential 2021-12-10 22:04:38 +01:00
Jakub Kuklis
d51f979c7a Adding endpoint validation 2021-12-10 22:04:38 +01:00
Jakub Kuklis
2b7725bfc7 Enable connecting directly to storage accounts and creating new containers 2021-12-10 22:04:38 +01:00
Jakub Kuklis
910db6ca57 Code cleanup addressing PR remarks 2021-12-10 22:04:38 +01:00
Jakub Kuklis
6acbec4455 Rebased on master 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f6c5a2c63c Removing file checks after changing blob filenames 2021-12-10 22:04:38 +01:00
Jakub Kuklis
3aa6a0ead3 Adding random postfix to blob names, needs code cleanup 2021-12-10 22:04:38 +01:00
Jakub Kuklis
5ed511f9da Working skeleton for Blob Storage integration test 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0c65cfed1f Addressing first PR remarks 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7c2e116214 Adding max_retries config 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a33cea0525 Refactors after proper rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3728dbbaf Rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b8ef47006c Minor refactors before PR 2021-12-10 22:04:37 +01:00
Ryad ZENINE
c4421d6866 Small changes following review 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7f595eedbc Catch Blob Storage exceptions 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3c4a92cc0 Code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c4691affb2 Adding a new test for read with offset 2021-12-10 22:04:37 +01:00
Jakub Kuklis
9a00acfb72 Fixing read size error plus minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fd8ebe0d9f Maximum read 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c9f3164cba Bringing back old single part upload limit 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0d7a4b10c3 Using unique_ptr for Blob Client in Read Buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
12924b3659 Minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fa2261dee6 Changing offset in ReadBuffer to size_t 2021-12-10 22:04:37 +01:00
Jakub Kuklis
5e2b2b1eee Better search for blobs with a given name 2021-12-10 22:04:37 +01:00
Jakub Kuklis
33247db3f4 Adding finalize() to WriteBuffer, always using block blob uploads, code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
bf2476a076 Shared pointer for Azure client and fixing path reference in ReadBuffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
60dd22dd4e Checking for success of file removal operation 2021-12-10 22:04:37 +01:00
Jakub Kuklis
486f267b3b Further code cleanup and error codes improvement 2021-12-10 22:04:37 +01:00
Jakub Kuklis
6755c5e897 File deletion prototype 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a0cfafa387 checkUniqueId implementation using ListBlobs 2021-12-10 22:04:37 +01:00
Jakub Kuklis
956348073b ReadBufferFromBlobStorage operating like a proper stream, rather than a single download-read buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
95816875f4 Change false to true in supportZeroCopyReplication 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b41ab197e8 Blob Storage endpoint url verification using re2 2021-12-10 22:04:37 +01:00
Jakub Kuklis
63ec8d4244 Adding multipart upload to WriteBuffer, retries in ReadBuffer, additional settings for buffers 2021-12-10 22:04:37 +01:00
Jakub Kuklis
02060791e3 Adding settings getter and applyNewSettings 2021-12-10 22:04:37 +01:00
Jakub Kuklis
3f28f2c62b Enabling local cache for DiskBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
e3daf8aa49 Increase default thread pool size 2021-12-10 22:04:37 +01:00
Jakub Kuklis
dbb98e3636 Code cleanup, leaving some debug information printing under #ifdef VERBOSE_DEBUG_MODE 2021-12-10 22:04:37 +01:00
Jakub Kuklis
8230f8039c Two column MergeTree table with Blob Storage disk work! Fixed buffer in ReadBufferFromBlobStorage to be permanent (class attribute) 2021-12-10 22:04:37 +01:00
Jakub Kuklis
2b4d3665ab Basic inserts and selects work for a MergeTree table with a Blob Disk, based on a single column table 2021-12-10 22:04:37 +01:00
Jakub Kuklis
4b51ddee1b Initial check with write and read works for the Blob Storage Disk 2021-12-10 22:04:37 +01:00
Jakub Kuklis
03f85dee41 Managed to read markers file in SELECT, getting error for CODEC version now 2021-12-10 22:04:37 +01:00
Jakub Kuklis
482a10e62e Uploading to Blob Storage on INSERT seems to do something reasonable 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c43b90c58c Progressing with WriteBufferFromBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fb4c1d6686 Starting insertion pipeline 2021-12-10 22:04:37 +01:00
Jakub Kuklis
6436f5b04d Adding an authenticated client to DiskBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
ee1a568919 The following query passes without erors: CREATE TABLE ... ENGINE = MergeTree ... storage_policy = 'blob_storage', debug constants present in the code 2021-12-10 22:04:37 +01:00
Jakub Kuklis
cacdac7ba6 Moving implementation to the .cpp file 2021-12-10 22:04:37 +01:00
Jakub Kuklis
88b5feb59d Registering the Blob Storage Disk 2021-12-10 22:04:36 +01:00
Jakub Kuklis
20dda7f666 Giving Disk Blob Storage files proper names 2021-12-10 22:04:36 +01:00
Jakub Kuklis
0cf338e165 Skeleton for DiskBlobStorage 2021-12-10 22:04:36 +01:00
Jakub Kuklis
43f420cba8 Adding a flag to use Azure Blob Storage 2021-12-10 22:04:36 +01:00
Jakub Kuklis
162e2b334b Initial working example of Azure Blob Storage manipulation from within ClickHouse 2021-12-10 22:04:36 +01:00
Vitaly Baranov
b5b195f4e2
Merge branch 'master' into kafka_protobuf_issue26643 2021-12-10 23:22:35 +03:00
Vitaly Baranov
82c2d8dd2c Add synchronization to ProtobufSchemas. 2021-12-10 23:18:47 +03:00
Azat Khuzhin
837ff82312 Suppress UBSan errors for avg() function 
CI: https://s3.amazonaws.com/clickhouse-test-reports/0/dd2ccd3b17e7c59ed9c8184f00f8dae85ee87d1f/fuzzer_astfuzzerubsan,actions//report.html
 2021-12-10 22:23:39 +03:00
Azat Khuzhin
fa6090f588 Fix processing initial table (--table/stdin) in clickhouse-local 
This patch will:
- fix the issue when table had been tried to create multiple times for
  --queries-files
- create these table for --interactive mode (before it works only if you
  had some queries already, i.e. when it run interactive after
  non-interactive)

This will also make ClientBase interface a little bit cleaner, by
removing one abstract method getQueryTextPrefix()
 2021-12-10 22:08:03 +03:00
Kseniia Sumarokova
1865e37109
Merge pull request #32498 from Vxider/rename-window-view-functions 
Rename window-view function to time window function
 2021-12-10 22:07:53 +03:00
Kruglov Pavel
764e205d36
Fix resetParser in MsgPack format 2021-12-10 21:37:08 +03:00
Vitaly Baranov
73092942ea Take into account nested structures while filling missing columns while reading protobuf. 2021-12-10 21:11:06 +03:00
avogar
1be84d80d4 Fix async inserts for some input formats 2021-12-10 20:54:08 +03:00
Dao
505fbc78c7 fix groupBitmapAnd function on distributed table 2021-12-11 00:18:11 +08:00
mergify[bot]
2ea137ba55
Merge branch 'master' into fix-use-quota-bug 2021-12-10 15:42:48 +00:00
Kseniia Sumarokova
f5e949b88e
Ping CI 2021-12-10 18:12:27 +03:00
tavplubix
25427719d4
Try fix 'Directory tmp_merge_<part_name>' already exists (#32201) 
* try fix 'directory tmp_merge_<part_name>' already exists

* fix

* fix

* fix

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-12-10 16:29:51 +03:00
Vitaly Baranov
bf80b04e43
Merge branch 'master' into fix-use-quota-bug 2021-12-10 16:08:13 +03:00
vdimir
ed4ea6fe3a
Handle const column in JoinCommon::removeColumnNullability 2021-12-10 15:57:36 +03:00
Nikolai Kochetov
7bf5c83561 Fix queries with hasColumnInTable constant condition and non existing column. 2021-12-10 15:45:04 +03:00
vxider
d021e13b8c rename window function name 2021-12-10 09:59:50 +00:00
Vitaly Baranov
84320f7ba9 Fix first time calculations of the ends of quota intervals. 2021-12-10 12:54:33 +03:00
vxider
52fa82b4d4 rename window-view function to time window function 2021-12-10 09:43:44 +00:00
zhanglistar
de3aad085d 1. fix ut fail. 2021-12-10 10:26:28 +08:00
Alexey Milovidov
5c90ed2ed9 Unambiguous formatting of distributed queries 2021-12-10 00:55:14 +03:00
Alexey Milovidov
dc3565b3fe Quote more identifiers 2021-12-09 23:51:52 +03:00
Nikolai Kochetov
e41974fabf
Merge branch 'master' into fix-31538 2021-12-09 19:00:22 +03:00
Nikolai Kochetov
272c0bb35a Fix crash in case of MATERIALIZE COLUMN with no default expression. 2021-12-09 18:58:28 +03:00
zhanglistar
212b86aa19
Merge branch 'ClickHouse:master' into keepersizeopt 2021-12-09 23:32:18 +08:00
zhanglistar
2509e879db 1. snapshot add size of Node. 2021-12-09 23:09:56 +08:00
kssenii
f5a77fca39 Done 2021-12-09 14:44:25 +00:00
Kruglov Pavel
ef42cd6821
Merge pull request #32138 from Avogar/fix-apply-lambda 
Fix 'APPLY lambda' parsing
 2021-12-09 17:09:11 +03:00
Nikolai Kochetov
f4f06ca368 Fix assert. 2021-12-09 16:59:21 +03:00
Nikolai Kochetov
b3bd9e6a37 Fix arraySlice with null args. 2021-12-09 16:31:13 +03:00
Kseniia Sumarokova
2091008c15
Merge pull request #32324 from Vxider/rename-window-functions 
Rename window functions in WindowView
 2021-12-09 16:19:21 +03:00
kssenii
40c9ffdfff Disable 2021-12-09 14:15:23 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
zhanglistar
91f26edeca
Update KeeperStorage.h 
delete trailing whitespaces.
 2021-12-09 18:15:53 +08:00
zhanglistar
359257483a
Update KeeperStorage.h 2021-12-09 18:05:11 +08:00
zhanglistar
21f50f95b2
Update KeeperStorage.h 
fix code style
 2021-12-09 18:04:29 +08:00
zhanglistar
448d22a23c
Merge branch 'ClickHouse:master' into keepersizeopt 2021-12-09 17:46:02 +08:00
mergify[bot]
b11a1ded8d
Merge branch 'master' into rename-window-functions 2021-12-09 08:10:41 +00:00
robot-clickhouse
f2ccddb7be Auto version update to [21.13.1.1] [54458] 2021-12-09 00:33:01 +03:00
tavplubix
2de543cbfb
Merge pull request #32389 from ClickHouse/tavplubix-patch-4 
Follow-up to #32140
 2021-12-09 00:18:26 +03:00
Anton Popov
ea67abf3f0 fix custom serializations in native protocol 2021-12-08 21:59:36 +03:00
Kruglov Pavel
7ba60aa770 Change error codes to SYNTAX_ERROR 2021-12-08 21:29:50 +03:00
avogar
417bd12629 Fix 'APPLY lambda' parsing 2021-12-08 21:29:50 +03:00
Azat Khuzhin
5fad4f6204 clickhouse-local: fix CREATE DATABASE with Atomic engine 
Before it fails to create due to "metadata" directory had not been
created, since metadata_path is different for Atomic database, see
InterpreterCreateQuery.cpp.
 2021-12-08 21:23:00 +03:00
Anton Popov
61a5f8a61a add comments 2021-12-08 18:56:30 +03:00
Anton Popov
d8367334a3 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-08 18:26:19 +03:00
Mikhail f. Shiryaev
03927f5fc6
Revert "graphite: split tagged/plain rollup rules (for merges perfomance)" 
This reverts commit 303552f515.
 2021-12-08 15:49:21 +01:00
Nikita Mikhaylov
4c4430329c
Merge pull request #32390 from nikitamikhaylov/gtest_executor 
Another attempt to fix unit test Executor::RemoveTasksStress
 2021-12-08 17:28:29 +03:00
Nikita Mikhaylov
7a8fe9a9ad
Merge pull request #32234 from FrankChen021/span_name 
Improve the operation name of an opentelemetry span
 2021-12-08 15:46:19 +03:00
Nikita Mikhaylov
9a6a1379fd
Merge pull request #32358 from nikitamikhaylov/mutation-livelock 
Fixed empty mutations livelock
 2021-12-08 15:43:19 +03:00
Nikita Mikhaylov
398d2f5984 Fix 2021-12-08 12:36:55 +00:00
tavplubix
785d79531d
Update StorageReplicatedMergeTree.cpp 2021-12-08 15:27:32 +03:00
Nikolai Kochetov
dd906b2a3e Remove comments. 2021-12-08 14:25:00 +03:00
Nikolai Kochetov
07b2d69ea2 Another try. 2021-12-08 14:22:43 +03:00
Kruglov Pavel
e0a8c5a4ac
Merge pull request #32348 from CurtizJ/fix-materialize-column 
Add cast to `MATERIALIZE COLUMN`
 2021-12-08 11:13:17 +03:00
Kruglov Pavel
3498e13551
Merge pull request #32270 from ClickHouse/fix-strict-resize 
Fix possible Pipeline stuck in case of StrictResize processor.
 2021-12-08 11:03:53 +03:00
zhanglistar
368fa9144a 1. opt keeper sizeInBytes caculation. 2021-12-08 14:52:21 +08:00
mergify[bot]
ed0c36bf32
Merge branch 'master' into rename-window-functions 2021-12-07 21:04:53 +00:00
mergify[bot]
f0d6fe53c9
Merge branch 'master' into mutation-livelock 2021-12-07 18:29:07 +00:00
Nikolai Kochetov
2c6cb902c4 Revert "Better fix." 
This reverts commit f07fc08a49.
 2021-12-07 20:54:49 +03:00
Anton Popov
dd7387729b fix usage of non-materialized skip indexes 2021-12-07 20:21:27 +03:00
tavplubix
4f46ac6b30
Remove LeaderElection (#32140) 
* remove LeaderElection

* try fix tests

* Update test.py

* Update test.py
 2021-12-07 19:55:55 +03:00
Nikita Mikhaylov
76f91c2c6c Done 2021-12-07 16:50:57 +00:00
Dmitry Novik
4afaa8697c remove ya.make 2021-12-07 16:47:42 +03:00
Anton Popov
ef7ae7cb75 fix MATERIALIZE COLUMN in case when data type of expression is not equal to data type of column 2021-12-07 16:45:30 +03:00
Dmitry Novik
f138122259 fix GROUPING SETS formating 2021-12-07 16:44:09 +03:00
mergify[bot]
79351e9d59
Merge branch 'master' into span_name 2021-12-07 12:05:48 +00:00
Nikita Mikhaylov
514120adfe
Merge pull request #27526 from mathalex/get_fuzz_data 
Added `getFuzzerData` function
 2021-12-07 15:05:16 +03:00
Nikita Mikhaylov
7e036d31e9 One more style related commit 2021-12-07 11:29:46 +00:00
tavplubix
6c16348faa
Fix division by zero in avgWeighted with Decimal argument (#32303) 
* fix division by zero

* Update src/AggregateFunctions/AggregateFunctionAvg.h

Co-authored-by: Dmitry Novik <n0vik@clickhouse.com>

* remove trash

* Update AggregateFunctionAvg.h

Co-authored-by: Dmitry Novik <n0vik@clickhouse.com>
 2021-12-07 13:32:26 +03:00
Nikita Mikhaylov
d3d4524390 Merge branch 'master' of github.com:ClickHouse/ClickHouse into get_fuzz_data 2021-12-07 10:32:03 +00:00
Mikhail f. Shiryaev
5fc20b3e6a
Merge pull request #25122 from msaf1980/rollup_rules_type 
GraphiteMergeTree Rollup rules type
 2021-12-07 10:18:08 +01:00
Stig Bakken
bb521cd363 Add TABLE OVERRIDE feature for MaterializedMySQL databases 2021-12-07 09:45:02 +01:00
vxider
c4c517bb8a rename window functions 2021-12-07 08:14:00 +00:00
dongyifeng
9df664e1c9
fix bug when remove unneeded columns in subquery (#32289) 2021-12-07 11:09:39 +03:00
Kseniia Sumarokova
926fd568c7
Merge pull request #32113 from FrankChen021/url_http_header 
Set Content-Type in HTTP packets issued from URL engine
 2021-12-07 08:52:36 +03:00
Kseniia Sumarokova
eab6f0ba49
Update FormatFactory.cpp 2021-12-06 23:35:29 +03:00
Kseniia Sumarokova
e2cf5c6075
Merge pull request #32232 from Vxider/fix-window-view-parser 
Fix window view parser
 2021-12-06 23:33:43 +03:00
Kseniia Sumarokova
7e261245d3
Merge pull request #32256 from taiyang-li/default_for_hdfs 
Support default expression for storage hdfs and optimize fetching when source is column oriented
 2021-12-06 23:32:01 +03:00
Nikolai Kochetov
f07fc08a49 Better fix. 2021-12-06 22:08:32 +03:00
Nikita Mikhaylov
fb24e7181f Better 2021-12-06 18:27:06 +00:00
Kseniia Sumarokova
a2dfa883cb
Merge pull request #32049 from leosunli/32039 
#32039 Make HDFS replication configurable in WriteBufferFromHDFSImpl#WriteBu…
 2021-12-06 20:24:29 +03:00
tavplubix
657db07795
Merge pull request #32288 from ClickHouse/fix_xml_dict_cyclic_dependency 
Fix cyclic dependency with XML dictionary and StorageDictionary
 2021-12-06 20:02:38 +03:00
tavplubix
d2a606b8af
Merge pull request #32173 from zzsmdfj/MaterializedMySQL_support_VARBINARY 
MaterializedMySQL support VARBINARY type
 2021-12-06 19:56:58 +03:00
Nikita Mikhaylov
c92cfc38d4
Fix possible crash in DataTypeAggregateFunction (#32287) 2021-12-06 19:47:30 +03:00
Nikolai Kochetov
ce568bff90 Merge branch 'master' into fix-31114 2021-12-06 19:47:13 +03:00
msaf1980
303552f515
graphite: split tagged/plain rollup rules (for merges perfomance) 2021-12-06 16:49:08 +01:00
Nikita Mikhaylov
fa22fdc097 Merge branch 'master' of github.com:ClickHouse/ClickHouse into get_fuzz_data 2021-12-06 14:30:49 +00:00
vxider
2f4f2d9d5e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse 2021-12-06 14:06:00 +00:00
Alexander Tokmakov
7797a72907 fix false-positive ciclyc dependency with XML dict 2021-12-06 16:35:43 +03:00
mergify[bot]
52904a4973
Merge branch 'master' into default_for_hdfs 2021-12-06 13:19:49 +00:00
Nikolai Kochetov
1d75d8fd45 Fix assert. 2021-12-06 15:52:08 +03:00
Nikolai Kochetov
f064f2cdaa
Use seq_cst semantic for MergeTreeBackgroundExecutor mertic. (#32125) 2021-12-06 15:21:39 +03:00
Maksim Kita
11df9a6bc4
Merge pull request #32187 from kitaisreal/dictionary-qualify-name-with-database-fix 
DictionariesLoader qualify name with database fix
 2021-12-06 15:11:09 +03:00
mergify[bot]
644f59843e
Merge branch 'master' into MaterializedMySQL_support_VARBINARY 2021-12-06 12:02:31 +00:00
Nikolai Kochetov
32b4875273 Fix possible Pipeline stuck in case of StrictResize processor. 2021-12-06 14:53:39 +03:00
Maksim Kita
1ec9039b1d ExternalDictionariesLoader fix getCurrentDatabase multiple times 2021-12-06 14:01:56 +03:00
Kruglov Pavel
007e323515
Merge pull request #31820 from kitaisreal/fix-recursive-user-defined-functions-crash 
Fix recursive user defined functions crash
 2021-12-06 13:52:56 +03:00
tavplubix
96be71997f
Merge pull request #32206 from ClickHouse/fix_database_lazy_terminate 
Fix uncaught exception in DatabaseLazy
 2021-12-06 13:32:09 +03:00
Kruglov Pavel
64263a043a
Merge pull request #32190 from Avogar/fix-parsing-quoted-null 
Fix parsing error in NaN deserialization for Quoted escaping rule.
 2021-12-06 12:29:02 +03:00
Kruglov Pavel
cc71c537bc
Merge pull request #32204 from Avogar/skip-quoted-values 
Improve skiping unknown fields with Quoted escaping rule in Template/CustomSeparated formats
 2021-12-06 12:28:14 +03:00
alexey-milovidov
50c23a9fd6
Merge pull request #31123 from amosbird/repl-improvement 
Better clickhouse-client multiline input
 2021-12-06 12:22:13 +03:00
vxider
e7a4312768 add comments 2021-12-06 08:41:56 +00:00
Nikolai Kochetov
91c4c89c66
Merge pull request #31636 from Algunenano/pull_timeout 
Check max_execution_time in the pipeline and pulling executors
 2021-12-06 11:33:39 +03:00
Frank Chen
59a701ee6e
Fix style 2021-12-06 16:31:47 +08:00
Frank Chen
bf3d3ea25c
Fix style 2021-12-06 16:20:10 +08:00
Vladimir C
0c5079bf16
Merge pull request #32117 from kitaisreal/dictionaries-custom-query-condition-fix 2021-12-06 10:59:56 +03:00
Vladimir C
c502454088
Merge pull request #32185 from kitaisreal/range-hashed-dictionary-added-update-field-support 2021-12-06 10:57:47 +03:00
Dmitriy Dorofeev
31648d95e2
use application/x-ndjson for streaming JSON (#32223) 2021-12-06 10:49:14 +03:00
liyang830
251a4963f0 feat: modify test 2021-12-06 15:38:23 +08:00
liyang830
ff01b97fb5 feat: modify need_reset_counters simple method 2021-12-06 15:32:12 +08:00
vxider
b68136d1e4 small code style update 2021-12-06 07:12:21 +00:00
leosunli
3946b156a1 Fix bug 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:32 +08:00
leosunli
6f611eec91 Fix bug 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:32 +08:00
Kseniia Sumarokova
4017bb8f78 Update WriteBufferFromHDFS.h 2021-12-06 14:32:32 +08:00
leosunli
8d04495840 set hdfs_replication default value 0 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:32 +08:00
leosunli
e81a1dbe6a Make HDFS replication configurable in WriteBufferFromHDFSImpl#WriteBufferFromHDFSImpl 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:31 +08:00
leosunli
5be58a8839 Make HDFS replication configurable in WriteBufferFromHDFSImpl#WriteBufferFromHDFSImpl 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:31 +08:00
sunlisheng
0f045fa20a rename replication 
Signed-off-by: sunlisheng <lisheng.sls@alibaba-inc.com>
 2021-12-06 14:32:31 +08:00
sunlisheng
ef77f45f59 Make HDFS replication configurable in WriteBufferFromHDFSImpl#WriteBufferFromHDFSImpl 
Signed-off-by: sunlisheng <lisheng.sls@alibaba-inc.com>
 2021-12-06 14:32:31 +08:00
taiyang-li
a98f740bf2 remove unused file 2021-12-06 12:01:39 +08:00
taiyang-li
9d1e63be0f 1. support default expression for storage hdfs; 2. optimize storage hdfs when format is column oriented 2021-12-06 11:54:45 +08:00
Azat Khuzhin
2b1789adc5 Pass through version for aggregate function from SerializationAggregateFunction too 
Suggested-by: @kssenii
 2021-12-05 21:45:07 +03:00
Azat Khuzhin
428e34d811 Fix versioning of aggregate functions (fixes performance tests) 2021-12-05 21:45:07 +03:00
Vitaly Baranov
d709782088
Merge pull request #31988 from vitlibar/fix-skipping-columns-while-writing-protobuf 
Fix skipping columns while writing protobuf
 2021-12-05 18:01:11 +03:00
alexey-milovidov
6cfb17d10f
Merge pull request #32219 from azat/fix-jemalloc-on-osx 
Fix jemalloc under osx
 2021-12-05 16:34:18 +03:00
Maksim Kita
4077d769e7
Merge pull request #31949 from Algunenano/parser_speedup 
Speedup query parsing
 2021-12-05 16:24:45 +03:00
Maksim Kita
3621779b9f
Merge pull request #32184 from kitaisreal/accurate-cast-or-default-remove-separate-branch 
Function accurateCastOrDefault remove separate branch
 2021-12-05 16:23:50 +03:00
Azat Khuzhin
82e0e9d27c Fix minmax partition index during merge in case of empty part in future parts 
It is possible to get empty part in future parts, during merging, that
can be created in case of there is information about this part in
zookeeper, but no replicas has it
(StorageReplicatedMergeTree::createEmptyPartInsteadOfLost())

And this may incorrectly initialize min value, which will be updated
after one more merge (since during that merge there will be no more
information about empty part) or OPTIMIZE.
 2021-12-04 21:49:49 +03:00
Vitaly Baranov
2e0b480044 Improve error handling while serializing protobufs. 2021-12-04 21:42:45 +03:00
Vitaly Baranov
15e3dbe3f2 Fix skipping columns in Nested while writing protobuf. 2021-12-04 18:00:02 +03:00
vxider
78ceb5c883 add illegal arg check 2021-12-04 14:18:10 +00:00
frank chen
55d50c0b4d Improve span operation name 
Signed-off-by: frank chen <frank.chen021@outlook.com>
 2021-12-04 21:56:52 +08:00
vxider
c522c06755 fix windowview parser 2021-12-04 12:30:04 +00:00
Azat Khuzhin
abab7108e4 Fix QueryProfiler building under osx 
Fixes: #32165
 2021-12-04 09:19:34 +03:00
Azat Khuzhin
22a74dc68d Make jemalloc under osx even more reliable 
- explicitly call zone_register() again
- explicitly call malloc(free()) to initialize jemalloc()
 2021-12-04 09:19:34 +03:00
Kseniia Sumarokova
7984b246ff
Merge pull request #32209 from ClickHouse/kssenii-patch-3 
Fix flacky fileLog test (probably)
 2021-12-03 22:53:12 +03:00
Kseniia Sumarokova
52987d382b
Merge pull request #32208 from ClickHouse/kssenii-patch-2 
Update ASTCreateQuery.cpp
 2021-12-03 19:43:10 +03:00
Maksim Kita
0372403ebd
Merge pull request #32112 from FrankChen021/content-type 
Returns Content-Type as application/json for JSONEachRow if output_format_json_array_of_rows is enabled
 2021-12-03 18:59:54 +03:00
Maksim Kita
e7b19da7bc
Merge pull request #32038 from FrankChen021/trace_finish_time_bug 
Fix a bug that opentelemetry span log duration is zero
 2021-12-03 18:59:22 +03:00
Kseniia Sumarokova
7b1d4631bf
Update FileLogSource.cpp 2021-12-03 18:56:29 +03:00
Kseniia Sumarokova
d38d2ba14f
Merge pull request #32178 from kssenii/fix-window-view-tests 
Fix window view tests
 2021-12-03 18:45:44 +03:00
Kseniia Sumarokova
738265eed2
Update ASTCreateQuery.cpp 2021-12-03 18:17:36 +03:00
Alexander Tokmakov
90eba0c0f6 fix uncaught exception in DatabaseLazy 2021-12-03 17:45:53 +03:00
frank chen
c319b6fa32 Fix style 
Signed-off-by: frank chen <frank.chen021@outlook.com>
 2021-12-03 22:09:04 +08:00
mergify[bot]
ca59d823cf
Merge branch 'master' into fix-parsing-quoted-null 2021-12-03 13:39:27 +00:00
avogar
7549619b25 Improve skiping unknown fields with Quoted escaping rule in Template/CustomSeparated formats 2021-12-03 16:25:35 +03:00
kssenii
adaa32b41b Fix inner table name 2021-12-03 15:52:57 +03:00
avogar
4f136cb30c Fix NaN deserialization for Quoted escaping rule 2021-12-03 15:37:39 +03:00
frank chen
898db5b468 Resolve review comments 
Signed-off-by: frank chen <frank.chen021@outlook.com>
 2021-12-03 19:47:05 +08:00
Maksim Kita
4c916a0e38 DictionariesLoader qualify name with database fix 2021-12-03 14:31:49 +03:00
Maksim Kita
8e37281a7f RangeHashedDictionary fix bytes_allocated with update_field 2021-12-03 14:10:47 +03:00
Maksim Kita
4bbb02bbae RangeHashedDictionary added update_field support 2021-12-03 14:06:58 +03:00
kssenii
0470525ea3 Fix 2021-12-03 13:39:27 +03:00
Maksim Kita
2d3f773147 Function accurateCastOrDefault remove separate branch 2021-12-03 13:25:55 +03:00
mergify[bot]
478c35538d
Merge branch 'master' into trace_finish_time_bug 2021-12-03 10:12:48 +00:00