thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
79,881 Commits 9,338 Branches 1,783 Tags 2.1 GiB
df0af75864
Commit Graph

21096 Commits

Author SHA1 Message Date
alexey-milovidov
c7a5fb7582
Merge pull request #32507 from DMokhnatkin/apache-arrow-fix 
The proper handler for apache arrow column duplication
 2021-12-16 01:55:13 +03:00
alexey-milovidov
d112b30d78
Merge pull request #32291 from cmsxbc/fix-multiply-row-policies-on-same-column 
Fix broken select query with multiply row policies on same column
 2021-12-16 01:52:12 +03:00
alexey-milovidov
5293555d4d
Merge pull request #31797 from aiven/kmichel-fix-base64encode-too-fast 
Fix base64Encode adding trailing bytes on small strings
 2021-12-16 01:35:22 +03:00
kssenii
7583c8007e Some review fixes 2021-12-16 00:38:46 +03:00
mergify[bot]
41ae7c8eb1
Merge branch 'master' into fix_IP_conversion 2021-12-15 19:19:47 +00:00
Alexander Tokmakov
1b02136941 fix skipping of some mutations 2021-12-15 21:19:29 +03:00
Dmitry Novik
64b0365848 Fix build 2021-12-15 20:40:36 +03:00
Raúl Marín
c61277fbac Move the logic to the saving part instead 
This way we can reuse scalars from other queries (non MVs) and
leave the door open for future improvements
 2021-12-15 17:47:17 +01:00
Raúl Marín
f367634271 Do not use scalar cache when processing MV blocks 2021-12-15 17:25:30 +01:00
Dmitrii Mokhnatkin
2147658432
Proper handler for apache arrow column duplication 2021-12-15 18:30:32 +03:00
Maksim Kita
655cc20525
Merge pull request #32758 from kitaisreal/sql-user-defined-functions-support-subqueries 
SQLUserDefinedFunctions support subqueries
 2021-12-15 15:57:20 +03:00
Maksim Kita
ac3cb8c12b CacheDictionary dictionary source access race fix 2021-12-15 15:55:28 +03:00
kssenii
dc1f0c58fa Merge remote-tracking branch 'upstream/master' into table-override-pg 2021-12-15 15:27:58 +03:00
Anton Popov
ec46cbef20
Merge pull request #32756 from CurtizJ/fix-index-hypothesis 
Fix race in skipping index of type `hypothesis`
 2021-12-15 15:26:35 +03:00
Alexander Tokmakov
07bfbc26d6 fix restarting with unavailable mysql 2021-12-15 15:08:08 +03:00
alesapin
31600cdcf9 Clang-tidy mocking me 2021-12-15 13:02:17 +03:00
gulige
b244a4b9e5
Merge branch 'ClickHouse:master' into master 2021-12-15 17:14:37 +08:00
Amos Bird
9c9f4a3e6c
build fix 2021-12-15 16:55:18 +08:00
alesapin
d1dcd0adbf Followup 2021-12-15 11:44:18 +03:00
mergify[bot]
3a7179756f
Merge branch 'master' into fix-32668 2021-12-15 08:31:10 +00:00
kssenii
2e813fe3cd Update defaults 2021-12-15 08:18:43 +00:00
alesapin
2e8b75ade4 Fix configs 2021-12-15 10:45:27 +03:00
alesapin
2753fe01a3 Merge branch 'master' into fix_special_build_check 2021-12-15 10:35:28 +03:00
gulige
574bee9040
fix code refactoring as @kitaisreal suggested 2021-12-15 11:48:12 +08:00
gulige
c0aa1c3405
missing code refactoring as @kitaisreal suggested 2021-12-15 10:58:58 +08:00
gulige
c90bfaf78c
code refactoring as @kitaisreal suggested 
Better to not use emplace with std::npos, just find element, if it already exists then checkColumnStructure, after that we can safely update index_by_name and insert element.
 2021-12-15 10:23:34 +08:00
alexey-milovidov
de66f669b6
Merge pull request #32755 from cmsxbc/fix-fuzzbits-with-multiply-same-fixedstring 
fix crash fuzzbits with multiply same fixedstring
 2021-12-15 03:58:59 +03:00
mergify[bot]
003aeda627
Merge branch 'master' into fix-multiply-row-policies-on-same-column 2021-12-15 00:05:49 +00:00
mergify[bot]
4300f77532
Merge branch 'master' into fix_IP_conversion 2021-12-14 20:12:34 +00:00
Anton Popov
2e388a72da
Merge pull request #32742 from nautaa/flush_in_memory_part 
flush all InMemoryDataParts when wal is not enabled
 2021-12-14 23:09:10 +03:00
Kseniia Sumarokova
bf415378be
Merge pull request #32607 from Vxider/watch-events-supports 
Events clause support for Window View watch query
 2021-12-14 23:08:24 +03:00
Dmitry Novik
e8a96b53b1 Remove unused code 2021-12-14 22:44:54 +03:00
Dmitry Novik
56a3f4a000 Cleanup code 2021-12-14 22:15:14 +03:00
kssenii
47f549166d Table Override for PostgreSQL 2021-12-14 20:36:43 +03:00
mergify[bot]
c915b26a17
Merge branch 'master' into fix-multiply-row-policies-on-same-column 2021-12-14 16:21:35 +00:00
alesapin
d7663b2179 Merge branch 'master' into fix_special_build_check 2021-12-14 19:08:28 +03:00
alesapin
884801e1bd Fixing 2021-12-14 19:08:08 +03:00
Anton Popov
16312e7e4a Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-14 18:58:17 +03:00
Maksim Kita
c4bc72672e SQLUserDefinedFunctions support subqueries 2021-12-14 18:46:21 +03:00
mergify[bot]
cda4b6d68b
Merge branch 'master' into fix-fuzzbits-with-multiply-same-fixedstring 2021-12-14 15:43:10 +00:00
Anton Popov
84d6715e3d fix race in skipping index of type hypothesis 2021-12-14 18:28:55 +03:00
cmsxbc
0b4a9ebce0
fix fuzzbits with multiply same fixedstring 2021-12-14 23:26:18 +08:00
Maksim Kita
233505b665
Merge pull request #32536 from azat/sparse_hashed-dict-fix 
Fix sparse_hashed dict performance with sequential keys (wrong hash function)
 2021-12-14 17:44:47 +03:00
Nikolai Kochetov
d394f0e753 Allpy some more optimizations to NO_QUERY ast. 2021-12-14 17:19:18 +03:00
alesapin
9974f10631 Fix clang tidy 2021-12-14 15:34:20 +03:00
nautaa
a96da58360 fix 2021-12-14 20:11:43 +08:00
nautaa
5545793555 Merge branch 'flush_in_memory_part' of github.com:nautaa/ClickHouse into flush_in_memory_part 2021-12-14 19:42:15 +08:00
nautaa
3469fb8165 fix 2021-12-14 19:40:35 +08:00
mergify[bot]
60a864f91c
Merge branch 'master' into flush_in_memory_part 2021-12-14 11:14:21 +00:00
Anton Popov
bda0cc2f76
Merge pull request #32530 from Avogar/fix-async-inserts 
Fix async inserts for some input formats
 2021-12-14 14:07:05 +03:00
Anton Popov
bbb981655d
Merge pull request #32695 from CurtizJ/fix-add-ttl 
Fix `MATERIALIZE TTL` with `TTL WHERE` and `TTL GROUP BY`
 2021-12-14 14:01:15 +03:00
nautaa
0da3881ff4 flush all InMemoryDataParts when wal is not enabled 2021-12-14 16:31:17 +08:00
Azat Khuzhin
33ffd343f3 Add ability to drain connections synchronously 
This is mostly for testing, to avoid hiding errors.
 2021-12-14 10:18:57 +03:00
Azat Khuzhin
510eab25a1 Add timeout into "Timeout exceeded while reading from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
eb9968babf Add timeout into "Read timeout while draining from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
5c198525d2 Disconnect connection on unknown packet. 2021-12-14 10:18:57 +03:00
Azat Khuzhin
a7dc6f309f Remove SparseHashMap.h 
It was added only for arcadia build, and used only in one place, no need
to have a separate typedef for it.
 2021-12-14 10:07:14 +03:00
Azat Khuzhin
e16891713d Fix sparse_hashed dict performance with sequential keys (wrong hash function) 
In #27152 the hash function for sparse_hash_map had been changed to
std::hash<> switch it back to DefaultHash<> (ClickHouse builtin), since
std::hash<> for numeric keys returns itself and this does not works
great with sparse_hash_map.

I've tried the example from #32480 and using some hash fixes the
performance of sparse_hashed layout.

Fixes: #32480

v2: Add comments for SparseHashMap
 2021-12-14 10:07:14 +03:00
mergify[bot]
a5599d9fe2
Merge branch 'master' into watch-events-supports 2021-12-14 06:15:58 +00:00
gulige
2f8998880d
to clear up compiler warning about "C-style cast". 
use std::npos to replace (size_t)-1
 2021-12-14 10:41:51 +08:00
mergify[bot]
96f6915233
Merge branch 'master' into wip_mv_into_join 2021-12-14 01:31:47 +00:00
mergify[bot]
e64718ae01
Merge branch 'master' into strange-code-tcphandler 2021-12-14 01:28:47 +00:00
Kseniia Sumarokova
c9ec664094
Update MySQLHelpers.h 2021-12-14 01:25:53 +03:00
mergify[bot]
2b181d7107
Merge branch 'master' into mongo-extend-auth-2 2021-12-13 22:21:10 +00:00
kssenii
f0d0714e47 Pass timeouts for mysql 2021-12-13 22:12:33 +00:00
Kseniia Sumarokova
dadaeabda7
Merge pull request #32670 from CurtizJ/fix-reading-in-order 
Fix reading in order of sorting key from `Distributed` and `Merge` tables
 2021-12-14 00:42:48 +03:00
Alexey Milovidov
6879f03cb6 Remove garbage 2021-12-13 21:12:47 +03:00
mergify[bot]
827dbbe406
Merge branch 'master' into strange-code-tcphandler 2021-12-13 18:09:00 +00:00
alexey-milovidov
730c16bd0c
Merge pull request #32649 from vdimir/issue_32504 
Reset pipeline in BlockIO::onFinish
 2021-12-13 21:05:33 +03:00
Kseniia Sumarokova
a9ca84ff7d
Merge pull request #32325 from stigsb/table-overrides 
Add TABLE OVERRIDE feature for MaterializedMySQL databases
 2021-12-13 20:39:42 +03:00
Kseniia Sumarokova
9eb2a4fe90
Merge pull request #32643 from nautaa/genarateRandom 
support Date32 for genarateRandom engine
 2021-12-13 20:38:41 +03:00
alexey-milovidov
fd9d40925a
Merge pull request #32640 from ClickHouse/fix-function-file 
Fix surprisingly bad code in function `file`
 2021-12-13 19:34:36 +03:00
kssenii
ddbeadd104 Done 2021-12-13 16:09:18 +00:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency 
Azure Blob Storage Disks
 2021-12-13 18:28:48 +03:00
Anton Popov
0b68980977 fix MATERIALIZE TTL with WHERE and GROUP BY 2021-12-13 17:21:05 +03:00
tavplubix
60fae3af76
Follow-up to #32140 (#32667) 
* Update LeaderElection.h

* Update LeaderElection.h
 2021-12-13 16:41:46 +03:00
Kruglov Pavel
52328f6abc
Merge pull request #32490 from ClickHouse/quote-more-identifiers 
Fix issue with ambiguous query formatting in distributed queries.
 2021-12-13 16:41:01 +03:00
Raúl Marín
589b5a1fdc Merge remote-tracking branch 'blessed/master' into wip_mv_into_join 2021-12-13 14:05:46 +01:00
Anton Popov
745fd4fc16 fix reading in order from Distributed and Merge tables 2021-12-13 15:35:25 +03:00
Anton Popov
a241103714
Merge pull request #32359 from CurtizJ/fix-non-materialized-skip-index 
Fix usage of non-materialized skip indexes
 2021-12-13 14:43:58 +03:00
mergify[bot]
fa8f9671f4
Merge branch 'master' into fix-async-inserts 2021-12-13 11:13:20 +00:00
Raúl Marín
b25cfa0b4d InterpreterInsertQuery::buildChain: Make sure to keep the context alive 2021-12-13 11:50:46 +01:00
Raúl Marín
c9e9bd78b2 Remove old unused file 2021-12-13 11:14:10 +01:00
vdimir
bda4ecbe90
Catch exceprion in ~ZlibDeflatingWriteBuffer 2021-12-13 12:29:35 +03:00
vdimir
7236313435
Reset pipeline in BlockIO::onFinish 2021-12-13 12:29:13 +03:00
mergify[bot]
77be64f1f4
Merge branch 'master' into genarateRandom 2021-12-13 09:18:52 +00:00
mergify[bot]
34417017e4
Merge branch 'master' into master 2021-12-13 09:18:49 +00:00
mergify[bot]
f5e2c38452
Merge branch 'master' into fix_groupBitmapAnd_function_distributed_table 2021-12-13 09:18:45 +00:00
Nikolai Kochetov
22e6fc1685
Merge pull request #32067 from amosbird/projection-fix23 
Fix detaching parts with projections
 2021-12-13 12:00:17 +03:00
nautaa
7cb3d24847 support Date32 for genarateRandom engine 2021-12-13 15:52:12 +08:00
mergify[bot]
47b18c959f
Merge branch 'master' into fix-non-materialized-skip-index 2021-12-13 07:03:02 +00:00
gulige
2f7edf82ab
avoid inconsistent state in Block 
In the member function insert(position, elem) to class Block, if function checkColumnStructure<void> throws an exception for columns with same name but unidentical structure, it will result in the mismatch between index_by_name and data. However we could not presume upon that caller will not use this block object anymore. So it seems better to guarantee consistency innerly.
 2021-12-13 13:53:12 +08:00
Alexey Milovidov
71926a3a97 Fix surprisingly bad code in function "file" 2021-12-13 07:57:54 +03:00
Vxider
f6dceb53a5 small update 2021-12-13 02:38:16 +00:00
Vxider
caad9f6c79 small update 2021-12-12 17:30:44 +00:00
mergify[bot]
dd4eaad2f3
Merge branch 'master' into watch-events-supports 2021-12-12 15:22:53 +00:00
tavplubix
be35ab171b
Merge pull request #32572 from azat/DROP-INSERT-UAF-fix 
Fix table lifetime in case of parallel DROP TABLE and INSERT
 2021-12-12 17:05:07 +03:00
alexey-milovidov
0486c5f341
Merge pull request #32220 from azat/mt-fix-minmax-for-empty-part 
[RFC] Fix minmax partition index during merge in case of empty part in future parts
 2021-12-12 14:45:10 +03:00
cmsxbc
e4b405716f fix multiply row policies on same column 2021-12-12 09:54:07 +00:00
mergify[bot]
72606749ab
Merge branch 'master' into watch-events-supports 2021-12-12 09:51:52 +00:00
Kseniia Sumarokova
5dc7d7e3ef
Merge pull request #32584 from kssenii/rabbitmq-fix 
rabbitmq fix
 2021-12-12 12:34:09 +03:00
alexey-milovidov
6779196c6b
Merge pull request #32587 from ClickHouse/reinterpret-perf 
Slight performance improvement of "reinterpret" function
 2021-12-12 12:14:43 +03:00
alexey-milovidov
09a6e826e7
Merge pull request #32591 from ClickHouse/fix-indecent-error-message 
Fix indecent error message
 2021-12-12 12:12:38 +03:00
Vxider
7eb18ea21e watch events support for window view 2021-12-12 07:45:55 +00:00
alexey-milovidov
26d606c158
Merge pull request #32456 from ClickHouse/fix-29890 
Fix arraySlice with null args.
 2021-12-12 06:38:17 +03:00
alexey-milovidov
71df622b1f
Merge pull request #31888 from ClickHouse/fix-31114 
Remove LowCardinality from window function args.
 2021-12-12 06:37:55 +03:00
Alexey Milovidov
2c68192bb6 Fix strange code in TCPHandler 2021-12-12 05:35:33 +03:00
Alexey Milovidov
453135eb63 Fix indecent error message 2021-12-12 05:24:47 +03:00
alexey-milovidov
151834d743
Merge pull request #32417 from azat/local-atomic-db-fix 
clickhouse-local: fix CREATE DATABASE with Atomic engine
 2021-12-12 04:33:59 +03:00
alexey-milovidov
33674cbe58
Merge pull request #32535 from azat/avg-ubsan 
Suppress UBSan errors for avg() function
 2021-12-12 04:32:44 +03:00
alexey-milovidov
b8a15b666d
Merge pull request #32571 from azat/mv-non-phys-table 
Fix LOGICAL_ERROR for MATERIALIZED VIEW over table functions (i.e. numbers())
 2021-12-12 03:55:51 +03:00
Alexey Milovidov
58ffe89643 Merge branch 'master' into quote-more-identifiers 2021-12-12 03:33:20 +03:00
Alexey Milovidov
3b0f026827 Slight performance improvement of "reinterpret" function 2021-12-12 03:29:26 +03:00
alexey-milovidov
eaf1d764a7
Merge pull request #32578 from bharatnc/ncb/fixes 
remove unused headers in test
 2021-12-12 01:05:12 +03:00
alexey-milovidov
a1cf07fdd3
Merge pull request #32580 from azat/remove-arcadia 
Remove arcadia build support
 2021-12-12 01:03:42 +03:00
kssenii
89db2c57f4 Fix 2021-12-12 00:19:06 +03:00
Vitaly Baranov
8f9b0b4ed0
Merge pull request #32575 from vitlibar/improve-quota-end-of-interval-calculation 
Improve quota's end-of-interval calculations.
 2021-12-11 22:16:04 +03:00
Azat Khuzhin
57c027be0e Remove arcadia build support 2021-12-11 21:25:23 +03:00
Vitaly Baranov
463ce1fcee
Merge pull request #27822 from filimonov/kafka_protobuf_issue26643 
Test for issue #26643
 2021-12-11 20:31:22 +03:00
alexey-milovidov
0bed774d83
Merge pull request #32574 from azat/remove-readline 
[RFC] Remove readline support
 2021-12-11 20:15:03 +03:00
bharatnc
a3dfb89927 remove unused headers in test 2021-12-11 08:30:22 -08:00
zhanglistar
2af221df18 1. update node size in set processor. 2021-12-11 22:24:59 +08:00
Vitaly Baranov
eea269a829 Improve quota's end-of-interval calculations. 2021-12-11 13:56:38 +03:00
Azat Khuzhin
4e4837758a Remove readline support 
- it was not nested for a long time
- replxx is an upstream way for completion
 2021-12-11 11:29:59 +03:00
Vitaly Baranov
abe9dd3368
Merge pull request #32531 from vitlibar/fix-nested-array-sizes-for-missing-columns 
Improve handling nested structures with missing columns while reading protobuf
 2021-12-11 11:08:34 +03:00
mergify[bot]
1e28f7340e
Merge branch 'master' into kafka_protobuf_issue26643 2021-12-11 08:05:20 +00:00
Azat Khuzhin
ca9c5dc4eb Fix LOGICAL_ERROR for MATERIALIZED VIEW over table functions (i.e. numbers()) 
Replace LOGICAL_ERROR with QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW
in this case.
 2021-12-11 11:04:47 +03:00
Azat Khuzhin
9a71598973 Fix table lifetime in case of parallel DROP TABLE and INSERT 
Stress tests founds [1]:

==527==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x37078ffd in unsigned long std::__1::__cxx_atomic_fetch_add<unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order) obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1050:12
    1 0x37078ffd in std::__1::__atomic_base<unsigned long, true>::fetch_add(unsigned long, std::__1::memory_order) obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1719:17
    2 0x37078ffd in std::__1::__atomic_base<unsigned long, true>::operator++() obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1756:57
    3 0x37078ffd in SimpleIncrement::get() obj-x86_64-linux-gnu/../src/Common/SimpleIncrement.h:20:16
    4 0x37078ffd in DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeDataWriter.cpp:276:46
    5 0x373c446c in DB::MergeTreeSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeSink.cpp:27:65

  Uninitialized value was created by a heap deallocation
    6 0x32d481e8 in DB::DatabaseCatalog::TableMarkedAsDropped::~TableMarkedAsDropped() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.h:248:12
    7 0x32d3c134 in DB::DatabaseCatalog::dropTableDataTask() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.cpp:908:1

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32534/fa6090f588dbf4cbb5f28bd2210847b070bb8218/stress_test__memory__actions_.html

The query was CREATE MATERIALIZED VIEW ... POPULATE AS SELECT ... from
00040_aggregating_materialized_view test.
 2021-12-11 11:00:31 +03:00
Vitaly Baranov
94e2e3625b
Merge pull request #31656 from sunny19930321/fix-use-quota-bug 
Fix use quota bug
 2021-12-11 11:00:23 +03:00
Kseniia Sumarokova
9ede6beca7
Merge pull request #32534 from azat/local-file-table-fix 
Fix processing initial table (--table/stdin) in clickhouse-local
 2021-12-11 10:45:19 +03:00
Kseniia Sumarokova
7e0bc290da
Merge pull request #32460 from kssenii/mongo-extend-auth 
Allow empty credentials for mongo engine
 2021-12-11 10:42:15 +03:00
alexey-milovidov
c90e588e24
Merge pull request #32506 from ClickHouse/fix-has-column-in-table-subquery 
Fix queries with hasColumnInTable constant condition and non existing column
 2021-12-11 02:50:06 +03:00
alexey-milovidov
17e5f5ccfe
Merge pull request #32508 from vdimir/issue_32458 
Handle const column in JoinCommon::removeColumnNullability
 2021-12-11 01:21:10 +03:00
Anton Popov
37837f3881
Merge pull request #32453 from kssenii/disable-optimizations-for-window-functions 
Disable some optimizations for window functions
 2021-12-11 00:37:24 +03:00
Jakub Kuklis
578aec97e9 Removing TODO and NOTE marks used mainly as hooks for review in the PR 2021-12-10 22:04:38 +01:00
Jakub Kuklis
a3bc6d2fb7 Style corrections 2021-12-10 22:04:38 +01:00
Jakub Kuklis
85807adec4 Fixing variable shadow error 2021-12-10 22:04:38 +01:00
Jakub Kuklis
74e26354cc Committing missing files 2021-12-10 22:04:38 +01:00
Jakub Kuklis
41a88e49c9 Abstracting metadata creation for remote disks registrations, putting Blob Storage auth in a separate file 2021-12-10 22:04:38 +01:00
Jakub Kuklis
e58f74d8dc Connecting to Azurite and invoking simple INSERTs works in the integration test 2021-12-10 22:04:38 +01:00
Jakub Kuklis
8afe69ff93 Comming back to StorageSharedKeyCredential for the integration test, get 400 Bad requests from Azurite when trying to create a container now 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f180a7ba5f Solved Invalid storage account error with connection strings 2021-12-10 22:04:38 +01:00
Jakub Kuklis
008141de36 Enable authentication via StorageSharedKeyCredential 2021-12-10 22:04:38 +01:00
Jakub Kuklis
d51f979c7a Adding endpoint validation 2021-12-10 22:04:38 +01:00
Jakub Kuklis
2b7725bfc7 Enable connecting directly to storage accounts and creating new containers 2021-12-10 22:04:38 +01:00
Jakub Kuklis
910db6ca57 Code cleanup addressing PR remarks 2021-12-10 22:04:38 +01:00
Jakub Kuklis
6acbec4455 Rebased on master 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f6c5a2c63c Removing file checks after changing blob filenames 2021-12-10 22:04:38 +01:00
Jakub Kuklis
3aa6a0ead3 Adding random postfix to blob names, needs code cleanup 2021-12-10 22:04:38 +01:00
Jakub Kuklis
5ed511f9da Working skeleton for Blob Storage integration test 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0c65cfed1f Addressing first PR remarks 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7c2e116214 Adding max_retries config 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a33cea0525 Refactors after proper rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3728dbbaf Rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b8ef47006c Minor refactors before PR 2021-12-10 22:04:37 +01:00
Ryad ZENINE
c4421d6866 Small changes following review 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7f595eedbc Catch Blob Storage exceptions 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3c4a92cc0 Code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c4691affb2 Adding a new test for read with offset 2021-12-10 22:04:37 +01:00
Jakub Kuklis
9a00acfb72 Fixing read size error plus minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fd8ebe0d9f Maximum read 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c9f3164cba Bringing back old single part upload limit 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0d7a4b10c3 Using unique_ptr for Blob Client in Read Buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
12924b3659 Minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fa2261dee6 Changing offset in ReadBuffer to size_t 2021-12-10 22:04:37 +01:00
Jakub Kuklis
5e2b2b1eee Better search for blobs with a given name 2021-12-10 22:04:37 +01:00
Jakub Kuklis
33247db3f4 Adding finalize() to WriteBuffer, always using block blob uploads, code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
bf2476a076 Shared pointer for Azure client and fixing path reference in ReadBuffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
60dd22dd4e Checking for success of file removal operation 2021-12-10 22:04:37 +01:00
Jakub Kuklis
486f267b3b Further code cleanup and error codes improvement 2021-12-10 22:04:37 +01:00
Jakub Kuklis
6755c5e897 File deletion prototype 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a0cfafa387 checkUniqueId implementation using ListBlobs 2021-12-10 22:04:37 +01:00
Jakub Kuklis
956348073b ReadBufferFromBlobStorage operating like a proper stream, rather than a single download-read buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
95816875f4 Change false to true in supportZeroCopyReplication 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b41ab197e8 Blob Storage endpoint url verification using re2 2021-12-10 22:04:37 +01:00
Jakub Kuklis
63ec8d4244 Adding multipart upload to WriteBuffer, retries in ReadBuffer, additional settings for buffers 2021-12-10 22:04:37 +01:00
Jakub Kuklis
02060791e3 Adding settings getter and applyNewSettings 2021-12-10 22:04:37 +01:00
Jakub Kuklis
3f28f2c62b Enabling local cache for DiskBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
e3daf8aa49 Increase default thread pool size 2021-12-10 22:04:37 +01:00
Jakub Kuklis
dbb98e3636 Code cleanup, leaving some debug information printing under #ifdef VERBOSE_DEBUG_MODE 2021-12-10 22:04:37 +01:00
Jakub Kuklis
8230f8039c Two column MergeTree table with Blob Storage disk work! Fixed buffer in ReadBufferFromBlobStorage to be permanent (class attribute) 2021-12-10 22:04:37 +01:00
Jakub Kuklis
2b4d3665ab Basic inserts and selects work for a MergeTree table with a Blob Disk, based on a single column table 2021-12-10 22:04:37 +01:00
Jakub Kuklis
4b51ddee1b Initial check with write and read works for the Blob Storage Disk 2021-12-10 22:04:37 +01:00
Jakub Kuklis
03f85dee41 Managed to read markers file in SELECT, getting error for CODEC version now 2021-12-10 22:04:37 +01:00
Jakub Kuklis
482a10e62e Uploading to Blob Storage on INSERT seems to do something reasonable 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c43b90c58c Progressing with WriteBufferFromBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fb4c1d6686 Starting insertion pipeline 2021-12-10 22:04:37 +01:00
Jakub Kuklis
6436f5b04d Adding an authenticated client to DiskBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
ee1a568919 The following query passes without erors: CREATE TABLE ... ENGINE = MergeTree ... storage_policy = 'blob_storage', debug constants present in the code 2021-12-10 22:04:37 +01:00
Jakub Kuklis
cacdac7ba6 Moving implementation to the .cpp file 2021-12-10 22:04:37 +01:00
Jakub Kuklis
88b5feb59d Registering the Blob Storage Disk 2021-12-10 22:04:36 +01:00
Jakub Kuklis
20dda7f666 Giving Disk Blob Storage files proper names 2021-12-10 22:04:36 +01:00
Jakub Kuklis
0cf338e165 Skeleton for DiskBlobStorage 2021-12-10 22:04:36 +01:00
Jakub Kuklis
43f420cba8 Adding a flag to use Azure Blob Storage 2021-12-10 22:04:36 +01:00
Jakub Kuklis
162e2b334b Initial working example of Azure Blob Storage manipulation from within ClickHouse 2021-12-10 22:04:36 +01:00
Vitaly Baranov
b5b195f4e2
Merge branch 'master' into kafka_protobuf_issue26643 2021-12-10 23:22:35 +03:00
Vitaly Baranov
82c2d8dd2c Add synchronization to ProtobufSchemas. 2021-12-10 23:18:47 +03:00
Azat Khuzhin
837ff82312 Suppress UBSan errors for avg() function 
CI: https://s3.amazonaws.com/clickhouse-test-reports/0/dd2ccd3b17e7c59ed9c8184f00f8dae85ee87d1f/fuzzer_astfuzzerubsan,actions//report.html
 2021-12-10 22:23:39 +03:00
Azat Khuzhin
fa6090f588 Fix processing initial table (--table/stdin) in clickhouse-local 
This patch will:
- fix the issue when table had been tried to create multiple times for
  --queries-files
- create these table for --interactive mode (before it works only if you
  had some queries already, i.e. when it run interactive after
  non-interactive)

This will also make ClientBase interface a little bit cleaner, by
removing one abstract method getQueryTextPrefix()
 2021-12-10 22:08:03 +03:00
Kseniia Sumarokova
1865e37109
Merge pull request #32498 from Vxider/rename-window-view-functions 
Rename window-view function to time window function
 2021-12-10 22:07:53 +03:00
Kruglov Pavel
764e205d36
Fix resetParser in MsgPack format 2021-12-10 21:37:08 +03:00
Vitaly Baranov
73092942ea Take into account nested structures while filling missing columns while reading protobuf. 2021-12-10 21:11:06 +03:00
avogar
1be84d80d4 Fix async inserts for some input formats 2021-12-10 20:54:08 +03:00
Dao
505fbc78c7 fix groupBitmapAnd function on distributed table 2021-12-11 00:18:11 +08:00
mergify[bot]
2ea137ba55
Merge branch 'master' into fix-use-quota-bug 2021-12-10 15:42:48 +00:00
Kseniia Sumarokova
f5e949b88e
Ping CI 2021-12-10 18:12:27 +03:00
tavplubix
25427719d4
Try fix 'Directory tmp_merge_<part_name>' already exists (#32201) 
* try fix 'directory tmp_merge_<part_name>' already exists

* fix

* fix

* fix

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-12-10 16:29:51 +03:00
Vitaly Baranov
bf80b04e43
Merge branch 'master' into fix-use-quota-bug 2021-12-10 16:08:13 +03:00
vdimir
ed4ea6fe3a
Handle const column in JoinCommon::removeColumnNullability 2021-12-10 15:57:36 +03:00
Nikolai Kochetov
7bf5c83561 Fix queries with hasColumnInTable constant condition and non existing column. 2021-12-10 15:45:04 +03:00
vxider
d021e13b8c rename window function name 2021-12-10 09:59:50 +00:00
Vitaly Baranov
84320f7ba9 Fix first time calculations of the ends of quota intervals. 2021-12-10 12:54:33 +03:00
vxider
52fa82b4d4 rename window-view function to time window function 2021-12-10 09:43:44 +00:00
zhanglistar
de3aad085d 1. fix ut fail. 2021-12-10 10:26:28 +08:00
Alexey Milovidov
5c90ed2ed9 Unambiguous formatting of distributed queries 2021-12-10 00:55:14 +03:00
Alexey Milovidov
dc3565b3fe Quote more identifiers 2021-12-09 23:51:52 +03:00
Nikolai Kochetov
e41974fabf
Merge branch 'master' into fix-31538 2021-12-09 19:00:22 +03:00
Nikolai Kochetov
272c0bb35a Fix crash in case of MATERIALIZE COLUMN with no default expression. 2021-12-09 18:58:28 +03:00
zhanglistar
212b86aa19
Merge branch 'ClickHouse:master' into keepersizeopt 2021-12-09 23:32:18 +08:00
zhanglistar
2509e879db 1. snapshot add size of Node. 2021-12-09 23:09:56 +08:00
kssenii
f5a77fca39 Done 2021-12-09 14:44:25 +00:00
Kruglov Pavel
ef42cd6821
Merge pull request #32138 from Avogar/fix-apply-lambda 
Fix 'APPLY lambda' parsing
 2021-12-09 17:09:11 +03:00
Nikolai Kochetov
f4f06ca368 Fix assert. 2021-12-09 16:59:21 +03:00
Nikolai Kochetov
b3bd9e6a37 Fix arraySlice with null args. 2021-12-09 16:31:13 +03:00
Kseniia Sumarokova
2091008c15
Merge pull request #32324 from Vxider/rename-window-functions 
Rename window functions in WindowView
 2021-12-09 16:19:21 +03:00
kssenii
40c9ffdfff Disable 2021-12-09 14:15:23 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
zhanglistar
91f26edeca
Update KeeperStorage.h 
delete trailing whitespaces.
 2021-12-09 18:15:53 +08:00
zhanglistar
359257483a
Update KeeperStorage.h 2021-12-09 18:05:11 +08:00
zhanglistar
21f50f95b2
Update KeeperStorage.h 
fix code style
 2021-12-09 18:04:29 +08:00
zhanglistar
448d22a23c
Merge branch 'ClickHouse:master' into keepersizeopt 2021-12-09 17:46:02 +08:00
mergify[bot]
b11a1ded8d
Merge branch 'master' into rename-window-functions 2021-12-09 08:10:41 +00:00
robot-clickhouse
f2ccddb7be Auto version update to [21.13.1.1] [54458] 2021-12-09 00:33:01 +03:00
tavplubix
2de543cbfb
Merge pull request #32389 from ClickHouse/tavplubix-patch-4 
Follow-up to #32140
 2021-12-09 00:18:26 +03:00
Anton Popov
ea67abf3f0 fix custom serializations in native protocol 2021-12-08 21:59:36 +03:00
Kruglov Pavel
7ba60aa770 Change error codes to SYNTAX_ERROR 2021-12-08 21:29:50 +03:00
avogar
417bd12629 Fix 'APPLY lambda' parsing 2021-12-08 21:29:50 +03:00
Azat Khuzhin
5fad4f6204 clickhouse-local: fix CREATE DATABASE with Atomic engine 
Before it fails to create due to "metadata" directory had not been
created, since metadata_path is different for Atomic database, see
InterpreterCreateQuery.cpp.
 2021-12-08 21:23:00 +03:00
Anton Popov
61a5f8a61a add comments 2021-12-08 18:56:30 +03:00
Anton Popov
d8367334a3 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-08 18:26:19 +03:00
Mikhail f. Shiryaev
03927f5fc6
Revert "graphite: split tagged/plain rollup rules (for merges perfomance)" 
This reverts commit 303552f515.
 2021-12-08 15:49:21 +01:00
Nikita Mikhaylov
4c4430329c
Merge pull request #32390 from nikitamikhaylov/gtest_executor 
Another attempt to fix unit test Executor::RemoveTasksStress
 2021-12-08 17:28:29 +03:00
Nikita Mikhaylov
7a8fe9a9ad
Merge pull request #32234 from FrankChen021/span_name 
Improve the operation name of an opentelemetry span
 2021-12-08 15:46:19 +03:00
Nikita Mikhaylov
9a6a1379fd
Merge pull request #32358 from nikitamikhaylov/mutation-livelock 
Fixed empty mutations livelock
 2021-12-08 15:43:19 +03:00
Nikita Mikhaylov
398d2f5984 Fix 2021-12-08 12:36:55 +00:00
tavplubix
785d79531d
Update StorageReplicatedMergeTree.cpp 2021-12-08 15:27:32 +03:00
Nikolai Kochetov
dd906b2a3e Remove comments. 2021-12-08 14:25:00 +03:00
Nikolai Kochetov
07b2d69ea2 Another try. 2021-12-08 14:22:43 +03:00
Kruglov Pavel
e0a8c5a4ac
Merge pull request #32348 from CurtizJ/fix-materialize-column 
Add cast to `MATERIALIZE COLUMN`
 2021-12-08 11:13:17 +03:00
Kruglov Pavel
3498e13551
Merge pull request #32270 from ClickHouse/fix-strict-resize 
Fix possible Pipeline stuck in case of StrictResize processor.
 2021-12-08 11:03:53 +03:00