thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
80,046 Commits 9,326 Branches 1,783 Tags 2.1 GiB
7dab7caa9d
Commit Graph

21155 Commits

Author SHA1 Message Date
Nikolai Kochetov
ffdcf52a9e
Merge pull request #32858 from ClickHouse/fix-has-column-in-table-subquery-2 
Always apply const-condition-if optimization.
 2021-12-17 20:39:30 +03:00
Nikolai Kochetov
93a33d52dc Merge branch 'fix-32668' of github.com:yandex/ClickHouse into fix-32668 2021-12-17 20:37:29 +03:00
Nikolai Kochetov
c8a92c046f Another try 2021-12-17 20:36:37 +03:00
Kseniia Sumarokova
f6e7e11742
Merge pull request #32907 from kssenii/url-better 
Better handling of globs for url storage
 2021-12-17 19:08:38 +03:00
Harry-Lee
ef36befd78 Issue 135: Fix null pointer dereference 2021-12-17 10:56:19 -05:00
root
6b3366dc70 [Issue #129] Fix Weak Regular Expression [v21.3-insecure] 
Problem Descripton:
==================
The “key_regexp” constant in the “DiskS3” class contains a regular expression (RegEx) for checking a key path.
However, the RegEx starts and ends with “.*”.
This is a wildcard that an attacker could use to exploit this file path check by padding the beginning and end with whatever path they’d like.

Recommendation:
==============
Tighten up the RegEx by replacing the “.*” at the beginning and end with more specific items.

Fix:
===
Replace the .* at the end with $ as there is no need to continue after the key file.
Leave the beginning .* as is because the path for the key can have any characters or in any path of the system.
 2021-12-17 10:30:16 -05:00
kssenii
40c266840c Fixes 2021-12-17 18:16:29 +03:00
Dao
2fb554c61f increase version to resolve conflict with master branch 2021-12-17 23:09:08 +08:00
vdimir
1eb032ac83
Style AggregateFunctionTopK.cpp 2021-12-17 18:04:59 +03:00
minhthucdao
001900655e
Merge branch 'master' into fix_groupBitmapAnd_function_distributed_table 2021-12-17 23:03:47 +08:00
Dao
52af96627c introduce new version and add test 2021-12-17 22:59:02 +08:00
vdimir
5c487b3bdc
Fix types for topKWeightedState 2021-12-17 17:54:12 +03:00
Kseniia Sumarokova
70937088e9
Merge pull request #32886 from kssenii/azure-rename 
Rename files and code from BlobStorage to AzureBlobStorage
 2021-12-17 16:59:08 +03:00
cmsxbc
ce6987ad4d
fix wrong testcase in gtest_Parser 2021-12-17 20:59:50 +08:00
alesapin
da1d04767c
Merge pull request #32366 from bigo-sg/keepersizeopt 
performance improve of keeper sizeInBytes calculation.
 2021-12-17 15:34:55 +03:00
Nikolai Kochetov
20b88d9b4c Always apply const if optimixation. 2021-12-17 15:31:30 +03:00
alesapin
6bd7e425c6
Merge pull request #22535 from CurtizJ/sparse-serialization 
Sparse serialization and ColumnSparse
 2021-12-17 15:26:17 +03:00
kssenii
b1794a47c3 StorageURL improve 2021-12-17 14:03:37 +03:00
Nikolai Kochetov
7b56e97cf8 Revert "Add debug logging" 
This reverts commit 49a9bc6b90.
 2021-12-17 13:26:05 +03:00
kssenii
ed42620530 Rename BlobStorage to AzureBlobStorage 2021-12-17 12:24:05 +03:00
Kseniia Sumarokova
77b3ebe44e
Merge pull request #32734 from kssenii/mysql-fix-timeouts 
Allow to control connection timeouts for mysql
 2021-12-17 10:13:33 +03:00
zzsmdfj
e9eed1f927 to #15182_MaterializeMySQL_support_bit_type 2021-12-17 15:03:24 +08:00
zhanglistar
e4006b2cfd
Merge branch 'ClickHouse:master' into keepersizeopt 2021-12-17 14:35:19 +08:00
tavplubix
db01e94f66
Merge pull request #32802 from ClickHouse/fix_restart_unavailable_mysql 
Fix restarting with unavailable MySQL
 2021-12-17 08:57:15 +03:00
alexey-milovidov
7dbdcf91bf
Merge pull request #32814 from ClickHouse/fix_skipped_mutations 
Fix skipping of some mutations
 2021-12-17 08:53:36 +03:00
kssenii
97222be3d9 Merge remote-tracking branch 'upstream/master' into table-override-pg 2021-12-17 01:14:28 +03:00
kssenii
ca283a4bb8 Fix optimization with lazy seek 2021-12-17 00:18:28 +03:00
alexey-milovidov
6cb54b4092
Merge pull request #31965 from azat/connection-drain-pull 
Add ability to drain connections synchronously
 2021-12-16 23:40:31 +03:00
Nikolai Kochetov
96384fe4f3
Update WindowTransform.cpp 2021-12-16 21:20:39 +03:00
Nikolai Kochetov
82f94a35f9
Update WindowTransform.cpp 2021-12-16 21:20:01 +03:00
Nikolai Kochetov
e56553ef11
Update InterpreterSelectQuery.cpp 2021-12-16 21:18:35 +03:00
Nikolai Kochetov
e28bcbfcea Fix MV query with multiple chunk result. 2021-12-16 21:17:05 +03:00
Nikolai Kochetov
49a9bc6b90 Add debug logging 2021-12-16 20:29:04 +03:00
Nikolai Kochetov
4799f07922
Merge branch 'master' into fix-has-column-in-table-subquery-2 2021-12-16 19:02:58 +03:00
Nikolai Kochetov
1b8ed6276a Allow string for constant if consition optmization. 2021-12-16 19:00:59 +03:00
Anton Popov
7e39fd14c2 fix test 2021-12-16 16:44:55 +03:00
mergify[bot]
4722bc6649
Merge branch 'master' into fix_skipped_mutations 2021-12-16 13:25:00 +00:00
mergify[bot]
cf37a1ba07
Merge branch 'master' into fix_restart_unavailable_mysql 2021-12-16 13:17:13 +00:00
Dmitry Novik
0f3e2bb32f Fix removing constant columns 2021-12-16 15:39:25 +03:00
凌涛
bf1098484e modify code style 2021-12-16 20:31:06 +08:00
凌涛
ad34217873 Add hints for clickhouse client options 2021-12-16 20:19:00 +08:00
Anton Popov
2945eac8c9 fix build 2021-12-16 15:05:42 +03:00
mergify[bot]
214f072c48
Merge branch 'master' into fix_echo_option_not_used_in_batch_mode 2021-12-16 11:57:23 +00:00
alesapin
c338bee718
Revert "Fix build issue related to azure blob storage" 2021-12-16 14:37:12 +03:00
alesapin
c0cebaea79
Merge pull request #32788 from amosbird/buildfix 
Fix build issue related to azure blob storage
 2021-12-16 13:45:22 +03:00
Vladimir C
2f2d98d8be
Merge pull request #32811 from Algunenano/materialized_view_scalar_subqueries 2021-12-16 13:35:22 +03:00
N. Kolotov
adf5fcdd0a Fixed --echo option is not used by clickhouse-client in batch mode with single query. 2021-12-16 13:10:49 +03:00
Maksim Kita
858d9f0f5f StorageLiveView fix function style 2021-12-16 12:29:20 +03:00
Maksim Kita
33374f8b0a
Merge pull request #31062 from Enmk/Governance/view_comment 
Views with comment
 2021-12-16 12:27:12 +03:00
Kseniia Sumarokova
947ef81f53
Update ReadWriteBufferFromHTTP.h 2021-12-16 09:14:09 +03:00
zhanglistar
bc4a2ca073
Update KeeperStorage.cpp 
modify potential overflow sub.
 2021-12-16 09:25:38 +08:00
alexey-milovidov
6d82644094
Update DatabaseMySQL.cpp 2021-12-16 02:02:36 +03:00
alexey-milovidov
6083869b5d
Merge pull request #32669 from Algunenano/wip_mv_into_join 
Fix LOGICAL_ERROR when the target of a materialized view is a JOIN or a SET table
 2021-12-16 02:01:31 +03:00
alexey-milovidov
c7a5fb7582
Merge pull request #32507 from DMokhnatkin/apache-arrow-fix 
The proper handler for apache arrow column duplication
 2021-12-16 01:55:13 +03:00
alexey-milovidov
d112b30d78
Merge pull request #32291 from cmsxbc/fix-multiply-row-policies-on-same-column 
Fix broken select query with multiply row policies on same column
 2021-12-16 01:52:12 +03:00
alexey-milovidov
5293555d4d
Merge pull request #31797 from aiven/kmichel-fix-base64encode-too-fast 
Fix base64Encode adding trailing bytes on small strings
 2021-12-16 01:35:22 +03:00
kssenii
7583c8007e Some review fixes 2021-12-16 00:38:46 +03:00
mergify[bot]
41ae7c8eb1
Merge branch 'master' into fix_IP_conversion 2021-12-15 19:19:47 +00:00
Alexander Tokmakov
1b02136941 fix skipping of some mutations 2021-12-15 21:19:29 +03:00
Dmitry Novik
64b0365848 Fix build 2021-12-15 20:40:36 +03:00
Raúl Marín
c61277fbac Move the logic to the saving part instead 
This way we can reuse scalars from other queries (non MVs) and
leave the door open for future improvements
 2021-12-15 17:47:17 +01:00
Raúl Marín
f367634271 Do not use scalar cache when processing MV blocks 2021-12-15 17:25:30 +01:00
Dmitrii Mokhnatkin
2147658432
Proper handler for apache arrow column duplication 2021-12-15 18:30:32 +03:00
Maksim Kita
655cc20525
Merge pull request #32758 from kitaisreal/sql-user-defined-functions-support-subqueries 
SQLUserDefinedFunctions support subqueries
 2021-12-15 15:57:20 +03:00
Maksim Kita
ac3cb8c12b CacheDictionary dictionary source access race fix 2021-12-15 15:55:28 +03:00
kssenii
dc1f0c58fa Merge remote-tracking branch 'upstream/master' into table-override-pg 2021-12-15 15:27:58 +03:00
Anton Popov
ec46cbef20
Merge pull request #32756 from CurtizJ/fix-index-hypothesis 
Fix race in skipping index of type `hypothesis`
 2021-12-15 15:26:35 +03:00
Alexander Tokmakov
07bfbc26d6 fix restarting with unavailable mysql 2021-12-15 15:08:08 +03:00
Raúl Marín
3de002c7c9 Merge remote-tracking branch 'blessed/master' into libcxx13_take2 2021-12-15 12:57:57 +01:00
alesapin
31600cdcf9 Clang-tidy mocking me 2021-12-15 13:02:17 +03:00
Raúl Marín
cd465ca0a0 Merge remote-tracking branch 'blessed/master' into libcxx13_take2 2021-12-15 10:55:23 +01:00
gulige
b244a4b9e5
Merge branch 'ClickHouse:master' into master 2021-12-15 17:14:37 +08:00
Amos Bird
9c9f4a3e6c
build fix 2021-12-15 16:55:18 +08:00
alesapin
d1dcd0adbf Followup 2021-12-15 11:44:18 +03:00
mergify[bot]
3a7179756f
Merge branch 'master' into fix-32668 2021-12-15 08:31:10 +00:00
kssenii
2e813fe3cd Update defaults 2021-12-15 08:18:43 +00:00
alesapin
2e8b75ade4 Fix configs 2021-12-15 10:45:27 +03:00
alesapin
2753fe01a3 Merge branch 'master' into fix_special_build_check 2021-12-15 10:35:28 +03:00
gulige
574bee9040
fix code refactoring as @kitaisreal suggested 2021-12-15 11:48:12 +08:00
gulige
c0aa1c3405
missing code refactoring as @kitaisreal suggested 2021-12-15 10:58:58 +08:00
gulige
c90bfaf78c
code refactoring as @kitaisreal suggested 
Better to not use emplace with std::npos, just find element, if it already exists then checkColumnStructure, after that we can safely update index_by_name and insert element.
 2021-12-15 10:23:34 +08:00
alexey-milovidov
de66f669b6
Merge pull request #32755 from cmsxbc/fix-fuzzbits-with-multiply-same-fixedstring 
fix crash fuzzbits with multiply same fixedstring
 2021-12-15 03:58:59 +03:00
mergify[bot]
003aeda627
Merge branch 'master' into fix-multiply-row-policies-on-same-column 2021-12-15 00:05:49 +00:00
mergify[bot]
4300f77532
Merge branch 'master' into fix_IP_conversion 2021-12-14 20:12:34 +00:00
Anton Popov
2e388a72da
Merge pull request #32742 from nautaa/flush_in_memory_part 
flush all InMemoryDataParts when wal is not enabled
 2021-12-14 23:09:10 +03:00
Kseniia Sumarokova
bf415378be
Merge pull request #32607 from Vxider/watch-events-supports 
Events clause support for Window View watch query
 2021-12-14 23:08:24 +03:00
Dmitry Novik
e8a96b53b1 Remove unused code 2021-12-14 22:44:54 +03:00
Dmitry Novik
56a3f4a000 Cleanup code 2021-12-14 22:15:14 +03:00
kssenii
47f549166d Table Override for PostgreSQL 2021-12-14 20:36:43 +03:00
mergify[bot]
c915b26a17
Merge branch 'master' into fix-multiply-row-policies-on-same-column 2021-12-14 16:21:35 +00:00
alesapin
d7663b2179 Merge branch 'master' into fix_special_build_check 2021-12-14 19:08:28 +03:00
alesapin
884801e1bd Fixing 2021-12-14 19:08:08 +03:00
Anton Popov
16312e7e4a Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-14 18:58:17 +03:00
Maksim Kita
c4bc72672e SQLUserDefinedFunctions support subqueries 2021-12-14 18:46:21 +03:00
mergify[bot]
cda4b6d68b
Merge branch 'master' into fix-fuzzbits-with-multiply-same-fixedstring 2021-12-14 15:43:10 +00:00
Anton Popov
84d6715e3d fix race in skipping index of type hypothesis 2021-12-14 18:28:55 +03:00
cmsxbc
0b4a9ebce0
fix fuzzbits with multiply same fixedstring 2021-12-14 23:26:18 +08:00
Maksim Kita
233505b665
Merge pull request #32536 from azat/sparse_hashed-dict-fix 
Fix sparse_hashed dict performance with sequential keys (wrong hash function)
 2021-12-14 17:44:47 +03:00
Nikolai Kochetov
d394f0e753 Allpy some more optimizations to NO_QUERY ast. 2021-12-14 17:19:18 +03:00
alesapin
9974f10631 Fix clang tidy 2021-12-14 15:34:20 +03:00
nautaa
a96da58360 fix 2021-12-14 20:11:43 +08:00
Raúl Marín
44f3b1c9d2 Merge remote-tracking branch 'blessed/master' into libcxx13_take2 2021-12-14 13:05:01 +01:00
nautaa
5545793555 Merge branch 'flush_in_memory_part' of github.com:nautaa/ClickHouse into flush_in_memory_part 2021-12-14 19:42:15 +08:00
nautaa
3469fb8165 fix 2021-12-14 19:40:35 +08:00
mergify[bot]
60a864f91c
Merge branch 'master' into flush_in_memory_part 2021-12-14 11:14:21 +00:00
Anton Popov
bda0cc2f76
Merge pull request #32530 from Avogar/fix-async-inserts 
Fix async inserts for some input formats
 2021-12-14 14:07:05 +03:00
Anton Popov
bbb981655d
Merge pull request #32695 from CurtizJ/fix-add-ttl 
Fix `MATERIALIZE TTL` with `TTL WHERE` and `TTL GROUP BY`
 2021-12-14 14:01:15 +03:00
nautaa
0da3881ff4 flush all InMemoryDataParts when wal is not enabled 2021-12-14 16:31:17 +08:00
Azat Khuzhin
33ffd343f3 Add ability to drain connections synchronously 
This is mostly for testing, to avoid hiding errors.
 2021-12-14 10:18:57 +03:00
Azat Khuzhin
510eab25a1 Add timeout into "Timeout exceeded while reading from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
eb9968babf Add timeout into "Read timeout while draining from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
5c198525d2 Disconnect connection on unknown packet. 2021-12-14 10:18:57 +03:00
Azat Khuzhin
a7dc6f309f Remove SparseHashMap.h 
It was added only for arcadia build, and used only in one place, no need
to have a separate typedef for it.
 2021-12-14 10:07:14 +03:00
Azat Khuzhin
e16891713d Fix sparse_hashed dict performance with sequential keys (wrong hash function) 
In #27152 the hash function for sparse_hash_map had been changed to
std::hash<> switch it back to DefaultHash<> (ClickHouse builtin), since
std::hash<> for numeric keys returns itself and this does not works
great with sparse_hash_map.

I've tried the example from #32480 and using some hash fixes the
performance of sparse_hashed layout.

Fixes: #32480

v2: Add comments for SparseHashMap
 2021-12-14 10:07:14 +03:00
mergify[bot]
a5599d9fe2
Merge branch 'master' into watch-events-supports 2021-12-14 06:15:58 +00:00
gulige
2f8998880d
to clear up compiler warning about "C-style cast". 
use std::npos to replace (size_t)-1
 2021-12-14 10:41:51 +08:00
mergify[bot]
96f6915233
Merge branch 'master' into wip_mv_into_join 2021-12-14 01:31:47 +00:00
mergify[bot]
e64718ae01
Merge branch 'master' into strange-code-tcphandler 2021-12-14 01:28:47 +00:00
Kseniia Sumarokova
c9ec664094
Update MySQLHelpers.h 2021-12-14 01:25:53 +03:00
mergify[bot]
2b181d7107
Merge branch 'master' into mongo-extend-auth-2 2021-12-13 22:21:10 +00:00
kssenii
f0d0714e47 Pass timeouts for mysql 2021-12-13 22:12:33 +00:00
Kseniia Sumarokova
dadaeabda7
Merge pull request #32670 from CurtizJ/fix-reading-in-order 
Fix reading in order of sorting key from `Distributed` and `Merge` tables
 2021-12-14 00:42:48 +03:00
Alexey Milovidov
6879f03cb6 Remove garbage 2021-12-13 21:12:47 +03:00
mergify[bot]
827dbbe406
Merge branch 'master' into strange-code-tcphandler 2021-12-13 18:09:00 +00:00
alexey-milovidov
730c16bd0c
Merge pull request #32649 from vdimir/issue_32504 
Reset pipeline in BlockIO::onFinish
 2021-12-13 21:05:33 +03:00
Kseniia Sumarokova
a9ca84ff7d
Merge pull request #32325 from stigsb/table-overrides 
Add TABLE OVERRIDE feature for MaterializedMySQL databases
 2021-12-13 20:39:42 +03:00
Kseniia Sumarokova
9eb2a4fe90
Merge pull request #32643 from nautaa/genarateRandom 
support Date32 for genarateRandom engine
 2021-12-13 20:38:41 +03:00
alexey-milovidov
fd9d40925a
Merge pull request #32640 from ClickHouse/fix-function-file 
Fix surprisingly bad code in function `file`
 2021-12-13 19:34:36 +03:00
kssenii
ddbeadd104 Done 2021-12-13 16:09:18 +00:00
Raúl Marín
cc6f7a71e5 Fix String reinterpret 
https://github.com/ClickHouse/ClickHouse/issues/32018#issuecomment-983595475
 2021-12-13 16:49:22 +01:00
Raúl Marín
61d959df8f Fix arrow build 2021-12-13 16:49:22 +01:00
Raúl Marín
d9e4544239 Adapt to arrow 6 2021-12-13 16:49:21 +01:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency 
Azure Blob Storage Disks
 2021-12-13 18:28:48 +03:00
Anton Popov
0b68980977 fix MATERIALIZE TTL with WHERE and GROUP BY 2021-12-13 17:21:05 +03:00
tavplubix
60fae3af76
Follow-up to #32140 (#32667) 
* Update LeaderElection.h

* Update LeaderElection.h
 2021-12-13 16:41:46 +03:00
Kruglov Pavel
52328f6abc
Merge pull request #32490 from ClickHouse/quote-more-identifiers 
Fix issue with ambiguous query formatting in distributed queries.
 2021-12-13 16:41:01 +03:00
Raúl Marín
589b5a1fdc Merge remote-tracking branch 'blessed/master' into wip_mv_into_join 2021-12-13 14:05:46 +01:00
Anton Popov
745fd4fc16 fix reading in order from Distributed and Merge tables 2021-12-13 15:35:25 +03:00
Anton Popov
a241103714
Merge pull request #32359 from CurtizJ/fix-non-materialized-skip-index 
Fix usage of non-materialized skip indexes
 2021-12-13 14:43:58 +03:00
mergify[bot]
fa8f9671f4
Merge branch 'master' into fix-async-inserts 2021-12-13 11:13:20 +00:00
Raúl Marín
b25cfa0b4d InterpreterInsertQuery::buildChain: Make sure to keep the context alive 2021-12-13 11:50:46 +01:00
Raúl Marín
c9e9bd78b2 Remove old unused file 2021-12-13 11:14:10 +01:00
vdimir
bda4ecbe90
Catch exceprion in ~ZlibDeflatingWriteBuffer 2021-12-13 12:29:35 +03:00
vdimir
7236313435
Reset pipeline in BlockIO::onFinish 2021-12-13 12:29:13 +03:00
mergify[bot]
77be64f1f4
Merge branch 'master' into genarateRandom 2021-12-13 09:18:52 +00:00
mergify[bot]
34417017e4
Merge branch 'master' into master 2021-12-13 09:18:49 +00:00
mergify[bot]
f5e2c38452
Merge branch 'master' into fix_groupBitmapAnd_function_distributed_table 2021-12-13 09:18:45 +00:00
Nikolai Kochetov
22e6fc1685
Merge pull request #32067 from amosbird/projection-fix23 
Fix detaching parts with projections
 2021-12-13 12:00:17 +03:00
nautaa
7cb3d24847 support Date32 for genarateRandom engine 2021-12-13 15:52:12 +08:00
mergify[bot]
47b18c959f
Merge branch 'master' into fix-non-materialized-skip-index 2021-12-13 07:03:02 +00:00
gulige
2f7edf82ab
avoid inconsistent state in Block 
In the member function insert(position, elem) to class Block, if function checkColumnStructure<void> throws an exception for columns with same name but unidentical structure, it will result in the mismatch between index_by_name and data. However we could not presume upon that caller will not use this block object anymore. So it seems better to guarantee consistency innerly.
 2021-12-13 13:53:12 +08:00
Alexey Milovidov
71926a3a97 Fix surprisingly bad code in function "file" 2021-12-13 07:57:54 +03:00
Vxider
f6dceb53a5 small update 2021-12-13 02:38:16 +00:00
Vxider
caad9f6c79 small update 2021-12-12 17:30:44 +00:00
mergify[bot]
dd4eaad2f3
Merge branch 'master' into watch-events-supports 2021-12-12 15:22:53 +00:00
tavplubix
be35ab171b
Merge pull request #32572 from azat/DROP-INSERT-UAF-fix 
Fix table lifetime in case of parallel DROP TABLE and INSERT
 2021-12-12 17:05:07 +03:00
alexey-milovidov
0486c5f341
Merge pull request #32220 from azat/mt-fix-minmax-for-empty-part 
[RFC] Fix minmax partition index during merge in case of empty part in future parts
 2021-12-12 14:45:10 +03:00
cmsxbc
e4b405716f fix multiply row policies on same column 2021-12-12 09:54:07 +00:00
mergify[bot]
72606749ab
Merge branch 'master' into watch-events-supports 2021-12-12 09:51:52 +00:00
Kseniia Sumarokova
5dc7d7e3ef
Merge pull request #32584 from kssenii/rabbitmq-fix 
rabbitmq fix
 2021-12-12 12:34:09 +03:00
alexey-milovidov
6779196c6b
Merge pull request #32587 from ClickHouse/reinterpret-perf 
Slight performance improvement of "reinterpret" function
 2021-12-12 12:14:43 +03:00
alexey-milovidov
09a6e826e7
Merge pull request #32591 from ClickHouse/fix-indecent-error-message 
Fix indecent error message
 2021-12-12 12:12:38 +03:00
Vxider
7eb18ea21e watch events support for window view 2021-12-12 07:45:55 +00:00
alexey-milovidov
26d606c158
Merge pull request #32456 from ClickHouse/fix-29890 
Fix arraySlice with null args.
 2021-12-12 06:38:17 +03:00
alexey-milovidov
71df622b1f
Merge pull request #31888 from ClickHouse/fix-31114 
Remove LowCardinality from window function args.
 2021-12-12 06:37:55 +03:00
Alexey Milovidov
2c68192bb6 Fix strange code in TCPHandler 2021-12-12 05:35:33 +03:00
Alexey Milovidov
453135eb63 Fix indecent error message 2021-12-12 05:24:47 +03:00
alexey-milovidov
151834d743
Merge pull request #32417 from azat/local-atomic-db-fix 
clickhouse-local: fix CREATE DATABASE with Atomic engine
 2021-12-12 04:33:59 +03:00
alexey-milovidov
33674cbe58
Merge pull request #32535 from azat/avg-ubsan 
Suppress UBSan errors for avg() function
 2021-12-12 04:32:44 +03:00
alexey-milovidov
b8a15b666d
Merge pull request #32571 from azat/mv-non-phys-table 
Fix LOGICAL_ERROR for MATERIALIZED VIEW over table functions (i.e. numbers())
 2021-12-12 03:55:51 +03:00
Alexey Milovidov
58ffe89643 Merge branch 'master' into quote-more-identifiers 2021-12-12 03:33:20 +03:00
Alexey Milovidov
3b0f026827 Slight performance improvement of "reinterpret" function 2021-12-12 03:29:26 +03:00
alexey-milovidov
eaf1d764a7
Merge pull request #32578 from bharatnc/ncb/fixes 
remove unused headers in test
 2021-12-12 01:05:12 +03:00
alexey-milovidov
a1cf07fdd3
Merge pull request #32580 from azat/remove-arcadia 
Remove arcadia build support
 2021-12-12 01:03:42 +03:00
kssenii
89db2c57f4 Fix 2021-12-12 00:19:06 +03:00
Vitaly Baranov
8f9b0b4ed0
Merge pull request #32575 from vitlibar/improve-quota-end-of-interval-calculation 
Improve quota's end-of-interval calculations.
 2021-12-11 22:16:04 +03:00
Azat Khuzhin
57c027be0e Remove arcadia build support 2021-12-11 21:25:23 +03:00
Vitaly Baranov
463ce1fcee
Merge pull request #27822 from filimonov/kafka_protobuf_issue26643 
Test for issue #26643
 2021-12-11 20:31:22 +03:00
alexey-milovidov
0bed774d83
Merge pull request #32574 from azat/remove-readline 
[RFC] Remove readline support
 2021-12-11 20:15:03 +03:00
bharatnc
a3dfb89927 remove unused headers in test 2021-12-11 08:30:22 -08:00
zhanglistar
2af221df18 1. update node size in set processor. 2021-12-11 22:24:59 +08:00
Vitaly Baranov
eea269a829 Improve quota's end-of-interval calculations. 2021-12-11 13:56:38 +03:00
Azat Khuzhin
4e4837758a Remove readline support 
- it was not nested for a long time
- replxx is an upstream way for completion
 2021-12-11 11:29:59 +03:00
Vitaly Baranov
abe9dd3368
Merge pull request #32531 from vitlibar/fix-nested-array-sizes-for-missing-columns 
Improve handling nested structures with missing columns while reading protobuf
 2021-12-11 11:08:34 +03:00
mergify[bot]
1e28f7340e
Merge branch 'master' into kafka_protobuf_issue26643 2021-12-11 08:05:20 +00:00
Azat Khuzhin
ca9c5dc4eb Fix LOGICAL_ERROR for MATERIALIZED VIEW over table functions (i.e. numbers()) 
Replace LOGICAL_ERROR with QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW
in this case.
 2021-12-11 11:04:47 +03:00
Azat Khuzhin
9a71598973 Fix table lifetime in case of parallel DROP TABLE and INSERT 
Stress tests founds [1]:

==527==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x37078ffd in unsigned long std::__1::__cxx_atomic_fetch_add<unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order) obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1050:12
    1 0x37078ffd in std::__1::__atomic_base<unsigned long, true>::fetch_add(unsigned long, std::__1::memory_order) obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1719:17
    2 0x37078ffd in std::__1::__atomic_base<unsigned long, true>::operator++() obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1756:57
    3 0x37078ffd in SimpleIncrement::get() obj-x86_64-linux-gnu/../src/Common/SimpleIncrement.h:20:16
    4 0x37078ffd in DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeDataWriter.cpp:276:46
    5 0x373c446c in DB::MergeTreeSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeSink.cpp:27:65

  Uninitialized value was created by a heap deallocation
    6 0x32d481e8 in DB::DatabaseCatalog::TableMarkedAsDropped::~TableMarkedAsDropped() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.h:248:12
    7 0x32d3c134 in DB::DatabaseCatalog::dropTableDataTask() obj-x86_64-linux-gnu/../src/Interpreters/DatabaseCatalog.cpp:908:1

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32534/fa6090f588dbf4cbb5f28bd2210847b070bb8218/stress_test__memory__actions_.html

The query was CREATE MATERIALIZED VIEW ... POPULATE AS SELECT ... from
00040_aggregating_materialized_view test.
 2021-12-11 11:00:31 +03:00
Vitaly Baranov
94e2e3625b
Merge pull request #31656 from sunny19930321/fix-use-quota-bug 
Fix use quota bug
 2021-12-11 11:00:23 +03:00
Kseniia Sumarokova
9ede6beca7
Merge pull request #32534 from azat/local-file-table-fix 
Fix processing initial table (--table/stdin) in clickhouse-local
 2021-12-11 10:45:19 +03:00
Kseniia Sumarokova
7e0bc290da
Merge pull request #32460 from kssenii/mongo-extend-auth 
Allow empty credentials for mongo engine
 2021-12-11 10:42:15 +03:00
alexey-milovidov
c90e588e24
Merge pull request #32506 from ClickHouse/fix-has-column-in-table-subquery 
Fix queries with hasColumnInTable constant condition and non existing column
 2021-12-11 02:50:06 +03:00
alexey-milovidov
17e5f5ccfe
Merge pull request #32508 from vdimir/issue_32458 
Handle const column in JoinCommon::removeColumnNullability
 2021-12-11 01:21:10 +03:00
Anton Popov
37837f3881
Merge pull request #32453 from kssenii/disable-optimizations-for-window-functions 
Disable some optimizations for window functions
 2021-12-11 00:37:24 +03:00
Jakub Kuklis
578aec97e9 Removing TODO and NOTE marks used mainly as hooks for review in the PR 2021-12-10 22:04:38 +01:00
Jakub Kuklis
a3bc6d2fb7 Style corrections 2021-12-10 22:04:38 +01:00
Jakub Kuklis
85807adec4 Fixing variable shadow error 2021-12-10 22:04:38 +01:00
Jakub Kuklis
74e26354cc Committing missing files 2021-12-10 22:04:38 +01:00
Jakub Kuklis
41a88e49c9 Abstracting metadata creation for remote disks registrations, putting Blob Storage auth in a separate file 2021-12-10 22:04:38 +01:00
Jakub Kuklis
e58f74d8dc Connecting to Azurite and invoking simple INSERTs works in the integration test 2021-12-10 22:04:38 +01:00
Jakub Kuklis
8afe69ff93 Comming back to StorageSharedKeyCredential for the integration test, get 400 Bad requests from Azurite when trying to create a container now 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f180a7ba5f Solved Invalid storage account error with connection strings 2021-12-10 22:04:38 +01:00
Jakub Kuklis
008141de36 Enable authentication via StorageSharedKeyCredential 2021-12-10 22:04:38 +01:00
Jakub Kuklis
d51f979c7a Adding endpoint validation 2021-12-10 22:04:38 +01:00
Jakub Kuklis
2b7725bfc7 Enable connecting directly to storage accounts and creating new containers 2021-12-10 22:04:38 +01:00
Jakub Kuklis
910db6ca57 Code cleanup addressing PR remarks 2021-12-10 22:04:38 +01:00
Jakub Kuklis
6acbec4455 Rebased on master 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f6c5a2c63c Removing file checks after changing blob filenames 2021-12-10 22:04:38 +01:00
Jakub Kuklis
3aa6a0ead3 Adding random postfix to blob names, needs code cleanup 2021-12-10 22:04:38 +01:00
Jakub Kuklis
5ed511f9da Working skeleton for Blob Storage integration test 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0c65cfed1f Addressing first PR remarks 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7c2e116214 Adding max_retries config 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a33cea0525 Refactors after proper rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3728dbbaf Rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b8ef47006c Minor refactors before PR 2021-12-10 22:04:37 +01:00
Ryad ZENINE
c4421d6866 Small changes following review 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7f595eedbc Catch Blob Storage exceptions 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3c4a92cc0 Code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c4691affb2 Adding a new test for read with offset 2021-12-10 22:04:37 +01:00
Jakub Kuklis
9a00acfb72 Fixing read size error plus minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fd8ebe0d9f Maximum read 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c9f3164cba Bringing back old single part upload limit 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0d7a4b10c3 Using unique_ptr for Blob Client in Read Buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
12924b3659 Minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fa2261dee6 Changing offset in ReadBuffer to size_t 2021-12-10 22:04:37 +01:00
Jakub Kuklis
5e2b2b1eee Better search for blobs with a given name 2021-12-10 22:04:37 +01:00
Jakub Kuklis
33247db3f4 Adding finalize() to WriteBuffer, always using block blob uploads, code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
bf2476a076 Shared pointer for Azure client and fixing path reference in ReadBuffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
60dd22dd4e Checking for success of file removal operation 2021-12-10 22:04:37 +01:00
Jakub Kuklis
486f267b3b Further code cleanup and error codes improvement 2021-12-10 22:04:37 +01:00
Jakub Kuklis
6755c5e897 File deletion prototype 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a0cfafa387 checkUniqueId implementation using ListBlobs 2021-12-10 22:04:37 +01:00
Jakub Kuklis
956348073b ReadBufferFromBlobStorage operating like a proper stream, rather than a single download-read buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
95816875f4 Change false to true in supportZeroCopyReplication 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b41ab197e8 Blob Storage endpoint url verification using re2 2021-12-10 22:04:37 +01:00
Jakub Kuklis
63ec8d4244 Adding multipart upload to WriteBuffer, retries in ReadBuffer, additional settings for buffers 2021-12-10 22:04:37 +01:00
Jakub Kuklis
02060791e3 Adding settings getter and applyNewSettings 2021-12-10 22:04:37 +01:00
Jakub Kuklis
3f28f2c62b Enabling local cache for DiskBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
e3daf8aa49 Increase default thread pool size 2021-12-10 22:04:37 +01:00
Jakub Kuklis
dbb98e3636 Code cleanup, leaving some debug information printing under #ifdef VERBOSE_DEBUG_MODE 2021-12-10 22:04:37 +01:00
Jakub Kuklis
8230f8039c Two column MergeTree table with Blob Storage disk work! Fixed buffer in ReadBufferFromBlobStorage to be permanent (class attribute) 2021-12-10 22:04:37 +01:00
Jakub Kuklis
2b4d3665ab Basic inserts and selects work for a MergeTree table with a Blob Disk, based on a single column table 2021-12-10 22:04:37 +01:00
Jakub Kuklis
4b51ddee1b Initial check with write and read works for the Blob Storage Disk 2021-12-10 22:04:37 +01:00
Jakub Kuklis
03f85dee41 Managed to read markers file in SELECT, getting error for CODEC version now 2021-12-10 22:04:37 +01:00
Jakub Kuklis
482a10e62e Uploading to Blob Storage on INSERT seems to do something reasonable 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c43b90c58c Progressing with WriteBufferFromBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fb4c1d6686 Starting insertion pipeline 2021-12-10 22:04:37 +01:00
Jakub Kuklis
6436f5b04d Adding an authenticated client to DiskBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
ee1a568919 The following query passes without erors: CREATE TABLE ... ENGINE = MergeTree ... storage_policy = 'blob_storage', debug constants present in the code 2021-12-10 22:04:37 +01:00
Jakub Kuklis
cacdac7ba6 Moving implementation to the .cpp file 2021-12-10 22:04:37 +01:00
Jakub Kuklis
88b5feb59d Registering the Blob Storage Disk 2021-12-10 22:04:36 +01:00