thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-04 05:22:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
56,397 Commits 9,182 Branches 1,782 Tags 2.3 GiB
3d522e5da9
Commit Graph

56397 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Amos Bird
d6c0479cf9
add normalizeQueryKeepNames and normalizedQueryHashKeepNames (#18910) 2021-01-26 17:51:30 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
alesapin
0ff05d2403 Fix deadlock in testkeeper 2021-01-26 17:30:29 +03:00
alexey-milovidov
985805fd01
Merge pull request #19583 from ClickHouse/merge-log-comment 
Merging #18549
 2021-01-26 17:11:07 +03:00
vdimir
bcae07128a
Merge pull request #19530 from taiyang-li/length_for_map 
enable length for map, which returns keys number in map
 2021-01-26 16:21:21 +03:00
vdimir
a4643bcff2
Whitespace fixes 2021-01-26 16:19:13 +03:00
Nikolai Kochetov
b0c324227d Remove comment. 2021-01-26 15:39:55 +03:00
feng lv
cea8ae162f fix test 2021-01-26 12:37:44 +00:00
Nikolai Kochetov
87246ea91b Fix header for MergeJoin with constants. 2021-01-26 15:37:42 +03:00
Azat Khuzhin
e97c01c3ea
Fix UAF of the CompressedWriteBuffer after Connection::disconnect (#19599) 
ASan report [1]:

<details>

Stacktrace with stripped shared_ptr and vector stuff:

```
==86==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0002b4888 at pc 0x00000a997056 bp 0x7f9e2ad55c00 sp 0x7f9e2ad55bf8
READ of size 8 at 0x60d0002b4888 thread T3 (TCPHandler)
    0 0xa997055 in DB::BufferBase::Buffer::end() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:40:46
    1 0xa997055 in DB::BufferBase::available() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:81:68
    2 0xa997055 in DB::BufferBase::hasPendingData() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:94:56
    3 0xa997055 in DB::WriteBuffer::nextIfAtEnd() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:67:14
    4 0xa997055 in DB::WriteBuffer::write(char const*, unsigned long) obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:78:13
    5 0x1dcff45e in DB::CompressedWriteBuffer::nextImpl() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:37:9
    6 0x1dcffb8a in DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:46:13
    7 0x1dcffb8a in DB::CompressedWriteBuffer::~CompressedWriteBuffer() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:54:9
    11 0xab600cf in DB::Connection::~Connection() obj-x86_64-linux-gnu/../src/Client/Connection.h:114:28
    15 0xac4adb9 in PoolBase<DB::Connection>::PooledObject::~PooledObject() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:35:12
    30 0xac485e4 in PoolBase<DB::Connection>::~PoolBase() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:105:26
    44 0xad2722f in DB::Cluster::ShardInfo::~ShardInfo() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:167:12
    52 0xad393b0 in DB::Cluster::~Cluster() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:30:7
    56 0x1f99f269 in DB::StorageDistributed::~StorageDistributed() obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:338:41
    69 0x1e231846 in DB::Context::~Context() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:501:19
    71 0x2073ccd3 in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:416:23
    72 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9

0x60d0002b4888 is located 56 bytes inside of 136-byte region [0x60d0002b4850,0x60d0002b48d8)
freed by thread T3 (TCPHandler) here:
    0 0xa93d682 in operator delete(void*, unsigned long) (/workspace/clickhouse+0xa93d682)
    1 0x2059d592 in std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518:9
    2 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19
    3 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::operator=(std::__1::shared_ptr<DB::WriteBuffer>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:5
    4 0x2059d592 in DB::Connection::disconnect() obj-x86_64-linux-gnu/../src/Client/Connection.cpp:143:9
    5 0x205d6e6d in DB::MultiplexedConnections::disconnect() obj-x86_64-linux-gnu/../src/Client/MultiplexedConnections.cpp:159:25
    6 0x1de6ec19 in DB::RemoteQueryExecutor::~RemoteQueryExecutor() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:86:34
    10 0x20bf0e2c in DB::RemoteSource::~RemoteSource() obj-x86_64-linux-gnu/../src/Processors/Sources/RemoteSource.cpp:22:29
    21 0x20869680 in DB::Pipe::~Pipe() obj-x86_64-linux-gnu/../src/Processors/Pipe.h:25:7
    22 0x20869680 in DB::QueryPipeline::reset() obj-x86_64-linux-gnu/../src/Processors/QueryPipeline.cpp:79:1
    23 0x1de2d89d in DB::BlockIO::reset() obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:45:14
    24 0x1de2d9f7 in DB::BlockIO::operator=(DB::BlockIO&&) obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:57:5
    25 0x20762731 in DB::QueryState::operator=(DB::QueryState&&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:31:8
    26 0x2073c70c in DB::QueryState::reset() obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:85:15
    27 0x2073c70c in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:399:19
    28 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
    29 0x266eeebe in Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
    30 0x266ef9db in Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:112:19
    31 0x269b1204 in Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
    32 0x269ab756 in Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
    33 0x7f9f06ea8608 in start_thread /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477:8

previously allocated by thread T3 (TCPHandler) here:
    0 0xa93ca1d in operator new(unsigned long) (/workspace/clickhouse+0xa93ca1d)
    1 0x2059b8cd in void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/new:235:10
    8 0x2059b8cd in DB::Connection::connect(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:112:15
    9 0x205a0a1d in DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:289:9
    10 0x205bdafa in DB::ConnectionPoolWithFailover::tryGetEntry(DB::IConnectionPool&, DB::ConnectionTimeouts const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::Settings const*, DB::QualifiedTableName const*) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:251:45
    11 0x205c06cf in DB::ConnectionPoolWithFailover::getManyChecked()::$_8::operator()() const obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:169:16
    20 0x205bd61f in DB::ConnectionPoolWithFailover::getManyChecked(DB::ConnectionTimeouts const&, DB::Settings const*, DB::PoolMode, DB::QualifiedTableName const&) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:172:12
    28 0x1de6f425 in DB::RemoteQueryExecutor::sendQuery() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:143:31
    29 0x1fdd1410 in DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:78:12
    30 0x1fdd74a8 in DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:131:32
    31 0x1d42cd79 in DB::TableFunctionRemote::getActualTableStructure(DB::Context const&) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:261:12
    32 0x1d42b4e2 in DB::TableFunctionRemote::executeImpl(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:222:26
    33 0x1e2c8b15 in DB::ITableFunction::execute(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/ITableFunction.cpp:24:16
    34 0x1e2417ff in DB::Context::executeTableFunction(std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1007:35
    35 0x1f15bd67 in DB::JoinedTables::getLeftTableStorage() obj-x86_64-linux-gnu/../src/Interpreters/JoinedTables.cpp:162:42
    36 0x1ebf393f in DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:306:33
    42 0x1eb3bf90 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterFactory.cpp:110:16
    43 0x1f4d9dee in DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:520:28
    44 0x1f4d7067 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:900:30
    45 0x2073b0bc in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:260:24
    46 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
```

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/19583/9f8ab99dd12a6f60a20f5f84ab2f5d53874c6ae7/fuzzer_asan/report.htmlfail1
 2021-01-26 14:27:58 +03:00
feng lv
35125c1b33 rewrite 2021-01-26 11:26:15 +00:00
bo zeng
c2d079d40e
fix issue #18086 (#18124) 
* fix issue #18086

* Update convertMySQLDataType.cpp

* add test case for type mapping for mysql

* remove type mapping test case from materialize_with_ddl.py, will add case in mysql engine.

* add test on types and values for mysql database engine.

* typo T_T

* add missing variables

* update the assert code to align with new test data.

* better code for testing, and remove null test since it is not supported.

* remove edge value from test since it always fail. should fix it in clickhouse-client.

* remove some testcases currently not support.

* correct test case

* correct test case. maybe I need to build up a test environment in my local pc...

* correct test data

* Update convertMySQLDataType.cpp

Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
 2021-01-26 13:37:57 +03:00
Nikolai Kochetov
d821cf11f8 Fix function neighbor for LowCardinality argument. 2021-01-26 13:31:29 +03:00
tavplubix
b8d6655131
Merge pull request #19572 from ClickHouse/fix_19568 
Fix DROP TABLE ON CLUSTER SYNC hang with Atomic database
 2021-01-26 12:51:07 +03:00
Maksim Kita
463a48dfe3
Merge pull request #19533 from kitaisreal/multi-if-constant-result-support 
Function multiIf constant result support
 2021-01-26 12:44:23 +03:00
Maksim Kita
16eb7b01b6
Merge pull request #19571 from kitaisreal/create-simple-dictionary-id-expression-fix 
Query CREATE DICTIONARY id expression fix
 2021-01-26 12:43:49 +03:00
tavplubix
27ddf78ba5
Merge pull request #19521 from adevyatova/annadevyatova-DOCSUP-5634-replace 
DOCSUP-5634: REPLACE TABLE
 2021-01-26 11:41:47 +03:00
Vladimir Chebotarev
68a671f009 Fixed documentation. 2021-01-26 11:33:21 +03:00
Vladimir Chebotarev
25037edfa6 Removed space in winking face smiley. 2021-01-26 11:28:47 +03:00
tavplubix
2d6a71fced
Merge pull request #19443 from ClickHouse/fix_virtual_parts_in_parts_to_do 
Addition to #15537
 2021-01-26 11:27:14 +03:00
tavplubix
3db7f48aa9
Merge pull request #19566 from ClickHouse/tavplubix-patch-3 
DateLUTImpl::addYears(...): suppress UBSan
 2021-01-26 11:23:41 +03:00
tavplubix
16561753e8
Update arcadia_skip_list.txt 2021-01-26 11:21:18 +03:00
alesapin
99fee8e117
Merge pull request #19598 from BohuTANG/20210126-remove-unused-header 
Remove unused include header
 2021-01-26 10:48:02 +03:00
Maksim Kita
869296f76e
Merge pull request #19601 from kitaisreal/date-time-function-zh-documentation-fix 
DateTime functions zh documentation fix
 2021-01-26 10:11:39 +03:00
Maksim Kita
aa88ac65dd DateTime functions zh documentation fix 2021-01-26 10:09:22 +03:00
BohuTANG
368121cc04 Remove unused include header 2021-01-26 14:10:33 +08:00
Azat Khuzhin
27a5794795 Fix UBsan report in lcm()/gcd() 
Changelog:
- Check against INT64_MIN/INT64_MAX
- Check against std::numeric_limits<>::min/max
- Move common code into separate header - GCDLCMImpl.h
- Forbid floats

UBsan report [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/19466/cb30a02540a0f223df6668c5f88ff84aa666ff54/fuzzer_ubsan/report.html#fail1
 2021-01-26 08:33:49 +03:00
bharatnc
31027dbaf7 update docs 2021-01-25 19:04:08 -08:00
Amos Bird
5998383a3b
Fix with aggregates 2021-01-26 10:29:13 +08:00
Amos Bird
0b74d146ef
Fix unused with column 2021-01-26 10:29:13 +08:00
Amos Bird
3ca0bdc848
Keep with statement 2021-01-26 10:29:13 +08:00
Amos Bird
a2ceca3fa1
Fix recursive propagation 2021-01-26 10:29:12 +08:00
Amos Bird
9e28fd4a8e
make clang happy 2021-01-26 10:29:12 +08:00
Amos Bird
66d84957e2
Get rid of useless with statement during analysis 2021-01-26 10:29:12 +08:00
Amos Bird
9546e61bd4
Normalize away WITH statement 2021-01-26 10:29:12 +08:00
Amos Bird
b6a5d2f8e9
Fix non-select queries 2021-01-26 10:29:12 +08:00
Amos Bird
3afcccaad2
Fix global with after union refactoring 2021-01-26 10:29:12 +08:00
Amos Bird
824aee4020
better cte and with global 2021-01-26 10:29:11 +08:00
bharatnc
db8975c38f remove redundant strcpy call 2021-01-25 17:38:55 -08:00
bharatnc
fc8e22075e refactor code to not use strings 2021-01-25 17:38:55 -08:00
bharatnc
a102c78328 tests for toIPv6 and IPv6StringToNum 2021-01-25 17:38:55 -08:00
bharatnc
7abedaeaa0 simplify logic flow 2021-01-25 17:38:55 -08:00
bharatnc
562d01e8d8 fix code and rebase 2021-01-25 17:38:55 -08:00
bharatnc
9152a7a4be init - make toIPv6 parse IPv4 addresses 2021-01-25 17:38:55 -08:00
Azat Khuzhin
f4a4d33c2d Suppress UBsan reports in addYears() 
https://clickhouse-test-reports.s3.yandex.net/19528/1c364b6ee3a44487529954cf2792e615f7de0f82/fuzzer_ubsan/report.html#fail1
 2021-01-26 01:12:40 +03:00
Alexey Milovidov
b2607a358d Applied a patch from Alexei Fedotov @leshikus 2021-01-26 00:28:43 +03:00
alexey-milovidov
8084e4d614
Merge pull request #19414 from ClickHouse/fix-15889-continuation 
Continue fix for Block structure mismatch in PipelineExecuting stream
 2021-01-26 00:05:35 +03:00
alexey-milovidov
7197552535
Merge pull request #19430 from ClickHouse/fix-14100 
Fix error Extremes transform was already added to pipeline
 2021-01-26 00:04:14 +03:00
Anna
c8b6774a75 correction 2021-01-25 23:48:54 +03:00
Alexey Milovidov
8dfa933028 Amend 2021-01-25 23:48:10 +03:00