thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 16:42:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
56,031 Commits 9,168 Branches 1,781 Tags 2.1 GiB
3e8d76da38
Commit Graph

56031 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Azat Khuzhin
3e8d76da38 Add pylintrc config 2021-01-26 23:35:56 +03:00
Azat Khuzhin
c4454ab095 Fix colored() usage in clickhouse-test 2021-01-26 23:23:00 +03:00
alexey-milovidov
e8dedbf210
Merge pull request #19532 from azat/UBsan-report-fixes-v2 
Forbid lcm/gcd for floats (and few UBsan reports fixes)
 2021-01-26 23:16:54 +03:00
alexey-milovidov
2892aa39c0
Merge pull request #19590 from ClickHouse/cassandra-patch-leshikus2 
Update Cassandra submodule with patch by @leshikus
 2021-01-26 21:27:54 +03:00
vdimir
fc8b8d23b2
Merge pull request #18812 from excitoon-favorites/betters3endpoints 
Added prefix-based S3 endpoint settings
 2021-01-26 20:47:57 +03:00
Nikolai Kochetov
1bc8c32e9f
Merge pull request #19209 from sevirov/sevirov-DOCSUP-4918-update_description_final 
DOCSUP-4918: Update the description of SELECT...FINAL
 2021-01-26 20:00:23 +03:00
Dmitriy
9f4e51d36b Merge remote-tracking branch 'remotes/clickhouse/master' into sevirov-DOCSUP-4918-update_description_final 2021-01-26 19:35:41 +03:00
Dmitriy
6a99955610 Update the description of the max_final_threads setting 
Поправил формулировку назначения настройки max_final_threads.
 2021-01-26 19:32:34 +03:00
Amos Bird
d6c0479cf9
add normalizeQueryKeepNames and normalizedQueryHashKeepNames (#18910) 2021-01-26 17:51:30 +03:00
alexey-milovidov
985805fd01
Merge pull request #19583 from ClickHouse/merge-log-comment 
Merging #18549
 2021-01-26 17:11:07 +03:00
vdimir
bcae07128a
Merge pull request #19530 from taiyang-li/length_for_map 
enable length for map, which returns keys number in map
 2021-01-26 16:21:21 +03:00
vdimir
a4643bcff2
Whitespace fixes 2021-01-26 16:19:13 +03:00
Azat Khuzhin
e97c01c3ea
Fix UAF of the CompressedWriteBuffer after Connection::disconnect (#19599) 
ASan report [1]:

<details>

Stacktrace with stripped shared_ptr and vector stuff:

```
==86==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0002b4888 at pc 0x00000a997056 bp 0x7f9e2ad55c00 sp 0x7f9e2ad55bf8
READ of size 8 at 0x60d0002b4888 thread T3 (TCPHandler)
    0 0xa997055 in DB::BufferBase::Buffer::end() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:40:46
    1 0xa997055 in DB::BufferBase::available() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:81:68
    2 0xa997055 in DB::BufferBase::hasPendingData() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:94:56
    3 0xa997055 in DB::WriteBuffer::nextIfAtEnd() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:67:14
    4 0xa997055 in DB::WriteBuffer::write(char const*, unsigned long) obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:78:13
    5 0x1dcff45e in DB::CompressedWriteBuffer::nextImpl() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:37:9
    6 0x1dcffb8a in DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:46:13
    7 0x1dcffb8a in DB::CompressedWriteBuffer::~CompressedWriteBuffer() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:54:9
    11 0xab600cf in DB::Connection::~Connection() obj-x86_64-linux-gnu/../src/Client/Connection.h:114:28
    15 0xac4adb9 in PoolBase<DB::Connection>::PooledObject::~PooledObject() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:35:12
    30 0xac485e4 in PoolBase<DB::Connection>::~PoolBase() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:105:26
    44 0xad2722f in DB::Cluster::ShardInfo::~ShardInfo() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:167:12
    52 0xad393b0 in DB::Cluster::~Cluster() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:30:7
    56 0x1f99f269 in DB::StorageDistributed::~StorageDistributed() obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:338:41
    69 0x1e231846 in DB::Context::~Context() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:501:19
    71 0x2073ccd3 in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:416:23
    72 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9

0x60d0002b4888 is located 56 bytes inside of 136-byte region [0x60d0002b4850,0x60d0002b48d8)
freed by thread T3 (TCPHandler) here:
    0 0xa93d682 in operator delete(void*, unsigned long) (/workspace/clickhouse+0xa93d682)
    1 0x2059d592 in std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518:9
    2 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19
    3 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::operator=(std::__1::shared_ptr<DB::WriteBuffer>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:5
    4 0x2059d592 in DB::Connection::disconnect() obj-x86_64-linux-gnu/../src/Client/Connection.cpp:143:9
    5 0x205d6e6d in DB::MultiplexedConnections::disconnect() obj-x86_64-linux-gnu/../src/Client/MultiplexedConnections.cpp:159:25
    6 0x1de6ec19 in DB::RemoteQueryExecutor::~RemoteQueryExecutor() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:86:34
    10 0x20bf0e2c in DB::RemoteSource::~RemoteSource() obj-x86_64-linux-gnu/../src/Processors/Sources/RemoteSource.cpp:22:29
    21 0x20869680 in DB::Pipe::~Pipe() obj-x86_64-linux-gnu/../src/Processors/Pipe.h:25:7
    22 0x20869680 in DB::QueryPipeline::reset() obj-x86_64-linux-gnu/../src/Processors/QueryPipeline.cpp:79:1
    23 0x1de2d89d in DB::BlockIO::reset() obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:45:14
    24 0x1de2d9f7 in DB::BlockIO::operator=(DB::BlockIO&&) obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:57:5
    25 0x20762731 in DB::QueryState::operator=(DB::QueryState&&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:31:8
    26 0x2073c70c in DB::QueryState::reset() obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:85:15
    27 0x2073c70c in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:399:19
    28 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
    29 0x266eeebe in Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
    30 0x266ef9db in Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:112:19
    31 0x269b1204 in Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
    32 0x269ab756 in Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
    33 0x7f9f06ea8608 in start_thread /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477:8

previously allocated by thread T3 (TCPHandler) here:
    0 0xa93ca1d in operator new(unsigned long) (/workspace/clickhouse+0xa93ca1d)
    1 0x2059b8cd in void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/new:235:10
    8 0x2059b8cd in DB::Connection::connect(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:112:15
    9 0x205a0a1d in DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:289:9
    10 0x205bdafa in DB::ConnectionPoolWithFailover::tryGetEntry(DB::IConnectionPool&, DB::ConnectionTimeouts const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::Settings const*, DB::QualifiedTableName const*) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:251:45
    11 0x205c06cf in DB::ConnectionPoolWithFailover::getManyChecked()::$_8::operator()() const obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:169:16
    20 0x205bd61f in DB::ConnectionPoolWithFailover::getManyChecked(DB::ConnectionTimeouts const&, DB::Settings const*, DB::PoolMode, DB::QualifiedTableName const&) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:172:12
    28 0x1de6f425 in DB::RemoteQueryExecutor::sendQuery() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:143:31
    29 0x1fdd1410 in DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:78:12
    30 0x1fdd74a8 in DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:131:32
    31 0x1d42cd79 in DB::TableFunctionRemote::getActualTableStructure(DB::Context const&) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:261:12
    32 0x1d42b4e2 in DB::TableFunctionRemote::executeImpl(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:222:26
    33 0x1e2c8b15 in DB::ITableFunction::execute(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/ITableFunction.cpp:24:16
    34 0x1e2417ff in DB::Context::executeTableFunction(std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1007:35
    35 0x1f15bd67 in DB::JoinedTables::getLeftTableStorage() obj-x86_64-linux-gnu/../src/Interpreters/JoinedTables.cpp:162:42
    36 0x1ebf393f in DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:306:33
    42 0x1eb3bf90 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterFactory.cpp:110:16
    43 0x1f4d9dee in DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:520:28
    44 0x1f4d7067 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:900:30
    45 0x2073b0bc in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:260:24
    46 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
```

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/19583/9f8ab99dd12a6f60a20f5f84ab2f5d53874c6ae7/fuzzer_asan/report.htmlfail1
 2021-01-26 14:27:58 +03:00
bo zeng
c2d079d40e
fix issue #18086 (#18124) 
* fix issue #18086

* Update convertMySQLDataType.cpp

* add test case for type mapping for mysql

* remove type mapping test case from materialize_with_ddl.py, will add case in mysql engine.

* add test on types and values for mysql database engine.

* typo T_T

* add missing variables

* update the assert code to align with new test data.

* better code for testing, and remove null test since it is not supported.

* remove edge value from test since it always fail. should fix it in clickhouse-client.

* remove some testcases currently not support.

* correct test case

* correct test case. maybe I need to build up a test environment in my local pc...

* correct test data

* Update convertMySQLDataType.cpp

Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
 2021-01-26 13:37:57 +03:00
tavplubix
b8d6655131
Merge pull request #19572 from ClickHouse/fix_19568 
Fix DROP TABLE ON CLUSTER SYNC hang with Atomic database
 2021-01-26 12:51:07 +03:00
Maksim Kita
463a48dfe3
Merge pull request #19533 from kitaisreal/multi-if-constant-result-support 
Function multiIf constant result support
 2021-01-26 12:44:23 +03:00
Maksim Kita
16eb7b01b6
Merge pull request #19571 from kitaisreal/create-simple-dictionary-id-expression-fix 
Query CREATE DICTIONARY id expression fix
 2021-01-26 12:43:49 +03:00
tavplubix
27ddf78ba5
Merge pull request #19521 from adevyatova/annadevyatova-DOCSUP-5634-replace 
DOCSUP-5634: REPLACE TABLE
 2021-01-26 11:41:47 +03:00
Vladimir Chebotarev
68a671f009 Fixed documentation. 2021-01-26 11:33:21 +03:00
Vladimir Chebotarev
25037edfa6 Removed space in winking face smiley. 2021-01-26 11:28:47 +03:00
tavplubix
2d6a71fced
Merge pull request #19443 from ClickHouse/fix_virtual_parts_in_parts_to_do 
Addition to #15537
 2021-01-26 11:27:14 +03:00
tavplubix
3db7f48aa9
Merge pull request #19566 from ClickHouse/tavplubix-patch-3 
DateLUTImpl::addYears(...): suppress UBSan
 2021-01-26 11:23:41 +03:00
tavplubix
16561753e8
Update arcadia_skip_list.txt 2021-01-26 11:21:18 +03:00
alesapin
99fee8e117
Merge pull request #19598 from BohuTANG/20210126-remove-unused-header 
Remove unused include header
 2021-01-26 10:48:02 +03:00
Maksim Kita
869296f76e
Merge pull request #19601 from kitaisreal/date-time-function-zh-documentation-fix 
DateTime functions zh documentation fix
 2021-01-26 10:11:39 +03:00
Maksim Kita
aa88ac65dd DateTime functions zh documentation fix 2021-01-26 10:09:22 +03:00
BohuTANG
368121cc04 Remove unused include header 2021-01-26 14:10:33 +08:00
Azat Khuzhin
27a5794795 Fix UBsan report in lcm()/gcd() 
Changelog:
- Check against INT64_MIN/INT64_MAX
- Check against std::numeric_limits<>::min/max
- Move common code into separate header - GCDLCMImpl.h
- Forbid floats

UBsan report [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/19466/cb30a02540a0f223df6668c5f88ff84aa666ff54/fuzzer_ubsan/report.html#fail1
 2021-01-26 08:33:49 +03:00
Azat Khuzhin
f4a4d33c2d Suppress UBsan reports in addYears() 
https://clickhouse-test-reports.s3.yandex.net/19528/1c364b6ee3a44487529954cf2792e615f7de0f82/fuzzer_ubsan/report.html#fail1
 2021-01-26 01:12:40 +03:00
Alexey Milovidov
b2607a358d Applied a patch from Alexei Fedotov @leshikus 2021-01-26 00:28:43 +03:00
alexey-milovidov
8084e4d614
Merge pull request #19414 from ClickHouse/fix-15889-continuation 
Continue fix for Block structure mismatch in PipelineExecuting stream
 2021-01-26 00:05:35 +03:00
alexey-milovidov
7197552535
Merge pull request #19430 from ClickHouse/fix-14100 
Fix error Extremes transform was already added to pipeline
 2021-01-26 00:04:14 +03:00
Anna
c8b6774a75 correction 2021-01-25 23:48:54 +03:00
Anna
7dedfba800 fix link 2021-01-25 23:32:05 +03:00
Dmitriy
d121eb4787 Update the description of the SELECT queries with FINAL 
Объяснил преимущество параллельности SELECT запросов с модификатором FINAL..
 2021-01-25 23:31:06 +03:00
Anna
2e317b22b7 add symlinks 2021-01-25 23:22:34 +03:00
Anna
6634432253 Correction 2021-01-25 23:17:08 +03:00
Alexey Milovidov
9f8ab99dd1 Use log_comment in clickhouse-test 2021-01-25 22:29:29 +03:00
Alexey Milovidov
a47b0190fd Add a test 2021-01-25 22:25:49 +03:00
Alexey Milovidov
cfd47b2980 Merge branch 'master' into issue18494 2021-01-25 22:18:23 +03:00
Anna
6137e5530f Update examples 2021-01-25 22:17:46 +03:00
alexey-milovidov
405e9c8f4b
Merge pull request #19525 from olgarev/revolg-DOCSUP-4970-Support_for_MySQL_style_global_variables_syntax 
DOCSUP-4970: Docs for global MySQL variables support
 2021-01-25 22:17:44 +03:00
Maksim Kita
ce9a6d32eb Updated tests with always false condition 2021-01-25 21:53:47 +03:00
Anna
cec83bb76a Update 2021-01-25 21:49:13 +03:00
Maksim Kita
f70a704e2c Added test 2021-01-25 21:22:27 +03:00
Anton Popov
658f24dcff
Merge pull request #19358 from CurtizJ/fix-subcolumns 
Fix several cases, while reading subcolumns
 2021-01-25 20:26:07 +03:00
Anna
2a66eab876
Merge branch 'master' into annadevyatova-DOCSUP-5634-replace 2021-01-25 20:10:42 +03:00
Nikolai Kochetov
a6f787faea
Merge pull request #19253 from ClickHouse/split-actions-dag 
Split actions dag
 2021-01-25 18:13:42 +03:00
alesapin
7d4bf6d56c
Merge pull request #19551 from ClickHouse/fix_01459_test 
Fix 01459_manual_write_to_replicas flaky test
 2021-01-25 18:05:20 +03:00
alesapin
b3726c1c8e
Merge pull request #19558 from ClickHouse/better_hung_check 
Check for hung queries or server hung in fast test
 2021-01-25 18:04:07 +03:00