thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-24 18:50:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
56,038 Commits 8,945 Branches 1,766 Tags 2 GiB
3562250594
Commit Graph

56038 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexey Milovidov
3562250594 Remove outdated index_granularity from docs 2021-01-27 03:58:48 +03:00
Maksim Kita
46be4d12ac
Merge pull request #19663 from ucasFL/fix-build 
fix special build on clang11
 2021-01-27 00:00:31 +03:00
Maksim Kita
81548e051e
Merge pull request #19527 from ucasFL/sign 
add sign math function
 2021-01-26 23:57:31 +03:00
alexey-milovidov
e8dedbf210
Merge pull request #19532 from azat/UBsan-report-fixes-v2 
Forbid lcm/gcd for floats (and few UBsan reports fixes)
 2021-01-26 23:16:54 +03:00
alexey-milovidov
2892aa39c0
Merge pull request #19590 from ClickHouse/cassandra-patch-leshikus2 
Update Cassandra submodule with patch by @leshikus
 2021-01-26 21:27:54 +03:00
vdimir
fc8b8d23b2
Merge pull request #18812 from excitoon-favorites/betters3endpoints 
Added prefix-based S3 endpoint settings
 2021-01-26 20:47:57 +03:00
Nikolai Kochetov
1bc8c32e9f
Merge pull request #19209 from sevirov/sevirov-DOCSUP-4918-update_description_final 
DOCSUP-4918: Update the description of SELECT...FINAL
 2021-01-26 20:00:23 +03:00
Dmitriy
9f4e51d36b Merge remote-tracking branch 'remotes/clickhouse/master' into sevirov-DOCSUP-4918-update_description_final 2021-01-26 19:35:41 +03:00
Dmitriy
6a99955610 Update the description of the max_final_threads setting 
Поправил формулировку назначения настройки max_final_threads.
 2021-01-26 19:32:34 +03:00
feng lv
2dd723e214 fix special build on clang11 2021-01-26 15:20:51 +00:00
Amos Bird
d6c0479cf9
add normalizeQueryKeepNames and normalizedQueryHashKeepNames (#18910) 2021-01-26 17:51:30 +03:00
alexey-milovidov
985805fd01
Merge pull request #19583 from ClickHouse/merge-log-comment 
Merging #18549
 2021-01-26 17:11:07 +03:00
vdimir
bcae07128a
Merge pull request #19530 from taiyang-li/length_for_map 
enable length for map, which returns keys number in map
 2021-01-26 16:21:21 +03:00
vdimir
a4643bcff2
Whitespace fixes 2021-01-26 16:19:13 +03:00
feng lv
cea8ae162f fix test 2021-01-26 12:37:44 +00:00
Azat Khuzhin
e97c01c3ea
Fix UAF of the CompressedWriteBuffer after Connection::disconnect (#19599) 
ASan report [1]:

<details>

Stacktrace with stripped shared_ptr and vector stuff:

```
==86==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0002b4888 at pc 0x00000a997056 bp 0x7f9e2ad55c00 sp 0x7f9e2ad55bf8
READ of size 8 at 0x60d0002b4888 thread T3 (TCPHandler)
    0 0xa997055 in DB::BufferBase::Buffer::end() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:40:46
    1 0xa997055 in DB::BufferBase::available() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:81:68
    2 0xa997055 in DB::BufferBase::hasPendingData() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:94:56
    3 0xa997055 in DB::WriteBuffer::nextIfAtEnd() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:67:14
    4 0xa997055 in DB::WriteBuffer::write(char const*, unsigned long) obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:78:13
    5 0x1dcff45e in DB::CompressedWriteBuffer::nextImpl() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:37:9
    6 0x1dcffb8a in DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:46:13
    7 0x1dcffb8a in DB::CompressedWriteBuffer::~CompressedWriteBuffer() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:54:9
    11 0xab600cf in DB::Connection::~Connection() obj-x86_64-linux-gnu/../src/Client/Connection.h:114:28
    15 0xac4adb9 in PoolBase<DB::Connection>::PooledObject::~PooledObject() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:35:12
    30 0xac485e4 in PoolBase<DB::Connection>::~PoolBase() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:105:26
    44 0xad2722f in DB::Cluster::ShardInfo::~ShardInfo() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:167:12
    52 0xad393b0 in DB::Cluster::~Cluster() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:30:7
    56 0x1f99f269 in DB::StorageDistributed::~StorageDistributed() obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:338:41
    69 0x1e231846 in DB::Context::~Context() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:501:19
    71 0x2073ccd3 in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:416:23
    72 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9

0x60d0002b4888 is located 56 bytes inside of 136-byte region [0x60d0002b4850,0x60d0002b48d8)
freed by thread T3 (TCPHandler) here:
    0 0xa93d682 in operator delete(void*, unsigned long) (/workspace/clickhouse+0xa93d682)
    1 0x2059d592 in std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518:9
    2 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19
    3 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::operator=(std::__1::shared_ptr<DB::WriteBuffer>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:5
    4 0x2059d592 in DB::Connection::disconnect() obj-x86_64-linux-gnu/../src/Client/Connection.cpp:143:9
    5 0x205d6e6d in DB::MultiplexedConnections::disconnect() obj-x86_64-linux-gnu/../src/Client/MultiplexedConnections.cpp:159:25
    6 0x1de6ec19 in DB::RemoteQueryExecutor::~RemoteQueryExecutor() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:86:34
    10 0x20bf0e2c in DB::RemoteSource::~RemoteSource() obj-x86_64-linux-gnu/../src/Processors/Sources/RemoteSource.cpp:22:29
    21 0x20869680 in DB::Pipe::~Pipe() obj-x86_64-linux-gnu/../src/Processors/Pipe.h:25:7
    22 0x20869680 in DB::QueryPipeline::reset() obj-x86_64-linux-gnu/../src/Processors/QueryPipeline.cpp:79:1
    23 0x1de2d89d in DB::BlockIO::reset() obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:45:14
    24 0x1de2d9f7 in DB::BlockIO::operator=(DB::BlockIO&&) obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:57:5
    25 0x20762731 in DB::QueryState::operator=(DB::QueryState&&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:31:8
    26 0x2073c70c in DB::QueryState::reset() obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:85:15
    27 0x2073c70c in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:399:19
    28 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
    29 0x266eeebe in Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
    30 0x266ef9db in Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:112:19
    31 0x269b1204 in Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
    32 0x269ab756 in Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
    33 0x7f9f06ea8608 in start_thread /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477:8

previously allocated by thread T3 (TCPHandler) here:
    0 0xa93ca1d in operator new(unsigned long) (/workspace/clickhouse+0xa93ca1d)
    1 0x2059b8cd in void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/new:235:10
    8 0x2059b8cd in DB::Connection::connect(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:112:15
    9 0x205a0a1d in DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:289:9
    10 0x205bdafa in DB::ConnectionPoolWithFailover::tryGetEntry(DB::IConnectionPool&, DB::ConnectionTimeouts const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::Settings const*, DB::QualifiedTableName const*) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:251:45
    11 0x205c06cf in DB::ConnectionPoolWithFailover::getManyChecked()::$_8::operator()() const obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:169:16
    20 0x205bd61f in DB::ConnectionPoolWithFailover::getManyChecked(DB::ConnectionTimeouts const&, DB::Settings const*, DB::PoolMode, DB::QualifiedTableName const&) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:172:12
    28 0x1de6f425 in DB::RemoteQueryExecutor::sendQuery() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:143:31
    29 0x1fdd1410 in DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:78:12
    30 0x1fdd74a8 in DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:131:32
    31 0x1d42cd79 in DB::TableFunctionRemote::getActualTableStructure(DB::Context const&) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:261:12
    32 0x1d42b4e2 in DB::TableFunctionRemote::executeImpl(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:222:26
    33 0x1e2c8b15 in DB::ITableFunction::execute(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/ITableFunction.cpp:24:16
    34 0x1e2417ff in DB::Context::executeTableFunction(std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1007:35
    35 0x1f15bd67 in DB::JoinedTables::getLeftTableStorage() obj-x86_64-linux-gnu/../src/Interpreters/JoinedTables.cpp:162:42
    36 0x1ebf393f in DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:306:33
    42 0x1eb3bf90 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterFactory.cpp:110:16
    43 0x1f4d9dee in DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:520:28
    44 0x1f4d7067 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:900:30
    45 0x2073b0bc in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:260:24
    46 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
```

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/19583/9f8ab99dd12a6f60a20f5f84ab2f5d53874c6ae7/fuzzer_asan/report.htmlfail1
 2021-01-26 14:27:58 +03:00
feng lv
35125c1b33 rewrite 2021-01-26 11:26:15 +00:00
bo zeng
c2d079d40e
fix issue #18086 (#18124) 
* fix issue #18086

* Update convertMySQLDataType.cpp

* add test case for type mapping for mysql

* remove type mapping test case from materialize_with_ddl.py, will add case in mysql engine.

* add test on types and values for mysql database engine.

* typo T_T

* add missing variables

* update the assert code to align with new test data.

* better code for testing, and remove null test since it is not supported.

* remove edge value from test since it always fail. should fix it in clickhouse-client.

* remove some testcases currently not support.

* correct test case

* correct test case. maybe I need to build up a test environment in my local pc...

* correct test data

* Update convertMySQLDataType.cpp

Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
 2021-01-26 13:37:57 +03:00
tavplubix
b8d6655131
Merge pull request #19572 from ClickHouse/fix_19568 
Fix DROP TABLE ON CLUSTER SYNC hang with Atomic database
 2021-01-26 12:51:07 +03:00
Maksim Kita
463a48dfe3
Merge pull request #19533 from kitaisreal/multi-if-constant-result-support 
Function multiIf constant result support
 2021-01-26 12:44:23 +03:00
Maksim Kita
16eb7b01b6
Merge pull request #19571 from kitaisreal/create-simple-dictionary-id-expression-fix 
Query CREATE DICTIONARY id expression fix
 2021-01-26 12:43:49 +03:00
tavplubix
27ddf78ba5
Merge pull request #19521 from adevyatova/annadevyatova-DOCSUP-5634-replace 
DOCSUP-5634: REPLACE TABLE
 2021-01-26 11:41:47 +03:00
Vladimir Chebotarev
68a671f009 Fixed documentation. 2021-01-26 11:33:21 +03:00
Vladimir Chebotarev
25037edfa6 Removed space in winking face smiley. 2021-01-26 11:28:47 +03:00
tavplubix
2d6a71fced
Merge pull request #19443 from ClickHouse/fix_virtual_parts_in_parts_to_do 
Addition to #15537
 2021-01-26 11:27:14 +03:00
tavplubix
3db7f48aa9
Merge pull request #19566 from ClickHouse/tavplubix-patch-3 
DateLUTImpl::addYears(...): suppress UBSan
 2021-01-26 11:23:41 +03:00
tavplubix
16561753e8
Update arcadia_skip_list.txt 2021-01-26 11:21:18 +03:00
alesapin
99fee8e117
Merge pull request #19598 from BohuTANG/20210126-remove-unused-header 
Remove unused include header
 2021-01-26 10:48:02 +03:00
Maksim Kita
869296f76e
Merge pull request #19601 from kitaisreal/date-time-function-zh-documentation-fix 
DateTime functions zh documentation fix
 2021-01-26 10:11:39 +03:00
Maksim Kita
aa88ac65dd DateTime functions zh documentation fix 2021-01-26 10:09:22 +03:00
BohuTANG
368121cc04 Remove unused include header 2021-01-26 14:10:33 +08:00
Azat Khuzhin
27a5794795 Fix UBsan report in lcm()/gcd() 
Changelog:
- Check against INT64_MIN/INT64_MAX
- Check against std::numeric_limits<>::min/max
- Move common code into separate header - GCDLCMImpl.h
- Forbid floats

UBsan report [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/19466/cb30a02540a0f223df6668c5f88ff84aa666ff54/fuzzer_ubsan/report.html#fail1
 2021-01-26 08:33:49 +03:00
Azat Khuzhin
f4a4d33c2d Suppress UBsan reports in addYears() 
https://clickhouse-test-reports.s3.yandex.net/19528/1c364b6ee3a44487529954cf2792e615f7de0f82/fuzzer_ubsan/report.html#fail1
 2021-01-26 01:12:40 +03:00
Alexey Milovidov
b2607a358d Applied a patch from Alexei Fedotov @leshikus 2021-01-26 00:28:43 +03:00
alexey-milovidov
8084e4d614
Merge pull request #19414 from ClickHouse/fix-15889-continuation 
Continue fix for Block structure mismatch in PipelineExecuting stream
 2021-01-26 00:05:35 +03:00
alexey-milovidov
7197552535
Merge pull request #19430 from ClickHouse/fix-14100 
Fix error Extremes transform was already added to pipeline
 2021-01-26 00:04:14 +03:00
Anna
c8b6774a75 correction 2021-01-25 23:48:54 +03:00
Anna
7dedfba800 fix link 2021-01-25 23:32:05 +03:00
Dmitriy
d121eb4787 Update the description of the SELECT queries with FINAL 
Объяснил преимущество параллельности SELECT запросов с модификатором FINAL..
 2021-01-25 23:31:06 +03:00
Anna
2e317b22b7 add symlinks 2021-01-25 23:22:34 +03:00
Anna
6634432253 Correction 2021-01-25 23:17:08 +03:00
Alexey Milovidov
9f8ab99dd1 Use log_comment in clickhouse-test 2021-01-25 22:29:29 +03:00
Alexey Milovidov
a47b0190fd Add a test 2021-01-25 22:25:49 +03:00
Alexey Milovidov
cfd47b2980 Merge branch 'master' into issue18494 2021-01-25 22:18:23 +03:00
Anna
6137e5530f Update examples 2021-01-25 22:17:46 +03:00
alexey-milovidov
405e9c8f4b
Merge pull request #19525 from olgarev/revolg-DOCSUP-4970-Support_for_MySQL_style_global_variables_syntax 
DOCSUP-4970: Docs for global MySQL variables support
 2021-01-25 22:17:44 +03:00
Maksim Kita
ce9a6d32eb Updated tests with always false condition 2021-01-25 21:53:47 +03:00
Anna
cec83bb76a Update 2021-01-25 21:49:13 +03:00
Maksim Kita
f70a704e2c Added test 2021-01-25 21:22:27 +03:00
Anton Popov
658f24dcff
Merge pull request #19358 from CurtizJ/fix-subcolumns 
Fix several cases, while reading subcolumns
 2021-01-25 20:26:07 +03:00