Commit Graph

71281 Commits

Author SHA1 Message Date
Nikita Mikhaylov
ab22fdaa80 Fix build 2021-08-25 11:17:45 +00:00
Nikolay Degterinsky
bfdfe4dca4
Merge pull request #28055 from yingfc/master
Fix `star schema` in zh docs
2021-08-25 14:14:08 +03:00
Maksim Kita
9462bd3842
Merge pull request #27993 from ClickHouse/update-libunwind-protect-from-wrong-cfa-expressions
Update libunwind
2021-08-25 14:07:57 +03:00
Maksim Kita
df1fe27791
Merge pull request #28105 from azat/enable-optimize_distributed_group_by_sharding_key
Enable optimize_distributed_group_by_sharding_key by default
2021-08-25 14:07:33 +03:00
gyuton
efeefcb437
DOCSUP-11587: Edited and translated to Russian (#27066)
* First Draft.

* Apply suggestions from code review

Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>

* Edited conrib.md

* Edited configuration-files.md

* Minor fixes

* Apply suggestions from code review

Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>

* Update docs/ru/development/contrib.md

Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>

* Small changes

Co-authored-by: George <gyuton@yandex-team.ru>
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-25 13:23:18 +03:00
alesapin
74bd5f9632
Merge pull request #28128 from ClickHouse/revert-28123-improve-01730_distributed_group_by_no_merge_order_by
Revert "Improve 01730_distributed_group_by_no_merge_order_by_long"
2021-08-25 12:38:23 +03:00
alesapin
6e40cad785
Revert "Improve 01730_distributed_group_by_no_merge_order_by_long" 2021-08-25 12:37:55 +03:00
alesapin
dc576e952a
Update KeeperStorage.cpp 2021-08-25 12:31:02 +03:00
mergify[bot]
eedf9f0936
Merge branch 'master' into break_some_tests 2021-08-25 08:38:00 +00:00
Nikita Mikhaylov
6cece7d1d9
Update build.sh 2021-08-25 11:22:16 +03:00
vdimir
b9b0b17ad0
chmod +x docker/test/fuzzer/generate-test-j2.py 2021-08-25 11:20:42 +03:00
vdimir
9ea6ee7fb2
Dummy change in 01720_join_implicit_cast.sql.j2 2021-08-25 11:18:31 +03:00
vdimir
0e37203dcd
Use jinja template tests in fuzzer 2021-08-25 11:18:30 +03:00
alesapin
a0b65b35af
Update tests/queries/skip_list.json 2021-08-25 10:21:44 +03:00
alesapin
875cfb9f98
Update tests/queries/skip_list.json 2021-08-25 10:21:38 +03:00
alesapin
9491c0d650
Update tests/queries/skip_list.json 2021-08-25 10:21:33 +03:00
Maksim Kita
dc852568c0
Merge pull request #27796 from ClickHouse/create-user-defined-lambda-function
Merging #23978
2021-08-25 10:08:20 +03:00
Maksim Kita
f8e2ed4e1d
Merge pull request #28123 from azat/improve-01730_distributed_group_by_no_merge_order_by
Improve 01730_distributed_group_by_no_merge_order_by_long
2021-08-25 10:07:54 +03:00
Azat Khuzhin
e8d87053c0 Fix data-race between setConnectionFD() and setTimer() from cancellation path
TSan report [1]:

    ==================
    WARNING: ThreadSanitizer: data race (pid=7)
      Write of size 8 at 0x7b5000d83768 by thread T539:
        0 Poco::Timespan::operator=(Poco::Timespan const&) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Timespan.cpp:61:8 (clickhouse+0x18e2da08)
        1 DB::RemoteQueryExecutorReadContext::setConnectionFD(int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutorReadContext.cpp:103:21 (clickhouse+0x142c4f41)
        2 DB::RemoteQueryExecutorRoutine::ReadCallback::operator()(int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutorReadContext.cpp:26:30 (clickhouse+0x142c7dae)
        3 decltype(std::__1::forward<DB::RemoteQueryExecutorRoutine::ReadCallback&>(fp)(std::__1::forward<int>(fp0), std::__1::forward<Poco::Timespan>(fp0), std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(fp0))) std::__1::__invoke<DB::RemoteQueryExecutorRoutine::ReadCallback&, int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(DB::RemoteQueryExecutorRoutine::ReadCallback&, int&&, Poco::Timespan&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse+0x142c7cf7)
        4 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::RemoteQueryExecutorRoutine::ReadCallback&, int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(DB::RemoteQueryExecutorRoutine::ReadCallback&, int&&, Poco::Timespan&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse+0x142c7cf7)
        5 std::__1::__function::__default_alloc_func<DB::RemoteQueryExecutorRoutine::ReadCallback, void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(int&&, Poco::Timespan&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse+0x142c7c21)
        6 void std::__1::__function::__policy_invoker<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::__call_impl<std::__1::__function::__default_alloc_func<DB::RemoteQueryExecutorRoutine::ReadCallback, void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> >(std::__1::__function::__policy_storage const*, int, Poco::Timespan&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse+0x142c7c21)
        7 std::__1::__function::__policy_func<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(int&&, Poco::Timespan&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse+0x1589a8c1)
        8 std::__1::function<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse+0x1589a8c1)
        9 DB::HedgedConnections::getReadyFileDescriptor(std::__1::function<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) obj-x86_64-linux-gnu/../src/Client/HedgedConnections.cpp:379:13 (clickhouse+0x1589a8c1)
        10 DB::HedgedConnections::getReadyReplicaLocation(std::__1::function<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) obj-x86_64-linux-gnu/../src/Client/HedgedConnections.cpp:317:20 (clickhouse+0x1589a8c1)
        11 DB::HedgedConnections::receivePacketUnlocked(std::__1::function<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, bool) obj-x86_64-linux-gnu/../src/Client/HedgedConnections.cpp:298:32 (clickhouse+0x1589b531)
        12 DB::RemoteQueryExecutorRoutine::operator()(boost::context::fiber&&) const obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutorReadContext.cpp:46:51 (clickhouse+0x142c7013)
        13 std::__1::enable_if<!(std::is_member_pointer<std::__1::decay<DB::RemoteQueryExecutorRoutine&>::type>::value), std::__1::result_of<DB::RemoteQueryExecutorRoutine& (boost::context::fiber&&)>::type>::type boost::context::detail::invoke<DB::RemoteQueryExecutorRoutine&, boost::context::fiber>(DB::RemoteQueryExecutorRoutine&, boost::context::fiber&&) obj-x86_64-linux-gnu/../contrib/boost/boost/context/detail/invoke.hpp:41:12 (clickhouse+0x142c6be1)
        14 boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine>::run() obj-x86_64-linux-gnu/../contrib/boost/boost/context/fiber_ucontext.hpp:291:17 (clickhouse+0x142c6be1)
        15 void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine> >(void*) obj-x86_64-linux-gnu/../contrib/boost/boost/context/fiber_ucontext.hpp:72:13 (clickhouse+0x142c5f79)
        16 <null> <null> (libc.so.6+0x5e65f)

      Previous read of size 8 at 0x7b5000d83768 by thread T321 (mutexes: write M861449115349678976, write M1059607378698108960, write M747733173224015304):
        0 Poco::Timespan::totalMicroseconds() const obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Timespan.h:213:9 (clickhouse+0x142c54b3)
        1 DB::RemoteQueryExecutorReadContext::setTimer() const obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutorReadContext.cpp:160:25 (clickhouse+0x142c54b3)
        2 DB::RemoteQueryExecutor::tryCancel(char const*, std::__1::unique_ptr<DB::RemoteQueryExecutorReadContext, std::__1::default_delete<DB::RemoteQueryExecutorReadContext> >*) obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:538:26 (clickhouse+0x142be3a5)
        3 DB::RemoteQueryExecutor::cancel(std::__1::unique_ptr<DB::RemoteQueryExecutorReadContext, std::__1::default_delete<DB::RemoteQueryExecutorReadContext> >*) obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:462:5 (clickhouse+0x142bdd8a)
        4 DB::RemoteSource::onCancel() obj-x86_64-linux-gnu/../src/Processors/Sources/RemoteSource.cpp:112:21 (clickhouse+0x15bf52c6)
        5 DB::IProcessor::cancel() obj-x86_64-linux-gnu/../src/Processors/IProcessor.h:237:9 (clickhouse+0x159db37e)
        6 DB::PipelineExecutor::cancel() obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:376:20 (clickhouse+0x159db37e)
        7 DB::PullingAsyncPipelineExecutor::cancel() obj-x86_64-linux-gnu/../src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:175:25 (clickhouse+0x159e9748)
        8 DB::TCPHandler::processOrdinaryQueryWithProcessors() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:700:26 (clickhouse+0x1594f554)
        9 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:329:17 (clickhouse+0x15949a2e)
        10 DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1658:9 (clickhouse+0x15959007)
        11 Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0x18cb2e42)
        12 Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:115:20 (clickhouse+0x18cb35cf)
        13 Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14 (clickhouse+0x18e29c21)
        14 Poco::(anonymous namespace)::RunnableHolder::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:55:11 (clickhouse+0x18e280af)
        15 Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27 (clickhouse+0x18e26827)

  [1]: https://clickhouse-test-reports.s3.yandex.net/27881/b328d9687238617e89c0806c9fa2dd7dd44b5fd7/integration_tests_(thread).htmlfail1
2021-08-25 09:42:30 +03:00
Azat Khuzhin
2df0411b3f Add TimerDescriptor::setRelative(uint64_t usec) 2021-08-25 09:42:30 +03:00
Azat Khuzhin
225f44248c Fix remote query cancelation (in case of remote server abnormaly terminated)
In case the remote server abnormally terminated, neither FIN nor RST
packet will be sent, and the initiator will not know that the connection
died (unless tcp_keep_alive_timeout > 0).

Fix this, using default timeouts.
2021-08-25 09:42:30 +03:00
Kseniia Sumarokova
994d3c04fe
Merge pull request #27919 from adevyatova/annadevyatova-DOCSUP-11585-datetime32
DOCSUP-11585: Document Date32 data type
2021-08-25 09:25:42 +03:00
Azat Khuzhin
af0a4d93ba Improve 01730_distributed_group_by_no_merge_order_by_long
Add a negativ case to show the difference w/ and w/o
optimize_aggregation_in_order=1
2021-08-25 09:05:47 +03:00
Azat Khuzhin
c3096805ac Update 01213_optimize_skip_unused_shards_DISTINCT 2021-08-25 08:41:15 +03:00
terrylin
eea87778d6 Merge branch 'materialize_ttl_recalculate_only' of https://github.com/lthaooo/ClickHouse into materialize_ttl_recalculate_only 2021-08-25 10:57:54 +08:00
terrylin
0094115426 improvement of materilize ttl 2021-08-25 10:57:17 +08:00
lthaooo
314ba1e682
Merge branch 'master' into materialize_ttl_recalculate_only 2021-08-25 10:56:24 +08:00
alexey-milovidov
b6d9fe6f88
Fix idiotic markup 2021-08-25 03:36:42 +03:00
alexey-milovidov
47cc6c5afe
Merge pull request #28063 from vitlibar/set-fixed-version-of-tzlocal
Set version of tzlocal to 2.1
2021-08-25 02:48:08 +03:00
alexey-milovidov
cca850b2ad
Merge pull request #28088 from nvartolomei/nv/config-reloader-crash-on-shutdown
Destroy `main_config_reloader` before shared context.
2021-08-25 02:47:00 +03:00
alexey-milovidov
7d1e74c31f
Update 85_bug-report.md 2021-08-25 02:26:31 +03:00
Nikita Mikhaylov
b330cb855f Fix build 2021-08-24 23:11:48 +00:00
Azat Khuzhin
37043b74bf Enable optimize_distributed_group_by_sharding_key by default
Since it requires optimize_skip_unused_shards this should be safe.
2021-08-24 23:49:50 +03:00
Alexey Milovidov
a3ecc2b705 Remove unused include 2021-08-24 23:47:13 +03:00
mergify[bot]
9520ceba7e
Merge branch 'master' into update-libunwind-protect-from-wrong-cfa-expressions 2021-08-24 20:45:38 +00:00
alexey-milovidov
91cffac7ef
Merge pull request #28095 from ClickHouse/benchmark-shiv
Add benchmark from Shiv Iyer
2021-08-24 23:43:16 +03:00
Maksim Kita
167d34051c
Merge pull request #28066 from ClickHouse/fix_zookeeper_secure_test
Fix zookeeper secure client test
2021-08-24 21:39:31 +03:00
Maksim Kita
0486d1bd37
Merge pull request #28084 from kssenii/nanodbc-update
Update nanodbc
2021-08-24 21:39:10 +03:00
Maksim Kita
027728a7a5 Updated libunwind to protect from wrong CFA expressions 2021-08-24 21:36:45 +03:00
Maksim Kita
de15612202
Merge pull request #28039 from azat/bump-replxx-v21.10
Bump replxx
2021-08-24 21:34:20 +03:00
kssenii
3cd56266c7 Merge branch 'pg-conflict' of github.com:kssenii/ClickHouse into pg-conflict 2021-08-24 18:29:05 +00:00
kssenii
8613d89001 Fix build 2021-08-24 18:28:38 +00:00
Anton Popov
47bb5eb330 support implicit conversions between index in operator [] and key of Map 2021-08-24 21:16:06 +03:00
Dmitry Novik
2da3893b1f refactor isCompatibleEnumTypes to remove PVS warning 2021-08-24 21:11:47 +03:00
Alexey Milovidov
4401b4e888 Add benchmark from Shiv Iyer 2021-08-24 20:49:35 +03:00
alexey-milovidov
19f67fefb2
Merge pull request #28047 from BraulioVM/build/add-git-status
print out git status information at configure stage
2021-08-24 20:09:10 +03:00
Dmitriy
7d9180cd8c Update other-functions.md
Поправил предупреждение.
2021-08-24 20:08:39 +03:00
Dmitriy
77296573e6
Update docs/ru/sql-reference/functions/other-functions.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-24 19:55:53 +03:00
Dmitriy
980720112e
Update docs/ru/sql-reference/functions/other-functions.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-24 19:55:42 +03:00
Dmitriy
15e986b3d9
Update docs/ru/sql-reference/functions/other-functions.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-08-24 19:55:35 +03:00