thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 10:22:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
134,090 Commits 9,328 Branches 1,783 Tags 2.1 GiB
ccafc523a9
Commit Graph

4208 Commits

Author SHA1 Message Date
Raúl Marín
b269f87f4c Better text_log with ErrnoException 2023-12-15 19:27:56 +01:00
Sema Checherinda
0dfe530a7f
Merge pull request #56744 from MikhailBurdukov/native_copy_for_s3_disks 
Enabled s3 `copyObject` for copy between s3 disks.
 2023-12-15 16:05:16 +01:00
Kseniia Sumarokova
06438cc17b
Merge pull request #57578 from ClickHouse/allow-to-change-some-cache-settings-without-restart 
Allow to apply some fs cache config settings changes without server restart
 2023-12-15 11:18:19 +01:00
Yakov Olkhovskiy
febdd542f8 Merge remote-tracking branch 'origin/master' into feature-server-iface-metrics 2023-12-14 18:59:44 +00:00
Azat Khuzhin
0e86e16a01
Revert "Replace --no-system-tables with loading virtual tables of system database lazily" 2023-12-14 09:07:47 +01:00
Kseniia Sumarokova
79db3c66df
Merge branch 'master' into allow-to-change-some-cache-settings-without-restart 2023-12-13 23:33:59 +01:00
Nikita Mikhaylov
8372c70958 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-the-limit-for-connections-per-endpoint 2023-12-13 18:29:56 +00:00
Alexey Milovidov
54698cd2de
Merge pull request #55876 from ClickHouse/keeper-http-readiness-endpoint 
Keeper HTTP readiness endpoint
 2023-12-13 17:07:57 +01:00
Alexander Tokmakov
3482224f83 Merge branch 'master' into keeper-http-readiness-endpoint 2023-12-12 21:19:57 +01:00
Han Fei
a96593c8b1 Merge branch 'master' into feature/more_warnings 2023-12-12 15:25:01 +01:00
Natalya Chizhonkova
3d51192bad Refactoring for UserDefinedSQLObjectsLoader 2023-12-12 17:03:49 +03:00
avogar
6088536f3c Remove outdated comment 2023-12-12 13:31:12 +00:00
avogar
6405bb4086 Change default database name in clickhouse-local to 'default' 2023-12-12 13:30:26 +00:00
alesapin
cbae29d26a
Merge pull request #57699 from hanfei1991/hanfei/keeper_soft_limit_ratio 
support keeper memory soft limit ratio
 2023-12-11 21:43:24 +01:00
Konstantin Bogdanov
771d5c58ea
Fix Keeper standalone build 2023-12-11 10:19:49 +01:00
Konstantin Bogdanov
89ba4a8452
Add HTTP readiness endpoint 2023-12-11 10:19:42 +01:00
MikhailBurdukov
119e451967 Merge branch 'master' into native_copy_for_s3_disks 2023-12-11 07:25:20 +00:00
Alexey Milovidov
e443567c0e
Merge pull request #52717 from azat/wait-threadpool 
[RFC] Correctly wait background threads
 2023-12-11 01:06:23 +01:00
Azat Khuzhin
8f5a60eb5a More descriptive comment for destroying context prio global thread pool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-10 15:39:43 +01:00
Azat Khuzhin
fcdead023c Proper server termination in case of intermediate failures during initialization 
For instance on CI it fails because KeeperDispatch::initialize() failed,
but there is already proper KeeperDispatch::shutdown() in dtor, the
problem is that you first need to shutdown global context, that will
terminate all thread pools, and only after you can join global thread
pool.

CI: https://s3.amazonaws.com/clickhouse-test-reports/52717/6ccbc2ea75c39234dff748cfb147bd44d65d3e75/integration_tests__asan__[3_4].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-10 08:01:40 +01:00
Azat Khuzhin
4f9d90761b Attach tables lazily only for clickhouse-local 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 23:08:29 +01:00
Azat Khuzhin
cc9afe6f6e Replace --no-system-tables with loading virtual tables of system database lazily 
First of all the problem is that that simple 'SELECT 1' cannot be runned
without system.one, which makes --no-system-tables almost useless:

    $ ./clickhouse-debug local --no-system-tables -q "select 1"
    Code: 81. DB::Exception: Database system does not exist. (UNKNOWN_DATABASE)

Secondly there are just too much flags, and this one
(--no-system-tables) is too damn specific.

This patch should improve startup time of the clickhouse-local almost
3x in debug builds.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 23:08:29 +01:00
Azat Khuzhin
b7edde3621 Shutdown embedded keeper even if it was not started successfully 
Since even when it does not, there are still few threads left that
should be joined (request, response, snapshot, snapshot s3):

There are some problems with keeper and invalid config, that got stuck in:

    libc.so.6`___lldb_unnamed_symbol3457 + 231
    libc.so.6`pthread_cond_timedwait + 571
    clickhouse`std::__1::__libcpp_condvar_timedwait[abi:v15000](__cv=0x00007f475c41a708, __m=0x00007f475c41a6b0, __ts=0x00007f45a99f1440) at __threading_support:341:10
    clickhouse`std::__1::condition_variable::__do_timed_wait(this=0x00007f475c41a708, lk=0x00007f45a99f1738, tp=time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1L, 1000000000L> > > @ 0x00007f45a99f1488) at condition_variable.cpp:76:14
    clickhouse`std::__1::cv_status std::__1::condition_variable::wait_for<long long, std::__1::ratio<1l, 1000000000l> >(this=0x00007f475c41a708, __lk=0x00007f45a99f1738, __d=0x00007f45a99f1590) at __mutex_base:457:9
    clickhouse`void std::__1::condition_variable::__do_timed_wait<std::__1::chrono::steady_clock>(this=0x00007f475c41a708, __lk=0x00007f45a99f1738, __tp=time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1L, 1000000000L> > > @ 0x00007f45a99f15a8) at __mutex_base:512:5
    clickhouse`std::__1::cv_status std::__1::condition_variable::wait_until<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >(this=0x00007f475c41a708, __lk=0x00007f45a99f1738, __t=0x00007f45a99f1698) at __mutex_base:415:5
    clickhouse`bool std::__1::condition_variable::wait_until<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> >, bool ConcurrentBoundedQueue<DB::KeeperStorage::RequestForSession>::popImpl<true>(DB::KeeperStorage::RequestForSession&, std::__1::optional<unsigned long>)::'lambda'()>(this=0x00007f475c41a708, __lk=0x00007f45a99f1738, __t=0x00007f45a99f1698, __pred=(unnamed class) @ 0x00007f45a99f1650)::'lambda'()) at __mutex_base:427:13
    clickhouse`bool std::__1::condition_variable::wait_for[abi:v15000]<long long, std::__1::ratio<1l, 1000l>, bool ConcurrentBoundedQueue<DB::KeeperStorage::RequestForSession>::popImpl<true>(DB::KeeperStorage::RequestForSession&, std::__1::optional<unsigned long>)::'lambda'()>(this=0x00007f475c41a708, __lk=0x00007f45a99f1738, __d=0x00007f45a99f1720, __pred=(unnamed class) @ 0x00007f45a99f16b8)::'lambda'()) at __mutex_base:471:12
    clickhouse`bool ConcurrentBoundedQueue<DB::KeeperStorage::RequestForSession>::popImpl<true>(this=0x00007f475c41a680, x=0x00007f45a99f1ba0, timeout_milliseconds= Has Value=true ) at ConcurrentBoundedQueue.h:75:50
    clickhouse`ConcurrentBoundedQueue<DB::KeeperStorage::RequestForSession>::tryPop(this=0x00007f475c41a680, x=0x00007f45a99f1ba0, milliseconds=5000) at ConcurrentBoundedQueue.h:161:16
    clickhouse`DB::KeeperDispatcher::requestThread(this=0x00007f475c8fac18) at KeeperDispatcher.cpp:88:33
    clickhouse`DB::KeeperDispatcher::initialize(this=0x00007f45a99f2020)::$_0::operator()() const at KeeperDispatcher.cpp:349:52
    clickhouse`decltype(__f=0x00007f45a99f2020)::$_0&>()()) std::__1::__invoke[abi:v15000]<DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&) at invoke.h:394:23
    clickhouse`decltype(__f=0x00007f45a99f2020, __t=size=0, (null)=__tuple_indices<> @ 0x00007f45a99f1d28) std::__1::__apply_tuple_impl[abi:v15000]<DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&, std::__1::tuple<>&>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&, std::__1::tuple<>&, std::__1::__tuple_indices<>) at tuple:1789:1
    clickhouse`decltype(__f=0x00007f45a99f2020, __t=size=0) std::__1::apply[abi:v15000]<DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&, std::__1::tuple<>&>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&, std::__1::tuple<>&) at tuple:1798:1
    clickhouse`ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::KeeperDispatcher::initialize(this=0x00007f475c9703c0)::$_0>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&&)::'lambda'()::operator()() at ThreadPool.h:245:13
    clickhouse`decltype(__f=0x00007f475c9703c0)::$_0>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&&)::'lambda'()&>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&&) at invoke.h:394:23
    clickhouse`void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::KeeperDispatcher::initialize(__args=0x00007f475c9703c0)::$_0>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&&)::'lambda'()&) at invoke.h:479:9
    clickhouse`std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&&)::'lambda'(), void ()>::operator(this=0x00007f475c9703c0)[abi:v15000]() at function.h:235:12
    clickhouse`void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::KeeperDispatcher::initialize(__buf=0x00007f45a99f22d8)::$_0>(DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::__1::shared_ptr<DB::Macros const> const&)::$_0&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) at function.h:716:16
    clickhouse`std::__1::__function::__policy_func<void ()>::operator(this=0x00007f45a99f22d8)[abi:v15000]() const at function.h:848:16
    clickhouse`std::__1::function<void ()>::operator(this=0x00007f45a99f22d8)() const at function.h:1187:12
    clickhouse`ThreadPoolImpl<std::__1::thread>::worker(this=0x00007f475c8229c0, thread_it=std::__1::list<std::__1::thread, std::__1::allocator<std::__1::thread> >::iterator @ 0x00007f45a99f23a8) at ThreadPool.cpp:427:13
    clickhouse`void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(this=0x00007f45cea0c8a8)>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const at ThreadPool.cpp:180:73
    clickhouse`decltype(__f=0x00007f45cea0c8a8)()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&) at invoke.h:394:23
    clickhouse`void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(__t=size=2, (null)=__tuple_indices<> @ 0x00007f45a99f2408)>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>) at thread:284:5
    clickhouse`void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()> >(__vp=0x00007f45cea0c8a0) at thread:295:5
    libc.so.6`___lldb_unnamed_symbol3481 + 755
    libc.so.6`___lldb_unnamed_symbol3865 + 11

This should fix the problem with
test_keeper_incorrect_config/test.py::test_invalid_configs test.

CI: https://s3.amazonaws.com/clickhouse-test-reports/52717/72b1052f7c2d453308262924e767ab8dc2206933/integration_tests__asan__[1_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 17:54:16 +01:00
Azat Khuzhin
55d5a3affa Correctly wait background threads 
There are some places that make it possible to trigger use-after-free
from threads because some global variable had been destroyed before, for
example some logger.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 17:54:16 +01:00
Han Fei
881e58dfcb support soft limit ratio 2023-12-08 21:48:27 +01:00
Nikita Mikhaylov
c979124cfe Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-the-limit-for-connections-per-endpoint 2023-12-08 16:25:02 +00:00
kssenii
199015576c Add a file which I forgot to commit 2023-12-07 18:30:43 +01:00
Nikolay Degterinsky
995b51ef73 Allow avoiding resolving hostnames in DDLWorker 2023-12-07 03:43:26 +00:00
Alexey Milovidov
0b597eadf8 Clarify the usage of prefer_not_to_merge option 2023-12-06 12:11:03 +01:00
jsc0218
cdd5280272
Merge pull request #57387 from evillique/better-disks 
Initialize only required disks in clickhouse-disks
 2023-12-05 13:59:04 -05:00
Nikita Mikhaylov
04d167c6d9 Better 2023-12-05 13:34:37 +01:00
Alexey Milovidov
cbea3545aa
Revert "add function getClientHTTPHeader" 2023-12-05 13:34:34 +03:00
凌涛
9e05e79d66
Merge branch 'master' into feature/more_warnings 2023-12-05 11:50:05 +08:00
凌涛
6794bbe196 rebase master 2023-12-05 11:42:25 +08:00
Sergei Trifonov
c073d03a20
Merge pull request #57236 from ClickHouse/multiline-charts 
Support multiple lines on dashboard charts
 2023-12-04 17:35:31 +01:00
serxa
1c650f380a remove debug print 2023-12-04 16:26:39 +00:00
Alexey Milovidov
02439eee58
Merge pull request #49351 from ClickHouse/async-loader-integration 
Asynchronous loading of tables
 2023-12-04 17:16:12 +01:00
Nikolay Degterinsky
93995f6c06 Fix test 2023-12-02 23:59:46 +00:00
Nikita Taranov
a32c5e976a
Don't print server revision in client on connect (#57435) 2023-12-02 16:32:34 +01:00
Yakov Olkhovskiy
416d28687e Merge remote-tracking branch 'origin/master' into feature-server-iface-metrics 2023-11-30 23:00:04 +00:00
Nikolay Degterinsky
e8203f8a76 Initialize only required disks 2023-11-30 03:09:55 +00:00
凌涛
d9f4b4d2c0 support parts, tables, databases 2023-11-30 10:56:31 +08:00
Antonio Andelic
93360e4e2c
Merge pull request #57299 from ClickHouse/keeper-changelog-snapshot-fix 
Keeper fix for changelog and snapshots
 2023-11-29 08:31:48 +01:00
János Benjamin Antal
4c2451f837
Merge pull request #56741 from ClickHouse/well-known-protos 2023-11-28 15:21:15 +01:00
Antonio Andelic
a61f32867e
Merge pull request #56488 from lingtaolf/feature/getHTTPHeader 
add function getClientHTTPHeader
 2023-11-28 11:41:56 +01:00
Antonio Andelic
889c53eca0 Correctly handle unknown changelog versions and snapshot deleting 2023-11-28 09:47:49 +00:00
serxa
0d39de9ee9 indicate that legend is not full by "..." 2023-11-28 01:18:41 +00:00
serxa
4979925095 fix legend + add string-labeled format 2023-11-28 00:36:38 +00:00
Yakov Olkhovskiy
b92030b485
Merge branch 'master' into feature-server-iface-metrics 2023-11-27 15:44:25 -05:00
János Benjamin Antal
adf21e37c4 Make tests use the installed protobuf files 2023-11-27 13:39:28 +00:00
János Benjamin Antal
4bfbbfbd75 Fix proto file installation 2023-11-27 13:39:01 +00:00
MikhailBurdukov
6f19e8ebd1
Merge branch 'master' into native_copy_for_s3_disks 2023-11-27 14:25:36 +03:00
serxa
62831d235f support multiple lines on dashboard charts 2023-11-26 23:28:11 +00:00
Vitaly Baranov
5769a88b92
Merge pull request #57133 from vitlibar/change-default-for-wait_dictionaries_load_at_startup 
Change the default for wait_dictionaries_load_at_startup to true
 2023-11-24 17:09:05 +01:00
alesapin
a3c2896e9d
Merge pull request #56846 from aalexfvk/add_cancellation_point_for_moving_background_operation 
Add cancellation hook for moving background operation
 2023-11-24 12:12:47 +01:00
Kseniia Sumarokova
650c1e959d
Merge pull request #57141 from ClickHouse/remove-test-case 
Follow up to #56541
 2023-11-24 11:45:29 +01:00
Alexey Milovidov
98cddf5312
Merge pull request #57153 from ClickHouse/better-server-exit-code 
Better server exit code
 2023-11-23 23:55:28 +01:00
Igor Nikonov
0434044a7e
Merge pull request #57054 from ClickHouse/pr-cleanup-narrow-dependency 
Parallel replicas: cleanup, narrow dependency
 2023-11-23 20:12:37 +01:00
Sergei Trifonov
9436ae6d7d
Merge pull request #56771 from ClickHouse/dashboards-table 
Load queries for dashboard page from new `system.dashboards` table
 2023-11-23 19:07:52 +01:00
Alexey Milovidov
b62e9d3a1a Address review comments 2023-11-23 18:13:12 +01:00
Alexey Milovidov
137bf2cf9c Whitespaces 2023-11-23 17:51:55 +01:00
Alexey Milovidov
bd4ffec361 Better exit codes from the server 2023-11-23 17:00:57 +01:00
Alexey Milovidov
5abed8f8a8 Better exit codes from the server 2023-11-23 17:00:02 +01:00
Vitaly Baranov
5f461ff780 Change the default for "wait_dictionaries_load_at_startup" to true, 
and use this setting only if "dictionaries_lazy_load" is false.
 2023-11-23 14:45:42 +01:00
Aleksei Filatov
1a03f5f7f4 Merge remote-tracking branch 'upstream/master' into add_cancellation_point_for_moving_background_operation 2023-11-23 16:43:33 +03:00
kssenii
ff3224e7f7 Load named collections in advance 2023-11-23 13:22:06 +01:00
serxa
663c8cd944 Merge branch 'master' into dashboards-table 2023-11-23 10:29:48 +00:00
serxa
0fc25fbda8 Merge branch 'master' into async-loader-integration 2023-11-23 09:32:43 +00:00
János Benjamin Antal
ab935e3dd7 Use the google proto files when importing protobuf schemas 2023-11-22 12:39:41 +00:00
János Benjamin Antal
6f652133a8 Install well-known protobuf types 2023-11-22 12:39:29 +00:00
Kseniia Sumarokova
1dfafcfe5a
Merge branch 'master' into add-system-s3-queue-log-to-config 2023-11-21 21:31:35 +01:00
vdimir
a139ae97eb
Merge pull request #52918 from ClickHouse/vdimir/s3_blob_log 
Add system table with blob storage operations log
 2023-11-21 17:40:42 +01:00
Igor Nikonov
63a1625b77 Merge remote-tracking branch 'origin/master' into pr-cleanup-narrow-dependency 2023-11-21 16:05:48 +00:00
Igor Nikonov
ce98dfb251 Settings pointer to reference 2023-11-21 16:04:54 +00:00
凌涛
e86aa433c5
Merge branch 'master' into feature/getHTTPHeader 2023-11-21 19:48:47 +08:00
kssenii
ad9d4a9bb8 Add system.s3queue_log to default config 2023-11-21 11:53:43 +01:00
Alexey Milovidov
2988becba6
Merge pull request #36589 from ClickHouse/remove-useless-install 
Remove useless "install" from CMake (step 1)
 2023-11-21 11:06:09 +01:00
vdimir
15234474d7
Implement system table blob_storage_log 2023-11-21 09:18:25 +00:00
凌涛
c5f16725ec add function getClientHTTPHeader 2023-11-21 11:12:01 +08:00
Alexey Milovidov
241cc2abf4 Merge branch 'master' into remove-useless-install 2023-11-20 01:12:08 +01:00
Nikolay Degterinsky
fd26fab07f Merge remote-tracking branch 'upstream/master' into local-send-fatal-log 2023-11-19 17:59:51 +00:00
Nikolay Degterinsky
cf46214440 Always send fatal level logs 2023-11-19 15:09:52 +00:00
serxa
9023f61349 Merge branch 'master' into async-loader-integration 2023-11-19 13:05:58 +00:00
Alexey Milovidov
d56cbda185 Add metrics for the number of queued jobs, which is useful for the IO thread pool 2023-11-18 19:07:59 +01:00
Nikolay Degterinsky
d1c56e3dab Send fatal logs by default in clickhouse-local 2023-11-18 17:40:15 +00:00
Raúl Marín
7074bdb3bb Merge remote-tracking branch 'blessed/master' into access-control-and-named-collections-by-default 2023-11-17 14:53:02 +01:00
Aleksei Filatov
163b8a814e Fix clang-tidy issues and disable parallel execution of tests 2023-11-17 11:02:07 +03:00
Alexey Milovidov
482d8ca9d8
Merge pull request #56102 from ClickHouse/coverage 
Granular code coverage with introspection
 2023-11-16 23:23:40 +01:00
serxa
dc644b9617 add backward compatibility to avoid breaking old URLs on new servers 2023-11-16 13:09:30 +00:00
serxa
74ee267524 refactoring: error is now used not only for auth 2023-11-16 12:17:22 +00:00
serxa
84656ed9af track if dashboard was customized, show with opacity 2023-11-16 11:38:32 +00:00
Vitaly Baranov
342e6c7546
Merge pull request #56782 from vitlibar/add-config-setting-wait_dictionaries_load_at_startup 
Add config setting "wait_dictionaries_load_at_startup"
 2023-11-16 11:51:29 +01:00
Alexey Milovidov
3067ca64df
Merge pull request #56573 from mkmkme/mkmkme/reload-config 
Update `query_masking_rules` when reloading the config
 2023-11-16 00:27:01 +01:00
serxa
e37fdaad33 fix issues with history and errors 2023-11-15 19:17:02 +00:00
Vitaly Baranov
d62048031f Add documentation. 2023-11-15 13:57:28 +01:00
Vitaly Baranov
308654195c Add config setting "wait_dictionaries_load_at_startup". 2023-11-15 11:48:50 +01:00
MikhailBurdukov
f29c072429 Test fix 2023-11-15 09:33:40 +00:00
serxa
bd66ac483c Load queries for dashboard page from new system.dashboards table 2023-11-14 23:03:06 +00:00
Sema Checherinda
b9cc1580e0
Merge pull request #56367 from canhld94/ch_table_reinit_new_disk 
Adding new disk to storage configuration without restart
 2023-11-14 15:54:22 +01:00
Mikhail Koviazin
9e1357dd7e
Update query_masking_rules when reloading the config 
Fixes #56449
 2023-11-14 14:03:28 +02:00
Alexey Milovidov
3ef14f6098 Merge branch 'master' of github.com:ClickHouse/ClickHouse into coverage 2023-11-14 06:08:32 +01:00
Alexey Milovidov
d7df884cbb Merge branch 'master' into filter-large-translation-units 2023-11-13 19:31:13 +01:00
Alexey Milovidov
c2127b05f6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into filter-large-translation-units 2023-11-13 17:54:15 +01:00
Alexey Milovidov
c1bba6ea4a Merge branch 'master' into filter-large-translation-units 2023-11-13 17:54:02 +01:00
Alexey Milovidov
8e820185cb Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-ccache-2 2023-11-13 11:17:18 +01:00
Alexey Milovidov
90936aa2c7
Merge pull request #56649 from chhetripradeep/pchhetri/enable-secure-flag-for-clickhouse-cloud-second-attempt 
Switch to SSL port for clickhouse-client for hostnames pointing to clickhouse cloud
 2023-11-13 10:18:29 +01:00
Alexey Milovidov
82c461031e Fix build 2023-11-13 10:09:23 +01:00
Pradeep Chhetri
be1e061764 Use SSL port for clickhouse-client for hostnames pointing to clickhouse cloud 2023-11-13 10:31:56 +08:00
Alexey Milovidov
7ac931c29d Merge branch 'master' into concurrency 2023-11-13 02:56:56 +01:00
Robert Schulze
3f49e1c5ed
Merge remote-tracking branch 'rschu1ze/master' into pcg 2023-11-12 11:43:19 +00:00
Alexey Milovidov
0279345871
Merge pull request #56634 from ClickHouse/short-names 
Introduce short names for `clickhouse-client` and `clickhouse-local`
 2023-11-12 05:27:18 +01:00
Alexey Milovidov
d2560f2197 Make clickhouse-local and clickhouse-client available under short names for usability 2023-11-12 02:42:51 +01:00
Alexey Milovidov
e207b1b14a Merge branch 'master' into access-control-and-named-collections-by-default 2023-11-12 00:56:06 +01:00
Alexey Milovidov
9f46d45066 Merge branch 'master' into symbolize-trace-log 2023-11-12 00:53:16 +01:00
Alexey Milovidov
ad03f1ee47
Merge pull request #56485 from ClickHouse/better-settings 
Better settings
 2023-11-12 00:01:38 +01:00
Alexey Milovidov
38ca18d8e7
Merge pull request #56617 from ClickHouse/publish-stripped-binary 
Publish stripped binary
 2023-11-11 22:18:26 +01:00
Alexey Milovidov
527ca9a1fc
Merge branch 'master' into symbolize-trace-log 2023-11-11 09:38:02 +01:00
Alexey Milovidov
04630c3e5a Merge branch 'master' into better-settings 2023-11-11 09:26:59 +01:00
Alexey Milovidov
356ae52e9b Enable access and named collections control by default 2023-11-11 08:46:10 +01:00
Alexey Milovidov
4b1fa685bb Publish stripped binary 2023-11-11 07:28:26 +01:00
Alexey Milovidov
df24ef42b1 Publish stripped binary 2023-11-11 07:27:10 +01:00
Alexey Milovidov
6078ff20bb Symbolize trace_log for exporting 2023-11-11 05:52:44 +01:00
Alexey Milovidov
96f73139b6 Check for large translation units 2023-11-10 06:13:55 +01:00
Alexey Milovidov
1685cdcb89 Reasonable value for HTTP keep-alive 2023-11-09 00:32:43 +01:00
Alexey Milovidov
2ce0ef9c8a Enable HTTP OPTIONS method by default 2023-11-09 00:23:20 +01:00
Oleg V. Kozlyuk
e341d8cea3
Add example to users.xml 2023-11-08 03:01:42 +03:00
Oleg V. Kozlyuk
0e52328795
Updated users.yml example 2023-11-08 02:59:54 +03:00
Robert Schulze
e46dbcb2f1
Use pcg/randomSeed() random generator/seed instead of std::mt19937/std::random_device 2023-11-07 18:54:50 +00:00
serxa
19aa1375fc Merge branch 'master' into async-loader-integration 2023-11-07 10:25:38 +00:00
Robert Schulze
8896134531
Cleanup IDataType convenience functions 2023-11-06 13:25:32 +00:00
serxa
d50747847d rework AsyncLoaderPoolId.h to match settings and metrics 2023-11-06 13:08:27 +00:00
serxa
28128d3f78 fix defaults for new pools in server config.xml 2023-11-06 12:23:09 +00:00
Duc Canh Le
4c21ba7b6f tables auto initialize new disks without restart 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-06 10:26:48 +00:00
Alexey Milovidov
cd94d02455 Merge branch 'master' into coverage 2023-11-04 21:49:59 +01:00
Yakov Olkhovskiy
4f11132ea2 fix clang tidy 2023-11-04 00:00:14 +00:00
serxa
77dc2e63b1 randomize async_load_databases in stress tests 2023-11-03 17:15:51 +00:00
vdimir
a70d0c5203
Merge pull request #56264 from azat/rocksdb/block-based-table-options 
Add ability to specify BlockBasedTableOptions for rocksdb
 2023-11-03 10:15:19 +01:00
serxa
e97edf5285 Merge branch 'master' into async-loader-integration 2023-11-02 14:56:32 +00:00
Azat Khuzhin
decf85baca Add ability to specify BlockBasedTableOptions for rocksdb 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-02 15:36:00 +01:00
Azat Khuzhin
cea78cd093
Add a warning if delayed accounting is not enabled (breaks OSIOWaitMicroseconds) (#56227) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-02 13:21:27 +01:00
Azat Khuzhin
c25d6cd624
Rename directory monitor concept into background INSERT (#55978) 
* Limit log frequence for "Skipping send data over distributed table" message

After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this
message.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename directory monitor concept into async INSERT

Rename the following query settings (with preserving backward
compatiblity, by keeping old name as an alias):
- distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms
- distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms
- distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts
- distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure

Rename the following table settings (with preserving backward
compatiblity, by keeping old name as an alias):
- monitor_batch_inserts -> async_insert_batch
- monitor_split_batch_on_failure -> async_insert_split_batch_on_failure
- directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms
- directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms

And also update all the references:

    $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename async_insert for Distributed into background_insert

This will avoid amigibuity between general async INSERT's and INSERT
into Distributed, which are indeed background, so new term express it
even better.

Mostly done with:

    $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g'

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Mark 02417_opentelemetry_insert_on_distributed_table as long

CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-01 15:09:39 +01:00
Kseniia Sumarokova
d026299901
Merge pull request #55949 from azat/connections_credentials-fix 
Fix overrides via connections_credentials in case of root directives exists
 2023-10-30 12:17:33 +01:00
Alexey Milovidov
56de2333f9 Add warning 2023-10-29 16:55:47 +01:00
Alexey Milovidov
70e3dd808c Granular code coverage with introspection 2023-10-29 02:07:24 +01:00
Alexey Milovidov
102feacac0
Merge pull request #55654 from ClickHouse/fix-garbage-config-index-mark-cache 
Fix misconfiguration of the index mark cache
 2023-10-28 14:57:29 +02:00
robot-ch-test-poll3
b6cb32b984
Merge pull request #56086 from kitaisreal/keeper-fix-run-without-config 
Keeper fix run without config
 2023-10-28 00:36:52 +02:00
Azat Khuzhin
8782873e4f Fix overrides via connections_credentials in case of root directives exists 
Before the following did not work, it always uses user `dev`, even with
`clickhouse-client --connection prod`:

  ```yaml
  user: dev

  connections_credentials:
    prod:
      name: prod
      user: prod
  ```

The problem was that before it was not possible to distinguish options
that had been set via command line options and via configuration file.

I've splitted this two actions, and embedded a call to
parseConnectionsCredentials() in between.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-27 20:03:50 +02:00
Maksim Kita
9b60a3d7aa Keeper run without config fix 2023-10-27 20:53:07 +03:00
Sergei Trifonov
a5d7c2b6a2
Merge branch 'master' into async-loader-integration 2023-10-27 10:39:12 +02:00
Yakov Olkhovskiy
0cf851316c use CH Buffer for HTTP out stream, add metrics for interfaces 2023-10-27 02:38:36 +00:00