thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 08:32:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
133,278 Commits 9,168 Branches 1,781 Tags 2.1 GiB
9038f0710a
Commit Graph

4191 Commits

Author SHA1 Message Date
Alexey Milovidov
04630c3e5a Merge branch 'master' into better-settings 2023-11-11 09:26:59 +01:00
Alexey Milovidov
356ae52e9b Enable access and named collections control by default 2023-11-11 08:46:10 +01:00
Alexey Milovidov
4b1fa685bb Publish stripped binary 2023-11-11 07:28:26 +01:00
Alexey Milovidov
df24ef42b1 Publish stripped binary 2023-11-11 07:27:10 +01:00
Alexey Milovidov
6078ff20bb Symbolize trace_log for exporting 2023-11-11 05:52:44 +01:00
Alexey Milovidov
96f73139b6 Check for large translation units 2023-11-10 06:13:55 +01:00
Alexey Milovidov
1685cdcb89 Reasonable value for HTTP keep-alive 2023-11-09 00:32:43 +01:00
Alexey Milovidov
2ce0ef9c8a Enable HTTP OPTIONS method by default 2023-11-09 00:23:20 +01:00
Oleg V. Kozlyuk
e341d8cea3
Add example to users.xml 2023-11-08 03:01:42 +03:00
Oleg V. Kozlyuk
0e52328795
Updated users.yml example 2023-11-08 02:59:54 +03:00
Robert Schulze
e46dbcb2f1
Use pcg/randomSeed() random generator/seed instead of std::mt19937/std::random_device 2023-11-07 18:54:50 +00:00
serxa
19aa1375fc Merge branch 'master' into async-loader-integration 2023-11-07 10:25:38 +00:00
Robert Schulze
8896134531
Cleanup IDataType convenience functions 2023-11-06 13:25:32 +00:00
serxa
d50747847d rework AsyncLoaderPoolId.h to match settings and metrics 2023-11-06 13:08:27 +00:00
serxa
28128d3f78 fix defaults for new pools in server config.xml 2023-11-06 12:23:09 +00:00
Duc Canh Le
4c21ba7b6f tables auto initialize new disks without restart 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-06 10:26:48 +00:00
Alexey Milovidov
cd94d02455 Merge branch 'master' into coverage 2023-11-04 21:49:59 +01:00
Yakov Olkhovskiy
4f11132ea2 fix clang tidy 2023-11-04 00:00:14 +00:00
serxa
77dc2e63b1 randomize async_load_databases in stress tests 2023-11-03 17:15:51 +00:00
vdimir
a70d0c5203
Merge pull request #56264 from azat/rocksdb/block-based-table-options 
Add ability to specify BlockBasedTableOptions for rocksdb
 2023-11-03 10:15:19 +01:00
serxa
e97edf5285 Merge branch 'master' into async-loader-integration 2023-11-02 14:56:32 +00:00
Azat Khuzhin
decf85baca Add ability to specify BlockBasedTableOptions for rocksdb 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-02 15:36:00 +01:00
Azat Khuzhin
cea78cd093
Add a warning if delayed accounting is not enabled (breaks OSIOWaitMicroseconds) (#56227) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-02 13:21:27 +01:00
Azat Khuzhin
c25d6cd624
Rename directory monitor concept into background INSERT (#55978) 
* Limit log frequence for "Skipping send data over distributed table" message

After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this
message.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename directory monitor concept into async INSERT

Rename the following query settings (with preserving backward
compatiblity, by keeping old name as an alias):
- distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms
- distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms
- distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts
- distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure

Rename the following table settings (with preserving backward
compatiblity, by keeping old name as an alias):
- monitor_batch_inserts -> async_insert_batch
- monitor_split_batch_on_failure -> async_insert_split_batch_on_failure
- directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms
- directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms

And also update all the references:

    $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename async_insert for Distributed into background_insert

This will avoid amigibuity between general async INSERT's and INSERT
into Distributed, which are indeed background, so new term express it
even better.

Mostly done with:

    $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g'

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Mark 02417_opentelemetry_insert_on_distributed_table as long

CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-01 15:09:39 +01:00
Kseniia Sumarokova
d026299901
Merge pull request #55949 from azat/connections_credentials-fix 
Fix overrides via connections_credentials in case of root directives exists
 2023-10-30 12:17:33 +01:00
Alexey Milovidov
56de2333f9 Add warning 2023-10-29 16:55:47 +01:00
Alexey Milovidov
70e3dd808c Granular code coverage with introspection 2023-10-29 02:07:24 +01:00
Alexey Milovidov
102feacac0
Merge pull request #55654 from ClickHouse/fix-garbage-config-index-mark-cache 
Fix misconfiguration of the index mark cache
 2023-10-28 14:57:29 +02:00
robot-ch-test-poll3
b6cb32b984
Merge pull request #56086 from kitaisreal/keeper-fix-run-without-config 
Keeper fix run without config
 2023-10-28 00:36:52 +02:00
Azat Khuzhin
8782873e4f Fix overrides via connections_credentials in case of root directives exists 
Before the following did not work, it always uses user `dev`, even with
`clickhouse-client --connection prod`:

  ```yaml
  user: dev

  connections_credentials:
    prod:
      name: prod
      user: prod
  ```

The problem was that before it was not possible to distinguish options
that had been set via command line options and via configuration file.

I've splitted this two actions, and embedded a call to
parseConnectionsCredentials() in between.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-27 20:03:50 +02:00
Maksim Kita
9b60a3d7aa Keeper run without config fix 2023-10-27 20:53:07 +03:00
Sergei Trifonov
a5d7c2b6a2
Merge branch 'master' into async-loader-integration 2023-10-27 10:39:12 +02:00
Yakov Olkhovskiy
0cf851316c use CH Buffer for HTTP out stream, add metrics for interfaces 2023-10-27 02:38:36 +00:00
Azat Khuzhin
caec685a49 Update formats list 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-26 17:35:22 +02:00
Azat Khuzhin
4cb8b186e7 Add more delimiters for completion 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-26 17:35:22 +02:00
Azat Khuzhin
b6435b32f0 Fix bash completion for mawk 
$ strings < /usr/bin/awk | grep awk
mawk 1.3%s%s %s, Copyright (C) Michael D. Brennan

$ awk '{print substr($0, 0, 1)}' <<<foo # returns empty string

$ awk '{print substr($0, 1, 1)}' <<<foo # returns empty string
f

$ awk -V | head -1
GNU Awk 5.2.2, API 3.2, PMA Avon 8-g1, (GNU MPFR 4.2.1, GNU MP 6.3.0)

$ awk '{print substr($0, 0, 1)}' <<<foo
f
$ awk '{print substr($0, 1, 1)}' <<<foo
f

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-26 17:35:22 +02:00
Alexander Tokmakov
fa963d60d0
Merge pull request #55641 from evillique/resubmit-cluster-groups 
Resubmit #54421
 2023-10-26 15:53:53 +02:00
serxa
08e63f2a24 fix build 2023-10-24 21:49:15 +00:00
xuzifu666
010cc6918a
Add get_direct_children_number command to keeper-client (#55898) 2023-10-24 17:02:54 +02:00
serxa
749cdb9954 Merge branch 'master' into async-loader-integration 2023-10-24 10:09:49 +00:00
serxa
d3f54a29e3 add startup waits in all DDL queries and global DDL worker 2023-10-24 10:09:10 +00:00
Kseniia Sumarokova
4e0122a299
Merge pull request #54422 from kssenii/s3-queue-fixes 
Fixes for storage `S3Queue`
 2023-10-18 21:51:51 +02:00
Alexey Milovidov
3864c6746e
Merge pull request #55649 from ClickHouse/dashboard-ipad 
Add support for touch devices
 2023-10-17 23:10:21 +02:00
vdimir
1d46ed75db
Merge pull request #55558 from ClickHouse/vdimir/mssql_odbc_cursor 
Fix 'Invalid cursor state' in odbc interacting with MS SQL Server
 2023-10-17 16:37:08 +02:00
kssenii
4464c86895 Merge remote-tracking branch 'origin/master' into s3-queue-fixes 2023-10-17 11:16:52 +02:00
Alexey Milovidov
6a752199e4 Merge remote-tracking branch 'origin/master' into fix-garbage-config-index-mark-cache 2023-10-17 04:52:13 +02:00
Alexey Milovidov
b242cdbb42 Merge remote-tracking branch 'origin/master' into dashboard-ipad 2023-10-17 04:51:59 +02:00
Alexey Milovidov
1e6be1ac86 Merge branch 'master' into dashboard-ipad 2023-10-17 04:50:38 +02:00
Azat Khuzhin
4cb3de3110 Add pager property into ClientBase 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-16 22:31:55 +02:00
Azat Khuzhin
3257d5643d Move adjusting input_format_values_allow_data_after_semicolon into the client base 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-16 22:31:55 +02:00
Kseniia Sumarokova
96c518be5b
Merge branch 'master' into s3-queue-fixes 2023-10-16 22:19:13 +02:00
serxa
0e17da22eb Merge branch 'master' into async-loader-integration 2023-10-16 11:08:29 +00:00
vdimir
aae3894c23
Throw an exception in odbc-bridge if more than one table matched a single query 2023-10-16 10:43:36 +00:00
vdimir
cf1deb7bd5
Fix 'Invalid cursor state' in odbc interacting with MS SQL Server 2023-10-16 10:43:33 +00:00
Antonio Andelic
a3360cb3f5
Merge pull request #55485 from guoxiaolongzte/keeper_client_add_get_all_children_number_command 
keeper_client add get_all_children_number command
 2023-10-16 08:45:38 +02:00
Alexey Milovidov
99aa8e6197 Fix misconfiguration of the mark cache size for secondary indices 2023-10-16 07:26:10 +02:00
Alexey Milovidov
750d45bad3 Add support for touch devices 2023-10-16 05:26:14 +02:00
Alexey Milovidov
95be5c7c07 Show move/maximize only if there is more than a single chart 2023-10-16 04:57:42 +02:00
alesapin
3b02748cb6 Fix some typos 2023-10-15 15:43:02 +02:00
Nikolay Degterinsky
c1fba94d67 Resubmit #54421 
This reverts commit 3e60451ac7.
 2023-10-14 19:04:01 +00:00
Alexey Milovidov
16bf5d2946
Revert "Removed "maximize" and "drag" buttons from dashboard in case of single chart" 2023-10-14 04:21:07 +03:00
Alexey Milovidov
8697e78cd8
Merge pull request #55581 from bhavuk2002/master 
Removed "maximize" and "drag" buttons from `dashboard` in case of single chart
 2023-10-14 03:15:31 +02:00
Alexander Tokmakov
3e60451ac7 Revert "Merge pull request #54421 from evillique/cluster-groups" 
This reverts commit 211c80e1a0, reversing
changes made to b53a98217c.
 2023-10-13 18:55:51 +02:00
bhavuk2002
fd5c864d19 using queries.length 2023-10-13 21:49:59 +05:30
bhavuk2002
d440ec5212 updated if condition 2023-10-13 21:47:29 +05:30
bhavuk2002
bbcd8a42ae rectified if sysntax 2023-10-13 21:46:27 +05:30
Alexander Tokmakov
e3e105d154
Merge pull request #55527 from azat/values-eof-check-fix 
Fix checking of non handled data for Values format
 2023-10-13 18:07:02 +02:00
Alexander Tokmakov
e40aa72883
Merge pull request #54826 from JackyWoo/lb_nearest_host_name 
Add load balancing hostname_levenshtein_distance
 2023-10-13 17:23:02 +02:00
kssenii
d64b990712 Merge remote-tracking branch 'origin/master' into s3-queue-fixes 2023-10-13 12:13:56 +02:00
bhavuk2002
1deb830dab updated dashboard.html 2023-10-13 10:57:33 +05:30
robot-clickhouse-ci-1
211c80e1a0
Merge pull request #54421 from evillique/cluster-groups 
Add replica groups to Replicated database engine
 2023-10-12 18:14:42 +02:00
Nikolay Degterinsky
20e16be27b Fixes & improvements 2023-10-12 15:02:08 +00:00
Azat Khuzhin
2cbb069b68 Add ability to ignore data after semicolon in Values format 
This is required for client, to handle comments in multiquery mode.

v0: separate context for input format
v2: cannot use separate context since params and stuff are changed in global context
v3: do not sent this setting to the server (breaks queries for readonly profiles)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-12 14:55:26 +02:00
JackyWoo
1f553f3d65 Merge branch 'master' into lb_nearest_host_name 2023-10-12 10:19:17 +08:00
guoxiaolong
8b47bf5e7e fix CheckSytel 2023-10-12 09:58:42 +08:00
Yakov Olkhovskiy
0738984edf
Merge pull request #42510 from ClibMouse/Kusto-phase2-oss-pr 
Support Kusto Query Language dialect - phase 2
 2023-10-11 08:35:26 -04:00
guoxiaolong
31ff669edc change the code to for loop 2023-10-11 17:47:38 +08:00
guoxiaolong
18a6e624c9 fix description 2023-10-11 17:24:08 +08:00
Yong Wang
16f992a06e Kusto-phase2: remove fuzzing test from KQL 2023-10-10 23:50:58 -07:00
guoxiaolong
3df0e77ea3 keeper_client add get_all_children_number command 2023-10-11 10:28:28 +08:00
JackyWoo
784fe880eb Merge branch 'master' into lb_nearest_host_name 2023-10-07 09:04:26 +08:00
Alexey Milovidov
8f9a227de1
Merge pull request #54931 from yariks5s/s3_style_url 
s3-style URL fix
 2023-09-29 07:12:01 +03:00
Nikolay Degterinsky
6539d52b9e
Merge pull request #50372 from helifu/master3 
Support dynamic adjustment of log level in Keeper
 2023-09-28 23:47:49 +02:00
yariks5s
ad94e08b0b made using macroses 2023-09-28 15:17:15 +00:00
Yong Wang
8c9da6bacf Merge remote-tracking branch 'origin/master' into Kusto-phase2-oss-pr 2023-09-27 22:35:14 -07:00
Alexey Milovidov
5d04858c07 I don't understand why backup log is not enabled by default 2023-09-28 07:20:05 +02:00
yariks5s
adf1af63a5 added tests, moved domains to config 2023-09-27 18:53:05 +00:00
kssenii
6b191a1afe Better 2023-09-27 14:54:31 +02:00
Robert Schulze
2d750e9e2f
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks 2023-09-27 12:15:29 +00:00
JackyWoo
5da03e8ba5 Add load balancing test_hostname_levenshtein_distance 2023-09-27 10:46:39 +08:00
Robert Schulze
cde10fe7b5
Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks 2023-09-26 18:59:41 +00:00
Alexey Milovidov
29d3b34dfc
Merge pull request #54090 from ClickHouse/remove-current-status 
Remove CurrentStatusInfo
 2023-09-26 21:38:07 +03:00
George Gamezardashvili
0ce30ab6d5
SSH keys authentication (#41109) 
Added new type of authentication based on SSH keys. It works only for Native TCP protocol.

Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2023-09-26 17:50:19 +02:00
Robert Schulze
9fff447716
Re-enable clang-tidy checks 2023-09-26 09:34:12 +00:00
Yong Wang
c936c08835 Kusto-phase2: fix the CMakeList files 2023-09-25 13:43:01 -07:00
Nikolay Degterinsky
bb0375d970 Fix data race in SYSTEM STOP LISTEN 2023-09-25 21:04:12 +02:00
alesapin
53158fa789
Merge pull request #54939 from evillique/fix-race 
Fix data race in SYSTEM STOP LISTEN
 2023-09-25 11:28:24 +02:00
Alexey Milovidov
51ef0ec4bc Remove status_info from example 2023-09-24 21:02:23 +02:00
Alexey Milovidov
3d5e7aefe9 Remove status_info from testdata 2023-09-24 21:01:55 +02:00
Alexey Milovidov
580f3c1a01 Merge branch 'master' into remove-current-status 2023-09-24 20:56:17 +02:00
Nikolay Degterinsky
c85d25cfe1 Fix data race in SYSTEM STOP LISTEN 2023-09-22 20:20:21 +00:00
Alexey Milovidov
6a4e4cc361
Merge pull request #54744 from azat/server-shutdown 
Disable killing the server by systemd
 2023-09-22 18:04:20 +03:00
Igor Nikonov
2b9b6265da
Merge pull request #54824 from ClickHouse/remove-check-of-config-file-sizes 
Remove config files sizes check
 2023-09-21 13:37:07 +02:00
Igor Nikonov
b5a6c85981
Merge branch 'master' into remove-check-of-config-file-sizes 2023-09-20 18:40:03 +02:00
Azat Khuzhin
c706101891 Fix throttling of BACKUPs from/to S3 (in case native copy was not used) 
In some cases native copy is not possible, and such requests should be
throttled.

v0: copyS3FileNativeWithFallback
v2: revert v0 and pass write_settings
v3: pass read_settings to copyFile()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-20 18:28:43 +02:00
Kruglov Pavel
165b0f0afe
Merge pull request #54807 from ClickHouse/own 
Make clickhouse-local logging (server_logs_file) prepend timestamps etc
 2023-09-20 13:20:04 +02:00
Igor Nikonov
eb8f9f1e76 Remove config files sizes check 2023-09-20 09:21:12 +00:00
Michael Kolupaev
fa51399ede Make clickhouse-local logging (server_logs_file) prepend timestamps etc 2023-09-19 21:39:59 +00:00
Azat Khuzhin
dde0a88067 Document shutdown_wait_unfinished/shutdown_wait_unfinished_queries in config.xml 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-19 22:09:38 +02:00
Azat Khuzhin
4d46753541 Convert shutdown_wait_unfinished into ServerSettings 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-09-19 22:09:38 +02:00
Mikhail f. Shiryaev
2e8e4b5766
Fix broken tests for clickhouse-diagnostics 2023-09-19 12:51:34 +02:00
kssenii
10518a9c83 Merge remote-tracking branch 'upstream/master' into kssenii-patch-6 2023-09-18 13:00:26 +02:00
Robert Schulze
f5e8028bb1
Merge pull request #54642 from rschu1ze/broken-re2st 
Remove broken lockless variant of re2
 2023-09-17 15:30:57 +02:00
János Benjamin Antal
afc753bb2d Fix typos 2023-09-16 00:11:59 +00:00
János Benjamin Antal
d975ee7608 Fix argument parsing bug 2023-09-16 00:07:54 +00:00
János Benjamin Antal
16e2f7f0b9 Add optional version argument to rm 2023-09-16 00:03:46 +00:00
Kruglov Pavel
8d01137d73
Merge pull request #54498 from bakam412/bugfix/local_multiquery_parameter 
Bugfix/local multiquery parameter
 2023-09-15 11:46:08 +02:00
Robert Schulze
7b378dbad3
Remove broken lockless variant of re2 2023-09-14 16:40:42 +00:00
Sergei Trifonov
12c71be9ca
Merge branch 'master' into async-loader-integration 2023-09-14 15:29:07 +02:00
Antonio Andelic
6306180333
Merge pull request #53049 from ClickHouse/keeper-batch-flushes 
Try batching multiple flush requests in Keeper
 2023-09-14 11:39:33 +02:00
serxa
28e65883aa rename settings 2023-09-13 12:22:46 +00:00
bakam412
7f52704383 Fix local_multiquery_parameter and fix unit tests 2023-09-13 10:02:14 +08:00
pufit
a54a7b726d
Merge pull request #54201 from ClickHouse/pufit/keeper-client-reconfig 
Implementing `reconfig`, `sync`, `exists` commands for keeper-client
 2023-09-12 10:58:02 -04:00
vdimir
aea9ac7de2
Merge pull request #54503 from ClickHouse/vdimir/local_database_arg 
clickhouse-local support --database command line argument
 2023-09-12 11:40:40 +02:00
vdimir
5061da4e8a
clickhouse-local support --database command line argument 2023-09-11 12:12:02 +00:00
kssenii
beb9a75a68 A few more 2023-09-11 12:40:25 +02:00
Antonio Andelic
5207997bd4 Merge branch 'master' into keeper-batch-flushes 2023-09-08 13:05:47 +00:00
pufit
926310728f Fix issues 2023-09-07 22:48:04 -04:00
Robert Schulze
bff06d18aa
Merge pull request #54249 from rschu1ze/multiple_query_arguments 
Allow multiple `--query` parameters in clickhouse-client/local.
 2023-09-07 21:20:07 +02:00
serxa
a2f8f6bf29 working on review comments 2023-09-07 13:21:43 +00:00
pufit
fef80f176e Fix typo 2023-09-06 21:56:14 -04:00
pufit
ef2350cfc7 Implementing exists, sync commands. Better tests 2023-09-06 20:36:39 -04:00
Antonio Andelic
824b90f754 Merge branch 'master' into keeper-batch-flushes 2023-09-06 13:29:35 +00:00
pufit
ed43a8f1f6 Merge branch 'master' into pufit/keeper-client-reconfig 2023-09-05 23:20:12 -04:00
Alexey Milovidov
ee088a0386 Fix bad punctuation in Keeper logs 2023-09-06 03:31:46 +02:00
robot-ch-test-poll
2e7edf4413
Merge pull request #53638 from arenadata/ADQM-987 
Add `backup_log` system table
 2023-09-05 17:03:41 +02:00
vdimir
f7e57edea8
Merge pull request #54113 from ClickHouse/vdimir/cluser_discovery_init_fix 2023-09-05 11:49:57 +02:00
János Benjamin Antal
3f7d8f1cae
Merge pull request #54099 from Tiakon/master 
Add the default value of task-upload-force
 2023-09-05 11:32:53 +02:00
Victor Krasnov
132ab6067b Merge branch 'master' into ADQM-987 2023-09-05 10:01:44 +03:00
Alexey Milovidov
b2821106fd
Merge pull request #53923 from myrrc/docs/ch-disks 
Add docs for clickhouse-disks
 2023-09-05 02:10:18 +03:00
robot-clickhouse
926c5636dd
Merge pull request #42599 from ClickHouse/build-fuzzer-protocol 
libFuzzer: add CI fuzzers build, add tcp protocol fuzzer, fix other fuzzers.
 2023-09-04 22:41:54 +02:00
Robert Schulze
11fea581bc
Fix tests 2023-09-04 15:34:42 +00:00
Mike Kot
11b1053f8d fix 2023-09-04 15:01:43 +00:00
Yakov Olkhovskiy
d513ceaef5 fix 2023-09-04 13:14:12 +00:00
Victor Krasnov
78760639d2 Merge branch 'master' into ADQM-987 2023-09-04 13:59:20 +03:00
Tiakon
8a2ace2975
Update programs/copier/ClusterCopierApp.cpp 
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
 2023-09-04 18:27:07 +08:00
Tiakon
74dedf2a5d
Merge branch 'ClickHouse:master' into master 2023-09-04 17:42:16 +08:00
Antonio Andelic
365bc584ec Merge branch 'master' into keeper-batch-flushes 2023-09-04 07:24:21 +00:00
Alexey Milovidov
7cb4d52d92 Deprecate metadata cache, part 2 2023-09-03 03:56:11 +02:00
pufit
096b44e1e5 Implementing reconfig command. Parser improvements 2023-09-01 23:34:47 -04:00
Sergei Trifonov
cf5ea46713
Merge pull request #53338 from azat/throttling-fixes 
Fix IO throttling during copying whole directories
 2023-09-01 18:29:15 +02:00
Victor Krasnov
52cce27e81 Merge branch 'master' into ADQM-987 2023-09-01 18:00:20 +03:00
Yakov Olkhovskiy
361b21b416 fix fuzzers, cmake refactor, add target fuzzers 2023-09-01 14:20:50 +00:00
kssenii
3f20d1f578 Rename 2023-09-01 12:55:58 +02:00
Antonio Andelic
7739dd7157 Fix integration tests 2023-08-31 14:52:42 +00:00
Mike Kot
ebcc97ad1b change case for copy operation in clickhouse-disks 2023-08-31 13:39:32 +00:00
vdimir
b9bb4af6be
Support setting up fail points in config 2023-08-31 11:45:39 +00:00
Tiakon
8c21bd8342
Update ClusterCopierApp.cpp 
Shows that the type of task-upload-force is Boolean.
 2023-08-31 10:42:45 +08:00
Alexey Milovidov
bb47e2cc23 Remove CurrentStatusInfo 2023-08-30 20:56:14 +02:00
Alexander Tokmakov
4d70624ca3
Merge pull request #53907 from arenadata/ADQM-1126 
Added validate_tcp_client_information server setting
 2023-08-30 15:10:13 +02:00
Azat Khuzhin
51e8b16916 Fix IO throttling during copying whole directories 
v2: pass local context during ALTER TABLE MOVE PARTITION TO VOLUME/DISK as well
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-29 23:13:46 +02:00
Alexey Milovidov
62747ea20f
Merge pull request #53739 from Hexta/arm-clocksource 
Do not warn about arch_sys_counter clock
 2023-08-29 21:57:39 +03:00
Mike Kot
061bbe13c0 add docs for clickhouse-disks 2023-08-29 12:25:04 +00:00
Alexey Gerasimchuck
de1ed97876 disabled validate_tcp_client_information 2023-08-29 08:59:49 +00:00
robot-ch-test-poll
5ccc75b0f7
Merge pull request #53769 from ClickHouse/pufit/keeper-client-node-from-config 
KeeperClient auto-detection of keeper node from clickhouse configs
 2023-08-29 09:33:11 +02:00
pufit
7f4fc3744d resolve issues 2023-08-28 22:23:01 -04:00
Alexey Gerasimchuck
e9757abea6 Added validate_tcp_client_information setting 2023-08-29 00:15:55 +00:00
Sergei Trifonov
c10e42ca05
Merge branch 'master' into async-loader-integration 2023-08-28 20:20:31 +02:00
Artur Malchanau
41508e129d
Merge branch 'ClickHouse:master' into arm-clocksource 2023-08-28 10:50:57 +03:00
Yong Wang
e5f968b0d8 Kusto-phase2: enabled fuzzing test for dialect prql 2023-08-26 07:40:40 -07:00
Yong Wang
1ec574057a Kusto-phase2: fixed some functional test 2023-08-26 07:40:40 -07:00
Larry Luo
4f02667861 Fix AST Fuzzer 2023-08-26 07:40:39 -07:00
Yong Wang
5cc3f4708c Kusto-phase2: restore to kusto dialect only, fix some style issue 2023-08-26 07:40:39 -07:00
Yong Wang
523bcf550d Kusto-phase2: fix operators, unit test and functional test. 2023-08-26 07:40:39 -07:00
HeenaBansal2009
2ddc6cec9b removed unwanted file 2023-08-26 07:40:32 -07:00
HeenaBansal2009
a4d828e0cf Fixed failed testescases 2023-08-26 07:40:32 -07:00
Victor Krasnov
921f0c8b5d Merge branch 'master' into ADQM-987 2023-08-25 21:20:23 +03:00
pufit
091a27359a More careful logs handling 2023-08-25 12:22:21 -04:00
Kruglov Pavel
d80be4673b
Merge pull request #53692 from Avogar/cache-count 
Cache number of rows in files for count in file/s3/url/hdfs/azure functions
 2023-08-25 09:08:52 +02:00
Yakov Olkhovskiy
415a993c91 fix fuzzers build 2023-08-24 23:22:39 +00:00
Yakov Olkhovskiy
bb1e3aaa29 fix 2023-08-24 23:21:37 +00:00
Alexander Tokmakov
1eff7bee5e
Merge pull request #53673 from arenadata/ADQM-1105 
Added client_info validation to the query packet in TCPHandler.
 2023-08-24 21:03:25 +03:00
Kruglov Pavel
44db5fa992
Merge branch 'master' into cache-count 2023-08-24 17:21:18 +02:00
Arthur Passos
2bade7db08
Add global proxy setting (#51749) 
* initial impl

* fix env ut

* move ut directory

* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider

* minor adjustment

* add a few tests, still incomplete

* add proxy support for url table function

* use proxy for select from url as well

* remove optional from return type, just returns empty config

* fix style

* style

* black

* ohg boy

* rm in progress file

* god pls don't let me kill anyone

* ...

* add use_aws guards

* remove hard coded s3 proxy resolver

* add concurrency-mt-unsafe

* aa

* black

* add logging back

* revert change

* imrpove code a bit

* helper functions and separate tests

* for some reason, this env test is not working..

* formatting

* :)

* clangtidy

* lint

* revert some stupid things

* small test adjusmtments

* simplify tests

* rename test

* remove extra line

* freaking style change

* simplify a bit

* fix segfault & remove an extra call

* tightly couple proxy provider with context..

* remove useless include

* rename config prefix parameter

* simplify provider a bit

* organize provider a bit

* add a few comments

* comment out proxy env tests

* fix nullptr in unit tests

* make sure old storage proxy config is properly covered without global context instance

* move a few functions from class to anonymous namespace

* fix no fallback for specific storage conf

* change API to accept http method instead of bool

* implement http/https distinction in listresolver, any still not implemented

* implement http/https distinction in remote resolver

* progress on code, improve tests and add url function working test

* use protcol instead of method for http and https

* small fix

* few more adjustments

* fix style

* black

* move enum to proxyconfiguration

* wip

* fix build

* fix ut

* delete atomicroundrobin class

* remove stale include

* add some tests.. need to spend some more time on the design..

* change design a bit

* progress

* use existing context for tests

* rename aux function and fix ut

* ..

* rename test

* try to simplify tests a bit

* simplify tests a bit more

* attempt to fix tests, accept more than one remote resolver

* use proper log id

* try waiting for resolver

* proper wait logic

* black

* empty

* address a few comments

* refactor tests

* remove old tests

* baclk

* use RAII to set/unset env

* black

* clang tidy

* fix env proxy not respecting any

* use log trace

* fix wrong logic in getRemoteREsolver

* fix wrong logic in getRemoteREsolver

* fix test

* remove unwanted code

* remove ClientConfigurationperRequest and auxilary classes

* remove unwanted code

* remove adapter test

* few adjustments and add test for s3 storage conf  with new proxy settings

* black

* use chassert for context

* Add getenv comment
 2023-08-24 16:07:26 +03:00
pufit
8c1f0251e6 Get keeper node from clickhouse configs 2023-08-23 22:18:24 -04:00
Yakov Olkhovskiy
e9ccfb24f1
Merge branch 'master' into build-fuzzer-protocol 2023-08-23 17:13:35 -04:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count 2023-08-23 15:18:02 +02:00
Artur Malchanau
97cdd0c276
Fix clang-tidy warning 2023-08-23 15:35:30 +03:00
helifu
7154cb0ed1 Support dynamic adjustment of log level 2023-08-23 19:42:03 +08:00
Artur Malchanau
0fd26152e3
Do not warn about arch_sys_counter clock 2023-08-23 12:51:27 +03:00
Robert Schulze
b70f3dbe41
Same variable name in LocalServer.cpp as in Server.cpp 2023-08-22 15:44:26 +00:00
Victor Krasnov
258f9927b4 Merge branch 'master' into ADQM-987 2023-08-22 16:49:32 +03:00
avogar
7f9e81d504 Clean up 2023-08-22 12:55:00 +00:00
Robert Schulze
f2fb7365e6
Add settings for 1. size_ratio (all caches) and 2. policy (index caches) 
1. is an expert-level setting, default is 0.5, applies only to SLRU.

Also, I noticed that we expose cache policy settings for the mark and
the uncompresed cache but not for the index mark and the index
uncompressed cache. Changed that as well, it simplifies the code a bit.
 2023-08-22 11:04:29 +00:00
Robert Schulze
acaa2eee58
Merge pull request #53556 from ClibMouse/feature/big-endian-UUID-correction 
Change Big Endian-UUID to work the same as Little Endian-UUID
 2023-08-22 11:57:44 +02:00
Alexey Gerasimchuck
3140958132 Added client_info validation 2023-08-22 03:52:57 +00:00
Robert Schulze
583d9b33f0
Merge pull request #51446 from rschu1ze/recfgable-caches 
Runtime-reconfigurable cache sizes
 2023-08-21 23:27:21 +02:00
Nikita Mikhaylov
145c99ee94
Merge pull request #53280 from evillique/better-stop-listen 
Add EXCEPT clause to SYSTEM STOP LISTEN query
 2023-08-21 19:03:49 +02:00
Robert Schulze
96c8b250ce
Allow to configure cache sizes at runtime 2023-08-21 13:51:10 +00:00
Victor Krasnov
a7813260c0 Merge branch 'master' into ADQM-987 2023-08-21 14:45:13 +03:00
Alexey Milovidov
47b565861b Add maximize and drag options for the dashboard 2023-08-21 06:23:58 +02:00
Alexey Milovidov
11757f6540 No Grammarly for SQL 2023-08-20 13:42:00 +02:00
Alexey Milovidov
6895f441bc Include params in the editor 2023-08-20 09:32:42 +02:00
Alexey Milovidov
7dd0e1f888 Resolve TODO 2023-08-20 09:19:10 +02:00
Alexey Milovidov
1a4b54875a Add mass editor 2023-08-20 09:17:22 +02:00
Alexey Milovidov
b2462a1bb7 Add mass editor 2023-08-20 09:16:38 +02:00
Alexey Milovidov
fe610d0394 Fix bad JavaScript 2023-08-20 08:07:11 +02:00
Alexey Milovidov
59fced8e99 Fix errors in dashboard 2023-08-20 08:00:04 +02:00
Kseniia Sumarokova
3d2fb2eced
Merge pull request #53124 from kssenii/allowed-caches-dir-for-dynamic-disks 
Allow to restrict fs caches paths (mainly useful for dynamic disks)
 2023-08-19 13:30:23 +04:00
kssenii
5382442da0 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-19 08:44:36 +04:00
Alexey Milovidov
68e694815c The dashboard just got better 2023-08-19 01:38:22 +02:00
kothiga
f33c585bc5
Addressing feedback. 2023-08-18 13:50:31 -07:00
kothiga
b905703909
Correct to-from pairing for transformEndianness. 2023-08-18 11:25:08 -07:00
Austin Kothig
6b42975d33
Change BE-UUID to work the same as LE-UUID. Included high and low getters to provide cleaner code when accessing undertype. 2023-08-18 08:19:46 -07:00
pufit
2e69a1d3a2 Fix keeper default path check 2023-08-17 22:07:39 -04:00
robot-ch-test-poll
ce58b90ea1
Merge pull request #53455 from rschu1ze/respect-changed-cgroups-limit 
Respect cgroup memory limit when reloading configuration
 2023-08-18 00:03:35 +02:00
Alexander Tokmakov
89c365803f
Merge pull request #53424 from ClickHouse/exception_message_patterns6 
Less exceptions with runtime format string
 2023-08-17 17:58:04 +03:00
Robert Schulze
ec7daed52f
Merge pull request #50276 from ClibMouse/feature/mergetree-checksum-big-endian-support 
MergeTree & SipHash checksum big-endian support
 2023-08-17 13:27:18 +02:00
Robert Schulze
5253a7f650
Respect cgroup memory limit when reloading configuration 
Cgroups allows to change the amount of memory available to a process
while it runs. The previous logic calculated the amount of available
memory only once at server startup. As a result, memory thresholds set
via cgroups were not picked up when the settings changed. We now always
incorporate the current limits during re-configuraton.

Note 1: getMemoryAmount() opens/reads a file which is potentially
        expensive. Should be fine though since that happens only when
        the server configuration changes.

Note 2: An better approach would be to treat cgroup limit changes as
        another trigger for ClickHouse server re-configuration (which
        currently only happens when the config files change). Shied away
        from that for now because of the case that when the cgroup limit
        is lowered, there is no guarantee that ClickHouse can shrink the
        memory amount accordingly in time (afaik, it does so only lazily
        by denying new allocations). As a result, the OOM killer would
        kill the server. The same will happen with this PR but at a
        lower implementation complexity.
 2023-08-17 09:22:13 +00:00
Nikolay Degterinsky
a893f75884 Merge remote-tracking branch 'upstream/master' into better-stop-listen 2023-08-16 23:06:18 +00:00
Alexander Tokmakov
ba44d7260e fix 2023-08-16 00:20:28 +02:00
kssenii
164b4f0ae3 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-15 20:30:01 +04:00
Robert Schulze
8343ebd4ec
Clarify comment 2023-08-15 14:44:13 +00:00
Alexander Tokmakov
83c10dad18 Merge branch 'master' into exception_message_patterns6 2023-08-15 15:20:09 +02:00
Robert Schulze
f317903073
Merge branch 'ClickHouse:master' into consistency-fixes-for-caches 2023-08-15 10:25:08 +02:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
ltrk2
2846ea49b4
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-14 13:02:30 -04:00
Robert Schulze
7ed5166c8a
Cosmetics: Make re-cfg of query cache similar to re-cfg of other caches 
This doesn't change semantics, it only changes in which places the
configuration is parsed. This makes the logic more consistent across
caches.
 2023-08-14 15:42:32 +00:00
Robert Schulze
ed90463888
Add comment 2023-08-14 15:42:32 +00:00
Robert Schulze
bb53d635df
More consistency: Run same mark cache size check in ch-local 
Server.cpp had a check that the configured mark cache size is not 0. Run
the same check in LocalServer.cpp.
 2023-08-14 15:42:32 +00:00
Robert Schulze
f126e54b0d
Respect max_cache_size for all caches 
So far, max_cache_size was only respected in Server.cpp, and in this
file only for the mark cache and the uncompressed cache. This commit
respects it also for the other cache types in Server.cpp, as well for
all caches in LocalServer.cpp.
 2023-08-14 15:42:32 +00:00
Robert Schulze
54e54aed74
Cosmetics: Introduce constants for default cache sizes and policies 2023-08-14 15:42:32 +00:00
Mikhail f. Shiryaev
3655df0f40
Attempt to address reset ENV in init.d script 2023-08-14 16:32:52 +02:00
Victor Krasnov
247cd63ca1 Merge branch 'master' into ADQM-987 
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/InterpreterSystemQuery.cpp
 2023-08-14 10:33:56 +03:00
kssenii
c3b8bc0267 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-13 21:14:42 +04:00
Alexey Milovidov
38a391ef57 Fix tidy 2023-08-12 16:34:35 +02:00
Alexey Milovidov
435cbb44a5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into benchmark-connect-in-parallel 2023-08-12 02:43:03 +02:00
Alexey Milovidov
97ec6a0e42
Merge pull request #35263 from sunny19930321/copier-check-drop-partition 
clickhouse-copier add check drop partition
 2023-08-12 03:37:42 +03:00
Alexey Milovidov
46c606b678
Merge pull request #53072 from ClickHouse/fix-comment 
Fix a comment
 2023-08-12 03:06:15 +03:00
Kseniia Sumarokova
ceb74ad3cb
Merge branch 'master' into allowed-caches-dir-for-dynamic-disks 2023-08-11 16:49:02 +02:00
Alexey Milovidov
a55f9eae84 Merge branch 'master' of github.com:ClickHouse/ClickHouse into allow-creating-system-tables-at-startup 2023-08-11 14:45:03 +02:00
Nikita Mikhaylov
dbe13e3016
Merge pull request #53110 from rmarduga/master 
Add hints for HTTP handlers
 2023-08-11 14:09:33 +02:00
ltrk2
1dc1b54c68
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-11 07:48:49 -04:00
robot-ch-test-poll
58cfede9f9
Merge pull request #53285 from ClickHouse/better-concurrency 
Change the default of `max_concurrent_queries` from 100 to 1000
 2023-08-11 13:01:48 +02:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache 
Better usage of ccache
 2023-08-11 04:43:45 +03:00
Alexey Milovidov
3b9d2c7080 Simplification 2023-08-11 03:06:07 +02:00
Alexey Milovidov
736ff1c127 Merge branch 'master' into allow-creating-system-tables-at-startup 2023-08-11 01:25:49 +02:00
Alexey Milovidov
45655928d1 clickhouse-benchmark: connect in parallel 2023-08-10 23:39:06 +02:00
Kseniia Sumarokova
99f7a83385
Merge branch 'master' into allowed-caches-dir-for-dynamic-disks 2023-08-10 22:26:33 +02:00
Alexey Milovidov
3aca240854 Change the default of max_concurrent_queries from 100 to 1000 2023-08-10 20:59:57 +02:00