thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 00:22:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
130,472 Commits 9,167 Branches 1,781 Tags 2.1 GiB
8fe2cd1a7e
Commit Graph

912 Commits

Author SHA1 Message Date
Alexander Sapin
4695ec6802 Add an ability to specify allocations size for sampling memory profiler 2023-07-21 14:56:29 +02:00
Dmitry Kardymon
5124f74fcc Merge remote-tracking branch 'origin/master' into ADQM-996 2023-07-21 05:54:36 +00:00
alesapin
cbcd48979c Fix race one more time 2023-07-20 13:04:43 +02:00
serxa
02fc66db10 avoid using bg pool during non-async start 2023-07-19 17:37:53 +00:00
Sergei Trifonov
b8a46ff822
Merge branch 'master' into async-loader-integration 2023-07-19 18:45:41 +02:00
serxa
6bca79fdff add auto-resolution for all resolvable deadlock types 2023-07-19 16:19:28 +00:00
alesapin
8be1454b28 Fix race 2023-07-18 16:06:03 +02:00
Sergei Trifonov
850ab23991
Merge branch 'master' into async-loader-integration 2023-07-17 19:14:54 +02:00
Nikita Mikhaylov
ee0bbc0e54
Merge branch 'master' into headers-blacklist 2023-07-17 19:08:52 +02:00
Nikita Mikhaylov
11cc8b4adf
Merge branch 'master' into add_delay_for_replicated 2023-07-17 19:04:29 +02:00
Nikita Mikhaylov
d5b97e4cc1
Merge branch 'master' into deprecate-metadata-cache 2023-07-17 15:00:27 +02:00
Dmitry Kardymon
771b1f8f47 Prototype (working) 2023-07-12 15:48:09 +00:00
Mike Kot
3e8906e5a2 fix build for non-ssl systems 2023-07-11 12:23:10 +00:00
Mike Kot
062b1c464c watch for certificate file updates in configreloader 2023-07-11 10:04:43 +00:00
Azat Khuzhin
a10aa9ad50 Force libunwind usage (removes gcc_eh support) 
libunwind is reentrant and signal safe, and works faster then then
gcc_eh (plus it has some custom patches for problems that have been
found during it's usage in ClickHouse).

gcc_eh may be missing in the system (if gcc was not installed), and
even if it exists clickhouse uses -nodefaultlibs, so some care should be
made to make it work.

Also this library is tiny and there shouln't be any problem to require
it always (there is already tendency to require some contrib libraries,
i.e. poco).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-08 20:55:50 +02:00
Alexey Milovidov
e4dd603919
Merge pull request #51497 from ClickHouse/fix-shutdown-race 
Fix possible race on shutdown wait
 2023-07-08 03:03:45 +03:00
Alexander Sapin
e618dd05cc Fix clang tidy and race 2023-07-07 13:03:44 +02:00
Alexey Milovidov
e303afbffb
Merge branch 'master' into fix-incorrect-warning 2023-07-07 13:44:21 +03:00
Nikolay Degterinsky
e98d136243
Merge branch 'master' into headers-blacklist 2023-07-07 04:44:06 +02:00
Alexey Milovidov
5416b7b6df Fix incorrect log level = warning 2023-07-06 03:04:58 +02:00
alesapin
88d3e1723a Fixes and comments 2023-07-05 18:31:47 +02:00
alesapin
baee73fd96 Make shutdown of replicated tables softer 2023-07-05 18:11:25 +02:00
Alexey Milovidov
31f68c2c67 Merge branch 'master' into deprecate-metadata-cache 2023-07-04 22:28:14 +02:00
Nikolay Degterinsky
8dfa773f44
Merge branch 'master' into headers-blacklist 2023-06-30 23:40:17 +02:00
serxa
40f721ae4f fix possible race on shutdown wait 2023-06-27 17:14:33 +00:00
Sergei Trifonov
e2798b5221
Merge branch 'master' into async-loader-integration 2023-06-26 19:18:23 +02:00
Azat Khuzhin
5d43a64112 Initialize text_log earlier to capture table startup messages 
While I was investigating some issues, I noticed that messages from
table startup are not appears in system.text_log due to too late
initialization.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-06-24 13:16:16 +02:00
Alexey Milovidov
a92513048d Add a warning 2023-06-23 02:36:00 +02:00
Sergei Trifonov
02e0d7e204
Merge branch 'master' into async-loader-integration 2023-06-20 12:03:29 +02:00
Andrey Zvonov
07191ce10d
Merge branch 'master' into zvonand-implicit-tz 2023-06-20 00:24:53 +02:00
Nikolay Degterinsky
9a25958be8 Add HTTP header filtering 2023-06-15 13:49:49 +00:00
Alexey Milovidov
598501011f Relax "too many parts" further 2023-06-11 17:51:54 +02:00
Andrey Zvonov
d95be4162f
Merge branch 'master' into zvonand-implicit-tz 2023-06-08 18:34:45 +03:00
Sergei Trifonov
0cbcff958a
Merge branch 'master' into async-loader-integration 2023-06-07 18:55:04 +02:00
Anton Popov
9cd0d5e6db move settings to server_settings 2023-06-07 11:49:41 +00:00
Anton Popov
c980771c22 Merge remote-tracking branch 'upstream/master' into HEAD 2023-06-07 11:23:13 +00:00
Nikita Mikhaylov
e87348010d
Rework loading and removing of data parts for MergeTree tables. (#49474) 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-06-06 14:42:56 +02:00
Sergei Trifonov
3ee782a0ec
Merge branch 'master' into async-loader-integration 2023-06-02 18:27:19 +02:00
serxa
aceca2e210 fix server 2023-06-02 16:24:56 +00:00
Nikita Mikhaylov
73db383727
Limit the number of in-flight tasks for loading outdated parts (#50450) 
* Done

* Update programs/local/LocalServer.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>

* Bump

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-06-02 14:10:26 +02:00
serxa
5048050085 use pool ids instead of priorities in load jobs 2023-05-31 16:53:10 +00:00
serxa
e1045f76de rework async loader initialization and reconfiguration for bg/fg pools 2023-05-31 16:23:41 +00:00
Andrey Zvonov
87fc780c14
Merge branch 'master' into zvonand-implicit-tz 2023-05-30 13:13:46 +03:00
Sergei Trifonov
994cfaf15b
Merge branch 'master' into async-loader-integration 2023-05-22 19:03:11 +02:00
zvonand
7dccb51bc0 fix conflict and update 2023-05-19 23:40:51 +02:00
kssenii
791bb6cd4c Fix style check 2023-05-19 17:35:01 +02:00
kssenii
0eab528f9f Move common code 2023-05-19 16:23:56 +02:00
Anton Popov
69ec65ab33 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-16 10:30:47 +00:00
Azat Khuzhin
424a20ee1c Log with warning if the server was terminated forcefully 
In case of it is terminated forcefully it will not be terminated
gracefully (i.e. run dtors and stuff), and by using warning log level
those messages will go to clickhouse-server.err.log, in which messages
are kept for a longer period then in clickhouse-server.log (at least
because it contains only warnings, errors and fatals only).

This will help with investigating some obscure issues.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-15 14:20:42 +02:00
serxa
9bf5292415 refact 2023-05-12 19:49:47 +00:00
Sergei Trifonov
426184ef73
Merge branch 'master' into async-loader-integration 2023-05-12 19:55:24 +02:00
serxa
4d31051bb9 add async_load_databases server setting 2023-05-12 17:54:51 +00:00
serxa
529a702391 fix LocalServer fast-tests 2023-05-11 13:23:53 +00:00
Sema Checherinda
8470aa8fbb
Merge branch 'master' into convertations_to_timespan 2023-05-11 08:24:40 +02:00
serxa
6bbc061ba0 start loader, fix helpers, fix deadlock 2023-05-10 19:56:48 +00:00
Sema Checherinda
f2ad1122a1 fix convertation 2023-05-10 17:50:42 +00:00
Andrey Zvonov
9237b904be
Merge branch 'master' into zvonand-implicit-tz 2023-05-10 01:25:33 +02:00
Sergei Trifonov
b479637e2c
Merge branch 'master' into async-loader-integration 2023-05-09 13:04:36 +02:00
Alexey Milovidov
a8bdb20fc4
Merge pull request #48787 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks [Resubmit]
 2023-05-09 07:58:36 +03:00
Alexander Tokmakov
abf6c60ad2 Merge branch 'master' into fix_dictionaries_loading_order 2023-05-08 00:31:03 +02:00
Alexey Milovidov
cfb35cb251 Fix typo: "as much slots" -> "as many slots" 2023-05-07 04:42:46 +02:00
zvonand
8338d54c34 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-implicit-tz 2023-05-05 15:50:26 +02:00
Alexander Tokmakov
6cf3e5b77a fix race on loading dictionaries 2023-05-05 15:25:18 +02:00
Konstantin Bogdanov
9d2a27cb7b
Invert shutdown_wait_unfinished_queries (#49427) 
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-03 19:04:48 +02:00
Nikita Mikhaylov
954e3b724c
Speedup outdated parts loading (#49317) 2023-05-03 18:56:45 +02:00
serxa
6eda6e8585 implement async loading interface in database engines 2023-05-02 09:18:50 +00:00
Sergei Trifonov
8dab49f328
Merge branch 'master' into async-loader-integration 2023-05-01 14:04:39 +02:00
serxa
abaf73268b integrate with Server and TablesLoader 2023-05-01 11:56:00 +00:00
serxa
86122409e2 more work 2023-04-26 18:27:23 +00:00
Anton Popov
125e5c50b8 allow to flush async insert queue 2023-04-25 21:30:03 +00:00
Alexey Milovidov
98ae9be734
Revert "Added tests for ClickHouse apps help and fixed help issues" 2023-04-21 01:54:34 +03:00
Andrey Zvonov
984e8fde41
Merge branch 'master' into zvonand-implicit-tz 2023-04-20 23:13:17 +02:00
Ilya Yatsishin
ebb1b990f2
Merge pull request #45819 from qoega/clickhouse-help 2023-04-18 21:46:36 +02:00
Dmitry Novik
cf5d9a175a Revert "Merge pull request #48760 from ClickHouse/revert-46089-background-memory-tracker" 
This reverts commit a61ed33223, reversing
changes made to 5f01b8a2b5.
 2023-04-14 16:34:19 +02:00
zvonand
089306820f Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-implicit-tz 2023-04-14 13:21:49 +02:00
Robert Schulze
7a21d5888c
Remove -Wshadow suppression which leaked into global namespace 2023-04-13 08:46:40 +00:00
zvonand
2a9f28b73b resolve 2023-04-12 12:54:39 +02:00
zvonand
a9499eed79 moved getting server TZ DateLUT to separate place, upd tests and fix 2023-04-12 12:47:05 +02:00
Yatsishin Ilya
b5b65d2149 Merge remote-tracking branch 'origin' into clickhouse-help 2023-04-11 11:24:48 +00:00
Alexey Milovidov
09ea79aaf7 Add support for {server_uuid} macro 2023-04-09 03:04:26 +02:00
Antonio Andelic
f5e97fbb66 Merge branch 'master' into alternative-keeper-configs 2023-04-03 15:06:52 +00:00
filimonov
fbb22348ea
Refactor reading the pool setting & from server config. (#48055) 
After #36425 there was a lot of confusions/problems with configuring pools - when the message was confusing, and settings need to be ajusted in several places.
See some examples in #44251, #43351, #47900, #46515.

The commit includes the following changes:
1) Introduced a unified mechanism for reading pool sizes from the configuration file(s). Previously, pool sizes were read from the Context.cpp with fallbacks to profiles, whereas main_config_reloader in Server.cpp read them directly without fallbacks.
2) Corrected the data type for background_merges_mutations_concurrency_ratio. It should be float instead of int.
3) Refactored the default values for settings. Previously, they were defined in multiple places throughout the codebase, but they are now defined in one place (or two, to be exact: Settings.h and ServerSettings.h).
4) Improved documentation, including the correct message in system.settings.

Additionally make the code more conform with #46550.
 2023-03-30 16:44:11 +02:00
Antonio Andelic
d6cbc5d05b Fix tests 2023-03-23 15:01:51 +00:00
Antonio Andelic
a4d1cd514d Refactor 2023-03-23 08:58:56 +00:00
Yatsishin Ilya
5786aba5eb Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-22 14:08:22 +00:00
Antonio Andelic
65b99ec6a7 Merge branch 'master' into alternative-keeper-configs 2023-03-22 12:22:57 +00:00
Robert Schulze
75ca2dd0e2
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase 2023-03-15 10:55:53 +00:00
Yatsishin Ilya
45c5275bfb Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-14 17:27:25 +00:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors 
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
 2023-03-13 15:11:12 +00:00
Alexander Tokmakov
7b1b238d0b Revert "Merge pull request #25674 from amosbird/distributedreturnconnection" 
This reverts commit 5ffd99dfd4, reversing
changes made to 2796aa333f.
 2023-03-11 19:09:47 +01:00
Yatsishin Ilya
6cde37093a Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-08 13:34:18 +00:00
Nikita Mikhaylov
5c4da5aa4a
Use separate thread pool for IO operations for backups (#47174) 2023-03-03 20:05:42 +01:00
Yatsishin Ilya
d4304d2028 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-02 16:16:18 +00:00
pufit
29296cb004
Merge pull request #46550 from ClickHouse/46231-systerm-server-settings 
Implement `system.server_settings`
 2023-02-28 11:23:15 -05:00
bkuschel
77607ba0d4 Fix setenv string call 2023-02-27 08:01:15 -05:00
Alexey Milovidov
83c41755ed Remove unused header 2023-02-25 03:21:14 +01:00
pufit
1157bf2d9f Removed unnecessary ifs 2023-02-22 15:15:09 -05:00
Yatsishin Ilya
80857638c1 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-22 13:58:17 +00:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
pufit
ff4e23aca8
Update programs/server/Server.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-02-19 16:48:55 -05:00
pufit
4496db66b2 Implement system.server_settings 2023-02-17 23:46:09 -05:00
Alexander Tokmakov
3457b68dcc fix 2023-02-17 17:57:49 +01:00
Alexander Tokmakov
9feb448a5e fix a race condition 2023-02-16 14:41:18 +01:00
serxa
c58b165b0f add config option to select scheduling policy 2023-02-11 16:18:42 +00:00
Yatsishin Ilya
1baa15d603 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-07 15:46:59 +00:00
Robert Schulze
9b841d67e8
Merge remote-tracking branch 'origin/master' into qc-max_cache_size 2023-02-05 16:47:02 +00:00
Boris Kuschel
1d4cf4fe69
Add encryption support with openssl 2023-02-04 14:59:34 -05:00
Yatsishin Ilya
f4cfd8a2d9 Merge remote-tracking branch 'origin' into clickhouse-help 2023-02-03 20:08:23 +00:00
Anton Popov
cdbe145bc1
Merge pull request #45796 from CurtizJ/fix-leak-in-azure-sdk 
Fix test `test_azure_blob_storage_zero_copy_replication ` (memory leak in azure sdk)
 2023-02-03 14:16:19 +01:00
Robert Schulze
b1889b0284
Make the naming a little less confusing 2023-02-02 19:18:39 +00:00
Robert Schulze
216636f984
Make query cache global settings re-configurable 2023-02-01 10:35:19 +00:00
Yatsishin Ilya
98edb9f06b Update help for clickhouse tools and add test 2023-01-31 12:19:37 +00:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache" 
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
 2023-01-31 09:54:34 +00:00
Anton Popov
839cd614fb fix memory leak in azure sdk 2023-01-31 01:01:10 +00:00
Alexander Tokmakov
6ecae8388e Merge branch 'master' into exception_message_patterns4 2023-01-24 14:42:36 +01:00
Alexey Milovidov
cb1e8afda6
Merge pull request #45524 from kssenii/fix-cache-policies-getter 
Add proper default to cache policy getter
 2023-01-24 12:46:33 +03:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
kssenii
3a4d5ba07d Fix cache policy getter 2023-01-23 22:07:06 +01:00
Robert Schulze
4f90824347
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-17 22:49:53 +00:00
Smita Kulkarni
09241150a0 Updated recording of startup time till after starting last listener and added stopping the StopWatch - Record server startup time in ProfileEvents 2023-01-16 08:32:25 +01:00
Smita Kulkarni
a0fe26f506 Addressed review comments and updated name to ServerStartupMilliseconds - Record server startup time in ProfileEvents 2023-01-13 14:38:54 +01:00
Smita Kulkarni
cf5cb0da97 Record server startup time in ProfileEvents 
Implementation:
* Added ProfileEvents::ServerStartupTime.
* Recorded time from start of main till listening to sockets.
Testing:
* Added a test 02532_profileevents_server_startup_time.sql
 2023-01-13 13:47:54 +01:00
Robert Schulze
475bb31f33
Make server configuratio hierarchical 2023-01-12 21:24:40 +00:00
Robert Schulze
923fa2c15a
Fix review comments, pt. II 2023-01-10 10:21:08 +00:00
Robert Schulze
45dbcf88e5
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-06 20:42:48 +00:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
Robert Schulze
e9e04166d9
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-02 08:20:27 +00:00
Alexey Milovidov
b859cc2ddc Fix build 2023-01-02 03:27:21 +01:00
Alexey Milovidov
6e946cc757 Less OOM in stress test 2023-01-01 23:22:26 +01:00
李扬
56b6378c12
Update programs/server/Server.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-12-28 15:08:50 +08:00
taiyang-li
437c31eca3 merge master and fix conflicts 2022-12-28 15:02:37 +08:00
Alexey Milovidov
1c8fa6eb26
Merge pull request #44595 from ClickHouse/wrong-place-of-log-message 
The position of the log message about the server environment was wrong
 2022-12-27 16:00:37 +03:00
Alexey Milovidov
b218d549bf Fix bad log message 2022-12-26 13:56:09 +01:00
Alexey Milovidov
d08e7e5f2d The position of the log message about the server environment was wrong 2022-12-26 13:30:13 +01:00
kssenii
8308177361 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 12:14:46 +01:00
vdimir
511e8d698d
Rename config param tmp_cache -> temporary_data_in_cache 2022-12-22 10:22:56 +00:00
vdimir
d30d25dbbe
Temporary files evict fs cache 2022-12-22 10:22:49 +00:00
kssenii
6bd4f8c029 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-20 21:17:28 +01:00
Sergei Trifonov
6f7bba159f
Merge pull request #41840 from ClickHouse/io-scheduler 
IO scheduling subsystem
 2022-12-19 15:15:19 +01:00
Robert Schulze
b93c56323c
Merge remote-tracking branch 'origin/master' into query-result-cache 2022-12-19 09:18:36 +00:00
Alexey Milovidov
30fd9b8317
Merge pull request #43597 from ClickHouse/sd_notify_followup_fixes 
Followup fixes for systemd notification (#43400)
 2022-12-17 04:35:05 +03:00
kssenii
30547d2dcd Replace old named collections code for url 2022-12-17 00:24:05 +01:00
Boris Kuschel
b1e401eb93
Merge branch 'ClickHouse:master' into openssl_support 2022-12-16 16:40:08 -05:00
Boris Kuschel
1b7c5758d2
Fix openssl Bug 2022-12-16 12:31:17 -08:00
Sergei Trifonov
f61a98b5dc
Merge branch 'master' into io-scheduler 2022-12-16 15:55:08 +01:00
Robert Schulze
4db33f16e3
Make DoS attack on cache less likely 2022-12-15 21:42:51 +00:00
Robert Schulze
537c64aad2
Merge remote-tracking branch 'origin/master' into query-result-cache 
Conflicts:
- tests/queries/0_stateless/02117_show_create_table_system.reference
 2022-12-15 12:17:58 +00:00