thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-28 10:31:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
111,628 Commits 9,189 Branches 1,782 Tags 2.2 GiB
e18d6dfac1
Commit Graph

3303 Commits

Author SHA1 Message Date
Alexander Gololobov
3608763a30 Wait for parent process to send MAINPID=... notification to the service manager 2022-12-06 16:14:46 +01:00
Alexander Gololobov
688e6fe714 Send MAINPID= notification from the parent (watchdog) process to make systemd handle READY=1 notifiaction from the child 2022-12-05 23:03:40 +01:00
Alexander Gololobov
58fcf0fdb0
Merge branch 'master' into sd_notify_followup_fixes 2022-12-05 15:03:44 +01:00
kssenii
dca3444283 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-12-04 15:56:02 +01:00
Vladimir C
7d6950d397
Revert "Temporary files evict fs cache" 2022-12-02 14:50:56 +01:00
Anton Popov
5d9077cdb6 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-02 12:06:45 +00:00
Vladimir C
117d01f2ac
Merge pull request #43457 from ClickHouse/vdimir/tmp-data-in-fs-cache 2022-12-01 13:41:40 +01:00
vdimir
98fe3c6c02
Temporary files evict fs cache 2022-12-01 11:49:17 +00:00
Antonio Andelic
1f93183dc7
Merge pull request #43087 from ClickHouse/keeper-prometheus 
Add support for embedded Prometheus endpoint in Keeper
 2022-12-01 12:46:47 +01:00
Kruglov Pavel
f6f71ac5f1
Merge branch 'master' into fix_bug_in_copier 2022-12-01 12:29:50 +01:00
Nikolay Degterinsky
a7eed2fd6d Throw an exception if something is broken 2022-12-01 02:59:35 +00:00
Vladimir C
53dc70a2d0
Merge pull request #38191 from BigRedEye/grace_hash_join 
Closes https://github.com/ClickHouse/ClickHouse/issues/11596
 2022-11-30 17:01:00 +01:00
Antonio Andelic
2f501d5327 Merge branch 'master' into keeper-prometheus 2022-11-30 13:24:37 +01:00
Antonio Andelic
fbda86ddc8 PR fixes 2022-11-30 13:24:08 +01:00
Alexey Milovidov
72bf1115e9
Merge branch 'master' into password-reset-message 2022-11-29 23:51:46 +01:00
Nikolay Degterinsky
b675898d37 Small naming fix 2022-11-29 16:36:05 +00:00
Sema Checherinda
7feea8fd9b
Merge pull request #43278 from mdelapenya/bump-testcontainers-go 
chore: bump testcontainers-go to latest released version
 2022-11-29 14:23:15 +01:00
Antonio Andelic
678958603b
Merge branch 'master' into keeper-prometheus 2022-11-28 09:48:57 +01:00
Manuel de la Peña
0cd49e84c6
Merge branch 'master' into bump-testcontainers-go 2022-11-28 08:32:17 +01:00
Azat Khuzhin
c35eb86a37 Fix example of colored prompt in client 
After switching to replxx you don't need the following anymore:
- RL_PROMPT_START_IGNORE (\001)
- RL_PROMPT_END_IGNORE   (\002)

And those symbols also breaks the client prompt (the text length
calculated incorrectly, it does not interpret "\x01" as an escape
sequence but instead just a bunch of bytes and shows extra whitespaces
after prompt).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 20:30:00 +01:00
Azat Khuzhin
6381feb46d Add ability to increase timeout for clickhouse start/stop/restart 
Increasing timeout may help in case of slow server (debug, sanitizers).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 09:40:39 +01:00
Nikolay Degterinsky
3fd10e0011 Better solution 2022-11-25 22:38:28 +00:00
Nikolay Degterinsky
6825d85daa Add complexity rules for passwords 2022-11-25 22:38:05 +00:00
Alexey Milovidov
cf27a0cdeb Fix log messages in clickhouse-copier 2022-11-25 22:29:02 +01:00
Alexey Milovidov
f0a3196fa8 xMerge branch 'master' into password-reset-message 2022-11-25 22:26:23 +01:00
Anton Popov
c811f34a41
Simplify AsynchronousInsertQueue and fix race (#43233) 2022-11-25 15:02:22 +01:00
Manuel de la Peña
d651b97bbb fix: typo 2022-11-25 10:09:32 +01:00
Manuel de la Peña
6a5ffabecf chore: create container for each test 2022-11-25 08:52:49 +01:00
Manuel de la Peña
8a957e904d fix: read frame columns properly 2022-11-25 08:52:01 +01:00
Manuel de la Peña
73dd5102d6 chore: run clickhouse container only for the duration of the test 2022-11-25 08:23:35 +01:00
Manuel de la Peña
9be878aae2 Merge branch 'master' into bump-testcontainers-go 
* master: (331 commits)
  Revert "Fix endian issue in integer hex string conversion"
  Update sparse-primary-indexes.md to correct typo
  to MaterializeMySQL_support_drop_mulit_table-change ParserIdentifier
  to MaterializeMySQL_support_drop_mulit_table-clean up unnecessary code
  Make insertRangeFrom() more exception safe (#43338)
  Update docs/en/sql-reference/statements/create/view.md
  Update docs/en/sql-reference/statements/alter/projection.md
  Analyzer table functions untuple fix
  Revert "Add table_uuid to system.parts"
  Update 00176_bson_parallel_parsing.sh
  Update test.py
  doc fix
  bump lib for diag
  Update kafka.md
  check ast limits for create_parser_fuzzer
  Disable compressed marks and index in stress tests
  add a test with weird strings
  Update .reference
  Add information about written rows in progress indicator
  Avoid possible DROP hung due to attached web disk
  ...
 2022-11-25 08:17:17 +01:00
Igor Nikonov
20e67b7140 Merge remote-tracking branch 'origin/master' into HEAD 2022-11-24 13:10:37 +00:00
Antonio Andelic
a6f38cb1cc
Merge branch 'master' into keeper-prometheus 2022-11-24 12:12:08 +01:00
Sergei Trifonov
8186a06c28
Merge branch 'master' into io-scheduler 2022-11-24 12:02:52 +01:00
Raúl Marín
4d3217cb18
Merge branch 'master' into perf_experiment 2022-11-24 10:32:12 +01:00
lzydmxy
0462132dd6 Ensure consistency when copier update status and current_partition_attach_is_done after partition attach is done 2022-11-24 11:15:44 +08:00
Alexander Gololobov
7b4e0e5bd8 Properly handle EINTR; some more comments 2022-11-23 23:37:02 +01:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Dale McDiarmid
60f7a40c0b bump lib for diag 2022-11-23 09:25:58 +00:00
Alexey Milovidov
c20dbb7770 Move password reset message from client to server 2022-11-22 19:34:38 +01:00
Raúl Marín
4aa29b6a63 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-22 19:09:00 +01:00
Alexander Gololobov
c636c323ea
Merge pull request #43400 from socketpair/sd_notify 
Fix #8685 - added systemd sd_notify implementation
 2022-11-22 18:07:50 +01:00
Alexander Gololobov
32d66659e5
Merge pull request #43364 from azat/server-stacktrace-on-fail 
Provide full stacktrace in case of uncaught exception during server startup
 2022-11-22 15:07:57 +01:00
Raúl Marín
ad95846b2b AST fuzzer: Print data of INSERT VALUES 2022-11-22 13:10:53 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Dmitry Novik
b17dc24a94 Do not reuse jemalloc memory in test 2022-11-21 16:10:47 +00:00
Alexander Gololobov
63196a1e9e Close the socket; some cleanups 2022-11-21 16:05:47 +01:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Коренберг ☢️ Марк
6dbeee69d6 Fix ##8685 - added systemd sd_notify implementation 2022-11-20 19:34:12 +02:00
Azat Khuzhin
44f23c2568 Make disks checks only for clickhouse-server 
This will fix clickhouse-disks

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 16:28:35 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
Azat Khuzhin
eb3d21831d Provide full stacktrace in case of uncaught exception during server start 
Without it, it is hard to understand where the problem is.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-18 13:24:57 +01:00
kssenii
5d9b34b59d More tests 2022-11-17 15:34:35 +01:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Manuel de la Peña
00204d296c chore: check process in the container, not in the host 2022-11-16 08:20:40 +01:00
Manuel de la Peña
cb91c2f682 chore: bump testcontainer-go to latest released version 2022-11-16 08:14:58 +01:00
Alexey Milovidov
b4165f95a9
Merge branch 'master' into build/poco-leftovers 2022-11-16 06:09:58 +01:00
Vitaly Baranov
8d72f75556 Make the password wiping an option of IAST::format(). 2022-11-16 01:35:06 +01:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Alexey Milovidov
2c7377be92
Merge pull request #43129 from Algunenano/skip_hash_in_sanitizer 
Skip hash logging on sanitizer builds
 2022-11-15 02:16:27 +01:00
Alexey Milovidov
372059fe87
Merge pull request #43191 from ClickHouse/fix-git-import-indent 
Fix the `indent` field in the `git-import` tool
 2022-11-15 01:14:15 +01:00
Kseniia Sumarokova
e58a7140be
Merge pull request #43194 from ClickHouse/async-metrics-documentation 
Add documentation for `system.asynchronous_metrics`.
 2022-11-14 12:21:34 +01:00
Raúl Marín
65f00cc246 Skip binary hash calculation if there isn't a checksum reference 2022-11-14 12:17:21 +01:00
Alexander Tokmakov
87550e33bb
Merge pull request #43092 from ClickHouse/make-clickhouse-local-progress-off-by-default 
Disable clickhouse local and client non-interactive progress by default.
 2022-11-14 14:14:29 +03:00
Antonio Andelic
1635d120a0
Merge pull request #43201 from ClickHouse/fix-tsan-race-config-reload 
Stop `ConfigReloader` first to avoid data race
 2022-11-14 11:58:43 +01:00
Kseniia Sumarokova
79206c6eb0
Merge pull request #43147 from kssenii/named-collections-refactoring 
Add generic implementation for arbitrary structured named collections, access type and system.named_collections
 2022-11-14 11:57:17 +01:00
Antonio Andelic
38ee1a81a3 Stop config reloader before anything else 2022-11-13 19:17:41 +00:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Alexey Milovidov
e4a6f2a8e1 Fix the indent field in the git-import tool 2022-11-12 18:54:48 +01:00
Yakov Olkhovskiy
60d116729c adjustments for progress option 2022-11-12 02:55:26 +00:00
kssenii
63de577172 Better, add tests 2022-11-10 22:47:30 +01:00
Azat Khuzhin
953457de28 Remove POCO_CLICKHOUSE_PATCH 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-10 22:41:26 +01:00
kssenii
bed7ac9442 Support arbitrary structure collections 2022-11-10 17:05:24 +01:00
Antonio Andelic
8aeb406d82 Fix standalone build 2022-11-10 10:49:00 +01:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Antonio Andelic
36d763011b Compile standalone keeper 2022-11-10 09:23:52 +01:00
Nikolai Kochetov
3eb6c03b47 Disable clickhouse local and clickhouse client non-interactive progress by default. 2022-11-09 19:46:47 +00:00
Antonio Andelic
a0c12c2428 Define LOGICAL_ERROR 2022-11-09 16:06:25 +01:00
Antonio Andelic
1ed3930809 Add more metrics for Keeper 2022-11-09 15:51:41 +01:00
vdimir
e3f211a943 dashboard.html: add debug mode to add add_http_cors_header 2022-11-09 15:13:22 +01:00
vdimir
a977ea3071 dashboard.html: scroll and focus to new chart 2022-11-09 15:13:22 +01:00
vdimir
6d59fcaec6 dashboard.html: show reload button on connection params change 2022-11-09 15:13:22 +01:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Antonio Andelic
e82a5d43b5 Extract HTTPContext for HTTPServer 2022-11-09 09:02:04 +01:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
Alexey Milovidov
641f8576c6
Merge pull request #42003 from ClickHouse/write-progress-to-tty 
Write progress directly to terminal
 2022-11-05 16:51:37 +01:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast() 
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
 2022-11-04 15:52:48 +00:00
Yakov Olkhovskiy
b30f2ac12d allow '--progress err' to preserve old behavior, adjust tests 2022-11-03 03:16:55 +00:00
vdimir
6a4247ca32
Merge branch 'master' into grace_hash_join 2022-10-31 09:54:37 +00:00
Vitaly Baranov
842e40082f Remove query_masking_rules for encrypt() function from the default config (now it's always masked). 2022-10-31 10:50:33 +01:00
Anton Popov
d09039418f Merge remote-tracking branch 'upstream/master' into HEAD 2022-10-28 14:35:45 +00:00
Robert Schulze
31ef6c66cb
Merge pull request #42513 from ClickHouse/update-libcxx-to-15 
Build with libcxx(abi) 15
 2022-10-27 10:30:36 +02:00
vdimir
506bf2d225
Merge branch 'master' into grace_hash_join 2022-10-26 12:25:50 +00:00
Robert Schulze
e996e2ff15
Fix clang-tidy 2022-10-26 07:48:37 +00:00
Alexey Milovidov
47c32678ce
Merge pull request #42477 from AlfVII/enable-compiled-expressions-cache-in-local 
Enable CompiledExpressionCache in clickhouse-local
 2022-10-25 04:57:48 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Alexey Milovidov
cd43595ea9 Merge branch 'master' into write-progress-to-tty 2022-10-23 01:41:17 +02:00
Azat Khuzhin
dbb4f8216f Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 
Conflicts:
- src/Functions/GregorianDate.h
 2022-10-22 08:20:05 +02:00
Azat Khuzhin
905a95e166 Review fixes 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 22:40:13 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Nikolay Degterinsky
d7d0794d2f Better logs in clickhouse-disks 2022-10-20 23:17:29 +00:00
Alexey Milovidov
58f238007a
Merge pull request #42508 from ClickHouse/remove-database-macro-from-server-prompt 
Remove support for the `{database}` macro from the client's prompt
 2022-10-20 21:14:06 +02:00
Robert Schulze
f435e4fbe2
Un-clusterf$ck incomplete type errors 2022-10-20 14:48:46 +00:00
Robert Schulze
50c932e566
Separate TaskShard and TaskTable 2022-10-20 14:00:20 +00:00
Robert Schulze
df318d8719
Un-inline TaskTableAndShard 2022-10-20 13:43:15 +00:00
Robert Schulze
f6de964eaa
Fix static_assert in random generator 2022-10-20 11:20:18 +00:00
Robert Schulze
cdae7082a3
Up-inline ShardPartition, ShardPartitionPiece and TaskCluster 2022-10-20 11:16:05 +00:00
Alexey Milovidov
fabc8f5a18 Remove support for {database} macro from the client's prompt 2022-10-19 23:27:12 +02:00
Alfonso Martinez
168df7d9ba Removed trailing whitespace 2022-10-19 13:34:31 +02:00
Alfonso Martinez
0aa26db64c Added end of line co comply with Clangformat 2022-10-19 12:38:20 +02:00
Alfonso Martinez
c0be79f26c CompiledExpressionCache enabled 2022-10-19 12:27:00 +02:00
Antonio Andelic
ee2f5977d5
Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-18 10:02:50 +02:00
Alexey Milovidov
ddfaadeea4
Merge pull request #42325 from gingerwizard/go_update_diagnostics 
go update for diagnostics tool
 2022-10-17 19:30:16 +02:00
Yakov Olkhovskiy
3ce1fa09d6
Merge pull request #41198 from ClickHouse/composable-protocol 
Composable protocol
 2022-10-17 11:23:12 -04:00
vdimir
adb63a5583
Merge branch 'master' into grace_hash_join 2022-10-17 12:32:56 +00:00
Antonio Andelic
7c7395575f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf 
Refactor the implementation of user-defined functions
 2022-10-15 12:33:10 +02:00
Dale McDiarmid
980f79f9a7 go update 2022-10-14 15:32:34 +01:00
Nikolay Degterinsky
51e55db617
Fix log-level 2022-10-14 05:41:58 +02:00
Vitaly Baranov
2e59d671fc Split UserDefinedSQLObjectsLoader to interface and implementation. 2022-10-13 10:32:32 +02:00
vdimir
0178307c27 Followup for TemporaryDataOnDisk 2022-10-12 15:25:23 +02:00
Yakov Olkhovskiy
6c8e52eb67
Merge branch 'master' into composable-protocol 2022-10-08 20:33:32 -04:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Yakov Olkhovskiy
2229087896
Merge branch 'master' into composable-protocol 2022-10-07 02:23:26 -04:00
Yakov Olkhovskiy
fc1de0a56a move some functionality to Server::buildProtocolStackFromConfig 2022-10-07 01:10:33 +00:00
vdimir
5a7c493559
Merge branch 'master' into pr/BigRedEye/38191 2022-10-06 14:43:35 +00:00
Robert Schulze
78be400ac0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 2022-10-06 12:22:43 +00:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
Anton Popov
190e1f6830
Merge branch 'master' into async-inserts-log 2022-10-06 01:59:27 +02:00
Alexey Milovidov
25d13b89e9
Merge pull request #41142 from MeenaRenganathan22/OpenSSL_InHouse 
Changes to support the OpenSSL In-house build.
 2022-10-05 23:02:04 +03:00
vdimir
ff55c369bc
Merge branch 'tmp-data-followup' 2022-10-05 18:10:05 +00:00
vdimir
ccd29e172f
Followup for TemporaryDataOnDisk 2022-10-05 16:42:27 +00:00
Anton Popov
f61e98f0e8
Merge pull request #40668 from ClickHouse/async-insert-max-data-size-fix 
Use settings from query context for async INSERTs
 2022-10-05 15:15:19 +02:00
Anton Popov
8ca73eab03
Merge branch 'master' into fuzz-table-definitions 2022-10-04 15:10:31 +02:00
Anton Popov
148995894f add system table asynchronous_inserts_log 2022-10-03 21:35:55 +00:00
Nikolai Kochetov
8b581a6b3c
Merge pull request #41847 from ekonkov/sudo 
replace back clickhouse su with sudo -u in start command
 2022-10-03 14:42:29 +02:00
Robert Schulze
db5ef7b3cb
Merge branch 'master' into generated-file-cleanup 2022-10-02 23:13:18 +02:00
Alexey Milovidov
daab7f7f61
Merge pull request #41987 from ClickHouse/musl-4 
Preparation for Musl build, part 4
 2022-10-02 02:35:01 +03:00
Alexey Milovidov
9de9af54fe Merge branch 'master' of github.com:ClickHouse/ClickHouse into write-progress-to-tty 2022-10-01 23:25:07 +02:00
Alexey Milovidov
5d710e21f1 Render progress directly in tty 2022-10-01 23:19:36 +02:00
Alexey Milovidov
8e4531d135 Preparation for Musl build, part 4 2022-10-01 16:29:41 +02:00
root
7e4edfeb5a Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:46 -07:00
root
4318ec125e Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:22 -07:00
vdimir
7ebc297f4c
Merge branch 'master' into pr/BigRedEye/38191 2022-09-30 09:40:47 +00:00
Robert Schulze
cc92a2d174
Merge branch 'master' into generated-file-cleanup 2022-09-30 09:56:31 +02:00
vdimir
0f1a7c252d
better TemporaryDataOnDisk 2022-09-29 09:51:46 +00:00
vdimir
0093752271
Upd log message 2022-09-29 09:51:43 +00:00
vdimir
15c7a3be34
Temp data on disk: build 2022-09-29 09:51:41 +00:00
vdimir
c0898ce289
Use abstraction for temporary data on disk in Sort and Aggregation 2022-09-29 09:51:41 +00:00
Yakov Olkhovskiy
5e40f2ebca review suggestions 2022-09-29 00:13:40 +00:00
Alexey Milovidov
1ee890a0ed
Merge branch 'master' into sudo 2022-09-29 03:00:55 +03:00
Anton Popov
c97bec829a slightly better 2022-09-28 21:47:10 +00:00
Sergei Trifonov
8edac7eba1
Merge branch 'master' into io-scheduler 2022-09-28 22:17:17 +02:00
Anton Popov
34bc16cd5b avoid more useless errors 2022-09-28 15:55:39 +00:00
Anton Popov
4ac5df2540 drop all fuzzed tables 2022-09-28 14:52:10 +00:00
Yakov Olkhovskiy
b5768be1ca
Merge pull request #41743 from ClickHouse/fix-cert-err-msg2 
Add parameter --accept-invalid-certificate and set appropriate invalid certificate handling
 2022-09-28 10:13:13 -04:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Robert Schulze
6d70b4a1f6
Generate config_version.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Anton Popov
c8beae8237 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-28 12:47:02 +00:00
Alexey Milovidov
c0d7fa74e4
Merge branch 'master' into sudo 2022-09-28 14:16:49 +03:00
Alexey Milovidov
59b37bf4a0
Merge pull request #40684 from ClickHouse/add-missing-tests-geobase 
Add missing tests for legacy geobase
 2022-09-28 14:08:30 +03:00
Evgeny Konkov
b08ab05fbb fix typo in comment 2022-09-28 12:28:59 +03:00
Robert Schulze
06507c40de
${ConfigIncludePath} --> ${CONFIG_INCLUDE_PATH} 2022-09-28 08:28:47 +00:00
Robert Schulze
1885bb0524
Make comment consistent accross generated files 2022-09-28 08:11:09 +00:00
Robert Schulze
de3264691c
Merge pull request #41790 from ClickHouse/print-git-hash 
Log git hash during startup
 2022-09-28 09:43:17 +02:00
Alexey Milovidov
fcd443f0e3
Merge branch 'master' into add-missing-tests-geobase 2022-09-28 01:58:02 +03:00
Alexey Milovidov
3d9d1369b1
Merge pull request #41518 from ClickHouse/filimonov-config-xml-obsolete-comment 
Remove obsolete comment from the config.xml
 2022-09-28 01:05:56 +03:00
Alexey Milovidov
f8d2735b66
Update Install.cpp 2022-09-27 23:35:22 +03:00
Alexey Milovidov
6d6e88a5c8
Merge pull request #41798 from ClickHouse/small-fix-in-dashboard 
Small fix in dashboard
 2022-09-27 23:20:33 +03:00
Evgeny Konkov
0c9d7a8054 replace back clickhouse su with sudo in start command in order to respect limits in /etc/security/limits.conf 2022-09-27 21:57:56 +03:00
serxa
b057d07977 IO scheduling subsystem 2022-09-27 13:26:41 +00:00
Robert Schulze
588a5e5a42
Simplify a bit 2022-09-27 07:47:18 +00:00
Antonio Andelic
655bde8e3f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-27 07:28:26 +00:00
Alexey Milovidov
6d7de37e3d Small fix in dashboard 2022-09-26 23:52:14 +02:00
Robert Schulze
99725e68d1
Fix standalone keeper build 2022-09-26 19:28:27 +00:00
Robert Schulze
9711950c35
Fix build 2022-09-26 15:05:36 +00:00
vdimir
e6a2959b63
Do not print tons of unknown files on server statup 2022-09-26 13:45:55 +00:00
kssenii
f12811012e Fix integration tests 2022-09-26 12:49:20 +02:00
Nikita Mikhaylov
502338560c Added a test 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
636cedf488 Better 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
9b35d54e8f First try 2022-09-26 10:24:39 +00:00
Yakov Olkhovskiy
1d6c6c7144
change to --accept-invalid-certificate 2022-09-25 20:54:04 -04:00
Yakov Olkhovskiy
57e875d08c add parameter --accept-certificate, set appropriate invalid certificate handling 2022-09-24 04:46:19 +00:00
Nikolay Degterinsky
a7f3c7086c
Merge pull request #41341 from evillique/mandatory-identification 
Add a setting requiring no_password to be explicitly specified when creating a user
 2022-09-22 15:19:43 +02:00
Nikita Mikhaylov
364aa1b41e
Fix race between ~BackgroundSchedulePool and ~DNSCacheUpdater (#41654) 2022-09-22 13:02:45 +02:00
Alexey Milovidov
45afacdae4
Merge pull request #41186 from ClickHouse/fix-three-fourth-of-trash 
Fix more than half of the trash
 2022-09-22 07:28:26 +03:00
Antonio Andelic
eefbbf53e8 Extract S3 logic 2022-09-21 11:53:54 +00:00
Nikolay Degterinsky
7292d47923
Merge branch 'master' into mandatory-identification 2022-09-21 12:18:17 +02:00
Nikolay Degterinsky
f0ca365fa0 Better 2022-09-21 10:16:49 +00:00
Alexey Milovidov
2c69ec4317 Fix help message in static-files-disk-uploader 2022-09-21 05:14:31 +02:00
Alexey Milovidov
45bd3cfc30 Merge branch 'master' into fix-three-fourth-of-trash 2022-09-20 21:27:41 +02:00
Nikolai Kochetov
46c955a469
Merge pull request #41387 from ClickHouse/run-dns-cache-updater-earlier 
Enable DNS cache updater earlier.
 2022-09-20 12:04:05 +02:00
Antonio Andelic
22fe3b947e
Merge pull request #41520 from ClickHouse/fix-log-saving-for-stateless-tests 
Add option for loading only system tables with clickhouse-local
 2022-09-20 08:23:47 +02:00
Alexey Milovidov
28b3965797
Merge branch 'master' into filimonov-config-xml-obsolete-comment 2022-09-20 06:01:40 +03:00
Sergei Trifonov
cf2db48c29
Merge pull request #40631 from ClickHouse/readonly-settings-allow 
Allow to modify constrained settings in readonly mode
 2022-09-20 02:18:14 +02:00
Antonio Andelic
0f92e5e576 Add option for loading only system tables from path using clickhouse-local 2022-09-19 14:18:42 +00:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage 2022-09-19 15:26:21 +02:00
filimonov
6f956329d5
Remove obsolete comment from the config.xml 
Remove obsolete comment, see commit c059d0a0ee
 2022-09-19 15:26:11 +02:00
Yakov Olkhovskiy
2f2080f5a4
Merge branch 'master' into composable-protocol 2022-09-19 08:15:07 -04:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
Alexey Milovidov
bb6f1bfce2 Fix 9/10 of trash 2022-09-19 08:53:20 +02:00
Vitaly Baranov
5365b105cc Add SYSTEM RELOAD USERS command. 2022-09-18 12:44:00 +02:00
Alexey Milovidov
515972f459 Merge branch 'master' into forbid-dlopen-2 2022-09-18 02:20:22 +02:00
Alexey Milovidov
bac578b23a
Merge pull request #41428 from ClickHouse/remove-dlopen 
Remove `dlopen`
 2022-09-18 00:09:57 +03:00
Alexey Milovidov
d9d1b558e9 Fix clang-tidy 2022-09-17 18:40:22 +02:00
Alexey Milovidov
ada7a44ae4 Remove -WithTerminatingZero methods 2022-09-17 05:34:18 +02:00
Alexey Milovidov
ecd4c29c2a Forbid invocations of dlopen 2022-09-17 03:55:39 +02:00
Alexey Milovidov
35cce03125 Remove dlopen 2022-09-17 03:02:34 +02:00
Nikolai Kochetov
8422fd5791
Merge branch 'master' into run-dns-cache-updater-earlier 2022-09-16 20:48:51 +02:00
Dmitry Novik
eacc790050
Merge branch 'master' into fix-odr-vialation 2022-09-16 18:44:53 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
Eugene Konkov
ce0508e223
Increase open files limit (#41345) 
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2022-09-16 12:41:36 +02:00
Nikolai Kochetov
ee361c6d46 Enable DNS cache updater earlier. 2022-09-15 17:31:28 +00:00
Nikolay Degterinsky
72d8a04ee6 Add setting 'allow_implicit_no_password' 2022-09-15 08:07:42 +00:00
Yakov Olkhovskiy
b939379da6 bug fix, merge fix, style fix 2022-09-14 16:29:26 +00:00
Yakov Olkhovskiy
5e3fde60f9
Merge branch 'master' into composable-protocol 2022-09-14 11:35:23 -04:00
Robert Schulze
fd97058e45
fix: incorporate review comments 2022-09-14 15:21:24 +00:00
Yakov Olkhovskiy
d550604e28 respect listen_host config param, fix updateServers and getListenTry 2022-09-13 23:12:53 +00:00
Dmitry Novik
990ce888bf
Merge branch 'master' into fix-odr-vialation 2022-09-13 16:53:57 +02:00
Sergei Trifonov
5ee2f1701c
Merge branch 'master' into kvm-clock 2022-09-13 12:45:24 +02:00
Sergei Trifonov
795c1c1438 fix warning text 2022-09-13 12:44:42 +02:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading 
Allow to load marks in threadpool in advance
 2022-09-13 12:31:58 +02:00
Sergei Trifonov
85e43566b6
Merge branch 'master' into kvm-clock 2022-09-13 00:17:09 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS 
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
  a mechanism to update a model explicitly. It also saves potentially
  unnecessary reloads of a model from disk after it's initial load.

  To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
  was changed from eager to lazy. This means that both statements
  previously immedately reloaded the specified/all models, whereas now
  the statements only trigger an unload and the first call to
  catboostEvaluate() does the actual load.

- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
  fields removed. The view was not documented in the past and for now it
  remains undocumented. The commit is thus not considered a breach of
  ClickHouse's public interface.
 2022-09-12 19:33:02 +00:00
Meena Renganathan
51d6611b96 Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default 2022-09-12 09:05:38 -07:00
Azat Khuzhin
b698a4ff65 Apply changes to http handlers on fly without server restart 
This has been implemented by simply restarting http servers in case of
http_handlers directive in configuration xml had been changed.

But, for this I have to change the handlers interface to accept
configuration separatelly, since the configuration that contains in the
server is the configuration with which server had been started.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Antonio Andelic
 2022-09-12 17:34:51 +02:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow 2022-09-12 16:20:40 +02:00
Sergei Trifonov
2c78ff9697
Merge branch 'master' into kvm-clock 2022-09-12 14:59:18 +02:00
Sergei Trifonov
22ea533179 Do not warn about kvm-clock 2022-09-12 14:56:01 +02:00
kssenii
0bb6f8792a Fix 2022-09-12 14:06:26 +02:00
Yakov Olkhovskiy
0d6d9e7e44
Merge branch 'master' into composable-protocol 2022-09-11 18:59:57 -04:00
Yakov Olkhovskiy
7c855c9da2 style fix 2022-09-11 22:44:27 +00:00
Yakov Olkhovskiy
ffa7d3b121 cleanup 2022-09-11 21:51:25 +00:00
Yakov Olkhovskiy
7985f4ba16 cleanup, respect USE_SSL 2022-09-11 21:35:38 +00:00
Yakov Olkhovskiy
0c62b5acfc split into diferent files 2022-09-11 19:00:40 +00:00
Alexey Milovidov
9a0892c40c
Merge pull request #41171 from ClickHouse/fix-half-of-trash 
Fix half of trash
 2022-09-11 21:40:22 +03:00
Yakov Olkhovskiy
745e759146 bugs fixed, cleanup, working state 2022-09-11 14:32:37 +00:00
kssenii
b7d751b782 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-11 13:23:30 +02:00
Alexey Milovidov
5802c2fdd2
Merge pull request #40713 from ClickHouse/remove-useless-line 
Remove one line from XML, because we do not care
 2022-09-11 09:53:30 +03:00
Yakov Olkhovskiy
c3ac0c434b some refactoring 2022-09-11 04:40:58 +00:00
Yakov Olkhovskiy
d001baec87 pass section config key to a factory 2022-09-11 00:40:40 +00:00
Yakov Olkhovskiy
772bf050da add PROXYv1 handler, add stack exchange data block, tuneup protocols config 2022-09-10 20:21:37 +00:00
Alexey Milovidov
61278c81e4 Merge branch 'master' into remove-useless-line 2022-09-10 05:07:10 +02:00
Alexey Milovidov
fa62c7e982 Fix half of trash 2022-09-10 04:08:16 +02:00
Dmitry Novik
323fdeff0b
Merge branch 'master' into fix-odr-vialation 2022-09-09 00:42:14 +02:00
Dmitry Novik
29b5d023ee Replace WITH_TEXT_LOG by WITHOUT_TEXT_LOG 2022-09-08 19:16:37 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view 
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
 2022-09-08 09:10:11 +00:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge 
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
 2022-09-08 09:01:32 +00:00
Yakov Olkhovskiy
8a7fe2888a protocols configuration processing 2022-09-08 06:12:33 +00:00