thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-11 17:02:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
120,257 Commits 9,299 Branches 1,782 Tags 2.1 GiB
2f99363db0
Commit Graph

3587 Commits

Author SHA1 Message Date
Han Fei
8b29f8406b
Merge branch 'master' into hanfei/gwp-asan 2023-01-20 11:03:12 +01:00
Stig Bakken
420c179b55 Add <storage_policy> config parameter for system logs 2023-01-19 10:25:28 +01:00
Bharat Nallan Chakravarthy
16a2585d55 fixes to odbc connection pooling 2023-01-18 16:38:56 -08:00
Robert Schulze
4f90824347
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-17 22:49:53 +00:00
Nikolay Degterinsky
d414d4aa7a Merge remote-tracking branch 'upstream/master' into bcrypt 2023-01-16 22:14:03 +00:00
Nikolay Degterinsky
09558003c2 Make workfactor changeable 2023-01-16 22:11:15 +00:00
Smita Kulkarni
09241150a0 Updated recording of startup time till after starting last listener and added stopping the StopWatch - Record server startup time in ProfileEvents 2023-01-16 08:32:25 +01:00
Smita Kulkarni
a0fe26f506 Addressed review comments and updated name to ServerStartupMilliseconds - Record server startup time in ProfileEvents 2023-01-13 14:38:54 +01:00
Smita Kulkarni
cf5cb0da97 Record server startup time in ProfileEvents 
Implementation:
* Added ProfileEvents::ServerStartupTime.
* Recorded time from start of main till listening to sockets.
Testing:
* Added a test 02532_profileevents_server_startup_time.sql
 2023-01-13 13:47:54 +01:00
Robert Schulze
843264ae3e
Add QRC settings to default config.xml 2023-01-13 10:32:22 +00:00
Robert Schulze
475bb31f33
Make server configuratio hierarchical 2023-01-12 21:24:40 +00:00
Han Fei
f5e736e240 introduce llvm/gwp-asan 2023-01-12 15:00:26 +01:00
Anton Popov
ca80240d84 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-12 12:13:45 +00:00
Robert Schulze
923fa2c15a
Fix review comments, pt. II 2023-01-10 10:21:08 +00:00
Robert Schulze
45dbcf88e5
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-06 20:42:48 +00:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
Alexey Milovidov
3a027d285f
Merge pull request #44828 from ClickHouse/remove-two-lines-of-code 
What if I will remove two lines of code?
 2023-01-04 04:50:52 +03:00
Robert Schulze
27f5aad49e
What happens if I remove 156 lines of code? 2023-01-03 18:51:16 +00:00
Robert Schulze
cfb6feffde
What happens if I remove these 139 lines of code? 2023-01-03 18:35:31 +00:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
Robert Schulze
e9e04166d9
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-02 08:20:27 +00:00
Alexey Milovidov
b859cc2ddc Fix build 2023-01-02 03:27:21 +01:00
Alexey Milovidov
47ae8c5c79 Remove more lines 2023-01-02 02:06:11 +01:00
Alexey Milovidov
6e946cc757 Less OOM in stress test 2023-01-01 23:22:26 +01:00
Nikolay Degterinsky
dfe93b5d82
Merge pull request #42284 from Algunenano/perf_experiment 
Performance experiment
 2022-12-30 03:14:22 +01:00
Raúl Marín
e915ce1e95 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-12-28 20:15:43 +01:00
Nikolay Degterinsky
e814179663 Add default password type 2022-12-28 13:26:23 +00:00
李扬
56b6378c12
Update programs/server/Server.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-12-28 15:08:50 +08:00
李扬
2405291612
Update programs/copier/ClusterCopierApp.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-12-28 15:07:06 +08:00
李扬
4e3098bbc2
Update programs/extract-from-config/ExtractFromConfig.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-12-28 15:06:57 +08:00
taiyang-li
437c31eca3 merge master and fix conflicts 2022-12-28 15:02:37 +08:00
Azat Khuzhin
b7f92454bf Move LineReader/ReplxxLineReader out from base 
This will simplify linking dependencies, and anyway this classes are not
base in any sense.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-27 14:23:51 +01:00
Alexey Milovidov
514ff2ba20
Merge pull request #44570 from ClickHouse/clickhouse-benchmark-better-arguments 
Better command line argument name in `clickhouse-benchmark`
 2022-12-27 16:10:21 +03:00
Alexey Milovidov
e1115828d0
Merge pull request #44593 from ClickHouse/better-diagnostic-in-stress-test 
Better diagnostics on server stop for the stress test
 2022-12-27 16:07:43 +03:00
Alexey Milovidov
1c8fa6eb26
Merge pull request #44595 from ClickHouse/wrong-place-of-log-message 
The position of the log message about the server environment was wrong
 2022-12-27 16:00:37 +03:00
Alexey Milovidov
1b21cc018e
Merge pull request #44342 from ClickHouse/add-test-29883 
Add a test for #29883
 2022-12-27 14:55:14 +03:00
Raúl Marín
fc1fa82a39
Merge branch 'master' into perf_experiment 2022-12-27 10:51:58 +01:00
Alexey Milovidov
b218d549bf Fix bad log message 2022-12-26 13:56:09 +01:00
Alexey Milovidov
d08e7e5f2d The position of the log message about the server environment was wrong 2022-12-26 13:30:13 +01:00
Alexey Milovidov
e98f217b87 Better diagnostics on stop 2022-12-26 12:15:58 +01:00
Alexey Milovidov
d4864c7d38 Better command line argument name in clickhouse-benchmark 2022-12-24 20:40:23 +01:00
kssenii
8308177361 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 12:14:46 +01:00
vdimir
511e8d698d
Rename config param tmp_cache -> temporary_data_in_cache 2022-12-22 10:22:56 +00:00
vdimir
d30d25dbbe
Temporary files evict fs cache 2022-12-22 10:22:49 +00:00
Alexey Milovidov
a6f50edde3
Merge branch 'master' into add-test-29883 2022-12-21 01:45:18 +03:00
kssenii
6bd4f8c029 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-20 21:17:28 +01:00
Anton Popov
56187c5f0e Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-20 13:37:23 +00:00
Raúl Marín
45d27f461b
Merge branch 'master' into perf_experiment 2022-12-20 09:07:48 +00:00
Sergei Trifonov
6f7bba159f
Merge pull request #41840 from ClickHouse/io-scheduler 
IO scheduling subsystem
 2022-12-19 15:15:19 +01:00
Robert Schulze
b93c56323c
Merge remote-tracking branch 'origin/master' into query-result-cache 2022-12-19 09:18:36 +00:00
Alexey Milovidov
1795c583a1 Add a test for #29883 2022-12-18 08:00:32 +01:00
Alexey Milovidov
30fd9b8317
Merge pull request #43597 from ClickHouse/sd_notify_followup_fixes 
Followup fixes for systemd notification (#43400)
 2022-12-17 04:35:05 +03:00
kssenii
30547d2dcd Replace old named collections code for url 2022-12-17 00:24:05 +01:00
Boris Kuschel
b1e401eb93
Merge branch 'ClickHouse:master' into openssl_support 2022-12-16 16:40:08 -05:00
Boris Kuschel
1b7c5758d2
Fix openssl Bug 2022-12-16 12:31:17 -08:00
Sergei Trifonov
f61a98b5dc
Merge branch 'master' into io-scheduler 2022-12-16 15:55:08 +01:00
Anton Popov
b6f2754afb Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-16 14:37:34 +00:00
Robert Schulze
4db33f16e3
Make DoS attack on cache less likely 2022-12-15 21:42:51 +00:00
Robert Schulze
537c64aad2
Merge remote-tracking branch 'origin/master' into query-result-cache 
Conflicts:
- tests/queries/0_stateless/02117_show_create_table_system.reference
 2022-12-15 12:17:58 +00:00
Anton Popov
87e02a1744 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-14 02:08:10 +00:00
kssenii
9aa6d31bce Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code 2022-12-13 22:25:10 +01:00
kssenii
fae817863c Apply new code of named collections to s3 2022-12-13 22:19:09 +01:00
Nikolay Degterinsky
443bc3e656
Merge pull request #43719 from evillique/password_complexity_rules 
Add password complexity rules
 2022-12-13 17:56:45 +01:00
Nikolay Degterinsky
9b6d31b95d
Merge branch 'master' into perf_experiment 2022-12-13 17:15:07 +01:00
Robert Schulze
b78652e138
Query result cache 2022-12-12 10:37:10 +00:00
Nikolay Degterinsky
19e95e8cd8 Better implementation of password rules 2022-12-11 23:59:04 +00:00
Boris Kuschel
592c73e2c3
Merge branch 'master' into openssl_support 2022-12-09 20:21:22 -05:00
Nikolay Degterinsky
f2c07c0337 Merge remote-tracking branch 'upstream/master' into password_complexity_rules 2022-12-10 00:46:05 +00:00
alesapin
7a9d5a5818 Return pragma 2022-12-08 18:54:24 +01:00
alesapin
965e3519e9 Remove unused includes 2022-12-08 18:53:05 +01:00
alesapin
350bacec60 Followup 2022-12-08 18:43:54 +01:00
alesapin
7a223f4956 Slightly better clickhouse disks and remove DiskMemory 2022-12-08 18:20:54 +01:00
Boris Kuschel
cc3314acbe
Merge branch 'master' into openssl_support 2022-12-08 07:40:43 -05:00
Alexey Milovidov
9c81fd7afc
Merge pull request #43517 from ClickHouse/password-reset-message 
Move password reset message from client to server
 2022-12-07 23:26:58 +03:00
Boris Kuschel
b7698dc609 Fix style issues 2022-12-07 13:22:46 -05:00
Boris Kuschel
f03f95044e OpenSSL code changes 2022-12-07 13:22:46 -05:00
Azat Khuzhin
a8faf196c4
Add --max-consecutive-errors for clickhouse-benchmark (#43344) 
* Align Benchmark::Benchmark()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add --max-consecutive-errors for clickhouse-benchmark

Unlike --continue_on_errors, it will not leave the benchmark forever if
server is unavailable.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
 2022-12-07 14:51:16 +01:00
Kruglov Pavel
d4cd53ccea
Merge pull request #43602 from lzydmxy/fix_bug_in_copier 
Ensure consistency when copier update `status` and `attach_is_done`
 2022-12-07 13:47:56 +01:00
Alexander Gololobov
3608763a30 Wait for parent process to send MAINPID=... notification to the service manager 2022-12-06 16:14:46 +01:00
Alexander Gololobov
688e6fe714 Send MAINPID= notification from the parent (watchdog) process to make systemd handle READY=1 notifiaction from the child 2022-12-05 23:03:40 +01:00
Alexander Gololobov
58fcf0fdb0
Merge branch 'master' into sd_notify_followup_fixes 2022-12-05 15:03:44 +01:00
kssenii
dca3444283 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-12-04 15:56:02 +01:00
Vladimir C
7d6950d397
Revert "Temporary files evict fs cache" 2022-12-02 14:50:56 +01:00
Anton Popov
5d9077cdb6 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-02 12:06:45 +00:00
Vladimir C
117d01f2ac
Merge pull request #43457 from ClickHouse/vdimir/tmp-data-in-fs-cache 2022-12-01 13:41:40 +01:00
vdimir
98fe3c6c02
Temporary files evict fs cache 2022-12-01 11:49:17 +00:00
Antonio Andelic
1f93183dc7
Merge pull request #43087 from ClickHouse/keeper-prometheus 
Add support for embedded Prometheus endpoint in Keeper
 2022-12-01 12:46:47 +01:00
Kruglov Pavel
f6f71ac5f1
Merge branch 'master' into fix_bug_in_copier 2022-12-01 12:29:50 +01:00
Nikolay Degterinsky
a7eed2fd6d Throw an exception if something is broken 2022-12-01 02:59:35 +00:00
Vladimir C
53dc70a2d0
Merge pull request #38191 from BigRedEye/grace_hash_join 
Closes https://github.com/ClickHouse/ClickHouse/issues/11596
 2022-11-30 17:01:00 +01:00
Antonio Andelic
2f501d5327 Merge branch 'master' into keeper-prometheus 2022-11-30 13:24:37 +01:00
Antonio Andelic
fbda86ddc8 PR fixes 2022-11-30 13:24:08 +01:00
Alexey Milovidov
72bf1115e9
Merge branch 'master' into password-reset-message 2022-11-29 23:51:46 +01:00
Nikolay Degterinsky
b675898d37 Small naming fix 2022-11-29 16:36:05 +00:00
Sema Checherinda
7feea8fd9b
Merge pull request #43278 from mdelapenya/bump-testcontainers-go 
chore: bump testcontainers-go to latest released version
 2022-11-29 14:23:15 +01:00
Antonio Andelic
678958603b
Merge branch 'master' into keeper-prometheus 2022-11-28 09:48:57 +01:00
Manuel de la Peña
0cd49e84c6
Merge branch 'master' into bump-testcontainers-go 2022-11-28 08:32:17 +01:00
Azat Khuzhin
c35eb86a37 Fix example of colored prompt in client 
After switching to replxx you don't need the following anymore:
- RL_PROMPT_START_IGNORE (\001)
- RL_PROMPT_END_IGNORE   (\002)

And those symbols also breaks the client prompt (the text length
calculated incorrectly, it does not interpret "\x01" as an escape
sequence but instead just a bunch of bytes and shows extra whitespaces
after prompt).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 20:30:00 +01:00
Azat Khuzhin
6381feb46d Add ability to increase timeout for clickhouse start/stop/restart 
Increasing timeout may help in case of slow server (debug, sanitizers).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-27 09:40:39 +01:00
Nikolay Degterinsky
3fd10e0011 Better solution 2022-11-25 22:38:28 +00:00
Nikolay Degterinsky
6825d85daa Add complexity rules for passwords 2022-11-25 22:38:05 +00:00
Alexey Milovidov
cf27a0cdeb Fix log messages in clickhouse-copier 2022-11-25 22:29:02 +01:00
Alexey Milovidov
f0a3196fa8 xMerge branch 'master' into password-reset-message 2022-11-25 22:26:23 +01:00
Anton Popov
c811f34a41
Simplify AsynchronousInsertQueue and fix race (#43233) 2022-11-25 15:02:22 +01:00
Manuel de la Peña
d651b97bbb fix: typo 2022-11-25 10:09:32 +01:00
Manuel de la Peña
6a5ffabecf chore: create container for each test 2022-11-25 08:52:49 +01:00
Manuel de la Peña
8a957e904d fix: read frame columns properly 2022-11-25 08:52:01 +01:00
Manuel de la Peña
73dd5102d6 chore: run clickhouse container only for the duration of the test 2022-11-25 08:23:35 +01:00
Manuel de la Peña
9be878aae2 Merge branch 'master' into bump-testcontainers-go 
* master: (331 commits)
  Revert "Fix endian issue in integer hex string conversion"
  Update sparse-primary-indexes.md to correct typo
  to MaterializeMySQL_support_drop_mulit_table-change ParserIdentifier
  to MaterializeMySQL_support_drop_mulit_table-clean up unnecessary code
  Make insertRangeFrom() more exception safe (#43338)
  Update docs/en/sql-reference/statements/create/view.md
  Update docs/en/sql-reference/statements/alter/projection.md
  Analyzer table functions untuple fix
  Revert "Add table_uuid to system.parts"
  Update 00176_bson_parallel_parsing.sh
  Update test.py
  doc fix
  bump lib for diag
  Update kafka.md
  check ast limits for create_parser_fuzzer
  Disable compressed marks and index in stress tests
  add a test with weird strings
  Update .reference
  Add information about written rows in progress indicator
  Avoid possible DROP hung due to attached web disk
  ...
 2022-11-25 08:17:17 +01:00
Igor Nikonov
20e67b7140 Merge remote-tracking branch 'origin/master' into HEAD 2022-11-24 13:10:37 +00:00
Antonio Andelic
a6f38cb1cc
Merge branch 'master' into keeper-prometheus 2022-11-24 12:12:08 +01:00
Sergei Trifonov
8186a06c28
Merge branch 'master' into io-scheduler 2022-11-24 12:02:52 +01:00
Raúl Marín
4d3217cb18
Merge branch 'master' into perf_experiment 2022-11-24 10:32:12 +01:00
lzydmxy
0462132dd6 Ensure consistency when copier update status and current_partition_attach_is_done after partition attach is done 2022-11-24 11:15:44 +08:00
Alexander Gololobov
7b4e0e5bd8 Properly handle EINTR; some more comments 2022-11-23 23:37:02 +01:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Dale McDiarmid
60f7a40c0b bump lib for diag 2022-11-23 09:25:58 +00:00
Alexey Milovidov
c20dbb7770 Move password reset message from client to server 2022-11-22 19:34:38 +01:00
Raúl Marín
4aa29b6a63 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-22 19:09:00 +01:00
Alexander Gololobov
c636c323ea
Merge pull request #43400 from socketpair/sd_notify 
Fix #8685 - added systemd sd_notify implementation
 2022-11-22 18:07:50 +01:00
Alexander Gololobov
32d66659e5
Merge pull request #43364 from azat/server-stacktrace-on-fail 
Provide full stacktrace in case of uncaught exception during server startup
 2022-11-22 15:07:57 +01:00
Raúl Marín
ad95846b2b AST fuzzer: Print data of INSERT VALUES 2022-11-22 13:10:53 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Dmitry Novik
b17dc24a94 Do not reuse jemalloc memory in test 2022-11-21 16:10:47 +00:00
Alexander Gololobov
63196a1e9e Close the socket; some cleanups 2022-11-21 16:05:47 +01:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Коренберг ☢️ Марк
6dbeee69d6 Fix ##8685 - added systemd sd_notify implementation 2022-11-20 19:34:12 +02:00
Azat Khuzhin
44f23c2568 Make disks checks only for clickhouse-server 
This will fix clickhouse-disks

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 16:28:35 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
Azat Khuzhin
eb3d21831d Provide full stacktrace in case of uncaught exception during server start 
Without it, it is hard to understand where the problem is.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-18 13:24:57 +01:00
kssenii
5d9b34b59d More tests 2022-11-17 15:34:35 +01:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Manuel de la Peña
00204d296c chore: check process in the container, not in the host 2022-11-16 08:20:40 +01:00
Manuel de la Peña
cb91c2f682 chore: bump testcontainer-go to latest released version 2022-11-16 08:14:58 +01:00
Alexey Milovidov
b4165f95a9
Merge branch 'master' into build/poco-leftovers 2022-11-16 06:09:58 +01:00
Vitaly Baranov
8d72f75556 Make the password wiping an option of IAST::format(). 2022-11-16 01:35:06 +01:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
李扬
fc7fcafe40
Merge branch 'master' into alternative_keeper 2022-11-15 10:34:07 +08:00
taiyang-li
64d7087084 merge master and fix conflict 2022-11-15 10:33:10 +08:00
Alexey Milovidov
2c7377be92
Merge pull request #43129 from Algunenano/skip_hash_in_sanitizer 
Skip hash logging on sanitizer builds
 2022-11-15 02:16:27 +01:00
Alexey Milovidov
372059fe87
Merge pull request #43191 from ClickHouse/fix-git-import-indent 
Fix the `indent` field in the `git-import` tool
 2022-11-15 01:14:15 +01:00
Kseniia Sumarokova
e58a7140be
Merge pull request #43194 from ClickHouse/async-metrics-documentation 
Add documentation for `system.asynchronous_metrics`.
 2022-11-14 12:21:34 +01:00
Raúl Marín
65f00cc246 Skip binary hash calculation if there isn't a checksum reference 2022-11-14 12:17:21 +01:00
Alexander Tokmakov
87550e33bb
Merge pull request #43092 from ClickHouse/make-clickhouse-local-progress-off-by-default 
Disable clickhouse local and client non-interactive progress by default.
 2022-11-14 14:14:29 +03:00
Antonio Andelic
1635d120a0
Merge pull request #43201 from ClickHouse/fix-tsan-race-config-reload 
Stop `ConfigReloader` first to avoid data race
 2022-11-14 11:58:43 +01:00
Kseniia Sumarokova
79206c6eb0
Merge pull request #43147 from kssenii/named-collections-refactoring 
Add generic implementation for arbitrary structured named collections, access type and system.named_collections
 2022-11-14 11:57:17 +01:00
Antonio Andelic
38ee1a81a3 Stop config reloader before anything else 2022-11-13 19:17:41 +00:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Alexey Milovidov
e4a6f2a8e1 Fix the indent field in the git-import tool 2022-11-12 18:54:48 +01:00
Yakov Olkhovskiy
60d116729c adjustments for progress option 2022-11-12 02:55:26 +00:00
kssenii
63de577172 Better, add tests 2022-11-10 22:47:30 +01:00
Azat Khuzhin
953457de28 Remove POCO_CLICKHOUSE_PATCH 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-10 22:41:26 +01:00
kssenii
bed7ac9442 Support arbitrary structure collections 2022-11-10 17:05:24 +01:00
Antonio Andelic
8aeb406d82 Fix standalone build 2022-11-10 10:49:00 +01:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Antonio Andelic
36d763011b Compile standalone keeper 2022-11-10 09:23:52 +01:00
Nikolai Kochetov
3eb6c03b47 Disable clickhouse local and clickhouse client non-interactive progress by default. 2022-11-09 19:46:47 +00:00
Antonio Andelic
a0c12c2428 Define LOGICAL_ERROR 2022-11-09 16:06:25 +01:00
Antonio Andelic
1ed3930809 Add more metrics for Keeper 2022-11-09 15:51:41 +01:00
vdimir
e3f211a943 dashboard.html: add debug mode to add add_http_cors_header 2022-11-09 15:13:22 +01:00
vdimir
a977ea3071 dashboard.html: scroll and focus to new chart 2022-11-09 15:13:22 +01:00
vdimir
6d59fcaec6 dashboard.html: show reload button on connection params change 2022-11-09 15:13:22 +01:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Antonio Andelic
e82a5d43b5 Extract HTTPContext for HTTPServer 2022-11-09 09:02:04 +01:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
Alexey Milovidov
641f8576c6
Merge pull request #42003 from ClickHouse/write-progress-to-tty 
Write progress directly to terminal
 2022-11-05 16:51:37 +01:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast() 
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
 2022-11-04 15:52:48 +00:00
Yakov Olkhovskiy
b30f2ac12d allow '--progress err' to preserve old behavior, adjust tests 2022-11-03 03:16:55 +00:00
vdimir
6a4247ca32
Merge branch 'master' into grace_hash_join 2022-10-31 09:54:37 +00:00
Vitaly Baranov
842e40082f Remove query_masking_rules for encrypt() function from the default config (now it's always masked). 2022-10-31 10:50:33 +01:00
Anton Popov
d09039418f Merge remote-tracking branch 'upstream/master' into HEAD 2022-10-28 14:35:45 +00:00
Robert Schulze
31ef6c66cb
Merge pull request #42513 from ClickHouse/update-libcxx-to-15 
Build with libcxx(abi) 15
 2022-10-27 10:30:36 +02:00
vdimir
506bf2d225
Merge branch 'master' into grace_hash_join 2022-10-26 12:25:50 +00:00
Robert Schulze
e996e2ff15
Fix clang-tidy 2022-10-26 07:48:37 +00:00
Alexey Milovidov
47c32678ce
Merge pull request #42477 from AlfVII/enable-compiled-expressions-cache-in-local 
Enable CompiledExpressionCache in clickhouse-local
 2022-10-25 04:57:48 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Alexey Milovidov
cd43595ea9 Merge branch 'master' into write-progress-to-tty 2022-10-23 01:41:17 +02:00
Azat Khuzhin
dbb4f8216f Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 
Conflicts:
- src/Functions/GregorianDate.h
 2022-10-22 08:20:05 +02:00
Azat Khuzhin
905a95e166 Review fixes 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 22:40:13 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Nikolay Degterinsky
d7d0794d2f Better logs in clickhouse-disks 2022-10-20 23:17:29 +00:00
Alexey Milovidov
58f238007a
Merge pull request #42508 from ClickHouse/remove-database-macro-from-server-prompt 
Remove support for the `{database}` macro from the client's prompt
 2022-10-20 21:14:06 +02:00
Robert Schulze
f435e4fbe2
Un-clusterf$ck incomplete type errors 2022-10-20 14:48:46 +00:00
Robert Schulze
50c932e566
Separate TaskShard and TaskTable 2022-10-20 14:00:20 +00:00
Robert Schulze
df318d8719
Un-inline TaskTableAndShard 2022-10-20 13:43:15 +00:00
Robert Schulze
f6de964eaa
Fix static_assert in random generator 2022-10-20 11:20:18 +00:00
Robert Schulze
cdae7082a3
Up-inline ShardPartition, ShardPartitionPiece and TaskCluster 2022-10-20 11:16:05 +00:00
Alexey Milovidov
fabc8f5a18 Remove support for {database} macro from the client's prompt 2022-10-19 23:27:12 +02:00
Alfonso Martinez
168df7d9ba Removed trailing whitespace 2022-10-19 13:34:31 +02:00
Alfonso Martinez
0aa26db64c Added end of line co comply with Clangformat 2022-10-19 12:38:20 +02:00
Alfonso Martinez
c0be79f26c CompiledExpressionCache enabled 2022-10-19 12:27:00 +02:00
Antonio Andelic
ee2f5977d5
Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-18 10:02:50 +02:00
Alexey Milovidov
ddfaadeea4
Merge pull request #42325 from gingerwizard/go_update_diagnostics 
go update for diagnostics tool
 2022-10-17 19:30:16 +02:00
Yakov Olkhovskiy
3ce1fa09d6
Merge pull request #41198 from ClickHouse/composable-protocol 
Composable protocol
 2022-10-17 11:23:12 -04:00
vdimir
adb63a5583
Merge branch 'master' into grace_hash_join 2022-10-17 12:32:56 +00:00
Antonio Andelic
7c7395575f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf 
Refactor the implementation of user-defined functions
 2022-10-15 12:33:10 +02:00
Dale McDiarmid
980f79f9a7 go update 2022-10-14 15:32:34 +01:00
Nikolay Degterinsky
51e55db617
Fix log-level 2022-10-14 05:41:58 +02:00
Vitaly Baranov
2e59d671fc Split UserDefinedSQLObjectsLoader to interface and implementation. 2022-10-13 10:32:32 +02:00
vdimir
0178307c27 Followup for TemporaryDataOnDisk 2022-10-12 15:25:23 +02:00
Yakov Olkhovskiy
6c8e52eb67
Merge branch 'master' into composable-protocol 2022-10-08 20:33:32 -04:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Yakov Olkhovskiy
2229087896
Merge branch 'master' into composable-protocol 2022-10-07 02:23:26 -04:00
Yakov Olkhovskiy
fc1de0a56a move some functionality to Server::buildProtocolStackFromConfig 2022-10-07 01:10:33 +00:00
vdimir
5a7c493559
Merge branch 'master' into pr/BigRedEye/38191 2022-10-06 14:43:35 +00:00
Robert Schulze
78be400ac0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 2022-10-06 12:22:43 +00:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
Anton Popov
190e1f6830
Merge branch 'master' into async-inserts-log 2022-10-06 01:59:27 +02:00
Alexey Milovidov
25d13b89e9
Merge pull request #41142 from MeenaRenganathan22/OpenSSL_InHouse 
Changes to support the OpenSSL In-house build.
 2022-10-05 23:02:04 +03:00
vdimir
ff55c369bc
Merge branch 'tmp-data-followup' 2022-10-05 18:10:05 +00:00
vdimir
ccd29e172f
Followup for TemporaryDataOnDisk 2022-10-05 16:42:27 +00:00
Anton Popov
f61e98f0e8
Merge pull request #40668 from ClickHouse/async-insert-max-data-size-fix 
Use settings from query context for async INSERTs
 2022-10-05 15:15:19 +02:00
Anton Popov
8ca73eab03
Merge branch 'master' into fuzz-table-definitions 2022-10-04 15:10:31 +02:00
Anton Popov
148995894f add system table asynchronous_inserts_log 2022-10-03 21:35:55 +00:00
Nikolai Kochetov
8b581a6b3c
Merge pull request #41847 from ekonkov/sudo 
replace back clickhouse su with sudo -u in start command
 2022-10-03 14:42:29 +02:00
Robert Schulze
db5ef7b3cb
Merge branch 'master' into generated-file-cleanup 2022-10-02 23:13:18 +02:00
Alexey Milovidov
daab7f7f61
Merge pull request #41987 from ClickHouse/musl-4 
Preparation for Musl build, part 4
 2022-10-02 02:35:01 +03:00
Alexey Milovidov
9de9af54fe Merge branch 'master' of github.com:ClickHouse/ClickHouse into write-progress-to-tty 2022-10-01 23:25:07 +02:00
Alexey Milovidov
5d710e21f1 Render progress directly in tty 2022-10-01 23:19:36 +02:00
Alexey Milovidov
8e4531d135 Preparation for Musl build, part 4 2022-10-01 16:29:41 +02:00
root
7e4edfeb5a Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:46 -07:00
root
4318ec125e Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:22 -07:00
vdimir
7ebc297f4c
Merge branch 'master' into pr/BigRedEye/38191 2022-09-30 09:40:47 +00:00
Robert Schulze
cc92a2d174
Merge branch 'master' into generated-file-cleanup 2022-09-30 09:56:31 +02:00
vdimir
0f1a7c252d
better TemporaryDataOnDisk 2022-09-29 09:51:46 +00:00
vdimir
0093752271
Upd log message 2022-09-29 09:51:43 +00:00
vdimir
15c7a3be34
Temp data on disk: build 2022-09-29 09:51:41 +00:00
vdimir
c0898ce289
Use abstraction for temporary data on disk in Sort and Aggregation 2022-09-29 09:51:41 +00:00
Yakov Olkhovskiy
5e40f2ebca review suggestions 2022-09-29 00:13:40 +00:00
Alexey Milovidov
1ee890a0ed
Merge branch 'master' into sudo 2022-09-29 03:00:55 +03:00
Anton Popov
c97bec829a slightly better 2022-09-28 21:47:10 +00:00
Sergei Trifonov
8edac7eba1
Merge branch 'master' into io-scheduler 2022-09-28 22:17:17 +02:00
Anton Popov
34bc16cd5b avoid more useless errors 2022-09-28 15:55:39 +00:00
Anton Popov
4ac5df2540 drop all fuzzed tables 2022-09-28 14:52:10 +00:00
Yakov Olkhovskiy
b5768be1ca
Merge pull request #41743 from ClickHouse/fix-cert-err-msg2 
Add parameter --accept-invalid-certificate and set appropriate invalid certificate handling
 2022-09-28 10:13:13 -04:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Robert Schulze
6d70b4a1f6
Generate config_version.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Anton Popov
c8beae8237 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-28 12:47:02 +00:00
Alexey Milovidov
c0d7fa74e4
Merge branch 'master' into sudo 2022-09-28 14:16:49 +03:00
Alexey Milovidov
59b37bf4a0
Merge pull request #40684 from ClickHouse/add-missing-tests-geobase 
Add missing tests for legacy geobase
 2022-09-28 14:08:30 +03:00
Evgeny Konkov
b08ab05fbb fix typo in comment 2022-09-28 12:28:59 +03:00
Robert Schulze
06507c40de
${ConfigIncludePath} --> ${CONFIG_INCLUDE_PATH} 2022-09-28 08:28:47 +00:00
Robert Schulze
1885bb0524
Make comment consistent accross generated files 2022-09-28 08:11:09 +00:00
Robert Schulze
de3264691c
Merge pull request #41790 from ClickHouse/print-git-hash 
Log git hash during startup
 2022-09-28 09:43:17 +02:00
Alexey Milovidov
fcd443f0e3
Merge branch 'master' into add-missing-tests-geobase 2022-09-28 01:58:02 +03:00
Alexey Milovidov
3d9d1369b1
Merge pull request #41518 from ClickHouse/filimonov-config-xml-obsolete-comment 
Remove obsolete comment from the config.xml
 2022-09-28 01:05:56 +03:00
Alexey Milovidov
f8d2735b66
Update Install.cpp 2022-09-27 23:35:22 +03:00
Alexey Milovidov
6d6e88a5c8
Merge pull request #41798 from ClickHouse/small-fix-in-dashboard 
Small fix in dashboard
 2022-09-27 23:20:33 +03:00
Evgeny Konkov
0c9d7a8054 replace back clickhouse su with sudo in start command in order to respect limits in /etc/security/limits.conf 2022-09-27 21:57:56 +03:00
serxa
b057d07977 IO scheduling subsystem 2022-09-27 13:26:41 +00:00
Robert Schulze
588a5e5a42
Simplify a bit 2022-09-27 07:47:18 +00:00
Antonio Andelic
655bde8e3f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-27 07:28:26 +00:00
Alexey Milovidov
6d7de37e3d Small fix in dashboard 2022-09-26 23:52:14 +02:00
Robert Schulze
99725e68d1
Fix standalone keeper build 2022-09-26 19:28:27 +00:00
Robert Schulze
9711950c35
Fix build 2022-09-26 15:05:36 +00:00
vdimir
e6a2959b63
Do not print tons of unknown files on server statup 2022-09-26 13:45:55 +00:00
kssenii
f12811012e Fix integration tests 2022-09-26 12:49:20 +02:00
Nikita Mikhaylov
502338560c Added a test 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
636cedf488 Better 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
9b35d54e8f First try 2022-09-26 10:24:39 +00:00
Yakov Olkhovskiy
1d6c6c7144
change to --accept-invalid-certificate 2022-09-25 20:54:04 -04:00
Yakov Olkhovskiy
57e875d08c add parameter --accept-certificate, set appropriate invalid certificate handling 2022-09-24 04:46:19 +00:00
Nikolay Degterinsky
a7f3c7086c
Merge pull request #41341 from evillique/mandatory-identification 
Add a setting requiring no_password to be explicitly specified when creating a user
 2022-09-22 15:19:43 +02:00
Nikita Mikhaylov
364aa1b41e
Fix race between ~BackgroundSchedulePool and ~DNSCacheUpdater (#41654) 2022-09-22 13:02:45 +02:00
Alexey Milovidov
45afacdae4
Merge pull request #41186 from ClickHouse/fix-three-fourth-of-trash 
Fix more than half of the trash
 2022-09-22 07:28:26 +03:00
Antonio Andelic
eefbbf53e8 Extract S3 logic 2022-09-21 11:53:54 +00:00
Nikolay Degterinsky
7292d47923
Merge branch 'master' into mandatory-identification 2022-09-21 12:18:17 +02:00
Nikolay Degterinsky
f0ca365fa0 Better 2022-09-21 10:16:49 +00:00
Alexey Milovidov
2c69ec4317 Fix help message in static-files-disk-uploader 2022-09-21 05:14:31 +02:00
Alexey Milovidov
45bd3cfc30 Merge branch 'master' into fix-three-fourth-of-trash 2022-09-20 21:27:41 +02:00
Nikolai Kochetov
46c955a469
Merge pull request #41387 from ClickHouse/run-dns-cache-updater-earlier 
Enable DNS cache updater earlier.
 2022-09-20 12:04:05 +02:00
Antonio Andelic
22fe3b947e
Merge pull request #41520 from ClickHouse/fix-log-saving-for-stateless-tests 
Add option for loading only system tables with clickhouse-local
 2022-09-20 08:23:47 +02:00
Alexey Milovidov
28b3965797
Merge branch 'master' into filimonov-config-xml-obsolete-comment 2022-09-20 06:01:40 +03:00
Sergei Trifonov
cf2db48c29
Merge pull request #40631 from ClickHouse/readonly-settings-allow 
Allow to modify constrained settings in readonly mode
 2022-09-20 02:18:14 +02:00
Antonio Andelic
0f92e5e576 Add option for loading only system tables from path using clickhouse-local 2022-09-19 14:18:42 +00:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage 2022-09-19 15:26:21 +02:00
filimonov
6f956329d5
Remove obsolete comment from the config.xml 
Remove obsolete comment, see commit c059d0a0ee
 2022-09-19 15:26:11 +02:00
Yakov Olkhovskiy
2f2080f5a4
Merge branch 'master' into composable-protocol 2022-09-19 08:15:07 -04:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
Alexey Milovidov
bb6f1bfce2 Fix 9/10 of trash 2022-09-19 08:53:20 +02:00
Vitaly Baranov
5365b105cc Add SYSTEM RELOAD USERS command. 2022-09-18 12:44:00 +02:00
Alexey Milovidov
515972f459 Merge branch 'master' into forbid-dlopen-2 2022-09-18 02:20:22 +02:00
Alexey Milovidov
bac578b23a
Merge pull request #41428 from ClickHouse/remove-dlopen 
Remove `dlopen`
 2022-09-18 00:09:57 +03:00
Alexey Milovidov
d9d1b558e9 Fix clang-tidy 2022-09-17 18:40:22 +02:00
Alexey Milovidov
ada7a44ae4 Remove -WithTerminatingZero methods 2022-09-17 05:34:18 +02:00
Alexey Milovidov
ecd4c29c2a Forbid invocations of dlopen 2022-09-17 03:55:39 +02:00
Alexey Milovidov
35cce03125 Remove dlopen 2022-09-17 03:02:34 +02:00
Nikolai Kochetov
8422fd5791
Merge branch 'master' into run-dns-cache-updater-earlier 2022-09-16 20:48:51 +02:00
Dmitry Novik
eacc790050
Merge branch 'master' into fix-odr-vialation 2022-09-16 18:44:53 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
Eugene Konkov
ce0508e223
Increase open files limit (#41345) 
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2022-09-16 12:41:36 +02:00
Nikolai Kochetov
ee361c6d46 Enable DNS cache updater earlier. 2022-09-15 17:31:28 +00:00
Nikolay Degterinsky
72d8a04ee6 Add setting 'allow_implicit_no_password' 2022-09-15 08:07:42 +00:00
Yakov Olkhovskiy
b939379da6 bug fix, merge fix, style fix 2022-09-14 16:29:26 +00:00
Yakov Olkhovskiy
5e3fde60f9
Merge branch 'master' into composable-protocol 2022-09-14 11:35:23 -04:00
Robert Schulze
fd97058e45
fix: incorporate review comments 2022-09-14 15:21:24 +00:00
Yakov Olkhovskiy
d550604e28 respect listen_host config param, fix updateServers and getListenTry 2022-09-13 23:12:53 +00:00
Dmitry Novik
990ce888bf
Merge branch 'master' into fix-odr-vialation 2022-09-13 16:53:57 +02:00
Sergei Trifonov
5ee2f1701c
Merge branch 'master' into kvm-clock 2022-09-13 12:45:24 +02:00
Sergei Trifonov
795c1c1438 fix warning text 2022-09-13 12:44:42 +02:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading 
Allow to load marks in threadpool in advance
 2022-09-13 12:31:58 +02:00
Sergei Trifonov
85e43566b6
Merge branch 'master' into kvm-clock 2022-09-13 00:17:09 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS 
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
  a mechanism to update a model explicitly. It also saves potentially
  unnecessary reloads of a model from disk after it's initial load.

  To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
  was changed from eager to lazy. This means that both statements
  previously immedately reloaded the specified/all models, whereas now
  the statements only trigger an unload and the first call to
  catboostEvaluate() does the actual load.

- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
  fields removed. The view was not documented in the past and for now it
  remains undocumented. The commit is thus not considered a breach of
  ClickHouse's public interface.
 2022-09-12 19:33:02 +00:00
Meena Renganathan
51d6611b96 Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default 2022-09-12 09:05:38 -07:00
Azat Khuzhin
b698a4ff65 Apply changes to http handlers on fly without server restart 
This has been implemented by simply restarting http servers in case of
http_handlers directive in configuration xml had been changed.

But, for this I have to change the handlers interface to accept
configuration separatelly, since the configuration that contains in the
server is the configuration with which server had been started.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Antonio Andelic
 2022-09-12 17:34:51 +02:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow 2022-09-12 16:20:40 +02:00
Sergei Trifonov
2c78ff9697
Merge branch 'master' into kvm-clock 2022-09-12 14:59:18 +02:00
Sergei Trifonov
22ea533179 Do not warn about kvm-clock 2022-09-12 14:56:01 +02:00
kssenii
0bb6f8792a Fix 2022-09-12 14:06:26 +02:00
Yakov Olkhovskiy
0d6d9e7e44
Merge branch 'master' into composable-protocol 2022-09-11 18:59:57 -04:00
Yakov Olkhovskiy
7c855c9da2 style fix 2022-09-11 22:44:27 +00:00
Yakov Olkhovskiy
ffa7d3b121 cleanup 2022-09-11 21:51:25 +00:00
Yakov Olkhovskiy
7985f4ba16 cleanup, respect USE_SSL 2022-09-11 21:35:38 +00:00
Yakov Olkhovskiy
0c62b5acfc split into diferent files 2022-09-11 19:00:40 +00:00
Alexey Milovidov
9a0892c40c
Merge pull request #41171 from ClickHouse/fix-half-of-trash 
Fix half of trash
 2022-09-11 21:40:22 +03:00
Yakov Olkhovskiy
745e759146 bugs fixed, cleanup, working state 2022-09-11 14:32:37 +00:00
kssenii
b7d751b782 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-11 13:23:30 +02:00
Alexey Milovidov
5802c2fdd2
Merge pull request #40713 from ClickHouse/remove-useless-line 
Remove one line from XML, because we do not care
 2022-09-11 09:53:30 +03:00
Yakov Olkhovskiy
c3ac0c434b some refactoring 2022-09-11 04:40:58 +00:00
Yakov Olkhovskiy
d001baec87 pass section config key to a factory 2022-09-11 00:40:40 +00:00
Yakov Olkhovskiy
772bf050da add PROXYv1 handler, add stack exchange data block, tuneup protocols config 2022-09-10 20:21:37 +00:00
Alexey Milovidov
61278c81e4 Merge branch 'master' into remove-useless-line 2022-09-10 05:07:10 +02:00
Alexey Milovidov
fa62c7e982 Fix half of trash 2022-09-10 04:08:16 +02:00
Dmitry Novik
323fdeff0b
Merge branch 'master' into fix-odr-vialation 2022-09-09 00:42:14 +02:00
Dmitry Novik
29b5d023ee Replace WITH_TEXT_LOG by WITHOUT_TEXT_LOG 2022-09-08 19:16:37 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view 
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
 2022-09-08 09:10:11 +00:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge 
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
 2022-09-08 09:01:32 +00:00
Yakov Olkhovskiy
8a7fe2888a protocols configuration processing 2022-09-08 06:12:33 +00:00
Sergei Trifonov
77ee4c04aa fix stateless tests 2022-09-06 20:28:50 +02:00
kssenii
487bc0fba3 Fix heap use after free 2022-09-06 16:50:58 +02:00
Sergei Trifonov
6217559175
Merge branch 'master' into readonly-settings-allow 2022-09-06 16:10:48 +02:00
Alexey Milovidov
51dc796275 Remove trash 2022-09-06 02:04:57 +02:00
Alexey Milovidov
b7bdd38e41 Revert "Remove trash" 
This reverts commit 494dfff73c.
 2022-09-06 02:04:36 +02:00
Alexey Milovidov
494dfff73c Remove trash 2022-09-06 02:04:18 +02:00
Alexey Milovidov
940a53e519
Merge pull request #40984 from Lucky-Chang/typo_fix 
Fix some typos and clang-tidy warnings
 2022-09-06 02:37:50 +03:00
Kruglov Pavel
95847775b6
Merge pull request #40120 from evillique/obfuscator-schema-inference 
Add schema inference to `clickhouse-obfuscator`
 2022-09-05 22:03:57 +02:00
kssenii
0f305f8a7a Fix test 2022-09-05 12:20:21 +02:00
Luck-Chang
1ac8e739c9 fix some typos and clang-tidy warnings 2022-09-05 09:50:24 +08:00
Alexey Milovidov
0e5c94ce80 Merge branch 'master' into add-missing-tests-geobase 2022-09-04 03:32:29 +02:00
Alexey Milovidov
13a129bee7 Merge branch 'master' into remove-useless-line 2022-09-04 03:24:40 +02:00
Alexander Gololobov
90baf74f13 Added mkdir command 2022-09-02 19:30:35 +02:00
kssenii
2dcf3fef94 Fix tests 2022-09-02 19:27:52 +02:00
Yakov Olkhovskiy
5727517713 add variadic constructor 2022-09-02 16:22:57 +00:00
Sergei Trifonov
005b2588d8
Merge branch 'master' into readonly-settings-allow 2022-09-02 16:37:43 +02:00
Sergei Trifonov
014d109175 fix build, fix docs, fix comments, logical fixes, test are still to be fixed and new test are to be added 2022-09-02 16:20:09 +02:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
Yakov Olkhovskiy
b6e3680d60 test with TLS 2022-09-02 04:47:04 +00:00
Sergei Trifonov
7fec55eea4 work in progress 2022-09-02 04:12:05 +02:00
Nikolay Degterinsky
e9232fc4e6 Better 2022-09-01 20:19:25 +00:00
Nikolay Degterinsky
5a611e272a Merge remote-tracking branch 'upstream/master' into obfuscator-schema-inference 2022-09-01 13:07:28 +00:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric 
Metric for the number of detached parts
 2022-09-01 12:24:42 +02:00
root
d5db88fa27 Structured-logging-custom-keys PR init 2022-08-30 23:20:19 -03:00
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
Alexander Tokmakov
1c6dea52e0
Update config.xml 2022-08-29 15:50:05 +03:00
Alexey Milovidov
84f9432e7e
Merge pull request #40724 from azat/update-bash-completion 
Update available formats for bash completion
 2022-08-29 00:14:48 +03:00
Alexey Milovidov
f462160bd1 Merge remote-tracking branch 'origin/master' into remove-useless-line 2022-08-28 21:00:29 +02:00
Alexey Milovidov
2fa8d212e6 Fix very strange test 2022-08-28 21:00:15 +02:00
Alexey Milovidov
deac6cfb5d Minor modifications 2022-08-28 20:49:39 +02:00
Azat Khuzhin
29877d3992 Update available formats for bash completion 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-28 17:22:32 +02:00
Alexey Milovidov
ddff5e9145
Merge pull request #40712 from ClickHouse/ci-cmake-self-extractable 
cmake: Disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING
 2022-08-28 17:12:19 +03:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Alexey Milovidov
632867dbff Fix error 2022-08-28 02:40:43 +02:00
Alexey Milovidov
3139d4a79a Fix error 2022-08-28 02:38:46 +02:00
Yakov Olkhovskiy
31a7ed09a1 disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING and add to env 2022-08-27 21:08:01 +00:00
Alexey Milovidov
b14bd43590 Fix error 2022-08-27 20:25:06 +02:00
Alexey Milovidov
b70373d429 Fix error 2022-08-27 20:23:15 +02:00
Alexey Milovidov
6414a01987 Rewrite NLP tests from integration to functional 2022-08-26 22:36:34 +02:00
Alexey Milovidov
77e15a24d0 Add missing tests for legacy geobase 2022-08-26 22:17:49 +02:00
Alexey Milovidov
34b5d267f3
Merge pull request #40461 from ClickHouse/dashboard 
Dashboard
 2022-08-23 15:34:28 +03:00
Alexey Milovidov
0252695c6e
Merge pull request #40224 from ClickHouse/alexey-milovidov-patch-4 
Check what will be if I enable `concurrency-mt-unsafe` in `clang-tidy`
 2022-08-22 23:12:01 +03:00
Alexey Milovidov
6444afac52 Merge branch 'master' into dashboard 2022-08-22 01:30:21 +02:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning 2022-08-21 18:24:17 +00:00
Alexey Milovidov
000f879779 Add comments 2022-08-21 13:31:04 +02:00
Alexey Milovidov
861e5dbb76 Add comments 2022-08-21 13:26:32 +02:00
Alexey Milovidov
ab079b270a Add comments 2022-08-21 13:07:46 +02:00
Alexey Milovidov
fa9df87198 Fix margin 2022-08-21 12:56:23 +02:00
Alexey Milovidov
458d21dd3c Add "readpassphrase" as a dependency for clickhouse-local 2022-08-21 12:12:11 +02:00
Alexey Milovidov
9635fa817b Embedded dashboard 2022-08-21 12:11:25 +02:00
Alexey Milovidov
4e0bf0dd34 Add a file 2022-08-21 12:11:04 +02:00
Alexey Milovidov
3649aac593 Fix style 2022-08-20 17:59:51 +02:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Sergei Trifonov
7bbbe0a4db fix darwin build 2022-08-18 19:31:25 +02:00
Sergei Trifonov
b064021cec fix build 2022-08-18 16:57:31 +02:00
Sergei Trifonov
42a711015f split concurrent_threads_soft_limit setting into two: concurrent_threads_soft_limit_{num,ratio_to_cores} 2022-08-18 14:51:08 +02:00
Nikita Mikhaylov
a7c3f23dfb
Merge branch 'master' into auto_convert_ordinary_to_atomic 2022-08-17 00:38:25 +02:00
Alexander Tokmakov
c9bb91c31f
Merge pull request #40232 from ClickHouse/fix_a_bug_with_symlinks_detection 
Fix a bug with symlinks detection
 2022-08-15 20:28:30 +03:00
Alexander Tokmakov
6f5a7c3bf7 fix a bug with symlinks detection 2022-08-15 12:30:47 +02:00
Alexey Milovidov
bada9ebefa
Merge branch 'master' into disable-zero-copy-replication 2022-08-14 10:40:16 +03:00
Alexey Milovidov
e774d28c43 Fix style 2022-08-14 04:16:48 +02:00
Alexey Milovidov
53ce2986de Display server-side time in clickhouse-benchmark by default 2022-08-14 03:33:42 +02:00
Alexey Milovidov
f0c8998471 Add warning message 2022-08-14 01:49:00 +02:00
Alexey Milovidov
42c358aa3c Add warning message 2022-08-14 01:47:36 +02:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master 
New caching strategies
 2022-08-12 17:23:37 +02:00
Nikita Taranov
17956cb668
Extend protocol with query parameters (#39906) 2022-08-12 14:28:35 +02:00
Anton Popov
0383250383 execute DROP queries for fuzzed tables 2022-08-12 00:32:21 +00:00
Nikolay Degterinsky
644be9dc42 Remove unneded ReadBuffers 2022-08-11 13:27:09 +00:00
Nikolay Degterinsky
1b8ca90fcc Add schema inference to clickhouse-obfuscator 2022-08-11 11:32:32 +00:00
Anton Popov
e347fa2f45 add fuzzer for table definition 2022-08-11 02:34:10 +00:00
Nikolay Degterinsky
496568101f
Merge pull request #39277 from ClibMouse/Structured-Logging-Support 
Structured logging support
 2022-08-10 22:57:48 +02:00
Kseniia Sumarokova
a4d513657c
Merge pull request #40010 from ClickHouse/catboost-further-preparation 
Further preparation for catboost integration into library-bridge
 2022-08-10 17:35:12 +02:00
Vladimir C
eb1ac41ac5
Merge pull request #40053 from ClickHouse/play-tab-in-textarea 
Play: recognize tab in textarea
 2022-08-10 12:53:28 +02:00
Robert Schulze
810221baf2
Assume unversioned server has version=0 and use tryParse() instead of from_chars() 2022-08-10 07:39:32 +00:00
Alexey Milovidov
34c8d2c3b4 Play: recognize tab in textarea 2022-08-10 08:33:58 +02:00
Mallik Hassan
666fb8dc42
Merge branch 'ClickHouse:master' into Structured-Logging-Support 2022-08-09 15:39:00 -03:00
Robert Schulze
e0d5020a92
Add simple versioning to the *-bridge-to-server protocol 
- In general, it is expected that clickhouse-*-bridges and
  clickhouse-server were build from the same source version (e.g. are
  upgraded "atomically"). If that is not the case, we should at least
  be able to detect the mismatch and abort.

- This commit adds a URL parameter "version", defined in a header shared
  by the server and bridges. The bridge returns an error in case of
  mismatch.

- The version is *not* send and checked for "ping" requests (used for
  handshake), only for regular requests send after handshake. This is
  because the internally thrown server-side exception due to HTTP
  failure does not propagate the exact HTTP error (it only stores the
  error as text), and as a result, the server-side handshake code
  simply retries in case of error with exponential backoff and finally
  fails with a "timeout error". This is reasonable as pings typically
  fail due to time out. However, without a rework of HTTP exceptions,
  version mismatch during ping would also appear as "timeout" which is
  too misleading. The behavior may be changed later if needed.

- Note that introducing a version parameter does not represent a
  protocol upgrade itself. Bridges older than the server will simply
  ignore the field. Only servers older than the bridges receive an error
  but such a situation should never occur in practice.
 2022-08-08 19:40:37 +00:00
Robert Schulze
9952ab1099
Prefix class names "LibraryBridge*Handler" with "ExternalDictionary" 
- necessary to disambiguate the names from "CatBoost"-"LibraryBridgeHandler"
  which will be added in a next step
 2022-08-08 17:16:46 +00:00
Yakov Olkhovskiy
d39e9f65de
Merge branch 'master' into fix-quota-key 2022-08-08 11:54:21 -04:00
Robert Schulze
ad0d060dc1
Merge pull request #39904 from ClickHouse/library-bridge-refactoring 
Prepare library-bridge for catboost integration
 2022-08-08 12:15:01 +02:00
Alexey Milovidov
64a1b0f2b8 Play UI: row numbers; cell selection; hysteresis 2022-08-08 04:25:03 +02:00
Alexey Milovidov
653b86d1e6 Play UI: row numbers; cell selection; hysteresis 2022-08-08 04:16:50 +02:00
Alexey Milovidov
2d83635643 Play UI: row numbers; cell selection; hysteresis 2022-08-08 03:57:02 +02:00
Alexey Milovidov
d914ddc766 Fix Play UI 2022-08-08 02:36:50 +02:00
alexX512
2be72560f4 Add loading of mark cache policy 2022-08-07 19:59:39 +00:00
alexX512
1133e42367 Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf 2022-08-07 19:16:40 +00:00
Yakov Olkhovskiy
29622746ea
Merge pull request #39936 from ClickHouse/cmake-fix-self-extracting-split 
Don't create self-extracting clickhouse for split build
 2022-08-07 07:23:24 -04:00
Yakov Olkhovskiy
b1f45fa787 Don't create self-extracting clickhouse for split build 2022-08-05 21:48:40 -04:00
Alexey Milovidov
ac93f0a2ca
Merge pull request #39910 from santrancisco/enhance-playui-authentication 
Move username and password from URL parameters to Basic Authentication
 2022-08-06 01:58:09 +03:00
Alexander Tokmakov
f2c4cad255 add flag that enables automatic canversion from Ordinary to Atomic 2022-08-05 21:41:25 +02:00
santrancisco
c7f0598a6f Fall back to url parameters if file is opened locally 2022-08-05 13:10:45 +10:00
santrancisco
a25b858097 Move username and password from URL parameter to Basic Authentication header 2022-08-05 09:51:12 +10:00
Anton Popov
4f2c73f9f6 Merge remote-tracking branch 'upstream/master' into HEAD 2022-08-04 23:08:15 +00:00
Robert Schulze
20bb8a248e
Prepare server-side BridgeHelper for catboost integration 
Wall of text, sorry, but I also had to document some stuff for myself:

There are three ways to communicate data using HTTP:
- the HTTP verb: for our purposes, PUT and GET,
- the HTTP path: '/ping', '/request' etc.,
- the HTTP URL parameter(s), e.g. 'method=libNew&dictionary_id=1234'

The bridge will use different handlers for communication with the
external dictionary library and for communication with the catboost
library. Handlers are created based on a combination of the HTTP verb
and the HTTP method. More specifically, there will be combinations
- GET + '/extdict_ping'
- PUT + '/extdict_request'
- GET + '/catboost_ping'
- PUT + '/catboost_request'.
For each combination, the bridge expects a certain set of URL
parameters, e.g. for the first combination parameter "dictionary_id" is
expected.

Starting with this commit, the library-bridge creates handlers based on
the first two combinations (the latter two combinations will be added
later). This makes the handler creation mechanism consistent with it's
counterpart in xdbc-bridge.

For that, it was necessary to make both IBridgeHelper methods
"getMainURI()" and "getPingURI()" pure virtual so that derived classes
(LibraryBridgeHelper and XDBCBridgeHelper) must provide custom URLs with
custom paths.

Side note 1: Previously, LibraryBridgeHelper sent HTTP URL parameter
"method=ping" during handshake (PING) but the library-bridge ignored
that parameter. We now omit this parameter, i.e.
LibraryBridgeHelper::PING was removed. Again, this makes things
consistent with xdbc-bridge.

Side note 2: xdbc-bridge is unchanged in this commit. Therefore,
XDBCBridgeHelper now uses the HTTP paths previously in the base class.
For funny reason, XDBCBridgeHelper did not use
IBridgeHelper::getMainURI() - it generates the URLs by itself. I kept it
that way for now but provided an implementation of getMainURI() anyways.
 2022-08-04 19:29:51 +00:00
Robert Schulze
ea73b98fb9
Prepare library-bridge for catboost integration 
- Rename generic file and identifier names in library-bridge to
  something more dictionary-specific. This is needed because later on,
  catboost will be integrated into library-bridge.

- Also: Some smaller fixes like typos and un-inlining non-performance
  critical code.

- The logic remains unchanged in this commit.
 2022-08-04 19:26:51 +00:00
Yakov Olkhovskiy
23037daf17
Merge branch 'master' into fix-quota-key 2022-08-04 12:14:49 -04:00
Alexey Milovidov
834cbbedce
Merge branch 'master' into escape_diag_creds 2022-08-04 01:59:50 +03:00
Yakov Olkhovskiy
2e34b384c1 update tcp protocol, add quota_key 2022-08-03 15:44:08 -04:00
Mallik Hassan
87bde54ee5
Merge branch 'ClickHouse:master' into Structured-Logging-Support 2022-08-02 10:44:05 -03:00
Alexey Milovidov
5f62863ab9
Merge branch 'master' into escape_diag_creds 2022-08-02 05:50:01 +03:00
Heena Bansal
d8db482b2e
Merge branch 'master' into Issue_39395 2022-08-01 12:22:16 -04:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling 
Use std::popcount, ::countl_zero, ::countr_zero functions
 2022-08-01 17:04:51 +02:00
HeenaBansal2009
50c98789b8 Updated as per comments 2022-08-01 07:05:50 -07:00
HeenaBansal2009
800ed546be Updated as per comments 2022-08-01 07:03:36 -07:00
Ilya Yatsishin
c882bdc88e
Merge pull request #35968 from ClickHouse/interserver_listen_port 2022-08-01 13:57:17 +02:00
Alexander Tokmakov
80f9ba9186
Merge pull request #39690 from ClickHouse/show-addresses-in-stack-traces 
Configuration parameter to hide addresses in stack traces
 2022-08-01 14:20:37 +03:00
Alexey Milovidov
6a2f7d0c8f
Merge branch 'master' into interserver_listen_port 2022-07-31 22:51:38 +03:00
Alexey Milovidov
ebc4974188
Merge branch 'master' into escape_diag_creds 2022-07-31 22:46:10 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions 
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
 2022-07-31 15:16:51 +00:00
Robert Schulze
dcc8751685
Disable harmful env var check to workaround failure to start the server 2022-07-31 08:55:07 +00:00
Robert Schulze
7c23e48b5b
Revert exclusion of libharmful (did not work anyways) 2022-07-31 08:05:12 +00:00
Robert Schulze
7fe106a0fb
Try to fix libharmful fail 2022-07-31 07:44:25 +00:00
Alexey Milovidov
fa9c3dcc48
Update programs/local/LocalServer.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-07-31 03:02:27 +03:00
Alexey Milovidov
a30dbed6b8
Update programs/local/LocalServer.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-07-31 03:02:20 +03:00
Alexey Milovidov
f80a4c184e
Merge branch 'master' into interserver_listen_port 2022-07-31 01:22:32 +03:00
Alexey Milovidov
9adfd259d0 Support in clickhouse-local 2022-07-30 01:50:45 +02:00
Alexey Milovidov
c0d7b6efc3 Suggestions from @tavplubix 2022-07-30 01:45:06 +02:00
Alexey Milovidov
4aa96767d5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into show-addresses-in-stack-traces 2022-07-30 01:44:04 +02:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-29 12:17:43 +00:00
Dale Mcdiarmid
d9c585ecf1 Escape credentials for diagnostics tool 2022-07-29 12:25:03 +01:00