Robert Schulze
9c066e964d
Less use of CH-specific bit_cast()
...
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)
Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
2022-11-04 15:52:48 +00:00
Yakov Olkhovskiy
b30f2ac12d
allow '--progress err' to preserve old behavior, adjust tests
2022-11-03 03:16:55 +00:00
vdimir
6a4247ca32
Merge branch 'master' into grace_hash_join
2022-10-31 09:54:37 +00:00
Vitaly Baranov
842e40082f
Remove query_masking_rules for encrypt() function from the default config (now it's always masked).
2022-10-31 10:50:33 +01:00
Anton Popov
d09039418f
Merge remote-tracking branch 'upstream/master' into HEAD
2022-10-28 14:35:45 +00:00
Robert Schulze
31ef6c66cb
Merge pull request #42513 from ClickHouse/update-libcxx-to-15
...
Build with libcxx(abi) 15
2022-10-27 10:30:36 +02:00
vdimir
506bf2d225
Merge branch 'master' into grace_hash_join
2022-10-26 12:25:50 +00:00
Robert Schulze
e996e2ff15
Fix clang-tidy
2022-10-26 07:48:37 +00:00
Alexey Milovidov
47c32678ce
Merge pull request #42477 from AlfVII/enable-compiled-expressions-cache-in-local
...
Enable CompiledExpressionCache in clickhouse-local
2022-10-25 04:57:48 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15
2022-10-24 08:29:37 +02:00
Alexey Milovidov
cd43595ea9
Merge branch 'master' into write-progress-to-tty
2022-10-23 01:41:17 +02:00
Azat Khuzhin
dbb4f8216f
Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
...
Conflicts:
- src/Functions/GregorianDate.h
2022-10-22 08:20:05 +02:00
Azat Khuzhin
905a95e166
Review fixes
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 22:40:13 +02:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Nikolay Degterinsky
d7d0794d2f
Better logs in clickhouse-disks
2022-10-20 23:17:29 +00:00
Alexey Milovidov
58f238007a
Merge pull request #42508 from ClickHouse/remove-database-macro-from-server-prompt
...
Remove support for the `{database}` macro from the client's prompt
2022-10-20 21:14:06 +02:00
Robert Schulze
f435e4fbe2
Un-clusterf$ck incomplete type errors
2022-10-20 14:48:46 +00:00
Robert Schulze
50c932e566
Separate TaskShard and TaskTable
2022-10-20 14:00:20 +00:00
Robert Schulze
df318d8719
Un-inline TaskTableAndShard
2022-10-20 13:43:15 +00:00
Robert Schulze
f6de964eaa
Fix static_assert in random generator
2022-10-20 11:20:18 +00:00
Robert Schulze
cdae7082a3
Up-inline ShardPartition, ShardPartitionPiece and TaskCluster
2022-10-20 11:16:05 +00:00
Alexey Milovidov
fabc8f5a18
Remove support for {database} macro from the client's prompt
2022-10-19 23:27:12 +02:00
Alfonso Martinez
168df7d9ba
Removed trailing whitespace
2022-10-19 13:34:31 +02:00
Alfonso Martinez
0aa26db64c
Added end of line co comply with Clangformat
2022-10-19 12:38:20 +02:00
Alfonso Martinez
c0be79f26c
CompiledExpressionCache enabled
2022-10-19 12:27:00 +02:00
Antonio Andelic
ee2f5977d5
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-10-18 10:02:50 +02:00
Alexey Milovidov
ddfaadeea4
Merge pull request #42325 from gingerwizard/go_update_diagnostics
...
go update for diagnostics tool
2022-10-17 19:30:16 +02:00
Yakov Olkhovskiy
3ce1fa09d6
Merge pull request #41198 from ClickHouse/composable-protocol
...
Composable protocol
2022-10-17 11:23:12 -04:00
vdimir
adb63a5583
Merge branch 'master' into grace_hash_join
2022-10-17 12:32:56 +00:00
Antonio Andelic
7c7395575f
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-10-17 11:02:36 +00:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf
...
Refactor the implementation of user-defined functions
2022-10-15 12:33:10 +02:00
Dale McDiarmid
980f79f9a7
go update
2022-10-14 15:32:34 +01:00
Nikolay Degterinsky
51e55db617
Fix log-level
2022-10-14 05:41:58 +02:00
Vitaly Baranov
2e59d671fc
Split UserDefinedSQLObjectsLoader to interface and implementation.
2022-10-13 10:32:32 +02:00
vdimir
0178307c27
Followup for TemporaryDataOnDisk
2022-10-12 15:25:23 +02:00
Yakov Olkhovskiy
6c8e52eb67
Merge branch 'master' into composable-protocol
2022-10-08 20:33:32 -04:00
Alexander Tokmakov
4175f8cde6
abort instead of __builtin_unreachable in debug builds
2022-10-07 21:49:08 +02:00
Yakov Olkhovskiy
2229087896
Merge branch 'master' into composable-protocol
2022-10-07 02:23:26 -04:00
Yakov Olkhovskiy
fc1de0a56a
move some functionality to Server::buildProtocolStackFromConfig
2022-10-07 01:10:33 +00:00
vdimir
5a7c493559
Merge branch 'master' into pr/BigRedEye/38191
2022-10-06 14:43:35 +00:00
Robert Schulze
78be400ac0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup
2022-10-06 12:22:43 +00:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup
...
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake
Logical merge conflicts:
- Functions/tryDecrypt.cpp
2022-10-06 08:43:25 +00:00
Anton Popov
190e1f6830
Merge branch 'master' into async-inserts-log
2022-10-06 01:59:27 +02:00
Alexey Milovidov
25d13b89e9
Merge pull request #41142 from MeenaRenganathan22/OpenSSL_InHouse
...
Changes to support the OpenSSL In-house build.
2022-10-05 23:02:04 +03:00
vdimir
ff55c369bc
Merge branch 'tmp-data-followup'
2022-10-05 18:10:05 +00:00
vdimir
ccd29e172f
Followup for TemporaryDataOnDisk
2022-10-05 16:42:27 +00:00
Anton Popov
f61e98f0e8
Merge pull request #40668 from ClickHouse/async-insert-max-data-size-fix
...
Use settings from query context for async INSERTs
2022-10-05 15:15:19 +02:00
Anton Popov
8ca73eab03
Merge branch 'master' into fuzz-table-definitions
2022-10-04 15:10:31 +02:00
Anton Popov
148995894f
add system table asynchronous_inserts_log
2022-10-03 21:35:55 +00:00
Nikolai Kochetov
8b581a6b3c
Merge pull request #41847 from ekonkov/sudo
...
replace back clickhouse su with sudo -u in start command
2022-10-03 14:42:29 +02:00
Robert Schulze
db5ef7b3cb
Merge branch 'master' into generated-file-cleanup
2022-10-02 23:13:18 +02:00
Alexey Milovidov
daab7f7f61
Merge pull request #41987 from ClickHouse/musl-4
...
Preparation for Musl build, part 4
2022-10-02 02:35:01 +03:00
Alexey Milovidov
9de9af54fe
Merge branch 'master' of github.com:ClickHouse/ClickHouse into write-progress-to-tty
2022-10-01 23:25:07 +02:00
Alexey Milovidov
5d710e21f1
Render progress directly in tty
2022-10-01 23:19:36 +02:00
Alexey Milovidov
8e4531d135
Preparation for Musl build, part 4
2022-10-01 16:29:41 +02:00
root
7e4edfeb5a
Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse
2022-09-30 15:50:46 -07:00
root
4318ec125e
Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse
2022-09-30 15:50:22 -07:00
vdimir
7ebc297f4c
Merge branch 'master' into pr/BigRedEye/38191
2022-09-30 09:40:47 +00:00
Robert Schulze
cc92a2d174
Merge branch 'master' into generated-file-cleanup
2022-09-30 09:56:31 +02:00
vdimir
0f1a7c252d
better TemporaryDataOnDisk
2022-09-29 09:51:46 +00:00
vdimir
0093752271
Upd log message
2022-09-29 09:51:43 +00:00
vdimir
15c7a3be34
Temp data on disk: build
2022-09-29 09:51:41 +00:00
vdimir
c0898ce289
Use abstraction for temporary data on disk in Sort and Aggregation
2022-09-29 09:51:41 +00:00
Yakov Olkhovskiy
5e40f2ebca
review suggestions
2022-09-29 00:13:40 +00:00
Alexey Milovidov
1ee890a0ed
Merge branch 'master' into sudo
2022-09-29 03:00:55 +03:00
Anton Popov
c97bec829a
slightly better
2022-09-28 21:47:10 +00:00
Sergei Trifonov
8edac7eba1
Merge branch 'master' into io-scheduler
2022-09-28 22:17:17 +02:00
Anton Popov
34bc16cd5b
avoid more useless errors
2022-09-28 15:55:39 +00:00
Anton Popov
4ac5df2540
drop all fuzzed tables
2022-09-28 14:52:10 +00:00
Yakov Olkhovskiy
b5768be1ca
Merge pull request #41743 from ClickHouse/fix-cert-err-msg2
...
Add parameter --accept-invalid-certificate and set appropriate invalid certificate handling
2022-09-28 10:13:13 -04:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h
...
Less duplication, less confusion ...
2022-09-28 13:31:57 +00:00
Robert Schulze
6d70b4a1f6
Generate config_version.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Anton Popov
c8beae8237
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-28 12:47:02 +00:00
Alexey Milovidov
c0d7fa74e4
Merge branch 'master' into sudo
2022-09-28 14:16:49 +03:00
Alexey Milovidov
59b37bf4a0
Merge pull request #40684 from ClickHouse/add-missing-tests-geobase
...
Add missing tests for legacy geobase
2022-09-28 14:08:30 +03:00
Evgeny Konkov
b08ab05fbb
fix typo in comment
2022-09-28 12:28:59 +03:00
Robert Schulze
06507c40de
${ConfigIncludePath} --> ${CONFIG_INCLUDE_PATH}
2022-09-28 08:28:47 +00:00
Robert Schulze
1885bb0524
Make comment consistent accross generated files
2022-09-28 08:11:09 +00:00
Robert Schulze
de3264691c
Merge pull request #41790 from ClickHouse/print-git-hash
...
Log git hash during startup
2022-09-28 09:43:17 +02:00
Alexey Milovidov
fcd443f0e3
Merge branch 'master' into add-missing-tests-geobase
2022-09-28 01:58:02 +03:00
Alexey Milovidov
3d9d1369b1
Merge pull request #41518 from ClickHouse/filimonov-config-xml-obsolete-comment
...
Remove obsolete comment from the config.xml
2022-09-28 01:05:56 +03:00
Alexey Milovidov
f8d2735b66
Update Install.cpp
2022-09-27 23:35:22 +03:00
Alexey Milovidov
6d6e88a5c8
Merge pull request #41798 from ClickHouse/small-fix-in-dashboard
...
Small fix in dashboard
2022-09-27 23:20:33 +03:00
Evgeny Konkov
0c9d7a8054
replace back clickhouse su with sudo in start command in order to respect limits in /etc/security/limits.conf
2022-09-27 21:57:56 +03:00
serxa
b057d07977
IO scheduling subsystem
2022-09-27 13:26:41 +00:00
Robert Schulze
588a5e5a42
Simplify a bit
2022-09-27 07:47:18 +00:00
Antonio Andelic
655bde8e3f
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-09-27 07:28:26 +00:00
Alexey Milovidov
6d7de37e3d
Small fix in dashboard
2022-09-26 23:52:14 +02:00
Robert Schulze
99725e68d1
Fix standalone keeper build
2022-09-26 19:28:27 +00:00
Robert Schulze
9711950c35
Fix build
2022-09-26 15:05:36 +00:00
vdimir
e6a2959b63
Do not print tons of unknown files on server statup
2022-09-26 13:45:55 +00:00
kssenii
f12811012e
Fix integration tests
2022-09-26 12:49:20 +02:00
Nikita Mikhaylov
502338560c
Added a test
2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
636cedf488
Better
2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
9b35d54e8f
First try
2022-09-26 10:24:39 +00:00
Yakov Olkhovskiy
1d6c6c7144
change to --accept-invalid-certificate
2022-09-25 20:54:04 -04:00
Yakov Olkhovskiy
57e875d08c
add parameter --accept-certificate, set appropriate invalid certificate handling
2022-09-24 04:46:19 +00:00
Nikolay Degterinsky
a7f3c7086c
Merge pull request #41341 from evillique/mandatory-identification
...
Add a setting requiring no_password to be explicitly specified when creating a user
2022-09-22 15:19:43 +02:00
Nikita Mikhaylov
364aa1b41e
Fix race between ~BackgroundSchedulePool and ~DNSCacheUpdater ( #41654 )
2022-09-22 13:02:45 +02:00
Alexey Milovidov
45afacdae4
Merge pull request #41186 from ClickHouse/fix-three-fourth-of-trash
...
Fix more than half of the trash
2022-09-22 07:28:26 +03:00
Antonio Andelic
eefbbf53e8
Extract S3 logic
2022-09-21 11:53:54 +00:00
Nikolay Degterinsky
7292d47923
Merge branch 'master' into mandatory-identification
2022-09-21 12:18:17 +02:00
Nikolay Degterinsky
f0ca365fa0
Better
2022-09-21 10:16:49 +00:00
Alexey Milovidov
2c69ec4317
Fix help message in static-files-disk-uploader
2022-09-21 05:14:31 +02:00
Alexey Milovidov
45bd3cfc30
Merge branch 'master' into fix-three-fourth-of-trash
2022-09-20 21:27:41 +02:00
Nikolai Kochetov
46c955a469
Merge pull request #41387 from ClickHouse/run-dns-cache-updater-earlier
...
Enable DNS cache updater earlier.
2022-09-20 12:04:05 +02:00
Antonio Andelic
22fe3b947e
Merge pull request #41520 from ClickHouse/fix-log-saving-for-stateless-tests
...
Add option for loading only system tables with clickhouse-local
2022-09-20 08:23:47 +02:00
Alexey Milovidov
28b3965797
Merge branch 'master' into filimonov-config-xml-obsolete-comment
2022-09-20 06:01:40 +03:00
Sergei Trifonov
cf2db48c29
Merge pull request #40631 from ClickHouse/readonly-settings-allow
...
Allow to modify constrained settings in readonly mode
2022-09-20 02:18:14 +02:00
Antonio Andelic
0f92e5e576
Add option for loading only system tables from path using clickhouse-local
2022-09-19 14:18:42 +00:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage
2022-09-19 15:26:21 +02:00
filimonov
6f956329d5
Remove obsolete comment from the config.xml
...
Remove obsolete comment, see commit c059d0a0ee
2022-09-19 15:26:11 +02:00
Yakov Olkhovskiy
2f2080f5a4
Merge branch 'master' into composable-protocol
2022-09-19 08:15:07 -04:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow
2022-09-19 12:52:36 +02:00
Alexey Milovidov
bb6f1bfce2
Fix 9/10 of trash
2022-09-19 08:53:20 +02:00
Vitaly Baranov
5365b105cc
Add SYSTEM RELOAD USERS command.
2022-09-18 12:44:00 +02:00
Alexey Milovidov
515972f459
Merge branch 'master' into forbid-dlopen-2
2022-09-18 02:20:22 +02:00
Alexey Milovidov
bac578b23a
Merge pull request #41428 from ClickHouse/remove-dlopen
...
Remove `dlopen`
2022-09-18 00:09:57 +03:00
Alexey Milovidov
d9d1b558e9
Fix clang-tidy
2022-09-17 18:40:22 +02:00
Alexey Milovidov
ada7a44ae4
Remove -WithTerminatingZero methods
2022-09-17 05:34:18 +02:00
Alexey Milovidov
ecd4c29c2a
Forbid invocations of dlopen
2022-09-17 03:55:39 +02:00
Alexey Milovidov
35cce03125
Remove dlopen
2022-09-17 03:02:34 +02:00
Nikolai Kochetov
8422fd5791
Merge branch 'master' into run-dns-cache-updater-earlier
2022-09-16 20:48:51 +02:00
Dmitry Novik
eacc790050
Merge branch 'master' into fix-odr-vialation
2022-09-16 18:44:53 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected
...
Move CatBoost evaluation into clickhouse-library-bridge
2022-09-16 13:12:09 +02:00
Eugene Konkov
ce0508e223
Increase open files limit ( #41345 )
...
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2022-09-16 12:41:36 +02:00
Nikolai Kochetov
ee361c6d46
Enable DNS cache updater earlier.
2022-09-15 17:31:28 +00:00
Nikolay Degterinsky
72d8a04ee6
Add setting 'allow_implicit_no_password'
2022-09-15 08:07:42 +00:00
Yakov Olkhovskiy
b939379da6
bug fix, merge fix, style fix
2022-09-14 16:29:26 +00:00
Yakov Olkhovskiy
5e3fde60f9
Merge branch 'master' into composable-protocol
2022-09-14 11:35:23 -04:00
Robert Schulze
fd97058e45
fix: incorporate review comments
2022-09-14 15:21:24 +00:00
Yakov Olkhovskiy
d550604e28
respect listen_host config param, fix updateServers and getListenTry
2022-09-13 23:12:53 +00:00
Dmitry Novik
990ce888bf
Merge branch 'master' into fix-odr-vialation
2022-09-13 16:53:57 +02:00
Sergei Trifonov
5ee2f1701c
Merge branch 'master' into kvm-clock
2022-09-13 12:45:24 +02:00
Sergei Trifonov
795c1c1438
fix warning text
2022-09-13 12:44:42 +02:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading
...
Allow to load marks in threadpool in advance
2022-09-13 12:31:58 +02:00
Sergei Trifonov
85e43566b6
Merge branch 'master' into kvm-clock
2022-09-13 00:17:09 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS
...
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
a mechanism to update a model explicitly. It also saves potentially
unnecessary reloads of a model from disk after it's initial load.
To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
was changed from eager to lazy. This means that both statements
previously immedately reloaded the specified/all models, whereas now
the statements only trigger an unload and the first call to
catboostEvaluate() does the actual load.
- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
fields removed. The view was not documented in the past and for now it
remains undocumented. The commit is thus not considered a breach of
ClickHouse's public interface.
2022-09-12 19:33:02 +00:00
Meena Renganathan
51d6611b96
Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default
2022-09-12 09:05:38 -07:00
Azat Khuzhin
b698a4ff65
Apply changes to http handlers on fly without server restart
...
This has been implemented by simply restarting http servers in case of
http_handlers directive in configuration xml had been changed.
But, for this I have to change the handlers interface to accept
configuration separatelly, since the configuration that contains in the
server is the configuration with which server had been started.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Antonio Andelic
2022-09-12 17:34:51 +02:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow
2022-09-12 16:20:40 +02:00
Sergei Trifonov
2c78ff9697
Merge branch 'master' into kvm-clock
2022-09-12 14:59:18 +02:00
Sergei Trifonov
22ea533179
Do not warn about kvm-clock
2022-09-12 14:56:01 +02:00
kssenii
0bb6f8792a
Fix
2022-09-12 14:06:26 +02:00
Yakov Olkhovskiy
0d6d9e7e44
Merge branch 'master' into composable-protocol
2022-09-11 18:59:57 -04:00
Yakov Olkhovskiy
7c855c9da2
style fix
2022-09-11 22:44:27 +00:00
Yakov Olkhovskiy
ffa7d3b121
cleanup
2022-09-11 21:51:25 +00:00
Yakov Olkhovskiy
7985f4ba16
cleanup, respect USE_SSL
2022-09-11 21:35:38 +00:00
Yakov Olkhovskiy
0c62b5acfc
split into diferent files
2022-09-11 19:00:40 +00:00
Alexey Milovidov
9a0892c40c
Merge pull request #41171 from ClickHouse/fix-half-of-trash
...
Fix half of trash
2022-09-11 21:40:22 +03:00
Yakov Olkhovskiy
745e759146
bugs fixed, cleanup, working state
2022-09-11 14:32:37 +00:00
kssenii
b7d751b782
Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading
2022-09-11 13:23:30 +02:00
Alexey Milovidov
5802c2fdd2
Merge pull request #40713 from ClickHouse/remove-useless-line
...
Remove one line from XML, because we do not care
2022-09-11 09:53:30 +03:00
Yakov Olkhovskiy
c3ac0c434b
some refactoring
2022-09-11 04:40:58 +00:00
Yakov Olkhovskiy
d001baec87
pass section config key to a factory
2022-09-11 00:40:40 +00:00
Yakov Olkhovskiy
772bf050da
add PROXYv1 handler, add stack exchange data block, tuneup protocols config
2022-09-10 20:21:37 +00:00
Alexey Milovidov
61278c81e4
Merge branch 'master' into remove-useless-line
2022-09-10 05:07:10 +02:00
Alexey Milovidov
fa62c7e982
Fix half of trash
2022-09-10 04:08:16 +02:00
Dmitry Novik
323fdeff0b
Merge branch 'master' into fix-odr-vialation
2022-09-09 00:42:14 +02:00
Dmitry Novik
29b5d023ee
Replace WITH_TEXT_LOG by WITHOUT_TEXT_LOG
2022-09-08 19:16:37 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view
...
- The deleted function modelEvaluate() was superseded by
catboostEvaluate().
- Also delete the external model repository, as modelEvaluate() was it's
last user. Additionally remove the system view SYSTEM.MODELS for
inspecting the repository.
- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
made a no-op instead of deleted.
Why?
The reason is that RBAC in distributed setups works by storing
privileges (granted and revoked) as plain SQL statements in Keeper.
Nodes read these statements at startup and parse them. If a privilege
for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
nodes would fail to come up.
Considered but rejected alternatives:
- Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
of no-op behavior, too brittle.
- Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
manipulation of Keeper nodes or via SQL by dropping the privileges.
Needs user intervention during upgrade.
2022-09-08 09:10:11 +00:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge
...
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.
SQL syntax:
SELECT
catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
ACTION AS target
FROM amazon_train
LIMIT 10
Required configuration:
<catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>
*** Implementation Details ***
The internal protocol between the server and the library-bridge is
simple:
- HTTP GET on path "/extdict_ping":
A ping, used during the handshake to check if the library-bridge runs.
- HTTP POST on path "extdict_request"
(1) Send a "catboost_GetTreeCount" request from the server to the
bridge, containing a library path (e.g /home/user/libcatboost.so) and
a model path (e.g. /home/user/model.bin). Rirst, this unloads the
catboost library handler associated to the model path (if it was
loaded), then loads the catboost library handler associated to the
model path, then executes GetTreeCount() on the library handler and
finally sends the result back to the server. Step (1) is called once
by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
library path handler is unloaded in the beginning because it contains
state which may no longer be valid if the user runs
catboost("/path/to/model.bin", ...) more than once and if "model.bin"
was updated in between.
(2) Send "catboost_Evaluate" from the server to the bridge, containing
the model path and the features to run the interference on. Step (2)
is called multiple times (once per chunk) by the server from function
FunctionCatBoostEvaluate::executeImpl(). The library handler for the
given model path is expected to be already loaded by Step (1).
Fixes #27870
2022-09-08 09:01:32 +00:00
Yakov Olkhovskiy
8a7fe2888a
protocols configuration processing
2022-09-08 06:12:33 +00:00
Sergei Trifonov
77ee4c04aa
fix stateless tests
2022-09-06 20:28:50 +02:00
kssenii
487bc0fba3
Fix heap use after free
2022-09-06 16:50:58 +02:00
Sergei Trifonov
6217559175
Merge branch 'master' into readonly-settings-allow
2022-09-06 16:10:48 +02:00
Alexey Milovidov
51dc796275
Remove trash
2022-09-06 02:04:57 +02:00
Alexey Milovidov
b7bdd38e41
Revert "Remove trash"
...
This reverts commit 494dfff73c
.
2022-09-06 02:04:36 +02:00
Alexey Milovidov
494dfff73c
Remove trash
2022-09-06 02:04:18 +02:00
Alexey Milovidov
940a53e519
Merge pull request #40984 from Lucky-Chang/typo_fix
...
Fix some typos and clang-tidy warnings
2022-09-06 02:37:50 +03:00
Kruglov Pavel
95847775b6
Merge pull request #40120 from evillique/obfuscator-schema-inference
...
Add schema inference to `clickhouse-obfuscator`
2022-09-05 22:03:57 +02:00
kssenii
0f305f8a7a
Fix test
2022-09-05 12:20:21 +02:00
Luck-Chang
1ac8e739c9
fix some typos and clang-tidy warnings
2022-09-05 09:50:24 +08:00
Alexey Milovidov
0e5c94ce80
Merge branch 'master' into add-missing-tests-geobase
2022-09-04 03:32:29 +02:00
Alexey Milovidov
13a129bee7
Merge branch 'master' into remove-useless-line
2022-09-04 03:24:40 +02:00
Alexander Gololobov
90baf74f13
Added mkdir command
2022-09-02 19:30:35 +02:00
kssenii
2dcf3fef94
Fix tests
2022-09-02 19:27:52 +02:00
Yakov Olkhovskiy
5727517713
add variadic constructor
2022-09-02 16:22:57 +00:00
Sergei Trifonov
005b2588d8
Merge branch 'master' into readonly-settings-allow
2022-09-02 16:37:43 +02:00
Sergei Trifonov
014d109175
fix build, fix docs, fix comments, logical fixes, test are still to be fixed and new test are to be added
2022-09-02 16:20:09 +02:00
Antonio Andelic
e64436fef3
Fix typos with new codespell
2022-09-02 08:54:48 +00:00
Yakov Olkhovskiy
b6e3680d60
test with TLS
2022-09-02 04:47:04 +00:00
Sergei Trifonov
7fec55eea4
work in progress
2022-09-02 04:12:05 +02:00
Nikolay Degterinsky
e9232fc4e6
Better
2022-09-01 20:19:25 +00:00
Nikolay Degterinsky
5a611e272a
Merge remote-tracking branch 'upstream/master' into obfuscator-schema-inference
2022-09-01 13:07:28 +00:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric
...
Metric for the number of detached parts
2022-09-01 12:24:42 +02:00
root
d5db88fa27
Structured-logging-custom-keys PR init
2022-08-30 23:20:19 -03:00
Sema Checherinda
e436b4f4cc
Metric for the number of detached parts
2022-08-30 02:12:57 +02:00
Alexander Tokmakov
1c6dea52e0
Update config.xml
2022-08-29 15:50:05 +03:00
Alexey Milovidov
84f9432e7e
Merge pull request #40724 from azat/update-bash-completion
...
Update available formats for bash completion
2022-08-29 00:14:48 +03:00
Alexey Milovidov
f462160bd1
Merge remote-tracking branch 'origin/master' into remove-useless-line
2022-08-28 21:00:29 +02:00
Alexey Milovidov
2fa8d212e6
Fix very strange test
2022-08-28 21:00:15 +02:00
Alexey Milovidov
deac6cfb5d
Minor modifications
2022-08-28 20:49:39 +02:00
Azat Khuzhin
29877d3992
Update available formats for bash completion
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-28 17:22:32 +02:00
Alexey Milovidov
ddff5e9145
Merge pull request #40712 from ClickHouse/ci-cmake-self-extractable
...
cmake: Disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING
2022-08-28 17:12:19 +03:00
Alexey Milovidov
b42d26acfe
Remove one line from XML, because we do not care
2022-08-28 02:44:02 +02:00
Alexey Milovidov
632867dbff
Fix error
2022-08-28 02:40:43 +02:00
Alexey Milovidov
3139d4a79a
Fix error
2022-08-28 02:38:46 +02:00
Yakov Olkhovskiy
31a7ed09a1
disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING and add to env
2022-08-27 21:08:01 +00:00
Alexey Milovidov
b14bd43590
Fix error
2022-08-27 20:25:06 +02:00
Alexey Milovidov
b70373d429
Fix error
2022-08-27 20:23:15 +02:00
Alexey Milovidov
6414a01987
Rewrite NLP tests from integration to functional
2022-08-26 22:36:34 +02:00
Alexey Milovidov
77e15a24d0
Add missing tests for legacy geobase
2022-08-26 22:17:49 +02:00
Alexey Milovidov
34b5d267f3
Merge pull request #40461 from ClickHouse/dashboard
...
Dashboard
2022-08-23 15:34:28 +03:00
Alexey Milovidov
0252695c6e
Merge pull request #40224 from ClickHouse/alexey-milovidov-patch-4
...
Check what will be if I enable `concurrency-mt-unsafe` in `clang-tidy`
2022-08-22 23:12:01 +03:00
Alexey Milovidov
6444afac52
Merge branch 'master' into dashboard
2022-08-22 01:30:21 +02:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning
2022-08-21 18:24:17 +00:00
Alexey Milovidov
000f879779
Add comments
2022-08-21 13:31:04 +02:00
Alexey Milovidov
861e5dbb76
Add comments
2022-08-21 13:26:32 +02:00
Alexey Milovidov
ab079b270a
Add comments
2022-08-21 13:07:46 +02:00
Alexey Milovidov
fa9df87198
Fix margin
2022-08-21 12:56:23 +02:00
Alexey Milovidov
458d21dd3c
Add "readpassphrase" as a dependency for clickhouse-local
2022-08-21 12:12:11 +02:00
Alexey Milovidov
9635fa817b
Embedded dashboard
2022-08-21 12:11:25 +02:00
Alexey Milovidov
4e0bf0dd34
Add a file
2022-08-21 12:11:04 +02:00
Alexey Milovidov
3649aac593
Fix style
2022-08-20 17:59:51 +02:00
Alexey Milovidov
74e1f4dc61
Fix clang-tidy
2022-08-20 17:09:20 +02:00
Sergei Trifonov
7bbbe0a4db
fix darwin build
2022-08-18 19:31:25 +02:00
Sergei Trifonov
b064021cec
fix build
2022-08-18 16:57:31 +02:00
Sergei Trifonov
42a711015f
split concurrent_threads_soft_limit setting into two: concurrent_threads_soft_limit_{num,ratio_to_cores}
2022-08-18 14:51:08 +02:00
Nikita Mikhaylov
a7c3f23dfb
Merge branch 'master' into auto_convert_ordinary_to_atomic
2022-08-17 00:38:25 +02:00
Alexander Tokmakov
c9bb91c31f
Merge pull request #40232 from ClickHouse/fix_a_bug_with_symlinks_detection
...
Fix a bug with symlinks detection
2022-08-15 20:28:30 +03:00
Alexander Tokmakov
6f5a7c3bf7
fix a bug with symlinks detection
2022-08-15 12:30:47 +02:00
Alexey Milovidov
bada9ebefa
Merge branch 'master' into disable-zero-copy-replication
2022-08-14 10:40:16 +03:00
Alexey Milovidov
e774d28c43
Fix style
2022-08-14 04:16:48 +02:00
Alexey Milovidov
53ce2986de
Display server-side time in clickhouse-benchmark by default
2022-08-14 03:33:42 +02:00
Alexey Milovidov
f0c8998471
Add warning message
2022-08-14 01:49:00 +02:00
Alexey Milovidov
42c358aa3c
Add warning message
2022-08-14 01:47:36 +02:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
...
New caching strategies
2022-08-12 17:23:37 +02:00
Nikita Taranov
17956cb668
Extend protocol with query parameters ( #39906 )
2022-08-12 14:28:35 +02:00
Anton Popov
0383250383
execute DROP queries for fuzzed tables
2022-08-12 00:32:21 +00:00
Nikolay Degterinsky
644be9dc42
Remove unneded ReadBuffers
2022-08-11 13:27:09 +00:00
Nikolay Degterinsky
1b8ca90fcc
Add schema inference to clickhouse-obfuscator
2022-08-11 11:32:32 +00:00
Anton Popov
e347fa2f45
add fuzzer for table definition
2022-08-11 02:34:10 +00:00
Nikolay Degterinsky
496568101f
Merge pull request #39277 from ClibMouse/Structured-Logging-Support
...
Structured logging support
2022-08-10 22:57:48 +02:00
Kseniia Sumarokova
a4d513657c
Merge pull request #40010 from ClickHouse/catboost-further-preparation
...
Further preparation for catboost integration into library-bridge
2022-08-10 17:35:12 +02:00
Vladimir C
eb1ac41ac5
Merge pull request #40053 from ClickHouse/play-tab-in-textarea
...
Play: recognize tab in textarea
2022-08-10 12:53:28 +02:00
Robert Schulze
810221baf2
Assume unversioned server has version=0 and use tryParse() instead of from_chars()
2022-08-10 07:39:32 +00:00
Alexey Milovidov
34c8d2c3b4
Play: recognize tab in textarea
2022-08-10 08:33:58 +02:00
Mallik Hassan
666fb8dc42
Merge branch 'ClickHouse:master' into Structured-Logging-Support
2022-08-09 15:39:00 -03:00
Robert Schulze
e0d5020a92
Add simple versioning to the *-bridge-to-server protocol
...
- In general, it is expected that clickhouse-*-bridges and
clickhouse-server were build from the same source version (e.g. are
upgraded "atomically"). If that is not the case, we should at least
be able to detect the mismatch and abort.
- This commit adds a URL parameter "version", defined in a header shared
by the server and bridges. The bridge returns an error in case of
mismatch.
- The version is *not* send and checked for "ping" requests (used for
handshake), only for regular requests send after handshake. This is
because the internally thrown server-side exception due to HTTP
failure does not propagate the exact HTTP error (it only stores the
error as text), and as a result, the server-side handshake code
simply retries in case of error with exponential backoff and finally
fails with a "timeout error". This is reasonable as pings typically
fail due to time out. However, without a rework of HTTP exceptions,
version mismatch during ping would also appear as "timeout" which is
too misleading. The behavior may be changed later if needed.
- Note that introducing a version parameter does not represent a
protocol upgrade itself. Bridges older than the server will simply
ignore the field. Only servers older than the bridges receive an error
but such a situation should never occur in practice.
2022-08-08 19:40:37 +00:00
Robert Schulze
9952ab1099
Prefix class names "LibraryBridge*Handler" with "ExternalDictionary"
...
- necessary to disambiguate the names from "CatBoost"-"LibraryBridgeHandler"
which will be added in a next step
2022-08-08 17:16:46 +00:00
Yakov Olkhovskiy
d39e9f65de
Merge branch 'master' into fix-quota-key
2022-08-08 11:54:21 -04:00
Robert Schulze
ad0d060dc1
Merge pull request #39904 from ClickHouse/library-bridge-refactoring
...
Prepare library-bridge for catboost integration
2022-08-08 12:15:01 +02:00
Alexey Milovidov
64a1b0f2b8
Play UI: row numbers; cell selection; hysteresis
2022-08-08 04:25:03 +02:00
Alexey Milovidov
653b86d1e6
Play UI: row numbers; cell selection; hysteresis
2022-08-08 04:16:50 +02:00
Alexey Milovidov
2d83635643
Play UI: row numbers; cell selection; hysteresis
2022-08-08 03:57:02 +02:00
Alexey Milovidov
d914ddc766
Fix Play UI
2022-08-08 02:36:50 +02:00
alexX512
2be72560f4
Add loading of mark cache policy
2022-08-07 19:59:39 +00:00
alexX512
1133e42367
Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf
2022-08-07 19:16:40 +00:00
Yakov Olkhovskiy
29622746ea
Merge pull request #39936 from ClickHouse/cmake-fix-self-extracting-split
...
Don't create self-extracting clickhouse for split build
2022-08-07 07:23:24 -04:00
Yakov Olkhovskiy
b1f45fa787
Don't create self-extracting clickhouse for split build
2022-08-05 21:48:40 -04:00
Alexey Milovidov
ac93f0a2ca
Merge pull request #39910 from santrancisco/enhance-playui-authentication
...
Move username and password from URL parameters to Basic Authentication
2022-08-06 01:58:09 +03:00
Alexander Tokmakov
f2c4cad255
add flag that enables automatic canversion from Ordinary to Atomic
2022-08-05 21:41:25 +02:00
santrancisco
c7f0598a6f
Fall back to url parameters if file is opened locally
2022-08-05 13:10:45 +10:00
santrancisco
a25b858097
Move username and password from URL parameter to Basic Authentication header
2022-08-05 09:51:12 +10:00
Anton Popov
4f2c73f9f6
Merge remote-tracking branch 'upstream/master' into HEAD
2022-08-04 23:08:15 +00:00
Robert Schulze
20bb8a248e
Prepare server-side BridgeHelper for catboost integration
...
Wall of text, sorry, but I also had to document some stuff for myself:
There are three ways to communicate data using HTTP:
- the HTTP verb: for our purposes, PUT and GET,
- the HTTP path: '/ping', '/request' etc.,
- the HTTP URL parameter(s), e.g. 'method=libNew&dictionary_id=1234'
The bridge will use different handlers for communication with the
external dictionary library and for communication with the catboost
library. Handlers are created based on a combination of the HTTP verb
and the HTTP method. More specifically, there will be combinations
- GET + '/extdict_ping'
- PUT + '/extdict_request'
- GET + '/catboost_ping'
- PUT + '/catboost_request'.
For each combination, the bridge expects a certain set of URL
parameters, e.g. for the first combination parameter "dictionary_id" is
expected.
Starting with this commit, the library-bridge creates handlers based on
the first two combinations (the latter two combinations will be added
later). This makes the handler creation mechanism consistent with it's
counterpart in xdbc-bridge.
For that, it was necessary to make both IBridgeHelper methods
"getMainURI()" and "getPingURI()" pure virtual so that derived classes
(LibraryBridgeHelper and XDBCBridgeHelper) must provide custom URLs with
custom paths.
Side note 1: Previously, LibraryBridgeHelper sent HTTP URL parameter
"method=ping" during handshake (PING) but the library-bridge ignored
that parameter. We now omit this parameter, i.e.
LibraryBridgeHelper::PING was removed. Again, this makes things
consistent with xdbc-bridge.
Side note 2: xdbc-bridge is unchanged in this commit. Therefore,
XDBCBridgeHelper now uses the HTTP paths previously in the base class.
For funny reason, XDBCBridgeHelper did not use
IBridgeHelper::getMainURI() - it generates the URLs by itself. I kept it
that way for now but provided an implementation of getMainURI() anyways.
2022-08-04 19:29:51 +00:00
Robert Schulze
ea73b98fb9
Prepare library-bridge for catboost integration
...
- Rename generic file and identifier names in library-bridge to
something more dictionary-specific. This is needed because later on,
catboost will be integrated into library-bridge.
- Also: Some smaller fixes like typos and un-inlining non-performance
critical code.
- The logic remains unchanged in this commit.
2022-08-04 19:26:51 +00:00
Yakov Olkhovskiy
23037daf17
Merge branch 'master' into fix-quota-key
2022-08-04 12:14:49 -04:00
Alexey Milovidov
834cbbedce
Merge branch 'master' into escape_diag_creds
2022-08-04 01:59:50 +03:00
Yakov Olkhovskiy
2e34b384c1
update tcp protocol, add quota_key
2022-08-03 15:44:08 -04:00
Mallik Hassan
87bde54ee5
Merge branch 'ClickHouse:master' into Structured-Logging-Support
2022-08-02 10:44:05 -03:00
Alexey Milovidov
5f62863ab9
Merge branch 'master' into escape_diag_creds
2022-08-02 05:50:01 +03:00
Heena Bansal
d8db482b2e
Merge branch 'master' into Issue_39395
2022-08-01 12:22:16 -04:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling
...
Use std::popcount, ::countl_zero, ::countr_zero functions
2022-08-01 17:04:51 +02:00
HeenaBansal2009
50c98789b8
Updated as per comments
2022-08-01 07:05:50 -07:00
HeenaBansal2009
800ed546be
Updated as per comments
2022-08-01 07:03:36 -07:00
Ilya Yatsishin
c882bdc88e
Merge pull request #35968 from ClickHouse/interserver_listen_port
2022-08-01 13:57:17 +02:00
Alexander Tokmakov
80f9ba9186
Merge pull request #39690 from ClickHouse/show-addresses-in-stack-traces
...
Configuration parameter to hide addresses in stack traces
2022-08-01 14:20:37 +03:00
Alexey Milovidov
6a2f7d0c8f
Merge branch 'master' into interserver_listen_port
2022-07-31 22:51:38 +03:00
Alexey Milovidov
ebc4974188
Merge branch 'master' into escape_diag_creds
2022-07-31 22:46:10 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
...
- Introduced with the C++20 <bit> header
- The problem with __builtin_c(l|t)z() is that 0 as input has an
undefined result (*) and the code did not always check. The std::
versions do not have this issue.
- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
src/Common/BitHelpers.h) because the std:: versions only accept
unsigned inputs (and they also check that) and the casting would be
ugly.
(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Robert Schulze
dcc8751685
Disable harmful env var check to workaround failure to start the server
2022-07-31 08:55:07 +00:00
Robert Schulze
7c23e48b5b
Revert exclusion of libharmful (did not work anyways)
2022-07-31 08:05:12 +00:00
Robert Schulze
7fe106a0fb
Try to fix libharmful fail
2022-07-31 07:44:25 +00:00
Alexey Milovidov
fa9c3dcc48
Update programs/local/LocalServer.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-07-31 03:02:27 +03:00
Alexey Milovidov
a30dbed6b8
Update programs/local/LocalServer.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-07-31 03:02:20 +03:00
Alexey Milovidov
f80a4c184e
Merge branch 'master' into interserver_listen_port
2022-07-31 01:22:32 +03:00
Alexey Milovidov
9adfd259d0
Support in clickhouse-local
2022-07-30 01:50:45 +02:00
Alexey Milovidov
c0d7b6efc3
Suggestions from @tavplubix
2022-07-30 01:45:06 +02:00
Alexey Milovidov
4aa96767d5
Merge branch 'master' of github.com:ClickHouse/ClickHouse into show-addresses-in-stack-traces
2022-07-30 01:44:04 +02:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-29 12:17:43 +00:00
Dale Mcdiarmid
d9c585ecf1
Escape credentials for diagnostics tool
2022-07-29 12:25:03 +01:00
Azat Khuzhin
b90152b6ec
Fix clickhouse-su building in splitted build
...
- Add status log message
- Add it to clickhouse-bundle in shared build
- Move clickhouse-su.cpp into su.cpp, since executable does not have
include directories of linked libraries (dbms here), only
clickhouse-lib-su does, hence it cannot find includes
CI: https://github.com/ClickHouse/ClickHouse/runs/7566319416?check_suite_focus=true
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-29 11:36:51 +03:00
root
39369be318
commented out formatting tag in config.xml
2022-07-28 23:56:26 -07:00
root
f9e4d6370d
modified integration test as suggested
2022-07-28 23:51:59 -07:00
Alexey Milovidov
552b517e59
Allow to hide addresses in stack traces
2022-07-29 00:13:02 +02:00
root
bec7408a0c
reflected change requests asked on July 27
2022-07-28 09:20:28 -07:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-28 15:54:22 +00:00
Sergei Trifonov
4588e146a6
Merge pull request #37285 from arenadata/ADQM-445
...
Add concurrent_threads_soft_limit parameter
2022-07-28 11:46:43 +02:00
Roman Vasin
f35349eb08
Rename total_max_threads to concurrent_threads_soft_limit
2022-07-28 07:46:20 +00:00
Roman Vasin
37845389c8
Merge branch 'concurrency-control' of github.com:ClickHouse/ClickHouse into ADQM-445
2022-07-27 13:31:22 +00:00
Antonio Andelic
e62526720f
Address PR comments
2022-07-27 07:51:30 +00:00
Antonio Andelic
35b61cc94e
Merge branch 'master' into keeper-version-check
2022-07-26 07:47:57 +00:00
Alexey Milovidov
75d0232265
Merge pull request #39541 from ClickHouse/obfuscator-save-load
...
Add save/load capabilities to Obfuscator
2022-07-25 21:18:28 +03:00
Mallik Hassan
b45e44b7e7
Merge branch 'ClickHouse:master' into Structured-Logging-Support
2022-07-25 12:10:41 -03:00
Yakov Olkhovskiy
d0f5dcad25
Merge pull request #38936 from ClickHouse/cmake-self-extracting-executable
...
Build self-extracting-executable utils
2022-07-25 08:13:25 -04:00
Alexey Milovidov
1f4e7ea34c
Add a test
2022-07-25 05:37:07 +02:00
Alexey Milovidov
dde89c9606
Fix errors
2022-07-25 03:56:32 +02:00
Alexey Milovidov
68f5a397d7
Fix errors
2022-07-25 03:49:09 +02:00
Alexey Milovidov
52ad48d1ba
Better documentation
2022-07-25 03:32:01 +02:00
Alexey Milovidov
c09413e3b9
Add save/load capabilities to Obfuscator
2022-07-25 03:27:10 +02:00
Alexey Milovidov
071374b152
Remove SPLIT_BINARY
2022-07-24 01:15:54 +02:00
Antonio Andelic
afb6cb6824
Add KeeperContext
2022-07-23 14:45:31 +00:00
Antonio Andelic
3040ff0959
Merge branch 'master' into keeper-version-check
2022-07-22 08:07:55 +00:00
jasperzhu
614f3b14a2
Merge branch 'master' into dev_intel_iaa_deflate
2022-07-22 11:05:44 +08:00
HeenaBansal2009
06e8b78efa
Added FT testcase
2022-07-21 19:46:36 -07:00
Kruglov Pavel
49c839bdc1
Merge pull request #39413 from vitlibar/fix-interactive-client-with-older-server
...
Fix interactive client with older server
2022-07-21 11:37:20 +02:00
Antonio Andelic
7d30ab80c4
Fix compatibility
2022-07-21 09:31:06 +00:00
Vitaly Baranov
da0bdd474b
Fix code style.
2022-07-21 10:48:53 +02:00
Yakov Olkhovskiy
e7438cc433
Merge branch 'master' into cmake-self-extracting-executable
2022-07-21 00:25:08 -04:00
Yakov Olkhovskiy
b766552481
add native build for cross-compilation
2022-07-20 23:09:05 -04:00
Nikolai Kochetov
91043351aa
Fixing build.
2022-07-20 20:30:16 +00:00
Vitaly Baranov
8873d16afe
Fix interactive client with older server after https://github.com/ClickHouse/ClickHouse/pull/38970
2022-07-20 11:51:55 +02:00
HeenaBansal2009
d67e061915
Clickhouse-local fixes
2022-07-19 21:38:36 -07:00
jasperzhu
c044c67745
Merge branch 'master' into dev_intel_iaa_deflate
2022-07-19 15:32:57 +08:00
Azat Khuzhin
4f41d21626
Fix leaking of logger in clickhouse-disks
...
CI found [1]:
Direct leak of 256 byte(s) in 1 object(s) allocated from:
0 0xd8cb88d in operator new(unsigned long) (/usr/bin/clickhouse+0xd8cb88d) (BuildId: 7a3fd7b485701220)
1 0xde8943e in DB::DisksApp::main() build_docker/../programs/disks/DisksApp.cpp:157:41
2 0x38dca887 in Poco::Util::Application::run() build_docker/../contrib/poco/Util/src/Application.cpp:334:8
3 0xde8d72c in mainEntryClickHouseDisks(int, char**) build_docker/../programs/disks/DisksApp.cpp:219:20
4 0xd8cf47f in main build_docker/../programs/main.cpp:445:12
5 0x7f060ddce082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
CI: https://s3.amazonaws.com/clickhouse-test-reports/39299/37b4b52c12698e711aa931f10aec3909bca287b6/integration_tests__asan__actions__[2/3].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 12:35:53 +03:00
jinjunzh
dbac3a35b2
fixed cosmetic issues
2022-07-15 17:23:37 -04:00
root
af718f7e04
updated config.xml
2022-07-15 13:06:39 -07:00
root
03194eaeb9
Feature - Structured Logging Support resubmit
2022-07-15 10:01:30 -07:00
Vitaly Baranov
6bf7bffbeb
Correct the list of always accessible system tables.
2022-07-15 15:44:29 +02:00
Vitaly Baranov
fbb2e14d54
Add new table function viewIfPermitted().
2022-07-15 15:44:29 +02:00
Vitaly Baranov
c85b2b5732
Add option enabling that SELECT from the system database requires grant.
2022-07-15 15:44:25 +02:00
Robert Schulze
62149111d3
Fix build
2022-07-14 14:36:31 +00:00
Anton Popov
14a3c7b8cf
print changed MergeTree settings on fail
2022-07-13 20:57:24 +00:00
Yakov Olkhovskiy
e5f165d909
Merge branch 'master' into cmake-self-extracting-executable
2022-07-13 16:09:18 -04:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
...
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.
As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.
In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).
(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Antonio Andelic
37f799550b
Merge pull request #38072 from lingpeng0314/master
...
Add Keeper related monitoring data
2022-07-10 09:39:57 +02:00
jinjunzh
816e974ca2
rename deflate to deflate_qpl
2022-07-09 14:42:01 -04:00
jinjunzh
825b7511e4
improve cosmetics
2022-07-09 14:13:12 -04:00
Nikita Mikhaylov
a44157d81d
Allow globs in keys for clickhouse-extract-from-config
tool ( #38966 )
2022-07-08 16:13:32 +02:00
Anton Popov
499262dba9
allow to randomize merge tree settings
2022-07-07 22:16:01 +00:00
jinjunzh
6fa036f6ee
remove qpl from clickhouse-keeper
2022-07-06 23:48:09 -04:00
Yakov Olkhovskiy
8a3f124982
add self-extracting to clickhouse-bundle
2022-07-06 22:01:21 -04:00
Yakov Olkhovskiy
c6db15458a
build utils/self-extracting-executable/compressor whenever we want to build compressed binary
2022-07-06 20:40:41 -04:00
jinjunzh
ba38969e9c
enable qpl for clickhouse-keeper build
2022-07-05 15:28:41 -04:00
Alexey Milovidov
1f2266e45f
Merge pull request #38619 from ClickHouse/filimonov-clickhouse-su
...
Fix exception messages in clickhouse su
2022-07-05 07:11:20 +03:00
Mikhail f. Shiryaev
7a58a8c19b
Merge pull request #38647 from ClickHouse/clickhouse-diagnostics
...
Clickhouse diagnostics
2022-07-04 12:14:33 +02:00
Mikhail f. Shiryaev
446ead1af4
Fix typos issues it clickhouse-diagnostics
2022-07-04 10:59:39 +02:00
Mikhail f. Shiryaev
664d43e348
Add disabling tag no_docker to some go tests
2022-07-04 10:59:39 +02:00
lingpeng0314
cca94f01ae
add function tryGetKeeperDispatcher for context
...
- to make sure other case will get error if dispatcher not initialized
2022-07-04 13:35:08 +08:00
Mikhail f. Shiryaev
3513b41683
Fix linter issue
2022-07-01 11:43:14 +02:00
Mikhail f. Shiryaev
1553f67809
Can set version in Makefile
2022-07-01 11:43:13 +02:00
Mikhail f. Shiryaev
a70cda9670
Create a path for clickhouse-diagnostics installation
2022-07-01 11:43:12 +02:00
Mikhail f. Shiryaev
51556704e0
Rename module to reflect the current state
2022-07-01 11:43:11 +02:00
Mikhail f. Shiryaev
9531bbf497
Move clickhouse-diagnostics to programs
2022-07-01 11:43:07 +02:00
Robert Schulze
59236d60c9
Merge pull request #38654 from ClickHouse/better-naming-for-split-debug-symbols
...
Better naming for stuff related to splitted debug symbols
2022-07-01 09:28:41 +02:00
lingpeng0314
03a5ce542d
Remove Exception in getKeeperDispatcher
...
- to handle the nil case by caller
2022-07-01 11:27:40 +08:00
Robert Schulze
bb358617e1
Better naming for stuff related to splitted debug symbols
...
The previous name was slightly misleading, e.g. it is not about
"intalling stripped binaries" but about splitting debug symbols from the
binary.
2022-06-30 23:41:27 +02:00
Yakov Olkhovskiy
dd132d1beb
Merge pull request #38447 from ClickHouse/utility-self-extracting
...
Compress clickhouse executable
2022-06-30 13:18:40 -04:00
Nikita Mikhaylov
e66114984c
Update programs/su/clickhouse-su.cpp
2022-06-30 12:44:59 +02:00
Nikita Mikhaylov
b2cc6787ee
Update programs/su/clickhouse-su.cpp
2022-06-30 12:44:30 +02:00
filimonov
3949480ed3
Fix exception messages in clickhouse su
...
Fix exception messages in
2022-06-30 11:38:05 +02:00
Yakov Olkhovskiy
78ea290789
add remove command
2022-06-29 06:47:21 -04:00
jinjunzh
fe451a4317
remove compress sync interface
2022-06-28 22:52:11 -04:00
Yakov Olkhovskiy
5d36994c4d
self-extracting requires utils (uses utils/self-extracting-executable/compressor)
2022-06-27 11:41:23 -04:00
Alexander Tokmakov
7c37079a96
Merge branch 'master' into deprecate_ordinary_database
2022-06-27 14:46:59 +02:00
mergify[bot]
4e5fd226c8
Merge branch 'master' into utility-self-extracting
2022-06-27 12:26:16 +00:00
Yakov Olkhovskiy
8ce6b8226d
Update CMakeLists.txt
2022-06-27 08:25:21 -04:00
Yakov Olkhovskiy
39ea5ffdcb
compress clickhouse executable, new target 'self-extracted' is added
2022-06-27 01:36:27 -04:00
jasperzhu
b2f98fa73b
Merge branch 'master' into dev_intel_iaa_deflate
2022-06-27 10:21:21 +08:00
Alexey Milovidov
b89f01438e
Lower mutex scope
2022-06-25 03:05:55 +02:00
Kseniia Sumarokova
e1823f4c0a
Merge pull request #38364 from kssenii/better-error-message-in-dbc
...
Better error message for failed odbc query
2022-06-24 18:30:54 +02:00
Alexander Tokmakov
31dcc7634e
Merge branch 'master' into deprecate_ordinary_database
2022-06-24 18:16:07 +02:00
kssenii
dc73042d62
Better error messafe
2022-06-24 01:05:33 +02:00
Alexander Gololobov
50438a4b05
Fixed uninitialized optional
2022-06-23 23:55:21 +02:00
Alexander Tokmakov
dbf2763788
automatically convert system database to Atomic
2022-06-23 21:38:43 +02:00
Alexander Gololobov
02f3921c69
Added --recursive to clickhouse-disks list
2022-06-23 20:39:20 +02:00
Kseniia Sumarokova
b844c36a9d
Merge pull request #38150 from kssenii/clickhouse-disks-fixes
...
Some fixes for clickhouse-disks
2022-06-23 10:21:44 +02:00
Kseniia Sumarokova
3f83d2f8b3
Update ICommand.cpp
2022-06-22 13:00:48 +02:00
Alexey Milovidov
ba05b3edab
Merge pull request #38197 from ClickHouse/play-charts
...
Add primitive charting capabilities
2022-06-22 06:42:05 +03:00
Robert Schulze
0d80874d40
Merge pull request #38068 from ClickHouse/clang-tsa
...
Support for Clang Thread Safety Analysis (TSA)
2022-06-21 20:19:33 +02:00
kssenii
47a118beab
More fixes
2022-06-21 16:40:22 +02:00
mergify[bot]
5b24939369
Merge branch 'master' into play-charts
2022-06-21 14:20:52 +00:00
Roman Vasin
3de3ae1b64
Set ConcurrencyControl::Unlimited when total_max_threads 0 or undefined
2022-06-21 16:39:53 +03:00
Alexey Milovidov
7e9a3796e9
Fix wrong docs
2022-06-21 00:09:55 +02:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa
2022-06-20 16:39:32 +02:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA)
...
- TSA is a static analyzer build by Google which finds race conditions
and deadlocks at compile time.
- It works by associating a shared member variable with a
synchronization primitive that protects it. The compiler can then
check at each access if proper locking happened before. A good
introduction are [0] and [1].
- TSA requires some help by the programmer via annotations. Luckily,
LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
std::shared_mutex and std::scoped_lock. This commit enables them
(--> contrib/libcxx-cmake/CMakeLists.txt).
- Further, this commit adds convenience macros for the low-level
annotations for use in ClickHouse (--> base/defines.h). For
demonstration, they are leveraged in a few places.
- As we compile with "-Wall -Wextra -Weverything", the required compiler
flag "-Wthread-safety-analysis" was already enabled. Negative checks
are an experimental feature of TSA and disabled
(--> cmake/warnings.cmake). Compile times did not increase noticeably.
- TSA is used in a few places with simple locking. I tried TSA also
where locking is more complex. The problem was usually that it is
unclear which data is protected by which lock :-(. But there was
definitely some weird code where locking looked broken. So there is
some potential to find bugs.
*** Limitations of TSA besides the ones listed in [1]:
- The programmer needs to know which lock protects which piece of shared
data. This is not always easy for large classes.
- Two synchronization primitives used in ClickHouse are not annotated in
libcxx:
(1) std::unique_lock: A releaseable lock handle often together with
std::condition_variable, e.g. in solve producer-consumer problems.
(2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
considered a design flaw + typically it is slower than a standard
mutex. In this commit, one std::recursive_mutex was converted to
std::mutex and annotated with TSA.
- For free-standing functions (e.g. helper functions) which are passed
shared data members, it can be tricky to specify the associated lock.
This is because the annotations use the normal C++ rules for symbol
resolution.
[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
mergify[bot]
f3264e50ec
Merge branch 'master' into play-charts
2022-06-20 14:01:47 +00:00
mergify[bot]
2d25daf64d
Merge branch 'master' into clickhouse-disks-fixes
2022-06-20 11:20:59 +00:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix
...
Send profile events for INSERT queries (previously only SELECT was supported)
2022-06-20 12:16:29 +02:00
lingpeng0314
1d463718ca
Change getKeeperDispatcher error type
2022-06-20 17:11:15 +08:00
Alexey Milovidov
240d896eb4
Add primitive charting capabilities
2022-06-18 18:20:53 +02:00
Alexey Milovidov
eba897d4ec
Add primitive charting capabilities
2022-06-18 18:15:45 +02:00
Maksim Kita
6b748f2193
AArch64 enable trace collection
2022-06-17 17:53:51 +02:00
kssenii
7520bfeab7
Fix
2022-06-16 19:50:09 +02:00
alesapin
ca33ff93cb
Merge pull request #37555 from ClickHouse/revert-37534-revert-37036-keeper-preprocess-operations
...
Add support for preprocessing ZooKeeper operations and real-time digest in `clickhouse-keeper`
2022-06-16 17:29:00 +02:00
Azat Khuzhin
3b29db6e9f
LocalServer: remove superfluous ProgressIndicator.h
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-16 11:59:01 +03:00
jasperzhu
bca18b2caf
Merge branch 'master' into dev_intel_iaa_deflate
2022-06-16 08:56:27 +08:00
Robert Schulze
b936abe813
+ comments about keeping stuff in sync
2022-06-16 00:07:59 +02:00
Robert Schulze
bc46cef63c
Minor follow-up
...
- change ELF section name to ".clickhouse.hash" (lowercase seems
standard)
- more expressive/concise integrity check messages at startup
2022-06-14 08:52:13 +00:00
Antonio Andelic
e5504f1b33
Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations
2022-06-13 15:43:10 +00:00
Robert Schulze
bc6f30fd40
Move binary hash to ELF section ".ClickHouse.hash"
2022-06-13 08:46:23 +00:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros
...
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.
__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
2022-06-10 10:22:31 +02:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type"
2022-06-07 13:53:10 +02:00
Varinara
1f8fab41f1
Fixe2: allow config path at any position
2022-06-06 16:52:58 +03:00
Varinara
6aa9e5d649
Fixe: allow config path at any position
2022-06-06 16:52:58 +03:00
Varinara
e35d1460bf
Corrections
2022-06-06 16:52:58 +03:00
Varinara
a330d3cb4a
Corrections
2022-06-06 16:52:58 +03:00