Alexey Milovidov
ab4db2d0c4
Fix 5/6 of trash
2022-09-19 08:50:53 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected
...
Move CatBoost evaluation into clickhouse-library-bridge
2022-09-16 13:12:09 +02:00
kssenii
52ef3758c4
Merge remote-tracking branch 'upstream/master' into fix-thread-status
2022-09-13 16:34:31 +02:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading
...
Allow to load marks in threadpool in advance
2022-09-13 12:31:58 +02:00
kssenii
b7d751b782
Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading
2022-09-11 13:23:30 +02:00
Alexander Tokmakov
9bcec2d49a
stop background ops before converting db engine
2022-09-09 18:27:19 +02:00
kssenii
6a885430d7
Add assertion
2022-09-09 13:36:23 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view
...
- The deleted function modelEvaluate() was superseded by
catboostEvaluate().
- Also delete the external model repository, as modelEvaluate() was it's
last user. Additionally remove the system view SYSTEM.MODELS for
inspecting the repository.
- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
made a no-op instead of deleted.
Why?
The reason is that RBAC in distributed setups works by storing
privileges (granted and revoked) as plain SQL statements in Keeper.
Nodes read these statements at startup and parse them. If a privilege
for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
nodes would fail to come up.
Considered but rejected alternatives:
- Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
of no-op behavior, too brittle.
- Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
manipulation of Keeper nodes or via SQL by dropping the privileges.
Needs user intervention during upgrade.
2022-09-08 09:10:11 +00:00
kssenii
e51313b6b3
Get rid of static threadpools
2022-09-07 17:48:11 +02:00
kssenii
487bc0fba3
Fix heap use after free
2022-09-06 16:50:58 +02:00
kssenii
4c8aa04b73
Revert "Fix"
...
This reverts commit ec55c90f36
2022-09-05 20:12:40 +02:00
kssenii
ec55c90f36
Fix
2022-09-01 17:04:34 +02:00
kssenii
a5f69517ae
Allow to load marks in threadpool
2022-08-30 20:20:05 +02:00
Frank Chen
f17d56b528
Merge branch 'master' into tracing_context_propagation
2022-08-30 14:24:36 +08:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330
...
Add config option warning_supress_regexp
2022-08-29 14:02:00 +02:00
Azat Khuzhin
c801469e9e
Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
...
The problem is that the buffer size validated after marks reading in
MergeTreeReaderStream::init(), since it requires to read marks first.
And later it is passed to AsynchronousReadBufferFromFileDescriptor,
which throws LOGICAL_ERROR because buffer_size < alignment.
Fix this my simply disallow such values for max_read_buffer_size (I
thougt about modifying createReadBufferFromFileBase(), but it is not
used for remote reads -- remote_fs_buffer_size).
Fixes : #40669
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-27 14:12:07 +02:00
vdimir
414872b1b9
Rename logger.warning_supress_regex -> warning_supress_regexp
2022-08-25 11:00:01 +02:00
vdimir
19cb068f64
Add config option warning_supress_regex
2022-08-25 11:00:01 +02:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation
2022-08-25 10:07:16 +08:00
vdimir
1321ac87b5
Minor fixes
2022-08-24 16:14:07 +00:00
kssenii
26384b2543
Fix
2022-08-22 00:02:05 +02:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation
2022-08-18 20:59:07 +08:00
Kseniia Sumarokova
d8fd1f9c37
Merge pull request #36171 from kssenii/make-cache-composable
...
Make cache composable, allow not to evict certain files (regarding .idx, .mrk, ..), delete old cache version
2022-08-18 12:37:34 +02:00
Vladimir C
7ac0eb1af1
Merge pull request #40035 from ClickHouse/vdimir/cluster-discovery-observer
...
Add observer mode to (zoo)keeper cluster discovery feature
2022-08-18 10:53:49 +02:00
kssenii
eb26b219b9
Merge master
2022-08-16 00:56:27 +02:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
...
New caching strategies
2022-08-12 17:23:37 +02:00
vdimir
cef6c4f8ce
Add observer mode to (zoo)keeper cluster discovery feature
2022-08-12 13:26:12 +00:00
Nikita Taranov
17956cb668
Extend protocol with query parameters ( #39906 )
2022-08-12 14:28:35 +02:00
alexX512
62f84b88eb
Review fixes
2022-08-07 20:02:21 +00:00
alexX512
2be72560f4
Add loading of mark cache policy
2022-08-07 19:59:39 +00:00
alexX512
1133e42367
Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf
2022-08-07 19:16:40 +00:00
Frank Chen
40c6e4c0d6
Merge remote-tracking branch 'origin/master' into tracing_context_propagation
2022-08-02 10:02:09 +08:00
kssenii
21ebf8874e
Merge remote-tracking branch 'upstream/master' into make-cache-composable
2022-07-19 13:36:42 +02:00
kssenii
0aed5a602a
Merge master
2022-07-18 16:16:28 +02:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef
...
- to be replaced by std::string_view
- suggested in #39262
2022-07-17 17:26:02 +00:00
kssenii
8ad26b3111
Merge master
2022-07-16 22:50:03 +02:00
Robert Schulze
deda29b46b
Pass const StringRef by value, not by reference
...
See #39224
2022-07-15 11:34:56 +00:00
Kruglov Pavel
a944a92d4c
Merge pull request #39224 from Avogar/string-view-by-value
...
Pass const std::string_view by value, not by reference
2022-07-15 13:05:56 +02:00
avogar
9291d33080
Pass const std::string_view & by value, not by reference
2022-07-14 16:11:57 +00:00
Sergei Trifonov
b2317ff710
renaming to be consistent
2022-07-14 17:33:22 +02:00
Sergei Trifonov
15ab3bc99f
use context->getWriteSettings()
2022-07-13 19:48:57 +02:00
Sergei Trifonov
f854507729
merge master and resolve conflicts
2022-07-13 19:07:28 +02:00
kssenii
0d2d3ac2b4
Merge master
2022-07-13 13:26:15 +02:00
Sergei Trifonov
43779ec280
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-11 14:59:39 +02:00
Frank Chen
da57a993e4
Fix CI
2022-07-09 13:43:10 +08:00
Frank Chen
9711559391
Remove extra code
2022-07-08 18:56:07 +08:00
Frank Chen
2e8c530bed
Move thread trace context out of ThreadStatus
2022-07-07 17:41:10 +08:00
kssenii
e55169ebad
Fix
2022-07-05 20:15:35 +02:00
lingpeng0314
cca94f01ae
add function tryGetKeeperDispatcher for context
...
- to make sure other case will get error if dispatcher not initialized
2022-07-04 13:35:08 +08:00
lingpeng0314
03a5ce542d
Remove Exception in getKeeperDispatcher
...
- to handle the nil case by caller
2022-07-01 11:27:40 +08:00
lingpeng0314
1d463718ca
Change getKeeperDispatcher error type
2022-06-20 17:11:15 +08:00
kssenii
4813b90281
Merge master
2022-06-13 21:32:00 +02:00
kssenii
aaf9a46f07
Merge master
2022-06-11 15:56:05 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size
...
Support to set max cache size for per query in local cache
2022-06-10 16:17:18 +02:00
KinderRiven
6883ecda96
fix
2022-06-08 14:19:50 +08:00
KinderRiven
3b2a427002
fix
2022-06-07 11:50:56 +08:00
KinderRiven
13e74bef5e
support skip_download_if_exceeds_query_cache
2022-06-06 02:36:23 +08:00
KinderRiven
3ffb0b3549
support_max_query_cache_size
2022-06-05 16:21:36 +08:00
Alexander Tokmakov
3d346c766a
better code
2022-06-01 16:49:26 +02:00
Alexander Tokmakov
4e52f45695
Merge branch 'master' into fix_trash
2022-05-28 19:43:19 +02:00
kssenii
25b1ed526d
Merge master
2022-05-27 16:50:45 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage
...
Separate object storage operations from disks
2022-05-27 13:59:50 +02:00
Nikolai Kochetov
fea2401f1f
Merge pull request #37532 from ClickHouse/add-separate-mutex-for-factories-info
...
Use a separate mutex for query_factories_info in Context.
2022-05-26 13:03:28 +02:00
Alexander Tokmakov
3d259fa0de
Update Context.cpp
2022-05-25 23:18:07 +03:00
kssenii
b89a4edc04
Merge master
2022-05-25 20:55:04 +02:00
alesapin
6f5c86e55e
Merge branch 'master' into i_object_storage
2022-05-25 20:49:01 +02:00
Alexander Tokmakov
779e6ea0b9
make it better, fix on cluster queries
2022-05-25 20:17:49 +02:00
Nikolai Kochetov
7b681fa8ac
Fixing build.
2022-05-25 17:15:23 +02:00
Nikolai Kochetov
6370c29049
Use a separate mutex for query_factories_info in Context.
2022-05-25 14:16:59 +00:00
Vitaly Baranov
497c70d786
Merge pull request #37269 from vitlibar/rework-access-control-notifications
...
Rework AccessControl's notifications.
2022-05-24 17:47:45 +02:00
Alexander Tokmakov
d0f998fb88
Merge branch 'master' into fix_trash
2022-05-23 21:25:56 +02:00
alesapin
06c3dd69c0
Move directories
2022-05-22 13:51:48 +02:00
kssenii
9aa20193d5
Merge master
2022-05-22 00:56:20 +02:00
kssenii
ffd8d9723a
Continuation
2022-05-22 00:21:40 +02:00
Vitaly Baranov
58f4a86ec7
Rework notifications used in access management.
2022-05-21 10:15:39 +02:00
alesapin
654b27e307
Merge branch 'master' into i_object_storage
2022-05-20 11:56:13 +02:00
alesapin
92c15ec97c
Get rid of IDiskRemote
2022-05-19 20:07:15 +02:00
Alexander Tokmakov
3bc39f9929
fixes
2022-05-18 21:57:20 +02:00
Alexander Tokmakov
9772924d06
Merge branch 'master' into fix_trash
2022-05-18 17:27:54 +02:00
Robert Schulze
43945cea1b
Fixing some warnings
2022-05-16 20:59:27 +02:00
kssenii
646cf4cc95
Better
2022-05-14 14:26:04 +02:00
kssenii
58faaf0ea3
Merge master
2022-05-13 18:51:07 +02:00
Azat Khuzhin
a0ba113e5e
Mark Context::getInterserverIOHandler() as const
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-12 16:54:01 +03:00
Azat Khuzhin
a04b6ad314
Mark Context::getInterserverCredentials() as const
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-12 16:54:01 +03:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log
...
Log into filesystem_cache_log when cache not even attempted
2022-05-12 12:38:36 +02:00
Kruglov Pavel
af3db5a992
Merge pull request #36726 from PolyProgrammist/s3clustermacro
...
Handle cluster macro in s3Cluster table function
2022-05-11 18:00:57 +02:00
Vasily Nemkov
de9a06df5b
Merge branch 'master' into fix_session_log_crash
2022-05-09 17:00:24 +04:00
Vasily Nemkov
58953b5f28
Fixed issues with concurrent access to User object
...
ContextAccess might be updated from another thread, nullifying UserPtr.
Can happen if user was dropped in concurrent query.
This could crash server under certain conditions.
2022-05-09 15:56:50 +03:00
kssenii
fd1ab053f8
Merge master
2022-05-06 12:50:33 +02:00
Vitaly Baranov
dd51265218
Merge pull request #36864 from vitlibar/backup-improvements-4
...
Backup improvements
2022-05-05 15:37:51 +02:00
alesapin
aba78e5ed0
Merge pull request #36859 from ClickHouse/revert-36858-revert-35637-memory-overcommit-free
...
Revert "Revert "Memory overcommit: continue query execution if memory is available""
2022-05-04 23:43:00 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3
2022-05-03 20:46:16 +00:00
kssenii
0266cdf125
Add entries to cache log when cache was not event attempted
2022-05-03 18:29:41 +02:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing
...
Parallel parsing of hive text format
2022-05-03 15:56:28 +02:00
Vitaly Baranov
5257ce31f8
Improved using ThreadPool for making backup or restoring, changed columns in system.backups.
2022-05-03 11:03:13 +02:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available""
2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available"
2022-05-03 00:36:50 +02:00
Dmitry Novik
71b6f89166
Merge pull request #35637 from ClickHouse/memory-overcommit-free
...
Memory overcommit: continue query execution if memory is available
2022-05-02 19:00:18 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
...
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
previously allowed.
Hence, this change
- removes shared_ptr_helper and as a result all inherited create() methods,
- instead, Storage objects are now created using make_shared<>() by the
caller (for that to work, many constructors had to be made public), and
- all Storage classes were marked as noncopyable using boost::noncopyable.
In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
KinderRiven
d595df1604
fix
2022-05-01 19:45:16 +08:00
KinderRiven
ce9a6965d0
impl system.log_table
2022-05-01 19:45:16 +08:00
KinderRiven
5a361a0db5
impl cache_log in filecache
2022-05-01 19:45:16 +08:00
KinderRiven
4db54ff6bc
add log record
2022-05-01 19:45:16 +08:00
KinderRiven
29f0471caf
add cache log
2022-05-01 19:45:16 +08:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase
...
Reorganize source files so that base won't depend on Common
2022-04-30 09:40:58 +03:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free
2022-04-29 19:31:59 +00:00
Nikolai Kochetov
5807ca47b0
Merge pull request #36425 from ClickHouse/pool-size-hot-reload
...
Allow to increase number of threads and tasks for background executors
2022-04-29 17:24:12 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common
2022-04-29 10:26:35 +08:00
Maksim Kita
04429d85db
Added user_defined_path config setting
2022-04-28 22:13:33 +02:00
Vadim Volodin
233f0b4b52
Handle cluster macro in s3Cluster table function
2022-04-28 18:38:17 +03:00
taiyang-li
99dee35b6e
parallel parsing of hive text format
2022-04-26 14:33:10 +08:00
alesapin
04afe552f8
Trying to avoid segfaults
2022-04-25 22:51:09 +02:00
Vitaly Baranov
ed0af9d4a0
Improve shutdown with async backup/restore.
2022-04-25 16:34:34 +02:00
Nikita Mikhaylov
9baf7b14dc
Fix build
2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
aaa7112405
Even better
2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
36bdee0499
Deleted settings
2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
4818b63979
Better
2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
8c325c2cdd
Better
2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
31ccb9c1c3
Done
2022-04-22 13:56:16 +00:00
kssenii
24e376337d
Merge with master
2022-04-21 12:33:56 +02:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings
...
Better settings configuration for s3
2022-04-20 10:22:06 +02:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains
...
Activate clang-tidy warning "readability-container-contains"
2022-04-20 08:15:46 +02:00
kssenii
7cfc0ac433
Merge with master
2022-04-19 15:13:45 +02:00
mergify[bot]
3c1dd44b80
Merge branch 'master' into fix-insert-select
2022-04-19 12:17:31 +00:00
kssenii
1f9a597b77
Merge master
2022-04-19 11:31:13 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
...
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexey Milovidov
242919eddd
Remove abbreviation
2022-04-18 01:02:49 +02:00
avogar
ae01646136
Place new logic of schema inference in insert select from table function under setting
2022-04-14 19:05:20 +00:00
kssenii
0912b038fa
Merge master
2022-04-13 15:30:01 +02:00
kssenii
37242a0103
Allow non-evictable file segments based on predicate
2022-04-12 16:55:25 +02:00
kssenii
9804c39de7
Composable cache
2022-04-12 14:33:13 +02:00
Alexander Tokmakov
49c35f3261
Merge branch 'master' into mvcc_prototype
2022-04-08 13:34:40 +02:00
kssenii
87855c4ae7
Merge master
2022-04-08 12:32:48 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements
...
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
2022-04-08 12:06:26 +02:00
Alexander Tokmakov
6a46da93ae
Merge branch 'master' into mvcc_prototype
2022-04-07 23:22:19 +02:00
Alexander Tokmakov
7f54e7b422
Merge branch 'master' into mvcc_prototype
2022-04-07 15:14:06 +02:00
Kruglov Pavel
73adbb4c15
Merge pull request #35986 from amosbird/better-scalar1
...
Fix performance regression of scalar query
2022-04-07 14:07:59 +02:00
Amos Bird
df06f9f974
Fix performance regression of scalar query
2022-04-06 17:50:22 +08:00
kssenii
4e50da722b
Better s3 settings
2022-04-04 16:14:56 +02:00
Alexander Tokmakov
a2167f12b8
Merge branch 'master' into mvcc_prototype
2022-04-04 14:24:23 +02:00
Azat Khuzhin
14538f6456
Add system.processors_profile_log
...
This is the system table that will contain Processors level profiling.
v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-04 13:19:41 +03:00
kssenii
6c8e073a61
Merge master
2022-04-01 16:48:29 +02:00
Kruglov Pavel
a217e9e9d0
Remove unused header
2022-03-31 14:27:43 +02:00
kssenii
0fc92fe2aa
Get rid of all "remote" mentions
2022-03-30 17:15:29 +02:00
avogar
849596c7a2
Improve schema inference for insert select queries
2022-03-30 12:44:53 +00:00
Alexander Tokmakov
287d858fda
Merge branch 'master' into mvcc_prototype
2022-03-29 16:24:12 +02:00
kssenii
d2a3cfe5dc
Cache on all write operations
2022-03-23 19:14:33 +01:00
taiyang-li
8dbf1c60e7
merge master and fix conflict
2022-03-23 11:36:50 +08:00
kssenii
59b7394caf
WriteSettings
2022-03-21 09:52:48 +01:00
Alexander Tokmakov
c2ac8d4a5c
review fixes
2022-03-16 21:05:34 +01:00
