thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-19 04:42:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
110,391 Commits 9,326 Branches 1,783 Tags 2.2 GiB
b541ec485d
Commit Graph

892 Commits

Author SHA1 Message Date
Smita Kulkarni
9aa06e48f6 Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-10-31 15:46:52 +01:00
Vladimir Chebotaryov
d17b7387f9 Reworked changes to std::shared_ptr<const RowPolicyFilter>. 2022-10-27 10:42:55 +03:00
Vladimir Chebotaryov
4cabe1f57c Added applied row-level policies to system.query_log. 2022-10-27 10:41:47 +03:00
Sergei Trifonov
34b43f8b4b
Merge branch 'master' into storage-system-moves 2022-10-25 22:56:45 +02:00
serxa
55a86bc2de Add active moves system table 2022-10-25 13:56:46 +00:00
Nikolai Kochetov
c4685f6994
Merge branch 'master' into fix-42575 2022-10-25 15:31:32 +02:00
Nikolai Kochetov
5c4444237e
Merge pull request #42617 from ClickHouse/revert-revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3 
Revert revert 41268 disable s3 parallel write for part moves to disk s3
 2022-10-25 14:29:27 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
Nikolai Kochetov
93310fe8b1 Fix typo. 2022-10-24 15:51:21 +00:00
Nikolai Kochetov
d5bd905a41 Try to fix data race on zookeeper vs DDLWorker at server shutdown. 2022-10-24 15:09:35 +00:00
Nikolai Kochetov
57ffcb7ed0 Revert "Merge pull request #41681 from ClickHouse/revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3" 
This reverts commit 7956c2becf, reversing
changes made to 57be648984.
 2022-10-20 19:51:27 +02:00
Kruglov Pavel
9588653a6e
Merge branch 'master' into improve-schema-inference-insert-select 2022-10-18 15:17:27 +02:00
Alexey Milovidov
e014293fe8 Fix tidy 2022-10-17 19:29:14 +02:00
Smita Kulkarni
3034229c37 Merge branch 'master' of github.com:ClickHouse/ClickHouse into 40907_Parameterized_views_as_table_functions 2022-10-17 18:37:31 +02:00
Smita Kulkarni
614fd4cf42 Added is_parameterized_view to Context and removed flag from ASTs, updated to use tryGetTable, addressed review comments - 40907 Parameterized views as table functions 2022-10-17 18:11:22 +02:00
Alexey Milovidov
f88ed8195b Fix trash 2022-10-17 04:21:08 +02:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf 
Refactor the implementation of user-defined functions
 2022-10-15 12:33:10 +02:00
Kruglov Pavel
11a9b5977a
Fix typos 2022-10-14 19:22:05 +02:00
nvartolomei
7d9f097d3b
Fix bad copy & paste when loading background_merges_mutations_concurrency_ratio setting (#42315) 2022-10-14 19:19:14 +02:00
Kruglov Pavel
f1dd247181
Merge branch 'master' into improve-schema-inference-insert-select 2022-10-14 17:16:58 +02:00
avogar
10335aec13 Improve using structure from insertion table in table functions 2022-10-14 15:09:35 +00:00
Vitaly Baranov
2e59d671fc Split UserDefinedSQLObjectsLoader to interface and implementation. 2022-10-13 10:32:32 +02:00
FArthur-cmd
b6304c46d4 move default settings to different part of ast 2022-10-11 18:25:28 +00:00
Smita Kulkarni
ec95294621 Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-10-05 11:48:14 +02:00
Smita Kulkarni
a0b1085f3b Fixed issues with using database name in SELECT and added tests - 40907 Parameterized views as table functions 2022-10-04 17:32:48 +02:00
Anton Popov
148995894f add system table asynchronous_inserts_log 2022-10-03 21:35:55 +00:00
kssenii
229414ed09 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-09-30 17:22:23 +02:00
kssenii
d538b5f3b8 Allow custom disk configuration 2022-09-30 17:20:10 +02:00
vdimir
f495361e28
fixes for TemporaryDataOnDisk 2022-09-29 10:09:29 +00:00
vdimir
14cd2b632c
fix Context::shutdown 2022-09-29 09:51:49 +00:00
vdimir
1234b70118
Handle decorator checking tmp disk type 2022-09-29 09:51:49 +00:00
vdimir
0f1a7c252d
better TemporaryDataOnDisk 2022-09-29 09:51:46 +00:00
Vladimir C
dd20fa37f5
Fix typos 2022-09-29 09:51:45 +00:00
vdimir
9f3f34548c
Allow to create temporaty streams on leaf TemporaryDataOnDisk 2022-09-29 09:51:45 +00:00
vdimir
15c7a3be34
Temp data on disk: build 2022-09-29 09:51:41 +00:00
vdimir
c0898ce289
Use abstraction for temporary data on disk in Sort and Aggregation 2022-09-29 09:51:41 +00:00
vdimir
367c77f0fb
Check if tmp volume contains only local disks 2022-09-29 09:51:39 +00:00
Sergei Trifonov
8edac7eba1
Merge branch 'master' into io-scheduler 2022-09-28 22:17:17 +02:00
Smita Kulkarni
f78f846503 Fixed issues executeTableFunctions in Context.cpp to fall back on TableFunction if not parameterized view & updated test - 40907 Parameterized views as table functions 2022-09-28 13:23:11 +02:00
Smita Kulkarni
e6672832b9 Removed unused errorcodes in Context.cpp - 40907 Parameterized views as table functions 2022-09-28 10:17:04 +02:00
Smita Kulkarni
618f63d6c7 Updated executeTableFunctions in Context.cpp to check for table/view & fallback to function to fix test fails - 40907 Parameterized views as table functions 2022-09-28 10:00:12 +02:00
kssenii
01c4299481 Support single disk instead of storage policy 2022-09-27 22:34:31 +02:00
Smita Kulkarni
bbc33a54b2 Addressed review comments - 40907 Parameterized views as table functions 
Implementation
* Fix for clang-today build fails - updated to use const in Context.cpp & const function in ActionsVisitior.cpp
* Updated to use QueryParameterVisitor to check if query has query parameters
* Updated executeTableFunction to check if table/table exists instead of try-catch approach
* Fixed small review comments and style comments.
Documentation:
* Addressed review comments and added the LIVE view part which was removed by mistake in the previous commits.
 2022-09-27 16:30:59 +02:00
serxa
b057d07977 IO scheduling subsystem 2022-09-27 13:26:41 +00:00
kssenii
0a801dad2a Merge remote-tracking branch 'upstream/master' into fix-thread-status 2022-09-23 19:39:07 +02:00
kssenii
30726721ad Fix threadpool reader (for local fs) 2022-09-23 19:35:16 +02:00
Smita Kulkarni
bb451b2b27 40907 Parameterized views as table functions 
Implementation:
* Updated parsers by adding a bool allow_query_parameters while creating ordinary view, which is used in interpreters to allow query parameters in SELECT.
* Added a check in ActionsVisitor if multiple parameters have same names while creating parameterised view.
* Added bool in StorageView to represent parameterized view.
* Updated processing of SELECT with parameter values to check for views and added substitution of values in the query parameters.

Testing:
* Added a test tests/queries/0_stateless/02428_parameterized_view.sql

Documentation:
* Updated the english documentation for VIEW.
 2022-09-23 16:46:57 +02:00
kssenii
ab702e43fc Merge remote-tracking branch 'upstream/master' into fix-thread-status 2022-09-23 15:21:33 +02:00
Nikita Mikhaylov
dbcba1490c
Try to make MergeTreeRestartingThread more reactive in case of expired sessions (#41092) 2022-09-23 14:40:04 +02:00
Alexander Tokmakov
021b882455 Revert "Disable parallel s3 multipart upload for part moves. (#41268)" 
This reverts commit 0414d95878.
 2022-09-22 13:59:25 +03:00
Alexey Milovidov
45afacdae4
Merge pull request #41186 from ClickHouse/fix-three-fourth-of-trash 
Fix more than half of the trash
 2022-09-22 07:28:26 +03:00
Nikolai Kochetov
0414d95878
Disable parallel s3 multipart upload for part moves. (#41268) 
* Disable parallel s3 multipart upload for part moves.

* Add setting s3_allow_parallel_part_upload
 2022-09-21 19:10:32 +02:00
Alexey Milovidov
ab4db2d0c4 Fix 5/6 of trash 2022-09-19 08:50:53 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
kssenii
52ef3758c4 Merge remote-tracking branch 'upstream/master' into fix-thread-status 2022-09-13 16:34:31 +02:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading 
Allow to load marks in threadpool in advance
 2022-09-13 12:31:58 +02:00
kssenii
b7d751b782 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-11 13:23:30 +02:00
Alexander Tokmakov
9bcec2d49a stop background ops before converting db engine 2022-09-09 18:27:19 +02:00
kssenii
6a885430d7 Add assertion 2022-09-09 13:36:23 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view 
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
 2022-09-08 09:10:11 +00:00
kssenii
e51313b6b3 Get rid of static threadpools 2022-09-07 17:48:11 +02:00
kssenii
487bc0fba3 Fix heap use after free 2022-09-06 16:50:58 +02:00
kssenii
4c8aa04b73 Revert "Fix" 
This reverts commit ec55c90f36.
 2022-09-05 20:12:40 +02:00
kssenii
ec55c90f36 Fix 2022-09-01 17:04:34 +02:00
kssenii
a5f69517ae Allow to load marks in threadpool 2022-08-30 20:20:05 +02:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330 
Add config option warning_supress_regexp
 2022-08-29 14:02:00 +02:00
Azat Khuzhin
c801469e9e Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks 
The problem is that the buffer size validated after marks reading in
MergeTreeReaderStream::init(), since it requires to read marks first.

And later it is passed to AsynchronousReadBufferFromFileDescriptor,
which throws LOGICAL_ERROR because buffer_size < alignment.

Fix this my simply disallow such values for max_read_buffer_size (I
thougt about modifying createReadBufferFromFileBase(), but it is not
used for remote reads -- remote_fs_buffer_size).

Fixes: #40669
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 14:12:07 +02:00
vdimir
414872b1b9 Rename logger.warning_supress_regex -> warning_supress_regexp 2022-08-25 11:00:01 +02:00
vdimir
19cb068f64 Add config option warning_supress_regex 2022-08-25 11:00:01 +02:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
vdimir
1321ac87b5
Minor fixes 2022-08-24 16:14:07 +00:00
kssenii
26384b2543 Fix 2022-08-22 00:02:05 +02:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation 2022-08-18 20:59:07 +08:00
Kseniia Sumarokova
d8fd1f9c37
Merge pull request #36171 from kssenii/make-cache-composable 
Make cache composable, allow not to evict certain files (regarding .idx, .mrk, ..), delete old cache version
 2022-08-18 12:37:34 +02:00
Vladimir C
7ac0eb1af1
Merge pull request #40035 from ClickHouse/vdimir/cluster-discovery-observer 
Add observer mode to (zoo)keeper cluster discovery feature
 2022-08-18 10:53:49 +02:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master 
New caching strategies
 2022-08-12 17:23:37 +02:00
vdimir
cef6c4f8ce
Add observer mode to (zoo)keeper cluster discovery feature 2022-08-12 13:26:12 +00:00
Nikita Taranov
17956cb668
Extend protocol with query parameters (#39906) 2022-08-12 14:28:35 +02:00
alexX512
62f84b88eb Review fixes 2022-08-07 20:02:21 +00:00
alexX512
2be72560f4 Add loading of mark cache policy 2022-08-07 19:59:39 +00:00
alexX512
1133e42367 Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf 2022-08-07 19:16:40 +00:00
Frank Chen
40c6e4c0d6 Merge remote-tracking branch 'origin/master' into tracing_context_propagation 2022-08-02 10:02:09 +08:00
kssenii
21ebf8874e Merge remote-tracking branch 'upstream/master' into make-cache-composable 2022-07-19 13:36:42 +02:00
kssenii
0aed5a602a Merge master 2022-07-18 16:16:28 +02:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef 
- to be replaced by std::string_view
- suggested in #39262
 2022-07-17 17:26:02 +00:00
kssenii
8ad26b3111 Merge master 2022-07-16 22:50:03 +02:00
Robert Schulze
deda29b46b
Pass const StringRef by value, not by reference 
See #39224
 2022-07-15 11:34:56 +00:00
Kruglov Pavel
a944a92d4c
Merge pull request #39224 from Avogar/string-view-by-value 
Pass const std::string_view by value, not by reference
 2022-07-15 13:05:56 +02:00
avogar
9291d33080 Pass const std::string_view & by value, not by reference 2022-07-14 16:11:57 +00:00
Sergei Trifonov
b2317ff710 renaming to be consistent 2022-07-14 17:33:22 +02:00
Sergei Trifonov
15ab3bc99f use context->getWriteSettings() 2022-07-13 19:48:57 +02:00
Sergei Trifonov
f854507729 merge master and resolve conflicts 2022-07-13 19:07:28 +02:00
kssenii
0d2d3ac2b4 Merge master 2022-07-13 13:26:15 +02:00
Sergei Trifonov
43779ec280 add max_remote_{read,write}_network_bandwidth_for_server settings 2022-07-11 14:59:39 +02:00
Frank Chen
da57a993e4 Fix CI 2022-07-09 13:43:10 +08:00
Frank Chen
9711559391 Remove extra code 2022-07-08 18:56:07 +08:00
Frank Chen
2e8c530bed Move thread trace context out of ThreadStatus 2022-07-07 17:41:10 +08:00
kssenii
e55169ebad Fix 2022-07-05 20:15:35 +02:00
lingpeng0314
cca94f01ae add function tryGetKeeperDispatcher for context 
- to make sure other case will get error if dispatcher not initialized
 2022-07-04 13:35:08 +08:00
lingpeng0314
03a5ce542d Remove Exception in getKeeperDispatcher 
- to handle the nil case by caller
 2022-07-01 11:27:40 +08:00
lingpeng0314
1d463718ca Change getKeeperDispatcher error type 2022-06-20 17:11:15 +08:00
kssenii
4813b90281 Merge master 2022-06-13 21:32:00 +02:00
kssenii
aaf9a46f07 Merge master 2022-06-11 15:56:05 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size 
Support to set max cache size for per query in local cache
 2022-06-10 16:17:18 +02:00
KinderRiven
6883ecda96 fix 2022-06-08 14:19:50 +08:00
KinderRiven
3b2a427002 fix 2022-06-07 11:50:56 +08:00
KinderRiven
13e74bef5e support skip_download_if_exceeds_query_cache 2022-06-06 02:36:23 +08:00
KinderRiven
3ffb0b3549 support_max_query_cache_size 2022-06-05 16:21:36 +08:00
Alexander Tokmakov
3d346c766a better code 2022-06-01 16:49:26 +02:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
kssenii
25b1ed526d Merge master 2022-05-27 16:50:45 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Nikolai Kochetov
fea2401f1f
Merge pull request #37532 from ClickHouse/add-separate-mutex-for-factories-info 
Use a separate mutex for query_factories_info in Context.
 2022-05-26 13:03:28 +02:00
Alexander Tokmakov
3d259fa0de
Update Context.cpp 2022-05-25 23:18:07 +03:00
kssenii
b89a4edc04 Merge master 2022-05-25 20:55:04 +02:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
Alexander Tokmakov
779e6ea0b9 make it better, fix on cluster queries 2022-05-25 20:17:49 +02:00
Nikolai Kochetov
7b681fa8ac Fixing build. 2022-05-25 17:15:23 +02:00
Nikolai Kochetov
6370c29049 Use a separate mutex for query_factories_info in Context. 2022-05-25 14:16:59 +00:00
Vitaly Baranov
497c70d786
Merge pull request #37269 from vitlibar/rework-access-control-notifications 
Rework AccessControl's notifications.
 2022-05-24 17:47:45 +02:00
Alexander Tokmakov
d0f998fb88 Merge branch 'master' into fix_trash 2022-05-23 21:25:56 +02:00
alesapin
06c3dd69c0 Move directories 2022-05-22 13:51:48 +02:00
kssenii
9aa20193d5 Merge master 2022-05-22 00:56:20 +02:00
kssenii
ffd8d9723a Continuation 2022-05-22 00:21:40 +02:00
Vitaly Baranov
58f4a86ec7 Rework notifications used in access management. 2022-05-21 10:15:39 +02:00
alesapin
654b27e307 Merge branch 'master' into i_object_storage 2022-05-20 11:56:13 +02:00
alesapin
92c15ec97c Get rid of IDiskRemote 2022-05-19 20:07:15 +02:00
Alexander Tokmakov
3bc39f9929 fixes 2022-05-18 21:57:20 +02:00
Alexander Tokmakov
9772924d06 Merge branch 'master' into fix_trash 2022-05-18 17:27:54 +02:00
Robert Schulze
43945cea1b
Fixing some warnings 2022-05-16 20:59:27 +02:00
kssenii
646cf4cc95 Better 2022-05-14 14:26:04 +02:00
kssenii
58faaf0ea3 Merge master 2022-05-13 18:51:07 +02:00
Azat Khuzhin
a0ba113e5e Mark Context::getInterserverIOHandler() as const 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:54:01 +03:00
Azat Khuzhin
a04b6ad314 Mark Context::getInterserverCredentials() as const 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:54:01 +03:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
Kruglov Pavel
af3db5a992
Merge pull request #36726 from PolyProgrammist/s3clustermacro 
Handle cluster macro in s3Cluster table function
 2022-05-11 18:00:57 +02:00
Vasily Nemkov
de9a06df5b
Merge branch 'master' into fix_session_log_crash 2022-05-09 17:00:24 +04:00
Vasily Nemkov
58953b5f28 Fixed issues with concurrent access to User object 
ContextAccess might be updated from another thread, nullifying UserPtr.
Can happen if user was dropped in concurrent query.

This could crash server under certain conditions.
 2022-05-09 15:56:50 +03:00
kssenii
fd1ab053f8 Merge master 2022-05-06 12:50:33 +02:00
Vitaly Baranov
dd51265218
Merge pull request #36864 from vitlibar/backup-improvements-4 
Backup improvements
 2022-05-05 15:37:51 +02:00
alesapin
aba78e5ed0
Merge pull request #36859 from ClickHouse/revert-36858-revert-35637-memory-overcommit-free 
Revert "Revert "Memory overcommit: continue query execution if memory is available""
 2022-05-04 23:43:00 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
kssenii
0266cdf125 Add entries to cache log when cache was not event attempted 2022-05-03 18:29:41 +02:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing 
Parallel parsing of hive text format
 2022-05-03 15:56:28 +02:00
Vitaly Baranov
5257ce31f8 Improved using ThreadPool for making backup or restoring, changed columns in system.backups. 2022-05-03 11:03:13 +02:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available"" 2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available" 2022-05-03 00:36:50 +02:00
Dmitry Novik
71b6f89166
Merge pull request #35637 from ClickHouse/memory-overcommit-free 
Memory overcommit: continue query execution if memory is available
 2022-05-02 19:00:18 +02:00