thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 04:22:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
147,908 Commits 9,192 Branches 1,782 Tags 2.2 GiB
2551f14141
Commit Graph

467 Commits

Author SHA1 Message Date
Robert Schulze
b70f3dbe41
Same variable name in LocalServer.cpp as in Server.cpp 2023-08-22 15:44:26 +00:00
Robert Schulze
f2fb7365e6
Add settings for 1. size_ratio (all caches) and 2. policy (index caches) 
1. is an expert-level setting, default is 0.5, applies only to SLRU.

Also, I noticed that we expose cache policy settings for the mark and
the uncompresed cache but not for the index mark and the index
uncompressed cache. Changed that as well, it simplifies the code a bit.
 2023-08-22 11:04:29 +00:00
Robert Schulze
96c8b250ce
Allow to configure cache sizes at runtime 2023-08-21 13:51:10 +00:00
Robert Schulze
8343ebd4ec
Clarify comment 2023-08-15 14:44:13 +00:00
Robert Schulze
ed90463888
Add comment 2023-08-14 15:42:32 +00:00
Robert Schulze
bb53d635df
More consistency: Run same mark cache size check in ch-local 
Server.cpp had a check that the configured mark cache size is not 0. Run
the same check in LocalServer.cpp.
 2023-08-14 15:42:32 +00:00
Robert Schulze
f126e54b0d
Respect max_cache_size for all caches 
So far, max_cache_size was only respected in Server.cpp, and in this
file only for the mark cache and the uncompressed cache. This commit
respects it also for the other cache types in Server.cpp, as well for
all caches in LocalServer.cpp.
 2023-08-14 15:42:32 +00:00
Robert Schulze
54e54aed74
Cosmetics: Introduce constants for default cache sizes and policies 2023-08-14 15:42:32 +00:00
Sergei Trifonov
e4ae2ece7c
Merge branch 'master' into async-loader-integration 2023-08-01 13:04:11 +02:00
Alexey Milovidov
a465dba311
Merge pull request #52628 from ClickHouse/load-suggestion-only-with-clickhouse-dialect 
Load suggestion only with `clickhouse` dialect
 2023-07-28 10:21:53 +03:00
Yakov Olkhovskiy
d35c87c1bd allow positional options for clickhouse-local and populate global udf settings 2023-07-27 05:06:16 +00:00
János Benjamin Antal
3a6aaa29c9 Do not load suggestions in case not ClickHouse dialects 2023-07-26 15:26:21 +00:00
Sergei Trifonov
8eaf3b6d94
Merge branch 'master' into async-loader-integration 2023-07-21 21:46:06 +02:00
serxa
02fc66db10 avoid using bg pool during non-async start 2023-07-19 17:37:53 +00:00
serxa
6bca79fdff add auto-resolution for all resolvable deadlock types 2023-07-19 16:19:28 +00:00
Sergei Trifonov
850ab23991
Merge branch 'master' into async-loader-integration 2023-07-17 19:14:54 +02:00
Vitaly Baranov
815a3857de Remove non-const function Context::getClientInfo(). 2023-07-17 15:02:07 +02:00
Alexey Milovidov
1a24bface0
Merge branch 'master' into clickhouse-local-default-settings 2023-07-06 03:30:11 +03:00
Azat Khuzhin
59f11863d7 Simplify settings overrides or clickhouse-local 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-06-24 23:42:31 +03:00
kssenii
0efaecab51 Revert "Merge pull request #51149 from ClickHouse/revert-48821-localfilefunction" 
This reverts commit a09e6bbb8e, reversing
changes made to ce38d64c5a.
 2023-06-21 00:29:46 +02:00
Alexander Tokmakov
caceb7c862
Revert "Added ability to implicitly use file/hdfs/s3 table functions in clickhouse-local" 2023-06-19 13:21:09 +03:00
Alexey Milovidov
df9abccfaa
Merge branch 'master' into localfilefunction 2023-06-17 00:45:54 +03:00
Sergei Trifonov
0cbcff958a
Merge branch 'master' into async-loader-integration 2023-06-07 18:55:04 +02:00
Nikita Mikhaylov
e87348010d
Rework loading and removing of data parts for MergeTree tables. (#49474) 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-06-06 14:42:56 +02:00
Sergei Trifonov
3ee782a0ec
Merge branch 'master' into async-loader-integration 2023-06-02 18:27:19 +02:00
serxa
866c9df70b fix clickhouse-local 2023-06-02 16:26:55 +00:00
Nikita Mikhaylov
73db383727
Limit the number of in-flight tasks for loading outdated parts (#50450) 
* Done

* Update programs/local/LocalServer.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>

* Bump

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-06-02 14:10:26 +02:00
Alexey Gerasimchuck
8574559e23 Disallow usage of --query and --queries-file 2023-05-28 22:25:40 +00:00
Sergei Trifonov
fe172f59db
Merge branch 'master' into async-loader-integration 2023-05-26 18:56:08 +02:00
alekseygolub
34fe8ba5cd
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-24 09:49:03 +03:00
Alexey Gerasimchuck
ab5e16a713 Changes after second review iteration 2023-05-23 00:27:17 +00:00
alekseygolub
70b9077bb2
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-21 17:49:29 +03:00
Alexey Gerasimchuck
4a6c7254e8 --multiquery <sql> -> -n -q <sql> syntax sugar 2023-05-17 03:43:35 +00:00
serxa
9bf5292415 refact 2023-05-12 19:49:47 +00:00
Sergei Trifonov
b479637e2c
Merge branch 'master' into async-loader-integration 2023-05-09 13:04:36 +02:00
Nikita Mikhaylov
954e3b724c
Speedup outdated parts loading (#49317) 2023-05-03 18:56:45 +02:00
alekseygolub
58cb6c7837 S3, HDFS only for explicit creation 2023-05-02 19:57:36 +00:00
serxa
6eda6e8585 implement async loading interface in database engines 2023-05-02 09:18:50 +00:00
alekseygolub
95522ad7a6 Added DatabaseHDFS 2023-05-01 21:46:17 +00:00
Sergei Trifonov
8dab49f328
Merge branch 'master' into async-loader-integration 2023-05-01 14:04:39 +02:00
serxa
abaf73268b integrate with Server and TablesLoader 2023-05-01 11:56:00 +00:00
Aleksei Golub
1846b76982 Added DatabaseS3 with test 2023-05-01 11:46:55 +03:00
Aleksei Golub
e20f92ce0f Fixed exceptions handling; Fixed style; 2023-04-27 21:26:36 +03:00
Aleksei Golub
267bbcab00 Added ability to implicitly use file table function in clickhouse-local 2023-04-16 15:48:55 +03:00
kssenii
599a02ca0d Fix 2023-04-13 18:53:16 +02:00
Yatsishin Ilya
5786aba5eb Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-22 14:08:22 +00:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors 
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
 2023-03-13 15:11:12 +00:00
Yatsishin Ilya
d4304d2028 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-02 16:16:18 +00:00
Alexey Milovidov
4134e56ef0 Increase the limit for opened files in clickhouse-local 2023-02-25 03:20:47 +01:00
Yatsishin Ilya
80857638c1 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-22 13:58:17 +00:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
Alexander Tokmakov
3457b68dcc fix 2023-02-17 17:57:49 +01:00
Alexander Tokmakov
9feb448a5e fix a race condition 2023-02-16 14:41:18 +01:00
Yatsishin Ilya
f4cfd8a2d9 Merge remote-tracking branch 'origin' into clickhouse-help 2023-02-03 20:08:23 +00:00
Yatsishin Ilya
ce553a2888 I like style check 2023-01-31 12:51:33 +00:00
Yatsishin Ilya
98edb9f06b Update help for clickhouse tools and add test 2023-01-31 12:19:37 +00:00
Anton Popov
839cd614fb fix memory leak in azure sdk 2023-01-31 01:01:10 +00:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
Robert Schulze
27f5aad49e
What happens if I remove 156 lines of code? 2023-01-03 18:51:16 +00:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
Azat Khuzhin
b7f92454bf Move LineReader/ReplxxLineReader out from base 
This will simplify linking dependencies, and anyway this classes are not
base in any sense.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-27 14:23:51 +01:00
kssenii
8308177361 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 12:14:46 +01:00
vdimir
d30d25dbbe
Temporary files evict fs cache 2022-12-22 10:22:49 +00:00
kssenii
30547d2dcd Replace old named collections code for url 2022-12-17 00:24:05 +01:00
kssenii
fae817863c Apply new code of named collections to s3 2022-12-13 22:19:09 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Azat Khuzhin
44f23c2568 Make disks checks only for clickhouse-server 
This will fix clickhouse-disks

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 16:28:35 +01:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Alexander Tokmakov
87550e33bb
Merge pull request #43092 from ClickHouse/make-clickhouse-local-progress-off-by-default 
Disable clickhouse local and client non-interactive progress by default.
 2022-11-14 14:14:29 +03:00
Yakov Olkhovskiy
60d116729c adjustments for progress option 2022-11-12 02:55:26 +00:00
kssenii
bed7ac9442 Support arbitrary structure collections 2022-11-10 17:05:24 +01:00
Nikolai Kochetov
3eb6c03b47 Disable clickhouse local and clickhouse client non-interactive progress by default. 2022-11-09 19:46:47 +00:00
Alexey Milovidov
641f8576c6
Merge pull request #42003 from ClickHouse/write-progress-to-tty 
Write progress directly to terminal
 2022-11-05 16:51:37 +01:00
Yakov Olkhovskiy
b30f2ac12d allow '--progress err' to preserve old behavior, adjust tests 2022-11-03 03:16:55 +00:00
Alexey Milovidov
47c32678ce
Merge pull request #42477 from AlfVII/enable-compiled-expressions-cache-in-local 
Enable CompiledExpressionCache in clickhouse-local
 2022-10-25 04:57:48 +02:00
Alexey Milovidov
cd43595ea9 Merge branch 'master' into write-progress-to-tty 2022-10-23 01:41:17 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Alfonso Martinez
168df7d9ba Removed trailing whitespace 2022-10-19 13:34:31 +02:00
Alfonso Martinez
0aa26db64c Added end of line co comply with Clangformat 2022-10-19 12:38:20 +02:00
Alfonso Martinez
c0be79f26c CompiledExpressionCache enabled 2022-10-19 12:27:00 +02:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf 
Refactor the implementation of user-defined functions
 2022-10-15 12:33:10 +02:00
Vitaly Baranov
2e59d671fc Split UserDefinedSQLObjectsLoader to interface and implementation. 2022-10-13 10:32:32 +02:00
Alexey Milovidov
9de9af54fe Merge branch 'master' of github.com:ClickHouse/ClickHouse into write-progress-to-tty 2022-10-01 23:25:07 +02:00
Alexey Milovidov
5d710e21f1 Render progress directly in tty 2022-10-01 23:19:36 +02:00
vdimir
0f1a7c252d
better TemporaryDataOnDisk 2022-09-29 09:51:46 +00:00
Antonio Andelic
0f92e5e576 Add option for loading only system tables from path using clickhouse-local 2022-09-19 14:18:42 +00:00
Alexey Milovidov
35cce03125 Remove dlopen 2022-09-17 03:02:34 +02:00
kssenii
0bb6f8792a Fix 2022-09-12 14:06:26 +02:00
kssenii
487bc0fba3 Fix heap use after free 2022-09-06 16:50:58 +02:00
kssenii
0f305f8a7a Fix test 2022-09-05 12:20:21 +02:00
kssenii
2dcf3fef94 Fix tests 2022-09-02 19:27:52 +02:00
Alexey Milovidov
458d21dd3c Add "readpassphrase" as a dependency for clickhouse-local 2022-08-21 12:12:11 +02:00
alexX512
2be72560f4 Add loading of mark cache policy 2022-08-07 19:59:39 +00:00
alexX512
1133e42367 Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf 2022-08-07 19:16:40 +00:00
Heena Bansal
d8db482b2e
Merge branch 'master' into Issue_39395 2022-08-01 12:22:16 -04:00
HeenaBansal2009
50c98789b8 Updated as per comments 2022-08-01 07:05:50 -07:00
HeenaBansal2009
800ed546be Updated as per comments 2022-08-01 07:03:36 -07:00
Alexey Milovidov
fa9c3dcc48
Update programs/local/LocalServer.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-07-31 03:02:27 +03:00
Alexey Milovidov
a30dbed6b8
Update programs/local/LocalServer.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-07-31 03:02:20 +03:00
Alexey Milovidov
9adfd259d0 Support in clickhouse-local 2022-07-30 01:50:45 +02:00
HeenaBansal2009
06e8b78efa Added FT testcase 2022-07-21 19:46:36 -07:00
HeenaBansal2009
d67e061915 Clickhouse-local fixes 2022-07-19 21:38:36 -07:00
Azat Khuzhin
3b29db6e9f LocalServer: remove superfluous ProgressIndicator.h 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-16 11:59:01 +03:00
Robert Schulze
c0d5705891
Merge pull request #37326 from ClickHouse/fix-marks-cache-size-cfg 
Provide default value for mark_cache_size
 2022-05-19 20:20:25 +02:00
Azat Khuzhin
29a8a00656 Add ability to pass QueryKind via clickhouse-client/local (useful for debugging) 
v2: fix LocalConnection::sendQuery() for Suggest (comes w/o client_info) [1]
    [1]: https://s3.amazonaws.com/clickhouse-test-reports/37290/7c85175963226ff78eec542efafcff4e650aa0f0/stateless_tests__ubsan__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 07:19:27 +03:00
Robert Schulze
8ca58cd5a5
Provide default value for mark_cache_size 
I played around with my local config.xml file. The minimal working example is this:
<?xml version="1.0"?>
<clickhouse>
    <mark_cache_size>5368709120</mark_cache_size>
    <listen_host>localhost</listen_host>
    <tcp_port>9000</tcp_port>
    <users_config>users.xml</users_config>
    <logger><console>true</console></logger>
</clickhouse>

Not specifying mark_cache_size made the server not start up:
  2022.05.18 12:15:06.549078 [ 8728320 ] {} <Error> Application: Not found: mark_cache_size

Looking at ClickHouse's ca. 100 server configuration options +
sub-options, it seems that mark_cache_size is NOT special enough to
require explicit configuration but instead that the behavior was
unintended because no default value was provided.
 2022-05-18 15:28:19 +02:00
Alexey Milovidov
1bdc51a496 Merge branch 'remind_for_external_option' of github.com:bigo-sg/ClickHouse into bigo-sg-remind_for_external_option 2022-05-07 12:36:57 +02:00
Alexey Milovidov
1ba42076e5
Merge pull request #36778 from ClickHouse/init-thread-pools-local-server 
Init thread pools for clickhouse-local
 2022-05-06 01:52:55 +03:00
Antonio Andelic
45a72d9aa9 Init thread pools for clickhouse-local 2022-04-29 07:41:10 +00:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Kseniia Sumarokova
d3763c4a62 Update ClientBase.h 2022-04-06 12:13:34 +02:00
kssenii
83488b2d13 Better 2022-04-05 14:48:26 +02:00
kssenii
1fb9440d47 Merge master 2022-03-30 17:37:37 +02:00
kssenii
95565b45d6 Fix 2022-03-29 13:33:17 +02:00
kssenii
98fe1ca1ec Fix 2022-03-29 12:05:27 +02:00
taiyang-li
1114c54899 merge master and fix conflict 2022-03-23 11:03:25 +08:00
Vitaly Baranov
39614e6e15
Merge pull request #35276 from vitlibar/fix-code-style-and-minor-corrections-after-allow-no-password 
Fix code style and other minor corrections after implementing allow_no_password.
 2022-03-16 18:55:05 +01:00
mergify[bot]
9cdd6ec7ab
Merge branch 'master' into test-mode-for-local 2022-03-15 08:28:40 +00:00
Vitaly Baranov
1eb2e8693e Fix code style and other minor corrections after implementing allow_no_password. 2022-03-14 20:55:34 +01:00
Vitaly Baranov
4af61fb9d3
Merge pull request #34738 from DevTeamBK/Issue-33953 
New setting in Server Configuration to on/off AuthType Plaintext_password and No_password
 2022-03-14 17:09:46 +01:00
kssenii
199188be08 Support test mode for clickhouse-local 2022-03-14 12:02:06 +01:00
HeenaBansal2009
3f031df225 Code refactoring 2022-03-10 22:22:51 -08:00
HeenaBansal2009
3ce9397246 Added Suggestions from Code review 2022-03-09 20:35:01 -08:00
taiyang-li
2283513fc8 merge master and solve conflict 2022-03-08 11:29:00 +08:00
taiyang-li
1b9987ceb1 fix mistake in check-style 2022-03-05 14:22:56 +08:00
taiyang-li
7f69507c95 finish dev 2022-03-01 17:22:12 +08:00
taiyang-li
b31440c77a Merge branch 'master' into add_metric_for_local 2022-03-01 12:29:36 +08:00
HeenaBansal2009
aa8494a808 Fix: System Reload Config Failure 2022-02-28 10:51:49 -08:00
HeenaBansal2009
1b263f0c15 Added FT testcase 2022-02-18 12:58:46 -08:00
Kruglov Pavel
adf58ea1ce
Merge pull request #34490 from Avogar/fix-client 
Refactor client fault tolerant connection
 2022-02-16 12:39:46 +03:00
taiyang-li
24bd47e556 finish dev 2022-02-15 20:11:13 +08:00
Anton Popov
2399f3e3bb
Merge pull request #34495 from Avogar/improve-local 
Improve schema inference in clickhouse-local
 2022-02-14 17:27:24 +03:00
alexey-milovidov
0250b28916
Merge pull request #34550 from kssenii/fix-test 
Tiny fixes for client/local suggestions
 2022-02-14 16:52:56 +03:00
Kruglov Pavel
2bf15d6b5b
Merge pull request #34352 from bigo-sg/clickhouse_local_format 
Default input and output formats that can be overriden by --input-format and --output-format.
 2022-02-14 14:09:59 +03:00
kssenii
39049251b7 Fix disable_suggestion argument for clickhouse-local 2022-02-12 20:26:01 +01:00
avogar
7d92d9d060 Fix tests 2022-02-10 18:37:27 +03:00
avogar
a69905d378 Fix 2022-02-10 18:23:52 +03:00
avogar
9bbbbdcdd6 Improve schema inference in clickhouse-local 2022-02-10 18:23:52 +03:00
avogar
75c56163a0 Refactor client fault tolerant connection 2022-02-10 12:43:08 +03:00
taiyang-li
0c47dd4ec2 format override input-format 2022-02-06 12:17:03 +08:00
avogar
8858b6bf86 Small improvement in schema inference from stdin in local 2022-01-28 19:17:35 +03:00
Kruglov Pavel
aa49a826b5
Try to fix tests 2022-01-25 14:11:02 +03:00
avogar
a82cdd4fb5 Fix tests 2022-01-25 10:55:09 +03:00
avogar
fed54e0204 Fix 2022-01-25 10:25:37 +03:00
avogar
a20fdc210a Add format autodetect in clickhouse-local 2022-01-20 15:55:28 +03:00
Azat Khuzhin
aee034a597 Use explicit template instantiation for SystemLog 
- Move some code into module part to avoid dependency from IStorage in SystemLog
- Remove extra headers from SystemLog.h
- Rewrite some code that was relying on headers that was included by SystemLog.h

v2: rebase
v3: squash move into module part with explicit template instantiation
    (to make each commit self compilable after rebase)
 2022-01-10 22:01:41 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
kssenii
38ca791465 Fix 2021-12-25 22:00:28 +03:00