thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 16:42:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
97,105 Commits 9,168 Branches 1,781 Tags 2.1 GiB
21ab379026
Commit Graph

30826 Commits

Author SHA1 Message Date
Antonio Andelic
97fd3d9123
Merge branch 'master' into fix-install-snapshot 2022-08-31 08:42:00 +02:00
Duc Canh Le
aebd1cb7de
Merge branch 'master' into ch_canh_intersect_distinct 2022-08-31 12:08:20 +08:00
Fangyuan Deng
5c2f35c302
Merge branch 'master' into UniqSketch 2022-08-31 10:23:46 +08:00
Mallik Hassan
87a28c25f5 Update src/Loggers/Loggers.cpp 
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
 2022-08-30 23:20:19 -03:00
root
f02f7ce99c addressed minor change requests 2022-08-30 23:20:19 -03:00
root
834ab5cd2b removed redundant initializer for base class 'OwnPatternFormatter' 2022-08-30 23:20:19 -03:00
root
790a437eb2 Addressed review requests: removed this pointer in functions and config variable in class 2022-08-30 23:20:19 -03:00
root
bb25abd0ae to resolve build error(no member named 'Util' in namespace 'Poco'), added <Poco/Util/AbstractConfiguration.h> in OwnJSONPatternFormatter.h 2022-08-30 23:20:19 -03:00
root
b8501bea06 style check correction 2022-08-30 23:20:19 -03:00
root
d5db88fa27 Structured-logging-custom-keys PR init 2022-08-30 23:20:19 -03:00
Anton Popov
3504781529
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-30 23:32:43 +02:00
Sema Checherinda
412973daee
Update src/Interpreters/AsynchronousMetrics.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-08-30 21:57:39 +02:00
Sema Checherinda
6b23abe2b6
Update src/Interpreters/AsynchronousMetrics.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-08-30 21:57:33 +02:00
Sema Checherinda
9bb4f556c3 work on remarks on the review 2022-08-30 20:18:35 +02:00
kssenii
23db13ca69 Merge master 2022-08-30 19:39:41 +02:00
kssenii
19c4442933 Fix 2022-08-30 19:38:57 +02:00
Robert Schulze
7a4978e600
docs: mention where else shortcuts are used 2022-08-30 17:00:25 +00:00
Robert Schulze
413a5759e1
style: eliminate some newlines 
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
 2022-08-30 17:00:20 +00:00
Robert Schulze
9baf0d745a
feat: make last command available in clickhouse-client interactive mode 2022-08-30 16:59:14 +00:00
Vitaly Baranov
77d741dc25 Add comments. 2022-08-30 18:58:13 +02:00
Vitaly Baranov
86872b2307 Fix locking while writing backup in multiple threads. 2022-08-30 18:10:54 +02:00
Alexander Tokmakov
6a50c20734
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-30 15:43:30 +03:00
Dmitry Novik
0a8378d9cd
Merge branch 'master' into betterorderbyoptimization 2022-08-30 14:23:22 +02:00
Alexander Tokmakov
6fdfb964d0
Revert "Add Annoy index" 2022-08-30 15:10:10 +03:00
vdimir
0f6f3c73b0
Minor fix 2022-08-30 11:57:28 +00:00
Vladimir C
7266c007b7
Merge pull request #39131 from amosbird/cte-global-with-fix1 
Fix unused unknown columns introduced by WITH statement
 2022-08-30 13:52:19 +02:00
Anton Popov
ee691961d9 fix build 2022-08-30 11:43:27 +00:00
Alexander Tokmakov
022f440ad0
Merge pull request #40769 from arthurpassos/caresptrresolver-hosts-patch 
Fix CaresPTRResolver not reading hosts file
 2022-08-30 14:35:10 +03:00
Kseniia Sumarokova
54f6f5bb2c
Merge branch 'master' into fix-write-through-cache-logical-error 2022-08-30 13:30:13 +02:00
Kruglov Pavel
624af5bffd
Update src/Storages/MergeTree/KeyCondition.cpp 2022-08-30 13:18:05 +02:00
Kruglov Pavel
be81d21fdf
Merge pull request #40597 from helifu/master 
Fix a crash while the grpc port is conflict
 2022-08-30 13:17:25 +02:00
Kseniia Sumarokova
5eb188311a
Merge pull request #40586 from kssenii/fix-race 
Fix possible race in fs cache
 2022-08-30 12:58:31 +02:00
Alexander Tokmakov
0ec7f068cc
Merge pull request #40677 from ClickHouse/query-insert-oom 
Avoid deadlock in case of new query and OOM
 2022-08-30 13:54:34 +03:00
Duc Canh Le
626b3e4147
Merge branch 'master' into ch_canh_intersect_distinct 2022-08-30 18:14:07 +08:00
Duc Canh Le
8590cc46c4 implement intersect + except distinct 2022-08-30 18:09:01 +08:00
Kseniia Sumarokova
487100a4fd
Merge pull request #40764 from kssenii/fix-logical-error-in-cache 
Remove incorrect check in fs cache
 2022-08-30 11:59:56 +02:00
Kseniia Sumarokova
c88db2ef97
Merge pull request #40751 from kssenii/fix-mysql-timeouts 
Fix issue with mysql db / table function timeouts
 2022-08-30 11:59:01 +02:00
Antonio Andelic
21cdb9a500 Don't remove from cache if it wasn't added during rollback 2022-08-30 08:14:53 +00:00
Antonio Andelic
2416ddfae7 Add extra message about startup preprocessing 2022-08-30 08:14:31 +00:00
Robert Schulze
c185353bf0
Minor: remove unnecessary ContextAccess::make() method 2022-08-30 08:06:42 +00:00
Robert Schulze
cc4225109f
Merge pull request #37215 from Vector-Similarity-Search-for-ClickHouse/annoy-2 
Test failures are unrelated, merging.
 2022-08-30 09:25:57 +02:00
Robert Schulze
566daad4a1
Merge pull request #40723 from k-morozov/ref/remove-unnecessary-allocation 
refactoring: remove unnecessary allocation
 2022-08-30 09:22:21 +02:00
Frank Chen
922a0dee74 Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-30 15:14:12 +08:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Frank Chen
83cbdef3c6 Fix context re-initialization for ThreadPool 2022-08-30 12:26:23 +08:00
Cory Levy
cd371110ad Fix timezone offset formatting 2022-08-29 21:01:27 -04:00
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
Robert Schulze
4ed1e1a5be
perf: don't copy data around unnecessarily 2022-08-29 20:27:06 +00:00
Robert Schulze
35a37c91f8
chore: incorporate review feedback 2022-08-29 20:27:06 +00:00
robot-clickhouse
64fa077148
style: fix style 2022-08-29 20:27:06 +00:00
Robert Schulze
4d511332c4
chore: delete obsolete modelEvaluate() function 
- superseded by catboostEvaluate() which no longer uses the internal
  repository for external models

- also removed was statement SYSTEM RELOAD MODELS and the monitoring view
  SYSTEM.SYSTEMMODELS
 2022-08-29 20:27:06 +00:00
Robert Schulze
6b2b3c1eb3
feat: implement catboost in library-bridge 
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
 2022-08-29 20:26:45 +00:00
Dmitry Novik
e25ed9547e
Update src/Interpreters/ProcessList.h 2022-08-29 20:26:37 +02:00
Dmitry Novik
865ee5d0d6 Refactor code 2022-08-29 20:24:35 +02:00
Arthur Passos
961365c7a4 Fix CaresPTRResolver not reading hosts file 2022-08-29 15:11:39 -03:00
Dmitry Novik
1169315580 Add OvercommitTracker blocking 2022-08-29 19:44:05 +02:00
Maksim Kita
88141cae98
Merge pull request #40732 from azat/thread-status-fix-leak 
Fix memory leak while pushing to MVs w/o query context (from Kafka/...)
 2022-08-29 19:36:25 +02:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
kssenii
db2bc31e17 Remove incorrect assertion 2022-08-29 19:32:47 +02:00
Konstantin Morozov
d185b7a332 refactoring: public ctors 2022-08-29 20:19:20 +03:00
FArthur-cmd
862b53b06f Merge branch 'annoy-2' of https://github.com/Vector-Similarity-Search-for-ClickHouse/ClickHouse into annoy-2 2022-08-29 16:43:39 +00:00
FArthur-cmd
3305af8db2 fix case when query is already matched 2022-08-29 16:43:24 +00:00
kssenii
545c6c8be4 Fix 2022-08-29 17:50:27 +02:00
Dmitry Novik
cfe509c3de Block overcommit tracker in ProcessList near allocations 2022-08-29 17:49:01 +02:00
Filatenkov Artur
d73f661732
Merge branch 'master' into annoy-2 2022-08-29 17:33:13 +03:00
Anton Popov
506bee1e2f avoid logical errors at some common operation with ColumnObject 2022-08-29 14:31:09 +00:00
kssenii
b1dab84d97 Review fixes 2022-08-29 16:23:14 +02:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
kssenii
877ade9a50 Merge remote-tracking branch 'upstream/master' into fix-race 2022-08-29 16:05:27 +02:00
Antonio Andelic
57471d11db Fix uncommitted auth 2022-08-29 12:55:30 +00:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330 
Add config option warning_supress_regexp
 2022-08-29 14:02:00 +02:00
Alexander Tokmakov
8bdb589c2b Merge branch 'master' into zookeeper_client_fault_injection 2022-08-29 13:34:57 +02:00
Antonio Andelic
54ce92a5c4 Merge branch 'master' into fix-install-snapshot 2022-08-29 11:27:34 +00:00
vdimir
24f62e8486
Throw an error in CreatingSetsOnTheFlyTransform in case of input for finished 2022-08-29 11:27:08 +00:00
vdimir
b0e2616aa9
Style fixes in CreateSetAndFilterOnTheFlyTransform and related 2022-08-29 11:26:21 +00:00
Alexander Tokmakov
69387acffa
Revert "Support for DWARF-5 in in house DWARF parser" 2022-08-29 14:25:53 +03:00
Alexander Tokmakov
8d90d30d37
Merge pull request #40589 from ClickHouse/remove_wrong_code_from_mutations 
Remove wrong code for skipping mutations in MergeTree
 2022-08-29 14:18:59 +03:00
avogar
85fa372ad7 Fix possible error Attempt to read after eof in CSV schema inference 2022-08-29 11:18:53 +00:00
Anton Popov
2a3e012931
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-29 13:17:26 +02:00
Alexander Tokmakov
eda0582ec0
Merge pull request #40641 from ClickHouse/fix_startup_of_dropped_replica 
Do not try to strartup dropped replica
 2022-08-29 14:15:15 +03:00
vdimir
3e06cf3a9d
Upd max_rows_in_set_to_optimize_join description 2022-08-29 10:10:47 +00:00
vdimir
7915b6948f
Fix build after rebase 2022-08-29 09:49:16 +00:00
vdimir
afb6b7d9cf
Test plan and pipeline for filtering step for join 2022-08-29 09:49:15 +00:00
vdimir
afeff512b5
Aux port for ReadHeadBalancedProcessor is empty Block 2022-08-29 09:49:14 +00:00
vdimir
95f87dc34e
fix sanitizer assert in CreateSetAndFilterOnTheFlyStep 2022-08-29 09:49:12 +00:00
vdimir
c67ab33d90
small fix CreateSetAndFilterOnTheFlyStep 2022-08-29 09:49:11 +00:00
vdimir
51e02d09f6
set preserves_sorting = true for CreateSetAndFilterOnTheFlyStep 2022-08-29 09:49:10 +00:00
vdimir
714c53ab24
fix typos 2022-08-29 09:49:09 +00:00
vdimir
8e1632f824
Create sets for joins: better code 2022-08-29 09:49:08 +00:00
vdimir
7228091ff1
rename CreateSetAndFilterOnTheFlyTransform 2022-08-29 09:49:07 +00:00
vdimir
67a9acc8db
rename CreatingSetOnTheFlyStep -> CreateSetAndFilterOnTheFlyStep 2022-08-29 09:49:07 +00:00
vdimir
d82a75ae75
cleanup PingPongProcessor 2022-08-29 09:49:06 +00:00
vdimir
e472e13c70
move PingPongProcessor/ReadHeadBalancedProceesor into separate file 2022-08-29 09:49:05 +00:00
vdimir
6ae15db3ad
check join kind for before applying max_rows_in_set_to_optimize_join 2022-08-29 09:49:04 +00:00
vdimir
51a51694d6
Create sets for joins: better code 2022-08-29 09:49:01 +00:00
vdimir
c778bba13f
Create sets for joins: wip 2022-08-29 09:47:00 +00:00
vdimir
31a167848d
Fix set finish condition in CreatingSetsOnTheFlyTransform 2022-08-29 09:46:59 +00:00
vdimir
71708d595f
Create sets for joins: wip 2022-08-29 09:46:59 +00:00
vdimir
8f06430ebd
Create sets for joins: upd 2022-08-29 09:46:58 +00:00
vdimir
3292566603
Format bytes in CreatingSetsOnTheFlyTransform logs 2022-08-29 09:46:57 +00:00
vdimir
031aaf3a45
Add Creating/FilterBySetsOnTheFlyStep for full sorting join 2022-08-29 09:46:57 +00:00
vdimir
c5bc7b0a0c
Resize pipeline after full sort join 2022-08-29 09:46:56 +00:00
Azat Khuzhin
f9812d9917 Fix memory leak while pushing to MVs w/o query context (from Kafka/...) 
While pushign to MVs, there is a low-level code that create
ThreadGroupStatus/ThreadStatus, it is required to gather some metrics
for system.query_views_log.

But, one should not use ThreadGroupStatus of the MainThreadStatus, since
this structure can hold some state, that may not be cleaned, plus this
may be racy, instead it is better to create new ThreadGroupStatus and
attach it instead.

Also this place misses detachQuery(), and because of this it leaks
ThreadGroupStatus::finished_threads_counters_memory. But it is only the
problem pushing to MVs is done w/o query context (i.e. from Kafka/...),
since when it has query context detachQuery() will be called eventually.

Before this patch series, when I've tried the reproducer with
500 MVs attached to Kafka engine (that @den-crane suggested), jemalloc
report looks like this:

    $ ../jeprof --text ~/ch/tmp/upstream/clickhouse-binary --base jeprof.44384.0.i0.heap jeprof.44384.167.i167.heap
    Using local file /home/azat/ch/tmp/upstream/clickhouse-binary.
    Using local file jeprof.44384.167.i167.heap.
    Total: 915.6 MB
       910.7  99.5%  99.5%    910.7  99.5% Snapshot (inline)
         9.5   1.0% 100.5%      9.5   1.0% std::__1::__libcpp_operator_new (inline)
         0.5   0.1% 100.6%      0.5   0.1% DB::TasksStatsCounters::create

And with focus to this place:

    $ ../jeprof --focus Snapshot --text ~/ch/tmp/upstream/clickhouse-binary --base jeprof.44384.0.i0.heap jeprof.44384.167.i167.heap
    Using local file /home/azat/ch/tmp/upstream/clickhouse-binary.
    Using local file jeprof.44384.167.i167.heap.
    Total: 915.6 MB
       910.7 100.0% 100.0%    910.7 100.0% Snapshot (inline)
         0.0   0.0% 100.0%    910.7 100.0% DB::QueryPipeline::reset
         0.0   0.0% 100.0%    910.7 100.0% DB::StorageKafka::streamToViews
         0.0   0.0% 100.0%    910.7 100.0% DB::StorageKafka::threadFunc
         0.0   0.0% 100.0%    910.7 100.0% ProfileEvents::Counters::getPartiallyAtomicSnapshot
         0.0   0.0% 100.0%    910.7 100.0% ~ThreadStatus
         0.0   0.0% 100.0%    910.7 100.0% ~ViewRuntimeData
         0.0   0.0% 100.0%    910.7 100.0% ~ViewRuntimeStats (inline)

Actually this report does not looks great (you understand it because I
stripped it), because --text does not that smart, but if you will use
--pdf for the report you will see the stacktrace (will attach pdf to the
pull request).

But after this patch series the process RSS does not goes beyond
~700MiB.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:36:33 +02:00
Azat Khuzhin
6da5707f8f Fix possible missing detachQuery() in case of exception in readers 
This can create leaks, since detachQuery() responsible for cleaning,
i.e. ThreadGroupStatus::finished_threads_counters_memory

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:30:17 +02:00
Azat Khuzhin
b16891da8d Avoid using of ThreadGroupStatus of the MainThreadStatus 
One should not use MainThreadStatus, since ThreadGroupStatus can hold
some states, and it is better not to play with this, since this may
create leaks.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:30:17 +02:00
Azat Khuzhin
9fff08eac7 WriteBufferFromS3: remove unused ThreadGroupStatus 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:30:17 +02:00
alesapin
133ca01447 Merge branch 'master' into stress_s3 2022-08-29 11:25:28 +02:00
alesapin
4ed375ca5b
Merge pull request #40720 from ClickHouse/fix_benign_race 
Fix benign race in database replicated worker
 2022-08-29 11:13:07 +02:00
Azat Khuzhin
269453a646 Avoid leaking of ThreadGroupStatus::finished_threads_counters_memory 
Cleanup them in ThreadStatus::detachQuery(), anyway they cannot be
received after by the client.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:08:50 +02:00
alesapin
7ce0afc0df
Merge pull request #40670 from Avogar/kafka 
Add setting to disable limit on kafka_num_consumers
 2022-08-29 10:53:35 +02:00
Amos Bird
15a69bce84
Use index when row_policy_filter is always false 2022-08-29 16:44:32 +08:00
Antonio Andelic
b4af21b6fb Small fixes 2022-08-29 07:06:51 +00:00
Amos Bird
abf51bccfd
Disable projection when grouping set is used. 2022-08-29 14:35:48 +08:00
Konstantin Morozov
6636bdec2a refactoring: fix format 2022-08-29 07:44:40 +03:00
Cory Levy
d0ace9a8fb Add the %z format descriptor to formatDateTime 2022-08-28 22:28:11 -04:00
Alexey Milovidov
71f6c52c2d
Merge pull request #40727 from amosbird/column-transformer-fix1 
Correct format of APPLY transformer param
 2022-08-29 03:48:04 +03:00
Alexey Milovidov
307e8f2da9
Merge pull request #40729 from ClickHouse/remove-useless-method 
Remove useless method
 2022-08-29 03:46:34 +03:00
Alexey Milovidov
ae97e880d7
Merge pull request #40710 from azat/DWARF-5 
Support for DWARF-5 in in house DWARF parser
 2022-08-29 00:13:33 +03:00
Alexey Milovidov
27782ceef8 Remove useless method 2022-08-28 22:33:42 +02:00
Konstantin Morozov
38a85ade27 refactoring: small up 2022-08-28 22:33:12 +03:00
Konstantin Morozov
b4de735a90 refactoring: fix space 2022-08-28 20:55:08 +03:00
Konstantin Morozov
08e9e799d0 refactoring: fix 2022-08-28 20:53:00 +03:00
Konstantin Morozov
75bd61fad5 Merge remote-tracking branch 'origin/ref/remove-unnecessary-allocation' into ref/remove-unnecessary-allocation 
# Conflicts:
#	src/Access/ContextAccess.cpp
#	src/Access/ContextAccess.h
 2022-08-28 20:50:29 +03:00
Konstantin Morozov
7cd2821aed refactoring: template make helper 2022-08-28 20:42:55 +03:00
Amos Bird
d1fbe51b81
Correct format of APPLY transformer param 2022-08-29 01:21:12 +08:00
Konstantin Morozov
4050ab819e refactoring: remove unnecessary allocation 2022-08-28 17:23:17 +03:00
Konstantin Morozov
d4d80dd3c1 refactoring: remove unnecessary allocation 2022-08-28 17:09:13 +03:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented 
All new table functions must be documented
 2022-08-28 17:05:20 +03:00
Alexey Milovidov
00074a5e14
Merge pull request #40711 from ClickHouse/all_new_functions_must_be_documented 
All new functions must be documented
 2022-08-28 17:05:05 +03:00
Azat Khuzhin
e03870bc8b Common/Dwarf: fix clang-tidy 2022-08-28 15:51:46 +02:00
alesapin
4f9faab924 Fix benign race in database replicated worker 2022-08-28 15:07:29 +02:00
Kseniia Sumarokova
a0bc5b6ea4
Merge pull request #40705 from azat/stress/max_read_buffer_size 
Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
 2022-08-28 12:41:16 +02:00
Robert Schulze
df934d8762
Merge pull request #40217 from zvonand/zvonand-minmax 
Fix conversion Date32 / DateTime64 / Date to narrow types
 2022-08-28 09:42:39 +02:00
Alexey Milovidov
d3a7945b1b
Merge pull request #40646 from ClickHouse/force-documentation-3 
Every profile event must have documentation
 2022-08-28 04:10:11 +03:00
Alexey Milovidov
5b57c91ad6 Rename table function MeiliSearch to meilisearch 2022-08-28 02:08:51 +02:00
Alexey Milovidov
1ff535a128 One step back 2022-08-28 02:00:09 +02:00
Alexey Milovidov
8a4bacc4b7 Update example 2022-08-28 00:30:14 +02:00
Alexey Milovidov
6ac9eabbd4 Register more aliases correctly 2022-08-28 00:17:13 +02:00
Alexey Milovidov
a6d99e795e Register more aliases correctly 2022-08-28 00:14:38 +02:00
Alexey Milovidov
e266688576 Remove something stupid 2022-08-27 23:58:34 +02:00
Alexey Milovidov
421d28542b Register all function aliases correctly 2022-08-27 23:53:51 +02:00
Alexey Milovidov
90cf0bb6e4 Fix style 2022-08-27 23:40:24 +02:00
Alexey Milovidov
365a600fdb Merge branch 'force-documentation-3' of github.com:ClickHouse/ClickHouse into force-documentation-3 2022-08-27 22:28:54 +02:00
Alexey Milovidov
6b2e227c8b Fix integration test 2022-08-27 22:28:38 +02:00
Azat Khuzhin
ee5696bb32 Add support for DWARF-5 
I have to do this, since I have libraries compiled with DWARF5 (i.e.
glibc).

ClickHouse changes:
- use camel_case
- add NOLINT
- avoid using folly:: (use std:: instead)
- avoid using boost:: (use std:: instead)

Refs: 490b287ca3
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 22:28:15 +02:00
Alexey Milovidov
6139cc8f7a Add Documentation to FunctionFactory 2022-08-27 22:06:57 +02:00
Alexey Milovidov
f86242c17e
Merge pull request #40699 from ClickHouse/remove-unbundled 
Remove the remaining parts of "unbundled" build
 2022-08-27 21:54:42 +03:00
Alexey Milovidov
e7054debc1
Merge pull request #40649 from ClickHouse/embedded-documentation 
A prototype for embedded documentation
 2022-08-27 21:28:11 +03:00
Alexey Milovidov
a6387ac0f7
Merge branch 'master' into force-documentation-3 2022-08-27 19:31:44 +03:00
Alexey Milovidov
6e564b18bf
Merge pull request #40600 from FrankChen021/check_url_arg 
Validate the CompressionMethod parameter of URL table engine
 2022-08-27 19:29:55 +03:00
Alexey Milovidov
7569cbb381
Merge pull request #40643 from ClickHouse/fix-recursive-buffer 
Fix stack overflow in recursive Buffer tables.
 2022-08-27 19:27:24 +03:00
Alexey Milovidov
779fac512d Revert wrong modification 2022-08-27 15:55:28 +02:00
Alexey Milovidov
2a931c69b7
Merge pull request #40682 from ClickHouse/remove-accurate-underscore-cast 
Remove `accurate_Cast` and `accurate_CastOrNull`, which are unused
 2022-08-27 16:53:30 +03:00
Alexey Milovidov
e04f69717b
Merge pull request #40673 from ClickHouse/vdimir/tmp-file-metrics-doc 
Add doc to ExternalSort/Agg/Join ProfileEvents
 2022-08-27 16:52:40 +03:00
Alexey Milovidov
36ca126a26
Merge pull request #40679 from ClickHouse/fix-typo 
Fix typo
 2022-08-27 16:52:21 +03:00
Azat Khuzhin
c801469e9e Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks 
The problem is that the buffer size validated after marks reading in
MergeTreeReaderStream::init(), since it requires to read marks first.

And later it is passed to AsynchronousReadBufferFromFileDescriptor,
which throws LOGICAL_ERROR because buffer_size < alignment.

Fix this my simply disallow such values for max_read_buffer_size (I
thougt about modifying createReadBufferFromFileBase(), but it is not
used for remote reads -- remote_fs_buffer_size).

Fixes: #40669
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 14:12:07 +02:00
Alexander Gololobov
6fc8f1e9e0
Merge pull request #40645 from ClickHouse/force-documentation-2 
Every current metric must have documentation
 2022-08-27 10:44:47 +02:00
Duc Canh Le
1691db9ce6
Merge branch 'master' into ch_canh_fix_json 2022-08-27 09:23:04 +08:00
Alexey Milovidov
91ed3ebe9a Remove the remaining parts of "unbundled" build 2022-08-27 02:18:47 +02:00
Azat Khuzhin
3c478da003 Move committing InMemory parts to WAL out of NOEXCEPT_SCOPE() 
Since this commit can definitelly throw (i.e. due to ENOSPC).

Note, that it should be safe, since rollback() will call dropPart() for
those parts.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-26 23:37:59 +02:00
Alexander Gololobov
89be7a5a25
Update ProfileEvents.cpp 2022-08-26 22:55:42 +02:00
Kseniia Sumarokova
11b82ffc5e
Merge branch 'master' into fix-race 2022-08-26 22:55:14 +02:00
Kseniia Sumarokova
0cf76dab77
Merge pull request #40608 from ClickHouse/kssenii-patch-4 
Fix stress test after #40420
 2022-08-26 22:54:37 +02:00
Alexey Milovidov
53343e0fc2 Remove accurate_Cast and accurate_CastOrNull, which are unused 2022-08-26 21:27:30 +02:00
Alexey Milovidov
c9858099ea
Update ProfileEvents.cpp 2022-08-26 22:10:10 +03:00
Alexander Gololobov
4fef408104
Merge branch 'master' into force-documentation-3 2022-08-26 20:29:06 +02:00
Alexey Milovidov
6940bdab3b
Merge branch 'master' into fix-recursive-buffer 2022-08-26 21:25:11 +03:00
Alexey Milovidov
02bfebf1d3 Fix typo 2022-08-26 20:08:33 +02:00
Alexey Milovidov
1ad44acce8
Merge pull request #40644 from ClickHouse/force-documentation 
Every setting must have documentation
 2022-08-26 21:06:57 +03:00
Dmitry Novik
5b7fe91675 Avoid deadlock in case of new query and OOM 2022-08-26 17:02:31 +02:00
Antonio Andelic
bff2b232ef Cache deltas for path 2022-08-26 14:03:38 +00:00
FArthur-cmd
f3d0083e57 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy-2 2022-08-26 13:53:53 +00:00
vdimir
c5a9b64080
Add doc to ExternalSort/Agg/Join ProfileEvents 2022-08-26 13:41:53 +00:00
Alexey Milovidov
46594e47bc Add comment 2022-08-26 15:28:18 +02:00
Alexey Milovidov
c2dffebe26
Merge branch 'master' into force-documentation-2 2022-08-26 16:10:58 +03:00
avogar
e31bba7731 Add docs 2022-08-26 12:52:15 +00:00
avogar
b8061532ac Add setting to disable limit on kafka_num_consumers 2022-08-26 12:49:39 +00:00
Kseniia Sumarokova
fcb53aacdb
Merge pull request #40642 from barumrho/barumrho/sqlite-int8-conversion 
Convert SQLite int8 column to ClickHouse int64
 2022-08-26 14:33:20 +02:00
Kseniia Sumarokova
4e5385298a
Update src/TableFunctions/TableFunctionURL.cpp 2022-08-26 14:22:29 +02:00
kssenii
470e18677f Ger rid of cache lock 2022-08-26 14:13:36 +02:00
Antonio Andelic
26d68dce23 Fix style 2022-08-26 10:55:07 +00:00
Antonio Andelic
dae65178b7 Improve preprocessing performance 2022-08-26 10:46:14 +00:00
Kruglov Pavel
e2e1ca26e9
Merge pull request #40628 from Avogar/fix-to-fixed-string 
Fix short circuit execution of toFixedString function
 2022-08-26 12:26:16 +02:00
Kruglov Pavel
58538d3f90
Merge pull request #40636 from Avogar/fix-url-format-detection 
Better format detection for url table function/engine
 2022-08-26 12:25:55 +02:00
Vladimir C
e067629e0d
Merge pull request #40239 from vdimir/vdimir/tmp-file-metrics 
More metrics for on-disk temporary data
 2022-08-26 11:28:01 +02:00
Robert Schulze
1c8c83ccf3
Merge pull request #40620 from zvonand/zvonand-b58 
Base58 fix handling leading 0 / '1'
 2022-08-26 10:18:26 +02:00
Alexander Gololobov
c7a792d929
small grammar fix 2022-08-26 10:03:43 +02:00
Alexander Gololobov
856f83caa1
Removed unnecessary commas 2022-08-26 10:00:20 +02:00
Lloyd-Pottiger
0d33327eda
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-26 15:36:21 +08:00
Frank Chen
cf1081eada Exception safe from destructor 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-26 11:18:16 +08:00
Frank Chen
2e874b384d
Update src/Storages/StorageURL.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-08-26 10:34:46 +08:00
Alexey Milovidov
a029116fe3 Add a comment 2022-08-26 04:01:34 +02:00
Alexey Milovidov
8afbfebd25 A prototype for embedded documentation 2022-08-26 03:41:52 +02:00
何李夫
2ec087192e
Merge branch 'master' into master 2022-08-26 09:09:29 +08:00
Alexey Milovidov
5b5a16320b Suggestions from @azat 2022-08-26 03:00:56 +02:00
Alexey Milovidov
91bd6f1d46 Write the documentation for all ProfileEvents. Enforce that documentation exists. 2022-08-26 01:35:03 +02:00
Alexey Milovidov
6659438d87 Every current metric must have documentation 2022-08-26 00:33:25 +02:00
Alexey Milovidov
a1acdf18ec Write the documentation 2022-08-26 00:14:46 +02:00
Kseniia Sumarokova
a8abf58f13
Update FileSegment.cpp 2022-08-25 23:15:24 +02:00
Barum Rho
a61827fb78 Convert SQLite int8 column to ClickHouse int64 
SQLite int8 column uses 64 bit integers.
 2022-08-25 16:15:50 -04:00
Alexander Tokmakov
b8e3a582f7 do not try to strartup dropped replica 2022-08-25 22:14:39 +02:00
Alexey Milovidov
ebe7d2d25e Fix stack overflow on recursive Buffer tables and system.tables 2022-08-25 21:26:32 +02:00
Azat Khuzhin
09aaa2b5dd Do not obtain storage snapshot for each INSERT block (improves performance) 
It depends on how much parts do you have, but for some workload with
InMemory only parts w/o merges, I got 5% increase.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-25 21:02:27 +02:00
Alexander Gololobov
6a69e08799
Merge pull request #40559 from ClickHouse/lwd_vertical_merge_fix 
Fix vertical merge of parts with lightweight deleted rows
 2022-08-25 20:47:44 +02:00
avogar
cef2972af4 Better format detection for url table function/engine 2022-08-25 18:31:06 +00:00
Alexey Milovidov
fcba079baa
Merge pull request #40572 from Avogar/fix-map-bug 
Fix possible logical error in arrayElement function with Map
 2022-08-25 20:17:22 +03:00
Alexander Tokmakov
eae2667a1c
Merge pull request #40148 from ClickHouse/replicate-tables-readonly 
`ReplicatedMergeTree` tables as readonly until attach is finished
 2022-08-25 19:39:19 +03:00
Alexander Tokmakov
a6f55b1430 fix 2022-08-25 16:27:12 +02:00
avogar
1844873dbc Fix short circuit execution of toFixedString function 2022-08-25 16:15:22 +02:00
zvonand
3b164f0b12 style fix 2022-08-25 17:07:41 +03:00
Vladimir C
ddde5096ef
Merge branch 'master' into vdimir/tmp-file-metrics 2022-08-25 15:23:35 +02:00
alesapin
10fdc1e286 Fix stupid bug 2022-08-25 14:52:01 +02:00
zvonand
b3360c169d Fix handling zeros/'1's at the beginning of input 2022-08-25 15:35:39 +03:00
Frank Chen
ef49cacefe Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:32:22 +08:00
zvonand
23357350d3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-minmax 2022-08-25 15:32:04 +03:00
Frank Chen
92f7ca3616 Move TracingContextOnThread::current() out of class for simplicity 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:23:56 +08:00
Alexander Tokmakov
0aea4c7184 fix 2022-08-25 14:22:27 +02:00
Frank Chen
bb00dcc19b Remove using namespace from header 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:20:13 +08:00
Frank Chen
c9ea4f9f77 Change compression_method from String to CompressionMethod 2022-08-25 19:18:04 +08:00
Kseniia Sumarokova
fac87d9c4d
Update FileSegment.cpp 2022-08-25 13:18:03 +02:00
alesapin
3ff6489fae Merge branch 'master' into stress_s3 2022-08-25 13:14:58 +02:00
alesapin
7d1e3ecfa3
Merge pull request #40506 from ClickHouse/fix_bad_exception 
Fix lost errors in AWS CPP SDK
 2022-08-25 13:07:35 +02:00
Kseniia Sumarokova
2089ce14dd
Merge branch 'master' into fix-race 2022-08-25 13:03:29 +02:00
Kseniia Sumarokova
44577ea8f6
Update FileSegment.cpp 2022-08-25 13:02:41 +02:00
Kseniia Sumarokova
7e0ff0dfaf
Update FileSegment.cpp 2022-08-25 12:53:51 +02:00
Kseniia Sumarokova
b0f5fd9bf6
Update FileSegment.cpp 2022-08-25 12:41:31 +02:00
alesapin
c4f8811701
Merge pull request #40395 from ClickHouse/data_source_description 
Add copy shortcut for backups if dest and source support it
 2022-08-25 12:36:30 +02:00
Kseniia Sumarokova
4b62d0f75f
Merge pull request #40511 from maks-buren630501/master 
Fix bug (issues 40262)
 2022-08-25 12:18:23 +02:00
pzhdfy
e85e885d40 add blank line 2022-08-25 18:15:01 +08:00
Kseniia Sumarokova
f6eea03120
Merge pull request #40569 from kssenii/add-comments 
Add some logs to fs cache
 2022-08-25 12:06:48 +02:00
zvonand
24dfa0c64a tryfix darwin 2022-08-25 12:28:15 +03:00
Kruglov Pavel
66e677b307
Merge pull request #40546 from Avogar/fix-inferring-datetimes 
Fix possible error 'Decimal math overflow' while parsing DateTime64
 2022-08-25 11:18:30 +02:00
Kruglov Pavel
25d7d05148
Merge pull request #40567 from ClickHouse/Avogar-patch-1 
Better exception messages in FunctionArrayMapped
 2022-08-25 11:14:06 +02:00
vdimir
414872b1b9 Rename logger.warning_supress_regex -> warning_supress_regexp 2022-08-25 11:00:01 +02:00
vdimir
19cb068f64 Add config option warning_supress_regex 2022-08-25 11:00:01 +02:00
Antonio Andelic
f90612f577 Remove shutdown assert 2022-08-25 07:07:54 +00:00
Frank Chen
221a71f801 Fix typo 2022-08-25 15:05:07 +08:00
Frank Chen
4c1fe0d623 validate parameter 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 14:49:17 +08:00
helifu
b52e098a40 Fix a crash while the grpc port is conflict 2022-08-25 10:56:53 +08:00
Frank Chen
558b179cb1 Fix typo 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 10:14:18 +08:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
zvonand
3797e7cfbe Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-minmax 2022-08-25 01:57:49 +03:00
Kseniia Sumarokova
ba4dfe8308
Update MaterializedPostgreSQLConsumer.cpp 2022-08-25 00:23:59 +02:00
Kseniia Sumarokova
fec58f7d1b
Merge pull request #40578 from ClickHouse/kssenii-patch-4 
Fix after #40420
 2022-08-24 23:53:26 +02:00
Alexey Milovidov
aaafb797c0
Merge pull request #40541 from ClickHouse/paranoid_check_for_s3_uploads 
Add paranoid option to check blobs to S3 after upload
 2022-08-24 23:51:22 +02:00
zvonand
a61fd73c88 fix test fails 2022-08-25 00:26:44 +03:00
Alexander Tokmakov
3cd26aafe4 remove wrong code for skipping mutations in MergeTree 2022-08-24 20:58:59 +02:00
Kseniia Sumarokova
4e7578cf3f
Merge pull request #40565 from FrankChen021/url_engine 
Fix segment fault when writing data to URL table engine
 2022-08-24 20:36:45 +02:00
Antonio Andelic
797e21761a Address PR comments 2022-08-24 18:32:03 +00:00
Nikita Taranov
ac34a17551
Merge branch 'master' into control_block_size_in_aggregator 2022-08-24 20:25:28 +02:00