thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
44,396 Commits 9,326 Branches 1,784 Tags 2.1 GiB
00d2219808
Commit Graph

3428 Commits

Author SHA1 Message Date
Alexey Milovidov
56b835effa Fix typo in setting name 2020-07-08 14:53:07 +03:00
Alexey Milovidov
f16afa4dee Fix typo in setting name 2020-07-08 14:49:44 +03:00
alexey-milovidov
df8cde7b54
Merge pull request #12279 from ClickHouse/fix-array-fill 
Fix arrayFill for empty arrays.
 2020-07-08 14:28:15 +03:00
alexey-milovidov
3f0c257389
Merge pull request #12275 from ClickHouse/supertype-low-cardinality 
Supertype for LowCardinality
 2020-07-08 14:27:48 +03:00
alexey-milovidov
51afbf8f8d
Merge pull request #12276 from ClickHouse/test-pod-array 
Added a test for PaddedPODArray just in case
 2020-07-08 14:27:16 +03:00
Artem Zuikov
26295f8695
try remove strange logic in DuplicateOrderByVisitor (#12267) 2020-07-08 14:24:50 +03:00
Vitaly Baranov
4e40965c88 Show error after TrieDictionary failed to load. 2020-07-08 13:55:39 +03:00
alesapin
84f8bf1e14
Merge pull request #10697 from CurtizJ/polymorphic-parts 
Polymorphic parts (in-memory format)
 2020-07-08 13:25:45 +03:00
Alexander Kuzmenkov
a420b1989a Merge remote-tracking branch 'origin/master' into HEAD 2020-07-08 13:18:09 +03:00
alesapin
f04a49a114
Merge pull request #12256 from ianton-ru/redirect_limit_exception 
throw exception on redirect limit in S3 request
 2020-07-08 10:33:42 +03:00
Alexey Milovidov
9c53a90e3d Non-significant changes 2020-07-08 06:16:20 +03:00
alexey-milovidov
d0d1590d0d
Merge pull request #12271 from ClickHouse/fix-logical-functions-review 
Fix minor issues in ternary logic
 2020-07-08 05:33:37 +03:00
Alexey Milovidov
34e67b521a Fix ugly ugliness 2020-07-08 04:56:50 +03:00
Alexey Milovidov
abf1e015b3 Add documentation for arrayFill 2020-07-08 04:44:24 +03:00
Alexey Milovidov
934d18b649 Added a test for PaddedPODArray just in case 2020-07-08 02:41:10 +03:00
alexey-milovidov
b4c2c4906e
Merge pull request #11873 from ClickHouse/initial-explain 
Initial explain
 2020-07-08 01:59:27 +03:00
Alexey Milovidov
3a1ed562db Merge branch 'master' into initial-explain 2020-07-08 01:58:25 +03:00
alexey-milovidov
ffdd6e1af6
Merge pull request #12173 from nikitamikhaylov/summing-bug-fix 
Exclude partition key columns from SummingMergeTree
 2020-07-08 01:46:12 +03:00
Alexey Milovidov
7138ccca2b Whitespace 2020-07-08 01:44:29 +03:00
Alexey Milovidov
60c1ffc892 Implement supertype for LowCardinality 2020-07-08 01:26:43 +03:00
Alexey Milovidov
1f978022c5 Implement getLeastSuperType for LowCardinality #8212 2020-07-08 01:10:23 +03:00
Azat Khuzhin
6a04de61b6 Allow isInjective() with empty block (is function injective with any arguments) 
Since most of the time function will ignore it anyway, and creating
arguments just for checking is function injective or not is overkill
 2020-07-08 00:28:49 +03:00
Azat Khuzhin
6e695809e4 Reset CurrentMetrics::MemoryTracking periodically to the process RSS 2020-07-08 00:19:46 +03:00
Alexey Milovidov
5ef4e90dfb Fix minor issues after #12196 2020-07-07 23:56:40 +03:00
Azat Khuzhin
fd5e08884a Bump arrow to 0.17 (and flatbuffers to v1.12, required by arrow) 
MOTIVATION:
- remove double-conversion external dependency
- remove flatc (but flatbuffers is still required, arrow just shipped
  with generated files and that's it)

CHANGED:
- remove pre-generated headers, it is shipped with the arrow
- remove flatc (see above)

NOTES (see tests changes):
- and snappy error is reported as unsupported compression.
 2020-07-07 23:54:32 +03:00
alexey-milovidov
0fdc29be1e
Merge pull request #12196 from ClickHouse/akz/logical-functions-normalized-uint8 
Fixed logical functions for UInt8 values when they are not equal to 0 or 1.
 2020-07-07 23:19:35 +03:00
Nikolai Kochetov
fd7fcb28d4 Merge branch 'master' into initial-explain 2020-07-07 22:52:09 +03:00
Nikolai Kochetov
6ed2908e2a Rename ident to indent. 2020-07-07 22:51:32 +03:00
alexey-milovidov
546fcc9ed6
Merge pull request #12151 from ClickHouse/fix-transform-query-aliases 
Fix transform query for external databases in presense of aliases
 2020-07-07 21:46:35 +03:00
Anton Ivashkin
78ba9c986f throw exception on redirect limit in S3 request 2020-07-07 16:20:48 +03:00
Alexander Kuzmenkov
ac436c79eb Merge remote-tracking branch 'origin/master' into HEAD 2020-07-07 15:42:11 +03:00
Nikita Mikhaylov
270551e599 done 2020-07-07 14:45:20 +03:00
Alexander Kuzmenkov
688f185f9e style fix for #12152 2020-07-07 13:28:45 +03:00
Alexander Kazakov
2906ae37f5 Simple (and fast) inplace fix for UInt8 -> bool 2020-07-07 13:26:11 +03:00
Anton Popov
416d9648c2 remove questionable functionality 2020-07-07 12:58:14 +03:00
alesapin
be85f9f98a
Merge pull request #12165 from vitlibar/fix-split-dictionary-source-table-name 
Fix splitting table name of dictionary source
 2020-07-07 11:49:36 +03:00
Azat Khuzhin
68ca3b7aec Do not try to adjust memory tracker amount if it is not larger then in total 2020-07-07 11:26:40 +03:00
Alexander Kuzmenkov
dce7709405
Merge pull request #12152 from BohuTANG/mysql_kill_query 
Support MySQL 'KILL QUERY [connection_id]'
 2020-07-07 11:07:20 +03:00
tavplubix
488a8cbc9f
Merge pull request #12120 from filimonov/query_context_for_system_logs 
Add query context for system logs and to Buffer
 2020-07-07 09:55:20 +03:00
Nikolai Kochetov
e9d1aea745 Fix tests. 2020-07-07 09:19:03 +03:00
Azat Khuzhin
f3ab0aa081 gtest_compressionCodec: suppress non instantiated gtest warning 
gtest reports:

    [ RUN      ] GoogleTestVerification.UninstantiatedParameterizedTestSuite<CodecTestPerformance>
    ../src/Compression/tests/gtest_compressionCodec.cpp:590: Failure
    Parameterized test suite CodecTestPerformance is defined via TEST_P, but never instantiated. None of the test cases will run. Either no INSTANTIATE_TEST_SUITE_P is provided or the only ones provided expand to nothing.

    Ideally, TEST_P definitions should only ever be included as part of binaries that intend to use them. (As opposed to, for example, being placed in a library that may be linked in to get other utilities.)

    To suppress this error for this test suite, insert the following line (in a non-header) in the namespace it is defined in:

    GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(CodecTestPerformance);
    [  FAILED  ] GoogleTestVerification.UninstantiatedParameterizedTestSuite<CodecTestPerformance> (0 ms)
 2020-07-07 02:00:08 +03:00
Azat Khuzhin
d68ecdc84c Cap max_memory_usage* limits to the process resident memory 
There are still some issues with memory tracking, but now with per-user
tracking:

    executeQuery: Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 437.72 GiB (attempt to allocate chunk of 4200926 bytes), maximum: 437.72 GiB (version 20.6.1.1) (from 10.7.140.7:31318)

Although the server is mostly idle:

    SELECT formatReadableSize(memory_usage)
    FROM system.processes

    ┌─formatReadableSize(memory_usage)─┐
    │ 289.28 MiB                       │
    │ 155.75 MiB                       │
    │ 0.00 B                           │
    └──────────────────────────────────┘

Refs: https://github.com/ClickHouse/ClickHouse/pull/10496/files#r450206865
Cc: @alexey-milovidov
 2020-07-07 01:51:21 +03:00
Alexander Kazakov
1c0c8a7bf6
Merge pull request #11892 from excitoon-favorites/relatives3paths 
Switched paths in S3 metadata to relative
 2020-07-07 00:13:07 +03:00
Vitaly Baranov
4733504b51 Don't split dictionary source's table name into schema and table name itself 
if ODBC driver doesn't support schema.
 2020-07-06 23:41:39 +03:00
Alexey Milovidov
2e8ba3427a Better assert 2020-07-06 23:40:32 +03:00
Azat Khuzhin
c59907c1d6 gtest_weak_hash_32: fix lack of operator<< for char8_t 2020-07-06 23:34:40 +03:00
Azat Khuzhin
0a24d9eabf gtest_compressionCodec: fix lack of operator<< for char8_t 2020-07-06 23:34:40 +03:00
Azat Khuzhin
dab5b5ad7b gtest_compressionCodec: use fmt over boost::format 
boost::format is not compiled under gcc10:

                     from ../src/Compression/tests/gtest_compressionCodec.cpp:14:
    /usr/include/boost/format/alt_sstream_impl.hpp: In instantiation of ‘boost::io::basic_altstringbuf<Ch, Tr, Alloc>::int_type boost::io::basic_altstringbuf<Ch, Tr, Alloc>::overflow(boost::io::basic_altstringbuf<Ch, Tr, Alloc>::int_type) [with Ch = char; Tr = std::char_traits<char>; Alloc = std::allocator<char>; boost::io::basic_altstringbuf<Ch, Tr, Alloc>::int_type = int]’:
    /usr/include/boost/format/alt_sstream_impl.hpp:227:9:   required from here
    /usr/include/boost/format/alt_sstream_impl.hpp:261:45: error: no matching function for call to ‘std::allocator<char>::allocate(std::size_t&, char*)’
      261 |                     newptr = alloc_.allocate(new_size, is_allocated_? oldptr : 0);

(although this is system-wide boost, it is pretty recent - 1.72)
 2020-07-06 23:34:40 +03:00
Azat Khuzhin
a276d0da4f gtest_compressionCodec: is_trivial+is_standard_layout over deprecated is_pod 2020-07-06 23:34:40 +03:00
Azat Khuzhin
128dd4fa8a Fix dictGet arguments check during GROUP BY injective functions elimination 
This patch changes the place where the dictionary will be loaded (during
syntax analysis), but I guess this is fine, it will be loaded anyway.

Fixes: #10342
 2020-07-06 22:53:17 +03:00
Azat Khuzhin
6310e49032 Fix dictGet with bad arguments during GROUP BY injective functions elimination 2020-07-06 22:53:17 +03:00
Mikhail Filimonov
93517b4e82 Same change for Kafka - just in case, and to make it conform. 2020-07-06 20:48:33 +02:00
alesapin
3f90f65678 Merge remote-tracking branch 'origin' into CurtizJ-polymorphic-parts 2020-07-06 21:34:17 +03:00
Nikita Mikhaylov
2b539e6ace fix style 2020-07-06 20:26:15 +03:00
Mikhail Filimonov
8038383f06 Fix #10437, CR fixes 2020-07-06 19:24:33 +02:00
alexey-milovidov
3059be55a3
Merge branch 'master' into fix-test-under-thread-fuzzer 2020-07-06 19:49:25 +03:00
Nikita Mikhaylov
53522c728b
Merge pull request #11662 from Avogar/orc_output_format 
Add ORCBlockOutputFormat
 2020-07-06 18:51:35 +04:00
Nikita Mikhaylov
d31ed58f01 done 2020-07-06 17:33:31 +03:00
Anton Popov
66bed10ec1
Merge pull request #12109 from PerformanceVision/initialize_aggregation 
Create initializeAggregation to initialize an aggregation function
 2020-07-06 15:12:30 +03:00
alesapin
8e767bf5d0
Merge pull request #12153 from ClickHouse/fix-mutations-interpreter 
Fix wrong logic in MutationsInterpreter
 2020-07-06 14:43:54 +03:00
Alexander Kuzmenkov
d648628a4b
Merge pull request #12075 from azat/StorageKafka-SIGSEGV-fix 
Fix SIGSEGV in StorageKafka on DROP TABLE
 2020-07-06 13:56:06 +03:00
Nikita Mikhaylov
5c6d6bdf54
Merge pull request #11995 from azat/load_balancing-priority 
Load balancing manual priority
 2020-07-06 13:32:20 +04:00
Nikita Mikhaylov
dac7a39256 fix ubsan final 2020-07-06 12:29:22 +03:00
alexey-milovidov
b6a19b5eff
Merge pull request #12148 from ClickHouse/fix-bad-redundant-order-by-optimization 
Fix bad code in redundant ORDER BY optimization
 2020-07-06 09:21:45 +03:00
alexey-milovidov
4ec787ce48
Merge pull request #12147 from ClickHouse/fix-logical-error-table-function-remote 
Change exception code from LOGICAL_ERROR to BAD_ARGUMENTS when the name of remote table is empty.
 2020-07-06 09:21:23 +03:00
Alexey Milovidov
1ca45c1b7e Fix mutations interpreter #9088 2020-07-06 09:05:48 +03:00
Alexey Milovidov
71b410eb10 Remove unused potentially dangerous function 2020-07-06 09:05:48 +03:00
Alexey Milovidov
2a026ee902 Merge branch 'master' into fix-test-under-thread-fuzzer 2020-07-06 08:49:20 +03:00
alexey-milovidov
3aad122153
Merge pull request #12092 from CurtizJ/fix-state-resample 
Fix segfault with -StateResample combinators
 2020-07-06 08:47:42 +03:00
Alexey Milovidov
0124eeb2d8 Added function "hasThreadFuzzer" 2020-07-06 06:21:29 +03:00
Guillaume Tassery
569a89280c update ya.make 2020-07-06 10:06:45 +07:00
Alexey Milovidov
6483ba7d52 Style 2020-07-06 05:07:55 +03:00
BohuTANG
cb04c503d7 Add MySQL to ClickHouse query replacement mapping table 2020-07-06 10:07:38 +08:00
Alexey Milovidov
f08ff9010e Whitespace 2020-07-06 04:57:10 +03:00
Alexey Milovidov
95a7a09c37 Fix transform query for external databases in presense of aliases #12032 2020-07-06 04:50:45 +03:00
Alexey Milovidov
de4459c25a Fix error 2020-07-06 04:25:45 +03:00
alexey-milovidov
ea0fb005fb
Merge pull request #12116 from vitlibar/fix-table-dictionary-dependency 
Fix handling dictionary-table dependency
 2020-07-06 04:19:43 +03:00
BohuTANG
37ac456439 Support KILL QUERY [connection_id] for MySQL 2020-07-06 09:02:02 +08:00
Alexey Milovidov
7f3a8f3eb9 Fix bad code in redundant ORDER BY optimization #10067 2020-07-06 02:50:20 +03:00
Anton Popov
e2e92afd98 fix segfault with -StateResample combinators 2020-07-06 02:17:46 +03:00
Alexey Milovidov
3ae9d3c9f3 Change exception code from LOGICAL_ERROR to BAD_ARGUMENTS when the name of remote table is empty 2020-07-06 02:13:44 +03:00
flynn
5816858403
add parseDateTimeBestEffortUS function (#12028) 
* add function parseDateTimeBestEffortUS
* add test
* add doc

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-07-05 21:09:20 +03:00
alexey-milovidov
6a013b966c
Update AggregateDescription.cpp 2020-07-05 19:27:54 +03:00
alexey-milovidov
a3f0ba17ea
Update SortDescription.h 2020-07-05 19:26:57 +03:00
alexey-milovidov
c08e24d9ab
Merge pull request #12136 from vivarum/vivarum-04659-new-ISO8601-year-modificators-for-formatDateTime 
New ISO8601 year modificators for formatDateTime() function
 2020-07-05 19:10:06 +03:00
alexey-milovidov
ab15c8d6ca
Merge pull request #12021 from ClickHouse/fix-if-fixed-string 
Fix function if with FixedString arguments of different sizes
 2020-07-05 19:02:21 +03:00
alexey-milovidov
0f1bfa5b19
Merge pull request #12042 from ClickHouse/fix-constraints 
Fix constraints check for constant columns, step 1.
 2020-07-05 19:01:33 +03:00
alexey-milovidov
eecf7ae3ee
Merge pull request #12140 from ClickHouse/fix-division 
Fix potential overflow in integer division
 2020-07-05 19:00:41 +03:00
myrrc
8c3417fbf7
ILIKE operator (#12125) 
* Integrated CachingAllocator into MarkCache

* fixed build errors

* reset func hotfix

* upd: Fixing build

* updated submodules links

* fix 2

* updating grabber allocator proto

* updating lost work

* updating CMake to use concepts

* some other changes to get it building (integration into MarkCache)

* further integration into caches

* updated Async metrics, fixed some build errors

* and some other errors revealing

* added perfect forwarding to some functions

* fix: forward template

* fix: constexpr modifier

* fix: FakePODAllocator missing member func

* updated PODArray constructor taking alloc params

* fix: PODArray overload with n restored

* fix: FakePODAlloc duplicating alloc() func

* added constexpr variable for alloc_tag_t

* split cache values by allocators, provided updates

* fix: memcpy

* fix: constexpr modifier

* fix: noexcept modifier

* fix: alloc_tag_t for PODArray constructor

* fix: PODArray copy ctor with different alloc

* fix: resize() signature

* updating to lastest working master

* syncing with 273267

* first draft version

* fix: update Searcher to case-insensitive

* added ILIKE test

* fixed style errors, updated test, split like and ilike,  added notILike

* replaced inconsistent comments

* fixed show tables ilike

* updated missing test cases

* regenerated ya.make

* Update 01355_ilike.sql

Co-authored-by: myrrc <me-clickhouse@myrrec.space>
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-07-05 18:57:59 +03:00
alexey-milovidov
cb126a23ef
Update StorageDictionary.h 2020-07-05 18:45:05 +03:00
alexey-milovidov
1a760e18a5
Update StorageDictionary.h 2020-07-05 18:44:28 +03:00
alexey-milovidov
ec563e54dc
Merge pull request #12138 from ClickHouse/codeql 
Fix warnings from CodeQL
 2020-07-05 18:22:49 +03:00
alexey-milovidov
f02edaa48c
Merge pull request #12137 from ClickHouse/fix-loop-geodistance 
Remove harmful code from "geoDistance"
 2020-07-05 18:22:15 +03:00
Vitaly Baranov
99e9b15e00 Make code clearer: use enum instead of bool internal. 2020-07-05 17:14:29 +03:00
alexey-milovidov
32a5dcd483
Update arraySum.cpp 2020-07-05 06:48:49 +03:00
alexey-milovidov
c257255759
Merge pull request #12134 from matwey/fix/missed_include 
Add missed <atomic> include in ProxyListConfiguration
 2020-07-05 04:22:42 +03:00
Alexey Milovidov
6e1c1b0dbb Whitespace 2020-07-05 03:35:57 +03:00
Alexey Milovidov
73a5c38398 Fix potential overflow in integer division #12119 2020-07-05 03:29:03 +03:00
Alexey Milovidov
d2e8c0df79 Minor modification 2020-07-05 02:39:27 +03:00
Alexey Milovidov
96e8e93c64 Fix warnings from CodeQL 2020-07-05 02:16:16 +03:00
alexey-milovidov
c696354dd4
Merge pull request #12115 from ianton-ru/disk-type-in-system-disks 
Add type column in system.disks
 2020-07-05 01:04:17 +03:00
alexey-milovidov
f8a43d5e07
Update formatDateTime.cpp 2020-07-05 01:00:30 +03:00
Alexey Milovidov
731d92657d Remove harmful code from "geoDistance" #12117 2020-07-05 00:57:57 +03:00
alexey-milovidov
e05283d5d3
Merge pull request #12060 from ClickHouse/fix-12053 
Check wrong type for filter.
 2020-07-04 23:46:34 +03:00
alexey-milovidov
cf1967bb29
Merge pull request #12089 from ClickHouse/fix-11595 
Fix result_rows and result_bytes metrics for selects.
 2020-07-04 23:41:51 +03:00
alexey-milovidov
c6def5f14b
Merge pull request #12108 from ClickHouse/fix-12086 
Fix Parquet for LowCardinality
 2020-07-04 23:39:38 +03:00
Alexey Milovidov
c4b6c780c6 Merge branch 'master' into fix-constraints 2020-07-04 23:36:48 +03:00
Alexey Milovidov
918e979449 Merge branch 'master' into fix-if-fixed-string 2020-07-04 22:00:27 +03:00
vivarum
d387fa719e New ISO8601 year modificators for formatDateTime 2020-07-04 21:54:42 +03:00
alexey-milovidov
f1151ed474
Merge pull request #12133 from ClickHouse/normalize-pid-file 
Normalize "pid" file handling #3501
 2020-07-04 21:36:27 +03:00
alexey-milovidov
d9ea8cfc42
Update StatusFile.cpp 2020-07-04 16:57:04 +03:00
Alexey Milovidov
176a7f2f72 Normalize "pid" file handling #3501 2020-07-04 16:54:24 +03:00
Anton Popov
73676f5022
Improve performace of reading in order of sorting key. (#11696) 
* simplify reading in order of sorting key

* add perf test for reading many parts

* Revert "simplify reading in order of sorting key"

This reverts commit 7267d7c46e.

* add threshold for preliminary merge for reading in order

* better threshold

* limit threads in test
 2020-07-04 15:48:51 +03:00
Matwey V. Kornilov
caafbe19a4 Add missed <atomic> include in ProxyListConfiguration 2020-07-04 15:32:02 +03:00
alesapin
a2b6d58053
Merge pull request #11069 from kssenii/add-storage-rabbitmq-read-only 
Add storage RabbitMQ
 2020-07-04 10:38:50 +03:00
alesapin
bfa8cdc7a4
Move skip lists for tests to skip_list.json file (#12107) 
* Add unbundled mode flag

* Move skip lists to clickhouse-test

* More verbose message about skip

* Make skip-list optional

* Update clickhouse-test
 2020-07-04 10:35:45 +03:00
alesapin
8dc204350f
Fix version column in replicated version collapsing merge tree (#12121) 2020-07-04 10:35:17 +03:00
alexey-milovidov
6677e0a503
Merge pull request #12073 from zhang2014/feature/atfer_column_modify_column 
ISSUES-4006 support first for ALTER ADD|MODIFY COLUMN
 2020-07-03 22:54:05 +03:00
Vitaly Baranov
ab343132de Fix handling dependency of table with ENGINE=Dictionary on dictionary. 2020-07-03 22:05:53 +03:00
Mikhail Filimonov
6e599533a7 Add missing query context for system logs 
Needed to allow attaching materialized views with joins or with
subqueries to system.logs.
 2020-07-03 19:19:32 +02:00
Nikita Mikhaylov
5c97106ea3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into Avogar_orc_output_format 2020-07-03 18:00:28 +03:00
Nikita Mikhaylov
fba02badcb fix test with UBSan 2020-07-03 17:53:54 +03:00
Vitaly Baranov
929d652297
Merge pull request #12083 from vitlibar/improve-revoke-command 
Improve REVOKE command
 2020-07-03 17:17:36 +03:00
Nikolai Kochetov
8349afa4ad Try fix tests. 2020-07-03 16:38:35 +03:00
Alexander Kuzmenkov
e7e4d29a54 more fuzzing 2020-07-03 15:52:16 +03:00
Anton Ivashkin
23b44ca6fe Add type column in system.disks 2020-07-03 15:45:54 +03:00
Guillaume Tassery
027c6b705f Create initializeAggregation to initialize an aggregation function based on a value 2020-07-03 19:18:47 +07:00
Nikolai Kochetov
6d08b7b45a Remove LC converting to Arrow. 2020-07-03 14:04:36 +03:00
Nikolai Kochetov
3d37a63247 Remove LC converting to Arrow. 2020-07-03 13:37:55 +03:00
Nikolai Kochetov
b3f1842035 Remove LC converting to Arrow. 2020-07-03 13:35:44 +03:00
alesapin
0a6f3ca9fd Add unbundled mode flag 2020-07-03 13:11:16 +03:00
Nikolai Kochetov
5e8d5bc8c0 Fix overlimiting the nuber of threads for union. 2020-07-03 11:10:52 +03:00
alesapin
c57edd2018 Tiny fixes 2020-07-03 11:02:35 +03:00
Nikolai Kochetov
17c89f3fe0
Merge pull request #12085 from ClickHouse/fix-11937 
Fix limiting the number of threads for VIEW.
 2020-07-03 10:35:04 +03:00
Azat Khuzhin
bd5e5e9462 kafka: remove outdated comment 
As stated by @filimonov it is not relevant (after #11599)
 2020-07-03 09:23:39 +03:00
Azat Khuzhin
de8bc99d48 kafka: avoid superior polling after DROP/DETACH TABLE 
Before this patch isStalled() was checked before polledDataUnusable(),
and after DROP TABLE isStalled() == true (although this looks tricky).
 2020-07-03 09:23:39 +03:00
Azat Khuzhin
8508200020 kafka: improve logging during engine shutdown 
This will help with tracking possible issues, when you need to know was
buffer released or not.
 2020-07-03 09:23:39 +03:00
Azat Khuzhin
e7179c402c kafka: check that the data is still usable after parsing 2020-07-03 09:23:38 +03:00
Azat Khuzhin
6104872cae kafka: fix SIGSEGV on DROP TABLE 
After #11599 it is possible that messages of the
ReadBufferFromKafkaConsumer will be cleaned-up right in
read_kafka_message callback (from KafkaBlockInputStream) if the stop
flag isset (i.e. DROP TABLE is waiting the consumer), and if
read_kafka_message already processed some rows it will not return 0 and
the loop after will try to get current topic from the buffer, which uses
messages in the underlying and this will got SIGSEGV:

    12:14:56.173262 [ 55421 ] {f7930856-d478-4e41-af56-24ce7b693e95} <Debug> executeQuery: (from 0.0.0.0:0, user: ) DROP TABLE IF EXISTS data.queue
    12:14:56.173285 [ 55421 ] {f7930856-d478-4e41-af56-24ce7b693e95} <Trace> StorageKafka (newly_queue): Waiting for cleanup
    12:14:56.180016 [ 55390 ] {} <Trace> BaseDaemon: Received signal 11
    12:14:56.180267 [ 4914 ] {} <Fatal> BaseDaemon: ########################################
    12:14:56.181879 [ 4914 ] {} <Fatal> BaseDaemon: (version 20.6.1.1, build id: 4CE0298F08583658) (from thread 55468) (no query) Received signal Segmentation fault (11)
    12:14:56.181900 [ 4914 ] {} <Fatal> BaseDaemon: Address: 0x8 Access: read. Address not mapped to object.
    12:14:56.181909 [ 4914 ] {} <Fatal> BaseDaemon: Stack trace:
    12:14:56.184676 [ 4914 ] {} <Fatal> BaseDaemon: 3. /ch/contrib/cppkafka/include/cppkafka/message.h:111: DB::KafkaBlockInputStream::readImpl() @ 0xe343f1c in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.185553 [ 4914 ] {} <Fatal> BaseDaemon: 4. /ch/contrib/libcxx/include/vector:1003: DB::IBlockInputStream::read() @ 0xd9d95bd in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.188238 [ 4914 ] {} <Fatal> BaseDaemon: 5. /ch/src/DataStreams/copyData.cpp:26: DB::copyData() @ 0xd9f712a in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.188780 [ 4914 ] {} <Fatal> BaseDaemon: 6. /ch/contrib/libcxx/include/vector:1532: DB::StorageKafka::streamToViews() @ 0xe335e73 in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.189331 [ 4914 ] {} <Fatal> BaseDaemon: 7. /ch/src/Storages/Kafka/StorageKafka.cpp:491: DB::StorageKafka::threadFunc() @ 0xe336738 in /usr/lib/debug/usr/bin/clickhouse

55421 thread (shows that it still waiting for deactivation):

    5  std::__1::lock_guard<>::lock_guard () at ../contrib/libcxx/include/__mutex_base:90
    6  DB::BackgroundSchedulePoolTaskInfo::deactivate (this=0x7fc7e4465f20) at ../src/Core/BackgroundSchedulePool.cpp:59
    7  DB::StorageKafka::shutdown (this=0x7fc7e45e4600) at ../contrib/libcxx/include/memory:3821

And just in case thread where read_kafka_message is called:

    0  DB::ReadBufferFromKafkaConsumer::nextImpl (this=0x7fd4901d4118) at ../contrib/libcxx/include/atomic:1491
    1  DB::ReadBuffer::next (this=0x7fd4901d4118) at ../src/IO/ReadBuffer.h:59
    2  DB::ReadBuffer::eof (this=0x7fd4901d4118) at ../src/IO/ReadBuffer.h:81
    3  DB::skipWhitespaceIfAny (buf=...) at ../src/IO/ReadHelpers.h:945
    4  DB::JSONEachRowRowInputFormat::readRow (ext=..., columns=..., this=0x7fd499a7a020) at ../src/Processors/Formats/Impl/JSONEachRowRowInputFormat.cpp:222
    5  DB::JSONEachRowRowInputFormat::readRow (this=0x7fd499a7a020, columns=..., ext=...) at ../src/Processors/Formats/Impl/JSONEachRowRowInputFormat.cpp:218
    6  DB::IRowInputFormat::generate (this=0x7fd499a7a020) at ../src/Processors/Formats/IRowInputFormat.cpp:64
    7  DB::ISource::work (this=0x7fd499a7a020) at ../src/Processors/ISource.cpp:48
    8  DB::KafkaBlockInputStream::<lambda()>::operator()(void) const () at ../contrib/libcxx/include/memory:3826
    9  DB::KafkaBlockInputStream::readImpl (this=0x7fd46e718820) at ../contrib/libcxx/include/new:340

Cc: @filimonov
 2020-07-03 09:22:44 +03:00
Anton Popov
4422df2e37 Merge remote-tracking branch 'upstream/master' into HEAD 2020-07-02 20:18:21 +03:00
alesapin
c6c7ee30d9 Less race conditions 2020-07-02 19:44:04 +03:00
Anton Popov
a5cbeda1e3 fix segfault with -StateResample combinators 2020-07-02 19:13:42 +03:00
Nikolai Kochetov
3f51419889 Fix tests. 2020-07-02 18:26:44 +03:00
Nikolai Kochetov
e7ab4df148 Fix result_rows and result_bytes metrics for selects. 2020-07-02 18:00:37 +03:00
Nikolai Kochetov
57d727d078 Fix result_rows and result_bytes metrics for selects. 2020-07-02 17:51:10 +03:00
alesapin
96df2e6b71 Better shutdown and conversion 2020-07-02 17:35:10 +03:00
alexey-milovidov
82061da77c
Merge pull request #12040 from azat/dist-fixes 
Tiny fixes for distributed queries
 2020-07-02 17:02:56 +03:00
alesapin
64583ceb22 Merge branch 'master' into add-storage-rabbitmq-read-only 2020-07-02 16:08:29 +03:00
Anton Popov
71059e4dff
Merge pull request #12062 from nvartolomei/nv/set-index-tuple-types 
Try fix pk in tuple performance
 2020-07-02 15:51:03 +03:00
Nikolai Kochetov
6527e04ec3 Fix limiting the number of threads for VIEW. 2020-07-02 15:35:51 +03:00
alesapin
f87c3af097
Merge pull request #12055 from ClickHouse/fix-9766 
Fix arrayMap with Tuple(LowCardinality) argument
 2020-07-02 13:40:05 +03:00
alesapin
65dae99c12
Merge pull request #12056 from ClickHouse/fix-11808 
Fix defaultValueOfArgumentType
 2020-07-02 13:35:38 +03:00
alesapin
cec02d15ed
Merge pull request #12067 from excitoon-favorites/betters3logs 
Moved useless S3 logging to TRACE level
 2020-07-02 13:21:55 +03:00
Vitaly Baranov
03b36c262e Improve REVOKE command: now it requires only grant/admin option for only 
access which will be revoked.
REVOKE ALL FROM user1 now revokes all granted roles.
 2020-07-02 12:54:24 +03:00
Nikolai Kochetov
924f1dfc46 Allow to parse EXPLAIN PLAN. 2020-07-02 11:40:41 +03:00
Nikolai Kochetov
79cd33a567 Fix tests. 2020-07-02 10:44:47 +03:00
Alexander Kazakov
cc2cadb7f5
Merge pull request #11893 from excitoon-favorites/autostartmoves 
In *MergeTree: Parts moving task shall be started if new storage policy needs them
 2020-07-02 10:44:38 +03:00
zhang2014
9ed5c6d925 ISSUES-4006 support first for ALTER ADD|MODIFY COLUMN 2020-07-01 22:58:52 +08:00
Vitaly Baranov
811d124a82
Merge pull request #12002 from vitlibar/fix-partial-revokes 
Fix partial revokes
 2020-07-01 17:15:01 +03:00
Nicolae Vartolomei
3854ce6d84 Rewrite Set lookup to make it more readable 2020-07-01 15:05:54 +01:00
Nicolae Vartolomei
f8ceca6942 Remove const specifier to allow auto-move (clangtidy) 2020-07-01 15:04:08 +01:00
hcz
e284bf83d0 Optimize and fix 2020-07-01 11:21:53 +08:00
Vladimir Chebotarev
a541683efa Moved useless S3 logging to TRACE level. 2020-07-01 01:04:30 +03:00
Anton Popov
68c9896fa2 fix arcadia build 2020-07-01 00:15:39 +03:00
Anton Popov
d45697a1b5 in-memory-parts: lazy creation of WAL 2020-06-30 21:47:12 +03:00
alexey-milovidov
948af0bfc5
Merge pull request #12045 from ClickHouse/fix_loading_of_tables 
Parse metadata in parallel when loading tables
 2020-06-30 21:16:40 +03:00
Nikolai Kochetov
3585700506 Fix tests. 2020-06-30 19:41:43 +03:00
Vitaly Baranov
c39eb8f71b Fix partial revokes (complex cases). 2020-06-30 18:47:02 +03:00
Nicolae Vartolomei
8f1845185e Try fix pk in tuple performance 
Possible approach for fixing #10574

The problem is that prepared sets are built correctly, it is a hash map of key -> set
where key is a hash of AST and list of data types (when we a list of
tuples of literals).

However, when the key is built from the index to try and find if there
exists a prepared set that would match it looks for data types of the
primary key (see how data_types is populated) because the primary key
has only one field (v in my example) it can not find the prepared set.

The patch looks for any prepared indexes where data types match for the
subset of fields found in primary key, we are not interested in other
fields anyway for the purpose of primary key pruning.
 2020-06-30 16:33:38 +01:00
Vitaly Baranov
40942a2164
Merge pull request #12031 from vitlibar/fix-access-rights-if-allow_introspection_functions-is-0 
Fix calculation of access rights when allow_introspection_functions = 0
 2020-06-30 18:32:18 +03:00
Nikolai Kochetov
ddc231b088 Check type of filter for prewhere. 2020-06-30 17:58:11 +03:00
Nikolai Kochetov
e3aede57c6 Check type of filter for prewhere. 2020-06-30 17:48:00 +03:00
Nikolai Kochetov
18059e9289 Check type of filter for prewhere. 2020-06-30 17:44:49 +03:00
Nikolai Kochetov
66da0733ab Check type of filter for prewhere. 2020-06-30 17:20:27 +03:00
Nikita Mikhaylov
c45a054f65 fix clang-tidy 2020-06-30 16:06:35 +03:00
Vladimir Chebotarev
bb8da71eff Moves task shall be started if new storage policy needs them. 2020-06-30 15:36:44 +03:00
alexey-milovidov
81b43dc481
Merge pull request #12020 from ClickHouse/fix-collations-order-by-tuple-const 
Fix ORDER BY tuple with COLLATE on const column
 2020-06-30 15:09:57 +03:00
alexey-milovidov
08b6ca19d8
Merge pull request #12043 from ClickHouse/topk-enum 
Make topK return Enum for Enum types
 2020-06-30 15:08:42 +03:00
Nikita Mikhaylov
b74b27854a yet another try 2020-06-30 14:38:09 +03:00
Nikolai Kochetov
e8ee5176cd Fix defaultValueOfArgumentType 2020-06-30 14:29:50 +03:00
Artem Zuikov
29178e26da
fix low card types in merge join (#12035) 2020-06-30 14:13:43 +03:00
Nikolai Kochetov
820d2c63df Try fix labda tuple(LC) argument. 2020-06-30 13:51:17 +03:00
Nikolai Kochetov
be51741566 Try fix labda tuple(LC) argument. 2020-06-30 13:19:06 +03:00
Nikita Mikhaylov
7c828861da fix arcadia 2020-06-30 13:18:38 +03:00
Alexander Kuzmenkov
3a50b81e31 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-30 12:25:52 +03:00
Alexander Kuzmenkov
5c417f45b8 streaming wip 2020-06-30 12:25:23 +03:00
Alexander Kuzmenkov
aa8461e996
Update ThreadProfileEvents.cpp 2020-06-30 10:47:19 +03:00
Alexander Kuzmenkov
1d838b7b3a
Update ThreadProfileEvents.cpp 2020-06-30 09:56:11 +03:00
Alexander Kuzmenkov
67a28971e8 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-30 09:09:36 +03:00
Vladimir Chebotarev
8128ca10f4 Switched paths in S3 metadata to relative. 2020-06-30 07:59:22 +03:00
kssenii
f797efb04c Fix build 2020-06-30 01:48:11 +00:00
Alexey Milovidov
a23df81dd5 Fix error 2020-06-30 03:53:30 +03:00
Alexander Tokmakov
987e64acfe parse metadata in parallel when loading tables 2020-06-30 03:12:16 +03:00
Alexey Milovidov
15cddc3c6e Make topK return Enum for Enum types 2020-06-30 02:17:20 +03:00
Alexey Milovidov
d97cb1e7b4 Fix constraints check for constant columns 2020-06-30 01:54:20 +03:00
Alexey Milovidov
5a35ff8938 Merge branch 'master' of github.com:yandex/ClickHouse into fix-collations-order-by-tuple-const 2020-06-30 01:51:04 +03:00
Alexey Milovidov
1194863b3d Fix error 2020-06-30 01:50:26 +03:00
Alexey Milovidov
0bf7a8c554 Add multiIf 2020-06-30 01:50:26 +03:00
Alexey Milovidov
6fa1204698 Fix function if with FixedString arguments of different sizes 2020-06-30 01:50:26 +03:00
alexey-milovidov
c1d2d2d7f7
Merge pull request #12039 from ClickHouse/fix-nullable-tuple-compare 
Fix nullable tuple compare
 2020-06-30 01:38:46 +03:00