thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-03 04:52:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,859 Commits 9,181 Branches 1,782 Tags 2.2 GiB
05953f4cfa
Commit Graph

1526 Commits

Author SHA1 Message Date
Amos Bird
acfbae7107
Fix replace,All functions so that they don't generate garbage to stderr 2021-02-23 22:39:05 +08:00
hexiaoting
e123c6ef9d Bug fix: Tuple cast to Map 2021-02-23 18:42:33 +08:00
alexey-milovidov
cb8cfdba77
Merge branch 'master' into fffix 2021-02-22 23:08:44 +03:00
Azat Khuzhin
521537eed8 Fix DateTime64 from Float 2021-02-21 22:36:40 +03:00
alexey-milovidov
125bf1d0d9
Merge pull request #20479 from ucasFL/tr 
fix transform with floating point key
 2021-02-21 17:37:20 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
Alexey Milovidov
4085782b05 More gentle exception messages 2021-02-21 15:34:48 +03:00
feng lv
5d3b8d99e7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into tr 2021-02-21 07:34:09 +00:00
feng lv
a33183ee1e fix transform with floating point key 
better

update test
 2021-02-21 05:20:42 +00:00
Alexey Milovidov
64e76a4a8d Minor changes in Decimal 2021-02-20 21:13:36 +03:00
vdimir
6cc2fb5e9f
Try to fix race in storage join: block parralel inserts 2021-02-20 18:00:59 +03:00
Maksim Kita
71c65b5d82
Merge pull request #20611 from kitaisreal/rename-untyped-function-reinterpret-as 
Rename untyped function reinterpretAs into reinterpret
 2021-02-19 14:35:00 +03:00
Amos Bird
77fd060665
Normalize function names 2021-02-18 20:20:28 +08:00
zlx19950903
c92e613b82
Add a function htmlOrXmlCoarseParse to extract content from html or xml format string. (#19600) 
* add html and xml coarse parse

* add test file

* add conditional check: hyperscan

* fix style error

* add conditional check

* bug fix

* delete unit

* typos check fix

* add unit test

* style check fix

* fix build error: case style

* acradis_skip test fix

* LINT error fix

* Remove comments

Co-authored-by: guojiantao <guojiantao15@mails.ucas.ac.cn>
Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
Co-authored-by: Ivan Lezhankin <ilezhankin@yandex-team.ru>
 2021-02-18 15:05:55 +03:00
Maksim Kita
4278098f9a Reinterpret function added Decimal, DateTim64 support 2021-02-18 13:09:01 +03:00
Maksim Kita
c608fa1e6a Added error reinterpretation tests 2021-02-17 12:53:12 +03:00
Maksim Kita
5f88f5817f Rename untyped function reinterpretAs into reinterpret 2021-02-17 11:23:24 +03:00
Nikita Mikhailov
7c5d845866 refactor function 2021-02-16 15:37:49 +03:00
Nikita Mikhailov
d41b2a4d06 Merge branch 'master' of github.com:ClickHouse/ClickHouse into keen-wolf-filefuncwithreadbuf 2021-02-15 16:15:52 +03:00
Nikita Mikhaylov
92e13d76d3
Merge pull request #18508 from depressed-pho/runningConcurrency 
Add function runningConcurrency()
 2021-02-15 14:39:38 +03:00
Alexey Milovidov
d529db5498 Adjustments 2021-02-14 23:57:25 +03:00
Alexey Milovidov
55c17ac93f Fix UBSan report in intDiv 2021-02-14 06:11:12 +03:00
alexey-milovidov
3e9f1dc83b
Merge pull request #20458 from ClickHouse/array-difference-ubsan 
Fix UBSan report in arrayDifference
 2021-02-13 19:34:44 +03:00
Alexey Milovidov
1c55be261c Fix UBSan report in arrayDifference 2021-02-13 03:45:06 +03:00
alexey-milovidov
371e84957a
Merge pull request #20303 from azat/constant-folding-fixes 
Do not allow early constant folding of explicitly forbidden functions
 2021-02-13 02:20:06 +03:00
Nikita Mikhailov
2a52aa8ca3 fix test 2021-02-12 20:25:40 +03:00
Nikita Mikhailov
fcd080631f Merge branch 'master' of github.com:ClickHouse/ClickHouse into keen-wolf-filefuncwithreadbuf 2021-02-12 17:53:18 +03:00
alesapin
a94e223cee
Merge pull request #20344 from azat/join_use_nulls-fix 
Fix null dereference with join_use_nulls=1
 2021-02-12 12:48:53 +03:00
alesapin
9aadbde86c
Merge pull request #20339 from kitaisreal/decimal-binary-operation-constants-fix 
Decimal binary operation constants fix
 2021-02-12 12:32:58 +03:00
alexey-milovidov
4e32dbd024
Merge pull request #19317 from abyss7/dwarf-folly 
Print inline frames for fatal stacktraces
 2021-02-12 02:38:50 +03:00
Azat Khuzhin
f6cfcd4da9 Fix null dereference with join_use_nulls=1 
Found with MSan [1], the following query triggers null dereference:

```sql
SELECT
    Y.id - 1
FROM X
RIGHT JOIN Y ON (X.id + 1) = Y.id
SETTINGS join_use_nulls=1; -- { serverError 53 }
```

```
Received signal 11
(version 21.3.1.5916, build id: 2E9E84AA32AEAAC7C8B6EB45DA3EC0B4F15E9ED4) (from thread 100) (query_id: 9ab8cb0d-be8d-445e-8498-930a7268488b) Received signal Segmentation fault (11)
Address: 0x10 Access: read. Address not mapped to object.
Stack trace: 0x2d079d65 0x29bf1f30 0x12b12220 0x12b13098 0x12b17b08 0x12b20459 0x2ae37913 0x2ae352d9 0x2c746072 0x2c7585dd 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d
4. ./obj-x86_64-linux-gnu/../contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:0: DB::ColumnConst::ColumnConst(COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, unsigned long) @ 0x2d079d65 in /workspace/clickhouse
5. ./obj-x86_64-linux-gnu/../src/Common/COW.h:0: DB::createBlockWithNestedColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) @ 0x29bf1f30 in /workspace/clickhouse
6. DB::FunctionOverloadResolverAdaptor::getReturnTypeDefaultImplementationForNulls(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::function<std::__1::shared_ptr<DB::IDataType const> (std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&)> const&) @ 0x12b12220 in /workspace/clickhouse
7. DB::FunctionOverloadResolverAdaptor::getReturnTypeWithoutLowCardinality(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x12b13098 in /workspace/clickhouse
8. DB::FunctionOverloadResolverAdaptor::getReturnType(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x12b17b08 in /workspace/clickhouse
9. DB::FunctionOverloadResolverAdaptor::build(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x12b20459 in /workspace/clickhouse
```

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/64c0bf98290362fa216c05b070aa122a12af3c25/fuzzer_msan/report.html#fail1
 2021-02-11 21:30:05 +03:00
Maksim Kita
369dc613c4 Decimal binary operation constants fix 2021-02-11 19:05:17 +03:00
alexey-milovidov
263d751d64
Merge pull request #20169 from ClickHouse/no-excessive-squash-memory 
Disable excessive squashing of blocks for StorageMemory #13052
 2021-02-11 16:38:24 +03:00
alesapin
c2bb2c2902
Merge pull request #20097 from ClickHouse/remove-adding-missed-step 
Build actions dag to evaluate missing defaults.
 2021-02-11 10:51:21 +03:00
Alexey Milovidov
f442b30f30 Fix test 2021-02-11 01:23:27 +03:00
Azat Khuzhin
6b82e8ad19 Mark ignore() as not suitable for constant folding 2021-02-11 00:39:03 +03:00
Nikolai Kochetov
af214e794f Review fixes. 2021-02-10 15:45:39 +03:00
Maksim Kita
a2560cecf4
Merge branch 'master' into fuse 2021-02-09 18:30:18 +03:00
Ivan Lezhankin
a287443438 Merge remote-tracking branch 'upstream/master' into dwarf-folly 2021-02-08 18:36:41 +03:00
Nikolai Kochetov
bd8f88f464 Fix style 2021-02-08 12:04:19 +03:00
alexey-milovidov
3b43b0a1be
Merge pull request #20131 from azat/DateTime64-fix 
DateTime64 fixes
 2021-02-07 16:55:28 +03:00
alexey-milovidov
0b043c41b6
Merge pull request #20160 from ClickHouse/cumulative-sum-normalized 
Fix UBSan report in arrayCumSum
 2021-02-07 14:28:16 +03:00
Amos Bird
1b32292dd0
bitmap function for all native integers 2021-02-07 11:32:54 +08:00
Azat Khuzhin
46ff7d2ab0 Fix DateTime64 overflows 2021-02-07 02:24:31 +03:00
Alexey Milovidov
1209c02869 Fix overflow in mapPopulateSeries 2021-02-06 23:18:42 +03:00
Alexey Milovidov
d9d49a4d30 Fix UBSan report in arrayCumSum 2021-02-06 22:22:21 +03:00
Alexey Milovidov
65902f4c6e Fix UBSan report in arrayCumSum 2021-02-06 22:21:22 +03:00
Alexey Milovidov
fd899daa00 Fix UBSan report in arrayCumSum 2021-02-06 22:17:19 +03:00
Azat Khuzhin
c4b5eed4ff Fix DateTime64 initialization (to match DateTime behaviour) 
There was no specializations for toDateTime64(<numeric>), and because of
this default decimal conversion was used, however this is not enough for
DateTime/DateTime64 types, since the date may overflow and the proper
check is required (like DateTime has), and this what UBsan found [1]:

    ../src/IO/WriteHelpers.h:812:33: runtime error: index 508 out of bounds for type 'const char [201]' Received signal -3 Received signal Unknown signal (-3)

Backtrace:

    (gdb) bt
    0  LocalDateTime::LocalDateTime (this=0x7fffffff8418, year_=1970, month_=1 '\001', day_=1 '\001', hour_=2 '\002', minute_=0 '\000', second_=254 '\376') at LocalDateTime.h:83
    1  0x00000000138a5edb in DB::writeDateTimeText<(char)45, (char)58, (char)32, (char)46> (datetime64=..., scale=7, buf=..., date_lut=...) at WriteHelpers.h:852
    2  0x0000000019c379b4 in DB::DataTypeDateTime64::serializeText (this=0x7ffff5c4b0d8, column=..., row_num=0, ostr=..., settings=...) at DataTypeDateTime64.cpp:66
    3  0x0000000019d297e4 in DB::IDataType::serializeAsText (this=0x7ffff5c4b0d8, column=..., row_num=0, ostr=..., settings=...) at IDataType.cpp:387

  [1]: https://clickhouse-test-reports.s3.yandex.net/19527/cea8ae162ffbf92e5ed29304ab010704c5d611c8/fuzzer_ubsan/report.html#fail1

Also fix CAST for DateTime64
 2021-02-06 13:04:53 +03:00
Azat Khuzhin
0627ba0e36 Fix timezone argument for DateTime64 
It should be marked with always const, otherwise it will bail:

    Code: 44, e.displayText() = DB::Exception: Illegal column String of time zone argument of function, must be constant string: While processing toDateTime(-1, 1, 'GMT'), Stack trace (when copying this message, always include the lines below):
 2021-02-06 13:04:53 +03:00
Azat Khuzhin
f4c2048bf2 Fix toDateTime64(toDate()/toDateTime()) for DateTime64 
Maybe this is even does not worth it, but at least there was code that
assume that this should work - ToDateTime64Transform in
FunctionsConversion.h.
 2021-02-06 13:04:53 +03:00
Nikolai Kochetov
9d62842a33 Fix tests. 2021-02-05 22:52:06 +03:00
alesapin
f81a407cdd Fix if with tuple then/else arguments 2021-02-05 22:39:26 +03:00
Nikolai Kochetov
9869f70a0d Remove AddMissed step and transform. 2021-02-05 14:41:44 +03:00
alexey-milovidov
34151eea2c
Merge pull request #20067 from ClickHouse/mod-negative-type-promotion 
Add type promotion for modulo of division of negative number
 2021-02-05 09:31:46 +03:00
Nikolai Kochetov
d9aa1096cf Build actions dag to evaluate missing defaults. 2021-02-04 23:36:50 +03:00
Alexey Milovidov
85b5805c3c Fix tests 2021-02-04 19:36:31 +03:00
Alexey Milovidov
05c5c8ed80 Avoid UBSan report in pointInPolygon 2021-02-03 21:23:54 +03:00
Nikolai Kochetov
c39c965612
Merge pull request #19962 from keen-wolf/timezoneOffset 
Implement the timezoneOffset function
 2021-02-03 20:37:34 +03:00
Nikolai Kochetov
ebab5df236 Update ya.make 2021-02-03 12:13:44 +03:00
Maksim Kita
ba45cf32ab
Merge pull request #19858 from kitaisreal/reinterpret-as-updated-to-big-integers 
Function reinterpretAs updated to support big integers
 2021-02-03 12:12:13 +03:00
PHO
09c49d3842 Add function runningConcurrency() 
Given a series of beginning time and ending time of events, this function calculates concurrency of the events at each of the data point, that is, the beginning time.
 2021-02-03 14:34:44 +09:00
keenwolf
9beb7689cc add comments and update test cases. 2021-02-02 23:35:45 +08:00
Vitaly Baranov
9427d5d94b
Merge pull request #19805 from vitlibar/allow-getting-return-type-of-dictget-without-loading-dictionary 
Fix starting the server with tables having default expressions containing dictGet()
 2021-02-02 18:19:55 +03:00
alexey-milovidov
695e28079d
Merge pull request #19960 from ClickHouse/json-extract-inaccurate-conversion-from-double-to-float 
Allow inaccurate conversion from double to float in function JSONExtract beacuse the users want that
 2021-02-02 15:15:37 +03:00
alexey-milovidov
4d0973cf25
Merge pull request #19959 from depressed-pho/fix-segv-fromModifiedJulianDay 
Fix a segfault in fromModifiedJulianDay()
 2021-02-02 13:50:14 +03:00
Alexey Milovidov
078011ef2d Allow conversion from double to float in function JSONExtract beacuse the users want that 2021-02-02 07:16:47 +03:00
PHO
5f7a852b92 Fix a segfault in fromModifiedJulianDay() 
It was crashing when the argument type was Nullable(T) where T was any integral type other than Int32.
 2021-02-02 12:43:36 +09:00
Alexey Milovidov
9930bb0bf6 Fix UBSan report in geoHashesInBox 2021-02-02 06:37:24 +03:00
keenwolf
d251807ea4 add function timezoneOffset 2021-02-02 11:01:20 +08:00
Alexey Milovidov
57664bffe0 Amend 2021-02-01 10:11:41 +03:00
Alexey Milovidov
e9a61c4738 More correct 2021-02-01 07:52:18 +03:00
Alexey Milovidov
1c0c3b3aab Merge branch 'master' into intdiv-ubsan 2021-02-01 07:46:40 +03:00
alexey-milovidov
a6b0fdd3eb
Merge pull request #19862 from ClickHouse/ubsan-gather-utils 
Fix UBSan report in GatherUtils
 2021-02-01 03:43:43 +03:00
Maksim Kita
d2b8d9886f Fixed fuzzer crash 2021-01-31 22:25:53 +03:00
Maksim Kita
a4812aa97b Updated tests 2021-01-31 18:32:47 +03:00
alexey-milovidov
342baa4223
Merge pull request #19720 from ClickHouse/unicode-asan 
Fix unsufficient args check (trash code) in StringSearcher
 2021-01-31 15:43:05 +03:00
alexey-milovidov
b80ad9f9bc
Merge pull request #19859 from ClickHouse/date-time-add-ubsan 
Fix UBSan report in DateTimeAddInterval
 2021-01-31 14:09:04 +03:00
vdimir
000b51f95a
Merge pull request #19518 from bharatnc/ncb/toIPv6-parses-IPv4 
toIPv6 function parses IPv4 addresses
 2021-01-31 13:56:45 +03:00
Alexey Milovidov
ddff28e594 Fix UBSan report in intDiv 2021-01-31 13:23:50 +03:00
Alexey Milovidov
37f88a1468 Whitespace 2021-01-31 12:02:54 +03:00
Alexey Milovidov
e23dc60e31 Fix UBSan report in GatherUtils 2021-01-31 10:12:17 +03:00
Alexey Milovidov
248540e58f Fix UBSan report in GatherUtils 2021-01-31 10:10:37 +03:00
Alexey Milovidov
0128d185ea Fix UBSan report in DateTimeAddInterval 2021-01-31 06:49:33 +03:00
Alexey Milovidov
0fa5142715 Remove tons of garbage 2021-01-31 05:36:52 +03:00
Alexey Milovidov
5c718c8754 Merge branch 'master' into unicode-asan 2021-01-31 02:40:24 +03:00
alexey-milovidov
8ac39e7f56
Merge pull request #19791 from ClickHouse/format-readable-time-delta-ubsan 
formatReadableTimeDelta: correctly output infinite values
 2021-01-31 01:37:59 +03:00
bharatnc
e967ba5f3b try fix clang-11-darwin build 2021-01-30 12:12:38 -08:00
Maksim Kita
155f6c0eb5 Function reinterpretAs updated to support big integers 2021-01-30 22:56:23 +03:00
vdimir
5cb99ae699
Use offsets in FunctionIPv6StringToNum for copying ipv4 to buffer 2021-01-30 20:20:11 +03:00
Vitaly Baranov
7c8deae0e1 Fix starting the server with tables having columns' default expressions containing dictGet(). 
Allow getting return type of dictGet() without loading dictionary.
 2021-01-30 19:06:48 +03:00
Azat Khuzhin
502d981ab4 Use BinaryArithmeticOverloadResolver::valid_on_float_arguments for lcm/gcd 2021-01-30 09:12:31 +03:00
Azat Khuzhin
7da4083237 Forbid bit functions for floats 
This will fix the following UB report [1]:

  "../src/Functions/bitAnd.cpp:24:61: runtime error: nan is outside the
range of representable values of type 'long' Received signal -3 Received
signal Unknown signal (-3)"

  [1]: https://clickhouse-test-reports.s3.yandex.net/19824/89c4055202b9d08459f90ee5791d4e3017b82fbf/fuzzer_ubsan/report.html#fail1
 2021-01-30 09:12:31 +03:00
Alexey Milovidov
d44ce00391 Merge branch 'master' into format-readable-time-delta-ubsan 2021-01-30 07:28:38 +03:00
alexey-milovidov
276e972166
Merge pull request #19789 from ClickHouse/fix-great-circle-angle 
Fix mistake in greatCircleAngle
 2021-01-29 21:35:26 +03:00
alexey-milovidov
4cfcc18a7f
Merge pull request #19786 from ClickHouse/int-round-ubsan 
Fix UBSan report in "round"
 2021-01-29 14:59:12 +03:00
alexey-milovidov
0679d81c0d
Merge pull request #19788 from ClickHouse/array-enumerate-extended-bug 
Fix trivial bug in arrayEnumerateUniq
 2021-01-29 14:58:38 +03:00
Maksim Kita
c7401f219f
Merge pull request #18236 from kitaisreal/dictionaries-support-for-nullable-and-array 
[WIP] Dictionaries support for Array and Nullable
 2021-01-29 12:02:23 +03:00
alexey-milovidov
76894ccb9f
Update greatCircleDistance.cpp 2021-01-29 08:52:52 +03:00
Alexey Milovidov
f06bb78c5d Merge branch 'master' into unicode-asan 2021-01-29 05:18:29 +03:00