Commit Graph

4566 Commits

Author SHA1 Message Date
Roman Vasin
1cce8c023d Refactor cutURLParameter to work directly on result column 2022-12-01 13:02:37 +00:00
Roman Vasin
620df1ec6f
Merge branch 'master' into ADQM-623 2022-12-01 15:30:36 +03:00
Roman Vasin
80598eebc9 FIx age() for case startdate > enddate 2022-12-01 12:20:20 +00:00
Vladimir C
a03fc9293c
Merge pull request #43274 from canhld94/ch_canh_fix_array_first_last
Close https://github.com/ClickHouse/ClickHouse/issues/43160
2022-12-01 13:14:48 +01:00
Roman Vasin
cea63dbca0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-626 2022-12-01 07:32:06 +00:00
taiyang-li
cae27d9536 fix code style 2022-12-01 11:01:14 +08:00
taiyang-li
f14582334c Add function concatWs 2022-12-01 10:56:53 +08:00
Roman Vasin
96d7cf319b
Merge branch 'master' into ADQM-623 2022-11-30 19:25:53 +03:00
Roman Vasin
48d808e475 Replace is_date_diff with is_relative 2022-11-30 16:22:30 +00:00
Roman Vasin
8160b286e2 Fix bug with week parameter for age() 2022-11-30 15:26:02 +00:00
Kruglov Pavel
3a0cf5bf5b
Merge pull request #43311 from Avogar/proper-fix-lc-with-if
Fix bad cast from LowCardinality column when using short circuit function execution
2022-11-30 16:06:43 +01:00
Igor Nikonov
3869a6c8f1
Merge pull request #43466 from zzsmdfj/issue_42742_fix_date_diff_wrong_result_build_by_darwin
Fix date_diff() for hour/minute on macOS
2022-11-30 12:23:53 +01:00
Yakov Olkhovskiy
77266ea754 cleanup 2022-11-29 17:34:16 +00:00
Roman Vasin
eff768cca7 Fix seconds in age(); make specific NOLINT; fix DATEDIFF in 02477_age test 2022-11-29 15:42:44 +00:00
Roman Vasin
1630a6fcb0 Remove function description 2022-11-29 12:58:57 +00:00
Antonio Andelic
a5c396ade9 Make better 2022-11-29 09:33:42 +00:00
Roman Vasin
d4aeb9342c Add check for array type 2022-11-29 07:53:16 +00:00
Nikolay Degterinsky
e6e6afe8f7
Merge pull request #43731 from amosbird/issue_43622
Fix issue #43622
2022-11-29 02:49:38 +01:00
Yakov Olkhovskiy
4d7d9cbd0b some comments and beautification 2022-11-29 01:07:25 +00:00
Alexey Milovidov
f5c8077819
Merge pull request #43539 from ClickHouse/substitute-udf-create
Substitute UDFs in `CREATE` queries
2022-11-28 17:05:05 +01:00
Roman Vasin
b7cac89a8f Add 02483_cuturlparameter_with_arrays test 2022-11-28 16:01:29 +00:00
Roman Vasin
75b2aaad64 Use {} in exception messages 2022-11-28 15:21:57 +00:00
Yakov Olkhovskiy
770b520ded
Merge branch 'master' into refactoring-ip-types 2022-11-28 08:50:19 -05:00
Igor Nikonov
f40f3949bd Use DateLUTImpl::Time instead of Int64 2022-11-28 13:42:22 +01:00
Igor Nikonov
b4d8aea731 Merge remote-tracking branch 'origin/master' into issue_42742_fix_date_diff_wrong_result_build_by_darwin 2022-11-28 11:34:43 +01:00
Roman Vasin
532e2c50d7 Fix bug with zeros after end of values 2022-11-28 09:53:34 +00:00
Robert Schulze
143637404e
Merge pull request #43078 from ClibMouse/s390x_ip_coding
s390x ip coding functions fixes
2022-11-28 09:11:42 +01:00
Amos Bird
65ce5abb63
Fix suspicious lowcard key 2022-11-27 18:40:26 +08:00
Duc Canh Le
9585ae4aba
Merge branch 'master' into ch_canh_fix_array_first_last 2022-11-26 17:40:38 +08:00
Nikita Taranov
d1c258cf20
Add xxh3 hash function (#43411)
* impl

* try fix

* add docs

* add test

* rm unused file

* excellent
2022-11-26 00:14:08 +01:00
Alexey Milovidov
dc7742e8f9
Merge pull request #43596 from nickitat/issue_43510
Fix monotonicity check for `toString(enum)`
2022-11-25 22:15:38 +01:00
Suzy Wang
d57a8b502d s390x crc32 fix 2022-11-25 13:12:11 -08:00
Yakov Olkhovskiy
5a86a46165 add description to dictionary functions 2022-11-25 20:34:34 +00:00
Vladimir C
ff5bb69584
Merge branch 'master' into zvonand-decdiv 2022-11-25 17:05:22 +01:00
Yakov Olkhovskiy
19a01b77eb refactoring on settings for casting, dirty hack for inaccessable context in cast function (will require review), docs appended 2022-11-25 15:57:49 +00:00
Roman Vasin
d0382270e5 Make array working on tables data correctly 2022-11-25 13:44:17 +00:00
Antonio Andelic
9f0fdd780b Modify ALTER before replicating 2022-11-25 10:16:20 +00:00
Roman Vasin
7bbb32f80a Refactor code by implementation of cuURL() 2022-11-25 08:25:48 +00:00
Roman Vasin
7084fbfee3 Make iteration through array of values 2022-11-25 06:51:08 +00:00
Yakov Olkhovskiy
08456a2421 special treatment for FixedString(16) as a binary representation for IPv6 in casting to better match previous behavior, some cleanup, test fixed 2022-11-24 05:00:28 +00:00
zvonand
e8d4176f5b fix ubsan(2) 2022-11-24 03:38:16 +03:00
zvonand
d42a29ac0f fix ubsan + expand docs 2022-11-24 01:23:59 +03:00
Nikita Taranov
dc9be92733 impl 2022-11-23 23:08:14 +01:00
Antonio Andelic
c52ca6e10e Merge branch 'master' into substitute-udf-create 2022-11-23 16:08:37 +00:00
Antonio Andelic
71fb4dc889 Fix visitor 2022-11-23 16:07:25 +00:00
Antonio Andelic
981e8a3a27 Modifiy visitor 2022-11-23 13:32:36 +00:00
Antonio Andelic
e84e05d849 Modify visitor 2022-11-23 10:27:38 +00:00
Duc Canh Le
769cbc58ea
Merge branch 'master' into ch_canh_fix_array_first_last 2022-11-23 14:33:35 +08:00
Yakov Olkhovskiy
2cbe748e68 functions fixed, test fixed 2022-11-22 21:34:32 +00:00
zvonand
92100c9aa4 fix typos 2022-11-22 18:01:54 +03:00
zvonand
faef0699b6 fix issues 2022-11-22 17:30:44 +03:00
Kruglov Pavel
98d6b96c82
Merge pull request #42033 from mark-polokhov/BSONEachRow
Add BSONEachRow input/output format
2022-11-22 14:45:21 +01:00
Kruglov Pavel
62bfb5547c
Merge pull request #43368 from Avogar/fix-if-nothing
Fix possible `Cannot create non-empty column with type Nothing` in functions if/multiIf
2022-11-22 14:11:13 +01:00
Yakov Olkhovskiy
988b030bfd bug fix, refactoring IPv6 to struct with overridden comparison 2022-11-22 12:57:19 +00:00
Roman Vasin
1e10d022ce Refactor FunctionCutURLParameter is based directly on IFunction 2022-11-22 11:27:52 +00:00
zzsmdfj
21f2cffa4d fix date_diff wrong result build by darwin 2022-11-22 18:58:37 +08:00
Anton Popov
5532e3db95 Merge branch 'filesystem-function' of https://github.com/ucasfl/ClickHouse into merging-42064 2022-11-21 18:23:11 +00:00
Yakov Olkhovskiy
ee3ce9de70 bugs fixed, some optimization, cleanup, test fixed 2022-11-21 07:45:42 +00:00
Duc Canh Le
fa70e335db minor fix 2022-11-21 10:53:54 +08:00
Duc Canh Le
0ce9a94c62
Merge branch 'master' into ch_canh_fix_array_first_last 2022-11-21 10:50:37 +08:00
Yakov Olkhovskiy
fc7f4baf23 fixed IPv6CIDRToRange, fixed comparison for ColumnIPv6 2022-11-19 06:55:05 +00:00
Yakov Olkhovskiy
dbaeabcf38 fixed some bugs, some functions corrected, some tests corrected 2022-11-18 20:10:27 +00:00
avogar
47959d6a70 Allow arguments with type Nothing in functions if/multiIf 2022-11-18 14:56:06 +00:00
Suzy Wang
8dc99e8d64
Merge branch 'master' into s390x_ip_coding 2022-11-17 23:35:06 -05:00
Nikolay Degterinsky
e7e21adf17
Merge pull request #43207 from freedomDR/improve_unhex
unhex support FixedString arguments
2022-11-18 03:09:58 +01:00
Alexey Milovidov
ef1278f5c4
Merge pull request #43313 from ClickHouse/pmod
`pmod`: compatibility with Spark, better documentation
2022-11-17 17:13:45 +01:00
Kruglov Pavel
1b68f605a2
Merge pull request #42761 from AlfVII/fix-slow-json-extract-with-low-cardinality
Fixed slowness in JSONExtract with LowCardinality(String) tuples
2022-11-17 12:49:18 +01:00
Nikita Taranov
11b535d443
impl (#43283) 2022-11-17 11:17:27 +01:00
Alexey Milovidov
10d7ce9815 pmod: compatibility with Spark, better documentation 2022-11-17 05:41:03 +01:00
Raevsky Rudolf
b8e227de81
Merge branch 'master' into typo-third 2022-11-17 00:37:30 +01:00
Raevsky Rudolf
318064ccce
Fixed: Typo 2022-11-17 00:25:57 +01:00
avogar
e98ac04678 Fix bad cast from LowCardinality column when using short circuit function execution 2022-11-16 22:34:24 +00:00
flynn
9f8dcfca34
Merge branch 'master' into filesystem-function 2022-11-16 23:28:10 +08:00
Duc Canh Le
84aea78d0f fix array first / last or null with nullable column 2022-11-16 13:11:05 +08:00
李扬
6393d11dab
fix doc 2022-11-16 09:34:45 +08:00
Yakov Olkhovskiy
813cb7fb0d merge master 2022-11-15 22:46:05 +00:00
Suzy Wang
97d76771a4 update format 2022-11-15 08:53:39 -08:00
Yakov Olkhovskiy
324c3bb81e fix bin, hex 2022-11-15 04:57:08 +00:00
freedomDR
5753be7b99 remove useless head file 2022-11-15 12:33:14 +08:00
freedomDR
7fdb2f400e unhex support FixedString 2022-11-15 12:33:14 +08:00
taiyang-li
7178d80c51 Merge branch 'function_factorial' of https://github.com/bigo-sg/ClickHouse into function_factorial 2022-11-15 09:46:44 +08:00
taiyang-li
5eba20e119 modify return type from Int64 to UInt64 2022-11-15 09:46:24 +08:00
李扬
10449247fd
Merge branch 'master' into function_factorial 2022-11-15 09:34:43 +08:00
李扬
1de5bb2392
Add function canonicalRand (#43124)
* add function canonicalRand

* add perf test

* revert rand.xml
2022-11-15 00:27:19 +01:00
李扬
ef45889ecb
Add function pmod which return non-negative result based on modulo (#42755)
* add function positive_modulo

* add ducument

* fix type deduction of positive_modulo

* add function positive_modulo

* add ducument

* fix type deduction of positive_modulo

* add notice

* fix typo

* fix typo

* fix bug

* fix ub error

* fix ub error
2022-11-15 00:15:58 +01:00
HarryLeeIBM
0133444433 Fix byte order issue of wide integer for s390x 2022-11-14 10:47:32 -08:00
zvonand
0b780346cd fix typos 2022-11-14 19:26:15 +03:00
zvonand
0a7e4ff3bf fix asan 2022-11-14 17:52:17 +03:00
Yakov Olkhovskiy
9aeebf3bdf
Merge branch 'master' into refactoring-ip-types 2022-11-14 09:21:54 -05:00
Yakov Olkhovskiy
4d144be39c replace domain IP types (IPv4, IPv6) with native 2022-11-14 14:17:17 +00:00
AlfVII
3768641bf7
Merge branch 'master' into fix-slow-json-extract-with-low-cardinality 2022-11-14 15:10:03 +01:00
Kruglov Pavel
48e6d39055
Merge pull request #43178 from Avogar/fix-if
Fix function if in case of NULL and const Nullable arguments
2022-11-14 14:52:30 +01:00
Alfonso Martinez
769cef9458 Replaced str with chars directly 2022-11-14 11:51:50 +01:00
Roman Vasin
ee0baa679b Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-623 2022-11-14 09:45:32 +00:00
zvonand
4392d04837 fix zero result 2022-11-14 12:27:31 +03:00
李扬
bf88816da4
Merge branch 'master' into function_factorial 2022-11-14 10:30:41 +08:00
zvonand
2390d59a2b fix memory access(2) 2022-11-14 03:07:21 +03:00
zvonand
e83deedb25 fix nullptr 2022-11-13 23:07:52 +03:00
zvonand
03bdb93dd9 fix fuzzer(3) 2022-11-13 13:34:33 +03:00
Alexander Gololobov
661c5cd0ae
fixed type 2022-11-12 23:58:09 +01:00
zvonand
1ed5a6c78d fix fuzzer(2) 2022-11-13 00:27:14 +03:00
Joanna Hulboj
42de468665 Fixed unexpected behaviour of Interval types with subquery and casting 2022-11-12 19:47:06 +00:00
zvonand
fce1573320 ast fix 2022-11-12 02:07:25 +03:00
zvonand
007e883c2b one more fix 2022-11-12 01:52:36 +03:00
zvonand
faf5185b22 one more fix 2022-11-11 23:59:03 +03:00
Nikolai Kochetov
505af344c4
Merge pull request #42612 from FArthur-cmd/UTC_timestamp
Add functions for PowerBI connect
2022-11-11 17:27:29 +01:00
zvonand
142c3d438c fix 2022-11-11 18:36:11 +03:00
zvonand
8f5e0a7e7a small datatype update 2022-11-11 17:54:41 +03:00
avogar
5ec748c664 Fix function if in case of NULL and const Nullable arguments 2022-11-11 14:49:49 +00:00
Alfonso Martinez
f24991fa56 Remove pushing a last 0 in FixedStrings 2022-11-11 13:02:30 +01:00
Alfonso Martinez
5d53ec9da8 Cleaned and replace some Strings by using chars directly 2022-11-11 11:07:05 +01:00
Roman Vasin
1f2f457e56 Fix style in function doc 2022-11-11 09:41:55 +00:00
Roman Vasin
67e66e40c3 Fix DateTime64 scale problem in age() 2022-11-11 09:25:08 +00:00
Roman Vasin
35a1c406ad Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-623 2022-11-11 07:35:44 +00:00
Roman Vasin
61b0c9d77c Add documentation of age() into code 2022-11-11 07:34:53 +00:00
avogar
9e89af28c6 Refactor BSONEachRow format, fix bugs, support more data types, support parallel parsing and schema inference 2022-11-10 20:15:14 +00:00
Alfonso Martinez
92028756f7 Merge remote-tracking branch 'upstream/master' into fix-slow-json-extract-with-low-cardinality 2022-11-10 17:22:58 +01:00
taiyang-li
f58e960a7d update as request 2022-11-10 21:07:44 +08:00
Roman Vasin
4962df7efb Fix quarter 2022-11-10 11:37:57 +00:00
Roman Vasin
8f3a7b5024 Change algorithm into more accurate 2022-11-10 10:27:18 +00:00
Alfonso Martinez
a1fd275209 Fixed duplicated includes 2022-11-10 09:13:52 +01:00
taiyang-li
d76c271242 add missed file 2022-11-10 16:08:22 +08:00
taiyang-li
628019d9d9 Merge branch 'enable_max_splits' of https://github.com/bigo-sg/ClickHouse into enable_max_splits 2022-11-10 10:47:10 +08:00
taiyang-li
c6449a1b23 fix bug 2022-11-10 10:46:51 +08:00
Suzy Wang
6ae4843b6d Update fixes as suggested 2022-11-09 13:54:32 -08:00
zvonand
90e3aed957 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-decdiv 2022-11-09 15:05:31 +03:00
李扬
9b2dd49b4a
Merge branch 'master' into enable_max_splits 2022-11-09 15:31:06 +08:00
Suzy Wang
90250ea6ec s390x ip coding functions fixes 2022-11-08 21:12:47 -08:00
Roman Vasin
44e033680e Refactor age() function 2022-11-08 16:39:40 +00:00
Nikolay Degterinsky
09aca50c3b
Merge pull request #42195 from evillique/tuple-of-intervals
Better INTERVAL parsing and execution
2022-11-08 16:21:39 +01:00
Robert Schulze
2c568df714
Merge pull request #42968 from ClickHouse/bitcast
Less use of CH-specific bit_cast()
2022-11-08 10:54:10 +01:00
Vladimir C
c4eb83408f
Merge pull request #37681 from xlwh/display_name
closes undefined
2022-11-08 10:53:58 +01:00
李扬
1728f1e436
Merge branch 'master' into enable_max_splits 2022-11-08 10:15:42 +08:00
Nikolay Degterinsky
2b286222f7
Merge branch 'master' into tuple-of-intervals 2022-11-08 00:15:35 +01:00
Roman Vasin
1d8e7728e1 Add age() function 2022-11-07 16:42:46 +00:00
Robert Schulze
58a12300ef
Update src/Functions/FunctionsStringSearch.h
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
2022-11-07 16:05:48 +00:00
Robert Schulze
a8098db25d
Minor cleanups
Semantics are unchanged.

Some special case handling was changed to early-out, because of that the
indentation changed but the logic is the same as before.
2022-11-07 14:03:46 +00:00
Vladimir C
0481999e35 Fix displayName test, add doc 2022-11-07 14:34:47 +01:00
xlwh
500609ccd3 Add miscellaneous function displayName() 2022-11-07 14:34:47 +01:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
李扬
70fd664dd1
Merge branch 'master' into ascii_function 2022-11-07 10:30:10 +08:00
Nikolay Degterinsky
fbf891d375
Merge pull request #42774 from alexon1234/add_format_redable_decimal_size
Published formatReadableDecimalSize so it can be used as function
2022-11-05 12:34:20 +03:00
Nikolay Degterinsky
3bcf151a9a Merge remote-tracking branch 'upstream/master' into tuple-of-intervals 2022-11-04 23:09:43 +00:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast()
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
2022-11-04 15:52:48 +00:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs
Mask some information in logs
2022-11-04 14:09:38 +01:00
Nikolay Degterinsky
16fc05b734
Merge branch 'master' into add_format_redable_decimal_size 2022-11-04 15:02:01 +03:00
Robert Schulze
6df8f7ef25
Merge pull request #42907 from ClickHouse/cleanup-regexpreplace
Cleanup implementation of regexpReplace(All|One)
2022-11-04 12:49:44 +01:00
Filatenkov Artur
7afc714b06
Merge branch 'master' into UTC_timestamp 2022-11-04 14:32:44 +03:00
Kruglov Pavel
e9a01a1946
Merge branch 'master' into fix-slow-json-extract-with-low-cardinality 2022-11-04 11:13:46 +01:00
李扬
fccf6c037f
Merge branch 'master' into ascii_function 2022-11-03 22:48:55 -05:00
李扬
e65b0f1c2e
Merge branch 'master' into enable_max_splits 2022-11-03 22:47:15 -05:00
zvonand
a7ab336af2 fixed types 2022-11-03 16:55:14 +03:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines 2022-11-03 13:22:06 +01:00
taiyang-li
f39cb41d3e change as requested 2022-11-03 16:44:57 +08:00
taiyang-li
0df7e95845 improve doc and uts 2022-11-03 16:12:19 +08:00
Alejandro
a1e158993d Added docs in the function 2022-11-03 09:04:22 +01:00
Alejandro
16bf5458f5 Published formatReadableDecimalSize 2022-11-03 09:04:22 +01:00
taiyang-li
a51a1b4394 rename alphaTokens to SplitByAlphaImpl 2022-11-03 15:06:58 +08:00
taiyang-li
5fa0968bd5 reset to original solution 2022-11-03 15:05:23 +08:00
taiyang-li
e78861ad1c Merge branch 'master' into enable_max_splits 2022-11-03 14:41:52 +08:00
taiyang-li
6770723027 modify as requested 2022-11-03 12:02:31 +08:00
taiyang-li
aeea21e4d7 Merge branch 'master' into ascii_function 2022-11-03 11:13:20 +08:00
Robert Schulze
494d6ca3df
Cleanup implementation of regexpReplace(All|One)
This is a pure refactoring, there are no semantic changes.

Cherry-picked from #42682.
2022-11-02 15:49:53 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
ltrk2
14141c9071 Add support for FixedString input to base64 functions 2022-11-02 08:14:28 -04:00
FArthur-cmd
84c88ddc91 fix typos 2022-11-01 17:29:15 +00:00
FArthur-cmd
34a33d1821 add documentation for functions 2022-11-01 15:09:05 +00:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14
Support `Object` type inside other types
2022-11-01 15:20:02 +01:00
Kruglov Pavel
c3a7211c1c
Merge pull request #41502 from marcioapm/coercing-json-extract
Add coercing in JSONExtract
2022-11-01 14:52:41 +01:00
Alexander Gololobov
f67f0183f6
Merge pull request #42812 from azat/utf8-fix-followup
Fix lowerUTF8()/upperUTF8() in case of symbol was in between 16-byte boundary
2022-11-01 12:01:00 +01:00
zvonand
2f1e4bbad7 added documentation 2022-11-01 13:21:17 +03:00
zvonand
83fb05e5ca Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-decdiv 2022-11-01 09:03:29 +03:00
Zhiguo Zhou
d18d08bcc7
Remove short-circuit evaluation in AssociativeApplierImpl::apply (#42214)
The short-circuit evaluation was implemented when applying the
saturable operators (and, or) on a vector of ColumnUInt8. However,
its control flow would be compiled as a series of conditional
branch instructions which are hard to predict by the hardware and
at the same time hinder the vectorization optimization by the
compiler. This commit removes the short-circuit and evaluates the
whole expression.
2022-10-31 21:36:15 +01:00
zvonand
46f1847dd9 fix segfaults invalid mem access, add tests 2022-10-31 23:18:08 +03:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Márcio Martins
399d024c2c Fix array index argument type check for JSONExtract* functions 2022-10-31 15:08:34 +01:00
Márcio Martins
c33bd7794f
Merge branch 'master' into coercing-json-extract 2022-10-31 15:04:48 +01:00
FArthur-cmd
81c1576e61 add tests 2022-10-31 13:03:13 +00:00
Vitaly Baranov
a30bfada63 Wipe passwords from backup logs too. 2022-10-31 10:50:33 +01:00
Alfonso Martinez
9e33b13737 Merge remote-tracking branch 'upstream/master' into fix-slow-json-extract-with-low-cardinality 2022-10-31 08:46:55 +01:00
Azat Khuzhin
5fe44f2736 Fix lowerUTF8()/upperUTF8() in case of symbol was in between 16-byte boundary
In lowerUTF8()/upperUTF8() there is an SSE optimization that handles
16 byte at a time, but only for ASCII, for UTF8 symbols converion will
be done by symbol.

Consider the following example:

    КВ АМ И СЖ
             ^ - offset is 15, length of sequence is 2
                 so first byte of a symbol is in first 16 bytes
                 second byte of a symbol is not ther

And in this case it will be handled incorrectly because it will try to
process oly these 16 bytes w/o looking forward.

This had been broken by #41286, before this patch it does not looks at
the row boundaries but only at the string end and so this sutation
wasn't possible.

Fixes: #42756
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-31 08:17:38 +01:00
Azat Khuzhin
32febf5155 Remove dead code in LowerUpperUTF8Impl::array()
AFAICS it was there before since it was possible to overrun the
expected_end, since utf8.convert() was called with "src_end - src" not
"expected_end - src".

Refs: 5a21f3908b054a0efc90c65a12fbe151c74d90dc:dbms/include/DB/Functions/FunctionsString.h
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-31 08:00:29 +01:00
Roman Vasin
24a4f4afe7
Merge branch 'master' into ADQM-605 2022-10-30 18:03:07 +03:00
FArthur-cmd
f5417da46b merge 2022-10-29 08:33:05 +00:00
FArthur-cmd
6cbf50a829 extract common part 2022-10-28 21:06:32 +00:00
avogar
8e13d1f1ec Improve and refactor Kafka/StorageMQ/NATS and data formats 2022-10-28 16:41:10 +00:00
Roman Vasin
5755728b33 Add Date32 and DateTime64 into exception messages 2022-10-28 13:11:09 +00:00
Roman Vasin
15f3f56812 Change 1 argument of function to first argument of function etc 2022-10-28 12:44:47 +00:00
zvonand
9e9222862d fixed division 2022-10-28 15:31:03 +03:00
Alfonso Martinez
c37b154254 Added reverted files and fixes for failing fuzzer tests 2022-10-28 12:37:59 +02:00
Vladimir C
31e8f92cd9
Merge pull request #42664 from ClickHouse/vdimir/followup-42274 2022-10-27 12:20:46 +02:00
Márcio Martins
1ca230ff6d Allow coercing in JSONExtract* functions 2022-10-27 10:26:55 +02:00
Roman Vasin
09fe9c3ed1 Use {} in exceptions 2022-10-26 16:07:56 +00:00
李扬
04f8f38580
Merge branch 'master' into ascii_function 2022-10-26 07:29:49 -05:00
Nikolai Kochetov
db422fa903
Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity
Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.
2022-10-26 14:11:23 +02:00
vdimir
14d0f6457b
Add tests and doc for some url-related functions 2022-10-26 10:52:57 +00:00
李扬
c7e5eb756b
Update src/Functions/ascii.cpp
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
2022-10-26 18:18:15 +08:00
李扬
48c37c52e6
Update src/Functions/ascii.cpp
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
2022-10-26 18:18:04 +08:00
Alfonso Martinez
348e153455 Merge branch 'fix-corner-case-low-cardinality-string-from-json' into fix-slow-json-extract-with-low-cardinality 2022-10-26 11:53:12 +02:00
Alfonso Martinez
241049763b Fixed failing tests 2022-10-26 11:50:45 +02:00
taiyang-li
05b77b3e34 fix comment 2022-10-26 16:25:25 +08:00
taiyang-li
fcbc217a7d enable limits for functions using FunctionTokens 2022-10-26 16:18:32 +08:00
taiyang-li
add5360a1b implement function ascii 2022-10-26 03:02:00 -05:00
Roman Vasin
3eca9ada3b Add Date32 to formatDateTIme 2022-10-26 08:00:12 +00:00
Alexey Milovidov
2b4be633ff
Merge branch 'master' into revert-41726-fix-slow-json-extract-with-low-cardinality 2022-10-25 22:57:12 +02:00
ltrk2
2c902bbc65
Implement support for different UUID binary formats (#42108)
* Implement support for different UUID binary formats

* Declare error codes that the implementation uses

* Make single-argument constructor explicit

* Modernize parts of the solution

* Make improvements based on review comments

* Declare an error code being used
2022-10-25 21:57:50 +02:00
Alfonso Martinez
8ef07bac89 Failing case corrected and test added 2022-10-25 17:40:54 +02:00
Anton Popov
e2ff7aa261
Merge pull request #42623 from CurtizJ/fix-bad-cast
Fix bad cast in `arrayElement` with `Map` with `Nullable` values and `Nullable` index
2022-10-25 14:59:52 +02:00
Vladimir C
8fabe1515c
Merge pull request #42274 from dentiscalprum/fix_domain 2022-10-25 13:56:58 +02:00
Nikita Taranov
49f6692a2e
Adapt internal data structures to 512-bit era (#42564)
* impl

* update tests

* fix tests
2022-10-25 13:56:28 +02:00
Maksim Kita
06fe6f3c8b
Merge pull request #31796 from kitaisreal/identifier-resolver
Added Analyzer
2022-10-25 12:36:08 +03:00
Alexey Milovidov
90c62fe46e
Merge branch 'master' into revert-41726-fix-slow-json-extract-with-low-cardinality 2022-10-25 09:31:46 +02:00
Robert Schulze
efd47c9eff
Merge pull request #42476 from ClickHouse/add-datetime-typedefs
Add convenience typedefs for Date/Date32/DateTime/DateTime64 columns
2022-10-25 08:45:57 +02:00
Alexey Milovidov
c48127534b
Merge pull request #41753 from pkit/morton_coding
add Morton Coding (ZCurve)
2022-10-25 06:10:37 +02:00
Anton Popov
eed21ad4ca
Revert "Low cardinality cases moved to the function for its corresponding type" 2022-10-25 01:30:32 +02:00
Anton Popov
b58ca49f05 fix bad cast in arrayElement with Map and Nullable index 2022-10-24 17:34:46 +00:00
Robert Schulze
655233e1c5
Add convenience typedefs for Date/Date32/DateTime/DateTime64 columns 2022-10-24 16:55:41 +00:00
Sergei Trifonov
bef3c71453
Merge pull request #41726 from AlfVII/fix-slow-json-extract-with-low-cardinality
Low cardinality cases moved to the function for its corresponding type
2022-10-24 18:18:40 +02:00
Nikolai Kochetov
14e3bb6354 Another case. 2022-10-24 13:58:26 +00:00
FArthur-cmd
e0c87b2ae2 small fix 2022-10-24 12:53:10 +00:00
FArthur-cmd
93d48a2403 add functions for powerBI connection 2022-10-24 11:04:45 +00:00
Maksim Kita
fe84e871f8 Fixed build 2022-10-24 10:22:20 +02:00
Maksim Kita
504cb2b38f Added GROUPING function support 2022-10-24 10:22:20 +02:00
Maksim Kita
75885ce2e1 Added Analyzer 2022-10-24 10:22:18 +02:00
Nikolay Degterinsky
6242e93c81 Fixes & better tests 2022-10-24 02:11:08 +00:00
Azat Khuzhin
bcc8d9f9d9 Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
* upstream/master:
  Fix strange code in date monotonicity
  Add a test
  Fix error
  Add tests
  Remove trash
2022-10-23 07:45:10 +02:00
Quanfa Fu
b07f65343d Add functions: domainRFC, topLevelDomainRFC, domainWithoutWWWRFC... 2022-10-23 12:01:26 +08:00
Alexey Milovidov
ebf330ca1a
Merge pull request #42573 from ClickHouse/remove-trash-6
Remove trash
2022-10-23 01:31:10 +02:00
Alexey Milovidov
6c69bdd34d
Merge pull request #42574 from ClickHouse/strange-date-monotonicity
Fix strange code in date monotonicity
2022-10-23 01:30:52 +02:00
Azat Khuzhin
a63dee2763 Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 2022-10-22 23:32:15 +02:00
Robert Schulze
b746224217
Merge pull request #42554 from arenadata/ADQM-604
Add support of Date32 arguments for dateName function
2022-10-22 17:20:10 +02:00
Alexey Milovidov
7bf1f4cdfa Fix strange code in date monotonicity 2022-10-22 09:02:20 +02:00
Azat Khuzhin
dbb4f8216f Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
Conflicts:
- src/Functions/GregorianDate.h
2022-10-22 08:20:05 +02:00
Alexey Milovidov
7fc3cd7fb0 Fix error 2022-10-22 07:38:22 +02:00
Alexey Milovidov
ef4b2b3f59 Remove trash 2022-10-22 07:25:37 +02:00
Nikolay Degterinsky
3f37636654 Merge branch 'master' into tuple-of-intervals 2022-10-22 02:44:35 +00:00
Alexey Milovidov
5fd7409ce1
Merge branch 'master' into morton_coding 2022-10-22 04:39:13 +02:00
Alexey Milovidov
acc96a5ace
Merge pull request #42464 from ClickHouse/julian_day_ubsan
Fix UBSan report in Julian Day functions
2022-10-22 04:15:12 +02:00
Azat Khuzhin
4eeb7337ed Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
Conflicts:
- src/Common/intExp.h
- src/Functions/DateTimeTransforms.h
- src/Functions/FunctionsConversion.h
2022-10-21 22:51:56 +02:00
Azat Khuzhin
905a95e166 Review fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 22:40:13 +02:00
Alexey Milovidov
592fbb3eaf
Merge pull request #42367 from ClickHouse/revert-40217
Revert #40217
2022-10-21 21:56:15 +02:00
Alexey Milovidov
3beace607b
Merge pull request #42465 from ClickHouse/decimal-asan
Fix buffer overflow in Decimal scale
2022-10-21 21:55:20 +02:00
Azat Khuzhin
3ccd5e8a18 Fix -Wshorten-64-to-32 with USE_BLAKE3
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:42 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Robert Schulze
2d448c139c
Merge pull request #42308 from arenadata/ADQM-584
Add support of Date32 arguments for date_diff function
2022-10-21 13:18:44 +02:00
Antonio Andelic
f3016fb86b Throw exception on invalid values 2022-10-21 09:15:24 +00:00
Alexey Milovidov
a7d2fd2cc5
Merge branch 'master' into decimal-asan 2022-10-21 01:28:34 +02:00
Alexey Milovidov
1a6fa07587 Fix clang-tidy 2022-10-21 00:08:17 +02:00
Robert Schulze
9b7294ef45
Merge remote-tracking branch 'origin/master' into revert-40217 2022-10-20 15:27:31 +00:00
Nikita Mikhaylov
9a73eb2fbb
Add functions to generate random values according to the distribution (#42411) 2022-10-20 17:25:28 +02:00