thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 16:42:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,473 Commits 9,168 Branches 1,781 Tags 2.1 GiB
6854bb46f5
Commit Graph

78473 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Azat Khuzhin
344298f403 Fix CAST from Nullable with cast_keep_nullable 
UBsan report [1]:

    SELECT toUInt32OrDefault(toNullable(0)) SETTINGS cast_keep_nullable = 1

    ../src/Common/assert_cast.h:50:12: runtime error: downcast of address 0x000029461170 which does not point to an object of type 'const DB::ColumnNullable'
    0x000029461170: note: object is of type 'DB::ColumnVector<unsigned int>'
     00 00 00 00  a0 51 3d 09 00 00 00 00  01 00 00 00 00 00 00 00  a0 3f 4e 29 00 00 00 00  a4 3f 4e 29
                  ^~~~~~~~~~~~~~~~~~~~~~~
                  vptr for 'DB::ColumnVector<unsigned int>'
        0 0x1e3e279d in DB::ColumnNullable const& assert_cast<DB::ColumnNullable const&, DB::IColumn const&>(DB::IColumn const&) obj-x86_64-linux-gnu/../src/Common/assert_cast.h:50:12
        1 0x1e3e279d in DB::ColumnNullable::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Columns/ColumnNullable.cpp:167:43
        2 0x12db4507 in DB::FunctionCastOrDefault::executeImpl() const (/src/ch/tmp/32019/clickhouse-ubsan+0x12db4507)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32019/9e1222a50b1805f696f4205e451365f8acdbc8a5/fuzzer_astfuzzerubsan,actions//report.html

And w/o -fsanitize=undefined:

    SELECT toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))) SETTINGS cast_keep_nullable = 1

    Received exception:
    Code: 12. DB::Exception: Parameters start = 0, length = 1 are out of bound in ColumnVector<T>::insertRangeFrom method (data.size() = 0).: While processing toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))). (PARAMETER_OUT_OF_BOUND)
 2021-12-03 02:02:42 +03:00
Anton Popov
f6be3d16fd
Merge pull request #24820 from kssenii/versioning 
Versioning of aggregate function states
 2021-12-03 01:41:44 +03:00
Azat Khuzhin
0b2de32228 Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset 
Even after timer_delete() the signal can be fired.
Reproducer:

    $ clickhouse-server & # with configured trace_log
    $ clickhouse-benchmark -c2 --query 'select * from numbers(1e6)' --query_profiler_cpu_time_period_ns=1 &
    ...
    2021.12.02 14:28:01.320288 [ 24885 ] {} <Debug> TCPHandler: Processed in 177.055205644 sec.
    User defined signal 2

CI failures:
- https://s3.amazonaws.com/clickhouse-test-reports/32067/8dbc7a8dae17090a18778f29629d8746a1bb9b72/stateful_tests__debug__actions_.html
- https://s3.amazonaws.com/clickhouse-test-reports/32064/c07450a7dce363b7a4c5ca3ab0e833c25e3d46c0/stateful_tests__debug__actions_.html

Fix this by do not reset the signal back, and introduce a flag to ignore
signals after disabling the timer.

Fixes: #31740
 2021-12-03 01:33:08 +03:00
Azat Khuzhin
96bd83c31e Do not reopen logs on USR1, HUP is enough 
USR1 is also used for query_profiler_real_time_period_ns, let's not
overlap.
 2021-12-03 01:32:57 +03:00
Azat Khuzhin
9276384977 clickhouse-test: do not use random generator with shared state 
Recently (#32094) test database had been overlapped, and random prefix
for database had been increased from 6 to 8.

But actually 6 bytes for random prefix should be enough (with existing
alphabet (0-9a-z) it is 36**6=2'176'782'336), and the real reason of
this overlap is that random generator by default uses shared state [1]:

    The functions supplied by this module are actually bound methods of
    a hidden instance of the random.Random class. You can instantiate your
    own instances of Random to get generators that don’t share state.

  [1]: https://docs.python.org/3/library/random.html

I've played a little bit with random in python, and using default random
generator it generates non-unique strings pretty fast, just in a few
runs, but using SystemRandom (that uses /dev/urandom) it takes ~1 minute.

Test:

```sh
    $ while /tmp/test.py | LANG=c sort -S5G | LANG=c uniq -d | tee /dev/stderr | wc -l | fgrep -q -x -c 0; do :; done
```

```python
    #!/usr/bin/env python3

    import multiprocessing
    import string
    import random

    def random_str(length=6):
        alphabet = string.ascii_lowercase + string.digits
        return ''.join(random.SystemRandom().choice(alphabet) for _ in range(length))

    def worker(_):
        print(random_str())

    with multiprocessing.Pool(processes=2) as pool:
        pool.map(worker, range(0, int(10e3)))
```

So let's switch to SystemRandom and use 6-byte prefix.
 2021-12-03 01:29:23 +03:00
Azat Khuzhin
bfaaa79cfc tests/ci: do not compress logs twice 
Cc: @alesapin
 2021-12-03 01:28:32 +03:00
Frank Chen
e895c85f49
Add exception/exception_code to trace span log (#32040) 2021-12-03 01:00:02 +03:00
kssenii
06312a2df9 Fix 2021-12-02 22:48:21 +03:00
mergify[bot]
e2cb91b560
Merge branch 'master' into add_one_more_group 2021-12-02 19:28:13 +00:00
alesapin
45021bd35c
Merge pull request #32121 from ClickHouse/add_assertion_in_keeper 
Add check for duplicate hostnames and IDs in KeeperConfig
 2021-12-02 22:24:46 +03:00
alesapin
308eadd83a
Merge pull request #32159 from ClickHouse/fix_ci 
Fix CI
 2021-12-02 22:04:48 +03:00
alesapin
f2140be90b Fix CI 2021-12-02 22:03:59 +03:00
tavplubix
4606376734
Update ReplicatedMergeTreeSink.cpp 2021-12-02 21:51:23 +03:00
tavplubix
4d96d6ab77
Merge pull request #32052 from stigsb/fix-flaky-create-table-like-test 
Fix flaky integration test for MaterializedMySQL CREATE TABLE LIKE
 2021-12-02 21:28:00 +03:00
Alexander Tokmakov
252192bc6d fix active replicas count in quorum inserts 2021-12-02 21:02:11 +03:00
alesapin
c14086d578
Merge pull request #32064 from ClickHouse/lightweight_checks_rerun 
Add ability for lightweight checks rerun
 2021-12-02 20:54:27 +03:00
alexey-milovidov
2a7a75ea88
Merge pull request #32156 from flickerbox/crb-update-featured-image 
Update featured image for 21.11 release blog post
 2021-12-02 20:35:48 +03:00
Cody Baker
be660ca25f Update featured image for 21.11 release blog post 2021-12-02 10:28:47 -07:00
Cody Baker
c9ae0aec32 Fix margin on form field 2021-12-02 10:02:41 -07:00
Cody Baker
72e47d9cc7 Update form field class 2021-12-02 10:01:54 -07:00
Cody Baker
8e72d6755d Merge branch 'master' into crb-support-page 
# Conflicts:
#	website/css/main.css
 2021-12-02 09:57:51 -07:00
Cody Baker
39a28d46f1 Add customer reference number field 2021-12-02 09:50:53 -07:00
alesapin
dd2ccd3b17
Merge pull request #32155 from ClickHouse/small_improvements 
Small improvements in lambda code
 2021-12-02 19:39:00 +03:00
alesapin
9e1437a7c4 Small improvements in lambda code 2021-12-02 19:38:18 +03:00
mergify[bot]
c399eaf945
Merge branch 'master' into add_one_more_group 2021-12-02 15:20:06 +00:00
Kruglov Pavel
afd74aa166
Update 02117_custom_separated_with_names_and_types.sh 2021-12-02 18:15:50 +03:00
kssenii
64fcbe347e Update 2021-12-02 16:08:11 +03:00
Nikolai Kochetov
7de34bb8e1
Merge pull request #32126 from ClickHouse/remove-02116_global_in_time_limit 
Remove 02116_global_in_time_limit.
 2021-12-02 16:07:18 +03:00
Nikolai Kochetov
4d10762727 Remove 02116_global_in_time_limit. 2021-12-02 16:05:06 +03:00
mergify[bot]
88e1a26e99
Merge branch 'master' into add_one_more_group 2021-12-02 12:56:57 +00:00
alesapin
b8bffcb14c
Merge pull request #32115 from ClickHouse/fix_flaky_keeper_tests 
Fix flaky keeper whitelist test
 2021-12-02 15:41:17 +03:00
tavplubix
1f4b14f68b
Merge pull request #32119 from ClickHouse/tavplubix-patch-4 
Fix data race in `removePartAndEnqueueFetch(...)`
 2021-12-02 15:25:38 +03:00
avogar
27e6e063d0 Change test 02117_custom_separated_with_names_and_types 2021-12-02 15:06:15 +03:00
Kruglov Pavel
049b2c0c14
Merge pull request #31072 from MaxWk/feature-support-bool-type 
Feature support bool type
 2021-12-02 14:50:29 +03:00
alesapin
437591f2fa Add check for duplicate hostnames and IDs in KeeperConfig 2021-12-02 14:46:33 +03:00
tavplubix
5400c5899d
Merge pull request #32063 from ClickHouse/fix_detach_dir_exists 
Fix 'directory exists' error when detaching part
 2021-12-02 14:04:21 +03:00
Anton Popov
a79dbb6b6a
Merge pull request #32079 from Avogar/parse-plus-in-float 
Allow plus before Float32/Float64
 2021-12-02 14:03:48 +03:00
alesapin
ff6cc14df3 Fix one more test 2021-12-02 13:41:04 +03:00
alesapin
7ef0737edc Move fuzzers and unit tests to another group 2021-12-02 13:23:32 +03:00
tavplubix
e9489ccd6f
Update StorageReplicatedMergeTree.cpp 2021-12-02 13:18:02 +03:00
Maksim Kita
2d359a5e79
Merge pull request #32110 from bharatnc/ncb/h3-func-looping 
make looping in H3 funcs uniform
 2021-12-02 13:15:36 +03:00
alesapin
e704140d07 Fix flaky whitelist test 2021-12-02 12:37:08 +03:00
Maksim Kita
a426ed0a5a
Merge pull request #31800 from kitaisreal/dict-get-with-type-nullable-fix 
Function dictGet with type Nullable fix
 2021-12-02 12:27:47 +03:00
alesapin
1cda5bfe4e
Merge pull request #32114 from ClickHouse/remove_pvs_from_master 
Remove PVS check from master
 2021-12-02 11:37:00 +03:00
alesapin
97e7da18bb Remove PVS check from master 2021-12-02 11:36:30 +03:00
Maksim Kita
9874931347
Merge pull request #32095 from FedericoCeratto/patch-2 
Fix syntax error: drop comma
 2021-12-02 11:20:52 +03:00
kssenii
5a273d4dda Update version 2021-12-02 08:02:07 +00:00
mergify[bot]
cd4a58c59d
Merge branch 'master' into patch-2 2021-12-02 08:01:05 +00:00
alesapin
d273a8db7c
Merge pull request #32094 from azat/clickhouse-test-longer-db 
Increase length of random database in clickhouse-test
 2021-12-02 10:58:40 +03:00
kssenii
8edf146687 Merge branch 'master' of github.com:ClickHouse/ClickHouse into versioning 2021-12-02 07:57:14 +00:00