Commit Graph

78483 Commits

Author SHA1 Message Date
Kseniia Sumarokova
a8dcbc988a
Merge pull request #32129 from kssenii/fix-postgres-tests
fixing postgres tests
2021-12-03 09:15:20 +03:00
alexey-milovidov
b7fb21e6f0
Merge pull request #32163 from azat/clickhouse-test-random
clickhouse-test: do not use random generator with shared state
2021-12-03 06:53:57 +03:00
alexey-milovidov
17fcf0a6d3
Merge pull request #32165 from azat/query-profiler-reset-fix
Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset
2021-12-03 06:51:21 +03:00
Azat Khuzhin
344298f403 Fix CAST from Nullable with cast_keep_nullable
UBsan report [1]:

    SELECT toUInt32OrDefault(toNullable(0)) SETTINGS cast_keep_nullable = 1

    ../src/Common/assert_cast.h:50:12: runtime error: downcast of address 0x000029461170 which does not point to an object of type 'const DB::ColumnNullable'
    0x000029461170: note: object is of type 'DB::ColumnVector<unsigned int>'
     00 00 00 00  a0 51 3d 09 00 00 00 00  01 00 00 00 00 00 00 00  a0 3f 4e 29 00 00 00 00  a4 3f 4e 29
                  ^~~~~~~~~~~~~~~~~~~~~~~
                  vptr for 'DB::ColumnVector<unsigned int>'
        0 0x1e3e279d in DB::ColumnNullable const& assert_cast<DB::ColumnNullable const&, DB::IColumn const&>(DB::IColumn const&) obj-x86_64-linux-gnu/../src/Common/assert_cast.h:50:12
        1 0x1e3e279d in DB::ColumnNullable::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Columns/ColumnNullable.cpp:167:43
        2 0x12db4507 in DB::FunctionCastOrDefault::executeImpl() const (/src/ch/tmp/32019/clickhouse-ubsan+0x12db4507)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32019/9e1222a50b1805f696f4205e451365f8acdbc8a5/fuzzer_astfuzzerubsan,actions//report.html

And w/o -fsanitize=undefined:

    SELECT toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))) SETTINGS cast_keep_nullable = 1

    Received exception:
    Code: 12. DB::Exception: Parameters start = 0, length = 1 are out of bound in ColumnVector<T>::insertRangeFrom method (data.size() = 0).: While processing toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))). (PARAMETER_OUT_OF_BOUND)
2021-12-03 02:02:42 +03:00
Anton Popov
f6be3d16fd
Merge pull request #24820 from kssenii/versioning
Versioning of aggregate function states
2021-12-03 01:41:44 +03:00
Azat Khuzhin
0b2de32228 Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset
Even after timer_delete() the signal can be fired.
Reproducer:

    $ clickhouse-server & # with configured trace_log
    $ clickhouse-benchmark -c2 --query 'select * from numbers(1e6)' --query_profiler_cpu_time_period_ns=1 &
    ...
    2021.12.02 14:28:01.320288 [ 24885 ] {} <Debug> TCPHandler: Processed in 177.055205644 sec.
    User defined signal 2

CI failures:
- https://s3.amazonaws.com/clickhouse-test-reports/32067/8dbc7a8dae17090a18778f29629d8746a1bb9b72/stateful_tests__debug__actions_.html
- https://s3.amazonaws.com/clickhouse-test-reports/32064/c07450a7dce363b7a4c5ca3ab0e833c25e3d46c0/stateful_tests__debug__actions_.html

Fix this by do not reset the signal back, and introduce a flag to ignore
signals after disabling the timer.

Fixes: #31740
2021-12-03 01:33:08 +03:00
Azat Khuzhin
96bd83c31e Do not reopen logs on USR1, HUP is enough
USR1 is also used for query_profiler_real_time_period_ns, let's not
overlap.
2021-12-03 01:32:57 +03:00
Azat Khuzhin
9276384977 clickhouse-test: do not use random generator with shared state
Recently (#32094) test database had been overlapped, and random prefix
for database had been increased from 6 to 8.

But actually 6 bytes for random prefix should be enough (with existing
alphabet (0-9a-z) it is 36**6=2'176'782'336), and the real reason of
this overlap is that random generator by default uses shared state [1]:

    The functions supplied by this module are actually bound methods of
    a hidden instance of the random.Random class. You can instantiate your
    own instances of Random to get generators that don’t share state.

  [1]: https://docs.python.org/3/library/random.html

I've played a little bit with random in python, and using default random
generator it generates non-unique strings pretty fast, just in a few
runs, but using SystemRandom (that uses /dev/urandom) it takes ~1 minute.

Test:

```sh
    $ while /tmp/test.py | LANG=c sort -S5G | LANG=c uniq -d | tee /dev/stderr | wc -l | fgrep -q -x -c 0; do :; done
```

```python
    #!/usr/bin/env python3

    import multiprocessing
    import string
    import random

    def random_str(length=6):
        alphabet = string.ascii_lowercase + string.digits
        return ''.join(random.SystemRandom().choice(alphabet) for _ in range(length))

    def worker(_):
        print(random_str())

    with multiprocessing.Pool(processes=2) as pool:
        pool.map(worker, range(0, int(10e3)))
```

So let's switch to SystemRandom and use 6-byte prefix.
2021-12-03 01:29:23 +03:00
Azat Khuzhin
bfaaa79cfc tests/ci: do not compress logs twice
Cc: @alesapin
2021-12-03 01:28:32 +03:00
Frank Chen
e895c85f49
Add exception/exception_code to trace span log (#32040) 2021-12-03 01:00:02 +03:00
kssenii
06312a2df9 Fix 2021-12-02 22:48:21 +03:00
mergify[bot]
e2cb91b560
Merge branch 'master' into add_one_more_group 2021-12-02 19:28:13 +00:00
alesapin
45021bd35c
Merge pull request #32121 from ClickHouse/add_assertion_in_keeper
Add check for duplicate hostnames and IDs in KeeperConfig
2021-12-02 22:24:46 +03:00
alesapin
308eadd83a
Merge pull request #32159 from ClickHouse/fix_ci
Fix CI
2021-12-02 22:04:48 +03:00
alesapin
f2140be90b Fix CI 2021-12-02 22:03:59 +03:00
tavplubix
4606376734
Update ReplicatedMergeTreeSink.cpp 2021-12-02 21:51:23 +03:00
tavplubix
4d96d6ab77
Merge pull request #32052 from stigsb/fix-flaky-create-table-like-test
Fix flaky integration test for MaterializedMySQL CREATE TABLE LIKE
2021-12-02 21:28:00 +03:00
Alexander Tokmakov
252192bc6d fix active replicas count in quorum inserts 2021-12-02 21:02:11 +03:00
alesapin
c14086d578
Merge pull request #32064 from ClickHouse/lightweight_checks_rerun
Add ability for lightweight checks rerun
2021-12-02 20:54:27 +03:00
alexey-milovidov
2a7a75ea88
Merge pull request #32156 from flickerbox/crb-update-featured-image
Update featured image for 21.11 release blog post
2021-12-02 20:35:48 +03:00
Cody Baker
be660ca25f Update featured image for 21.11 release blog post 2021-12-02 10:28:47 -07:00
Cody Baker
c9ae0aec32 Fix margin on form field 2021-12-02 10:02:41 -07:00
Cody Baker
72e47d9cc7 Update form field class 2021-12-02 10:01:54 -07:00
Cody Baker
8e72d6755d Merge branch 'master' into crb-support-page
# Conflicts:
#	website/css/main.css
2021-12-02 09:57:51 -07:00
Cody Baker
39a28d46f1 Add customer reference number field 2021-12-02 09:50:53 -07:00
alesapin
dd2ccd3b17
Merge pull request #32155 from ClickHouse/small_improvements
Small improvements in lambda code
2021-12-02 19:39:00 +03:00
alesapin
9e1437a7c4 Small improvements in lambda code 2021-12-02 19:38:18 +03:00
mergify[bot]
c399eaf945
Merge branch 'master' into add_one_more_group 2021-12-02 15:20:06 +00:00
Kruglov Pavel
afd74aa166
Update 02117_custom_separated_with_names_and_types.sh 2021-12-02 18:15:50 +03:00
kssenii
64fcbe347e Update 2021-12-02 16:08:11 +03:00
Nikolai Kochetov
7de34bb8e1
Merge pull request #32126 from ClickHouse/remove-02116_global_in_time_limit
Remove 02116_global_in_time_limit.
2021-12-02 16:07:18 +03:00
Nikolai Kochetov
4d10762727 Remove 02116_global_in_time_limit. 2021-12-02 16:05:06 +03:00
mergify[bot]
88e1a26e99
Merge branch 'master' into add_one_more_group 2021-12-02 12:56:57 +00:00
alesapin
b8bffcb14c
Merge pull request #32115 from ClickHouse/fix_flaky_keeper_tests
Fix flaky keeper whitelist test
2021-12-02 15:41:17 +03:00
tavplubix
1f4b14f68b
Merge pull request #32119 from ClickHouse/tavplubix-patch-4
Fix data race in `removePartAndEnqueueFetch(...)`
2021-12-02 15:25:38 +03:00
avogar
27e6e063d0 Change test 02117_custom_separated_with_names_and_types 2021-12-02 15:06:15 +03:00
Kruglov Pavel
049b2c0c14
Merge pull request #31072 from MaxWk/feature-support-bool-type
Feature support bool type
2021-12-02 14:50:29 +03:00
alesapin
437591f2fa Add check for duplicate hostnames and IDs in KeeperConfig 2021-12-02 14:46:33 +03:00
tavplubix
5400c5899d
Merge pull request #32063 from ClickHouse/fix_detach_dir_exists
Fix 'directory exists' error when detaching part
2021-12-02 14:04:21 +03:00
Anton Popov
a79dbb6b6a
Merge pull request #32079 from Avogar/parse-plus-in-float
Allow plus before Float32/Float64
2021-12-02 14:03:48 +03:00
alesapin
ff6cc14df3 Fix one more test 2021-12-02 13:41:04 +03:00
alesapin
7ef0737edc Move fuzzers and unit tests to another group 2021-12-02 13:23:32 +03:00
tavplubix
e9489ccd6f
Update StorageReplicatedMergeTree.cpp 2021-12-02 13:18:02 +03:00
Maksim Kita
2d359a5e79
Merge pull request #32110 from bharatnc/ncb/h3-func-looping
make looping in H3 funcs uniform
2021-12-02 13:15:36 +03:00
alesapin
e704140d07 Fix flaky whitelist test 2021-12-02 12:37:08 +03:00
Maksim Kita
a426ed0a5a
Merge pull request #31800 from kitaisreal/dict-get-with-type-nullable-fix
Function dictGet with type Nullable fix
2021-12-02 12:27:47 +03:00
alesapin
1cda5bfe4e
Merge pull request #32114 from ClickHouse/remove_pvs_from_master
Remove PVS check from master
2021-12-02 11:37:00 +03:00
alesapin
97e7da18bb Remove PVS check from master 2021-12-02 11:36:30 +03:00
Maksim Kita
9874931347
Merge pull request #32095 from FedericoCeratto/patch-2
Fix syntax error: drop comma
2021-12-02 11:20:52 +03:00
kssenii
5a273d4dda Update version 2021-12-02 08:02:07 +00:00