thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-13 19:14:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,861 Commits 9,130 Branches 1,774 Tags 2 GiB
0eab165761
Commit Graph

78861 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Maksim Kita
ca2a70cd8d
Merge pull request #32080 from azat/cast-fix-ubsan 
Fix CAST from Nullable with cast_keep_nullable
 2021-12-03 13:05:38 +03:00
Maksim Kita
6ec559f103
Update JSONEachRowRowOutputFormat.h 2021-12-03 12:48:28 +03:00
tavplubix
17f8193435
Merge pull request #32157 from ClickHouse/fix_active_replicas_count_quorum 
Fix active replicas count in quorum inserts
 2021-12-03 12:39:36 +03:00
alesapin
caa481aba8 Use ccache in fast test 2021-12-03 12:28:17 +03:00
alesapin
99a5ca2646 Fix style (got tired of it) 2021-12-03 12:19:39 +03:00
Kruglov Pavel
17d6f539b3
Merge pull request #32019 from azat/dist-desc 
Fix excessive DESC TABLE for remote() over identifier
 2021-12-03 12:15:05 +03:00
mergify[bot]
51b8148cc0
Merge branch 'master' into fix-test-executable-storage-input 2021-12-03 09:11:05 +00:00
Maksim Kita
74979a5f01
Merge pull request #31095 from amosbird/addprojection 
Function name normalization for ALTER queries
 2021-12-03 12:07:07 +03:00
SuperDJY
5d7dfc6eb9
fix storage join settings with persistency (#32066) 2021-12-03 12:06:58 +03:00
Kruglov Pavel
3d047747ed
Merge pull request #32123 from Avogar/fix-tests 
Change test 02117_custom_separated_with_names_and_types
 2021-12-03 12:06:50 +03:00
alesapin
ec0bfa7bcf Better output for some actions 2021-12-03 11:33:16 +03:00
Azat Khuzhin
3e96b28843 perf: convert killall to pkill (since killall has some magic, see -e option) 2021-12-03 10:56:42 +03:00
Azat Khuzhin
fb0cc62509 perf: fix waiting of the server after running tests 
killall requires strict match, i.e. "clickhouse-server" not
"clickhouse":

    2021-12-03 05:24:56      + env kill -- -21700
    2021-12-03 05:24:56      kill: (-21700): No such process
    2021-12-03 05:24:56      + killall clickhouse
    2021-12-03 05:24:56      clickhouse: no process found
    2021-12-03 05:24:56      + echo Servers stopped.
    2021-12-03 05:24:56      Servers stopped.
    2021-12-03 05:24:56      + analyze_queries

    $ tail -n1 *-server-log.log
    ==> left-server-log.log <==
    2021.12.03 05:26:59.530647 [ 450 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1668052

    ==> right-server-log.log <==
    2021.12.03 05:27:20.873136 [ 466 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 9605

    ==> setup-server-log.log <==
    2021.12.03 02:47:14.844395 [ 96 ] {} <Information> Application: Child process exited normally with code 0.

As you can see killall instantly fails with no such process, while this
cannot be true since it was there, and also according to logs there were
messages after running analyze_queries() from compare.sh

This should fix problems like in [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/32080/344298f4037f88b114b8e798bb30036b24be8f16/performance_comparison/report.html#fail1
 2021-12-03 10:55:35 +03:00
alesapin
7699d3f35e
Merge pull request #32162 from azat/double-gz 
tests/ci: do not compress logs twice
 2021-12-03 10:20:35 +03:00
alesapin
83c086b051
Merge pull request #32120 from ClickHouse/add_one_more_group 
Move fuzzers and unit tests to another group
 2021-12-03 10:14:09 +03:00
zzsmdfj
4fbfc7c56f MaterializedMySQL support VARBINARY type 2021-12-03 15:08:03 +08:00
Kseniia Sumarokova
03c7907b37
Merge pull request #32172 from ClickHouse/kssenii-patch-2 
Update materialized-postgresql.md
 2021-12-03 09:28:46 +03:00
Kseniia Sumarokova
9093ad6596
Update materialized-postgresql.md 2021-12-03 09:27:22 +03:00
Kseniia Sumarokova
a8dcbc988a
Merge pull request #32129 from kssenii/fix-postgres-tests 
fixing postgres tests
 2021-12-03 09:15:20 +03:00
alexey-milovidov
b7fb21e6f0
Merge pull request #32163 from azat/clickhouse-test-random 
clickhouse-test: do not use random generator with shared state
 2021-12-03 06:53:57 +03:00
alexey-milovidov
17fcf0a6d3
Merge pull request #32165 from azat/query-profiler-reset-fix 
Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset
 2021-12-03 06:51:21 +03:00
Azat Khuzhin
344298f403 Fix CAST from Nullable with cast_keep_nullable 
UBsan report [1]:

    SELECT toUInt32OrDefault(toNullable(0)) SETTINGS cast_keep_nullable = 1

    ../src/Common/assert_cast.h:50:12: runtime error: downcast of address 0x000029461170 which does not point to an object of type 'const DB::ColumnNullable'
    0x000029461170: note: object is of type 'DB::ColumnVector<unsigned int>'
     00 00 00 00  a0 51 3d 09 00 00 00 00  01 00 00 00 00 00 00 00  a0 3f 4e 29 00 00 00 00  a4 3f 4e 29
                  ^~~~~~~~~~~~~~~~~~~~~~~
                  vptr for 'DB::ColumnVector<unsigned int>'
        0 0x1e3e279d in DB::ColumnNullable const& assert_cast<DB::ColumnNullable const&, DB::IColumn const&>(DB::IColumn const&) obj-x86_64-linux-gnu/../src/Common/assert_cast.h:50:12
        1 0x1e3e279d in DB::ColumnNullable::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Columns/ColumnNullable.cpp:167:43
        2 0x12db4507 in DB::FunctionCastOrDefault::executeImpl() const (/src/ch/tmp/32019/clickhouse-ubsan+0x12db4507)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32019/9e1222a50b1805f696f4205e451365f8acdbc8a5/fuzzer_astfuzzerubsan,actions//report.html

And w/o -fsanitize=undefined:

    SELECT toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))) SETTINGS cast_keep_nullable = 1

    Received exception:
    Code: 12. DB::Exception: Parameters start = 0, length = 1 are out of bound in ColumnVector<T>::insertRangeFrom method (data.size() = 0).: While processing toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))). (PARAMETER_OUT_OF_BOUND)
 2021-12-03 02:02:42 +03:00
Anton Popov
f6be3d16fd
Merge pull request #24820 from kssenii/versioning 
Versioning of aggregate function states
 2021-12-03 01:41:44 +03:00
Azat Khuzhin
0b2de32228 Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset 
Even after timer_delete() the signal can be fired.
Reproducer:

    $ clickhouse-server & # with configured trace_log
    $ clickhouse-benchmark -c2 --query 'select * from numbers(1e6)' --query_profiler_cpu_time_period_ns=1 &
    ...
    2021.12.02 14:28:01.320288 [ 24885 ] {} <Debug> TCPHandler: Processed in 177.055205644 sec.
    User defined signal 2

CI failures:
- https://s3.amazonaws.com/clickhouse-test-reports/32067/8dbc7a8dae17090a18778f29629d8746a1bb9b72/stateful_tests__debug__actions_.html
- https://s3.amazonaws.com/clickhouse-test-reports/32064/c07450a7dce363b7a4c5ca3ab0e833c25e3d46c0/stateful_tests__debug__actions_.html

Fix this by do not reset the signal back, and introduce a flag to ignore
signals after disabling the timer.

Fixes: #31740
 2021-12-03 01:33:08 +03:00
Azat Khuzhin
96bd83c31e Do not reopen logs on USR1, HUP is enough 
USR1 is also used for query_profiler_real_time_period_ns, let's not
overlap.
 2021-12-03 01:32:57 +03:00
Azat Khuzhin
9276384977 clickhouse-test: do not use random generator with shared state 
Recently (#32094) test database had been overlapped, and random prefix
for database had been increased from 6 to 8.

But actually 6 bytes for random prefix should be enough (with existing
alphabet (0-9a-z) it is 36**6=2'176'782'336), and the real reason of
this overlap is that random generator by default uses shared state [1]:

    The functions supplied by this module are actually bound methods of
    a hidden instance of the random.Random class. You can instantiate your
    own instances of Random to get generators that don’t share state.

  [1]: https://docs.python.org/3/library/random.html

I've played a little bit with random in python, and using default random
generator it generates non-unique strings pretty fast, just in a few
runs, but using SystemRandom (that uses /dev/urandom) it takes ~1 minute.

Test:

```sh
    $ while /tmp/test.py | LANG=c sort -S5G | LANG=c uniq -d | tee /dev/stderr | wc -l | fgrep -q -x -c 0; do :; done
```

```python
    #!/usr/bin/env python3

    import multiprocessing
    import string
    import random

    def random_str(length=6):
        alphabet = string.ascii_lowercase + string.digits
        return ''.join(random.SystemRandom().choice(alphabet) for _ in range(length))

    def worker(_):
        print(random_str())

    with multiprocessing.Pool(processes=2) as pool:
        pool.map(worker, range(0, int(10e3)))
```

So let's switch to SystemRandom and use 6-byte prefix.
 2021-12-03 01:29:23 +03:00
Azat Khuzhin
bfaaa79cfc tests/ci: do not compress logs twice 
Cc: @alesapin
 2021-12-03 01:28:32 +03:00
Frank Chen
e895c85f49
Add exception/exception_code to trace span log (#32040) 2021-12-03 01:00:02 +03:00
Alexey
6e27c9c625 More default argument values 
Actual interactive mode call conditions
 2021-12-02 20:19:50 +00:00
kssenii
06312a2df9 Fix 2021-12-02 22:48:21 +03:00
mergify[bot]
e2cb91b560
Merge branch 'master' into add_one_more_group 2021-12-02 19:28:13 +00:00
alesapin
45021bd35c
Merge pull request #32121 from ClickHouse/add_assertion_in_keeper 
Add check for duplicate hostnames and IDs in KeeperConfig
 2021-12-02 22:24:46 +03:00
alesapin
308eadd83a
Merge pull request #32159 from ClickHouse/fix_ci 
Fix CI
 2021-12-02 22:04:48 +03:00
alesapin
f2140be90b Fix CI 2021-12-02 22:03:59 +03:00
tavplubix
4606376734
Update ReplicatedMergeTreeSink.cpp 2021-12-02 21:51:23 +03:00
tavplubix
4d96d6ab77
Merge pull request #32052 from stigsb/fix-flaky-create-table-like-test 
Fix flaky integration test for MaterializedMySQL CREATE TABLE LIKE
 2021-12-02 21:28:00 +03:00
Alexander Tokmakov
252192bc6d fix active replicas count in quorum inserts 2021-12-02 21:02:11 +03:00
alesapin
c14086d578
Merge pull request #32064 from ClickHouse/lightweight_checks_rerun 
Add ability for lightweight checks rerun
 2021-12-02 20:54:27 +03:00
alexey-milovidov
2a7a75ea88
Merge pull request #32156 from flickerbox/crb-update-featured-image 
Update featured image for 21.11 release blog post
 2021-12-02 20:35:48 +03:00
Cody Baker
be660ca25f Update featured image for 21.11 release blog post 2021-12-02 10:28:47 -07:00
Cody Baker
c9ae0aec32 Fix margin on form field 2021-12-02 10:02:41 -07:00
Cody Baker
72e47d9cc7 Update form field class 2021-12-02 10:01:54 -07:00
Cody Baker
8e72d6755d Merge branch 'master' into crb-support-page 
# Conflicts:
#	website/css/main.css
 2021-12-02 09:57:51 -07:00
Cody Baker
39a28d46f1 Add customer reference number field 2021-12-02 09:50:53 -07:00
alesapin
dd2ccd3b17
Merge pull request #32155 from ClickHouse/small_improvements 
Small improvements in lambda code
 2021-12-02 19:39:00 +03:00
alesapin
9e1437a7c4 Small improvements in lambda code 2021-12-02 19:38:18 +03:00
mergify[bot]
c399eaf945
Merge branch 'master' into add_one_more_group 2021-12-02 15:20:06 +00:00
Kruglov Pavel
afd74aa166
Update 02117_custom_separated_with_names_and_types.sh 2021-12-02 18:15:50 +03:00
Raúl Marín
755ba5dc09 Don't forget to check the output of checkTimeLimitSoft 2021-12-02 15:09:53 +01:00
Raúl Marín
37572f7362 02122_join_group_by_timeout: Unify max process timeouts 2021-12-02 14:57:42 +01:00