thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,661 Commits 9,328 Branches 1,783 Tags 2.1 GiB
40c9ffdfff
Commit Graph

78661 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
avogar
7549619b25 Improve skiping unknown fields with Quoted escaping rule in Template/CustomSeparated formats 2021-12-03 16:25:35 +03:00
Raúl Marín
0c1e89b043 Add test 2021-12-03 14:18:01 +01:00
kssenii
adaa32b41b Fix inner table name 2021-12-03 15:52:57 +03:00
avogar
4f136cb30c Fix NaN deserialization for Quoted escaping rule 2021-12-03 15:37:39 +03:00
alesapin
adde67358a
Merge pull request #32175 from ClickHouse/better_logs 
Better output for some actions
 2021-12-03 15:32:02 +03:00
alesapin
a625dd41e8
Merge branch 'master' into better_logs 2021-12-03 15:31:45 +03:00
frank chen
898db5b468 Resolve review comments 
Signed-off-by: frank chen <frank.chen021@outlook.com>
 2021-12-03 19:47:05 +08:00
Maksim Kita
4c916a0e38 DictionariesLoader qualify name with database fix 2021-12-03 14:31:49 +03:00
alesapin
597c578809
Merge pull request #31263 from den-crane/Docs/alter-partition-verbose-result 
docs. alter-partition-verbose-result
 2021-12-03 14:15:30 +03:00
Maksim Kita
8e37281a7f RangeHashedDictionary fix bytes_allocated with update_field 2021-12-03 14:10:47 +03:00
Maksim Kita
4bbb02bbae RangeHashedDictionary added update_field support 2021-12-03 14:06:58 +03:00
alesapin
4f30f69d54
Merge pull request #32177 from ClickHouse/use_ccache_in_fast_test 
Use ccache in fast test
 2021-12-03 13:57:22 +03:00
kssenii
0470525ea3 Fix 2021-12-03 13:39:27 +03:00
mergify[bot]
48ed754fab
Merge branch 'master' into use_ccache_in_fast_test 2021-12-03 10:28:03 +00:00
Maksim Kita
2d3f773147 Function accurateCastOrDefault remove separate branch 2021-12-03 13:25:55 +03:00
alesapin
d8604acb8e Bump 2021-12-03 13:18:24 +03:00
mergify[bot]
478c35538d
Merge branch 'master' into trace_finish_time_bug 2021-12-03 10:12:48 +00:00
Maksim Kita
ca2a70cd8d
Merge pull request #32080 from azat/cast-fix-ubsan 
Fix CAST from Nullable with cast_keep_nullable
 2021-12-03 13:05:38 +03:00
Maksim Kita
6ec559f103
Update JSONEachRowRowOutputFormat.h 2021-12-03 12:48:28 +03:00
tavplubix
17f8193435
Merge pull request #32157 from ClickHouse/fix_active_replicas_count_quorum 
Fix active replicas count in quorum inserts
 2021-12-03 12:39:36 +03:00
alesapin
caa481aba8 Use ccache in fast test 2021-12-03 12:28:17 +03:00
alesapin
99a5ca2646 Fix style (got tired of it) 2021-12-03 12:19:39 +03:00
Kruglov Pavel
17d6f539b3
Merge pull request #32019 from azat/dist-desc 
Fix excessive DESC TABLE for remote() over identifier
 2021-12-03 12:15:05 +03:00
mergify[bot]
51b8148cc0
Merge branch 'master' into fix-test-executable-storage-input 2021-12-03 09:11:05 +00:00
Maksim Kita
74979a5f01
Merge pull request #31095 from amosbird/addprojection 
Function name normalization for ALTER queries
 2021-12-03 12:07:07 +03:00
SuperDJY
5d7dfc6eb9
fix storage join settings with persistency (#32066) 2021-12-03 12:06:58 +03:00
Kruglov Pavel
3d047747ed
Merge pull request #32123 from Avogar/fix-tests 
Change test 02117_custom_separated_with_names_and_types
 2021-12-03 12:06:50 +03:00
alesapin
ec0bfa7bcf Better output for some actions 2021-12-03 11:33:16 +03:00
Azat Khuzhin
3e96b28843 perf: convert killall to pkill (since killall has some magic, see -e option) 2021-12-03 10:56:42 +03:00
Azat Khuzhin
fb0cc62509 perf: fix waiting of the server after running tests 
killall requires strict match, i.e. "clickhouse-server" not
"clickhouse":

    2021-12-03 05:24:56      + env kill -- -21700
    2021-12-03 05:24:56      kill: (-21700): No such process
    2021-12-03 05:24:56      + killall clickhouse
    2021-12-03 05:24:56      clickhouse: no process found
    2021-12-03 05:24:56      + echo Servers stopped.
    2021-12-03 05:24:56      Servers stopped.
    2021-12-03 05:24:56      + analyze_queries

    $ tail -n1 *-server-log.log
    ==> left-server-log.log <==
    2021.12.03 05:26:59.530647 [ 450 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1668052

    ==> right-server-log.log <==
    2021.12.03 05:27:20.873136 [ 466 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 9605

    ==> setup-server-log.log <==
    2021.12.03 02:47:14.844395 [ 96 ] {} <Information> Application: Child process exited normally with code 0.

As you can see killall instantly fails with no such process, while this
cannot be true since it was there, and also according to logs there were
messages after running analyze_queries() from compare.sh

This should fix problems like in [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/32080/344298f4037f88b114b8e798bb30036b24be8f16/performance_comparison/report.html#fail1
 2021-12-03 10:55:35 +03:00
alesapin
7699d3f35e
Merge pull request #32162 from azat/double-gz 
tests/ci: do not compress logs twice
 2021-12-03 10:20:35 +03:00
alesapin
83c086b051
Merge pull request #32120 from ClickHouse/add_one_more_group 
Move fuzzers and unit tests to another group
 2021-12-03 10:14:09 +03:00
zzsmdfj
4fbfc7c56f MaterializedMySQL support VARBINARY type 2021-12-03 15:08:03 +08:00
Kseniia Sumarokova
03c7907b37
Merge pull request #32172 from ClickHouse/kssenii-patch-2 
Update materialized-postgresql.md
 2021-12-03 09:28:46 +03:00
Kseniia Sumarokova
9093ad6596
Update materialized-postgresql.md 2021-12-03 09:27:22 +03:00
Kseniia Sumarokova
a8dcbc988a
Merge pull request #32129 from kssenii/fix-postgres-tests 
fixing postgres tests
 2021-12-03 09:15:20 +03:00
alexey-milovidov
b7fb21e6f0
Merge pull request #32163 from azat/clickhouse-test-random 
clickhouse-test: do not use random generator with shared state
 2021-12-03 06:53:57 +03:00
alexey-milovidov
17fcf0a6d3
Merge pull request #32165 from azat/query-profiler-reset-fix 
Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset
 2021-12-03 06:51:21 +03:00
Azat Khuzhin
344298f403 Fix CAST from Nullable with cast_keep_nullable 
UBsan report [1]:

    SELECT toUInt32OrDefault(toNullable(0)) SETTINGS cast_keep_nullable = 1

    ../src/Common/assert_cast.h:50:12: runtime error: downcast of address 0x000029461170 which does not point to an object of type 'const DB::ColumnNullable'
    0x000029461170: note: object is of type 'DB::ColumnVector<unsigned int>'
     00 00 00 00  a0 51 3d 09 00 00 00 00  01 00 00 00 00 00 00 00  a0 3f 4e 29 00 00 00 00  a4 3f 4e 29
                  ^~~~~~~~~~~~~~~~~~~~~~~
                  vptr for 'DB::ColumnVector<unsigned int>'
        0 0x1e3e279d in DB::ColumnNullable const& assert_cast<DB::ColumnNullable const&, DB::IColumn const&>(DB::IColumn const&) obj-x86_64-linux-gnu/../src/Common/assert_cast.h:50:12
        1 0x1e3e279d in DB::ColumnNullable::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Columns/ColumnNullable.cpp:167:43
        2 0x12db4507 in DB::FunctionCastOrDefault::executeImpl() const (/src/ch/tmp/32019/clickhouse-ubsan+0x12db4507)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32019/9e1222a50b1805f696f4205e451365f8acdbc8a5/fuzzer_astfuzzerubsan,actions//report.html

And w/o -fsanitize=undefined:

    SELECT toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))) SETTINGS cast_keep_nullable = 1

    Received exception:
    Code: 12. DB::Exception: Parameters start = 0, length = 1 are out of bound in ColumnVector<T>::insertRangeFrom method (data.size() = 0).: While processing toUInt32OrDefault(toNullable(1), toNullable(toUInt32(1))). (PARAMETER_OUT_OF_BOUND)
 2021-12-03 02:02:42 +03:00
Anton Popov
f6be3d16fd
Merge pull request #24820 from kssenii/versioning 
Versioning of aggregate function states
 2021-12-03 01:41:44 +03:00
Azat Khuzhin
0b2de32228 Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset 
Even after timer_delete() the signal can be fired.
Reproducer:

    $ clickhouse-server & # with configured trace_log
    $ clickhouse-benchmark -c2 --query 'select * from numbers(1e6)' --query_profiler_cpu_time_period_ns=1 &
    ...
    2021.12.02 14:28:01.320288 [ 24885 ] {} <Debug> TCPHandler: Processed in 177.055205644 sec.
    User defined signal 2

CI failures:
- https://s3.amazonaws.com/clickhouse-test-reports/32067/8dbc7a8dae17090a18778f29629d8746a1bb9b72/stateful_tests__debug__actions_.html
- https://s3.amazonaws.com/clickhouse-test-reports/32064/c07450a7dce363b7a4c5ca3ab0e833c25e3d46c0/stateful_tests__debug__actions_.html

Fix this by do not reset the signal back, and introduce a flag to ignore
signals after disabling the timer.

Fixes: #31740
 2021-12-03 01:33:08 +03:00
Azat Khuzhin
96bd83c31e Do not reopen logs on USR1, HUP is enough 
USR1 is also used for query_profiler_real_time_period_ns, let's not
overlap.
 2021-12-03 01:32:57 +03:00
Azat Khuzhin
9276384977 clickhouse-test: do not use random generator with shared state 
Recently (#32094) test database had been overlapped, and random prefix
for database had been increased from 6 to 8.

But actually 6 bytes for random prefix should be enough (with existing
alphabet (0-9a-z) it is 36**6=2'176'782'336), and the real reason of
this overlap is that random generator by default uses shared state [1]:

    The functions supplied by this module are actually bound methods of
    a hidden instance of the random.Random class. You can instantiate your
    own instances of Random to get generators that don’t share state.

  [1]: https://docs.python.org/3/library/random.html

I've played a little bit with random in python, and using default random
generator it generates non-unique strings pretty fast, just in a few
runs, but using SystemRandom (that uses /dev/urandom) it takes ~1 minute.

Test:

```sh
    $ while /tmp/test.py | LANG=c sort -S5G | LANG=c uniq -d | tee /dev/stderr | wc -l | fgrep -q -x -c 0; do :; done
```

```python
    #!/usr/bin/env python3

    import multiprocessing
    import string
    import random

    def random_str(length=6):
        alphabet = string.ascii_lowercase + string.digits
        return ''.join(random.SystemRandom().choice(alphabet) for _ in range(length))

    def worker(_):
        print(random_str())

    with multiprocessing.Pool(processes=2) as pool:
        pool.map(worker, range(0, int(10e3)))
```

So let's switch to SystemRandom and use 6-byte prefix.
 2021-12-03 01:29:23 +03:00
Azat Khuzhin
bfaaa79cfc tests/ci: do not compress logs twice 
Cc: @alesapin
 2021-12-03 01:28:32 +03:00
Frank Chen
e895c85f49
Add exception/exception_code to trace span log (#32040) 2021-12-03 01:00:02 +03:00
kssenii
06312a2df9 Fix 2021-12-02 22:48:21 +03:00
mergify[bot]
e2cb91b560
Merge branch 'master' into add_one_more_group 2021-12-02 19:28:13 +00:00
alesapin
45021bd35c
Merge pull request #32121 from ClickHouse/add_assertion_in_keeper 
Add check for duplicate hostnames and IDs in KeeperConfig
 2021-12-02 22:24:46 +03:00
alesapin
308eadd83a
Merge pull request #32159 from ClickHouse/fix_ci 
Fix CI
 2021-12-02 22:04:48 +03:00
alesapin
f2140be90b Fix CI 2021-12-02 22:03:59 +03:00