thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-21 09:10:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
56,890 Commits 8,936 Branches 1,766 Tags 2 GiB
14fe627d32
Commit Graph

6250 Commits

Author SHA1 Message Date
Alexey Milovidov
f467104919 Merge branch 'point-in-polygon-ubsan' of github.com:yandex/ClickHouse into point-in-polygon-ubsan 2021-02-04 08:27:15 +03:00
Alexey Milovidov
47a56b9d1d Merge branch 'master' into point-in-polygon-ubsan 2021-02-04 08:26:41 +03:00
Azat Khuzhin
984445b44b Fix UBsan reports in quantileTiming 
UBsan reports [1]:

    ../src/AggregateFunctions/QuantileTiming.h:442:27: runtime error: 1.84467e+19 is outside the range of representable values of type 'unsigned long' Received signal -3 Received signal Unknown signal (-3)

  [1]: https://clickhouse-test-reports.s3.yandex.net/19971/e15f5d9cb5b36482d1ae9ca069074fb200f2ab37/fuzzer_ubsan/report.html#fail1

Follow-up for: #19394
 2021-02-04 02:01:48 +03:00
Azat Khuzhin
cc43f92719 Fix tests *.reference files 
Using:

    sed -i 's/MergeTree  with order/MergeTree with order/g' tests/queries/0_stateless/*.reference

Fixes: dc2afc4795 ("Fix double whitespace #18383")
 2021-02-03 22:45:54 +03:00
alexey-milovidov
ca06d3b317
Merge branch 'master' into fix-rocksdb 2021-02-03 21:28:50 +03:00
alexey-milovidov
402c018d1e
Merge pull request #19947 from azat/dist-INSERT-block-structure-mismatch 
Fix "Block structure mismatch" for INSERT into Distributed
 2021-02-03 21:27:45 +03:00
alexey-milovidov
0aca4a740c
Update 01700_point_in_polygon_ubsan.sql 2021-02-03 21:26:03 +03:00
Alexey Milovidov
eb8b2e883a Add a test 2021-02-03 21:25:04 +03:00
Nikolai Kochetov
c39c965612
Merge pull request #19962 from keen-wolf/timezoneOffset 
Implement the timezoneOffset function
 2021-02-03 20:37:34 +03:00
Alexey Milovidov
82ab793731 Fix missing type check in StorageEmbeddedRocksDB 2021-02-03 20:35:23 +03:00
Alexander Kuzmenkov
ec382d8812 BETWEEN CURRENT ROW AND CURRENT ROW 2021-02-03 17:55:40 +03:00
Alexander Kuzmenkov
35754abb4a CURRENT ROW frame start for RANGE frame 2021-02-03 17:22:37 +03:00
Alexander Kuzmenkov
7c55ecf67d cleanup 2021-02-03 16:41:59 +03:00
Alexander Kuzmenkov
d8a9d347fe
Merge pull request #20000 from ClickHouse/aku/perf 
update perf tests
 2021-02-03 16:06:58 +03:00
Alexander Kuzmenkov
f31d2206a7 more fuzzing and less bugs 2021-02-03 15:50:25 +03:00
Pavel Kovalenko
b7151aa754 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskDecorator.h
#	src/Disks/IDisk.h
#	src/Disks/S3/DiskS3.cpp
 2021-02-03 14:22:18 +03:00
Alexander Kuzmenkov
53052f7040
Update group_by_sundy_li.xml 2021-02-03 12:22:02 +03:00
Alexander Kuzmenkov
bb45c5a8ff Merge remote-tracking branch 'origin/master' into HEAD 2021-02-03 12:13:23 +03:00
Maksim Kita
ba45cf32ab
Merge pull request #19858 from kitaisreal/reinterpret-as-updated-to-big-integers 
Function reinterpretAs updated to support big integers
 2021-02-03 12:12:13 +03:00
Alexander Kuzmenkov
1b47cf08a9
Merge pull request #19921 from ClickHouse/aku/window-prototype-2 
UNBOUNDED FOLLOWING frame end
 2021-02-03 12:10:37 +03:00
alexey-milovidov
830b0eb36e
Merge pull request #19974 from qoega/kerberized-hdfs-port-clash 
Fixed port clash from test_storage_kerberized_hdfs test.
 2021-02-03 10:16:14 +03:00
Alexander Kuzmenkov
a164abf23e add frame fuzzing and one query that fails under msan 2021-02-03 09:42:54 +03:00
Alexander Kuzmenkov
7e945bab03 fix the calculation for moving frame start 2021-02-03 08:53:21 +03:00
alexey-milovidov
a4d86c43e9
Merge pull request #19946 from ClickHouse/improve-performance-of-aggregate-functions 
Improve performance of aggregate functions
 2021-02-03 08:46:51 +03:00
Alexander Kuzmenkov
d0bac8bfd9 Merge remote-tracking branch 'origin/master' into HEAD 2021-02-03 08:45:16 +03:00
Alexander Kuzmenkov
85f8918753 fix the row order to be like in the original test 2021-02-03 07:47:53 +03:00
Azat Khuzhin
ac510f9770 test_read_temporary_tables_on_failure: increase timeout to avoid false-positive 
In [1]:

    start_cluster = <helpers.cluster.ClickHouseCluster object at 0x7f2d46895dd8>

        def test_different_versions(start_cluster):
            with pytest.raises(QueryTimeoutExceedException):
                node.query("SELECT sleep(3)", timeout=1)
            with pytest.raises(QueryRuntimeException):
    >           node.query("SELECT 1", settings={'max_concurrent_queries_for_user': 1})
    E           Failed: DID NOT RAISE <class 'helpers.client.QueryRuntimeException'>

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/b68508002d134ead05bf2ca0e22a13a34a5c55c6/integration_tests_(thread).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
33628e1bd9 test_materialize_mysql_database: add retries for test_select_without_columns_8_0 
Here is an example:

    2021.01.31 14:04:43.615618 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:43.616453 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_1_1_0 to 0_1_1_0.
    ...
    2021.01.31 14:04:43.786823 [ 12 ] {4a0d1f80-4a96-4809-a84e-d55d7cd25cdd} <Debug> executeQuery: (from 172.18.0.1:45162, using production parser) SELECT count(_version) FROM db.t FORMAT TSV
    ...
    2021.01.31 14:04:44.566037 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:44.566943 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_2_2_0 to 0_2_2_0.
    ...
    2021.01.31 14:04:52.343911 [ 54 ] {} <Error> MaterializeMySQLSyncThread: Code: 1002, e.displayText() = DB::Exception: Master maybe lost, Stack trace (when copying this message, always include the lines below):

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/978f23e3437d0d0eb64727e87ffe1b80289c9fc2/integration_tests_(release).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
3d3d6777d3 test_dictionaries_dependency_xml: decrease dictionaries lifetime 
Since ExternalLoader::PeriodicUpdater::check_period_sec = 5, and so if
it will be scheduled too early the reload will be skipped, and indeed
this is what you can see in logs [1], the reload is done each 10
seconds, not 5:

    2021.01.31 14:20:22.590999 [ 48 ] {} <Trace> ExternalDictionariesLoader: Supposed update time for 'dep_x' is 2021-01-31 14:20:27 (loaded, lifetime [5, 5], no errors)
    2021.01.31 14:20:22.591016 [ 48 ] {} <Trace> ExternalDictionariesLoader: Next update time for 'dep_x' was set to 2021-01-31 14:20:27
    ...
    2021.01.31 14:20:32.164882 [ 50 ] {} <Trace> ExternalDictionariesLoader: Start loading object 'dep_x'

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/37797fdf5b30dc97147e73b3ac8ca9025b80aaed/integration_tests_(release).html#fail1
 2021-02-02 22:04:32 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
keenwolf
b7e4f7bd1e modify file attribute 2021-02-03 00:41:11 +08:00
Alexander Kuzmenkov
264aea20be Merge remote-tracking branch 'origin/master' into HEAD 2021-02-02 19:12:46 +03:00
keenwolf
9beb7689cc add comments and update test cases. 2021-02-02 23:35:45 +08:00
Vitaly Baranov
9427d5d94b
Merge pull request #19805 from vitlibar/allow-getting-return-type-of-dictget-without-loading-dictionary 
Fix starting the server with tables having default expressions containing dictGet()
 2021-02-02 18:19:55 +03:00
Alexander Kuzmenkov
d2a0e44a3a
Merge pull request #19299 from ClickHouse/aku/window-prototype 
[wip] support RANGE frame for window functions
 2021-02-02 17:59:16 +03:00
Alexander Kuzmenkov
6317208add
Merge branch 'master' into aku/perf 2021-02-02 17:24:17 +03:00
Alexey Milovidov
b9647e5326 Fix double whitespace 2021-02-02 17:22:03 +03:00
Alexander Kuzmenkov
b4afc49d3c update perf tests 
sed -i s'/^<test.*$/<test>/g' tests/performance/*.xml

WITH ceil(max(q[3]), 1) AS h
SELECT concat('sed -i s\'/^<test.*$/<test max_ignored_relative_change="', toString(h), '">/g\' tests/performance/', test, '.xml') AS s
FROM
(
    SELECT
        test,
        query_index,
        count(*),
        min(event_time),
        max(event_time) AS t,
        arrayMap(x -> floor(x, 3), quantiles(0, 0.5, 0.95, 1)(stat_threshold)) AS q,
        median(stat_threshold) AS m
    FROM perftest.query_metrics_v2
    WHERE ((pr_number != 0) AND (event_date > '2021-01-01')) AND (metric = 'client_time') AND (abs(diff) < 0.05) AND (old_value > 0.1)
    GROUP BY
        test,
        query_index,
        query_display_name
    HAVING (t > '2021-01-01 00:00:00') AND ((q[3]) > 0.1)
    ORDER BY test DESC
)
GROUP BY test
ORDER BY h DESC
FORMAT PrettySpace
 2021-02-02 17:21:43 +03:00
alexey-milovidov
695e28079d
Merge pull request #19960 from ClickHouse/json-extract-inaccurate-conversion-from-double-to-float 
Allow inaccurate conversion from double to float in function JSONExtract beacuse the users want that
 2021-02-02 15:15:37 +03:00
alexey-milovidov
4d0973cf25
Merge pull request #19959 from depressed-pho/fix-segv-fromModifiedJulianDay 
Fix a segfault in fromModifiedJulianDay()
 2021-02-02 13:50:14 +03:00
alexey-milovidov
05b8164133
Merge pull request #19956 from ClickHouse/geohash-ubsan 
Fix UBSan report in geoHashesInBox
 2021-02-02 12:52:01 +03:00
Maksim Kita
c889f51e9e Updated performance benchmark 2021-02-02 12:50:58 +03:00
feng lv
0edf65c094 fix test 
fix

update

fix spell
 2021-02-02 09:22:30 +00:00
Alexey Milovidov
0b166e1cbd Add performance test 2021-02-02 12:11:20 +03:00
Yatsishin Ilya
19d6e867c6 HDFS test had a clash with 9000 ClickHouse port. 2021-02-02 12:08:47 +03:00
Kruglov Pavel
fd5adf1ff6
Merge pull request #19925 from ClickHouse/fix_insert_values_with_semicolon 
Fix logical error in INSERT VALUES
 2021-02-02 12:05:19 +03:00
tavplubix
befee42ab0
Merge pull request #19684 from yiguolei/master 
Should fail ddl query as soon as possible if table is shutdown
 2021-02-02 11:46:58 +03:00
tavplubix
dbcddbcebd
Merge pull request #19770 from stigsb/materializemysql_system_parts 
Show MaterializeMySQL tables in system.parts
 2021-02-02 11:43:42 +03:00
alexey-milovidov
f5dc38a33c
Merge pull request #19584 from azat/client-complete-error-codes 
client: more suggestions
 2021-02-02 11:05:17 +03:00
alesapin
a9c50caa9a
Merge pull request #19942 from ClickHouse/fix_test_keeper_b2b_test 
Fix test keeper integration tests
 2021-02-02 10:17:27 +03:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
keenwolf
f21e22ae2c add test for function timezoneOffset 2021-02-02 12:22:45 +08:00
Alexey Milovidov
078011ef2d Allow conversion from double to float in function JSONExtract beacuse the users want that 2021-02-02 07:16:47 +03:00
PHO
5f7a852b92 Fix a segfault in fromModifiedJulianDay() 
It was crashing when the argument type was Nullable(T) where T was any integral type other than Int32.
 2021-02-02 12:43:36 +09:00
Alexey Milovidov
9930bb0bf6 Fix UBSan report in geoHashesInBox 2021-02-02 06:37:24 +03:00
Alexey Milovidov
b9fff3676b Add exception for ANTLR tests 2021-02-02 06:03:10 +03:00
Alexander Kuzmenkov
d084625436 cleanup 2021-02-02 03:51:35 +03:00
Alexander Kuzmenkov
78a9624fc5 CURRENT ROW frame start for ROWS frame 2021-02-02 02:27:23 +03:00
Alexander Kuzmenkov
ef46c36317 start offset for ROWS frame 2021-02-02 02:27:17 +03:00
Azat Khuzhin
37797fdf5b Merge remote-tracking branch 'upstream/master' into client-complete-error-codes 2021-02-02 00:14:53 +03:00
alexey-milovidov
6b2f0435c8
Merge pull request #19375 from Avogar/select-final 
Improve do_not_merge_across_partitions_select_final optimization.
 2021-02-01 20:31:08 +03:00
alexey-milovidov
1f0a91e0ab
Merge pull request #19677 from kitaisreal/executable-dictionary-implicit-key-option 
ExecutableDictionarySource added implicit_key option
 2021-02-01 20:13:25 +03:00
alexey-milovidov
2901591f81
Merge pull request #19876 from ClickHouse/intdiv-ubsan 
Fix UBSan report in intDiv
 2021-02-01 20:12:30 +03:00
alesapin
e6fd6abe6f Fix test keeper integration tests 2021-02-01 18:07:34 +03:00
alesapin
2aa8a6304b
Merge pull request #15450 from CurtizJ/fix-ttl-group-by 
Fix some cases of TTL expressions
 2021-02-01 16:48:07 +03:00
Nikolai Kochetov
19e4a33f9d
Merge pull request #19544 from amosbird/limitconcurrency 
Per MergeTree table query limit
 2021-02-01 16:09:12 +03:00
alexey-milovidov
f09158de0c
Merge pull request #19884 from kitaisreal/bloom-filter-index-crash-fix 
BloomFilter index crash fix
 2021-02-01 15:42:56 +03:00
alexey-milovidov
9161a5f570
Merge pull request #19875 from ClickHouse/text-log-fix-deadlock 
Fix deadlock in system.text_log
 2021-02-01 15:42:16 +03:00
Alexander Tokmakov
accf4d262e fix logical error 2021-02-01 15:40:57 +03:00
tavplubix
2a20f20f21
Merge pull request #19790 from taiyang-li/fix_client_select 
fix client bug while executing select
 2021-02-01 13:17:14 +03:00
tavplubix
1e44e3f341
Update 01671_ddl_hang_timeout.sh 2021-02-01 12:53:13 +03:00
Maksim Kita
beee7a4f02 Added test to arcadia_skip_list 2021-02-01 12:35:47 +03:00
Maksim Kita
d3f299269f Fixed code review issues 2021-02-01 12:35:47 +03:00
Maksim Kita
d194294d0b Fixed tests 2021-02-01 12:35:47 +03:00
Maksim Kita
dafb0ef4e9 ExecutableDictionarySource added implicit_key option 2021-02-01 12:35:47 +03:00
Maksim Kita
9a1e52a80d Added performance test 2021-02-01 12:25:52 +03:00
Stig Bakken
bedc472bb0 Show MaterializeMySQL tables in system.parts 2021-02-01 16:59:10 +08:00
alexey-milovidov
ef72ba7349
Merge pull request #19861 from amosbird/fixcte 
Fix crash when pushing down predicates to union distinct subquery
 2021-02-01 10:28:41 +03:00
alesapin
d9598c47b4
Merge pull request #19702 from ClickHouse/fix_rare_bug_after_part_corruption 
Fix incorrect virtual_parts after part corruption
 2021-02-01 10:09:48 +03:00
yiguolei
fa03fbdc5b fix functional test 2021-02-01 13:16:54 +08:00
alexey-milovidov
2459777f57
Update 01681_bloom_filter_nullable_column.sql 2021-02-01 07:55:58 +03:00
Alexey Milovidov
1c0c3b3aab Merge branch 'master' into intdiv-ubsan 2021-02-01 07:46:40 +03:00
yiguolei
bef5af3f57 fix functional test 2021-02-01 11:35:02 +08:00
alexey-milovidov
08f713f177
Merge pull request #14822 from vzakaznikov/live_view_periodic_refresh 
Adding support for periodically refreshed LIVE VIEW tables
 2021-02-01 06:11:30 +03:00
yiguolei
768e461d28 add functional test 2021-02-01 10:40:48 +08:00
Alexey Milovidov
efb9293587 Adjust test scale 2021-02-01 05:28:18 +03:00
alexey-milovidov
2193baafde
Merge pull request #19852 from vzakaznikov/update_testflows_to_support_retries_during_container_start 
Adding retries for docker-compose start, stop and restart in TestFlows tests
 2021-02-01 03:52:38 +03:00
alexey-milovidov
7567bb2e31
Merge pull request #19860 from ClickHouse/hyperscan-assert 
Fix debug assertion in Hyperscan
 2021-02-01 03:45:08 +03:00
alexey-milovidov
a6b0fdd3eb
Merge pull request #19862 from ClickHouse/ubsan-gather-utils 
Fix UBSan report in GatherUtils
 2021-02-01 03:43:43 +03:00
alexey-milovidov
9407c5baf4
Merge pull request #19868 from amosbird/fixif 
Fix argMinMaxIf crash
 2021-02-01 03:42:48 +03:00
Alexander Tokmakov
84d3d6e728 fix 'Unmatched parentheses', add test 2021-02-01 02:10:41 +03:00
Maksim Kita
65183f1fea Fixed tests 2021-01-31 23:54:52 +03:00
Maksim Kita
1aa8d7fdfb BloomFilter index crash fix 2021-01-31 21:49:27 +03:00
Maksim Kita
a4812aa97b Updated tests 2021-01-31 18:32:47 +03:00
alexey-milovidov
342baa4223
Merge pull request #19720 from ClickHouse/unicode-asan 
Fix unsufficient args check (trash code) in StringSearcher
 2021-01-31 15:43:05 +03:00
alexey-milovidov
b80ad9f9bc
Merge pull request #19859 from ClickHouse/date-time-add-ubsan 
Fix UBSan report in DateTimeAddInterval
 2021-01-31 14:09:04 +03:00
vdimir
000b51f95a
Merge pull request #19518 from bharatnc/ncb/toIPv6-parses-IPv4 
toIPv6 function parses IPv4 addresses
 2021-01-31 13:56:45 +03:00
Alexey Milovidov
4eb21ca396 Add a test 2021-01-31 13:25:31 +03:00
Alexey Milovidov
051e4832b9 Add a test 2021-01-31 13:05:00 +03:00
Amos Bird
18f3df31d5
Fix crash when pushing down predicates to union distinct subquery 2021-01-31 16:29:29 +08:00