Commit Graph

93261 Commits

Author SHA1 Message Date
Azat Khuzhin
18eb8b6d48 Fix UB (stack-use-after-scope) in extactAll()
After #37544 OptimizedRegularExpressionImpl started to be moved, but
StringSearcher is not copyable since it holds pointers that goes out of
scope after move (before Regexps::get() returns std::shared_ptr<Regexp>
but it had been replaced with Regexps::createRegexp() that returns
Regexp object).

<details>

<summary>ASan report</summary>

    ==48348==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fff577239a9 at pc 0x00001518209b bp 0x7fff57723820 sp 0x7fff57723818
    READ of size 1 at 0x7fff577239a9 thread T0
        0 0x1518209a in char8_t const* DB::StringSearcher<true, true>::search<char8_t>(char8_t const*, char8_t const*) const /bld/./src/Common/StringSearcher.h:730:41
        1 0x1518dd3f in char8_t const* DB::StringSearcher<true, true>::search<char8_t>(char8_t const*, unsigned long) const /bld/./src/Common/StringSearcher.h:751:16
        2 0x1518dd3f in OptimizedRegularExpressionImpl<false>::match(char const*, unsigned long, std::__1::vector<OptimizedRegularExpressionDetails::Match, std::__1::allocator<OptimizedRegularExpressionDetails::Match> >&, unsigned int) const /bld/./src/Common/OptimizedRegularExpression.cpp:463:54
        3 0x1811cb42 in DB::ExtractAllImpl::get(char const*&, char const*&) /bld/./src/Functions/FunctionsStringArray.h:588:18
        4 0x1811aa62 in DB::FunctionTokens<DB::ExtractAllImpl>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const /bld/./src/Functions/FunctionsStringArray.h:704:30
        5 0x14fe17b4 in DB::IFunction::executeImplDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const /bld/./src/Functions/IFunction.h:409:16

    Address 0x7fff577239a9 is located in stack of thread T0 at offset 201 in frame
        0 0x1518d98f in OptimizedRegularExpressionImpl<false>::match(char const*, unsigned long, std::__1::vector<OptimizedRegularExpressionDetails::Match, std::__1::allocator<OptimizedRegularExpressionDetails::Match> >&, unsigned int) const /bld/./src/Common/OptimizedRegularExpression.cpp:439

</details>

CI: https://s3.amazonaws.com/clickhouse-test-reports/39342/c6f7698f9ad6ae22199182ebf7c3b2dac77d69d8/fuzzer_astfuzzerasan,actions//report.html
Fixes: #37544 (cc @rschu1ze)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-20 10:18:44 +03:00
Alexander Tokmakov
988569bdb1
Merge pull request #39366 from ClickHouse/fix_race_in_replication_queue
Do not enqueue uneeded parts for check
2022-07-20 00:35:02 +03:00
Sergei Trifonov
499818751e
Merge pull request #39280 from ClickHouse/avg-cpu-progress
fix cpu usage metric in client
2022-07-19 15:17:44 +02:00
Robert Schulze
7146685f9d
Merge pull request #39299 from azat/fix-lsan-v3
[RFC] Fix LSan by fixing getauxval()
2022-07-19 14:27:41 +02:00
Alexander Tokmakov
a761da9f1b add comment 2022-07-19 13:58:59 +02:00
alesapin
bdc09c8319
Merge pull request #39303 from ClickHouse/whitespaces
Whitespaces
2022-07-19 13:17:29 +02:00
Alexander Tokmakov
3e7414b356 add comment 2022-07-19 12:47:20 +02:00
Kruglov Pavel
12221cffc9
Merge pull request #39071 from jiahui-97/parse_timedelta
implementation of parseTimeDelta function
2022-07-19 12:39:56 +02:00
Alexander Tokmakov
acb1fa53ed
Merge pull request #39341 from ClickHouse/fix_memory_exceptions_with_transactions
Fix memory exceptions with transactions
2022-07-19 13:31:15 +03:00
Alexander Tokmakov
1fff3b24b8
Merge pull request #39337 from ClickHouse/minor_fix_for_ordinary_database
Do not start on unexpected Ordinary metadata
2022-07-19 13:30:56 +03:00
Alexander Tokmakov
115c8d9b5f
Merge pull request #39339 from ClickHouse/fix_flaky_01174_select_insert_isolation
Fix flaky 01174_select_insert_isolation
2022-07-19 13:29:27 +03:00
Nikolai Kochetov
d0db7135cb
Merge pull request #38946 from amosbird/nullable-partition-fix1
Fix incorrect partition pruning when there is a nullable partition
2022-07-19 12:14:36 +02:00
Kseniia Sumarokova
f168e0c5d0
Merge pull request #39272 from kssenii/fixes-for-materialized-pg
Fix segfault in materialised postgresql
2022-07-19 11:56:10 +02:00
Nikolay Degterinsky
142701223d
Merge pull request #39361 from ClickHouse/Avogar-patch-1
Fix typo
2022-07-19 11:50:49 +02:00
Vitaly Baranov
6b03f200e4
Merge pull request #39322 from vitlibar/auto-set-test-name-in-integration-tests
Auto set test name in integration tests
2022-07-19 10:44:38 +02:00
Mikhail f. Shiryaev
617f774829
Merge pull request #39235 from ClickHouse/backport-script
Backport script
2022-07-19 10:36:24 +02:00
jiahui-97
e7af88b688 implementation of parseTimeDelta function
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
2022-07-19 09:33:02 +08:00
Alexander Tokmakov
d82f378a9d do not enqueue uneeded parts for check 2022-07-18 23:37:07 +02:00
Kruglov Pavel
1513285166
Fix typo 2022-07-18 20:54:13 +02:00
Kruglov Pavel
07210c7b55
Merge pull request #39360 from ClickHouse/Avogar-patch-1
Fix typo
2022-07-18 20:53:33 +02:00
Vitaly Baranov
725ad5bf22
Update cluster.py 2022-07-18 20:49:30 +02:00
Alexander Tokmakov
ffc50fe6ab
Merge pull request #39334 from ClickHouse/try_fix_flaky_test_store_cleanup
Try fix flaky test_store_cleanup
2022-07-18 21:44:40 +03:00
Kruglov Pavel
9dd370e2ec
Fix typo 2022-07-18 20:15:18 +02:00
Robert Schulze
6df3c9d799
Merge pull request #39300 from ClickHouse/stringref-to-stringview
First try at reducing the use of StringRef
2022-07-18 19:20:16 +02:00
Alexander Tokmakov
bc9c359abe
Update 01174_select_insert_isolation.sh 2022-07-18 18:47:35 +03:00
Alexander Tokmakov
51c4ffad8c block memory exceptions in TransactionLog 2022-07-18 17:40:09 +02:00
Alexander Tokmakov
64e2e10e2b
Merge pull request #39318 from ClickHouse/fix_build_clang_13
Fix build clang-13
2022-07-18 18:33:24 +03:00
Alexander Tokmakov
ae96911735 fix flaky 01174_select_insert_isolation 2022-07-18 17:31:36 +02:00
Alexander Tokmakov
225a5c3c7e
Update tests/integration/helpers/cluster.py 2022-07-18 18:19:02 +03:00
Alexander Tokmakov
23312164bf do not start on unexpected Ordinary metadata 2022-07-18 17:05:03 +02:00
Dmitry Novik
855ad30a5d
Merge pull request #39305 from ClickHouse/window-case-test
Add test for window function inside CASE
2022-07-18 16:10:11 +02:00
Alexander Tokmakov
dfccb677f5 try fix flaky test_store_cleanup 2022-07-18 16:03:06 +02:00
Azat Khuzhin
f187c3499a Fix getauxval() by using procfs with fallback to environ
getauxval() from glibc-compatibility did not work always correctly:

- It does not work after setenv(), and this breaks vsyscalls,
  like sched_getcpu() [1] (and BaseDaemon.cpp always set TZ if timezone
  is defined, which is true for CI [2]).

  Also note, that fixing setenv() will not fix LSan,
  since the culprit is getauxval()

  [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1163404
  [2]: ClickHouse#32928 (comment)

- Another think that is definitely broken is LSan (Leak Sanitizer), it
  relies on worked getauxval() but it does not work if __environ is not
  initialized yet (there is even a commit about this).

  And because of, at least, one leak had been introduced [3]:

    [3]: ClickHouse#33840

Fix this by using /proc/self/auxv with fallback to environ solution to
make it compatible with environment that does not allow reading from
auxv (or no procfs).

v2: add fallback to environ solution
v3: fix return value for __auxv_init_procfs()
Refs: #33957
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 16:49:49 +03:00
Azat Khuzhin
51e7c41883 Add a test for LSan
v2: fix type check
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 16:49:49 +03:00
Kseniia Sumarokova
b4eae94889
Merge pull request #39259 from rnbondarenko/patch-1
removed default queue arguments
2022-07-18 15:32:31 +02:00
Robert Schulze
33aa9593ca
Merge pull request #39265 from jrdi/fix-macosx-endian
Fix macosx compilation due to endian.h
2022-07-18 15:32:17 +02:00
Robert Schulze
e2aa2d9850
Merge pull request #39295 from azat/cmake-cache-fix
Fix running cmake with predefined cache (for development only)
2022-07-18 15:31:18 +02:00
Kseniia Sumarokova
a75cead5d0
Merge pull request #39314 from azat/disks-leak
Fix leaking of logger in clickhouse-disks
2022-07-18 15:30:35 +02:00
Robert Schulze
74e55e42f6
Merge pull request #39167 from ClickHouse/multiStringAllPositions-non-const-needle
multiStringAllPositions() with non-const needle
2022-07-18 15:30:14 +02:00
Vitaly Baranov
ad115a9a3d
Merge pull request #39254 from vitlibar/backup-checks-destination-not-in-use
Check that the destination for a backup is not in use.
2022-07-18 15:19:43 +02:00
Vitaly Baranov
e67a447fc3 Fix style. 2022-07-18 15:17:53 +02:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings
Add more settings for randomization
2022-07-18 15:05:41 +02:00
Vitaly Baranov
bd563c210e Update tests/integration/helpers/cluster.py
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2022-07-18 15:04:03 +02:00
Vitaly Baranov
15882b1d86 Auto set test's name in integration tests. 2022-07-18 15:03:57 +02:00
Alexander Tokmakov
966a59e617
Merge pull request #39315 from ClickHouse/tavplubix-patch-1
Update exception message
2022-07-18 15:16:49 +03:00
Vitaly Baranov
74a0f67676 Fix setting name in integration tests. 2022-07-18 13:46:26 +02:00
Sergei Trifonov
30cb4e168e cleanup 2022-07-18 13:31:32 +02:00
Sergei Trifonov
0ffe5bd39c fix typo and style 2022-07-18 13:26:33 +02:00
Robert Schulze
1ced1d11fd
Merge pull request #39285 from azat/bg-exe
[RFC] Remove superior atomic from MergeTreeBackgroundExecutor and annotations for TSA
2022-07-18 12:46:54 +02:00
alesapin
00e5bdb84a Fix build clang-13 2022-07-18 12:44:58 +02:00