thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-20 08:40:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
91,646 Commits 8,927 Branches 1,765 Tags 2 GiB
0d626982ee
Commit Graph

27783 Commits

Author SHA1 Message Date
Nikolai Kochetov
dcf6e0d060 Roll back some changes from IPartMetadataManager 2022-06-21 13:05:49 +00:00
Alexander Tokmakov
f3ea8ad745 support CREATE EMPTY AS SELECT 2022-06-21 15:03:58 +02:00
Kruglov Pavel
8bf78dbc95
Add comment 2022-06-21 14:32:12 +02:00
Nikolai Kochetov
854d148d73 Add some comments. Remove some commented code. 2022-06-21 12:31:02 +00:00
kssenii
4656dd3de3 Headers for url in ast 2022-06-21 14:25:05 +02:00
Nikolai Kochetov
46c4fd6cf7 Fixing build. 2022-06-21 11:15:06 +00:00
Kruglov Pavel
3c5a627bd7
Merge pull request #38239 from CurtizJ/fix-reading-from-s3 
Fix reading from s3 in some corner cases
 2022-06-21 12:57:55 +02:00
mergify[bot]
f304fc9179
Merge branch 'master' into fix-alter-columns-with-dots 2022-06-21 10:40:53 +00:00
mergify[bot]
bb79eb73e6
Merge branch 'master' into fix-short-circuit 2022-06-21 10:40:07 +00:00
Nikolai Kochetov
1e8c9ecd4c Merge branch 'master' into refactor-something-in-part-volumes 2022-06-21 12:37:21 +02:00
Kruglov Pavel
b9b58b4305
Merge pull request #37759 from Avogar/fix-nothing-error 
Fix possible logical error with type Nothing in some functions
 2022-06-21 12:35:05 +02:00
alesapin
fafcb876be
Merge pull request #38182 from ClickHouse/disk_transaction 
Disk transaction
 2022-06-21 12:28:49 +02:00
Saulius Valatka
160c95a6b8 unpoisson memory read by io_uring, handle interrupted syscalls, do not throw from monitor thread 2022-06-21 13:20:21 +03:00
Saulius Valatka
7b5e13a9eb implement pending request queue to prevent CQ overflows, more error checking, reduce size of io_uring 2022-06-21 13:17:56 +03:00
Nikolai Kochetov
b8d27aa8dd
Merge pull request #37469 from azat/projections-optimize_aggregation_in_order 
Implement in order aggregation (optimize_aggregation_in_order) for projections for tables with fully materialized projections
 2022-06-21 12:17:35 +02:00
Kseniia Sumarokova
71ee5dbbb2
Merge pull request #38227 from kssenii/diff-reduce 
buffer's getFileSize small changes
 2022-06-21 12:12:22 +02:00
Nikolai Kochetov
e45d552f4d Do not create empty folder for in-memory parts. 2022-06-21 09:59:46 +00:00
mergify[bot]
bdd5b94192
Merge branch 'master' into keeper-sequential-consistency-reads 2022-06-21 08:13:04 +00:00
Nikolai Kochetov
5232c17b89 Fix style 2022-06-21 07:30:45 +00:00
Nikolai Kochetov
3429a5e07a Fix 2022-06-21 07:26:43 +00:00
Saulius Valatka
6bbabf59d9 simplify build, add read method randomized tests, fix typos 2022-06-21 09:59:46 +03:00
Saulius Valatka
68eef98128 check for io_uring support, fail if not supported 2022-06-21 09:59:46 +03:00
Saulius Valatka
0dc7854af1 add initial io_uring support 2022-06-21 09:59:46 +03:00
Saulius Valatka
3f755a2505 add liburing to build 2022-06-21 09:59:46 +03:00
Alexander Tokmakov
75b8800b5c
Merge pull request #37822 from ClickHouse/fix_flaky_tests_with_transactions 
Try fix flaky tests with transactions
 2022-06-21 09:38:25 +03:00
Larry Luo
bbd73ba727 use utility methods to access x509 struct fields. 2022-06-20 21:27:33 -04:00
Anton Popov
dcc9cd0367
fix style check 2022-06-20 22:50:39 +02:00
Maksim Kita
cb018348cf
Merge pull request #38022 from kitaisreal/sorting-added-batch-queue-variants 
Sorting added batch queue variants
 2022-06-20 22:35:44 +02:00
zvonand
22af00b757 rename variable + fix handling of ENABLE_LIBRARIES 2022-06-20 23:53:47 +05:00
Nikolai Kochetov
dccf90b1ea Cleanup. 2022-06-20 18:18:17 +00:00
mergify[bot]
9bdd9e14a6
Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-20 18:11:30 +00:00
Dmitry Novik
f6692c34e6
Merge pull request #34632 from excitoon-favorites/optimizedprocessing 
Optimized processing of ORDER BY in window functions
 2022-06-20 20:03:26 +02:00
Kseniia Sumarokova
2d039b739c
Update ReadBufferFromFileDescriptor.cpp 2022-06-20 19:13:20 +02:00
Kseniia Sumarokova
e48ce50863
Update ArrowBufferedStreams.cpp 2022-06-20 19:12:51 +02:00
Anton Popov
464cb59920 slightly better code 2022-06-20 16:22:36 +00:00
Anton Popov
59a9ce6def fix reading from s3 in some corner cases 2022-06-20 15:24:38 +00:00
kssenii
cea81822bd Fix 2022-06-20 17:19:36 +02:00
mergify[bot]
b440ee84ae
Merge branch 'master' into fix-short-circuit 2022-06-20 15:14:19 +00:00
zvonand
d4e5686b99 minor: fix message for base64 2022-06-20 20:13:09 +05:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa 2022-06-20 16:39:32 +02:00
zvonand
78d55d6f46 small fixes 2022-06-20 19:30:54 +05:00
Antonio Andelic
0c76fc1121 Fix style 2022-06-20 14:21:59 +00:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA) 
- TSA is a static analyzer build by Google which finds race conditions
  and deadlocks at compile time.

- It works by associating a shared member variable with a
  synchronization primitive that protects it. The compiler can then
  check at each access if proper locking happened before. A good
  introduction are [0] and [1].

- TSA requires some help by the programmer via annotations. Luckily,
  LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
  std::shared_mutex and std::scoped_lock. This commit enables them
  (--> contrib/libcxx-cmake/CMakeLists.txt).

- Further, this commit adds convenience macros for the low-level
  annotations for use in ClickHouse (--> base/defines.h). For
  demonstration, they are leveraged in a few places.

- As we compile with "-Wall -Wextra -Weverything", the required compiler
  flag "-Wthread-safety-analysis" was already enabled. Negative checks
  are an experimental feature of TSA and disabled
  (--> cmake/warnings.cmake). Compile times did not increase noticeably.

- TSA is used in a few places with simple locking. I tried TSA also
  where locking is more complex. The problem was usually that it is
  unclear which data is protected by which lock :-(. But there was
  definitely some weird code where locking looked broken. So there is
  some potential to find bugs.

*** Limitations of TSA besides the ones listed in [1]:

- The programmer needs to know which lock protects which piece of shared
  data. This is not always easy for large classes.

- Two synchronization primitives used in ClickHouse are not annotated in
  libcxx:
  (1) std::unique_lock: A releaseable lock handle often together with
      std::condition_variable, e.g. in solve producer-consumer problems.
  (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
      considered a design flaw + typically it is slower than a standard
      mutex. In this commit, one std::recursive_mutex was converted to
      std::mutex and annotated with TSA.

- For free-standing functions (e.g. helper functions) which are passed
  shared data members, it can be tricky to specify the associated lock.
  This is because the annotations use the normal C++ rules for symbol
  resolution.

[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
 2022-06-20 16:13:25 +02:00
Antonio Andelic
777ae72259 Merge branch 'master' into keeper-sequential-consistency-reads 2022-06-20 14:06:45 +00:00
Antonio Andelic
df31b562ec Use sync after reconnect to avoid stale reads 2022-06-20 14:05:51 +00:00
Nikolai Kochetov
39b02d4ab8 Continure resolving conflicts. 2022-06-20 15:39:05 +02:00
Nikolai Kochetov
7452d04e3a Merge branch 'master' into refactor-something-in-part-volumes 2022-06-20 15:31:02 +02:00
kssenii
5dd1bb2fd8 improvements for getFileSize 2022-06-20 15:22:56 +02:00
Nikolai Kochetov
ef8b967c3f Fix more tests. 2022-06-20 13:07:14 +00:00
Antonio Andelic
6c225bb04d
Update src/Common/ZooKeeper/ZooKeeperImpl.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-06-20 14:55:59 +02:00
Dmitry Novik
77177917bd
Merge pull request #37848 from ClickHouse/window-function-expression 
Support expressions with window functions
 2022-06-20 14:21:13 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6 
Backup Improvements 6
 2022-06-20 14:13:30 +02:00
alesapin
7ae775d2db Review fixes 2022-06-20 14:01:05 +02:00
alesapin
e3184d264b Merge branch 'master' into disk_transaction 2022-06-20 13:47:30 +02:00
mergify[bot]
4ed606a275
Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-20 10:17:51 +00:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix 
Send profile events for INSERT queries (previously only SELECT was supported)
 2022-06-20 12:16:29 +02:00
Kseniia Sumarokova
c83594284d
Merge pull request #38205 from azat/fix-window-view 
(Window View is an experimental feature) Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
 2022-06-20 11:47:45 +02:00
Nikolai Kochetov
fab62513aa
Merge pull request #38029 from ClickHouse/fix-possible-crash-after-removing-replica-in-distributed 
Fix possible crash in Distributed async insert in case of removing a replica from config.
 2022-06-20 11:38:35 +02:00
Vitaly Baranov
a6fc0dea4e Fix clang-tidy more. 2022-06-20 11:04:37 +02:00
Vladimir Chebotarev
aef6fe6008 Rebase fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
bfb67ded2c Rebase fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
b81df2be83 Update src/Interpreters/ExpressionAnalyzer.cpp 
Co-authored-by: Dmitry Novik <d1mas1k4@yandex.ru>
 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
92a553fb77 Build fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
6a363b7429 Build fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
d41c97ea1d Review fixes. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
4f38e01343 Unused code. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
cc45f15eae Build fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
3c2a63b87a Fix test. 2022-06-20 05:15:07 +03:00
Vladimir Chebotarev
e50210969f Style. 2022-06-20 05:15:07 +03:00
Vladimir Chebotarev
7f9557f8a3 Added optimize_read_in_window_order setting. 2022-06-20 05:15:07 +03:00
Vladimir Chebotarev
ec22f6d539 Draft. 2022-06-20 05:15:07 +03:00
Vitaly Baranov
638ea23399 Fix build. 2022-06-20 03:44:59 +02:00
Vitaly Baranov
2c8788266c Fix tests. 2022-06-20 03:44:49 +02:00
Alexey Milovidov
e2fb85c072 Merge branch 'master' into enable-enable_positional_arguments 2022-06-20 00:37:36 +02:00
mergify[bot]
670a63865e
Merge branch 'master' into window-function-expression 2022-06-19 22:14:54 +00:00
alesapin
ae8108d30b Add comments, better interface 2022-06-19 23:48:46 +02:00
zvonand
832fd6e0a9 Added tests + minor updates 2022-06-19 23:10:28 +05:00
Alexey Milovidov
f024956f62 Merge branch 'master' of github.com:ClickHouse/ClickHouse into revert-35914-FIPS_compliance 2022-06-19 18:59:53 +02:00
Alexey Milovidov
7700c26076
Merge pull request #38196 from ClickHouse/revert-38194-revert-37015-zstd_window_log_max 
Revert "Revert "Add a setting to use more memory for zstd decompression""
 2022-06-19 19:58:03 +03:00
Alexey Milovidov
c941d0c8e2
Merge pull request #33653 from ClickHouse/enable-some-settings 
Enable some settings
 2022-06-19 19:52:56 +03:00
Alexey Milovidov
354a10d1e1
Merge pull request #35075 from azat/dist-insert-hung-fix 
Disable send_logs_level for INSERT into Distributed to avoid possible hung
 2022-06-19 19:39:56 +03:00
alesapin
2e6e3f082a Merge branch 'master' into disk_transaction 2022-06-19 18:21:02 +02:00
Azat Khuzhin
bc3e73d776 Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure 
Caching header of the source table in the WINDOW VIEW should not be
done, since there is no ability to get notification when it had been
changed (ALTER or CREATE/DROP).

And this fires on [CI], when the following tests had been executed in
order in stress tests:
- 01050_window_view_parser_tumble (leaves wm for mt)
- 01748_partition_id_pruning (cache input_header)
- 01188_attach_table_from_path (insert into mt with wm attached and
  incorrect structure)

  [CI]: https://s3.amazonaws.com/clickhouse-test-reports/38056/109980eb275c064d08bc031bfdc14d95b9a7272b/stress_test__undefined__actions_.html

Follow-up for: #37965 (@Vxider)
Fixes: #37815
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-19 19:16:39 +03:00
alesapin
119ce90e5b Fix strange diff 2022-06-19 18:03:43 +02:00
alesapin
08b5dd091b Fix double negative 2022-06-19 17:12:04 +02:00
Vitaly Baranov
8a7c970ce0 Fix style. 2022-06-19 15:58:26 +02:00
alesapin
50801e41c5 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-19 14:05:46 +02:00
Vitaly Baranov
115be82440 DiskAccessStorage is now allowed to backup by default. 2022-06-19 13:16:36 +02:00
Vitaly Baranov
9f197defda Add support for setting 'allow_backup' to skip access entities from putting to backup. 2022-06-19 12:49:50 +02:00
Vitaly Baranov
01aaaf7395 More accurate access checking for RESTORE. 2022-06-19 11:26:41 +02:00
Alexey Milovidov
abced45397 Enable "enable_positional_arguments" 2022-06-19 05:55:12 +02:00
Alexey Milovidov
04b7a34870 Maybe fix error 2022-06-19 05:21:59 +02:00
Alexey Milovidov
0cf88e0950
Revert "ClickHouse's boringssl module updated to the official version of the FIPS compliant." 2022-06-18 23:16:18 +03:00
Alexey Milovidov
67a66721f2 Merge branch 'master' into enable-some-settings 2022-06-18 19:16:23 +02:00
Maksim Kita
b35998d7ea SortCursor improve performance for random data 2022-06-18 18:20:01 +02:00
Maksim Kita
6adf6b1707 Fixed tests 2022-06-18 18:20:01 +02:00
Maksim Kita
282f65c5e2 Fixed performance tests 2022-06-18 18:20:01 +02:00
Maksim Kita
ace900a1f8 Fixed tests 2022-06-18 18:20:01 +02:00
Maksim Kita
dbbf499005 Fixed unit tests 2022-06-18 18:20:01 +02:00
Maksim Kita
33f4b4d834 Fixed tests 2022-06-18 18:20:01 +02:00
Maksim Kita
ef084ad12a Fixed tests 2022-06-18 18:20:00 +02:00
Maksim Kita
fd7e533a09 MergingSortedAlgorithm added batch sorting queue 2022-06-18 18:20:00 +02:00
Maksim Kita
c86191a6b9 SortingTransform add cursor order 2022-06-18 18:20:00 +02:00
Maksim Kita
d82ab22333 Fixed style check 2022-06-18 18:20:00 +02:00
Maksim Kita
3664f02690 Replaced SortingHeap with SortingQueue 2022-06-18 18:20:00 +02:00
Maksim Kita
9670504781 Fixed tests 2022-06-18 18:20:00 +02:00
Maksim Kita
9f55a2cbac Fixed tests 2022-06-18 18:20:00 +02:00
Maksim Kita
d7e3e85d2f Sorting added batch queue variants 2022-06-18 18:20:00 +02:00
Alexey Milovidov
73709b0488
Revert "Revert "Add a setting to use more memory for zstd decompression"" 2022-06-18 15:55:35 +03:00
Antonio Andelic
dd75e9cba5
Merge pull request #38149 from vdimir/multiple_joins_original_names_34697 
Add setting multiple_joins_try_to_keep_original_names
 2022-06-18 14:47:31 +02:00
alesapin
e6b0648f99 Merge branch 'master' into disk_transaction 2022-06-18 14:26:21 +02:00
alesapin
1d7cf28cab
Merge pull request #38190 from ClickHouse/fix_36123 
Check row size to avoid out of bounds access in PostgreSQLSource
 2022-06-18 14:18:35 +02:00
alesapin
16e8b85fbf
Revert "Add a setting to use more memory for zstd decompression" 2022-06-18 14:08:14 +02:00
Vitaly Baranov
de9a07d18d Fix RESTORE ALL for tables without database in backup. 2022-06-18 14:07:01 +02:00
Vitaly Baranov
36475c5b98 Fix handling empty files in backups. 2022-06-18 12:28:32 +02:00
Dmitry Novik
8b917d12f2 Small refactoring 2022-06-18 01:40:30 +00:00
Alexey Milovidov
e20259e9ca
Merge pull request #37015 from wuxiaobai24/zstd_window_log_max 
Add a setting to use more memory for zstd decompression
 2022-06-18 04:19:27 +03:00
Alexey Milovidov
1b9a47f825
Merge pull request #38157 from qoega/remove-trash 
Remove processor description from span attributes - it is not working
 2022-06-18 04:09:13 +03:00
Alexander Gololobov
54e599161d Check row size to avoid out of bounds access 2022-06-18 00:16:45 +02:00
Maksim Kita
7e759cac39
Merge pull request #38184 from kitaisreal/sql-user-defined-function-loader-allow-symlinks 
SQLUserDefinedFunctionsLoader allow symlinks in user_defined directory
 2022-06-17 23:03:52 +02:00
zvonand
f4b3af091d fix zero byte 2022-06-17 23:48:14 +05:00
Antonio Andelic
f72e509b3b
Merge pull request #38052 from amosbird/join_regression_fix 
Fix significant join performance regression
 2022-06-17 19:55:33 +02:00
Vitaly Baranov
a0c558a17e Implement backup/restore for ACL system tables (system.users, system.roles, etc.) 2022-06-17 18:14:31 +02:00
Alexander Tokmakov
83adf56383 fix race 2022-06-17 18:13:57 +02:00
Maksim Kita
ed03bb5dc6
Merge pull request #38127 from iyupeng/2022061601.fix_merge_sorter 
Fix redundant memory reservation for output block in MergeSorter
 2022-06-17 17:50:26 +02:00
Maksim Kita
965b706350 SQLUserDefinedFunctionsLoader allow symlinks in user_defined directory 2022-06-17 17:43:42 +02:00
alesapin
93e465809d Fix style check 2022-06-17 16:49:49 +02:00
Alexander Tokmakov
e9c2157f01 Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-17 15:57:30 +02:00
Alexander Gololobov
f69f24ffbd
Merge pull request #38135 from vdimir/issue_37928 
Try to fix temporary name clash in partial merge join
 2022-06-17 15:10:19 +02:00
mergify[bot]
23e0b16058
Merge branch 'master' into fix-possible-crash-after-removing-replica-in-distributed 2022-06-17 11:50:44 +00:00
Alexander Tokmakov
5ffc38d9b9 fix style check, add comments 2022-06-17 13:47:14 +02:00
alesapin
7524500ca9 Remove logic from disk 2022-06-17 13:45:17 +02:00
avogar
23f48a9fb9 Fix bug with nested short-circuit functions 2022-06-17 11:44:49 +00:00
Maksim Kita
77c519ae08
Merge pull request #38142 from kitaisreal/executable-user-defined-functions-parameters-direct-fix 
Disable parameters for non direct executable user defined functions
 2022-06-17 12:57:27 +02:00
alesapin
e5b405701a Merge branch 'master' into disk_transaction 2022-06-17 12:16:12 +02:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache 
Extract some diff from pr #36171
 2022-06-17 12:07:31 +02:00
Andrey Zvonov
f987f461e5 fix style -- rm unused ErrorCode 2022-06-17 15:00:32 +05:00
Igor Nikonov
baebbc084f
Merge pull request #38027 from ClickHouse/decimal_rounding_fix 
Fix: rounding for Decimal128/Decimal256 with more than 19-digits long scale
 2022-06-17 09:48:18 +02:00
Alexander Gololobov
59d32f8c96
Merge pull request #38132 from Avogar/fix-mapped-array 
Fix logical error in array mapped functions with const nullable column
 2022-06-17 08:49:41 +02:00
mergify[bot]
296905771f
Merge branch 'master' into better-support-gcp 2022-06-17 01:19:11 +00:00
zvonand
c1b2b669ab remove wrong code 2022-06-17 01:52:45 +05:00
Yatsishin Ilya
6fdcac1c9d Remove processor description from span attributes - it is not working anyway. 2022-06-16 19:49:07 +00:00
Maksim Kita
f8f6a95547
Merge pull request #38144 from kitaisreal/sort-description-compile-fix-typo 
SortDescription compile fix typo
 2022-06-16 20:16:14 +02:00
Alexander Tokmakov
39c0219c11 fixes 2022-06-16 19:41:32 +02:00
Dmitry Novik
500895306f Fix typo 2022-06-16 16:06:59 +00:00
Dmitry Novik
644399db8a Add comment and update docs 2022-06-16 15:59:51 +00:00
vdimir
2a9942ee80
Add setting multiple_joins_try_to_keep_original_names 2022-06-16 15:50:03 +00:00
Dmitry Novik
8479d880f3 Remove redundant code 2022-06-16 15:49:49 +00:00
Dmitry Novik
376412e417 Small refactoring 2022-06-16 15:41:04 +00:00
alesapin
ca33ff93cb
Merge pull request #37555 from ClickHouse/revert-37534-revert-37036-keeper-preprocess-operations 
Add support for preprocessing ZooKeeper operations and real-time digest in `clickhouse-keeper`
 2022-06-16 17:29:00 +02:00