thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
93,214 Commits 9,315 Branches 1,783 Tags 2.1 GiB
a6655bea1c
Commit Graph

3789 Commits

Author SHA1 Message Date
kssenii
a6655bea1c Some fixes 2022-07-18 14:23:08 +02:00
kssenii
8ad26b3111 Merge master 2022-07-16 22:50:03 +02:00
Kseniia Sumarokova
3f43641b18
Update MergedColumnOnlyOutputStream.cpp 2022-07-16 16:32:33 +02:00
alesapin
5cf4bd50de
Merge pull request #39214 from azat/s3-fix-fetching-memory-part 
Fix fetch of in-memory part with allow_remote_fs_zero_copy_replication
 2022-07-15 17:52:05 +02:00
Alexander Tokmakov
5ef7f5f494
Merge pull request #39229 from azat/fix-noexcept-scope 
[RFC] Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
 2022-07-15 18:14:56 +03:00
Kruglov Pavel
a944a92d4c
Merge pull request #39224 from Avogar/string-view-by-value 
Pass const std::string_view by value, not by reference
 2022-07-15 13:05:56 +02:00
Alexander Tokmakov
435d98738d
Update ReplicatedMergeTreeQueue.cpp 2022-07-14 22:41:01 +03:00
avogar
9291d33080 Pass const std::string_view & by value, not by reference 2022-07-14 16:11:57 +00:00
Azat Khuzhin
b20407fab9 Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception) 
Current implementation of NOEXCEPT_SCOPE will not work, you cannot
rethrow exception outside the catch block, this will simply terminate
(via std::terminate) the program.
In other words NOEXCEPT_SCOPE macro will simply call std::terminate on
exception and will lost original exception.

But if NOEXCEPT_SCOPE will accept the code that should be runned w/o
exceptions, then it can catch exception and log it, rewrite it in this
way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-14 17:16:18 +03:00
Azat Khuzhin
3e77b80b88 Fix fetch of in-memory part with allow_remote_fs_zero_copy_replication 
CI founds the following error during trying to fetch in-memory part [1]:

    2022.07.13 08:06:54.231033 [ 141093 ] {} <Error> InterserverIOHTTPHandler: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/disks/s3/store/886/88635b40-e4e3-4fe6-a0a0-1b6755463358/all_0_0_0/data.bin, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

<details>

<summary>stacktrace</summary>

    2022.07.13 08:06:54.101825 [ 189342 ] {8dbd11b3-f38a-4d5d-9ded-148987adb71d} <Debug> executeQuery: (from [::1]:54570) (comment: 01643_replicated_merge_tree_fsync_smoke.sql) select 'memory in_memory_parts_insert_sync'; (stage: Complete)
    2022.07.13 08:06:54.131309 [ 691 ] {} <Debug> test_26u6kx.rep_fsync_r2 (39c3823c-22e5-4c05-9dec-cdffd8872c40): Fetching part all_0_0_0 from /clickhouse/tables/test_26u6kx/rep_fsync/replicas/r1
    2022.07.13 08:06:54.231033 [ 141093 ] {} <Error> InterserverIOHTTPHandler: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/disks/s3/store/886/88635b40-e4e3-4fe6-a0a0-1b6755463358/all_0_0_0/data.bin, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
    0. DB::Exception::Exception() @ 0xba0191a in /usr/bin/clickhouse
    1. DB::throwFromErrnoWithPath() @ 0xba029ca in /usr/bin/clickhouse
    2. DB::OpenedFile::open() const @ 0x156e7fb0 in /usr/bin/clickhouse
    3. DB::OpenedFile::getFD() const @ 0x156e8003 in /usr/bin/clickhouse
    4. DB::ReadBufferFromFilePReadWithDescriptorsCache::ReadBufferFromFilePReadWithDescriptorsCache() @ 0x156e5f23 in /usr/bin/clickhouse
    5. ? @ 0x156e53f0 in /usr/bin/clickhouse
    6. DB::createReadBufferFromFileBase() @ 0x156e52b5 in /usr/bin/clickhouse
    7. DB::DiskLocal::readFile() const @ 0x15e45ea8 in /usr/bin/clickhouse
    8. DB::MetadataStorageFromDisk::readFileToString() const @ 0x15e6ab8b in /usr/bin/clickhouse
    9. DB::MetadataStorageFromDisk::readMetadataUnlocked() const @ 0x15e6cdeb in /usr/bin/clickhouse
    10. DB::MetadataStorageFromDisk::getSerializedMetadata() const @ 0x15e6cfc4 in /usr/bin/clickhouse
    11. DB::DiskObjectStorage::getSerializedMetadata() const @ 0x15e19e2e in /usr/bin/clickhouse
    12. DB::DiskDecorator::getSerializedMetadata() const @ 0x15e54ed1 in /usr/bin/clickhouse
    13. DB::DiskDecorator::getSerializedMetadata() const @ 0x15e54ed1 in /usr/bin/clickhouse
    14. DB::DataPartsExchange::Service::sendPartFromDiskRemoteMeta() @ 0x1700bb9e in /usr/bin/clickhouse
    15. DB::DataPartsExchange::Service::processQuery(DB::HTMLForm const&, DB::ReadBuffer&, DB::WriteBuffer&, DB::HTTPServerResponse&) @ 0x1700a649 in /usr/bin/clickhouse
    16. DB::InterserverIOHTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTTPServerResponse&, DB::InterserverIOHTTPHandler::Output&) @ 0x17433c53 in /usr/bin/clickhouse
    17. DB::InterserverIOHTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x174344f1 in /usr/bin/clickhouse
    18. DB::HTTPServerConnection::run() @ 0x1768714d in /usr/bin/clickhouse
    19. Poco::Net::TCPServerConnection::start() @ 0x1a398093 in /usr/bin/clickhouse
    20. Poco::Net::TCPServerDispatcher::run() @ 0x1a399411 in /usr/bin/clickhouse
    21. Poco::PooledThread::run() @ 0x1a54b7bb in /usr/bin/clickhouse
    22. Poco::ThreadImpl::runnableEntry(void*) @ 0x1a548ec0 in /usr/bin/clickhouse
    23. ? @ 0x7fdf1c204609 in ?
    24. clone @ 0x7fdf1c129133 in ?
     (version 22.7.1.1781 (official build))

</details>

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/8b6e31cc615ca52c80724b6e5097777cb9514f07/stateless_tests__release__s3_storage__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-14 11:43:35 +03:00
kssenii
615221f73b Minor changes 2022-07-13 19:22:47 +02:00
kssenii
0944183eec Fix merge 2022-07-13 16:50:31 +02:00
Azat Khuzhin
3720a3843c Add merge_algorithm to system.part_log 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-13 17:16:25 +03:00
kssenii
0d2d3ac2b4 Merge master 2022-07-13 13:26:15 +02:00
Nikolai Kochetov
8efbe6d44d
Merge pull request #39119 from ClickHouse/removed-projection-and-old-part 
Properly remove projection from part in case it was removed from table metadata.
 2022-07-12 16:24:30 +02:00
Robert Schulze
69de9ee0e8
Merge pull request #39108 from ClickHouse/prefix_cmake_commands 
Prefix overridden add_executable() command with "clickhouse_"
 2022-07-12 15:05:48 +02:00
Alexander Tokmakov
c99fe6e6c8
Merge branch 'master' into removed-projection-and-old-part 2022-07-12 14:00:55 +03:00
Alexander Tokmakov
b35ad0dbdc
Merge pull request #39091 from ClickHouse/tavplubix-patch-1 
Follow-up to #39036
 2022-07-12 13:47:37 +03:00
Nikolai Kochetov
945b5d4b3f
Merge pull request #38993 from azat/fix-fsync_part_directory-on-fetch 
Fix fsync_part_directory for fetches
 2022-07-12 11:48:42 +02:00
Nikolai Kochetov
cd4211298e Fix typo. 2022-07-12 09:37:00 +00:00
Nikolai Kochetov
03d9301425 Fix log. 2022-07-12 08:31:09 +00:00
Nikolai Kochetov
631be9219b Properly remove projection from part in case it was removed from table metadata. 2022-07-12 08:28:22 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_" 
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
 2022-07-11 19:36:18 +02:00
Alexander Tokmakov
e9ddbc247f
Merge pull request #39016 from Algunenano/clearOldLogs_zkerror 
clearOldLogs: Don't report KEEPER_EXCEPTION on concurrent deletes
 2022-07-11 15:16:18 +03:00
Alexander Tokmakov
048d27189e
Update MergeTreeData.cpp 2022-07-11 15:07:31 +03:00
Alexander Tokmakov
51130d723c
Update MergeTreeData.cpp 2022-07-08 20:17:29 +03:00
Raúl Marín
a965438eea clearOldLogs: Don't report errors on concurrent deletes 2022-07-08 13:57:53 +02:00
Azat Khuzhin
b2f7f9c0f0 Fix fsync_part_directory for fetches 
Before it produces the following error:

    2022.07.07 23:40:03.131044 [ 44869 ] {} <Error> default.rep_data: auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(DB::StorageReplicatedMergeTree::LogEntryPtr &) const: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/data/tmp-fetch_36_25458_25458_0, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

    0. Poco::Exception::Exception() @ 0x1764634c in /usr/bin/clickhouse
    1. DB::Exception::Exception() @ 0xb42caba in /usr/bin/clickhouse
    2. DB::throwFromErrnoWithPath() @ 0xb42d79b in /usr/bin/clickhouse
    3. DB::LocalDirectorySyncGuard::LocalDirectorySyncGuard() @ 0x1404bbd4 in /usr/bin/clickhouse
    4. DB::DiskLocal::getDirectorySyncGuard() const @ 0x140466a4 in /usr/bin/clickhouse
    5. DB::DataPartsExchange::Fetcher::downloadPartToDisk() @ 0x15122af1 in /usr/bin/clickhouse
    6. DB::DataPartsExchange::Fetcher::fetchPart() @ 0x1511de39 in /usr/bin/clickhouse

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-08 13:48:51 +03:00
Raúl Marín
33ffb71453 Avoid 'Node exists' errors when creating Ephemeral Locks 2022-07-07 16:10:04 +02:00
Alexander Tokmakov
fdd2db4ace
Move check for denied allocations (#38858) 
* move check for denied allocations

* remove include

* make hardening softer
 2022-07-07 14:24:36 +02:00
Vitaly Baranov
ed27987646
Merge pull request #38861 from vitlibar/backup-improvements-9 
Backup Improvements 9
 2022-07-07 02:24:47 +02:00
Vitaly Baranov
7f84cf3968 Fix style. 2022-07-06 16:36:59 +02:00
Maksim Kita
b94489d52c
Merge pull request #38859 from kitaisreal/merge-tree-merge-disable-batch-optimization 
MergeTree merge disable batch optimization
 2022-07-06 15:59:40 +02:00
Raúl Marín
86d1c319f4 Remove unnecessary log 2022-07-06 12:23:45 +02:00
Maksim Kita
bdc21737d5 MergeTree merge disable batch optimization 2022-07-05 16:15:00 +02:00
Vitaly Baranov
43d35eec1b Write unfinished mutations to backup. 2022-07-05 14:51:09 +02:00
kssenii
b52084265c Merge master 2022-07-04 21:37:43 +02:00
Dmitry Novik
c66e2c8095
Merge pull request #37155 from amosbird/projection-fix-two 
Fix intermediate header in aggregate projection
 2022-07-04 16:32:34 +02:00
Vitaly Baranov
1dba3d9f9c Make temporary files in better places. 2022-07-04 11:02:58 +02:00
Vitaly Baranov
92e0ee0b6f More detailed error messages. 2022-07-03 14:20:19 +02:00
alesapin
b2db49dbf1
Merge branch 'master' into better_data_part_storage_builder 2022-07-02 15:39:44 +02:00
alesapin
0d7298f3a9 Fix typos 2022-07-01 15:26:27 +02:00
alesapin
57284e6a9d Fix possible deadlocks with MergeTreeData::Transaction 2022-07-01 15:16:32 +02:00
Vitaly Baranov
cadb496894
Merge pull request #38537 from vitlibar/backup-improvements-8 
Backup Improvements 8
 2022-07-01 11:20:46 +02:00
Amos Bird
d73f975849
Fix intermediate header in aggregate projection 2022-07-01 16:47:32 +08:00
alesapin
09faa1641e Some comments 2022-06-30 23:09:32 +02:00
alesapin
979565edf0 Merge branch 'master' into better_data_part_storage_builder 2022-06-30 22:54:42 +02:00
alesapin
eb5046ab26 Simplify everything 2022-06-30 22:51:27 +02:00
Alexander Tokmakov
d2cbdc7c53
Update ReplicatedMergeTreeQueue.cpp 2022-06-30 17:23:53 +03:00
Vitaly Baranov
e367d96964 Fix style. 2022-06-30 15:10:33 +02:00