Fix processing pending batch for Distributed async INSERT after restart

After abnormal server restart current_batch.txt (that contains list of
files to send to the remote shard), may not have all files, if it was
terminated between unlink .bin files and truncation of current_batch.txt

But it should be fixed in a more reliable way, though to backport the
patch I kept it simple.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
This commit is contained in:
Azat Khuzhin 2023-05-15 15:38:50 +02:00
parent 25912a2673
commit f2a023140e

View File

@ -201,6 +201,14 @@ void DistributedAsyncInsertBatch::sendBatch()
{
for (const auto & file : files)
{
/// In case of recovery it is possible that some of files will be
/// missing, if server had been restarted abnormally
if (recovered && !fs::exists(file))
{
LOG_WARNING(parent.log, "File {} does not exists, likely due abnormal shutdown", file);
continue;
}
ReadBufferFromFile in(file);
const auto & distributed_header = DistributedAsyncInsertHeader::read(in, parent.log);