Commit Graph

1241 Commits

Author SHA1 Message Date
Alexey Milovidov
78d3079f0c Non-significant changes 2022-08-09 21:32:42 +02:00
Alexander Tokmakov
d9190a8121
Merge pull request #39872 from ClickHouse/store_cleanup_all_disks
Cleanup unused dirs from `store/` on all disks
2022-08-05 12:21:59 +03:00
Kseniia Sumarokova
bd5610b732
Merge branch 'master' into update-azure-lib 2022-08-03 23:00:57 +02:00
Alexander Tokmakov
b8ffaabdd8 add chmod and stat to IDisk 2022-08-03 19:47:36 +02:00
Anton Popov
2a5b023b0f
Merge pull request #39800 from CurtizJ/fix-cannot-read-all-data
Fix `CANNOT_READ_ALL_DATA` with `pread_threadpool`.
2022-08-02 16:06:13 +02:00
Alexey Milovidov
b1919d045f
Update ThreadPoolReader.cpp 2022-08-02 05:34:14 +03:00
Anton Popov
64cbecf0c8 fix build on non linux systems 2022-08-02 00:13:20 +00:00
Anton Popov
43e8ca5ba8 fix CANNOT_READ_ALL_DATA with pread_threadpool 2022-08-01 18:40:21 +00:00
Azat Khuzhin
3e627e2861 Add profile events for fsync
The following new provile events had been added:

- FileSync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.
- DirectorySync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.
- FileSyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.
- DirectorySyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.

v2: rewrite test to sh with retries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-31 23:19:30 +03:00
kssenii
687f0fabfc Move some dependencies 2022-07-31 20:17:22 +03:00
Alexander Tokmakov
344b69dea4
Merge pull request #39183 from azat/default-disk-check
Forbid defining non-default disk with default path from <path>
2022-07-27 15:14:50 +03:00
Kseniia Sumarokova
77ddb8eda0
Merge pull request #39506 from CurtizJ/fix-direct-io
Fix reading from async read buffer with direct io
2022-07-23 06:08:48 +02:00
Anton Popov
c1a5514e46 fix reading from async read buffer with direct io 2022-07-22 16:58:30 +00:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code
Remove some dead and commented code
2022-07-22 13:24:34 +03:00
Alexander Tokmakov
a8da5d96fc remove some dead and commented code 2022-07-21 15:05:48 +02:00
Nikolai Kochetov
91043351aa Fixing build. 2022-07-20 20:30:16 +00:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings
Add more settings for randomization
2022-07-18 15:05:41 +02:00
alesapin
afd5c40335
Merge pull request #39190 from ClickHouse/fix_race_with_concurrent_hardlinks
Fix extremely rare race condition in hardlinks creation
2022-07-16 13:57:02 +02:00
Anton Popov
76eec62469
Merge branch 'master' into randomize-more-settings 2022-07-15 14:12:49 +02:00
alesapin
687961cf42 Fix style 2022-07-15 00:38:58 +02:00
alesapin
f0d4a5c93a Merge branch 'master' into fix_race_with_concurrent_hardlinks 2022-07-14 19:03:58 +02:00
Sergei Trifonov
b2317ff710 renaming to be consistent 2022-07-14 17:33:22 +02:00
Anton Popov
d4cb2184c2
Merge branch 'master' into randomize-more-settings 2022-07-14 17:01:07 +02:00
Sergei Trifonov
8a693469f0 Merge branch 'master' into remote-io-bandwidth-limits 2022-07-14 12:38:35 +02:00
Sergei Trifonov
af4a5a5f3a fix conflicts 2022-07-13 19:44:29 +02:00
Kseniia Sumarokova
61ce5161f3
Merge pull request #39106 from kssenii/cache-cacth-disk-full-on-cache-write
fs cache: continue without exception when getting no disk space left error
2022-07-13 19:08:13 +02:00
Sergei Trifonov
f854507729 merge master and resolve conflicts 2022-07-13 19:07:28 +02:00
alesapin
f81650e94a Fix extremely rare condition in hardlinks creation 2022-07-13 19:03:48 +02:00
Azat Khuzhin
ad6b50b087 Forbid defining non-default disk with default path from <path>
Suppose you have the following configuration:

    <path>/var/lib/clickhouse/</path>
    <storage_configuration>
        <disks>
            <data><path>/var/lib/clickhouse/</path></data>
        </disks>
        <policies>
            <default>
                <volumes>
                    <main>
                        <disk>data</disk>
                    </main>
                </volumes>
            </default>
        </policies>
    </storage_configuration>

In this case disks will have two disks:
- 'data'    disk with path '/var/lib/clickhouse/'
- 'default' disk with path '/var/lib/clickhouse/'

And in this case MergeTree engine will complain on ATTACH for table that
uses 'default' policy:

    2022.06.20 07:49:15.165393 [ 242 ] {e8f50978-218a-426f-babc-637a8d03b1c6} <Error> TCPHandler: Code: 479. DB::Exception: Part `0_0_0_1` was found on disk `default` which is not defined in the storage policy. (UNKNOWN_DISK), Stack trace (when copying this message, always include the lines below):

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 17:40:08 +03:00
Kseniia Sumarokova
d6c145e5e1
Update CachedReadBufferFromRemoteFS.cpp 2022-07-13 11:48:17 +02:00
kssenii
9973fb2f1d Fix merge 2022-07-13 11:19:14 +02:00
kssenii
6f29eada80 Merge master 2022-07-12 19:28:18 +02:00
Robert Schulze
8b6e31cc61
Merge pull request #37882 from excitoon-favorites/nodeleteobjects
Fixes for objects removal in `S3ObjectStorage`
2022-07-12 19:09:55 +02:00
kssenii
c38f726537 Remove redundant code 2022-07-12 18:28:20 +02:00
kssenii
4d48d6e713 STUPID BUG 2022-07-12 18:27:22 +02:00
kssenii
112a764794 Process no disk space left
:wq# Please enter the commit message for your changes. Lines starting
2022-07-11 22:12:40 +02:00
kssenii
e8daa50bad Fix tests 2022-07-11 21:24:18 +02:00
Sergei Trifonov
6ed777d2a8 add azure support 2022-07-11 19:19:57 +02:00
Kseniia Sumarokova
00dedc4e00
Merge pull request #39092 from ClickHouse/kssenii-patch-4
Update registerDiskS3.cpp
2022-07-11 18:05:28 +02:00
Sergei Trifonov
add0d1698e respect new limits in HDFS 2022-07-11 17:30:23 +02:00
Kseniia Sumarokova
e82485f65a
Update DiskObjectStorageTransaction.cpp 2022-07-11 16:50:09 +02:00
Sergei Trifonov
43779ec280 add max_remote_{read,write}_network_bandwidth_for_server settings 2022-07-11 14:59:39 +02:00
Kseniia Sumarokova
24f933cf75
Update registerDiskS3.cpp 2022-07-11 14:21:18 +02:00
Kseniia Sumarokova
3998a10c01
Update registerDiskS3.cpp 2022-07-11 14:19:09 +02:00
Vladimir Chebotarev
92880d7840 Fixed test. 2022-07-11 08:31:58 +03:00
kssenii
dd9651a6ca Better 2022-07-10 22:42:16 +02:00
Kseniia Sumarokova
18f93b15fc
Merge pull request #39070 from kssenii/fix-cache-bug
Fix for fs cache
2022-07-10 16:54:08 +02:00
kssenii
7501d5e59c Fix 2022-07-10 12:42:43 +02:00
Anton Popov
28d9fc44cf fix reading with mmap from Log storages 2022-07-09 23:16:16 +00:00
kssenii
4c88527c8e Review fixes 2022-07-09 19:35:07 +02:00