thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 08:32:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
130,825 Commits 9,168 Branches 1,781 Tags 2.1 GiB
217570a165
Commit Graph

31 Commits

Author SHA1 Message Date
Azat Khuzhin
4179ff7111 Adjust the test_storage_delta::test_multiple_log_files after MinIO upgrade 
After upgrade MinIO stopped handling of objects with leading slash:

    (Pdb) p minio_client.fput_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json', file_path='/test_multiple_log_files/_delta_log/00000000000000000001.json').http_headers
    HTTPHeaderDict({'Accept-Ranges': 'bytes', 'Content-Length': '0', 'ETag': '"f1f3276c787b3f6724b9a8edb313c4fa"', 'Server': 'MinIO', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Vary': 'Origin, Accept-Encoding', 'X-Amz-Id-2': '7987905dee74cdeb212432486a178e511309594cee7cb75f892cd53e35f09ea4', 'X-Amz-Request-Id': '178B9B0904A9640D', 'X-Content-Type-Options': 'nosniff', 'X-Xss-Protection': '1; mode=block', 'Date': 'Fri, 06 Oct 2023 19:22:27 GMT'})
    (Pdb) p list(minio_client.list_objects('root', '/test_multiple_log_files/_delta_log/'))
    []
    (Pdb) p minio_client.get_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json').read()
    b'{"commitInfo":{"timestamp":1696618985577,"operation":"WRITE","operationParameters":{"mode":"Overwrite","partitionBy":"[]"},"readVersion":0,"isolationLevel":"Serializable","isBlindAppend":false,"operationMetrics":{"numFiles":"1","numOutputRows":"100","numOutputBytes":"2459"},"engineInfo":"Apache-Spark/3.3.2 Delta-Lake/2.3.0","txnId":"2a69a4c9-f72e-4ac9-9374-bdff96707edf"}}\n{"add":{"path":"part-00000-9ff4728b-d5fe-49ac-a773-e2d6e01f4139-c000.parquet","partitionValues":{},"size":2459,"modificationTime":1696618985310,"dataChange":true,"stats":"{\\"numRecords\\":100,\\"minValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"1\\"},\\"maxValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":99,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"99\\"},\\"nullCount\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":0}}"}}\n{"remove":{"path":"part-00000-7c29f5d9-e14d-419a-a273-58d07775e1e9-c000.parquet","deletionTimestamp":1696618985576,"dataChange":true,"extendedFileMetadata":true,"partitionValues":{},"size":2459}}\n'
    (Pdb) p list(minio_client.list_objects(bucket_name='root', prefix='test_multiple_log_files/', recursive=True))
    [<minio.datatypes.Object object at 0x7ffff1ec2fe0>, <minio.datatypes.Object object at 0x7ffff1ec39d0>, <minio.datatypes.Object object at 0x7ffff1ec2590>, <minio.datatypes.Object object at 0x7ffff1ec3dc0>, <minio.datatypes.Object object at 0x7ffff1ec3a00>, <minio.datatypes.Object object at 0x7ffff1ec3fa0>, <minio.datatypes.Object object at 0x7ffff1ec3f40>, <minio.datatypes.Object object at 0x7ffff1ec2770>]

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:30:13 +02:00
kssenii
8073e0bad1 Fix tests 2023-06-14 13:45:53 +02:00
vdimir
e34dcabeca
Fix order by in test_storage_delta 2023-04-18 14:25:21 +00:00
kssenii
6f53784f22 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-13 15:56:40 +02:00
kssenii
b9811573be Better 2023-04-13 15:10:49 +02:00
kssenii
37691e573a Fix black check, add with_spark 2023-04-13 11:12:24 +02:00
kssenii
0c8d65b759 Fix java error 2023-04-12 23:12:49 +02:00
kssenii
18723b1a44 Better 2023-04-12 14:38:39 +02:00
kssenii
e32c98e412 Close spark session 2023-04-11 17:25:32 +02:00
kssenii
c59d7a4bb3 Fix 2023-04-05 20:32:37 +02:00
kssenii
e632dc5cdd Try to understand why some tests fail in CI, but locally pass 2023-04-05 16:28:27 +02:00
kssenii
c379eb7c29 Fix style check 2023-04-04 22:59:16 +02:00
kssenii
8c0be0c3be Checkpoints 2023-04-04 22:40:05 +02:00
kssenii
bbe8c121a1 Fix black 2023-04-03 23:05:25 +02:00
kssenii
75b11bceb5 Fix style check 2023-04-03 21:27:05 +02:00
kssenii
18a9a670c3 Better 2023-04-03 20:56:10 +02:00
kssenii
bb1df7cbb6 Fix test 2023-03-30 18:29:55 +02:00
kssenii
60efa3cae7 Fixes for hudi 2023-03-30 18:25:54 +02:00
kssenii
1573790fe7 Better 2023-03-28 20:51:25 +02:00
kssenii
36cc6fee51 Rewrite data lakes (part 1) 2023-03-24 22:35:12 +01:00
kssenii
19819f12f3 Better 2023-03-21 20:19:30 +01:00
kssenii
d9053b8a7c Fix 2023-03-21 17:42:50 +01:00
kssenii
0240ad4c68 Add spark to tests, rewrite tests, fix bug 2023-03-08 15:45:35 +01:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
flynn
da77199bf5 fix style 
fix style
 2022-11-10 03:35:38 +00:00
flynn
6cbfab4e95 fix test 2022-11-10 03:10:25 +00:00
flynn
0ffdf3e0c7 fix style 2022-11-09 12:42:13 +00:00
flynn
c226a3cc9a fix style 2022-11-09 12:32:12 +00:00
flynn
592f6b31d9 fix and add test 2022-11-09 10:04:53 +00:00
Daniil Rubin
f75376b7f8 Apply black formatter to tests 2022-09-07 11:05:40 +03:00
Daniil Rubin
4cc397da84 Add tests for deltalake storage 2022-09-07 11:05:40 +03:00