thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
151,201 Commits 9,328 Branches 1,783 Tags 2.1 GiB
9d5f1e1a69
Commit Graph

47 Commits

Author SHA1 Message Date
kssenii
7e006f6581 Update delta lake test 2024-08-15 14:09:48 +02:00
Michael Stetsyuk
2e5f45a7ad rename: S3DiskNoKeyErrors -> DiskS3NoSuchKeyErrors 2024-08-14 17:53:33 +00:00
Michael Stetsyuk
092c837119 randomize table name in test_storage_delta 2024-08-02 12:14:37 +01:00
Michael Stetsyuk
9c05a0ad5a rm dirs in test_storage_delta 2024-08-02 09:34:32 +01:00
Michael Stetsyuk
1d85f9b1cb fix remove_local_directory_contents 2024-08-01 14:58:11 +01:00
robot-clickhouse
7dbd3d7534 Automatic style fix 2024-07-31 13:51:48 +00:00
Michael Stetsyuk
31c142a96d make it possible to rerun test_storage_delta and test_checking_s3_blobs_paranoid 2024-07-31 14:44:54 +01:00
Michael Stetsyuk
8b52d7b711 fxs 2024-07-30 13:35:19 +01:00
kssenii
8a202d91ad Properly read schema and partition columns from checkpoint file 2024-07-09 16:30:39 +02:00
kssenii
a8f0cfe580 Fixes 2024-05-13 20:57:19 +02:00
kssenii
fb4a230eee Support reading partitioed DeltaLake columns 2024-05-10 12:16:52 +02:00
Kseniia Sumarokova
50a0494df7
Update test.py 2024-01-24 16:53:10 +01:00
kssenii
a4082f5b97 Add a test 2024-01-23 13:28:25 +01:00
robot-clickhouse
48f0580512 Automatic style fix 2024-01-22 17:39:44 +00:00
kssenii
1aa8e62bb7 Fix 2024-01-22 18:26:44 +01:00
Azat Khuzhin
4179ff7111 Adjust the test_storage_delta::test_multiple_log_files after MinIO upgrade 
After upgrade MinIO stopped handling of objects with leading slash:

    (Pdb) p minio_client.fput_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json', file_path='/test_multiple_log_files/_delta_log/00000000000000000001.json').http_headers
    HTTPHeaderDict({'Accept-Ranges': 'bytes', 'Content-Length': '0', 'ETag': '"f1f3276c787b3f6724b9a8edb313c4fa"', 'Server': 'MinIO', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Vary': 'Origin, Accept-Encoding', 'X-Amz-Id-2': '7987905dee74cdeb212432486a178e511309594cee7cb75f892cd53e35f09ea4', 'X-Amz-Request-Id': '178B9B0904A9640D', 'X-Content-Type-Options': 'nosniff', 'X-Xss-Protection': '1; mode=block', 'Date': 'Fri, 06 Oct 2023 19:22:27 GMT'})
    (Pdb) p list(minio_client.list_objects('root', '/test_multiple_log_files/_delta_log/'))
    []
    (Pdb) p minio_client.get_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json').read()
    b'{"commitInfo":{"timestamp":1696618985577,"operation":"WRITE","operationParameters":{"mode":"Overwrite","partitionBy":"[]"},"readVersion":0,"isolationLevel":"Serializable","isBlindAppend":false,"operationMetrics":{"numFiles":"1","numOutputRows":"100","numOutputBytes":"2459"},"engineInfo":"Apache-Spark/3.3.2 Delta-Lake/2.3.0","txnId":"2a69a4c9-f72e-4ac9-9374-bdff96707edf"}}\n{"add":{"path":"part-00000-9ff4728b-d5fe-49ac-a773-e2d6e01f4139-c000.parquet","partitionValues":{},"size":2459,"modificationTime":1696618985310,"dataChange":true,"stats":"{\\"numRecords\\":100,\\"minValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"1\\"},\\"maxValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":99,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"99\\"},\\"nullCount\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":0}}"}}\n{"remove":{"path":"part-00000-7c29f5d9-e14d-419a-a273-58d07775e1e9-c000.parquet","deletionTimestamp":1696618985576,"dataChange":true,"extendedFileMetadata":true,"partitionValues":{},"size":2459}}\n'
    (Pdb) p list(minio_client.list_objects(bucket_name='root', prefix='test_multiple_log_files/', recursive=True))
    [<minio.datatypes.Object object at 0x7ffff1ec2fe0>, <minio.datatypes.Object object at 0x7ffff1ec39d0>, <minio.datatypes.Object object at 0x7ffff1ec2590>, <minio.datatypes.Object object at 0x7ffff1ec3dc0>, <minio.datatypes.Object object at 0x7ffff1ec3a00>, <minio.datatypes.Object object at 0x7ffff1ec3fa0>, <minio.datatypes.Object object at 0x7ffff1ec3f40>, <minio.datatypes.Object object at 0x7ffff1ec2770>]

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:30:13 +02:00
kssenii
8073e0bad1 Fix tests 2023-06-14 13:45:53 +02:00
vdimir
e34dcabeca
Fix order by in test_storage_delta 2023-04-18 14:25:21 +00:00
kssenii
6f53784f22 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-13 15:56:40 +02:00
kssenii
b9811573be Better 2023-04-13 15:10:49 +02:00
kssenii
37691e573a Fix black check, add with_spark 2023-04-13 11:12:24 +02:00
kssenii
0c8d65b759 Fix java error 2023-04-12 23:12:49 +02:00
kssenii
18723b1a44 Better 2023-04-12 14:38:39 +02:00
kssenii
e32c98e412 Close spark session 2023-04-11 17:25:32 +02:00
kssenii
c59d7a4bb3 Fix 2023-04-05 20:32:37 +02:00
kssenii
e632dc5cdd Try to understand why some tests fail in CI, but locally pass 2023-04-05 16:28:27 +02:00
kssenii
c379eb7c29 Fix style check 2023-04-04 22:59:16 +02:00
kssenii
8c0be0c3be Checkpoints 2023-04-04 22:40:05 +02:00
kssenii
bbe8c121a1 Fix black 2023-04-03 23:05:25 +02:00
kssenii
75b11bceb5 Fix style check 2023-04-03 21:27:05 +02:00
kssenii
18a9a670c3 Better 2023-04-03 20:56:10 +02:00
kssenii
bb1df7cbb6 Fix test 2023-03-30 18:29:55 +02:00
kssenii
60efa3cae7 Fixes for hudi 2023-03-30 18:25:54 +02:00
kssenii
1573790fe7 Better 2023-03-28 20:51:25 +02:00
kssenii
13f29a7242 Better 2023-03-28 18:57:24 +02:00
kssenii
36cc6fee51 Rewrite data lakes (part 1) 2023-03-24 22:35:12 +01:00
kssenii
19819f12f3 Better 2023-03-21 20:19:30 +01:00
kssenii
d9053b8a7c Fix 2023-03-21 17:42:50 +01:00
kssenii
0240ad4c68 Add spark to tests, rewrite tests, fix bug 2023-03-08 15:45:35 +01:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
flynn
da77199bf5 fix style 
fix style
 2022-11-10 03:35:38 +00:00
flynn
6cbfab4e95 fix test 2022-11-10 03:10:25 +00:00
flynn
0ffdf3e0c7 fix style 2022-11-09 12:42:13 +00:00
flynn
c226a3cc9a fix style 2022-11-09 12:32:12 +00:00
flynn
592f6b31d9 fix and add test 2022-11-09 10:04:53 +00:00
Daniil Rubin
f75376b7f8 Apply black formatter to tests 2022-09-07 11:05:40 +03:00
Daniil Rubin
4cc397da84 Add tests for deltalake storage 2022-09-07 11:05:40 +03:00