ClickHouse/tests/integration/compose/docker_compose_minio.yml

version: '2.3'

services:
  minio1:
    image: minio/minio:RELEASE.2023-09-30T07-02-29Z
    volumes:
      - data1-1:/data1
      - ${MINIO_CERTS_DIR:-}:/certs
    expose:
      - ${MINIO_PORT:-9001}
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
      MINIO_PROMETHEUS_AUTH_TYPE: public
    command: server --console-address 127.0.0.1:19001 --address :9001 --certs-dir /certs /data1-1
    depends_on:
      - proxy1
      - proxy2

  # HTTP proxies for Minio.
  proxy1:
    image: clickhouse/s3-proxy
    expose:
      - "8080" # Redirect proxy port
      - "80"   # Reverse proxy port
      - "443"  # Reverse proxy port (secure)

  proxy2:
    image: clickhouse/s3-proxy
    expose:
      - "8080"
      - "80"
      - "443"

  # Empty container to run proxy resolver.
  resolver:
    image: clickhouse/python-bottle:${DOCKER_PYTHON_BOTTLE_TAG:-latest}
    expose:
      - "8080"
    tty: true
    depends_on:
      - proxy1
      - proxy2

volumes:
  data1-1:
Bump compose version to 2.3, and make tmpfs for zookeeper optional 2020-05-19 15:27:10 +00:00			`version: '2.3'`
Use Minio for S3 Storage integration tests. 2019-11-20 11:56:38 +00:00
			`services:`
			`minio1:`
Upgrade MinIO to support accepting non signed requests During fixing one issue with multiple reads of input files while sending them to S3 this had been pops up. The problem is in this place [1]. [1]: https://github.com/minio/minio/blob/200caab82b622a0ec6c9776ee9b555aedbc9b320/cmd/object-handlers.go#L1573-L1591 And this had been added only in [2]. [2]: https://github.com/minio/minio/pull/16484 But even this implementation is not complete, since it does not correctly support STREAMING-UNSIGNED-PAYLOAD-TRAILER, but this will be the first step, to see if there are still problems with newer MinIO. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> 2023-10-03 09:39:06 +00:00			`image: minio/minio:RELEASE.2023-09-30T07-02-29Z`
Use Minio for S3 Storage integration tests. 2019-11-20 11:56:38 +00:00			`volumes:`
revert minio changes 2023-01-05 09:12:56 +00:00			`- data1-1:/data1`
S3 HTTPS integration test. 2020-07-10 19:42:18 +00:00			`- ${MINIO_CERTS_DIR:-}:/certs`
s3 catch up 2021-05-12 07:03:53 +00:00			`expose:`
Fix compose files to work without correct environment 2023-01-02 16:45:50 +00:00			`- ${MINIO_PORT:-9001}`
Use Minio for S3 Storage integration tests. 2019-11-20 11:56:38 +00:00			`environment:`
revert minio changes 2023-01-05 09:12:56 +00:00			`MINIO_ACCESS_KEY: minio`
			`MINIO_SECRET_KEY: minio123`
add profiling events from S3 request to system.events and system.query_log 2020-07-10 09:32:34 +00:00			`MINIO_PROMETHEUS_AUTH_TYPE: public`
tests: use explicit address for console to avoid EADDRINUSE The problem is that MinIO uses dynamic port for console address by default, which may be below ip_local_port_range, and in this case it ca be reused in a short time, sicne first MinIO allocates the socket, then close this socket, and only after try to bind to it. And even though this is a problem of MinIO I'm not a go developer to fix it. v2: use long notation of the 127.0.0.1 (that version of MinIO on CI cannot handle 127.1, while 2023-07-21T21-12-44Z can) Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> 2023-08-02 09:05:21 +00:00			`command: server --console-address 127.0.0.1:19001 --address :9001 --certs-dir /certs /data1-1`
Handle exceptions in ProxyResolverConfiguration properly. 2020-05-12 14:34:37 +00:00			`depends_on:`
S3 HTTPS integration test. 2020-07-10 19:42:18 +00:00			`- proxy1`
			`- proxy2`
Use Minio for S3 Storage integration tests. 2019-11-20 11:56:38 +00:00
S3 HTTPS integration test. 2020-07-10 19:42:18 +00:00			`# HTTP proxies for Minio.`
S3 with proxy integration test and fixes. 2020-04-29 08:39:00 +00:00			`proxy1:`
Preparation 2021-09-06 11:33:39 +00:00			`image: clickhouse/s3-proxy`
s3 catch up 2021-05-12 07:03:53 +00:00			`expose:`
S3 HTTPS integration test. 2020-07-10 19:42:18 +00:00			`- "8080" # Redirect proxy port`
			`- "80" # Reverse proxy port`
			`- "443" # Reverse proxy port (secure)`
S3 with proxy integration test and fixes. 2020-04-29 08:39:00 +00:00
			`proxy2:`
Preparation 2021-09-06 11:33:39 +00:00			`image: clickhouse/s3-proxy`
s3 catch up 2021-05-12 07:03:53 +00:00			`expose:`
S3 HTTPS integration test. 2020-07-10 19:42:18 +00:00			`- "8080"`
			`- "80"`
			`- "443"`
S3 with proxy integration test and fixes. 2020-04-29 08:39:00 +00:00
S3 HTTPS integration test. 2020-07-10 19:42:18 +00:00			`# Empty container to run proxy resolver.`
Add possibility to use proxy resolver in DiskS3. 2020-05-08 10:53:12 +00:00			`resolver:`
Add a tag processing for python-bottle 2023-12-21 15:11:54 +00:00			`image: clickhouse/python-bottle:${DOCKER_PYTHON_BOTTLE_TAG:-latest}`
s3 catch up 2021-05-12 07:03:53 +00:00			`expose:`
S3 HTTPS integration test. 2020-07-10 19:42:18 +00:00			`- "8080"`
Add possibility to use proxy resolver in DiskS3. 2020-05-08 10:53:12 +00:00			`tty: true`
Handle exceptions in ProxyResolverConfiguration properly. 2020-05-12 14:34:37 +00:00			`depends_on:`
			`- proxy1`
			`- proxy2`
Add possibility to use proxy resolver in DiskS3. 2020-05-08 10:53:12 +00:00
Use Minio for S3 Storage integration tests. 2019-11-20 11:56:38 +00:00			`volumes:`
			`data1-1:`