boto3

在开始之前，您必须生成 Access Key。所有示例都将使用 access_key_id 和 access_key_secret 变量，分别代表您生成的 Access Key ID（访问密钥 ID） 和 Secret Access Key（秘密访问密钥） 值。

配置 boto3 ↗ 以使用您的 R2 端点：

import boto3

s3 = boto3.client(
    service_name="s3",
    endpoint_url="https://<ACCOUNT_ID>.r2.cloudflarestorage.com",
    aws_access_key_id="<ACCESS_KEY_ID>",
    aws_secret_access_key="<SECRET_ACCESS_KEY>",
    region_name="auto",
)

如果设置了 AWS_ACCESS_KEY_ID 和 AWS_SECRET_ACCESS_KEY 环境变量 ↗，可以省略 aws_access_key_id 和 aws_secret_access_key。

使用客户端的常见操作：

# Get object metadata
s3.head_object(Bucket="my-bucket", Key="dog.png")

# Get object
response = s3.get_object(Bucket="my-bucket", Key="dog.png")

# Upload single file
with open("./dog.png", "rb") as f:
    s3.upload_fileobj(f, "my-bucket", "dog.png")

# Delete object
s3.delete_object(Bucket="my-bucket", Key="dog.png")

优化上传性能

对于大型对象（例如训练数据或视频等多 GB 文件），upload_fileobj 可能成为吞吐量瓶颈。其内部线程池受 Python GIL ↗ 限制，通过 TransferConfig 增加 max_concurrency 在超过约 10 个线程后收益递减。

请改用低级分片 API 配合 ThreadPoolExecutor：

import boto3
import math
import os
from concurrent.futures import ThreadPoolExecutor

s3 = boto3.client(
    service_name="s3",
    endpoint_url="https://<ACCOUNT_ID>.r2.cloudflarestorage.com",
    aws_access_key_id="<ACCESS_KEY_ID>",
    aws_secret_access_key="<SECRET_ACCESS_KEY>",
    region_name="auto",
)

bucket = "my-bucket"
key = "large-file.bin"
file_path = "./large-file.bin"
part_size = 16 * 1024 * 1024  # 16 MiB per part
max_workers = 10

# Step 1: Create the multipart upload
upload_id = None
mpu = s3.create_multipart_upload(Bucket=bucket, Key=key)
upload_id = mpu["UploadId"]

def upload_part(part_number, data):
    response = s3.upload_part(
        Bucket=bucket,
        Key=key,
        UploadId=upload_id,
        PartNumber=part_number,
        Body=data,
    )
    return {"PartNumber": part_number, "ETag": response["ETag"]}

try:
    file_size = os.path.getsize(file_path)
    part_count = math.ceil(file_size / part_size)

    # Step 2: Upload parts in parallel
    with ThreadPoolExecutor(max_workers=max_workers) as pool:
        futures = []
        with open(file_path, "rb") as f:
            for i in range(part_count):
                data = f.read(part_size)
                futures.append(pool.submit(upload_part, i + 1, data))

        parts = [future.result() for future in futures]

    # Step 3: Complete the upload
    s3.complete_multipart_upload(
        Bucket=bucket,
        Key=key,
        UploadId=upload_id,
        MultipartUpload={"Parts": parts},
    )
    print("Multipart upload complete.")
except Exception:
    if upload_id:
        try:
            s3.abort_multipart_upload(Bucket=bucket, Key=key, UploadId=upload_id)
        except Exception:
            pass
    raise

有关分片上传的更多信息，包括分片大小限制和生命周期管理，请参阅上传对象。

生成预签名 URL

生成预签名链接，以共享存储桶的临时公开读取或写入访问权限。

# Generate presigned URL for reading (GET)
get_url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "my-bucket", "Key": "dog.png"},
    ExpiresIn=3600,  # Valid for 1 hour
)

# Generate presigned URL for writing (PUT)
put_url = s3.generate_presigned_url(
    "put_object",
    Params={
        "Bucket": "my-bucket",
        "Key": "dog.png",
        "ContentType": "image/png",
    },
    ExpiresIn=3600,
)

https://<ACCOUNT_ID>.r2.cloudflarestorage.com/my-bucket/dog.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Date=<timestamp>&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature>
https://<ACCOUNT_ID>.r2.cloudflarestorage.com/my-bucket/dog.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Date=<timestamp>&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Signature=<signature>

使用预签名 PUT URL 上传。使用带有 ContentType 的预签名 URL 时，客户端必须包含匹配的 Content-Type 请求头：

curl -X PUT "https://<ACCOUNT_ID>.r2.cloudflarestorage.com/my-bucket/dog.png?X-Amz-Algorithm=..." \
  -H "Content-Type: image/png" \
  --data-binary @dog.png

使用 CORS 和 Content-Type 限制上传

生成用于上传的预签名 URL 时，您可以通过以下方式限制滥用和误用：

限制 Content-Type：在预签名 URL 参数中指定允许的内容类型。如果客户端发送不同的 Content-Type 请求头，上传将失败。
配置 CORS：在存储桶上设置 CORS 规则，以控制哪些源可以上传文件。通过 Cloudflare 仪表板 ↗ 向存储桶设置添加 JSON 策略来配置 CORS：

[
	{
		"AllowedOrigins": ["https://example.com"],
		"AllowedMethods": ["PUT"],
		"AllowedHeaders": ["Content-Type"],
		"ExposeHeaders": ["ETag"],
		"MaxAgeSeconds": 3600
	}
]

然后生成带有 Content-Type 限制的预签名 URL：

put_url = s3.generate_presigned_url(
    "put_object",
    Params={
        "Bucket": "my-bucket",
        "Key": "dog.png",
        "ContentType": "image/png",
    },
    ExpiresIn=3600,
)

客户端使用此预签名 URL 时必须：

从允许的源发起请求（由 CORS 强制执行）
包含 Content-Type: image/png 请求头（由签名强制执行）