Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/sachnun/hugbucket/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

HugBucket exposes a path-style S3-compatible HTTP API on port 9000. All requests use the /{bucket}/{key} path format — virtual-hosted-style addressing is not supported. The gateway translates standard S3 REST operations into Hugging Face Storage Bucket API calls, so any S3-compatible tool (AWS CLI, boto3, S3 Browser, etc.) works out of the box.

​
Authentication

HugBucket implements AWS Signature V4 authentication (header-based and presigned URLs) as well as Signature V2 presigned URLs for compatibility with older clients. Credentials are configured via environment variables:
VariableDescription
AWS_ACCESS_KEY_IDAccess key presented to S3 clients
AWS_SECRET_ACCESS_KEYSecret key used to verify request signatures
If both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are left empty, authentication is completely disabled and any client can access the gateway without credentials. Only do this in trusted, isolated environments.
The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values you choose are arbitrary — they are not your actual AWS credentials. HugBucket uses the Hugging Face token (HF_TOKEN) for all backend access.

​
Supported operations

OperationHTTP methodPath
ListBucketsGET/
CreateBucketPUT/{bucket}
DeleteBucketDELETE/{bucket}
HeadBucketHEAD/{bucket}
GetBucketLocationGET/{bucket}?location
ListObjectsV2GET/{bucket}?list-type=2
PutObjectPUT/{bucket}/{key}
GetObjectGET/{bucket}/{key}
HeadObjectHEAD/{bucket}/{key}
DeleteObjectDELETE/{bucket}/{key}
DeleteObjects (batch)POST/{bucket}?delete
CopyObjectPUT + x-amz-copy-source header/{bucket}/{key}
InitiateMultipartUploadPOST/{bucket}/{key}?uploads
UploadPartPUT/{bucket}/{key}?partNumber=N&uploadId=...
CompleteMultipartUploadPOST/{bucket}/{key}?uploadId=...
AbortMultipartUploadDELETE/{bucket}/{key}?uploadId=...

​
Using with AWS CLI

Pass --endpoint-url to point the AWS CLI at your HugBucket instance. The --no-sign-request flag is only needed when authentication is disabled. 
aws --endpoint-url http://localhost:9000 s3 ls
When credentials are set, configure them in your shell or ~/.aws/credentials: 
export AWS_ACCESS_KEY_ID=hugbucket
export AWS_SECRET_ACCESS_KEY=hugbucket

​
Using with boto3

Set endpoint_url when creating the S3 client and supply the same credentials you configured on the server: 
import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:9000",
    aws_access_key_id="hugbucket",
    aws_secret_access_key="hugbucket",
    region_name="us-east-1",
)

# List all buckets
response = s3.list_buckets()
for bucket in response["Buckets"]:
    print(bucket["Name"])

# Upload an object
s3.put_object(Bucket="my-bucket", Key="hello.txt", Body=b"Hello, world!")

# Download an object
response = s3.get_object(Bucket="my-bucket", Key="hello.txt")
print(response["Body"].read())

​
Multipart upload

HugBucket supports the full S3 multipart upload flow, which tools like the AWS CLI and boto3 use automatically for large files.
1

Initiate

The client sends POST /{bucket}/{key}?uploads. HugBucket creates an in-memory upload record and returns a unique uploadId.
2

Upload parts

The client sends one or more PUT /{bucket}/{key}?partNumber=N&uploadId=... requests. Each part is stored in memory, keyed by part number. An MD5 ETag is returned for each part.
3

Complete

The client sends POST /{bucket}/{key}?uploadId=.... HugBucket concatenates all parts in order (freeing each as it goes to keep peak memory at ~1x the file size) and writes the result to Hugging Face storage as a single object.
4

Abort (optional)

The client sends DELETE /{bucket}/{key}?uploadId=.... HugBucket discards all buffered parts and removes the upload record. Aborting an upload that is mid-completion returns 409 InvalidRequest.

​
Idempotent completion

If a network timeout causes the client to retry CompleteMultipartUpload while the upload is still being written:
  • In progress — the retry waits for the original request to finish, then returns the same result.
  • Already completed — the cached response body is returned immediately.
  • Previously failed — the upload state is reset so the client can re-upload parts and retry.

​
Stale upload cleanup

A background task runs every 10 minutes and removes uploads that have been inactive for longer than the TTL defined by multipart_upload_ttl (default: 86400 seconds / 24 hours). Uploads actively being completed are never removed by the cleanup pass.

​
Range requests

GetObject supports the Range: bytes=<start>-<end> header. HugBucket responds with HTTP 206 Partial Content and the Content-Range header. If the requested range falls outside the object size, 416 Range Not Satisfiable is returned. Range support enables parallel/resumable downloads, streaming media players, and tools like wget --continue.

​
Directory handling

S3 has no native directory concept — “folders” are simulated by object key prefixes. When a client performs a PUT with a key ending in / and a zero-byte body (the standard S3 folder-creation pattern), HugBucket stores a .hugbucket_keep marker object in that prefix instead of a bare trailing-slash key. HeadObject on a trailing-slash key checks for this marker and returns 200 with Content-Type: application/x-directory if the directory exists.

​
Ignored paths

The following paths are rejected with 404 NoSuchBucket and never forwarded to Hugging Face, to avoid unnecessary API calls triggered by browsers and crawlers:
  • favicon.ico
  • robots.txt
  • sitemap.xml
  • .well-known

Build docs developers (and LLMs) love

Get started for freeTalk to us