2

S3 Object Versioning

Chris Lu edited this page 2026-01-29 22:49:54 -08:00

Table of Contents

• S3 Object Versioning
• Configuration
• Check Versioning Status
• Suspend Versioning
• List Object Versions
• Access Specific Versions
• Get a specific version of an object:
• Copy a specific version:
• Delete a specific version:
• Versioning Behavior
• When Versioning is Enabled:
• When Versioning is Suspended:
• Storage Layout
• Limitations
• Best Practices

S3 Object Versioning

SeaweedFS supports S3 object versioning, which allows you to keep multiple variants of an object in the same bucket. This provides data protection against accidental deletion or modification.

Configuration

export S3_ENDPOINT=http://localhost:8333

To enable versioning on a bucket, use the PutBucketVersioning API:

aws --endpoint-url $S3_ENDPOINT s3api put-bucket-versioning \
    --bucket my-bucket \
    --versioning-configuration Status=Enabled

Check Versioning Status

To check the versioning status of a bucket:

aws --endpoint-url $S3_ENDPOINT s3api get-bucket-versioning --bucket my-bucket

Response:

{
    "Status": "Enabled"
}

Suspend Versioning

To suspend versioning (not disable completely):

aws --endpoint-url $S3_ENDPOINT s3api put-bucket-versioning \
    --bucket my-bucket \
    --versioning-configuration Status=Suspended

List Object Versions

To list all versions of objects in a bucket:

aws --endpoint-url $S3_ENDPOINT s3api list-object-versions --bucket my-bucket

Response includes both object versions and delete markers:

{
    "Versions": [
        {
            "Key": "example.txt",
            "VersionId": "v_1234567890abcdef",
            "IsLatest": true,
            "LastModified": "2023-12-01T10:00:00Z",
            "ETag": "\"abcdef1234567890\"",
            "Size": 1024
        }
    ],
    "DeleteMarkers": [
        {
            "Key": "deleted-file.txt",
            "VersionId": "v_fedcba0987654321",
            "IsLatest": true,
            "LastModified": "2023-12-01T11:00:00Z"
        }
    ]
}

Access Specific Versions

Get a specific version of an object:

aws --endpoint-url $S3_ENDPOINT s3api get-object \
    --bucket my-bucket \
    --key example.txt \
    --version-id v_1234567890abcdef \
    output.txt

Copy a specific version:

aws --endpoint-url $S3_ENDPOINT s3api copy-object \
    --copy-source my-bucket/example.txt?versionId=v_1234567890abcdef \
    --bucket my-bucket \
    --key example-copy.txt

Delete a specific version:

aws --endpoint-url $S3_ENDPOINT s3api delete-object \
    --bucket my-bucket \
    --key example.txt \
    --version-id v_1234567890abcdef

Versioning Behavior

When Versioning is Enabled:

• PUT Object: Creates a new version with a unique version ID
• GET Object: Returns the latest version (unless version ID is specified)
• DELETE Object: Creates a delete marker (soft delete)
• DELETE Object with version ID: Permanently deletes that specific version

When Versioning is Suspended:

• PUT Object: Overwrites the object with version ID "null"
• GET Object: Returns the current version
• DELETE Object: Permanently deletes the object

Storage Layout

SeaweedFS stores versioned objects using the following structure:

/buckets/my-bucket/
├── example.txt          # Current version (if versioning suspended)
└── example.txt.versions/
    ├── v_1234567890abcdef  # Version 1
    ├── v_fedcba0987654321  # Version 2
    └── v_abcdef1234567890  # Version 3 (latest)

Limitations

• Version ID Format: SeaweedFS generates version IDs in the format v_<32-char-hex>
• Restore Operations: Only partial support for RestoreObject API
• Lifecycle Policies: Version-specific lifecycle rules are not fully implemented
• MFA Delete: Not currently supported

Best Practices

1. Enable versioning before storing important data to ensure all versions are captured
2. Monitor storage usage as versioning can increase storage consumption
3. Implement lifecycle policies to automatically clean up old versions
4. Use version-specific operations when you need to access historical data
5. Consider performance impact when listing many versions of objects

Introduction

• Quick Start with weed mini
• Components
• Getting Started
• Production Setup
• Benchmarks
• FAQ
• Applications

API

• Master Server API
• Volume Server API
• Filer Server API
• Client Libraries
• SeaweedFS Java Client

Configuration

• Replication
• Store file with a Time To Live
• Failover Master Server
• Erasure coding for warm storage
• Server Startup via Systemd
• Environment Variables

Filer

• Filer Setup
• Directories and Files
• File Operations Quick Reference
• Data Structure for Large Files
• Filer Data Encryption
• Filer Commands and Operations
• Filer JWT Use
• TUS Resumable Uploads

Filer Stores

• Filer Cassandra Setup
• Filer Redis Setup
• Super Large Directories
• Path-Specific Filer Store
• Choosing a Filer Store
• Customize Filer Store

Management

• Admin UI
• Worker

Advanced Filer Configurations

• Migrate to Filer Store
• Add New Filer Store
• Filer Store Replication
• Filer Active Active cross cluster continuous synchronization
• Filer as a Key-Large-Value Store
• Path Specific Configuration
• Filer Change Data Capture

FUSE Mount

• FIO benchmark
• fstab

WebDAV

Cloud Drive

• Cloud Drive Benefits
• Cloud Drive Architecture
• Configure Remote Storage
• Mount Remote Storage
• Cache Remote Storage
• Cloud Drive Quick Setup
• Gateway to Remote Object Storage

AWS S3 API

• Amazon S3 API
• S3 Conditional Operations
• S3 CORS
• S3 Object Lock and Retention
• S3 Object Versioning
• S3 API Benchmark
• S3 API FAQ
• S3 Bucket Quota
• S3 Rate Limiting
• S3 API Audit log
• S3 Nginx Proxy
• Docker Compose for S3

S3 Table Bucket

• S3 Table Bucket

S3 Authentication & IAM

• S3 Configuration - Start Here
• S3 Credentials (-s3.config)
• OIDC Integration (-s3.iam.config)
• S3 Policy Variables
• Amazon IAM API
• AWS IAM CLI

Server-Side Encryption

• Server-Side Encryption
• Server-Side Encryption SSE-KMS
• Server-Side Encryption SSE-C

S3 Client Tools

• AWS CLI with SeaweedFS
• s3cmd with SeaweedFS
• rclone with SeaweedFS
• restic with SeaweedFS
• nodejs with Seaweed S3

Machine Learning

• TensorFlow with SeaweedFS

HDFS

• Hadoop Compatible File System
• run Spark on SeaweedFS
• run HBase on SeaweedFS
• run Presto on SeaweedFS
• Hadoop Benchmark
• HDFS via S3 connector

Replication and Backup

• Async Replication to another Filer [Deprecated]
• Async Backup
• Async Filer Metadata Backup
• Async Replication to Cloud [Deprecated]
• Kubernetes Backups and Recovery with K8up

Metadata Change Events

• Filer Metadata Events
• Filer Notification Webhook

Messaging

• Structured Data Lake with SMQ and SQL
• Seaweed Message Queue
• SQL Queries on Message Queue
• SQL Quick Reference
• PostgreSQL-compatible Server weed db
• Pub-Sub to SMQ to SQL
• Kafka to Kafka Gateway to SMQ to SQL

Use Cases

• Use Cases
• Actual Users

Operations

• System Metrics
• weed shell
• Data Backup
• Deployment to Kubernetes and Minikube

Advanced

• Large File Handling
• Optimization
• Volume Management
• Tiered Storage
• Cloud Tier
• Cloud Monitoring
• Load Command Line Options from a file
• SRV Service Discovery
• Volume Files Structure

Security

• Security Overview
• Security Configuration
• Cryptography and FIPS Compliance
• Run Blob Storage on Public Internet

Misc Use Case Examples

• UrBackup with SeaweedFS
• Docker Image Registry with SeaweedFS
• SeaweedFS in Docker Swarm
• Words from SeaweedFS Users
• Independent Benchmarks
• Hardware