lsm/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-02-09 09:17:28 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
Page: Security Overview
Pages
AWS CLI with SeaweedFS AWS IAM CLI Actual Users Admin UI Amazon IAM API Amazon S3 API Applications Async Backup Async Filer Metadata Backup Async Replication to Cloud Async Replication to another Filer Benchmark SeaweedFS as a GlusterFS replacement Benchmarks from jinleileiking Benchmarks Cache Remote Storage Choosing a Filer Store Client Libraries Cloud Drive Architecture Cloud Drive Benefits Cloud Drive Quick Setup Cloud Monitoring Cloud Tier Components Configure Remote Storage Cryptography and FIPS Compliance Customize Filer Store Data Backup Data Structure for Large Files Deployment to Kubernetes and Minikube Directories and Files Docker Compose for S3 Docker Image Registry with SeaweedFS Environment Variables Erasure Coding for warm storage Error reporting to sentry FAQ FIO benchmark FUSE Mount Failover Master Server File Operations Quick Reference Filer Active Active cross cluster continuous synchronization Filer Cassandra Setup Filer Change Data Capture Filer Commands and Operations Filer Data Encryption Filer JWT Use Filer Metadata Events Filer Notification Webhook Filer Redis Setup Filer Server API Filer Setup Filer Store Replication Filer Stores Filer as a Key Large Value Store Gateway to Remote Object Storage Getting Started HDFS via S3 connector Hadoop Benchmark Hadoop Compatible File System Hardware Hobbyest Tinkerer scale on premises tutorial Home Independent Benchmarks Kafka to Kafka Gateway to SMQ to SQL Kubernetes Backups and Recovery with K8up Large File Handling Load Command Line Options from a file Master Server API Migrate to Filer Store Mount Remote Storage OIDC Integration Optimization Path Specific Configuration Path Specific Filer Store PostgreSQL compatible Server weed db Production Setup Pub Sub to SMQ to SQL Quick Start with weed mini Replication Run Blob Storage on Public Internet Run Presto on SeaweedFS S3 API Audit log S3 API Benchmark S3 API FAQ S3 Bucket Quota S3 CORS S3 Conditional Operations S3 Configuration S3 Credentials S3 Nginx Proxy S3 Object Lock and Retention S3 Object Versioning S3 Policy Variables S3 Rate Limiting S3 Table Bucket Commands S3 Table Bucket SQL Queries on Message Queue SQL Quick Reference SRV Service Discovery Seaweed Message Queue SeaweedFS Java Client SeaweedFS in Docker Swarm Security Configuration Security Overview Server Side Encryption SSE C Server Side Encryption SSE KMS Server Side Encryption Server Startup via Systemd Store file with a Time To Live Structured Data Lake with SMQ and SQL Super Large Directories System Metrics TUS Resumable Uploads TensorFlow with SeaweedFS Tiered Storage UrBackup with SeaweedFS Use Cases Volume Files Structure Volume Management Volume Server API WebDAV Words from SeaweedFS Users Worker fstab nodejs with Seaweed S3 rclone with SeaweedFS restic with SeaweedFS run HBase on SeaweedFS run Spark on SeaweedFS s3cmd with SeaweedFS weed shell
Clone
21
Security Overview
Chris Lu edited this page 2026-01-12 13:00:52 -08:00
Table of Contents

Overview

Since SeaweedFS is a distributed system with many volume servers, the volume servers have the risk of being changed without proper access control. We want to have the freedom to place a volume server anywhere we want, with the confidence that nobody can tamper the data.

Looking for FIPS 140-2/140-3 compliance information? See Cryptography and FIPS Compliance.

Understanding Communication Layers

SeaweedFS has two separate communication layers that must be secured independently:

gRPC Communication (Control Plane)

Used for metadata operations and cluster coordination:

  • S3 ↔ Filer: Getting bucket info, file metadata
  • Filer ↔ Master: Getting volume assignments, cluster info
  • Volume ↔ Master: Heartbeats, volume management
  • Configured via: [grpc.*] sections in security.toml

HTTP/HTTPS Communication (Data Plane)

Used for actual file data uploads/downloads:

  • S3/Filer/Client → Volume Server: Uploading and downloading actual file content
  • S3 → Filer: File data operations (when S3 acts as a proxy)
  • Configured via: [https.*] sections in security.toml

Configuration Sections Summary

Configuration Block Purpose
[grpc.client] Used by clients (S3, mount, filer.copy, backup, etc.) to connect to any gRPC server
[grpc.filer], [grpc.master], [grpc.volume], [grpc.s3] Server-side gRPC mTLS - secures gRPC endpoints
[https.client] Client-side HTTPS configuration - tells clients (S3/Filer/etc.) to use HTTPS for HTTP connections
[https.volume], [https.master], [https.filer] Server-side HTTPS configuration - makes servers accept HTTPS connections

Important:

  • [https.client] with enabled = true is required for clients to use HTTPS when connecting to HTTPS-enabled servers
  • If you enable [https.filer] on the Filer server, you must also enable [https.client] so that S3 API can connect via HTTPS
  • Otherwise, you'll get "client sent an HTTP request to an HTTPS server" errors

Security Features

We will address the volume servers first. The following items are not covered, yet:

  • master server http REST services

Starting with version 2.84, the Filer HTTP REST services can be secured with a JWT, by setting jwt.filer_signing.key and jwt.filer_signing.read.key in security.toml.

In summary, here are what can be achieved.

Server Service Note
master gRPC secured by mutual TLS
volume gRPC secured by mutual TLS
filer gRPC secured by mutual TLS
master http "weed master -disableHttp", disable http operations, only gRPC operations are allowed.
filer http Before version 2.84: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does not work with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer.

Starting with version 2.84: secured by JWT, by setting jwt.filer_signing.key and jwt.filer_signing.read.key in security.toml. This now works with the Amazon S3 API.
volume http write set jwt.signing.key in security.toml in master and volume servers to check token for write operations
volume http read unprotected, but url is not guessable

Generate security.toml file

See Security Configuration

Servers in SeaweedFS usually support 2 kinds of operations: gRPC and REST.

Securing gRPC operations

The following operations are implemented via gRPC.

  • requests from filer to master
  • requests from master to volume servers
  • delete operations from filer or other clients (mount, s3, filer.copy, filer.replicate, etc) to volume servers
  • requests from clients to filer

All gRPC operations can optionally be secured via mutual TLS, by customizing the security.toml file. See Security Configuration.

Securing Volume Servers

Besides gRPC mentioned above, volume servers can only be changed by file upload, update, and delete operations. Json Web Token (JWT) is used to authorize access for each file id.

JWT-based access control

To enable JWT-based access control,

  1. generate security.toml file by weed scaffold -config=security
  2. set jwt.signing.key to a secret string
  3. copy the same security.toml file to the masters and all volume servers.

Re-enabling Volume UI

By default, if the jwt.signing.key is set, the web UI on the volume servers is disabled. You can re-enable the web UI by setting access.ui=true in security.toml. Despite some information leakage (as the UI is unauthenticted), this should not pose a security risk, as the UI is purely read-only.

How JWT-based access control works

  • To upload a new file, when requesting a new fileId via http://<master>:<port>/dir/assign, the master will use the jwt.signing.key to generate and sign a JWT, and set it to response header Authorization. The JWT is valid for 10 seconds.
  • To update or delete a file by fileId, the JWT can be read from the response header Authorization of http://<master>:<port>/dir/lookup?fileId=xxxxx.
  • When sending upload/update/delete HTTP operations to a volume server, the request header Authorization should be the JWT string. The operation is authorized after the volume server validates the JWT with jwt.signing.key.

JWT Summary:

  • JWT is set in /dir/assign or /dir/lookup response header Authorization
  • JWT is read from request header Authorization
  • JWT is valid for 10 seconds.
  • JWT only has permission to create/modify/delete one fileId.
  • The volume server HTTP access is only for read, and only if the fileId is known. There are no way to iterate all files.
  • All other volume server HTTP accesses are disabled when jwt.signing is enabled.

JWT for Read Access Control

The volume server can also check JWT for reads. This mode does not work with weed filer. But this could be useful if the volume server is exposed to public and you do not want anyone to access it with a URL, e.g., paid content.

  • To enable it, set the jwt.signing.read.key in security.toml file.
  • To obtain a JWT for read, the JWT can be read from the response header Authorization of http://<master>:<port>/dir/lookup?fileId=xxxxx&read=yes.

Securing Filer HTTP with JWT

To enable JWT-based access control for the Filer,

  1. generate security.toml file by weed scaffold -config=security
  2. set jwt.filer_signing.key to a secret string - and optionally jwt.filer_signing.read.key as well to a secret string
  3. copy the same security.toml file to the filers and all S3 proxies.

If jwt.filer_signing.key is configured: When sending upload/update/delete HTTP operations to a filer server, the request header Authorization should be the JWT string (Authorization: Bearer [JwtToken]). The operation is authorized after the filer validates the JWT with jwt.filer_signing.key.

The JwtToken can be generated by calling security.GenJwtForFilerServer(signingKey SigningKey, expiresAfterSec int) in github.com/seaweedfs/seaweedfs/weed/security package. 9b94177380/weed/security/jwt.go (L53)

If jwt.filer_signing.read.key is configured: When sending GET or HEAD requests to a filer server, the request header Authorization should be the JWT string (Authorization: Bearer [JwtToken]). The operation is authorized after the filer validates the JWT with jwt.filer_signing.read.key.

The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer.

JWT Scoping

The Filer HTTP JWT can optionally include claims to restrict access to specific paths and HTTP methods.

  • allowed_prefixes: A list of path prefixes. If present, the request path must match at least one of these prefixes.
  • allowed_methods: A list of HTTP methods (e.g., "GET", "POST"). If present, the request method must match one of these methods.

Example JWT payload:

{
  "exp": 17000,
  "allowed_prefixes": ["/hls/movie123/", "/subtitles/movie123/"],
  "allowed_methods": ["GET", "HEAD"]
}

This is useful for generating short-lived, scoped tokens for direct client access (e.g., for downloading content) without exposing global read/write permissions.

Introduction

API

Configuration

Filer

Filer Stores

Management

Advanced Filer Configurations

FUSE Mount

WebDAV

Cloud Drive

AWS S3 API

S3 Table Bucket

S3 Authentication & IAM

Server-Side Encryption

S3 Client Tools

Machine Learning

HDFS

Replication and Backup

Metadata Change Events

Messaging

Use Cases

Operations

Advanced

Security

Misc Use Case Examples