Table of Contents
SeaweedFS uses Prometheus to store the metrics and Grafana to visualize them.
SeaweedFS supports both push and pull metrics.
Push Metrics
SeaweedFS can publish metrics to Prometheus Push Gateway, and the gateway pass along to the Prometheus server.
Master | Volume Server | Filer => Prometheus Push Gateway => Prometheus Server => Grafana
Note: Setting Prometheus, push gateway, and Grafana can be simplified with this https://github.com/evnsio/prom-stack
Configuration
Just add a metrics address to weed master or weed server command line options. If you have multiple masters, please add it to all the master command line options.
weed master -metrics.address=<protocol><prometheus_gateway_host_name>:<prometheus_gateway_port>
# example
weed master -metrics.address=localhost:9091 # Defaults to http://localhost:9091
weed master -metrics.address=https://example.com
weed server -metrics.address=<protocol><prometheus_gateway_host_name>:<prometheus_gateway_port>
# example
weed server -metrics.address=localhost:9091 # Defaults to http://localhost:9091
weed server -metrics.address=https://example.com
The SeaweedFS filer or volume servers will read this metrics configuration from the master, and report the metrics directly to the Prometheus Gateway. Filer and volume servers need to be restarted for the changes to take effect.
Pull Metrics
SeaweedFS can also start with ports accepting Prometheus metrics queries.
weed server -metricsPort=1234
weed master -metricsPort=1235
weed volume -metricsPort=1236
weed filer -metricsPort=1237
Note: All server should be running on different ports for accepting prometheus metrics queries.
And then you can configure your Prometheus to crawl them periodically.
Dashboard
The dashboard is shared at https://github.com/seaweedfs/seaweedfs/blob/master/other/metrics/grafana_seaweedfs.json
If you modify the dashboard, please share your revisions.
Example Dashboard
Introduction
API
Configuration
- Replication
- Store file with a Time To Live
- Failover Master Server
- Erasure coding for warm storage
- Server Startup via Systemd
- Environment Variables
Filer
- Filer Setup
- Directories and Files
- File Operations Quick Reference
- Data Structure for Large Files
- Filer Data Encryption
- Filer Commands and Operations
- Filer JWT Use
- TUS Resumable Uploads
Filer Stores
- Filer Cassandra Setup
- Filer Redis Setup
- Super Large Directories
- Path-Specific Filer Store
- Choosing a Filer Store
- Customize Filer Store
Management
Advanced Filer Configurations
- Migrate to Filer Store
- Add New Filer Store
- Filer Store Replication
- Filer Active Active cross cluster continuous synchronization
- Filer as a Key-Large-Value Store
- Path Specific Configuration
- Filer Change Data Capture
FUSE Mount
WebDAV
Cloud Drive
- Cloud Drive Benefits
- Cloud Drive Architecture
- Configure Remote Storage
- Mount Remote Storage
- Cache Remote Storage
- Cloud Drive Quick Setup
- Gateway to Remote Object Storage
AWS S3 API
- Amazon S3 API
- S3 Conditional Operations
- S3 CORS
- S3 Object Lock and Retention
- S3 Object Versioning
- S3 API Benchmark
- S3 API FAQ
- S3 Bucket Quota
- S3 Rate Limiting
- S3 API Audit log
- S3 Nginx Proxy
- Docker Compose for S3
S3 Table Bucket
S3 Authentication & IAM
- S3 Configuration - Start Here
- S3 Credentials (
-s3.config) - OIDC Integration (
-s3.iam.config) - S3 Policy Variables
- Amazon IAM API
- AWS IAM CLI
Server-Side Encryption
S3 Client Tools
- AWS CLI with SeaweedFS
- s3cmd with SeaweedFS
- rclone with SeaweedFS
- restic with SeaweedFS
- nodejs with Seaweed S3
Machine Learning
HDFS
- Hadoop Compatible File System
- run Spark on SeaweedFS
- run HBase on SeaweedFS
- run Presto on SeaweedFS
- Hadoop Benchmark
- HDFS via S3 connector
Replication and Backup
- Async Replication to another Filer [Deprecated]
- Async Backup
- Async Filer Metadata Backup
- Async Replication to Cloud [Deprecated]
- Kubernetes Backups and Recovery with K8up
Metadata Change Events
Messaging
- Structured Data Lake with SMQ and SQL
- Seaweed Message Queue
- SQL Queries on Message Queue
- SQL Quick Reference
- PostgreSQL-compatible Server weed db
- Pub-Sub to SMQ to SQL
- Kafka to Kafka Gateway to SMQ to SQL
Use Cases
Operations
Advanced
- Large File Handling
- Optimization
- Volume Management
- Tiered Storage
- Cloud Tier
- Cloud Monitoring
- Load Command Line Options from a file
- SRV Service Discovery
- Volume Files Structure
Security
- Security Overview
- Security Configuration
- Cryptography and FIPS Compliance
- Run Blob Storage on Public Internet