Files
seaweedfs/weed/admin
Chris Lu 891a2fb6eb Admin: misc improvements on admin server and workers. EC now works. (#7055)
* initial design

* added simulation as tests

* reorganized the codebase to move the simulation framework and tests into their own dedicated package

* integration test. ec worker task

* remove "enhanced" reference

* start master, volume servers, filer

Current Status
 Master: Healthy and running (port 9333)
 Filer: Healthy and running (port 8888)
 Volume Servers: All 6 servers running (ports 8080-8085)
🔄 Admin/Workers: Will start when dependencies are ready

* generate write load

* tasks are assigned

* admin start wtih grpc port. worker has its own working directory

* Update .gitignore

* working worker and admin. Task detection is not working yet.

* compiles, detection uses volumeSizeLimitMB from master

* compiles

* worker retries connecting to admin

* build and restart

* rendering pending tasks

* skip task ID column

* sticky worker id

* test canScheduleTaskNow

* worker reconnect to admin

* clean up logs

* worker register itself first

* worker can run ec work and report status

but:
1. one volume should not be repeatedly worked on.
2. ec shards needs to be distributed and source data should be deleted.

* move ec task logic

* listing ec shards

* local copy, ec. Need to distribute.

* ec is mostly working now

* distribution of ec shards needs improvement
* need configuration to enable ec

* show ec volumes

* interval field UI component

* rename

* integration test with vauuming

* garbage percentage threshold

* fix warning

* display ec shard sizes

* fix ec volumes list

* Update ui.go

* show default values

* ensure correct default value

* MaintenanceConfig use ConfigField

* use schema defined defaults

* config

* reduce duplication

* refactor to use BaseUIProvider

* each task register its schema

* checkECEncodingCandidate use ecDetector

* use vacuumDetector

* use volumeSizeLimitMB

* remove

remove

* remove unused

* refactor

* use new framework

* remove v2 reference

* refactor

* left menu can scroll now

* The maintenance manager was not being initialized when no data directory was configured for persistent storage.

* saving config

* Update task_config_schema_templ.go

* enable/disable tasks

* protobuf encoded task configurations

* fix system settings

* use ui component

* remove logs

* interface{} Reduction

* reduce interface{}

* reduce interface{}

* avoid from/to map

* reduce interface{}

* refactor

* keep it DRY

* added logging

* debug messages

* debug level

* debug

* show the log caller line

* use configured task policy

* log level

* handle admin heartbeat response

* Update worker.go

* fix EC rack and dc count

* Report task status to admin server

* fix task logging, simplify interface checking, use erasure_coding constants

* factor in empty volume server during task planning

* volume.list adds disk id

* track disk id also

* fix locking scheduled and manual scanning

* add active topology

* simplify task detector

* ec task completed, but shards are not showing up

* implement ec in ec_typed.go

* adjust log level

* dedup

* implementing ec copying shards and only ecx files

* use disk id when distributing ec shards

🎯 Planning: ActiveTopology creates DestinationPlan with specific TargetDisk
📦 Task Creation: maintenance_integration.go creates ECDestination with DiskId
🚀 Task Execution: EC task passes DiskId in VolumeEcShardsCopyRequest
💾 Volume Server: Receives disk_id and stores shards on specific disk (vs.store.Locations[req.DiskId])
📂 File System: EC shards and metadata land in the exact disk directory planned

* Delete original volume from all locations

* clean up existing shard locations

* local encoding and distributing

* Update docker/admin_integration/EC-TESTING-README.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* check volume id range

* simplify

* fix tests

* fix types

* clean up logs and tests

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-07-30 12:38:03 -07:00
..
2025-07-01 01:28:09 -07:00
2025-07-01 08:41:53 -07:00

SeaweedFS Admin Component

A modern web-based administration interface for SeaweedFS clusters built with Go, Gin, Templ, and Bootstrap.

Features

  • Dashboard: Real-time cluster status and metrics
  • Master Management: Monitor master nodes and leadership status
  • Volume Server Management: View volume servers, capacity, and health
  • Object Store Bucket Management: Create, delete, and manage Object Store buckets with web interface
  • System Health: Overall cluster health monitoring
  • Responsive Design: Bootstrap-based UI that works on all devices
  • Authentication: Optional user authentication with sessions
  • TLS Support: HTTPS support for production deployments

Building

Using the Admin Makefile

The admin component has its own Makefile for development and building:

# Navigate to admin directory
cd weed/admin

# View all available targets
make help

# Generate templates and build
make build

# Development mode with template watching
make dev

# Run the admin server
make run

# Clean build artifacts
make clean

Using the Root Makefile

The root SeaweedFS Makefile automatically integrates the admin component:

# From the root directory
make install          # Builds weed with admin component
make full_install     # Full build with all tags
make test            # Runs tests including admin component

# Admin-specific targets from root
make admin-generate  # Generate admin templates
make admin-build     # Build admin component
make admin-run       # Run admin server
make admin-dev       # Development mode
make admin-clean     # Clean admin artifacts

Manual Building

If you prefer to build manually:

# Install templ compiler
go install github.com/a-h/templ/cmd/templ@latest

# Generate templates
templ generate

# Build the main weed binary
cd ../../../
go build -o weed ./weed

Development

Template Development

The admin interface uses Templ for type-safe HTML templates:

# Watch for template changes and auto-regenerate
make watch

# Or manually generate templates
make generate

# Format templates
make fmt

File Structure

weed/admin/
├── Makefile              # Admin-specific build tasks
├── README.md             # This file
├── S3_BUCKETS.md         # Object Store bucket management documentation
├── admin.go              # Main application entry point
├── dash/                 # Server and handler logic
│   ├── admin_server.go   # HTTP server setup
│   ├── handler_admin.go  # Admin dashboard handlers
│   ├── handler_auth.go   # Authentication handlers
│   └── middleware.go     # HTTP middleware
├── static/               # Static assets
│   ├── css/admin.css     # Admin-specific styles
│   └── js/admin.js       # Admin-specific JavaScript
└── view/                 # Templates
    ├── app/              # Application templates
    │   ├── admin.templ   # Main dashboard template
    │   ├── s3_buckets.templ # Object Store bucket management template
    │   └── *_templ.go    # Generated Go code
    └── layout/           # Layout templates
        ├── layout.templ  # Base layout template
        └── layout_templ.go # Generated Go code

S3 Bucket Management

The admin interface includes comprehensive Object Store bucket management capabilities. See S3_BUCKETS.md for detailed documentation on:

  • Creating and deleting Object Store buckets
  • Viewing bucket contents and metadata
  • Managing bucket permissions and settings
  • API endpoints for programmatic access

Usage

Basic Usage

# Start admin interface on default port (23646)
weed admin

# Start with custom configuration
weed admin -port=8080 -masters="master1:9333,master2:9333"

# Start with authentication
weed admin -adminUser=admin -adminPassword=secret123

# Start with HTTPS
weed admin -port=443 -tlsCert=/path/to/cert.pem -tlsKey=/path/to/key.pem

Configuration Options

Option Default Description
-port 23646 Admin server port
-masters localhost:9333 Comma-separated master servers
-adminUser admin Admin username (if auth enabled)
-adminPassword "" Admin password (empty = no auth)
-tlsCert "" Path to TLS certificate
-tlsKey "" Path to TLS private key

Docker Usage

# Build Docker image with admin component
make docker-build

# Run with Docker
docker run -p 23646:23646 seaweedfs/seaweedfs:latest admin -masters=host.docker.internal:9333

Development Workflow

Quick Start

# Clone and setup
git clone <seaweedfs-repo>
cd seaweedfs/weed/admin

# Install dependencies and build
make install-deps
make build

# Start development server
make dev

Making Changes

  1. Template Changes: Edit .templ files in view/

    • Templates auto-regenerate in development mode
    • Use make generate to manually regenerate
  2. Go Code Changes: Edit .go files

    • Restart the server to see changes
    • Use make build to rebuild
  3. Static Assets: Edit files in static/

    • Changes are served immediately

Testing

# Run admin component tests
make test

# Run from root directory
make admin-test

# Lint code
make lint

# Format code
make fmt

Production Deployment

Security Considerations

  1. Authentication: Always set adminPassword in production
  2. HTTPS: Use TLS certificates for encrypted connections
  3. Firewall: Restrict admin interface access to authorized networks

Example Production Setup

# Production deployment with security
weed admin \
  -port=443 \
  -masters="master1:9333,master2:9333,master3:9333" \
  -adminUser=admin \
  -adminPassword=your-secure-password \
  -tlsCert=/etc/ssl/certs/admin.crt \
  -tlsKey=/etc/ssl/private/admin.key

Monitoring

The admin interface provides endpoints for monitoring:

  • GET /health - Health check endpoint
  • GET /metrics - Prometheus metrics (if enabled)
  • GET /api/status - JSON status information

Troubleshooting

Common Issues

  1. Templates not found: Run make generate to create template files
  2. Build errors: Ensure templ is installed with make install-templ
  3. Static files not loading: Check that static/ directory exists and has proper files
  4. Connection errors: Verify master and filer addresses are correct

Debug Mode

# Enable debug logging
weed -v=2 admin

# Check generated templates
ls -la view/app/*_templ.go view/layout/*_templ.go

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Run tests: make test
  5. Format code: make fmt
  6. Submit a pull request

Architecture

The admin component follows a clean architecture:

  • Presentation Layer: Templ templates + Bootstrap CSS
  • HTTP Layer: Gin router with middleware
  • Business Logic: Handler functions in dash/ package
  • Data Layer: Communicates with SeaweedFS masters and filers

This separation makes the code maintainable and testable.