lsm/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2025-10-08 03:14:24 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
11,576 Commits 53 Branches 313 Tags
5e79436498a25f6b085f77980b0ffb377409174c
Commit Graph

1238 Commits

Author SHA1 Message Date
chrislu
5e79436498 reference seaweedfs.com 2025-06-26 11:09:17 -07:00
chrislu
ab49540d2b use master.toml value if not empty 
fix https://github.com/seaweedfs/seaweedfs/issues/6922
 2025-06-25 17:54:56 -07:00
chrislu
2f1b3d68d7 pass volume version when creating a volume 2025-06-19 01:15:25 -07:00
Erik Jacobson
77397be070 allow distro mount command to use disableXAttr (#6872) 2025-06-16 08:45:05 -07:00
chrislu
c79e73aa2a mount: complete fix for freebsd 
fix https://github.com/seaweedfs/seaweedfs/issues/6645
 2025-06-12 08:19:47 -07:00
chrislu
2f3de5e199 fix build
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Details
2025-06-03 22:50:45 -07:00
chrislu
bd4891a117 change version directory 2025-06-03 22:46:10 -07:00
Luna Yao
5e354a18a3 Update filer upsert sqlstring for postgresql (#6835) 2025-06-02 10:46:50 -07:00
Aleksey Kosov
283d9e0079 Add context with request (#6824) 2025-05-28 11:34:02 -07:00
Weihao Jiang
ea70d17c5f Make fuse command linux/MacOS only (#6811)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details
2025-05-23 08:21:46 -07:00
bwlfhu
f1181f1121 Fix mysql tls enable (#6807)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Details
2025-05-22 21:55:21 -07:00
chrislu
f3d44b1eb6 Update fuse.go
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Details 
Changed the signal from SIGUSR1 to SIGTERM. This should fix the compilation error since SIGTERM is available on all Unix-like systems including macOS. The functionality remains similar - it will still allow the master process to wait for a signal before exiting, just using a different signal type.
 2025-05-22 11:32:08 -07:00
chrislu
25d6310307 fix error format
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Details
2025-05-22 09:56:31 -07:00
Weihao Jiang
874b4a5535 Ensure weed fuse master process exits after mounted (#6809) 
* Ensure fuse master process wait for mounted

* Validate parent PID input in fuse command
 2025-05-22 09:50:07 -07:00
Aleksey Kosov
165af32d6b added context to filer_client method calls (#6808) 
Co-authored-by: akosov <a.kosov@kryptonite.ru>
 2025-05-22 09:46:49 -07:00
Mohamed Sekour
93aed187e9 Add SFTP Server Support (#6753) 
* Add SFTP Server Support

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* fix s3 tests and helm lint

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* increase  helm chart version

* adjust version

---------

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2025-05-05 11:43:49 -07:00
ARibster
a2c5510ae1 Addendum to issue #6733 (#6756) 
add s3.ip.bind command line parameter to filer
 2025-05-05 10:30:54 -07:00
chrislu
ed15357c7a adds tls examples
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Details 
related to https://github.com/seaweedfs/seaweedfs/pull/6738
 2025-05-04 05:46:33 -07:00
ARibster
d424cbbcd5 fix: S3 Feature: please add s3.idleTimeout command line parameter #6746 (#6747) 
ildeTimeout command line parameter for s3
 2025-05-01 13:09:47 -07:00
ARibster
9f4be83e88 fix #6733 add s3.ip.bind command line parameter (#6741)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details
2025-04-28 04:49:01 -07:00
Konstantin Lebedev
aa9121aa9d Revert "[filer] void panic if s3opt.tlsCACertificate is nill" (#6694) 
Revert "[filer] void panic if s3opt.tlsCACertificate is nill (#6685)"

This reverts commit 4dca59ed27.
 2025-04-07 11:09:40 -07:00
Konstantin Lebedev
4dca59ed27 [filer] void panic if s3opt.tlsCACertificate is nill (#6685) 
* fix issues/6684

* add s3.cacert.file option for filer
 2025-04-07 09:45:49 -07:00
MaratKarimov
ba3afd1803 Tarantool filer store (#6669) 
Co-authored-by: Marat Karimov <m.karimov@digitalms.ru>
 2025-03-29 21:12:06 -07:00
Chris Lu
669a3917af add lua redis store to ensure atomicity (#6671) 2025-03-28 11:27:39 -07:00
chrislu
138b66231a fix help message
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
2025-03-25 09:30:56 -07:00
Karol Będkowski
95746f48e6 feat/fix: add s3.cacert.file/tlsVerifyClientCert to filer command (#6659) 
This prevent crash filler with nil pointer dereference as s3 expect this
parameters.

New two parameters are added to filer command - copy of s3 parameters:
- s3.cacert.file - path to the TLS CA certificate file
- s3.tlsVerifyClientCert - whether to verify the client's certificate
 2025-03-24 10:14:31 -07:00
Weihao Jiang
75ef324533 Allow standalone webdav server to bind specific address (#6403) 
* Allow standalone webdav to bind specific address

* Rename bindIp to ipBind

* Bind embedded webdav IP to its parent
 2025-03-17 23:01:26 -07:00
Chris Lu
02773a6107 Accumulated changes for message queue (#6600)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details 
* rename

* set agent address

* refactor

* add agent sub

* pub messages

* grpc new client

* can publish records via agent

* send init message with session id

* fmt

* check cancelled request while waiting

* use sessionId

* handle possible nil stream

* subscriber process messages

* separate debug port

* use atomic int64

* less logs

* minor

* skip io.EOF

* rename

* remove unused

* use saved offsets

* do not reuse session, since always session id is new after restart

remove last active ts from SessionEntry

* simplify printing

* purge unused

* just proxy the subscription, skipping the session step

* adjust offset types

* subscribe offset type and possible value

* start after the known tsns

* avoid wrongly set startPosition

* move

* remove

* refactor

* typo

* fix

* fix changed path
 2025-03-09 23:49:42 -07:00
lizhengui007
61249d8dde clear cache directory when mount exits (#6605)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details 
Signed-off-by: lizhengui <lizhengui@virtaitech.com>
Co-authored-by: lizhengui <lizhengui@virtaitech.com>
 2025-03-06 08:14:17 -08:00
chrislu
314f211260 Update filer.toml
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details
2025-02-26 17:52:45 -08:00
chrislu
9bd4c47299 update redis support status 2025-02-26 17:48:44 -08:00
chrislu
4613186e8a not planning to redis3 2025-02-26 17:35:31 -08:00
Chris Lu
df436d342b add cassandra2 (#6582) 2025-02-26 17:29:22 -08:00
Karol Będkowski
441614b386 fix: s3 command ignore -tlsVerifyClientCert and -cacert.file arguments (#6547)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
helm: lint and test charts / lint-test (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details 
s3 command ignore tlsVerifyClientCert and cacert.file arguments from
command line. On startS3Server instead of use real values (in s3opt),
default values (from s3Options, always empty) are checked.

Now on right values are checked and if user provide this arguments
RequireAndVerifyClientCert is set and/or ca certificate is loaded.
 2025-02-13 13:27:38 -08:00
Lisandro Pin
e8d8bfcccc Nit: remove missing newlines on weed shell commands output. (#6524) 
Nit: remove missing newlines on `weed` commands output.
 2025-02-07 10:27:04 -08:00
Ethan Mosbaugh
9ebc132ffc fix: bucket-hook fails with gnu wget (#6521)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
2025-02-06 23:11:17 -08:00
Chris Lu
cc05874d06 Add message queue agent (#6463) 
* scaffold message queue agent

* adjust proto, add mq_agent

* add agent client implementation

* remove unused function

* agent publish server implementation

* adding agent
 2025-01-20 22:19:27 -08:00
Tom Crasset
c5f21b2b01 fix S3 per-user-directory Policy (#6443)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details 
* fix S3 per-user-directory Policy

* Delete docker/config.json

* add tests

* remove logs

* undo modifications of weed/shell/command_volume_balance.go

* remove modifications of docker-compose

* fix failing test

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2025-01-17 01:03:17 -08:00
chrislu
ec155022e7 "golang.org/x/exp/slices" => "slices" and go fmt 2024-12-19 19:25:06 -08:00
zemul
e77e50886e mount metacache add ttl (#6360) 
* fix:mount deadlock

* fix

* feat: metaCache ttl

* Update weed/command/mount.go

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

* fix InodeEntry

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-12-16 20:19:32 -08:00
vadimartynov
914ff0e718 Replaced https.client.enabled with false in scaffold/security (#6320) 2024-12-05 10:39:28 -08:00
Konstantin Lebedev
b65eb2ec45 [security] reload whiteList on http seerver (#6302) 
* reload whiteList

* white_list add to scaffold
 2024-12-02 10:38:10 -08:00
Numblgw
aebf3952b7 filer sync: source path and exclude path support dir suffix (#6268) 
filer sync: source path and exclude paht support dir suffix

Co-authored-by: liguowei <liguowei@xinye.com>
 2024-11-21 08:25:12 -08:00
zemul
6d3e38f439 Adjust threshold config (#6246) 
* fix:mount deadlock

* fix

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
 2024-11-17 20:51:26 -08:00
Max Denushev
a5fe6e21bc feat(filer.backup): add ignore errors option (#6235) 
* feat(filer.backup): add ignore errors option

* feat(filer.backup): fix 404 error wrap

* feat(filer.backup): fix wrapping function

* feat(filer.backup): fix wrapping errors in genProcessFunction

* Update weed/command/filer_backup.go

* Update weed/command/filer_backup.go

* Update weed/command/filer_backup.go

---------

Co-authored-by: Max Denushev <denushev@tochka.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-11-14 08:40:55 -08:00
chrislu
113c9ce6a8 remove the direct_io flag, as it is not well-supported on macOS 2024-11-07 14:18:59 -08:00
Chris Lu
dc784bf217 merge current message queue code changes (#6201) 
* listing files to convert to parquet

* write parquet files

* save logs into parquet files

* pass by value

* compact logs into parquet format

* can skip existing files

* refactor

* refactor

* fix compilation

* when no partition found

* refactor

* add untested parquet file read

* rename package

* refactor

* rename files

* remove unused

* add merged log read func

* parquet wants to know the file size

* rewind by time

* pass in stop ts

* add stop ts

* adjust log

* minor

* adjust log

* skip .parquet files when reading message logs

* skip non message files

* Update subscriber_record.go

* send messages

* skip message data with only ts

* skip non log files

* update parquet-go package

* ensure a valid record type

* add new field to a record type

* Update read_parquet_to_log.go

* fix parquet file name generation

* separating reading parquet and logs

* add key field

* add skipped logs

* use in memory cache

* refactor

* refactor

* refactor

* refactor, and change compact log

* refactor

* rename

* refactor

* fix format

* prefix v to version directory
 2024-11-04 12:08:25 -08:00
chrislu
6c7824c813 consistent sorted tags elastic,gocdk,rclone,sqlite,tikv,ydb 2024-10-21 22:53:29 -07:00
chrislu
804e765397 remove unused
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
2024-10-13 23:12:10 -07:00
Guang Jiong Lou
6c986e9d70 improve worm support (#5983) 
* improve worm support

Signed-off-by: lou <alex1988@outlook.com>

* worm mode in filer

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* move to fs configure

Signed-off-by: lou <alex1988@outlook.com>

* remove flag

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* support worm hardlink

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* typo

Signed-off-by: lou <alex1988@outlook.com>

* sync filer conf

Signed-off-by: lou <alex1988@outlook.com>

---------

Signed-off-by: lou <alex1988@outlook.com>
 2024-09-16 21:02:21 -07:00