Commit Graph

11517 Commits

Author SHA1 Message Date
dependabot[bot]
e9c2d5d4f8
chore(deps): bump github.com/redis/go-redis/v9 from 9.8.0 to 9.10.0 (#6850)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.8.0 to 9.10.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.8.0...v9.10.0)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-version: 9.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 10:38:36 -07:00
dependabot[bot]
6e5e6df326
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.80.0 to 1.80.1 (#6852)
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3

Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.80.0 to 1.80.1.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.80.0...service/s3/v1.80.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.80.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 10:38:22 -07:00
dependabot[bot]
f3b1748eb7
chore(deps): bump go.etcd.io/etcd/client/v3 from 3.6.0 to 3.6.1 (#6855)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.0 to 3.6.1.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.0...v3.6.1)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 10:37:57 -07:00
dependabot[bot]
dcabb71a28
chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.67 to 1.17.68 (#6853) 2025-06-09 09:17:09 -07:00
dependabot[bot]
1023971dbc
chore(deps): bump modernc.org/sqlite from 1.37.1 to 1.38.0 (#6854) 2025-06-09 09:16:49 -07:00
chrislu
33ecc8442e refactor
Some checks failed
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
helm: lint and test charts / lint-test (push) Has been cancelled
2025-06-08 22:11:09 -07:00
chrislu
81aeec74a4 3.90 2025-06-08 20:59:43 -07:00
chrislu
60f11f6510 add a readme file for volume needle data layout
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
2025-06-07 15:52:51 -07:00
chrislu
61c4f01e05 refactor
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
2025-06-06 08:55:32 -07:00
chrislu
ce51d60127 include new changes related to FreeBSD
50f6569d1a
2025-06-06 08:48:17 -07:00
chrislu
a489d99333 fix tests 2025-06-06 08:25:04 -07:00
Lisandro Pin
00c621abb8
Fix dumb typo in 08556257 (#6844) 2025-06-06 05:59:11 -07:00
SmoothDenis
c99530ec66
fix(filer): add missing return after KvDelete in KvPut with empty value (#6846) 2025-06-06 05:58:03 -07:00
chrislu
7439af0eca refactoring 2025-06-06 01:35:48 -07:00
chrislu
cc135c63f7 a bit refactoring 2025-06-06 01:26:54 -07:00
chrislu
c4695fc3b3 refactor needle write for different versions 2025-06-06 00:35:13 -07:00
Lisandro Pin
bed0a64693
New needle_map.CompactMap() implementation for reduced memory usage (#6842)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
* Rework `needle_map.CompactMap()` to maximize memory efficiency.

* Use a memory-efficient structure for `CompactMap` needle value entries.

This slightly complicates the code, but makes a **massive** difference
in memory efficiency - preliminary results show a ~30% reduction in
heap usage, with no measurable performance impact otherwise.

* Clean up type for `CompactMap` chunk IDs.

* Add a small comment description for `CompactMap()`.

* Add the old version of `CompactMap()` for comparison purposes.
2025-06-05 14:03:29 -07:00
chrislu
d8ddc22fc2 update to latest hanwen/go-fuse/v2
https://github.com/seaweedfs/seaweedfs/issues/6645#issuecomment-2933832235

fix https://github.com/seaweedfs/seaweedfs/issues/6645
2025-06-05 09:20:14 -07:00
chrislu
35f0daa198 the isFsync parameter is essentially IsAsyncWrite and it needs to be turned off if s.isStopping
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
d8c574a5ef (r159132764)
2025-06-05 00:19:10 -07:00
chrislu
2f3de5e199 fix build
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
2025-06-03 22:50:45 -07:00
chrislu
bd4891a117 change version directory 2025-06-03 22:46:10 -07:00
chrislu
7039d5003c gorocksdb 1.10.1 ~ rocksdb 10.2.1 2025-06-03 22:46:10 -07:00
Chris Lu
7151a54b28 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
Some checks failed
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
helm: lint and test charts / lint-test (push) Has been cancelled
2025-06-02 23:57:54 -07:00
Chris Lu
b25561d0d7 3.89 2025-06-02 23:56:58 -07:00
dependabot[bot]
9994617dad
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.78.2 to 1.80.0 (#6830)
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3

Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.78.2 to 1.80.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.78.2...service/s3/v1.80.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.80.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:55:52 -07:00
dependabot[bot]
7265a9c3b7
chore(deps): bump modernc.org/sqlite from 1.37.0 to 1.37.1 (#6831)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.37.0 to 1.37.1.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.37.0...v1.37.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.37.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:54:18 -07:00
dependabot[bot]
4217f0dcaf
chore(deps): bump github.com/prometheus/client_golang from 1.21.1 to 1.22.0 (#6832)
chore(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.1 to 1.22.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-version: 1.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:52:23 -07:00
dependabot[bot]
aeb3bcdfc1
chore(deps): bump docker/build-push-action from 6.17.0 to 6.18.0 (#6834)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.17.0 to 6.18.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](1dc7386353...263435318d)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 6.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:52:11 -07:00
dependabot[bot]
2c494751e5
chore(deps): bump github.com/fluent/fluent-logger-golang from 1.9.0 to 1.10.0 (#6833)
chore(deps): bump github.com/fluent/fluent-logger-golang

Bumps [github.com/fluent/fluent-logger-golang](https://github.com/fluent/fluent-logger-golang) from 1.9.0 to 1.10.0.
- [Changelog](https://github.com/fluent/fluent-logger-golang/blob/master/CHANGELOG.md)
- [Commits](https://github.com/fluent/fluent-logger-golang/compare/v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: github.com/fluent/fluent-logger-golang
  dependency-version: 1.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:52:07 -07:00
Chris Lu
d40746f34e
fix insert beyond look back window (#6838) 2025-06-02 23:43:01 -07:00
Lisandro Pin
7204731749
Minor fix for the CompactMap() performance test. (#6836)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
helm: lint and test charts / lint-test (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Per-entry memory usage is based on `TotalAllocs`, which is incorrect - that
value is a cummulative of heap usage, which doesn't decrease when objects
are freeed.

`Allocs` is instead an accurate represeentation of actual memory usage
at the time metrics are reported.
2025-06-02 17:09:01 -07:00
Luna Yao
5e354a18a3
Update filer upsert sqlstring for postgresql (#6835) 2025-06-02 10:46:50 -07:00
Chris Lu
90802cb201
revert part of d8c574a5ef (#6829) 2025-06-01 12:27:49 -07:00
FQHSLycopene
ee0c14673d
Fix TTL Behavior for Directories in Path-Specific Configuration (#6827) 2025-05-29 02:38:12 -07:00
Lisandro Pin
9ffc8bcb54
Further improve memory usage of needle_map.CompactMap(). (#6825) 2025-05-28 11:42:00 -07:00
Aleksey Kosov
283d9e0079
Add context with request (#6824) 2025-05-28 11:34:02 -07:00
Bruce Zou
62aaaa18f3
unlimit the list entries in completeMultipartUpload (#6822) 2025-05-27 03:41:27 -07:00
royatwp
2786bea839
Unable to upload empty files to seaweed via. multipart when length is unknown (#6729) 2025-05-26 23:25:01 -07:00
chrislu
6677f1f563 3.88 2025-05-26 21:54:49 +02:00
Chris Lu
c1c09b60a1
Update README.md 2025-05-26 12:53:39 -07:00
dependabot[bot]
aa5b927a18
chore(deps): bump golang.org/x/image from 0.24.0 to 0.27.0 (#6817)
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.24.0 to 0.27.0.
- [Commits](https://github.com/golang/image/compare/v0.24.0...v0.27.0)

---
updated-dependencies:
- dependency-name: golang.org/x/image
  dependency-version: 0.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-26 12:51:01 -07:00
dependabot[bot]
2d77c15569
chore(deps): bump github.com/rclone/rclone from 1.69.1 to 1.69.3 (#6816) 2025-05-26 08:50:01 -07:00
dependabot[bot]
dd5a24fb89
chore(deps): bump github.com/cognusion/imaging from 1.0.1 to 1.0.2 (#6818) 2025-05-26 08:49:39 -07:00
dependabot[bot]
a907e71a96
chore(deps): bump google.golang.org/api from 0.232.0 to 0.234.0 (#6819) 2025-05-26 08:49:22 -07:00
Mohamed Sekour
27a392f706
Fix sftp performances and add seaweedfs all-in-one deployment (#6792)
* improve perfs & fix rclone & refactoring
Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* improve perfs on download + add seaweedfs all-in-one deployment

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* use helper for topologySpreadConstraints and fix create home dir of sftp users

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* fix helm lint

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* add missing ctx param

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

---------

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
2025-05-26 00:50:48 -07:00
Weihao Jiang
ea70d17c5f
Make fuse command linux/MacOS only (#6811)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
2025-05-23 08:21:46 -07:00
Lisandro Pin
2e1506c31e
Rewrite needle_map.CompactMap() for more efficient memory usage (#6813) 2025-05-23 07:05:08 -07:00
bwlfhu
f1181f1121
Fix mysql tls enable (#6807)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
2025-05-22 21:55:21 -07:00
chrislu
f3d44b1eb6 Update fuse.go
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Changed the signal from SIGUSR1 to SIGTERM. This should fix the compilation error since SIGTERM is available on all Unix-like systems including macOS. The functionality remains similar - it will still allow the master process to wait for a signal before exiting, just using a different signal type.
2025-05-22 11:32:08 -07:00
chrislu
896096c41b fix compilation on windows 2025-05-22 11:25:56 -07:00