Commit Graph

11528 Commits

Author SHA1 Message Date
chrislu
c26299b05a 3.91 3.91 2025-06-15 20:42:16 -07:00
NyaMisty
f894e7b7a5 Support filtering source disk type in volume.tier.upload (#6868) 2025-06-15 20:30:04 -07:00
NyaMisty
53e5c84523 Fix wrong error handling in volume.tier.upload when stream == nil but copyErr != nil (#6867) 2025-06-15 20:28:40 -07:00
NyaMisty
cdc543aa9e Correctly sort in volume.list to ensure output consistency (#6866) 2025-06-15 20:27:48 -07:00
Gerry Hernandez
e653de54b4 FUSE Mount: Fix buffer allocation during copy (#6863)
Fix buffer allocation during FUSE copy
2025-06-13 12:27:39 -07:00
chrislu
c79e73aa2a mount: complete fix for freebsd
fix https://github.com/seaweedfs/seaweedfs/issues/6645
2025-06-12 08:19:47 -07:00
chrislu
e71d681fee refactor 2025-06-11 20:46:13 -07:00
chrislu
7c4d98446b refactor 2025-06-11 20:13:06 -07:00
chrislu
f27e195354 refactoring 2025-06-11 20:13:06 -07:00
Bruce Zou
fa730abec7 fix rocksdb enumerate (#6858) 2025-06-11 17:55:58 -07:00
dependabot[bot]
f7df4856e5 chore(deps): bump github.com/cloudflare/circl from 1.3.7 to 1.6.1 (#6856)
Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.3.7 to 1.6.1.
- [Release notes](https://github.com/cloudflare/circl/releases)
- [Commits](https://github.com/cloudflare/circl/compare/v1.3.7...v1.6.1)

---
updated-dependencies:
- dependency-name: github.com/cloudflare/circl
  dependency-version: 1.6.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-10 19:19:12 -07:00
dependabot[bot]
e9c2d5d4f8 chore(deps): bump github.com/redis/go-redis/v9 from 9.8.0 to 9.10.0 (#6850)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.8.0 to 9.10.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.8.0...v9.10.0)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-version: 9.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 10:38:36 -07:00
dependabot[bot]
6e5e6df326 chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.80.0 to 1.80.1 (#6852)
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3

Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.80.0 to 1.80.1.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.80.0...service/s3/v1.80.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.80.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 10:38:22 -07:00
dependabot[bot]
f3b1748eb7 chore(deps): bump go.etcd.io/etcd/client/v3 from 3.6.0 to 3.6.1 (#6855)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.0 to 3.6.1.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.0...v3.6.1)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 10:37:57 -07:00
dependabot[bot]
dcabb71a28 chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.67 to 1.17.68 (#6853) 2025-06-09 09:17:09 -07:00
dependabot[bot]
1023971dbc chore(deps): bump modernc.org/sqlite from 1.37.1 to 1.38.0 (#6854) 2025-06-09 09:16:49 -07:00
chrislu
33ecc8442e refactor
Some checks failed
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
helm: lint and test charts / lint-test (push) Has been cancelled
2025-06-08 22:11:09 -07:00
chrislu
81aeec74a4 3.90 3.90 2025-06-08 20:59:43 -07:00
chrislu
60f11f6510 add a readme file for volume needle data layout
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
2025-06-07 15:52:51 -07:00
chrislu
61c4f01e05 refactor
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
2025-06-06 08:55:32 -07:00
chrislu
ce51d60127 include new changes related to FreeBSD
50f6569d1a
2025-06-06 08:48:17 -07:00
chrislu
a489d99333 fix tests 2025-06-06 08:25:04 -07:00
Lisandro Pin
00c621abb8 Fix dumb typo in 08556257 (#6844) 2025-06-06 05:59:11 -07:00
SmoothDenis
c99530ec66 fix(filer): add missing return after KvDelete in KvPut with empty value (#6846) 2025-06-06 05:58:03 -07:00
chrislu
7439af0eca refactoring 2025-06-06 01:35:48 -07:00
chrislu
cc135c63f7 a bit refactoring 2025-06-06 01:26:54 -07:00
chrislu
c4695fc3b3 refactor needle write for different versions 2025-06-06 00:35:13 -07:00
Lisandro Pin
bed0a64693 New needle_map.CompactMap() implementation for reduced memory usage (#6842)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
* Rework `needle_map.CompactMap()` to maximize memory efficiency.

* Use a memory-efficient structure for `CompactMap` needle value entries.

This slightly complicates the code, but makes a **massive** difference
in memory efficiency - preliminary results show a ~30% reduction in
heap usage, with no measurable performance impact otherwise.

* Clean up type for `CompactMap` chunk IDs.

* Add a small comment description for `CompactMap()`.

* Add the old version of `CompactMap()` for comparison purposes.
2025-06-05 14:03:29 -07:00
chrislu
d8ddc22fc2 update to latest hanwen/go-fuse/v2
https://github.com/seaweedfs/seaweedfs/issues/6645#issuecomment-2933832235

fix https://github.com/seaweedfs/seaweedfs/issues/6645
2025-06-05 09:20:14 -07:00
chrislu
35f0daa198 the isFsync parameter is essentially IsAsyncWrite and it needs to be turned off if s.isStopping
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
d8c574a5ef (r159132764)
2025-06-05 00:19:10 -07:00
chrislu
2f3de5e199 fix build
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
2025-06-03 22:50:45 -07:00
chrislu
bd4891a117 change version directory 2025-06-03 22:46:10 -07:00
chrislu
7039d5003c gorocksdb 1.10.1 ~ rocksdb 10.2.1 2025-06-03 22:46:10 -07:00
Chris Lu
7151a54b28 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
Some checks failed
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
helm: lint and test charts / lint-test (push) Has been cancelled
3.89
2025-06-02 23:57:54 -07:00
Chris Lu
b25561d0d7 3.89 2025-06-02 23:56:58 -07:00
dependabot[bot]
9994617dad chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.78.2 to 1.80.0 (#6830)
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3

Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.78.2 to 1.80.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.78.2...service/s3/v1.80.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.80.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:55:52 -07:00
dependabot[bot]
7265a9c3b7 chore(deps): bump modernc.org/sqlite from 1.37.0 to 1.37.1 (#6831)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.37.0 to 1.37.1.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.37.0...v1.37.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.37.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:54:18 -07:00
dependabot[bot]
4217f0dcaf chore(deps): bump github.com/prometheus/client_golang from 1.21.1 to 1.22.0 (#6832)
chore(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.1 to 1.22.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-version: 1.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:52:23 -07:00
dependabot[bot]
aeb3bcdfc1 chore(deps): bump docker/build-push-action from 6.17.0 to 6.18.0 (#6834)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.17.0 to 6.18.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](1dc7386353...263435318d)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 6.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:52:11 -07:00
dependabot[bot]
2c494751e5 chore(deps): bump github.com/fluent/fluent-logger-golang from 1.9.0 to 1.10.0 (#6833)
chore(deps): bump github.com/fluent/fluent-logger-golang

Bumps [github.com/fluent/fluent-logger-golang](https://github.com/fluent/fluent-logger-golang) from 1.9.0 to 1.10.0.
- [Changelog](https://github.com/fluent/fluent-logger-golang/blob/master/CHANGELOG.md)
- [Commits](https://github.com/fluent/fluent-logger-golang/compare/v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: github.com/fluent/fluent-logger-golang
  dependency-version: 1.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-06-02 23:52:07 -07:00
Chris Lu
d40746f34e fix insert beyond look back window (#6838) 2025-06-02 23:43:01 -07:00
Lisandro Pin
7204731749 Minor fix for the CompactMap() performance test. (#6836)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
helm: lint and test charts / lint-test (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Per-entry memory usage is based on `TotalAllocs`, which is incorrect - that
value is a cummulative of heap usage, which doesn't decrease when objects
are freeed.

`Allocs` is instead an accurate represeentation of actual memory usage
at the time metrics are reported.
2025-06-02 17:09:01 -07:00
Luna Yao
5e354a18a3 Update filer upsert sqlstring for postgresql (#6835) 2025-06-02 10:46:50 -07:00
Chris Lu
90802cb201 revert part of d8c574a5ef (#6829) 2025-06-01 12:27:49 -07:00
FQHSLycopene
ee0c14673d Fix TTL Behavior for Directories in Path-Specific Configuration (#6827) 2025-05-29 02:38:12 -07:00
Lisandro Pin
9ffc8bcb54 Further improve memory usage of needle_map.CompactMap(). (#6825) 2025-05-28 11:42:00 -07:00
Aleksey Kosov
283d9e0079 Add context with request (#6824) 2025-05-28 11:34:02 -07:00
Bruce Zou
62aaaa18f3 unlimit the list entries in completeMultipartUpload (#6822) 2025-05-27 03:41:27 -07:00
royatwp
2786bea839 Unable to upload empty files to seaweed via. multipart when length is unknown (#6729) 2025-05-26 23:25:01 -07:00
chrislu
6677f1f563 3.88 3.88 2025-05-26 21:54:49 +02:00