lsm/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2025-11-24 08:46:54 +08:00
Code Issues Actions 17 Packages Projects Releases Wiki Activity
11,542 Commits 51 Branches 316 Tags
5d8a391b956cb981b273e73e41093c679f9a2085
Commit Graph

825 Commits

Author SHA1 Message Date
Chris Lu
5d8a391b95 filer store: fix nil for mongodb (#6886) 
fix https://github.com/seaweedfs/seaweedfs/issues/6885
 2025-06-16 14:21:05 -07:00
Bruce Zou
fa730abec7 fix rocksdb enumerate (#6858) 2025-06-11 17:55:58 -07:00
FQHSLycopene
ee0c14673d Fix TTL Behavior for Directories in Path-Specific Configuration (#6827) 2025-05-29 02:38:12 -07:00
Aleksey Kosov
283d9e0079 Add context with request (#6824) 2025-05-28 11:34:02 -07:00
bwlfhu
f1181f1121 Fix mysql tls enable (#6807)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Details
2025-05-22 21:55:21 -07:00
Aleksey Kosov
165af32d6b added context to filer_client method calls (#6808) 
Co-authored-by: akosov <a.kosov@kryptonite.ru>
 2025-05-22 09:46:49 -07:00
SmoothDenis
45964c2f86 fix: ydb filer bugs (#6778)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details 
* fix: ydb filer bugs

* fix(ydb): correct DeleteEntry log argument types

* fix(ydb): bucket creation & deletion logic
 2025-05-12 01:20:25 -07:00
Kuzmin Anton
9c1048bacb Add prefix listing in mongodb_store.go (#6777)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
test s3 over https using aws-cli / awscli-tests (push) Waiting to run
Details
2025-05-11 07:13:49 -07:00
Jade Devin Cabatlao
21514e1fea feat(redis): add mTLS support for Redis connection initialization (#6738) 
* feat(redis): add mTLS support for Redis connection initialization

- Enhanced the Redis2Store initialization to support mutual TLS (mTLS) by adding configuration options for CA certificate, client certificate, and client key paths.
- Updated the Redis client setup to use TLS configuration when mTLS is enabled, ensuring secure connections to the Redis server.

* feat(redis): extend Redis3Store initialization to support mTLS

- Added configuration options for enabling mutual TLS (mTLS) in Redis3Store.
- Implemented logic to load client certificates and CA certificates for secure Redis connections.
- Updated the Redis client setup to utilize TLS configuration when mTLS is enabled.

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2025-05-04 05:31:27 -07:00
MaratKarimov
ba3afd1803 Tarantool filer store (#6669) 
Co-authored-by: Marat Karimov <m.karimov@digitalms.ru>
 2025-03-29 21:12:06 -07:00
Aleksey Kosov
43c3e80970 added a check for the nil value when requesting FindEntry. (#6651)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details 
Co-authored-by: akosov <a.kosov@kryptonite.ru>
 2025-03-21 00:40:28 -07:00
chrislu
9bd4c47299 update redis support status 2025-02-26 17:48:44 -08:00
chrislu
476aabb04b Update README.txt 2025-02-26 17:33:11 -08:00
Chris Lu
df436d342b add cassandra2 (#6582) 2025-02-26 17:29:22 -08:00
chrislu
299e372b48 prevent reading wrong data 
fix https://github.com/seaweedfs/seaweedfs/issues/6565
 2025-02-21 00:57:26 -08:00
Aleksey Kosov
41f5cc69b4 Cassandra loss dir (#6559)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details 
removed initialization of the local err

Co-authored-by: akosov <a.kosov@kryptonite.ru>
 2025-02-17 23:49:16 -08:00
Numblgw
c2b8942769 stop retry when all nodes have tried it (#6551) 
Co-authored-by: liguowei <liguowei@xinye.com>
 2025-02-17 15:46:31 -08:00
chrislu
42efade0dc adjust fix
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details 
fix https://github.com/seaweedfs/seaweedfs/issues/6497
 2025-02-01 14:11:57 -08:00
chrislu
a75271dd43 ensure correct auto bucket creation 
fix https://github.com/seaweedfs/seaweedfs/issues/6497
 2025-02-01 13:26:12 -08:00
MaratKarimov
4fe1248aa8 filer store YDB: fix upsert with TTL (#6469)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
2025-01-22 07:45:54 -08:00
Guang Jiong Lou
3b1ac77e1f worm grace period and retention time support (#6404) 
Signed-off-by: lou <alex1988@outlook.com>
 2024-12-31 18:41:43 -08:00
chrislu
ec155022e7 "golang.org/x/exp/slices" => "slices" and go fmt 2024-12-19 19:25:06 -08:00
chrislu
54204bc89b format error 2024-11-21 08:52:39 -08:00
Bruce
15fc58bfd9 remove dup code for section.reader access (#6169) 2024-11-05 19:08:59 -08:00
chrislu
706d1a8e22 fix tests 2024-11-05 08:52:07 -08:00
chrislu
90acfd9394 remove case when read request is out side of the file size 2024-11-05 08:42:44 -08:00
chrislu
98b519b113 fix FUSE mount on mac 2024-11-05 08:28:54 -08:00
Chris Lu
dc784bf217 merge current message queue code changes (#6201) 
* listing files to convert to parquet

* write parquet files

* save logs into parquet files

* pass by value

* compact logs into parquet format

* can skip existing files

* refactor

* refactor

* fix compilation

* when no partition found

* refactor

* add untested parquet file read

* rename package

* refactor

* rename files

* remove unused

* add merged log read func

* parquet wants to know the file size

* rewind by time

* pass in stop ts

* add stop ts

* adjust log

* minor

* adjust log

* skip .parquet files when reading message logs

* skip non message files

* Update subscriber_record.go

* send messages

* skip message data with only ts

* skip non log files

* update parquet-go package

* ensure a valid record type

* add new field to a record type

* Update read_parquet_to_log.go

* fix parquet file name generation

* separating reading parquet and logs

* add key field

* add skipped logs

* use in memory cache

* refactor

* refactor

* refactor

* refactor, and change compact log

* refactor

* rename

* refactor

* fix format

* prefix v to version directory
 2024-11-04 12:08:25 -08:00
Bruce
0060a2cf9c Fix 6181/6182 (#6183) 
* set larger buf size for LogBuffer

* jump to next day when no more entry found

* Update weed/filer/filer_notify_read.go

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-31 08:40:05 -07:00
Konstantin Lebedev
bae93e611d [filer] avoid 500 if table doesn't exist (#6075)
Some checks failed
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
helm: lint and test charts / lint-test (push) Has been cancelled
Details
2024-09-27 07:15:54 -07:00
Guang Jiong Lou
6c986e9d70 improve worm support (#5983) 
* improve worm support

Signed-off-by: lou <alex1988@outlook.com>

* worm mode in filer

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* move to fs configure

Signed-off-by: lou <alex1988@outlook.com>

* remove flag

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* support worm hardlink

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* typo

Signed-off-by: lou <alex1988@outlook.com>

* sync filer conf

Signed-off-by: lou <alex1988@outlook.com>

---------

Signed-off-by: lou <alex1988@outlook.com>
 2024-09-16 21:02:21 -07:00
Guang Jiong Lou
367e7638d7 fix invalid file read (#6024) 2024-09-16 08:45:50 -07:00
Bruce
5428229347 fix file read crash (#6021) 2024-09-14 08:33:35 -07:00
Aleksey Kosov
7340c62c47 changing FindEntry error handling in cassandra store (#6015) 2024-09-13 06:53:04 -07:00
Eugeniy E. Mikhailov
dab0bb8097 Feature limit caching to prescribed number of bytes per file (#6009) 
* feature: we can check if a fileId is already in the cache

We using this to protect cache from adding the same needle to
the cache over and over.

* fuse mount: Do not start dowloader if needle is already in the cache

* added maxFilePartSizeInCache property to ChunkCache

If file very large only first maxFilePartSizeInCache bytes
are going to be put to the cache (subject to the needle size
constrains).

* feature: for large files put in cache no more than prescribed number of bytes

Before this patch only the first needle of a large file was intended for
caching. This patch uses maximum prescribed amount of bytes to be put in
cache. This allows to bypass default 2MB maximum for a file part stored
in the cache.

* added dummy mock methods to satisfy interfaces of ChunkCache
 2024-09-11 21:09:20 -07:00
chrislu
f23015ac5b weed mount caching 2024-09-10 10:30:05 -07:00
chrislu
93decd1114 remove unused 2024-09-10 10:29:25 -07:00
chrislu
69fcdd0840 adjust logging 2024-09-10 10:28:49 -07:00
zouyixiong
d8e18a8eb7 bug fixed to filer store elastic deleteEntry (#5988) 2024-09-09 02:15:25 -07:00
chrislu
18afdb15b6 Revert "weed mount, weed dav add option to force cache" 
This reverts commit 7367b976b0.
 2024-09-04 01:38:29 -07:00
chrislu
7367b976b0 weed mount, weed dav add option to force cache 2024-09-04 01:19:14 -07:00
chrislu
d49ecde535 rename functions 2024-08-01 23:27:23 -07:00
chrislu
4fee496b49 conditional delete 2024-08-01 13:00:54 -07:00
wyang
0581ce6096 fix delete chunk failed if volumeSever specified grpc.port (#5820) 
Co-authored-by: Yang Wang <yangwang@weride.ai>
 2024-07-26 11:14:19 -07:00
chrislu
aed2a18223 get, set and add path conf 2024-07-20 11:27:11 -07:00
chrislu
a8fa78b892 refactoring 2024-07-17 23:14:36 -07:00
vadimartynov
86d92a42b4 Added tls for http clients (#5766) 
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
 2024-07-16 23:14:09 -07:00
wusong
ce61a66b65 Fix mmap write fail (#5791) 
* [mount] fix GetAttr blocks count

Signed-off-by: wang wusong <wangwusong@virtaitech.com>

* [mount] fix mmap read

Signed-off-by: wang wusong <wangwusong@virtaitech.com>

---------

Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2024-07-16 07:59:55 -07:00
chrislu
4e7d8eb3f1 small refactoring 2024-07-11 14:50:11 -07:00
chrislu
2addcd1623 fix test 2024-07-11 07:53:02 -07:00