Chris Lu
cc05874d06
Add message queue agent ( #6463 )
...
* scaffold message queue agent
* adjust proto, add mq_agent
* add agent client implementation
* remove unused function
* agent publish server implementation
* adding agent
2025-01-20 22:19:27 -08:00
Tom Crasset
c5f21b2b01
fix S3 per-user-directory Policy ( #6443 )
...
go: build dev binaries / cleanup (push) Has been cancelled
docker: build dev containers / build-dev-containers (push) Has been cancelled
End to End / FUSE Mount (push) Has been cancelled
go: build binary / Build (push) Has been cancelled
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
* fix S3 per-user-directory Policy
* Delete docker/config.json
* add tests
* remove logs
* undo modifications of weed/shell/command_volume_balance.go
* remove modifications of docker-compose
* fix failing test
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com >
2025-01-17 01:03:17 -08:00
chrislu
ec155022e7
"golang.org/x/exp/slices" => "slices" and go fmt
2024-12-19 19:25:06 -08:00
zemul
e77e50886e
mount metacache add ttl ( #6360 )
...
* fix:mount deadlock
* fix
* feat: metaCache ttl
* Update weed/command/mount.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com >
* fix InodeEntry
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com >
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com >
2024-12-16 20:19:32 -08:00
vadimartynov
914ff0e718
Replaced https.client.enabled with false in scaffold/security ( #6320 )
2024-12-05 10:39:28 -08:00
Konstantin Lebedev
b65eb2ec45
[security] reload whiteList on http seerver ( #6302 )
...
* reload whiteList
* white_list add to scaffold
2024-12-02 10:38:10 -08:00
Numblgw
aebf3952b7
filer sync: source path and exclude path support dir suffix ( #6268 )
...
filer sync: source path and exclude paht support dir suffix
Co-authored-by: liguowei <liguowei@xinye.com >
2024-11-21 08:25:12 -08:00
zemul
6d3e38f439
Adjust threshold config ( #6246 )
...
* fix:mount deadlock
* fix
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com >
2024-11-17 20:51:26 -08:00
Max Denushev
a5fe6e21bc
feat(filer.backup): add ignore errors option ( #6235 )
...
* feat(filer.backup): add ignore errors option
* feat(filer.backup): fix 404 error wrap
* feat(filer.backup): fix wrapping function
* feat(filer.backup): fix wrapping errors in genProcessFunction
* Update weed/command/filer_backup.go
* Update weed/command/filer_backup.go
* Update weed/command/filer_backup.go
---------
Co-authored-by: Max Denushev <denushev@tochka.com >
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com >
2024-11-14 08:40:55 -08:00
chrislu
113c9ce6a8
remove the direct_io flag, as it is not well-supported on macOS
2024-11-07 14:18:59 -08:00
Chris Lu
dc784bf217
merge current message queue code changes ( #6201 )
...
* listing files to convert to parquet
* write parquet files
* save logs into parquet files
* pass by value
* compact logs into parquet format
* can skip existing files
* refactor
* refactor
* fix compilation
* when no partition found
* refactor
* add untested parquet file read
* rename package
* refactor
* rename files
* remove unused
* add merged log read func
* parquet wants to know the file size
* rewind by time
* pass in stop ts
* add stop ts
* adjust log
* minor
* adjust log
* skip .parquet files when reading message logs
* skip non message files
* Update subscriber_record.go
* send messages
* skip message data with only ts
* skip non log files
* update parquet-go package
* ensure a valid record type
* add new field to a record type
* Update read_parquet_to_log.go
* fix parquet file name generation
* separating reading parquet and logs
* add key field
* add skipped logs
* use in memory cache
* refactor
* refactor
* refactor
* refactor, and change compact log
* refactor
* rename
* refactor
* fix format
* prefix v to version directory
2024-11-04 12:08:25 -08:00
chrislu
6c7824c813
consistent sorted tags elastic,gocdk,rclone,sqlite,tikv,ydb
2024-10-21 22:53:29 -07:00
chrislu
804e765397
remove unused
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
2024-10-13 23:12:10 -07:00
Guang Jiong Lou
6c986e9d70
improve worm support ( #5983 )
...
* improve worm support
Signed-off-by: lou <alex1988@outlook.com >
* worm mode in filer
Signed-off-by: lou <alex1988@outlook.com >
* update after review
Signed-off-by: lou <alex1988@outlook.com >
* update after review
Signed-off-by: lou <alex1988@outlook.com >
* move to fs configure
Signed-off-by: lou <alex1988@outlook.com >
* remove flag
Signed-off-by: lou <alex1988@outlook.com >
* update after review
Signed-off-by: lou <alex1988@outlook.com >
* support worm hardlink
Signed-off-by: lou <alex1988@outlook.com >
* update after review
Signed-off-by: lou <alex1988@outlook.com >
* typo
Signed-off-by: lou <alex1988@outlook.com >
* sync filer conf
Signed-off-by: lou <alex1988@outlook.com >
---------
Signed-off-by: lou <alex1988@outlook.com >
2024-09-16 21:02:21 -07:00
zouyixiong
881c9a009e
[master] master missing start LoopPushingMetric routine fixed. ( #6018 )
2024-09-13 20:01:34 -07:00
chrislu
d660d5c7d4
increasing default cache size
2024-09-10 10:30:19 -07:00
chrislu
eb02946c97
support write once read many
...
fix https://github.com/seaweedfs/seaweedfs/issues/5954
2024-09-04 02:25:07 -07:00
chrislu
18afdb15b6
Revert "weed mount, weed dav add option to force cache"
...
This reverts commit 7367b976b0
2024-09-04 01:38:29 -07:00
chrislu
7367b976b0
weed mount, weed dav add option to force cache
2024-09-04 01:19:14 -07:00
chrislu
66ac82bb8f
default cacheDirWrite to cacheDir
2024-09-04 00:05:58 -07:00
Shaun Alexander
0d435f9da4
filer.remote.gateway not filer.remote.sync for filer.remote.gateway help string. ( #5935 )
2024-08-23 20:42:59 -07:00
chrislu
4463296811
add parallel vacuuming
2024-08-21 22:53:54 -07:00
chrislu
2074e7db67
fix invalid toml
2024-08-08 00:03:42 -07:00
qinguoyi
bf2366ba3a
Fix KeyMaterial nil bug ( #5849 )
2024-08-02 11:06:48 -07:00
qinguoyi
e88da0be1b
Fix KeyMaterial nil bug ( #5848 )
2024-08-02 09:47:11 -07:00
qinguoyi
606d0cba64
Fix extractToFile gz file nil bug ( #5847 )
2024-08-02 09:46:37 -07:00
qinguoyi
b74b506e52
add command backup destory volume error log ( #5846 )
2024-08-02 00:04:57 -07:00
chrislu
d49ecde535
rename functions
2024-08-01 23:27:23 -07:00
zuzuviewer
5ee50d26d8
fix CVE-2016-2183 ( #5844 )
...
* fix CVE-2016-2183
* allow tls.min_version and tls.max_version not set
2024-08-01 20:00:22 -07:00
Konstantin Lebedev
e1f9336688
[filer] fix panic filer on webdav.maxMB ( #5822 )
...
fix panic filer on webdav.maxMB
2024-07-27 13:40:22 -07:00
vadimartynov
86d92a42b4
Added tls for http clients ( #5766 )
...
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
chrislu
19d14b4c71
minor
2024-07-16 09:21:01 -07:00
vadimartynov
b796c21fa9
Added loadSecurityConfigOnce ( #5792 )
2024-07-16 09:15:55 -07:00
chrislu
3a82f5ffad
ensure metadata follow a specific folder
...
fix https://github.com/seaweedfs/seaweedfs/issues/5774
2024-07-12 11:17:30 -07:00
vadimartynov
ec9e7493b3
-metricsIp cmd flag ( #5773 )
...
* Added/Updated:
- Added metrics ip options for all servers;
- Fixed a bug with the selection of the binIp or ip parameter for the metrics handler;
* Fixed cmd flags
2024-07-12 10:56:26 -07:00
vadimartynov
de60f383de
Added tls for http handlers ( #5764 )
...
* Added https handler for filer
* Added example for security.toml
2024-07-11 07:53:18 -07:00
chrislu
8943a572d8
log error message
2024-07-03 09:50:47 -07:00
Konstantin Lebedev
5ffacbb6ea
refactor all methods strings to const ( #5726 )
2024-07-01 01:00:39 -07:00
chrislu
ac6fd36c06
using fixed bootstrap from peer filer
2024-06-30 23:16:54 -07:00
Nikita Borzykh
64756c2d79
fix: correct command flag for master.raftBootstrap option in server.go ( #5701 )
2024-06-22 19:28:37 -07:00
chrislu
a829f36d17
adjust error handling
...
fix https://github.com/seaweedfs/seaweedfs/issues/5677
2024-06-17 10:59:41 -07:00
vadimartynov
8aae82dd71
Added context for the MasterClient's methods to avoid endless loops ( #5628 )
...
* Added context for the MasterClient's methods to avoid endless loops
* Returned WithClient function. Added WithClientCustomGetMaster function
* Hid unused ctx arguments
* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions
* Changed the context termination check in the tryConnectToMaster function
* Added a child context to the tryConnectToMaster function
* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
2024-06-14 11:40:34 -07:00
MeenakshiSachdeva
d2516d99cc
Meenakshi fix filer sync ( #5672 )
...
* Fix Filer Sync Issue: 5455
* Fix Filer Sync Issue: 5455
2024-06-10 11:16:29 -07:00
MeenakshiSachdeva
1caa012a76
Fix Filer Sync Issue: 5455 ( #5663 )
2024-06-10 09:19:38 -07:00
sb
a3a00d9499
Feature/mongodb security ( #5602 )
2024-05-18 12:19:10 -07:00
sb
3fae87632f
add filer configuration options for username/password and tls ( #5601 )
2024-05-18 09:02:03 -07:00
Konstantin Lebedev
d389c5b27e
fix: recreate index include deleted files ( #5579 )
...
* fix: recreate index include deleted files
https://github.com/seaweedfs/seaweedfs/issues/5508
* fix: counting the number of files
* fix: log
2024-05-12 11:31:34 -07:00
Nikita Korolev
2dd2bb3e16
fix filer address parsing ( #5415 )
2024-03-24 11:20:33 -07:00
Nikita Borzykh
1f3742850d
Add TLS support, initial etcd connection check, fix key_prefix handling for etcd backend meta storage ( #5403 )
2024-03-24 08:36:21 -07:00
Konstantin Lebedev
d821cb3b18
fix: sync without dir /buckets/some/.uploads/hash_hash ( #5402 )
2024-03-20 12:54:29 -07:00
