lsm/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2025-09-23 01:15:35 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
10,958 Commits 48 Branches 313 Tags
4af21b0dfced547b540a6c20b03151cbc4faa6b5
Commit Graph

533 Commits

Author SHA1 Message Date
chrislu
43d9f68219 more consistent server header for s3 apis 2024-09-09 09:07:58 -07:00
Konstantin Lebedev
16984ef07a [s3] reducing the number of metrics for 403 (#5961) 
reducing the number of metrics
 2024-09-04 11:06:54 -07:00
chrislu
6ce27affa4 also use /healthz for most consistent health check 2024-08-21 10:55:39 -07:00
Oleg Salionov
4cfc7d3b10 Dont try lazy decode content in proxyToFiler if no accept-encoding provided (#5907) 2024-08-18 21:51:17 -07:00
Kamran Sarwar
63c707f9c1 added access-control-expose-headers to put request (#5876) 2024-08-09 15:52:05 -07:00
chrislu
aed2a18223 get, set and add path conf 2024-07-20 11:27:11 -07:00
vadimartynov
86d92a42b4 Added tls for http clients (#5766) 
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
 2024-07-16 23:14:09 -07:00
Konstantin Lebedev
9c440d472c [s3] revert skip deletion error, since the error file was not found is already skipped on the side of the grpc function (#5760) 
* revert skip deletion error, since the error file was not found is already skipped on the side of the grpc function

* fix response error

* fix test_lifecycle_get

* Revert "fix test_lifecycle_get"

This reverts commit 8f991bdcf9.
 2024-07-10 08:47:43 -07:00
chrislu
b8121e4b24 simplify 2024-07-08 11:10:41 -07:00
chrislu
f618121e63 Fix S3 deletion in deep folders, and names with empty spaces 
fix https://github.com/seaweedfs/seaweedfs/issues/5748
 2024-07-08 10:20:09 -07:00
Konstantin Lebedev
f77eee667d add s3test for sql (#5718) 
* add s3test for sql

* fix test test_bucket_listv2_delimiter_basic for s3

* fix action s3tests

* regen s3 api xsd

* rm minor s3 test test_bucket_listv2_fetchowner_defaultempty

* add docs

* without xmlns
 2024-07-04 11:00:41 -07:00
Konstantin Lebedev
5ffacbb6ea refactor all methods strings to const (#5726) 2024-07-01 01:00:39 -07:00
Lê Hoàng Nam
1f2dc54647 fix(s3api): make post api upload object default return 204 (#5648) 2024-06-04 06:18:47 -07:00
Riccardo Bertossa
f6e8a9bf9c added s3 iam DeleteBucket permission management (#5599) 2024-05-17 04:54:09 -07:00
chrislu
1917f70f78 also delete parent folder if empty 
fix https://github.com/seaweedfs/seaweedfs/issues/5567
 2024-05-07 08:46:19 -07:00
chrislu
e6bfd061c7 AllowEmptyFolder checks during object deletion 2024-04-29 06:25:06 -07:00
chrislu
d521466a37 split file 2024-04-29 06:23:42 -07:00
chrislu
a1b59948cc rename files 2024-04-29 05:33:56 -07:00
Konstantin Lebedev
3d3ee04eb9 [s3] Put bucket lifecycle configuration (#5510) 2024-04-27 07:39:22 -07:00
Konstantin Lebedev
33537ae29f [s3] fix s3 test_multipart_get_part (#5476) 
* try fix s3  test_multipart_get_part

* add passed s3 tests

* fix SeaweedFSUploadId

* rm spaces

* convert part request to range

* add passed s3 tests of multipart
 2024-04-14 10:41:32 -07:00
clonefetch
9e07a87fcb chore: fix function names in comment (#5478) 2024-04-08 07:19:02 -07:00
Konstantin Lebedev
3e25ed1b11 [s3] add s3 pass test_multipart_upload_size_too_small (#5475) 
* add s3 pass test_multipart_upload_size_too_small

* refactor metric names

* return ErrNoSuchUpload if empty parts

* fix test
 2024-04-07 11:52:35 -07:00
Konstantin Lebedev
35cba720a5 [s3] add s3 pass test_multipart_upload (#5474) 
add s3 pass test_multipart_upload
 2024-04-07 11:51:22 -07:00
Konstantin Lebedev
d42a04cceb [s3] fix s3 test_multipart_resend_first_finishes_last (#5471) 
* try fix s3 test
https://github.com/seaweedfs/seaweedfs/pull/5466

* add error handler metrics

* refactor

* refactor multipartExt

* delete bad entry parts
 2024-04-06 10:56:39 -07:00
Nikita Borzykh
7aa25c1137 fix: correctly form partNumber from new entry.Name format in "listObjectParts" (#5470) 2024-04-04 09:16:16 -07:00
skycope
b19c9847c6 fix completed multiupload lost data (#5460) 
If there are putObjectPart requests with the same uploadId during
completeMultiPart, it can result in data loss. putObjectPart requests
might be due to timeout retries.

Co-authored-by: Yang Wang <yangwang@weride.ai>
 2024-04-02 20:18:42 -07:00
Nikita Borzykh
df9031e19a fix: ensure complete gRPC stream consumption in doListFilerEntries to prevent context canceled errors. (#5441) 2024-04-01 15:55:16 -07:00
adasauce
61f4e40ad9 fix s3api: delimeter properly takes prefixes into account (#5411) 2024-03-22 07:03:34 -07:00
Konstantin Lebedev
170b63d6f8 [filer.backup] add param uploader_part_size for S3sink (#5352) 
* fix: install cronie

* chore: refactor configure S3Sink

* chore: refactor cinfig

* add filer-backup compose file

* fix: X-Amz-Meta-Mtime and resolve with comments

* fix: attr mtime

* fix: MaxUploadPartst is reduced to the maximum allowable

* fix: env and force set max MaxUploadParts

* fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
 2024-03-07 08:35:51 -08:00
adasauce
3e3e0fa240 feat: add support for S3 ListObjects and ListObjectsV2 (#5350) 2024-03-06 07:24:44 -08:00
7y-9
41ec9d7759 refactor: put the auth outside (#5313) 2024-02-19 03:07:48 -08:00
7y-9
f037c09c11 fix: only admin auth can delete S3 bucket (#5312) 2024-02-19 00:38:05 -08:00
sxlehua
ffe5f19aa0 Add s3 postpolicy support header (#5280) 
* Add s3 postpolicy support header

* optimized code
 2024-02-05 07:55:07 -08:00
sxlehua
0b49c16361 Adapt S3 POST ContentType (#5275) 2024-02-03 22:33:34 -08:00
Konstantin Lebedev
a7fc723ae0 chore: add status code for request_total metrics (#5188) 2024-01-10 10:05:27 -08:00
Konstantin Lebedev
4f6172f369 Fix missing VersionConfiguration node in get-bucket-versioning response (#5162) 
* fix: s3 response for get bucket version
https://github.com/seaweedfs/seaweedfs/issues/5155

* fix: s3 response for get bucket version
https://github.com/seaweedfs/seaweedfs/issues/5155
 2024-01-03 12:27:13 -08:00
chrislu
ee1c9bc314 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-12-22 11:33:02 -08:00
chrislu
7a0d1e7fc1 minor 2023-12-22 11:29:54 -08:00
jerebear12
06343f8976 Set allowed origins in config (#5109) 
* Add a way to use a JWT in an HTTP only cookie

If a JWT is not included in the Authorization header or a query string, attempt to get a JWT from an HTTP only cookie.

* Added a way to specify allowed origins header from config

* Removed unecessary log

* Check list of domains from config or command flag

* Handle default wildcard and change name of config value to cors
 2023-12-20 16:21:11 -08:00
Konstantin Lebedev
5e41ab1370 s3 api add not implemented response for PutBucketVersioning 2023-11-13 06:22:23 -08:00
Konstantin Lebedev
7f6152f130 s3 api add default response for GetBucketVersioning 2023-11-13 06:22:23 -08:00
chrislu
612fd7206e fix 2023-10-19 08:10:33 -07:00
Konstantin Lebedev
6c82d7ddfc [s3] do reload s3 static config (#4923) 
* do reload s3 config

* print error on reload s3 config

* print success msg

* Update weed/s3api/s3api_server.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-10-18 23:26:49 -07:00
chrislu
cbc24c7b24 disconnect from old subscribers 2023-10-13 16:48:14 -07:00
chrislu
7b3c63a8af handle wrapped errors 2023-10-05 08:15:11 -07:00
chrislu
4e493d596c minor 2023-10-05 08:13:53 -07:00
chrislu
b02fdeabff RetryForever => RetryUntil 2023-10-01 11:33:56 -07:00
chrislu
645ae8c57b Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"" 
This reverts commit 8cb42c39
 2023-09-25 09:35:16 -07:00
Konstantin Lebedev
f8b94cac0e [s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface

https://github.com/seaweedfs/seaweedfs/issues/4519

* fix: test bucket acl default and
adjust the variable names

* fix: s3 api config test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-25 08:34:12 -07:00
Konstantin Lebedev
750a0ba1b2 [iam] Replace action read/write to readAcp/writeAcp for handlers with acl (#4858) 
Replace action read/write to readAcp/writeAcp for handlers with acl query
 https://github.com/seaweedfs/seaweedfs/issues/4519

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-21 14:07:04 -07:00