26 Commits

Author SHA1 Message Date
Chris Lu
12f50d37fa test versioning also (#7000)
* test versioning also

* fix some versioning tests

* fall back

* fixes

Never-versioned buckets: No VersionId headers, no Status field
Pre-versioning objects: Regular files, VersionId="null", included in all operations
Post-versioning objects: Stored in .versions directories with real version IDs
Suspended versioning: Proper status handling and null version IDs

* fixes

Bucket Versioning Status Compliance
Fixed: New buckets now return no Status field (AWS S3 compliant)
Before: Always returned "Suspended" 
After: Returns empty VersioningConfiguration for unconfigured buckets 
2. Multi-Object Delete Versioning Support
Fixed: DeleteMultipleObjectsHandler now fully versioning-aware
Before: Always deleted physical files, breaking versioning 
After: Creates delete markers or deletes specific versions properly 
Added: DeleteMarker field in response structure for AWS compatibility
3. Copy Operations Versioning Support
Fixed: CopyObjectHandler and CopyObjectPartHandler now versioning-aware
Before: Only copied regular files, couldn't handle versioned sources 
After: Parses version IDs from copy source, creates versions in destination 
Added: pathToBucketObjectAndVersion() function for version ID parsing
4. Pre-versioning Object Handling
Fixed: getLatestObjectVersion() now has proper fallback logic
Before: Failed when .versions directory didn't exist 
After: Falls back to regular objects for pre-versioning scenarios 
5. Enhanced Object Version Listings
Fixed: listObjectVersions() includes both versioned AND pre-versioning objects
Before: Only showed .versions directories, ignored pre-versioning objects 
After: Shows complete version history with VersionId="null" for pre-versioning 
6. Null Version ID Handling
Fixed: getSpecificObjectVersion() properly handles versionId="null"
Before: Couldn't retrieve pre-versioning objects by version ID 
After: Returns regular object files for "null" version requests 
7. Version ID Response Headers
Fixed: PUT operations only return x-amz-version-id when appropriate
Before: Returned version IDs for non-versioned buckets 
After: Only returns version IDs for explicitly configured versioning 

* more fixes

* fix copying with versioning, multipart upload

* more fixes

* reduce volume size for easier dev test

* fix

* fix version id

* fix versioning

* Update filer_multipart.go

* fix multipart versioned upload

* more fixes

* more fixes

* fix versioning on suspended

* fixes

* fixing test_versioning_obj_suspended_copy

* Update s3api_object_versioning.go

* fix versions

* skipping test_versioning_obj_suspend_versions

* > If the versioning state has never been set on a bucket, it has no versioning state; a GetBucketVersioning request does not return a versioning state value.

* fix tests, avoid duplicated bucket creation, skip tests

* only run s3tests_boto3/functional/test_s3.py

* fix checking filer_pb.ErrNotFound

* Update weed/s3api/s3api_object_versioning.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers_copy.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/s3/versioning/s3_versioning_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-19 21:43:34 -07:00
Chris Lu
02773a6107 Accumulated changes for message queue (#6600)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
docker: build dev containers / build-dev-containers (push) Waiting to run
End to End / FUSE Mount (push) Waiting to run
go: build binary / Build (push) Waiting to run
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
* rename

* set agent address

* refactor

* add agent sub

* pub messages

* grpc new client

* can publish records via agent

* send init message with session id

* fmt

* check cancelled request while waiting

* use sessionId

* handle possible nil stream

* subscriber process messages

* separate debug port

* use atomic int64

* less logs

* minor

* skip io.EOF

* rename

* remove unused

* use saved offsets

* do not reuse session, since always session id is new after restart

remove last active ts from SessionEntry

* simplify printing

* purge unused

* just proxy the subscription, skipping the session step

* adjust offset types

* subscribe offset type and possible value

* start after the known tsns

* avoid wrongly set startPosition

* move

* remove

* refactor

* typo

* fix

* fix changed path
2025-03-09 23:49:42 -07:00
chrislu
3a2e21fee7 fix EC reading on nLargeBlockRows
fix https://github.com/seaweedfs/seaweedfs/issues/5465
2024-08-13 12:01:46 -07:00
chrislu
8b2c39f2c0 reduce binary size 2023-09-21 13:22:08 -07:00
chrislu
4d6c18d86f pub sub initial tests 2023-08-27 17:50:59 -07:00
aronneagu
2b783738d6 Update Makefile - use binary in current directory (#4027) 2022-12-01 14:09:14 -08:00
chrislu
02b38f894b debug 2022-08-07 00:55:49 -07:00
chrislu
dc0f48682c unused 2022-03-03 03:41:35 -08:00
chrislu
f87da798a4 to be re-written following fuse virtual file system 2022-02-11 03:09:30 -08:00
chrislu
9b954dc0d4 adjust make file 2022-01-13 01:33:13 -08:00
Chris Lu
c77c0de418 make install 2021-11-09 12:34:57 -08:00
Chris Lu
cc278a5928 Update Makefile 2021-09-04 13:46:49 -07:00
Chris Lu
40dc283b2d fix locating data chunks 2021-08-15 23:07:58 -07:00
Chris Lu
5571f4f70a master: add master.follower to handle read file id lookup requests 2021-08-12 18:10:59 -07:00
Chris Lu
13e45e1605 filer.remote.sync can work now 2021-08-08 01:21:42 -07:00
Chris Lu
30dbe98dff Update Makefile 2021-07-01 01:19:31 -07:00
Chris Lu
a4bb37a5fe add debug info 2021-05-01 16:10:26 -07:00
Chris Lu
e52c94640e filer.backup: added to replace filer.replicate 2021-02-28 16:22:27 -08:00
Chris Lu
41c0f3ad24 filer: support path-specific filer store 2020-12-19 01:27:09 -08:00
Chris Lu
b52ae9cef8 add s3 debug 2020-12-08 22:27:04 -08:00
Chris Lu
9d80a3428c add debug_webdav 2020-10-16 12:53:56 -07:00
Chris Lu
b18f21cce1 mount: fix bound tree with filer.path
fix https://github.com/chrislusf/seaweedfs/issues/1528
2020-10-12 21:58:37 -07:00
Chris Lu
5d6753fb98 shell: add volumeServer.leave command 2020-09-13 21:25:51 -07:00
Chris Lu
c0ee78d2fa adjust make file 2020-09-12 04:05:42 -07:00
Chris Lu
c45ba5d7d4 fix listObjectsV2 response format
fix https://github.com/chrislusf/seaweedfs/issues/1426 issue 1
2020-08-19 12:08:04 -07:00
Chris Lu
51ecb49db3 for debugging 2020-08-18 17:35:39 -07:00