lsm/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2025-08-01 20:47:01 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,642 Commits 47 Branches 311 Tags 202 MiB
7ad5a90127
Commit Graph

484 Commits

Author SHA1 Message Date
Chris Lu
afc1532b24 file rename 2018-06-24 23:19:56 -07:00
Chris Lu
233a9a9b6c fix log error 2018-06-24 16:12:41 -07:00
Chris Lu
49f4216280 fix compilation error 2018-06-24 15:37:54 -07:00
Chris Lu
7f6500ae07 super block reserves a couple of bits for future extension 2018-06-24 15:23:03 -07:00
Chris Lu
f339e9c284 add support for extra super block info 2018-06-24 15:19:57 -07:00
Chris Lu
9ba335a7c4 prepare for flexible super block 2018-06-24 11:37:08 -07:00
Chris Lu
c0122eeef3 start 0.91 beta 2018-06-24 09:29:34 -07:00
Chris Lu
cf76711789 releasing 0.90 version 2018-06-23 18:36:27 -07:00
Chris Lu
3a6f9c9bcf increase test size 2018-06-23 18:30:13 -07:00
Chris Lu
7f7e4e9885 fix error for deleted files during compaction 
deletion during commit may cause trouble when make up the difference during commitCompact()
 2018-06-23 18:24:59 -07:00
Chris Lu
3699996985 add compaction test 2018-06-23 16:48:19 -07:00
Chris Lu
14c6f12b0c adjust interface parameter name and doc 2018-06-17 13:24:57 -07:00
Chris Lu
db0c25b3d8 delete old entry only after successful insert 2018-06-17 13:02:22 -07:00
Chris Lu
eb2acd11c2 decouple from viper for filer store 2018-06-17 13:01:57 -07:00
Chris Lu
0467195f07 fix file closing on windows 2018-06-13 00:36:51 -07:00
Chris Lu
4433b4c006 treat 0 sized file as normal 
fix https://github.com/chrislusf/seaweedfs/issues/409
 2018-06-12 21:11:26 -07:00
Chris Lu
03f50180f3 simplifying the leader election by raft 
fixing https://github.com/chrislusf/seaweedfs/issues/629
 2018-06-12 01:54:09 -07:00
Chris Lu
69b4f93830 fix compilation for go tip 2018-06-12 00:58:43 -07:00
Chris Lu
17918f0b82 fix compilation error on go tip 2018-06-11 23:51:36 -07:00
Chris Lu
eaf000eff1 fix error on go tip 2018-06-11 23:39:42 -07:00
Chris Lu
d2a811eef2 fix compilation 2018-06-11 23:29:09 -07:00
Chris Lu
5bd72696ac weed mount add ttl option 2018-06-11 23:13:33 -07:00
Chris Lu
98110c1697 add collection, replication, ttl to each file 2018-06-10 16:57:32 -07:00
Chris Lu
f1273073fc switch to fs.NodeRequestLookuper 
in order to set the entry valid duration
 2018-06-07 00:07:37 -07:00
Chris Lu
cec1d97035 recursively move files and directories 2018-06-06 23:39:30 -07:00
Chris Lu
ae23e46313 better error log 2018-06-06 23:06:19 -07:00
Chris Lu
e755540be9 fix caching during directory listing 2018-06-06 22:55:59 -07:00
Chris Lu
daabdfe357 remove nodemap, fix directory listing cache 2018-06-06 22:48:51 -07:00
Chris Lu
5c25d29272 support renaming files 2018-06-06 22:11:01 -07:00
Chris Lu
b3447f4375 adjust logging 2018-06-06 02:21:36 -07:00
Chris Lu
6816661b0f fixed file handle by file full path 2018-06-06 02:09:57 -07:00
Chris Lu
299312c805 use separate filer grpc port 2018-06-05 23:37:41 -07:00
Chris Lu
95fe745a0c compatible with previous json rendering 2018-06-04 19:28:59 -07:00
Chris Lu
ebf58709d3 remove unused register file function 2018-06-01 23:24:34 -07:00
Chris Lu
43e3f5724c use fixed list of masters in both filer and volume servers 2018-06-01 00:39:39 -07:00
Chris Lu
a6f7f9b0b8 add missing modify and create times 2018-05-31 22:50:19 -07:00
Chris Lu
7efeb146c5 fix log 2018-05-31 22:49:55 -07:00
Chris Lu
4d1eedfa28 prepare to merge with master 2018-05-30 22:29:18 -07:00
Chris Lu
dc13e10637 fix copy error on 0 size files 2018-05-30 22:28:14 -07:00
Chris Lu
a218eaf1f0 fix log 2018-05-30 22:09:24 -07:00
Chris Lu
430eb67489 handle large file copy when write request is larger than buffer 2018-05-30 22:02:21 -07:00
Chris Lu
44acf4b756 fix file name 
f.Name() is not the base file name
 2018-05-30 20:48:13 -07:00
Chris Lu
0301504184 add mime, use simple insert and update filer store API 
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
 2018-05-30 20:24:57 -07:00
Chris Lu
26e7cd8c75 assign a different volume on large file copying 2018-05-30 01:05:26 -07:00
Chris Lu
2fe0d479f1 filer.copy supports chunking 2018-05-30 00:54:56 -07:00
Chris Lu
2d7dea1a09 a little refactoring 2018-05-29 23:52:27 -07:00
Chris Lu
8a48baa056 add single chunk file copying to new filer 2018-05-29 23:46:45 -07:00
Chris Lu
78aabf66ce add error details 2018-05-29 23:46:17 -07:00
Chris Lu
4e3ea49cff properly working local write buffer 2018-05-29 01:21:21 -07:00
Chris Lu
c4b92e17d0 fix isPerfectAppend 2018-05-28 22:45:52 -07:00
Chris Lu
74332e1a61 minor 2018-05-28 14:32:16 -07:00
Chris Lu
be0e88a606 fix chunk size limit for default 0 2018-05-28 13:44:27 -07:00
Chris Lu
5c4480ec6c add mountOptions.chunkSizeLimitMB, remove cmdMount.IsDebug 2018-05-28 13:42:25 -07:00
Chris Lu
8ab7dd9d08 weed mount add options for collection and replication 2018-05-28 13:24:48 -07:00
Chris Lu
d0b238d2db cache local writes before flushing to volume server 2018-05-28 12:30:17 -07:00
Chris Lu
07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu
dd5661b297 adding favicon to filer also 2018-05-28 02:35:58 -07:00
Chris Lu
9b603f5ffa split filer read into 2 files 2018-05-28 02:24:14 -07:00
Chris Lu
8db9319a06 refactoring: go fmt, reorg 2018-05-27 23:59:49 -07:00
Chris Lu
480a073f1f refactoring: split into 4 files 2018-05-27 23:53:10 -07:00
Chris Lu
5b844d7011 pass reference of master server 
SendHeartbeat passes lock by value: weed_server.MasterServer contains
sync.Mutex
 2018-05-27 11:58:00 -07:00
Chris Lu
8b0718ac92 go vet 2018-05-27 11:56:49 -07:00
Chris Lu
458ada173e go fmt 2018-05-27 11:52:26 -07:00
Chris Lu
8647191bee removing filer package! 2018-05-27 11:52:15 -07:00
Chris Lu
03d76479a5 add reds 2018-05-27 11:14:29 -07:00
Chris Lu
a6a6b87f1b adjust readme 2018-05-27 11:14:12 -07:00
Chris Lu
ab86d263c9 add Cassandra 2018-05-27 00:01:15 -07:00
Chris Lu
603de2d5db skip deletion if entry not found 2018-05-27 00:00:56 -07:00
Chris Lu
f124ebab1d fix ordering 2018-05-27 00:00:11 -07:00
Chris Lu
1c0a2a7db8 reformat 2018-05-26 23:59:56 -07:00
Chris Lu
869161a261 support both mysql and postgres 2018-05-26 22:02:49 -07:00
Chris Lu
87b3b84471 simplifying dirhash to 64bit integer 2018-05-26 21:24:03 -07:00
Chris Lu
955eae3500 this works great and support long file names 2018-05-26 14:08:55 -07:00
Chris Lu
2da84ed331 working fine now 
but index length can be improved
 2018-05-26 13:35:56 -07:00
Chris Lu
68bcaff14d mysql can compile, not yet tested! 2018-05-26 05:32:15 -07:00
Chris Lu
c1e353491a refactoring variable names 2018-05-26 04:50:55 -07:00
Chris Lu
828e4a5ace fix test error 2018-05-26 04:13:50 -07:00
Chris Lu
9e77563c99 add leveldb store 
1. switch to viper for filer store configuration
2. simplify FindEntry() return values, removing “found”
3. add leveldb store
 2018-05-26 03:49:46 -07:00
Chris Lu
c34feca59c refactoring 2018-05-25 23:27:06 -07:00
Chris Lu
6de84c64c6 adding create time 2018-05-25 23:26:40 -07:00
Chris Lu
72fc001d0d remove unused code 2018-05-25 01:30:16 -07:00
Chris Lu
ac66c133a5 do not read attributes when file is opened 2018-05-25 01:27:21 -07:00
Chris Lu
6d1bcd4b8c use existing attributes instead of fetching from filer 2018-05-25 01:22:31 -07:00
Chris Lu
0a223838bd refactoring 2018-05-25 00:57:25 -07:00
Chris Lu
f8776ad5cd add breadcrumb to filer UI 2018-05-24 23:42:13 -07:00
Chris Lu
a01557f4cb adjust filer web page 2018-05-24 23:21:12 -07:00
Chris Lu
c11d84f314 fix reading from a url 2018-05-24 23:20:56 -07:00
Chris Lu
7b81cf3762 better logs 2018-05-24 23:20:26 -07:00
Chris Lu
9f4f8de9ad skip printout 2018-05-24 23:20:12 -07:00
Chris Lu
e18c7e160c avoid empty chunk view 2018-05-24 23:19:56 -07:00
Chris Lu
d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu
00d0274fd7 prepare to read from multiple file chunks 2018-05-23 22:28:54 -07:00
Chris Lu
849b6ec28d seems editing already working 
Need to handle multiple chunks read.
Need to cache local file changes.
 2018-05-23 20:55:24 -07:00
Chris Lu
d84b80b795 similar changes as master 2018-05-23 03:23:47 -07:00
Chris Lu
536559f62d copy works, edit somehow still fails 2018-05-23 03:08:46 -07:00
Chris Lu
1675243f29 maybe speed up a little when loading index 2018-05-23 02:38:11 -07:00
Chris Lu
e97c60cc10 avoid overwriting variables 2018-05-23 02:36:19 -07:00
Chris Lu
468514f525 reduce logs 2018-05-23 02:36:06 -07:00
Chris Lu
69b9d8c3c2 fix boltdb variable usage 2018-05-22 10:18:09 -07:00
Chris Lu
873868cc10 not working now 
need to add file handler
 2018-05-22 04:31:44 -07:00
Chris Lu
7362de9a18 weed mount can work well 
TODO: somehow filer url is returning empty content
 2018-05-22 03:26:38 -07:00
Chris Lu
9dd228747c filer copy added uid/gid 2018-05-21 01:25:30 -07:00
Chris Lu
f07482382b able to update file content 
having some issue when vi reports file changed.
 2018-05-21 00:00:28 -07:00
Chris Lu
cd47528a75 remove unused code 2018-05-20 17:08:54 -07:00
Chris Lu
cc66e25cd2 merge intervals 2018-05-20 17:06:09 -07:00
Chris Lu
7ca5052942 create files correctly! 2018-05-19 13:51:44 -07:00
Chris Lu
793dd81ca2 skip permission checking when creating dir or files 2018-05-19 12:40:24 -07:00
Chris Lu
ab4ddb1e0e fix directory creation, directory listing 2018-05-19 12:07:15 -07:00
Chris Lu
e31c514b00 adding modified time to file chunk 2018-05-16 00:54:44 -07:00
Chris Lu
6bf31467c7 adding empty fsync 
less error now. But still:

vi on write: E514: write error (file system full?)
cp: Input/output error
 2018-05-16 00:54:27 -07:00
Chris Lu
b303a02461 cp file can work 
1. consolidate to filer_pb.FileChunk
2. dir add file, mkdir
3. file flush, write

updates having issue
 2018-05-16 00:08:44 -07:00
Chris Lu
c7a71d35b0 fix on pagination 2018-05-14 20:27:48 -07:00
Chris Lu
58954bf46f pass file attributes from filer to mount 2018-05-14 02:02:17 -07:00
Chris Lu
67401f1908 fix gRpc nil response 2018-05-14 00:17:22 -07:00
Chris Lu
c5cf9bd290 properly working filer 2018-05-13 23:56:16 -07:00
Chris Lu
f01d5616b3 add better listing directory entries 2018-05-13 14:02:29 -07:00
Chris Lu
a4740ca836 fix compilation error for windows 2018-05-13 00:49:59 -07:00
Chris Lu
2dd5b777f7 prepare for 0.77 release 
right before changing to filer2
 2018-05-13 00:26:23 -07:00
Chris Lu
75a2b05aa3 remove Size from Attr 2018-05-13 00:11:26 -07:00
Chris Lu
fc7f81f6e7 remove nlink property from directory node 2018-05-12 22:42:28 -07:00
Chris Lu
9ecbf92569 more tests 2018-05-12 13:50:11 -07:00
Chris Lu
81af1bafba file store adds memdb implementation 2018-05-12 13:45:29 -07:00
Chris Lu
a808df5019 visit parent folder first 2018-05-11 02:27:57 -07:00
Chris Lu
8a1d640dc4 starting filer2 2018-05-11 02:20:15 -07:00
Chris Lu
9f345da20f mv filer proto to filer_pb 2018-05-09 23:18:02 -07:00
Chris Lu
942c2cbd7b mv pb to master_pb 2018-05-09 23:11:54 -07:00
Chris Lu
57c5405657 add todo 2018-05-09 04:18:31 -07:00
Chris Lu
6db1c8bafb Revert "adjust the needle mapper" 
This reverts commit e2c7be3a2e.
 2018-05-09 03:57:07 -07:00
Chris Lu
e2c7be3a2e adjust the needle mapper 
as discussed on
https://github.com/chrislusf/seaweedfs/pull/642#issuecomment-387606960
 2018-05-09 03:50:14 -07:00
Chris Lu
43a69d20bf change filer API to gRPC 2018-05-08 01:59:43 -07:00
Chris Lu
4936d6c342 adjusting for FUSE 2018-05-07 00:37:47 -07:00
Chris Lu
cb6b1d8d41 fix broken master health checking 2018-05-07 00:37:33 -07:00
Chris Lu
be58993f47 ensure the same Node is returned 2018-05-05 23:50:34 -07:00
Chris Lu
b857cf9d9b format 2018-05-05 23:39:29 -07:00
Chris Lu
456738ba64 refactoring fuse 2018-05-05 22:47:16 -07:00
Chris Lu
2cada598c6 fix test 2018-05-05 15:08:23 -07:00
Chris Lu
f3e07c231b adding folder icon 2018-05-05 14:41:40 -07:00
Chris Lu
7f49514f6f fix filer UI 2018-05-05 14:10:50 -07:00
Chris Lu
fffb14bc87 better support FUSE Lookup() 2018-05-05 02:01:50 -07:00
Chris Lu
050ab19264 add design 2018-05-04 23:38:42 -07:00
Chris Lu
94a35f25f3 simple refactoring 2018-05-03 00:16:02 -07:00
WenLong LI
b9068c1304
use volumes variable 
fix leveldb mount bug
 2018-05-03 12:13:53 +08:00
liwenlong05
02a1597642 fix leveldb mount bug 2018-05-03 11:20:38 +08:00
mtolman
e4b1e5c3c8
Fix for issue #635 
Fix for the following issue on master: https://github.com/chrislusf/seaweedfs/issues/635
 2018-04-17 13:26:48 -06:00
Chris Lu
d49e91cd78 cassandra use local quorum for better performance 2018-04-16 01:33:05 -07:00
Chris Lu
15b8499f8d refactoring 2018-04-16 01:16:06 -07:00
Chris Lu
d712c87ec0 update glide 2018-04-16 00:59:10 -07:00
Chris Lu
6448e0c25d update glide 2018-04-16 00:38:49 -07:00
Chris Lu
8c64d06b3a update redis library 2018-04-16 00:27:35 -07:00
Chris Lu
9cd6b33e97 fix https://github.com/chrislusf/seaweedfs/issues/603 2018-02-27 14:18:00 -08:00
Chris Lu
4b9b9547d4 unused option 2018-02-27 13:57:15 -08:00
Chris Lu
fc3924ac70 set file name to base file name 
fixing https://github.com/chrislusf/seaweedfs/issues/632
 2018-02-26 08:01:23 -08:00
tengattack
371ad2e397 fix reserve volume on node 2018-01-17 19:53:41 +08:00
Chris Lu
61ff330e99 adjust instructions 2018-01-15 20:33:12 -08:00
Chris Lu
1d09291f95 update upstream 2018-01-15 20:32:57 -08:00
Chris Lu
70f6740309 better fix for single master restart without peers changing 2017-12-06 00:14:14 -08:00
Chris Lu
9026b3e86e always remember the max volume id 2017-11-28 17:08:59 -08:00
Chris Lu
97f757c669 add debug message 
display the error message for debugging
https://github.com/chrislusf/seaweedfs/issues/603
 2017-11-25 01:31:13 -08:00
Chris Lu
ef6224e45d filer default to auto chunking file size to 32MB 2017-09-15 08:24:30 -07:00
Yang Pengcheng
a2112e1be0 fix ldb dir deleting 2017-08-30 22:55:03 +08:00
Chris Lu
3288d729ae fix compilation error 2017-08-30 02:24:19 -07:00
Chris Lu
58344980e4 preallocate disk space during compaction also, add cleanup for failed compaction 2017-08-29 23:59:53 -07:00
Chris Lu
f7c22f0159 fix bug when collecting vacuum errors 2017-08-29 23:11:20 -07:00
chanyoung
52fc476a6f filer: fixup print error message when make new directory 2017-08-15 15:54:12 +09:00
Chris Lu
dbee816205 fix typo 
fix https://github.com/chrislusf/seaweedfs/issues/549
 2017-08-10 10:26:39 -07:00
Chris Lu
3a8746ca54 better error message 2017-08-10 10:26:39 -07:00
kelgon
3bf883327e (fix #543)added body to error when Post encounter 4xx response 2017-07-28 16:11:23 +08:00
Chris Lu
e0dfa7a069 ensure 30GB or less 
avoid https://github.com/chrislusf/seaweedfs/issues/535
 2017-07-16 21:40:55 -07:00
Xinran Xu
4a0939a4e4 Fix panic error when target collection is not exist. 
Fix: interface conversion: interface {} is nil, not *topology.Collection.
 2017-07-14 20:04:33 +08:00
Chris Lu
f9a1769df1 fix nil raft server caused crash 
fix https://github.com/chrislusf/seaweedfs/issues/524
 2017-07-03 22:35:28 -07:00
Chris Lu
1fbb8723dc add cpu and mem pprof 2017-06-22 01:33:58 -07:00
Chris Lu
2c7dad589d stop when dn.parent is nil 
fix https://github.com/chrislusf/seaweedfs/issues/473
 2017-06-16 08:27:50 -07:00
Chris Lu
72e89b615b 301 is reported as 404 for http post 
fix https://github.com/chrislusf/seaweedfs/issues/512
 2017-06-15 21:21:32 -07:00
Chris Lu
5047bdb4a2 skip bytes cache 2017-06-03 11:44:24 -07:00
Chris Lu
f5bed84340 add volume id to error message 2017-06-03 01:58:28 -07:00
Chris Lu
83efe1547d revert webp support due to "go test" 
related to https://github.com/chai2010/webp/issues/9
 2017-05-28 10:25:40 -07:00
Chris Lu
478fe0ecf2 filer add readonly public port 2017-05-27 20:14:22 -07:00
Chris Lu
7ecc0f4b11 rename variables 2017-05-27 18:11:18 -07:00
Chris Lu
021ef0d14b adjust version to 0.76 2017-05-26 23:04:38 -07:00
Chris Lu
e3fc676260 add doc for 0.75 2017-05-26 23:01:40 -07:00
Chris Lu
82c3ccc8dd add btree for volume index 2017-05-26 22:51:25 -07:00
Chris Lu
bd97cbc523 skip busy reporting back to master 
possible fix for https://github.com/chrislusf/seaweedfs/issues/499
 2017-05-23 21:12:03 -07:00
Chris Lu
a7faefd4ce fix compilation error 2017-05-22 23:04:43 -07:00
Chris Lu
b3513d3167 skip only readonly volumes 
fix https://github.com/chrislusf/seaweedfs/issues/405
 2017-05-22 17:05:27 -07:00
Chris Lu
4ce6586710 use local random generator to avoid global lock 2017-05-19 23:33:36 -07:00
rmn
366fe0d394 Scale images to fit or fill 2017-05-05 12:17:30 +03:00
Vitaly Baev
7ab9160ee1 WebP resize support 2017-03-31 09:56:00 +03:00
Chris Lu
a337b844ec properly send heartbeats 
fix https://github.com/chrislusf/seaweedfs/issues/476
 2017-03-27 10:11:54 -07:00
sparklxb
c46e91d229 complement weed mount: add read and delete 2017-03-17 15:02:27 +08:00
Chris Lu
37b79174d1 Merge pull request #469 from sparklxb/master 
complement weed mount
 2017-03-14 08:21:29 -07:00
Stepan Bujnak
14ec3bfc1d Show contents of manifest file when cm=false is passed 2017-03-12 18:40:24 -07:00
Stepan Bujnak
a72c92e55f Added locking on the filer move operation 2017-03-11 02:52:04 -08:00
sparklxb
3a67b6ddc1 complement weed mount 2017-03-10 14:46:00 +08:00
sparklxb
7f85e15990 complement weed mount 2017-03-10 10:28:19 +08:00
sparklxb
813a4f7456 Fixes #266 2017-03-06 13:04:52 +08:00
sparklxb
4049c72e6c fix bug: replication, metadata not replicated 2017-03-04 11:24:12 +08:00
Chris Lu
30f138a538 fix tests 2017-02-12 22:18:48 -08:00
Chris Lu
41f686ab9d remove weeds.conf 2017-02-12 21:58:44 -08:00
Chris Lu
98d66a569b avoid nil 
fix https://github.com/chrislusf/seaweedfs/issues/452
 2017-02-11 10:21:28 -08:00
vancepym
ee6067e98c Fix: the maximum-length character exceed 255 
The length of a CHAR column is fixed to the length that you declare when you create the table. The length can be any value from 0 to 255. When CHAR values are stored, they are right-padded with spaces to the specified length. 

see https://dev.mysql.com/doc/refman/5.7/en/char.html
 2017-01-23 20:33:00 +08:00