Notes on Using SeaweedFS

You should download the large_disk version.

Run ./weed version, if you see “Seaweed 8000GB”, then you are using the large disk version. If you see “Seaweed 30GB”, you are using the regular version.

For simplicity all commands are run under the root user.

Start a master node on Aloha

nohup ./weed master -mdir=”weed-idx” -ip=172.25.17.14 > master.out 2>&1 &

Check the log, you should see “Start Seaweed Master 8000GB” which again verifies you are using the large disk version.

Go to http://172.25.17.14:9333/, this webpage should be up.

Start a volume node on la

The volume in SeaweedFS terminology doesn’t refer to the volume as in hard disk. The SeaweedFS volume is just a container of files and directories.

nohup ./weed volume -max=0 -mserver=”aloha:9333″ -dir=”/data/brick1/sw” > volume.out 2>&1 &

$ df -h /data/brick1
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3       1.8T   77M  1.7T   1% /data/brick1

Refresh http://172.25.17.14:9333/, Max under Cluster status becomes 59 from 0. 59*30=1770 GB, which matches the size of /dev/sda3.

The Topology section will show 172.25.17.12:8080. Click the link and you will see the actual file distribution on that volume node.

Volume 1, 2, and 4 has files, while the others are empty for now. (If this is the first time you ever use Seaweedfs, all volumes should be empty for you.)

Test

Up to now, the SeaweedFS storage is ready to use. Let’s create a file and test it.

echo ‘hello seaweedfs’ > README.md

$ curl http://aloha:9333/dir/assign
{"fid":"6,081bb85359","url":"172.25.17.12:8080","publicUrl":"172.25.17.12:8080","count":1}
$ curl -F file=@README.md http://172.25.17.12:8080/6,081bb85359
{"name":"README.md","size":16,"eTag":"f8debcaf"}

Go to http://172.25.17.12:8080/ui/index.html, Volume 6 should contain 1 file (if it was 0).

You may argue this way to upload file is inconvenient, and I agree. Next section starts the filer service and provides a POSIX (FUSE) access point.

Start a filer node on aloha

Reference: https://github.com/chrislusf/seaweedfs/wiki/Directories-and-Files

nohup ./weed filer -master=172.25.17.14:9333 > filer.out 2>&1 &

8888 appeared below is the default communication port of filer.

Test

$ cp README.md README2.md
$ curl -F "filename=@README2.md" "http://aloha:8888/path/to/sources/"
{"name":"README2.md","size":16}
$ curl "http://localhost:8888/path/to/sources/README2.md"
hello seaweedfs

We can see the benefit of running the filer service is that we are able to access files using paths instead of the esoteric ids like 6,081bb85359.

The next step is to mount the SeaweedFS storage to our file system.

Run FUSE Mount on aloha

First check the mount point. I’m going to mount SeaweedFS storage to /home/shared-la.

$ ll /home/shared-la
ls: cannot access '/home/shared-la': Transport endpoint is not connected
$ df -h
df: /home/shared-la: Transport endpoint is not connected

It turns out my mount point is not “clean”, so I will run sudo umount /home/shared-la.

ll /home/shared-la
total 0

nohup ./weed mount -filer=172.25.17.14:8888 -dir=/home/shared-la -filer.path=/ > mount.out 2>&1 &

Test

ll -R /home/shared-la
/home/shared-la:
total 0
drwxrwx--x 1 root root 0 2022-05-27 11:16 path

/home/shared-la/path:
total 0
drwxrwx--x 1 root root 0 2022-05-27 11:16 to

/home/shared-la/path/to:
total 0
drwxrwx--x 1 root root 0 2022-05-27 11:16 sources

/home/shared-la/path/to/sources:
total 1.5K
-rw-rw---- 1 root root 839 2022-05-27 11:16 nfs.out
-rw-rw---- 1 root root  16 2022-05-27 13:39 README2.md

The README2.md uploaded through filer service shows up here, but the README.md uploaded through the native API doesn’t.

$ cat README2.md
cat: README2.md: Input/output error
$ sudo chown $(whoami): README2.md
$ cat README2.md
hello seaweedfs

Files uploaded through the filer API has owner and groups as root. The permissions and groups are enforced by FUSE. You can chown.

df -h /home/shared-la reports the total storage size is around 210GB. Figure says there are only 7 volumes on the volume node and each each volume holds up to 30GB despite of the large disk version, hence the total size 210GB. After you filled up the 210GB, Seaweedfs will generate another 7 volumes. You may pre-generate all volumes at once.

Add New Volume Node on Eureka

nohup ./weed volume -max=0 -mserver=”aloha:9333″ -dir=”/data/brick1/sw” > volume.out 2>&1 &