GlusterFS is an open source distributed parallel file system capable of scaling to several petabytes.
GlusterFS is a cluster/network file system. GlusterFS comes with two components, a server and a client.
The storage server (or each server in a cluster) runs glusterfsd and clients use the mount command or glusterfs client to mount the file systems served, using FUSE.
The goal here is to run 2 servers that will perform complete replication of part of a filesystem.
Be careful not to run this type of architecture on the Internet as performance will be catastrophic. Indeed, when a node wants to read access a file, it must contact all other nodes to see if there are any discrepancies. Only then does it authorize reading, which can take a long time depending on the architectures.
### file: server-volume.vol.sample######################################## GlusterFS Server Volume File ########################################### CONFIG FILE RULES:### "#" is comment character.### - Config file is case sensitive### - Options within a volume block can be in any order.### - Spaces or tabs are used as delimitter within a line. ### - Multiple values to options will be : delimitted.### - Each option should end within a line.### - Missing or commented fields will assume default values.### - Blank/commented lines are allowed.### - Sub-volumes should already be defined above before referring.volume posix1
type storage/posix
option directory /var/www
end-volume
volume locks1
type features/locks
subvolumes posix1
end-volume
volume brick1
type performance/io-threads
option thread-count 8 subvolumes locks1
end-volume
volume server-tcp
type protocol/server
option transport-type tcp
option auth.addr.brick1.allow *
option transport.socket.listen-port 6996 option transport.socket.nodelay on
subvolumes brick1
end-volume
### file: client-volume.vol.sample######################################## GlusterFS Client Volume File ########################################### CONFIG FILE RULES:### "#" is comment character.### - Config file is case sensitive### - Options within a volume block can be in any order.### - Spaces or tabs are used as delimitter within a line. ### - Each option should end within a line.### - Missing or commented fields will assume default values.### - Blank/commented lines are allowed.### - Sub-volumes should already be defined above before referring.# RAID 1# TRANSPORT-TYPE tcpvolume ed-1
type protocol/client
option transport-type tcp
option remote-host ed
option transport.socket.nodelay on
option transport.remote-port 6996 option remote-subvolume brick1
end-volume
volume rafiki-1
type protocol/client
option transport-type tcp
option remote-host rafiki
option transport.socket.nodelay on
option transport.remote-port 6996 option remote-subvolume brick1
end-volume
volume mirror-0
type cluster/replicate
subvolumes rafiki-1 ed-1
end-volume
volume readahead
type performance/read-ahead
option page-count 4 subvolumes mirror-0
end-volume
volume iocache
type performance/io-cache
option cache-size `echo$(($(grep 'MemTotal' /proc/meminfo | sed 's/[^0-9]//g')/5120))`MB
option cache-timeout 1 subvolumes readahead
end-volume
volume quickread
type performance/quick-read
option cache-timeout 1 option max-file-size 64kB
subvolumes iocache
end-volume
volume writebehind
type performance/write-behind
option cache-size 4MB
subvolumes quickread
end-volume
volume statprefetch
type performance/stat-prefetch
subvolumes writebehind
end-volume
If you want to force data synchronization for a client, it’s simple. Just go to the directory where the glusterfs share is located (here /mnt/glusterfs), then perform a directory traversal like this:
1
ls -lRa
This will read everything and therefore copy everything.
If you look in your logs and see something like this:
(/var/log/glusterfs/glusterfsd.vol.log)
1
2
3
4
5
6
7
...
+------------------------------------------------------------------------------+
[2010-10-17 00:40:30] W [afr.c:2743:init] www-replicate: Volume is dangling.
[2010-10-17 00:40:30] C [posix.c:4936:init] www-posix: Extended attribute not supported, exiting.
[2010-10-17 00:40:30] E [xlator.c:844:xlator_init_rec] www-posix: Initialization of volume 'www-posix' failed, review your volfile again
[2010-10-17 00:40:30] E [glusterfsd.c:591:_xlator_graph_init] glusterfs: initializing translator failed
[2010-10-17 00:40:30] E [glusterfsd.c:1395:main] glusterfs: translator initialization failed. exiting
It means you have permission problems. In my case, this happened in an OpenVZ container. To solve the problem, here’s the solution to apply on the host machine (not in the VE) (warning, this requires stopping, applying configurations, then restarting the VE):
If you want to do glusterfs in a VE, you may encounter permission problems:
fuse: failed to open /dev/fuse: Permission denied
To work around them, we’ll create the fuse device from the host on the VE in question and add admin rights to it (not great in terms of security, but no choice):