Creating Filesystems with Ruby and FUSE

Posted by Steve on Fri 7 Nov 2008 at 19:30

The FUSE project allows you to create filesystems in userspace - which means you can create a filesystem without having to get your hands dirty and modify your kernel source. This is insanely cool, and can be used for many purposes. Here we're going to look at using the Ruby bindings to create a simple filesystem.

The FUSE project allows you to create a filesystem entirely in userspace merely by writing a couple of primitives - such as:

  • Get contents of directory.
  • Get file permissions.
  • Write to file.

With enough of the basic primitives implemented you'll end up with a working filesystem. Unfortunately unless you're a fan of working in C this process is a little more complex than it should be. To ease this there are bindings for Perl, Ruby, and other languages which makes getting started very simple.

My recent FUSE experiments have been Ruby-based, using the libfusefs-ruby1.8 packages.

Assuming you're working with a recent kernel you should be able to get started writing a filesystem very quickly. Here's what it took for me:

trashvm:~# apt-get install libfusefs-ruby1.8 fuse-utils ruby1.8
Reading package lists... Done
Building dependency tree... Done
The following extra packages will be installed:
  libfuse2 libruby1.8
The following NEW packages will be installed
  fuse-utils libfuse2 libfusefs-ruby1.8 libruby1.8
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 1716kB of archives.
After unpacking 6476kB of additional disk space will be used.
..

Once I installed the packages I inserted the fuse kernel module:

trashvm:~# modprobe fuse

Assuming this completes without errors you're ready to begin! Lets create a simple filesystem that just proves we can. Create the file hello.rb:

require 'fusefs'

class HelloDir
  def contents(path)
    ['hello.txt']
  end
  def file?(path)
    path == '/hello.txt'
  end
  def read_file(path)
    "Hello, World!\n"
  end
end

hellodir = HelloDir.new
FuseFS.set_root( hellodir )

# Mount under a directory given on the command line.
FuseFS.mount_under ARGV.shift
FuseFS.run

Once you've done that you can mount your filesystem by running:

trashvm:~# mkdir hello/
trashvm:~# ruby hello.rb hello/ &

Once you've done this you should find you've got a directory called "hello/" which has our filesystem mounted in it. This filesystem just creates a file called "hello.txt" - and when you read it you'll get the expected contents:

trashvm:~/hello# ls
hello.txt
trashvm:~/hello# cat hello
Hello, World!

You can now kill the userspace program and all will be gone:

trashvm:~# kill -9 %1
[1]+  Killed                  ruby1.8 hello.rb hello
trashvm:~# umount  hello
trashvm:~# ls hello
trashvm:~#

All gone. Now is time to confess that I didn't create that hello.rb file - it is one of the examples included with the libfusefs-ruby1.8 package. If you take a look beneath /usr/share/doc/libfusefs-ruby1.8 directory you'll find several examples which each work in a similar way to the one we've demonstrated.

So, what could you do with a FUSE filesystem? Well if you're creative there are almost no limits. As a more useful example I've placed a simple filesystem I created online at the foot of this article. This allows you to mount MySQL as a filesystem:

skx@gold:/mysql/yawns$ ls
about_pages.data   notifications.data             related.data
about_pages.sql    notifications.sql              related.sql
adverts.data       permissions.data               scratchpads.data
adverts.sql        permissions.sql                scratchpads.sql
articles.data      poll_anon_voters.data          sessions.data
articles.sql       poll_anon_voters.sql           sessions.sql
bookmarks.data     poll_answers.data              submission_notes.data
bookmarks.sql      poll_answers.sql               submission_notes.sql
comments.data      poll_questions.data            submissions.data
comments.sql       poll_questions.sql             submissions.sql
db_version.data    poll_submissions_answers.data  tags.data
db_version.sql     poll_submissions_answers.sql   tags.sql
ip_blacklist.data  poll_submissions.data          tips.data
ip_blacklist.sql   poll_submissions.sql           tips.sql
messages.data      poll_voters.data               users.data
messages.sql       poll_voters.sql                users.sql
news.data          preferences.data               weblogs.data
news.sql           preferences.sql                weblogs.sql

Each directory beneath /mysql is a database, and each table of that database is presented as two files:

  • foo.sql - Just the table structure.
  • foo.data - The contents of the table.

The code is very simple and mostly revolves around invoking mysqldump correctly, but despite that it is surprisingly useful.

I'd love to hear suggestions on interesting uses for FUSE, and what you're doing with it!

This is the code which implements the MySQL filesystem. It is very short as it only involves implementing a couple of methods:

  • contents - Return the filenames in a given directory.
  • directory? - Is the given name a directory?
  • file? - Is the given name a file?
  • read_file - Return the contents of a file.

 

 


Posted by dkg (216.254.xx.xx) on Fri 7 Nov 2008 at 21:55
[ View dkg's Scratchpad | View Weblogs ]
Nice article, Steve! And thanks for the example, too. It makes me want to play with FUSE.

I find it interesting that FUSE is bringing over big pieces of the translator concept from microkernels like Hurd to Linux (which i guess is more of a hybrid than a monolithic kernel these days).

With subsystems like this, I'm curious (as always) about how they handle error cases, and whether it's possible to cleanly drop privileges for the process that backs the mountpoint. Have you experimented with either of these questions?

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Sat 8 Nov 2008 at 00:23
[ View Steve's Scratchpad | View Weblogs ]

The basic error cases seemed to be handled fairly well, but I admit I've not hit anything too weird in my current use.

The only thing to say is that sometimes errors weren't clear. For example trying to bind two filesystems to the same directory failed - and the error was misleading.

Again permissions have mostly worked as expected; if you're a member of the fuse group you can bind/unbind new filesystems - but in this piece I only mentioned root because that is simpler.

I've written a couple of toy filesystems to do different tasks, but so far nothing too useful. I'm sure there are a few killer applications out there not already implemented it just takes that spark of creativity to come up with them.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (62.142.xx.xx) on Mon 10 Nov 2008 at 07:50
It appears that you have a bug in your mysqlfs.rb.

The constructor should be initialize not initialise

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Mon 10 Nov 2008 at 09:19
[ View Steve's Scratchpad | View Weblogs ]

Thanks, that was a last minute addition that was clearly wrong.

Updated now.

Steve

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 496 votes ~ 5 comments )