Do it Yourself NAS?

So the question has got to be, can we build the filestore solution ourselves? 

Now when we talk about providing student filestore we are talking about providing shares (aka mount points) and lots of them to a filestore that has a lot of small files with a lot of churn.

This has typically been done with nas devices, purely because san storage is expensive. Of course you don't actually need a nas device you could build one yourself. Linux + Samba + JBOD would give you something that could export shares to nfs and cifs. In fact Auspex, one of the early nas manufacturers did exactly this - if you opened up their boxes there was a sun file server inside the case controlling the disks and the shares.

The following links describe how you could do this with various implementations of OpenSolaris:

• http://schlaepfer.nine.ch/twiki/bin/view/Schlaepfer/SelfMadeNas2

• http://www.sun.com/aboutsun/pr/2008-04/sunflash.20080429.1.xml
• http://www.nexenta.com/corp/
• http://blogs.sun.com/icedawn/entry/bondin

and the choice of OpenSolaris  is a good one given ZFS's ability to do copy on write and snapshots. 

Why, because backing up big rapidly changing filestores is difficult due to the treewalk program. If you snapshot the filestore and then do an image level backup of the filestore you can post process it to do a synthetic incremental file level backup, which is kind of useful given that users want to restore files, not volumes.

Now the reason that everyone doesn't simply build their own NAS is that simple home built solutions don't scale well. Using a full size operating system is inefficient, as is using a conventional filesystem. ZFS addresses some of the filesystem problems, which was why NetApp got all antsy.

Big NAS solutions can handle lots of connections very efficiently, and that's down to the combination of a very efficient filesystem and very efficient minamalist operating system that is optimised to handle files and not do a lot else. FreeNAS  tries to address this by using a cut down version of BSD and get some efficiency gains that way. And because it's BSD, you can run samba and netatalk to give cifs and afp support.

But it's a single box solution, and therefore limited as to connections it can handles etc etc. But it might be fun to build one to see what it can do.

Now our apple based design tries to address these problems:

The client machine connects through a content server switch (CSS) to one of the front end servers. The CSS load balances on a round robin basis, which means we can spread the load between multiple servers, add new servers, drop servers out for maintenance etc. Even though this is a NAS style solution the storage is provided by an Apple XSAN solution. 

It's more complicated than I've drawn it with metadata controllers etc but essentially it's connected together by fibre through a switch, meaning that all the filers can see all the storage. This granular design makes adding more storage reasonably simple. 

We also have some other things behind the scenes, such as backup drive, which is essentially provided by running rdiff when users log out to copy their changes across so we have a live copy we can then backup to tape or whatever.

It works. My problem when I start reading about clustered filestores such as exanet  and isilon  is twofold:

1. Is their solution any more than our solution placed in a box with a badge on the outside?
2. If it is more, what is the extra?

Now I'm sure they can do other clever things and the management is slicker etc etc, but is the functionality noticably greater?

And does that functionality provide a unique advantage?