The problem:
We need a common filestore to support windows and macintosh clients, and potentially in the future linux clients. The filestore needs to be able to allow people to access their files from all three platforms as students may use a mac lab for one course module and be working on a term paper in a linux lab.
This isn't a new problem. Been here before and all that. First of all with Dec Pathworks to share Windows 3.0 filestore with VMS using DecNet and then Sun's PC-NFS to share Windows 3.11 filestore with Unix filestore over NFS. This was 15 years ago and it worked robustly. In fact it worked so well that we tried to integrate Apple System 7 machines into the equation. And that was a failure as the Apple file system use what's known as Apple double, where a file actually consists of two objects, a resource fork and data fork. The data fork contains the information, the resource fork contains all sorts of extra stuff about what application can open it etc. In essence, the structured metadata.
Unix, of course is not so sophisticated, files are collections of bytes which may contain a combination of metadata and data, or just raw data. Windows fakes it by flle type associations and application specific registry keys but really the files are just collections of bytes. This means that Unix systems can access windows files without too much difficulty and vice versa. The only real problem is that (post NT) Windows has a richer and more complex permissions model than Unix/Linux.
So one has to fake up the apple double file format on linux by storing two files, one for the data fork and one for the resource fork. The typical way it can be done is to create a file called .fred for the resource fork and fred for the data fork. Filenames starting with a dot are normally invisible under unix, just as if they had the hidden +h attribute set under windows.
Back in 1993 our approach was to use GatorBoxes to act as protocol converters. It wasn't very reliable and we had problems with resource fork corruption, load, and hardware reliability. Fortunately, round about the time this was happening Apple looked about to go down the toilet so we canned Macintosh support and proceeded to ignore the problem.
Over the years we carried on with the linux/windows common filestore using Network Appliance boxes with native sharing, or standard Unix boxes with Samba. Both work well, both have kinks, but basically you can share unix filestore between linux boxes on nfs and windows boxes on cifs without too much difficulty, and if you want to share windows filestore linux boxes can mount cifs shares.
Fast forward to today.
OS X is now commonplace. However OS X still has the Apple resource fork/data fork model, which makes sharing files between platforms just as messy as it was fifteen years ago. OS X can mount cifs filestores and by extension a unix filestore exported using samba, but that's not the point. You've lost the richness of the resource fork. NetApp have a paper describing an approach you could take but it's not a true solution and involves installing extra software on the client. The alternative would be a 'samba for afp', ie an application to do protocol translation the way that sambas does for cifs, and which also could cope with Apple Double.
There is such an application - Netatalk. Netatalk solves the resource fork problem by storing them in a .AppleDouble directory, ie a hidden magic directory. And that brings all the problems of damaged resource forks changed documents and luddites who get rid of .AppleDouble directory to free up filespace.
Up to now we've been running another sort of solution. A cluster of Apple Xserves and Xraids that run Samba and also export the filestore using AFP. This means that we have native filestore handling of Apple files, and samba provides access to windows and linux hosts. That works well, but it has scaling problems, and Apple's canning Xraid filestores doesn't help.
So potentially we need another solution. If we didn't have OS X, NetApp or BlueArc would do the job perfectly. Exanet hints that they can support AFP. That would certainly support our heterogenous client problem. But of course it may just be Netatalk in disguise - after all a lot of cifs emulators are built around the samba source so I see no reason why the same shouldn't be true for AFP support. In fact it's even more likely as AFP support is a minority religion, and until recently hadn't been a 'must have' for corporates. Macs simply lived in their own world and mounted foreign filestores if they really had to. The fact that things really havn't moved on since 1993 makes this painfully obvious.
Now we have to not only share the file store between platforms but that users will use different platforms to access the same files on different occasions. And that's the rub. If all we needed was a common filestore and mac users only ever used macs and accessed mostly their files something like netatalk could work. If however users flit between platforms we buy ourselves a bag of problems.
The cynic in me says 'thin client and webdav'. It might be the simplest if not the cheapest ...
3 comments:
I've come to the conclusion that trying to provide unified filestore across OSs is just setting yourself up for pain.
Mixed mode on the netapps has been nothing but trouble for us, and we're not even trying to add macs into the equation.
Given people an easy way access their files via a browser instead. We're currently looking at SSL VPNs.
nice article.
have you had a look at a company called Helios and wht they do to solve the "unified" filestore probem as you call it?
We've tried Helios. Like Exanet their AFP support is essentially netatalk with the consequent restrictions.
Basically if you want to do AFP you need something like the Queensland Brain Institute solution where you use an Xserve to front a NFS based resource.
Some experimentation required ...
Post a Comment