EMC rumoured to be upping costs for Mozy

EMC don't seem to be able to make storage pay as a service. According to techworld EMC are set to hike Mozy service fees.

And that's always the thing about backup. It's expensive to do in terms of infrastructure and on the whole delivers little benefit to most people - the losses are usually small accidental deletions. On the other hand when you do have a major los you're damn glad you have it. On a ay to day basis a cost benefit analysis would probably show it's not worth doing. On the other hand it can make the difference between staying in business and not ...

Facebook ...

late to the party, I let myself be assimilated by the Facebook borg a little before christmas last year.

Why?

Curiousity. Students used it, it was mentioned on a couple of podcasts I regularly listened to, and it seemed to be taking over the world. So I joined it. And initially it was kind of fascinating. All these people I know who were on it, news of former colleagues, reconnecting with old friends, etc. Definitely a rush. We're social beings after all.

And well, what's it for? Jokes about "well the earth moved for me" alerted me to yesterday's small earthquake in england, but otherwise not much. Yes sure you can play games with friends far away, share photos and the rest, but you can do that anyway. What Facebook sells is the illusion of community in an increasingly fractured socially disconnected online world, allowing you to conenct with people that you sort of know and to keep up with their doings.

In short facebook is a sociological phenomenon, not a technical phenomenon, and something that says more about the way we live and how tenuous our connections are with people we consider friends. There's an xkcd cartoon that seems appropriate.

Solving the docx problem

Vista is approaching and the need to support it is appearing increasingly inevitable. Laptop HALs is one pressing reason. And in a strange but horribly related way Office 2007 is becoming impossible to resist, and with it the docx support problem.

Just to complicate it, we won't be upgrading everyone simultaenously, nor will be upgrading Mac users to Office 2008 immediately. And of course there are linux users. A minority, but if our plans for a linux based open source desktop go anywhere perhaps a growing minority. (The linux desktop may turn out be our 'get out of jail' strategy to avoid a widespread deployment of Vista)

So docX is not a possibility. Getting an ODF plugin for Office isn't really an option as people need to remember to export the files as ODF. And there isn't one for the Mac and converting the Mac Office users to Neo Office isn't an option. Too much work, too much training too much support.

So we need a left field solution - and that is - make the doc format the default document format for Office 2007. There's a registry hack that fixes that, and there's probably equivalent hacks for the new version of powerpoint and excel. What's more, there's a similar hack to allow Open Office to save in word formal by default.

So we've created a default document format. Admittedly based on Microsoft's proprietary format but .doc is the lowest common denominator. And with a default document format as standard, we don't care which platform or application people use, and we can set up an automatic workflow for normalising or archiving documents using either Xena or Docvert to do the automatic conversion.

Archiving everything as PDF seems like a document archiving strategy that's relatively future proof, even if that's a nother proprietary format. But that's an argument for another day :-)

Give kids a laptop and ...

Oh and another thing about the storage as a service model. Here in Oz the government is very keen to start giving kids laptops as education aids - you can already claim buying your kid a laptop for school against tax (within reason - $700 max). And the schools are getting more computers, better connections and so on.

But no one has asked, how are they going to back it up - you don't want little Johnny to lose his school graduation project a week before final exams do you. And the whole ideas of online portfolios for kids to lodge their final project reports, artwork etc is calling out for some sort resilient distributed backup.

And again a little voice is whispering in my ear 'rsync and cloud computing" as a solution to this ...

Zmanda, moonwalk, S3 and offiste DR ...

Zmanda, Moonwalk and the rest are interesting because they allow you to move the data offsite. Moonwalk is perhaps the most interesting as it allows integration into an HSM style policy based system, and one could imagine a scenario where files are backed up to some local clustered storage and then you write the "last changed" files to some local storage, perhaps a tape based archive that keeps all the deltas and an offsite location that keeps only the deltas and perhas does a consolidation task.

Thus in the event of a complete disaster we can revert to a known state - well that's true for unstructured data. For structured data like databases we need to do something different, such as writing dumps in multiple locations.

And of course for data we don't care a lot about we just do an rsync (or more accurately and rdiff) to ensure that we have a recent copy. (In fact for the UK Mirror Service - basically a local cache of key internet resources - we did exactly that, no backup, rsync the servers, and if both sites lost the same files pull them again next time we do a consistency check against the location being mirrored. This of course worked as we assumed that the mirrored sites were backed up by the host sites)

Simple.

Then costs come into the equation. Backups are expensive. Infrastructure costs alone head towards the million dollar mark. For a secondary disaster recovery site double it and then add some for network traffic, which you'd pay anyway if your data crossed a public network. So the cost benefit analysis of using something like S3 comes down to "are our costs going to be less outsourcing than doing it ourselves?"

If you are a relatively small site and you have enough bandwidth to easily trickle the deltas across, I'd guess yes, after all it's only your critical unstructured data. If you're large it might be worth doing s cross hosting deal.

But the key is data classification to maintain costs - you only want to save the stuff you can't afford to lose.

The other key thing is not to get held up on particular vendor's offerings -it's the technology that's important at this stage. Zmanda and Moonwalk in combination with S3 are demonstrations of what can be done. The message is that the economies of cloud computing architectures make realtime access to offsite data stores a distinct possibility.

After all backup is only copying data and putting it somewhere safe ...

S3 and tiered storage

In an earlier post I wrote how a company called zmanda was using Amazon's S3 service for backup. So I guess it's no surprise that another company, moonwalk, have fiited S3 into a tierd storage model, which is not only cool, but also has some interesting implications for disaster recovery and the like.

This morning's SMH had an article covering moonwalk's developments - sort of zmanda and a bit more.

Worth checking out

Docvert - cool document conversion

One of the great problems with digital preservation is getting documents into a standard format for indexing and preservation. The National Archive uses Xena to normalise the document to XML, but I've just come across Docvert, which is kind of like Xena on drugs. Very powerful, very slick. An excellent solution to the bulk document conversion problem and one which can be pipelined for onward processing of the document.

economy computing ...

I've always wanted a linux machine at home to play with, but the realities of life have meant up to now that we've had to have a more mainstream machine, as we need to share it. And so we've soldiered on with a win2k laptop, which is now close to 7 years old.

Well we've broken free of our Microsoft addiction with a sexy new aluminium imac, having finally convinced Judi that the operating system is irrelevant and iWork will do what we need to do domestically. [.doc != Word != XP].

What finally convinced her was work giving her a vista thinkpad with Office 2007 which immediately caused all the problems about being able to exchange documents with people with non-upgraded machines and having to spend time hunting around Office 2007 to do what could be done with a few keystrokes in Office 2003.

So, the linux machine. Well we had a couple of old pc's, one I'd bought in, wait for it, 1998, and one I'd bought for $20 from a previous employer. (It would have cost them at least that to send it to auction or dispose of environmentally).

Well I'd never even powered up the $20 special, despite it sitting in the garage for two years, but mid last week I did, and after sitting in the garage for two years it powered up perfectly, and then hung trying to connect to non existent novell network.

Powered it down, restarted with an ubuntu install cd, and an hour later, hey presto, a working linux machine.

Now that still left me with an old 1998 pc. Opened it up, pulled the disk and the memory, which was pc-100 like the $20 special, and installed them in the special. A bit of fiddling with the bios and reinstall, with my usual failed first attempt at partitioning (I don't know, it's not just intuitive to me) and then, again a working machine.

Stripped the case of cd drive and power supply just in case I even need spares and then it was ready for the scrap heap.

The only problem was the monitor. It again was 1998 vintage and getting distinctly soft and slightly fuzzy. Keyboard and mouse were not a problem, I've a couple of spare old style pc keyboards and mice, but the only other monitor I had wasn't that hot either.

However Canberra being Canberra, there's a lot of surplus ex government equipment around, most of it sold at auction. I didn't want a pallet load of monitors but I found an online auction site that was selling off Sun VGA compatible LCD's individually.

$63 and a week later I had a nice four year old Sun LCD in good condition, plugged it in to the linux machine, and there it was, super sharp.

So sharp in fact that when Judi first saw it she asked me why I was setting up the imac in the shed.

But for $83 I've second machine for writing, email and web when the imac is tied up with other things. And it isn't Microsoft!

So far so good. Now the linux special isn't connected to the network, but we got ourselves a new adsl home gateway through our isp and I persuaded Judi the linux machine (the designer in her) looked good enough to live in the study - I could have used a wireless bridge, but as the shed is made of tin it functions as a really nice Faraday cage making networking a tad difficult.

So, set up the gateway. First wireless conenction. Just works. Connect old W2k laptop via ethernet, just works. Share disks. Just works. Then the damned box just dies - won't do ethernet though wireless works. Reboot it, powercycle it etc. Splurg. dead. Wireless works, we lit did till sometime on sunday morning. Reconnected old modem and emailed our isp's tech support. Of course at 1100 on Sunday you don't get much - but I think they owe us an adsl 2 box that works.

Funny isn't it - the old stuff just works and the new stuff ...