hadoop in a word. Very involved in the development of the hadopp file system and I guess it's going to be their googlefs equivalent for content hosting.
I'm also guessing that flickr will be in the [does | will do] box as well.
Which is kind of interesting. And leveraging off opensource save some of your development costs as well.
And as a bit of non sequiter (ok I want to keep track of the url), there's an interesting tale from the NYT about hadoop, Amazon's cloud computing environment adn digital content management
See also:
ReplyDeleteAmazon EC2 and S3 (I use these)
s3.amazonews.com
In particular, Hapdoop on S3 not hadoop-fs:
http://wiki.apache.org/lucene-hadoop/AmazonS3
Amazon's Dynamo:
http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf
It's a key-value system not a normal filesystem and only available internally. Still a good read.
The Ceph filesystem
http://lwn.net/Articles/258516/
Interestingly, only the last of these even tries to preserve posix semantics.