Wednesday, 8 May 2019

recovering data from garages

Earlier today a former colleague retweeted this:

and strangely, I've been here before.

When I was managing the ANU's various ANDS funded data capture projects we made use of company in Perth that specialised in reading old tapes - in particular for the mining industry, but they would read anything - for a fee of course.

As part of the DC7A project, we used this company to read seismological data that was locked away on piles of DAT tapes that no one could read any more, due to no one on campus having suitable hardware.

As is the way in universities,  a researcher in social sciences, who worked in PNG heard of this.

He'd recently found some old 9 track tapes in a colleague's garage, and he recognised them as likely to hold a copy of some data from the PNG government. More importantly he thought that it might be data that the PNG government had lost as a result of a hardware failure.

Details were sketchy, there were some paper labels that identified them as 9track ascii tapes, but that was about it.

Any way I talked to our data recovery company and they were happy to give it a go.

Fortunately, despite languishing in a Canberra garage, the tapes were readable and were in a straight forward comma delimited format, rather than some old proprietary compressed data format, or some strange format used by some now forgotten data manipulation software.

So the data was recoverable and could be returned to the PNG government.

Now I'm not blogging about this seven years after the event to show how good I am, but rather to show that old data can (with a bit of luck) be recovered.

But to simplify your task do the following:

  • try and find if there's anyone still left who remembers the days of tapes - hopefully they might be able to help interpret the (paper) labels stuck on the tapes.
  • talk to the people who are going to read the tapes. Chances are the tape will be in a 9 track format and be ascii encoded, unless it came from somewhere that used IBM or Amdahl mainframes where it might be EBCDIC
  • don't be put off by people mentioning dead manufacturers like Prime or Data General, 9 track was a fairly standard format
  • When you finally get to read the data, remember that even though it's been recorded in a standard way it doesn't mean that the data isn't in some proprietary format - again if you can find someone who knew about the original data they might remember the name of the software package used
do some detective work, and chances are you might luck out, and don't be afraid to ask questions ...

Sunday, 28 April 2019

Espresso book machines revisited

A long time ago, over ten years ago in fact, I became quite excited about the espresso book machine.

At the time it seemed to offer the promise of small run book publishers, such as your typical small university press, the opportunity to avoid the costs of printing and holding stock, as well as the potential to on demand reprints of out of print books.

Well, ten years on, the landscape hasn't quite changed as I imagined it. Yes, there are various printers, mostly in India, who will do a cheap reprint of an out of print nineteenth century book, by printing a copy of a scanned edition downloaded from the internet archive, something for which you basically need a laptop, an internet connection, and a laser printer, and access to the equipment required to bind a book, which in a low cost country such as India, where labour is cheap and there is a well established book printing industry, it's probably cost effective to have a semi manual process.

But recently I've bought a couple of scholarly short run Australian books. Even though they were ordered through Amazon Australia's marketplace, due to the mysteries of the book trade, they came from online booksellers in the UK, and they had the look and feel of a print on demand book.

Strangely the front matter that contains the copyright statement and the NLA cataloguing in publication data, didn't list a printer, but at the back of the book there was a QR code and the text Lightning Source Milton Keynes, followed by what was obviously a reference number of some kind.

Being curious I turned to Google to discover that Lightning Source have a pretty informative wikipedia page,

Basically Lightning Source is an offshoot of the same company that developed the espresso machine and provides a print on demand service to small publishing houses - just as I thought would happen all these years ago - and what's more the espresso book machine is most decidedly not dead ...

Sunday, 14 April 2019

Not another bloody thinkpad ...

I've recently blogged about how I finally got around to getting myself a new larger screen laptop to replace my old Dell Inspiron, and of course I bought myself an old Thinkpad around about a year ago, which did a stellar job of replacing my official HP Probook when I dropped coffee on it.

Well I've been so impressed by both of my Lenovo machines I've gone and bought myself a Lenovo Thinkpad Yoga 11E, one of the old touch screen models you can use as a bulky tablet.

Windows 10, 128GB SSD and 4GB RAM, and a reasonably specified processor - all for around $200. I even get 3 months warranty from the refurbisher.

So a bargain, and quite a rational purchase.

I'll explain why:

To get the most out of my old Thinkpad I really should upgrade it to Windows 10, and guess what, the upgrade cost is near enough what I just paid for the Yoga. Now if that was the only consideration I'd probably just have bought the upgrade, but I've two other pressure points:

  • My Chromebook has gone end of life - no more updates, and gradually things will cease to work. At what point it becomes unusable is unknown but what's clear is that the replacement cost will be around $400. One of my major uses of my Chromebook is reading my email and rss feeds in bed - the Yoga with it's touch screen etc is a more than decent replacement
  • My MacBook Air (a 2012 machine) is probably going to drop off the OS X supported device  list sometime soon. On top of that it could probably do with a new battery - it used to manage a couple of hours between charges, it's now managing barely an hour. A new third party battery replacement kit is around $150 if you fit it yourself, or a bit over $200 if you have a repair shop do it for you. The Yoga is heavier than the air and little bit bulkier, but could feasibly make a decent travel computer, and being roughly the same form factor as the Air will fit in both the travel backpacks I own.
So, at the moment, I seem to own a stupid number of computers. However, the old 2008 vintage iMac I use when working with old documents is showing its age, it's already unsupported as regards MacOS and I expect that Google will soon stop supporting Chrome for that version of the operating system, and it will eventually fade away. 

The Air will obviously last a little longer, but one can see the writing on the wall, as one can with the Chromebook. I expect to keep on using my old unupgraded Thinkpad X230 for another couple of years at least.

The Yoga, being ruggedised for educational use, should last as long, and survive trains planes and car trips reasonably well. It also has a decent thinkpad style keyboard to type on (as good as the X230's) which adds to its attractiveness, so I reckon at $200 it's a bargain, and while $200 is a reasonable amount of money, it's not much more than a night in a decent city centre hotel ...

Thursday, 4 April 2019

Coffee 0 HP Probook 1

As I'm sure you're all aware, about six weeks ago I was stupid enough to pour coffee over my work laptop.

Well, it went off to the repair shop, and obviously my prompt if panicked reaction saved the day.

It was stripped down, cleaned up in an isopropyl alcohol bath. The processor daughter card was damaged, but that was replaced with a refurbished spare - tracking one down was the reason it took six weeks to repair my laptop, and it's back, almost as good as new.

All the data has survived, not that it wasn't backed up. The only problems are that it seemed to have lost its network configuration data - hardly a problem really, and the SSID was tied to the processor, so naturally excel whinges that it hasn't been properly activated, again something that just requires the contacting corporate IT dance .

Resyncing the data back wasn't a problem either, all I needed to do was download the data from OneDrive to cover the missing days and open OneNote, and tell it to do a sync. Fifteen minutes work at most.

Obviously before I say it's really fixed I need to use it for a few days, rather than a quick click around but everything looks great.

Oh, and if you're worried that you might be at risk of spilling something on your laptop, check out this sensible advice from the NYT...

[The original title of this post was 'Coffee 0 HP Powerbook 1' - complete brain snap on my part, the laptop in question is a ProBook - a 6470b to be exact]

Sunday, 31 March 2019

Power outages and documentation

As a rider to my use of coffee to prove a documentation methodolgy, we had another proof of the scheme's robustness a couple of days ago.

Under the scheme, data is saved twice, once to the computer's local drive and secondly to a USB stick. The data on the computer's local drive is also backed up to OneDrive, and entrusted to Microsoft to look after.

The crucial point is that you don't need a functioning internet connection to carry out documentation - as long as you have access to one somewhere in the piece to back the data up everything is fine as you always have at least two copies of the data - very useful as I found in the coffee pouring incident as I was able to check and confirm that all the data had been backed up.

This time it was the power company. The power went off with an unscheduled outage, and more importantly stayed off. However as I had (conservatively) about three and a half hours of battery life left on my computer and the same on my phone - I use my phone to take pictures of the artefacts and transfer the data to my computer. Normally I recharge my phone as I go from my laptop, but obviously I didn't do that once the power went off - a severe case of robbing Peter to pay Paul.

So, with three and a half hours worth of power I could stay working.

Which I did - the only limiting factor was that it began to cloud over in the early afternoon, and the light began to go, making it difficult to work.

Once home, I powered up my laptop, let it sync to OneDrive, and hey presto, we were done and backed up...

Saturday, 30 March 2019

Ok, finally got myself a new computer


about a month ago I finally got round to buying myself a new computer.

Lenovo had a special offer on their AMD Ryzen systems where you got a 512GB SSD for the cost of the standard 256GB unit, and the one thing I'm hungry for is storage.

So I went for it.

Of course as it was a special build to order configuration I had to be patient and wait for it but it eventually arrived yesterday.

Out of the box it just worked. I can't say I took to the slightly shouty voice enabled activation assistant, but, but it all just worked.

And once it was configured, all I needed to do was add the tools I use, much as I last year with my old thinkpad.

Speed to set up, download and configure were impressive, and while the keyboard wouldn't be my first choice (I prefer older clacky ones), it's pretty nice to type on.

The only annoyance was that to install Dropbox, I had to unlink some of my older machines, as Dropbox now limits free accounts to three clients, but then there's also for use with older machines, and I guess I could start using Box more ...

[update 31/03/2019]

which indeed I've done. I've added the box client to my new computer and to my ipad (on which I'd never got round to installing dropbox) - and we'll see how this goes ...

The use case is of course slightly different - when dropbox, box, and the rest first came on the scene there was little in the way of cloud based storage, and sharing files between machines essentially meant copying them between machines.

Dropbox like services' unique proposition was that the files were always in sync providing you had a working connection.

Things of course are different these days. Be it OneDrive, Google drive or Amazon's services there are lots of way to both share files between machines and ensure that they stored securely. For exampl, if I'm working in a library somewhere with my ipad, I can easily save the notes I've written by sending them to OneDrive from pages, or indeed saving them to icloud.

What Dropbox (and the rest) now have as their unique proposition is  now 'save once, sync everywhere' without people having to go looking for the latest version.

Given the chaos I've seen with shared editing of funding proposals, that's a pretty powerful proposition for a group, but for an individual, especially as the first tier up costs the same as any other storage solution - say A$15 a month for a terabyte - perhaps less so.

As I said, we'll see how this goes ...

Thursday, 21 March 2019

Things I didn't know: part 183 - Alt F4 and shutdown

Once, a very long time ago, I used to be a power windows user.

Built and maintained windows based network installs, that sort of thing. Then I got to be a boss and had people to do things for me.

So while I learned and did new things I definitely lost my technical edge. In fact the last version of Windows I really knew anything about was XP, and even then I really just used my XP machine for remote console stuff and getting onto linux machines.

And when you work like that you really don't know a lot about the underlying OS, it's only a vehicle to get you there.

So much so that for the last 10 years or so before I retired I never really used Windows. Ok I did a bit of windows 7 at home, but 90% was mac or linux, and as I say it could have been BeOS as long as it ran a standard, and recent, web browser and you could run an ssh tool without having to put a snail in your left ear and dance round a tree at midnight. Standardish applications were useful, but once LibreOffice cracked Microsoft Office file level compatibility it really didn't matter what you used (unless of course you had to deal with a funding proposal created with some really bizarre Word template),

But I did use windows 7 enough to know about automatic updates and windows' irritating habit of sitting there and fiddling about installing updates when all you wanted to do was shut down and go to the pub.

Of course what you do in a work situation, or at home, is mutter under your breath and leave it to it. That is providing you don't want to take your machine home with you.

I did use to idly wonder what you did if it wanted to fiddle with itself when you were in a situation where you simply had to do shutdown - such as in an airport and about to go through security.

I thought there must be a way of doing it, but never bothered to find out, after all I had a MacBook or Dell XPS with Ubuntu as my work laptop(s).

Well yesterday it happened to me. I've been using my Thinkpad for the documentation project ever since I accidentally dropped coffee on my office laptop. And when you are working in BYOD mode, of course you want to take your machine home with you (not to mention that our NBN FTTC link at home is a lot faster than the Trust's ADSL link, so it sometimes makes sense to finish off stuff at home).

Got the dread little orange shield and exclamation mark thing on shutdown on my Thinkpad. And it was a time when I had to shut it down properly.

So I googled, and discovered all about Alt F4 and accessing the full shutdown dialogue.

And it works. Like a dream.

And you can use the same trick on Windows 10, which is kind of useful to know as I've finally bought myself a new Windows 10 laptop ...

Thursday, 7 March 2019

DNA testing of old family documents

Over the holiday season I spent a bit of time messing about with family history research.

One of the things I found was that if your ancestors got up in the morning, went to work, didn't end up in court, or burn down public buildings, they don't leave much of a trace in public documentation sources like newspaper archives.

Just the same as they don't end up in military records, or indeed convict records. but if they come from a country with a functioning public records system, you can at least trace the shape of their lives back to the 1850's, and possibly earlier, that is providing no one burned down the records office or pulped the records as no longer required.

The other sort of thing one can have are collections of letters passed down. This is particularly so in countries built on the back of migrant communities such as Australia, where people's ancestors may have come from places without a public records system, or one where war, revolution, and ordinary disasters has introduced gaps into the system.

However, even though the records may have gone a lot of places had an excellent postal system, which means that if you have the shoebox of granny's letters when she was still a schoolgirl in the Ukraine, you can trace your missing ancestors - maybe.

Certainly you can trace the aunts an uncles from the contents and perhaps a little more from old love letters, but you still come up against the problem that paper burns.

But if you could trace the DNA, maybe you might get a match from the stamp, or the envelope - sealed with a loving kiss - which I guess might well be driving the move to offer DNA testing of old letters - and that might offer people closure of some sort, particularly if your family was torn apart by the chaos that engulfed places like the Ukraine in the early part of the twentieth century.

There are of course other scenarios, Indian sailors who jumped ship and went to work in a curry house is one that comes to mind, or indeed trace the movements of British soldiers around the empire, and indeed the wives and sweethearts (and perhaps children) they left behind ...

Saturday, 2 March 2019

My Chromebook's gone end of life

This morning (2nd March here in possum town, but still 1st of March in the US) I made myself a cup of tea, fed the cat, took Judi a cup of tea, and powered up my chromebook to read my email.

All what happens most mornings, except that this time a little window popped open to tell me my Chromebook was end  of life and no longer supported.

Well this seems a little ridiculous -  while it's almost exactly five years old, it still works well, does its job.

So my plan is this:

  • procrastinate until it becomes unusable
    • I only really care about email and a couple of web based services
Loading an alternative OS, such as gallium OS or neverware  isn't really an option as my Chromebook is one of the Samsung Eyxnos powered devices - an HP11-1101 in fact and the firmware (it is said) restricts matters as regards alternative OS's.

We'll see how this goes. I guess long term the answer is to buy one of these very cheap eMMC memory based windows laptops - after all all I need is Chrome, and perhaps Thunderbird ...

[update 03/03/2019]

And slightly bizarrely, this morning I got an operating system update, so I guess that in this case unsupported means no guarantees about updates continuing to work, rather than we don't want to talk to you any more ...

Thursday, 28 February 2019

Using coffee to prove a documentation scheme

Last week I did something I have never done before.

I dropped coffee on the keyboard of my laptop while down at Chiltern working on the documentation project.

Being the highly trained IT professional I once claimed to be, I screamed 'Sh-i-i-t!!' while simultaeneously pulling out the power cable and inverting the laptop in the hope that any coffee that had got into the innards would go back out the same way without causing any damage.

Not a chance. It shut down before I managed to turn it off, so I pulled the battery out and then carefully carried it to my car and drove home, where I have minature screwdrivers and the like.

I took the back off, used J's hairdryer on a low setting to evaporate off any moisture and then put the device in an open book shape on our outdoor table on the back deck, out of direct sunlight - as we're still getting 35C in the afternoon I reckoned that should dry it out nicely.

Given I drink my coffee black and sugarless I thought there was a slim chance of the laptop still being in the land of the living, and certainly when I tried to power it up the fan started and it began to boot, and failed on self test, while blowing the smell of java (what else) out of itself.

Probably that meant there was still some coffee trapped inside somewhere, and probably the next stage would be to take the individual boards out and clean them with isopropyl alchohol and blast the case with compressed air.

Well, I don't have a suitable home workshop, so I took it back into work, said what I'd done, about which they were really nice.

The laptop was an old one - you can pick up the same machine from the various specialist refurbishers for something round about $400 - and a little more for one with an SSD in place of the hard disk - and no data was lost, so off it went to a repair centre to decide if it was fixable.

(We have an insurance based maintenance scheme - so basically if it costs more to fix than replace it goes in the bin).

But this of course left me without a laptop to work on, so I went into BYOD mode with the old Thinkpad I bought last year.

I'd designed the whole documentation methodology to be platform and application agnostic - the only dependency was OneNote for the supporting material, and Windows 7 supports that albeit with a different client interface, just as it supports OneDrive.

Despite having previously suggested Gnumeric as an alternative to Excel I decided to use Libre Office Calc as the windows port of Gnumeric is now  deprecated

Libre Office Calc opened the data spreadsheets without difficulty, and easily handled the day to day documentation created in Markdown with CodeWriter.

The net result was a seamless changeover, in fact more seamless that I hoped - everything just works and the Thinkpad as a slightly nicer keyboard to type on.

So, while I didn't mean to, I think I've demonstrated that designing data collection protocols focused on some standard formats, rather on a particular set of tools allows simple migration to other tools, and potentially other platforms, although the dependence on OneNote and OneDrive is a constraint here - but perhaps only to the ideologically pure, OS X and both Windows 7 and 10 support One Note and One Drive, so in practical terms it's not a problem ...

Friday, 22 February 2019

We have internet again ...

Now I've already written elsewhere about our internet troubles, but I'm glad to say we're all fixed now.

It did mean that for a couple of days while we waited for the NBN man we were dependent on our 4g modem, but the speeds were quite usable as long as we didn't stream any content - you could probably run a pop up business on a Telstra 4G modem.

Well, the NBN man turned up when he said he would, listened to my story about the guys working in the street, agreed it was most likely a cable break, pulled out his reflectometer, and voila, there was a break at 33m.

So off he went, looked at the street wiring cabinet, and there it was - our cable was broken off. Quick re kroning job later and we were up and running - time to fix 10 minutes, something I found quite impressive.

Now we were up and running I could back up my last set of data from the project, upload my photos from Norfolk Island, sync things and I was away.

We also got our internet tv service back. And thereby hangs a tale.

Our internet tv box uses an antenna connection for all the free to air channels, which had meant that when the internet died we could still watch them via the box.

Unknown to us it had quietly downloaded a new version of the system before the internet broke, but because we'd never powercycled the box we were running on the previous version.

Now, we'd had a notice from the power company to say that during the week we were aways they would be turning off the power in the street to do some routine work. When we got back we had clocks to reset and so on, but what we didn't realise is that the internet tv box rebooted into automatic configuration mode for the new version of the software, and imediately hung because there was no internet.

Not good design. Ideally it should have a 'carry on without an internet connection mode', to deal with scenarios like this.

Fortunately I'd never disconnected our old hard disk video recorder, which had a separate connection, so we could watch free to air tv via that.

Still, everything's tickety boo now, but the next time I see a cable van in the in the street, I'm taking a picture in case there needs to be some followup ... 

Tuesday, 22 January 2019

Newspaper access solved

I've recently written about my experiences doing family history online, and if you've been following my stuff more generally, you'l also know I use Trove, the NLA's  digital archive quite a lot.

What I don't have access to is UK newspapers, or I didn't until now.

Now I live in rural Victoria, and while I knew the State Library had access to a lot of these online, I thought that you had to visit the State Library itself to use some of the resources. (I remember discussions with database vendors in the early days of CDROM networking where access was restricted to a block of ip addresses - something they called access within a single building and I called frustrating).

Anyway I discovered that the State Library provides networked access to a shedload of resources including the Times and the Irish Times archives, but you need to (a) sign up and (b) prove you are a Victorian resident, usually by showing some ID to the membership team at the State Library, or by a more complex postal procedure.

Well, I was in Melbourne for other reasons, so I made time to go to the State Library and sign up for resource access.

I was so curious to see if it worked, that when I got back to our AirBnB apartment, I tried it using my iPad over our 4G modem - and it just worked!

Having spent a good part of my professional life trying to get these things to work I was quite astounded at just how good the service was (mind you, to get the best you probably need something with a little more poke than an iPad).

The service looks to be provided via Ex Libris, nothing special there, and uses the standard ProQuest databases. And it works.

I'm quietly happy ...

router spam (sort of)

I've written before about our new 4G portable router, including its ability to be used like a pager to receive SMS requests.

Well, a few days ago I turned it on, and it told me I had a new SMS. I assumed it was a warning of a planned service outage, so I clicked on it to see if it was relevant to where I was.

It wasn't.

It was an SMS spam message telling me that there was an inheritance waiting for me and to email a totally unlikely looking email address.

Needless to say I didn't ...

Monday, 7 January 2019

I bought an ipad ...

Until a few weeks ago I was possibly almost unique in the western world for never having laid a finger on an ipad in any sort of serious way.

Sure I'd fondled them in an Apple store, looked at them when people showed me documents and images on them, but I'd never used one or owned one.

Not that I was tablet agnostic - I bought myself an Android tablet in 2011, and while I've been through several since, for a long time they did the job - as a notetaker, for research work in public libraries, and a few other tasks.

While most people use a tablet to surf the web and check their email in bed, I mostly use a Chromebook - principally because it has a keyboard and I can write on it, so my tablet use has gradually declined.

At the same time, I've begun to listen to podcasts more and more, and I've got some reference material in pdf's which is mostly digitised nineteenth century directories (family history folks!) and so on.

And with it's sudden wifi wierdness my Pixi wasn't cutting it anymore.

So I bought myself a refurbished iPad mini. They're reasonably cheap, as a lot of them come out of point of sales devices, and since they've usually spent a large part of their lives in a protective housing, they're usually in pretty good order.

To it, I added the logitech canvas keyboard - they were on discount on amazon, I only paid around fifty bucks for mine, and that's given me a device about the size of an A5 notebook on which I can type, listen to podcasts, and do web based stuff, be it research or fun.

I've yet to use it for serious typing, but I've downloaded both a plain text editor and a copy of pages, and I guess I could always use Google docs if necessary.

What I can report is for scrolling through things such as the 1865 Scottish County directory - all 500 odd pages of it - iBooks certainly does the job better than anything I've found for Android, and of course one gets the Apple niceness - one suspects that some of the people who put the environment together wear ties and nicely pressed chinos - which after linux, windows and android comes as a pleasant change ...