Thursday 17 February 2022

A tweak to the documentation methodology

 When I started on the project to document the contents of Dow's Pharmacy down in Chiltern I needed a documentation methodology - basically a standard procedure for documenting the artefacts.

Basically the procedure was

  • photograph the artefact
  • write a short standard description of the object in an excel spreadsheet and record the filenames of the photographs and what they are (one packet can look very much like another)
  • record the manufacturer name
  • save the spreadsheet and image files in a self documenting file structure (not quite true - each section has a short markdown description listing objects and locations as a sort of finding aid)
  • backup the saved information to a USB drive
Any additional background material was saved in OneNote. The day's work was backed up to OneDrive at the end of the day.

The reason for doing the backup at the end of the day was quite pragmatic - when I started in 2017 we had a rather slow ADSL link, and doing a backup to OneDrive during the day would slow the link to a near unusable state for anyone else.

In fact I often used to back up the data at home where we had a faster connection, even in the ADSL days. Once we had the NBN at home, it was a no brainer - a days work could be backed up in a few minutes. In fact it took longer to check that everything had been uploaded properly than it did to do the upload.

This methodology has proven robust, and has allowed me to work when the power was out, or indeed when the internet was off.

In fact, it's a methodology I'd certainly continue to use.

Recently however, there's been a bit of creep. Now that we have a reasonably fast NBN connection I've been doing a couple of backups to OneDrive during the day, rather than one big backup at close of play.
 
And then my work computer died on me. Not quite true, but the battery is buggered and not worth replacing.

This does mean I could keep on working normally as long as the power was on, but there would always be a risk of losing work (in rural Victoria the power can and does go off unpredictably during stormy weather).

So I changed computers. My new one is faster than the old one, but instead of a 500Mb hard disk, has a 128Mb SSD, which is just a little too small to hold a copy of the data set.

So, the simplest solution seemed to be to save the data directly to OneDrive, and then do backups of the days work to a USB stick - basically at morning tea, lunch, and at the end of the day, or roughly every two hours. There's no afternoon tea backup as I usually finish documenting at around 1500 or 1530, after which time I do a quality check to make sure that everything has been saved correctly and that OneNote has synced itself correctly.

Unlike my previous procedure, there's an assumption that there will be a working network connection and that OneDrive is online, so the procedure is slightly less robust, but in the event of a problem I can always drop back to the previous procedure, and back up the data once everything is back online.

The way OneDrive synchronisation works helps us here - the file is actually saved locally, and then uploaded in the background, meaning that even if the OneDrive upload fails I can still back the local copy up to the USB backup.

Yesterday was the first time I switched to this way of working and it seems to be robust enough. I probably need to test it in a number of situations, but I'm happy so far.


No comments: