DM Blog

When Backups Fail…


it’s been over two weeks since my last article, not because I’ve lost interest in blogging (not at all), but because I’ve been super busy with a surge of requests for arrangements lately. I don’t know why it is, but it seems that whenever I get an order for charts, I always get at least three more orders within a day or two. And each client will usually order two or three arrangements at a time from me… Sometimes I wish the requests would come more evenly spread out and not all at once, but on the other hand, it’s also nice to be able to focus on only one thing at a time and then switch to something else for a while – it keeps me interested, focused and lets me get very deep into something while preventing long-term mental fatigue.

Of course, the disadvantage of having all these orders come in at the same time is the pressure it puts on me to meet the deadlines for delivery. Since I require a full payment to be made up-front before I even start working, I don’t think people should have to wait so long to get the arrangements in their hands. I always under-promise and over-deliver, but I also work myself very hard – just because I have a lot of other orders in the queue doesn’t mean I’m promising far-future delivery dates for new requests that come in. I suppose I should get a little better at how I manage this so that I’m not working myself day and night, even if it is for only a two week period at a time…

Two weeks ago, the last of the six arrangements (that were "due" that week) that I was working on was "Han Cogido La Cosa" by Grupo Niche. I also had another five arrangements in the queue that I needed to finish the following week (the first week of April) and orders were already starting to pile up for this week too (the second week of April). So there really couldn’t have been a worse time to have something go wrong with my computer.

MacBook Pro

I love my MacBook Pro – I can’t imagine using anything else and I’m on it all the time. it’s on and in use for more than twelve hours a day, seven days a week, but probably closer to sixteen hours a day. When I’m not working on music, web stuff or editing photos, I’m probably watching a movie on it with Michelle. But I’ve had things go wrong with my older MacBook Pros (and PowerBooks) in the past, so I always make sure to keep backups of everything, just in case. Always.

For most things, I just let Time Machine backup to my Time Capsule every hour, but whenever I’m working in Finale, I always have two additional backups while I’m working. Finale has an option that allows you to save a backup of your file to a separate folder every time you save (I hit Command+S almost obsessively, by habit) and another setting to "autosave" a file to a separate folder at a specified interval. I have mine set to autosave every five minutes, so that’s a total of three backups – my main Finale project file, a backup file in another folder, an autosave file in yet another folder, and then my Time Machine backup every hour. Better safe than sorry, right?

When I first started transcribing and arranging charts, I’d do a lot of the transcribing work with paper and a pencil and then enter the notes into Finale, but I’ve gotten so quick at using Finale that it actually takes me longer to do it that way now, so I’ve started using Finale right off the bat. (Actually, I’ve gotten so quick at using Finale that it’s much quicker for me to create even a simple lead sheet in Finale than it is to use manuscript; and my handwriting is terrible anyways.) Of course, this means that if something goes wrong in Finale (or on my computer) while I’m working, I now have nothing on paper that I can go back to, so that’s why I’m so obsessive about having all these backups. (I do however keep paper copies of all my arrangements – scores and parts – once they’re done. it’s a lot of printing, but worth it.) You can probably figure out where this story is going…

So two weeks ago, I was creating a modified arrangement of "Han Cogido La Cosa" and all was going well, completely normal. I finished the transcribing portion, finished the modifications I needed to make to the arrangement (re-orchestrating it for different horn parts), finished entering articulations, dynamics, rehearsal markings, etc.; basically, I was completely done and even finished laying out/formatting the parts and score pages. I had been working on it since twelve noon and it was 3am when I was finally ready to start saving everything as a PDF and send it off to my client. Oh, that blissful feeling of completing a project!

So just as I started to save everything as a PDF, the application froze at the Finder prompt where it asks you to select which folder to save the PDFs into. I waited to see if it would clear after a while but it wouldn’t budge, so I force-quit the application and thought I’d try re-opening it. (Sometimes Finale has attitude, it’s "special" that way…) I had just saved the file seconds before so I wasn’t worried about losing anything, plus I knew I had many backups anyways. When I re-opened Finale my file opened up fine, but the application froze again at the same Finder prompt, (asking me where to save the PDFs). My computer had been slow and acting up a little in the last hour or so, (even spotlight was taking forever to open anything and it wasn’t indexing anything either), so I thought I should close everything, restart my computer and try creating the PDFs again.

I restarted my MacBook Pro, got up to put some stuff away and got ready for bed while it restarted. For more than five minutes, it was stuck on this grey screen and wouldn’t completely shut down so I held down the power button to shut it off and then pressed it again to turn it back on. It rebooted normally after that so I went to open my Finale file, but Finale kept giving me an error (-39) and a message saying something about not being able to open the file. I had a look at the file and noticed it was last modified at 3:08am but was showing a size of 0KB. No big deal, right? OK, so the file was probably corrupt, but that’s why I have backups in separate folders. Only, when I went to check the backup folder, that file was also showing 0KB. The file in the autosaves folder? Also 0KB. Yikes. At least I’d have my Time Machine backup though, and at most it would only be an hour old, right? Well, when I went to check my Time Machine backups I discovered that for some weird, unknown reason, Time Machine hadn’t made any backups of my computer since 2pm the day before. So if I was going to get anything back from my Time Machine backup, it was only going to be two out of fifteen hours of work!

How could it be that I had not one, but three backups, and all three failed me? I stayed up for two more hours trying to see if there was any way to get my file back (I even checked system folders and Finale’s temp folder) but had no luck. It was now five in the morning, I was tired and sleepy, and I realized that I’d have to get up the next day and start over again from the very beginning. Listening, transcribing, everything all over again…

When I got up three hours later, I started thinking – "What if I start over and work on this for hours, only to have the same thing happen to me again?" How could I be sure that it wouldn’t happen again? Well, I couldn’t.

After getting off the phone with AppleCare, they determined that since I had a similar issue back in November, it was likely that my hard drive was no good (even though it was passing tests left and right) and told me to take my computer to the apple store to have the drive replaced. I knew it would likely be a couple days before They’d get a replacement drive in and that didn’t help my present situation of having all these charts to finish, (and now behind schedule as a result of losing this file). I thought to buy another external drive so I could transfer everything, (including the OS), to that drive and run everything off of that in the meantime. I was told it was likely that there was a bad sector on my old drive, in which case, attempting to save anything to that disk was indeed risky, and they confirmed that my idea of running everything off an external drive was a great way around that.

So that’s what I’ve been doing, and even now as I write this, I’m running everything off an external firewire drive. (it’s cool that you can do that, no?) Apple was very quick at replacing my drive – it only took three days for the drive to come in, (over a weekend), and the replacement was done in less than fifteen minutes while I waited. They also suggested that I don’t transfer everything back to the new drive since some of the other files, including those used by the OS, could have also been corrupted by the bad drive. Makes sense. So when I get the time, I’ll be starting fresh from a new installation of Lion and selectively transferring files over as I need them. I’ve already installed Lion on the new drive, but I haven’t had the time to set up anything else so that’s why I’m still running everything off the external drive. I’m kind of looking forward to starting fresh, but not looking forward to re-installing all of my applications, having to set my preferences and settings in each one, and worst of all, customizing Apache and PHP all over again…

I’ve now had my hard drive, bottom case, top case, video card and logic board all replaced on this computer (covered by AppleCare) – pretty much everything except for the DVD drive and the display. I can’t imagine not buying AppleCare, ever. Completely worth it!

Of course, I did finish that arrangement of "Han Cogido La Cosa", again, and it wasn’t fun having to do the exact same work twice, especially the listening/transcribing part. But the worst thing about this situation was knowing that this arrangement was for a brand new client who had never ordered charts from me before – this was the first time – and he was a little reluctant to order at first, too. I was worried that the delay would be enough reason for him to never order from me again (almost all my clients become repeat customers) but it was nice to get an email response saying: "The charts look great. Worth waiting for." Crisis averted? Maybe.

The irony of this whole story is that I was so obsessive about keeping backups that I even had three backups going on at the same time, but in the end, none of them helped. Since then, it occurred to me that I should change one thing about how I work in Finale – sure, I had my main file plus two backups going, but even though they were being saved to separate folders, they were still on the same physical drive. So now, I’ve moved all my Finale projects over to my external "music" drive (which I was using only for Logic and Pro Tools) and I’ve kept the autosaves and backup folder on the main hard drive. At least now I have three different drives in the mix – the main hard drive, an external drive, and the Time Capsule drive – but against all reason, I’m still reluctant to believe that even that would have prevented this from happening…

Actually, the big let down was Time Machine – at least the other backups were there, even if they too were corrupt (which makes sense if they were all being saved to the same bad sector on the drive), but Time Machine didn’t even try to back anything up for more than fourteen hours. No warning, no error message, nothing at all to even give me an opportunity to look into it and fix it. I still don’t know why and I probably never will.

I was playing catch-up for most of last week trying to get back on track, but I’m finally all caught up now. Not finished, just caught up. And I took the Easter weekend off to spend time with Michelle and our three dogs, (and felt guilty about not working on charts), so I better get back to work!