The Support Gamble

 

keep-calm-the-server-is-down

You spend money to save money

Annual support contracts and updating your End of Life products offer critical assistance in unexpected situations. Period.

Have you ever found yourself reviewing those yearly support contracts, struggling to sign that dotted line? Have you been overruled in your arguments for maintenance and forced to coast along without plans to replace or upgrade your EOL hardware or worse software versions?

I have, and it’s not an experience I’ll repeat.

 

Rolling the dice on support and expired equipment

A few months after my start date I was in the process of migrating to a full virtual shop, the company didn’t have support for several old servers nor their EOL Microsoft Exchange version. It’s a huge risk, one that I wouldn’t wish on anyone. In fact, today I’d quit before I’d let anyone managing the company purse strings walk me out onto this ledge again.

2014officespace15

It was the last week of December and the holidays were upon us. The office was closing early for last minute shopping, friendly cocktails, family gatherings, or some much-needed rest. I, on the other hand, was living the nightmare of a network outage. A “glitch” had just crippled MS Exchange, along with the company’s core database server.

Historically, during the first hour of any outage (after fielding countless help desk calls, calmly instructing your team, meeting with management, assessing the damage, and referring to your DR binder), I have continued to frantically google for help instructions, but this is serious, and the clock is ticking so I keep my head down, assess the magnitude of the outage, estimate downtime and get moving. Oh, but first I’m tasked with broadcasting the company-wide voice message stating there is an outage NO EMAIL, NO DATABASE so STOP trying to log in! –no one wants to attach their voice to this news but I kept the message calm, commanding, and to the point – the old “we will update you with the latest news as it becomes available”.

 

No means No. No support. Period.

ifESSEH

Without any support it’s a bleak picture: no helpline to dial, no magic wand, no soothing voice on the phone offering to take over, remote in, or walk you through recovery. Just me – or possibly that new network vendor that’s been after our business. Now, there’s an idea. I make the call and without the slightest hesitation, they prepare to step in. These situations are what they’ve built their business around, it’s why they succeed and survive. While all eyes are cast in my direction I announce to the management team that help is on the way. The look of relief that flashes across their faces tells me they too realize I’m out of my element. After all, I run the daily stuff.  This is beyond my scope and even they don’t expect me to steer this ship alone.

While awaiting the arrival of my new-found rescue team, I start to decipher if this is going to be a bare metal rebuild; or, fingers crossed, just a software reload. Pull all the power out of that server and start the rebooting – and praying, lots of praying. By the way, where are those install files? We needed to find the files and get organized; pull it together.

The once friendly-now nervous-faces are still “popping by” to check and see how much longer I think it’ll be before they can check email and that ginormous lifeline they call the company database. I’m 3 hours into this and the stress is building at a pace I hope to out-run.

Recovery and the weeks that follow

The disaster recovery process began officially at 3 PM, December 22nd, and although it would only be another 16 hours before employees could be logging in again, the deep dive of recovery would last another 3 weeks.

GIF

Years later, my memory is a bit fuzzy and perhaps it’s best to forget. However, this I do recall: my stomach was a mess, my head was throbbing, I had a sense of constant panic, my Christmas shopping wasn’t completed, the Holiday was here, and the little money we saved on support was about to put me over the edge. I didn’t feel any Holiday spirit and the remaining days of family and friendly gatherings were a blur. All I could think of was…this could happen again at any time to any one of our servers…and again, it would be my responsibility to rebuild and keep the company connected and working. My first goal is now to sign on the dotted line and employ support.

Support Contracts provide added value for your internal IT staff

Typically, IT staff is in the business of connectivity, and break-fix events. Long days of keeping the organization connected: telephones, voice mail, faxes, printers, email, databases, data files, internet, WIFI, moves, adds and changes, and all things attached to a cable or a plug. That’s just the front of the house. The back of the house is the server room, the backups, the APC’s, the firewalls, the gateways, keeping things current, patched, running, secure, and even cooled. The days of paginating a word document, or helping staff use pivot tables in excel are few and far between. It’s all about connectivity. Downtime and loss of connectivity are not an option and not an option you should be willing to back.

There’s no hiding in IT

About 4 months later I’m at a company function, sitting with a large group of employees I’ve never met. Several people mention the time the email and database were down, and how inconvenient it was, they wondered too if the President and CEO knew how much money they lost during the incident. After all, they couldn’t sell a thing during those 2 days without email or the database. Since I worked in the IT department, they wondered too if I was or knew of anyone involved with the event –  I rose to excuse myself and replied simply “No”.

1479.gif

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s