#TidbitTrenches: Production fixes

Why you shouldn’t rush to a fix in production.

Recently we ran into an issue with one of our Production SQL Server boxes. We knew the fix: we’d have to upgrade to SQL Server Enterprise. And quick. My server guy asked me if he needed to block out time THAT DAY so we could get things rolling. It’s a reasonable question, he needs to make time in his schedule to do such tasks. I said no.

WHAT? WHY? Because years of experience (and impulsivity) have taught me you first stop and think.

Some examples of things that I had to think about in this situation:

  • Did I want to upgrade our version along with our edition? (Yes, yes, I did)
  • If yes, what version – the latest? (Probably not)
  • Do I want a new server or is this an in-place upgrade? (New Server)
  • What else does could this affect? (Lots)

Tons of other things to think about, but since this is post is supposed to be a tidbit, we will stop there. Here’s the thing – we could have affected a lot.

Let’s address each of these examples to give us an idea of what can go wrong. First off, our current SQL Server version is in extended support and we are missing out on features I want to implement. These features match with our company’s goals over the next few years, and our next big lift should be more Azure focused than SQL Server version focused. That means I don’t want to install another version 1 year from now. I’m a busy gal and 1 year goes by quickly!

Does that mean I want the latest version? More bang for the buck – right? Brent Ozar’s post Which Version of SQL Server Should You Use? gave me pause. And when I thought about it more, I realized we may have compatibility issues with other apps that interact with our databases on that server. (BTW: checked with one of the vendors and they confirmed they’ve done ZERO testing on the latest SQL Server version and do not know any current clients that are using it.) So I needed to really weigh benefits versus risk on what version we should go to.

What about in-place upgrades? While I’ve done more in-place upgrades than I care to count, occasionally they can cause unexpected issues; I didn’t want to add more variables when I was already tackling an issue fix. Full stop.

And finally – what else could this affect? This is such a fundamentally important question that really I could have just had that as a singular bullet point. How does your data flow? Are apps that are part of the data flow process going to play nice with your changes? What is connecting to it? Do you even know? What is your plan when you get a ton of support calls because things aren’t working that you didn’t even know about? If you’ve never encounter this type of scenario – go take a look at the book: The Phoenix Project. I listened to the audio version years ago and seriously LOL’d at some parts because I felt like I had lived it. I’m not alone in this.

Final thought: Before being the superhero and trying to fix something super fast in Production, stop and think. You’ll probably find a temporarily solution that will hold you over until you’ve tested out any major changes. And that’s your tidbit from the trenches for today.

%d bloggers like this: