The up-shift

Lately, I haven’t felt much in control of my own professional destiny. As easy as it would be to point fingers, ultimately, it’s on me. In my time as a Senior DBA, I’ve accomplished a lot: migrations, upgrades, automation (lots of automation), and all sorts of other projects. I’ve found a large sense of purpose in my work and it’s helped me when other parts of my life weren’t so great, and to top it all off I work with and for the best people.

Now, though, the projects aren’t quite there any more. We’ve been “steady state” with our SQL Server footprint for a while now, and with no major upgrades in the near future, and most of the daily stuff automated, I’ve had to go looking for work to do. Sometimes it’s there, sometimes it isn’t. Meanwhile, it’s not like the rest of the company’s projects fall on the wayside. So to stay relevant, it’s time to step outside my daily role and do something different, and even a little scary.

It’s not goodbye to SQL Server…

No, I’m not changing careers. Instead, I’m taking my talents to a new, major project in our IT shop. We’re looking to change our entire process around predictive analytics for some core business processes, and I’ll be joining a team that’s responsible for it’s implementation.

What am I going to be working with? Quite a lot, it turns out:

• I’ll be working Azure Data Lake to aggregate and store lots of structured and unstructured data from a variety of sources.
• Azure Databricks will be our interface to things like Spark jobs and notebook integration for analysis, so I’ll need to learn all I can about that.
• I’ll also help bring advanced ML models in Python. While I’ve messed around a little with this already (with curious results), this will be my first “production” implementation of something like this.
• Deploying trained models with the use of containers. This part still both excites and confuses me but hey, what kind of blog post wouldn’t be complete without mentioning Docker or Kubernetes?

And I know next to nothing about most of those things, except maybe a passing knowledge of Python. I’m equal parts excited and scared.

Acting like I’ve been there before

Rather than jumping in head first, though, I’ve got some more immediate things to work on for this team. Most notably? Automation, especially around the Azure infrastructure and Databricks API. From the jump, I’m going to be working on bringing some PowerShell to bear on the project, creating a series of functions and modules to support our overall pipeline. And since I’ve been working on the DevOps team for a couple years now, I can see how all the pieces fit together, and how we’ll ultimately leverage the tools we have for our existing release pipeline.

Oh and to top it all off? I’ll continue to leverage SQL Server a lot. Remember how I said I’ll be helping move a lot of data around from various places? Our existing data plays a large part in this project too, so I’ll be looking at some pretty wild stuff to help move things around both in batch and in real-time (which is a teaser, I think I have a pretty rad solution for that, but that’s for another blog post).

Pedal on the floor

So that’s what I have going on. In a way, it’s equal parts exciting (it’s a new direction), scary (I’m well outside my comfort zone on a lot of this technology), and bittersweet (there is something to be said for stepping away, even if it’s temporary, from something you put so much time, effort, and love into). Without this change though, I could have become just a little *too* comfortable. I want to keep pushing myself and I’m glad I get to work on stuff like this, and that the leaders I work with see me as someone who can bring value.

That also means I’m not slowing down on my speaking, online teaching, and blogging either. Especially with the latter, as I continue to learn and share some of the things I’ve started to implement. As always, stay tuned… and thanks for reading!