A journey through the Foundry: Becoming an analytics engineer at dbt Labs

Data is an industry of sidesteppers. Most folks in the field stumble into it, look around, and if they like what they see, they’ll build a career here. This is particularly true in the analytics engineering space. Every AE I’ve talked to had envisioned themselves doing something different before finding this work in a moment of serendipity. This raises the question, how can someone become an analytics engineer intentionally? This is the question dbt Labs’ Foundry Program aims to address.

About the Foundry

The Foundry Program is an apprenticeship designed to turn data newbies into fully-fledged analytics engineers over the course of six months. As one of the inaugural foundry apprentices, I’m here to share my journey into analytics engineering along with the takeaways I picked up along the way.

We’re continuing to improve the program with each iteration but the curriculum for my cohort was split into two parts—three months of training followed by three months of hands-on work.

Where I started

Before diving into the foundry experience, I’d like to tell you a bit about my background before dbt Labs. In my previous job, I had done some very basic work with data in Excel. Prior to dbt, I had also done a data science bootcamp. The first time I heard about analytics engineering was when I saw a post about the foundry program in Code for Philadelphia’s Slack channel. Even as someone who didn’t understand what analytics engineering was, I was struck by dbt Labs’ strong opinions about analytics and data: there was a vision towards the future informed by lessons from the past (i.e. reflecting on the history of software engineering). There was a desire to optimize the way data was done, transparency on the plan to get there, and where better to get my feet wet than a company committed to doing analytics in the best way possible?

The Foundry journey

Ramping up

My first couple weeks at dbt Labs was a whirlwind of information and discovery. Week two was when I began to understand what analytics engineering really meant: the organization of data. There was a lot to love about it; there was a promise of both the technical and the creative (the code and the problem solving). As someone who loves organizing, analytics engineering was a natural fit. It came with a KonMari zen.

I had originally focused my job search on data analytics, but for me, analytics engineering was a much better fit. It felt less like reaching around for a lightbulb moment and more like building a library I can take pride in.

As my knowledge of the “what” and “why” behind analytics engineering was growing, I started learning the “how”. SQL, Jinja, best practices, and the in’s and out’s of working in a dbt project. The best part was applying my knowledge to exercises. I remember going through my refactored code from an exercise with Dave Connors, my foundry mentor (shout out to Dave! He was a huge help during my apprenticeship). Going through my modeling and the different ways an AE could refactor the code showed me the creative problem solving that this job requires. Often there’s a clear best path in the code. But sometimes there isn’t. Playing with those trade offs made me feel like a kid in a candy store.

Along the way, I was able to utilize some great resources. My apprenticeship was on our professional services team, who excelled not only in dbt work but in conveying an understanding of dbt and the analytics engineering way of thinking to our clients. Within our team as well as the larger dbt community, there was a culture of sharing perspectives, sharing solutions, and growing as a space. We have guides on analytics engineering, articles on the MDS ecosystem, and a number of robust writers sharing their latest takes on the field. These are invaluable resources for hopeful analytics engineers.

Of course, I can’t talk about the Community without mentioning Coalesce. My first Coalesce came on the heels of the training section of my apprenticeship, right before I dove into real hands-on consulting work. It was amazing to see so many folks excited and engaged in analytics engineering. Talks ranged from getting-your-hands-dirty technical problems to reflections on the broader industry. Coalesce 2021 reaffirmed for me that the real magic of this field wasn’t dbt, but the Community that had coalesced around it.

On the ground

And then it was time for the real work. I was paired on projects with more senior team members. The need to prove myself gave way to some imposter syndrome. Was I ready? Had I learned enough and was I capable of applying the knowledge when it really came down to it? As is often the case when you shift from an academic application to a practical one, I found that there were challenges I hadn’t anticipated.

The first project I worked on was a solutions review on a client’s project, where we review the project and suggest where it can be improved, as well as highlighting where it shines. I was armed with dbt Labs’ best practices, but when I first opened up a DAG of over 200 models, I was overwhelmed and didn’t know where to start. That’s when I learned that context gathering (like going through the DAG and project before diving into the work) is a very important part of the job! In the long term, the contributions I made to those initial client engagements were the first step in growing my confidence.

Post-Foundry

Once the foundry wrapped up, I was offered a permanent position on the professional service’s team! I continue to benefit from the knowledge loop, but now I’m also able to contribute to it. I’ve worked on more dbt projects. I’ve made package contributions. I’ve gone from being a starry-eyed Coalesce attendee to being a starry-eyed Coalesce attendee and co-facilitating workshops at Coalesce. Over a year later, I can happily say that the Foundry program brought me where I wanted to be.

If you’re looking for resources to help a hopeful analytics engineer (whether you are one or a manager of one), feel free to reach out to me on the community Slack (@Wasila)!