For the past few weeks, we’ve been beating the Postgres drum. In this edition of Collapsing the Modern Data Stack, we’re not putting the sticks down completely…but we’ll be changing the rhythm a bit. We’re going to jump into the frontend (UI/UX) vs. data platform (backend) fray. And I’ll argue that a beautiful data platform is actually more valuable than a beautiful frontend, even though I’m certainly on the record of talking about how important UI/UX is to the success of a digital product. My key point is that data tooling vendors are making your backend a mess.

In fact, let’s put a finer point on it. I say vendors have more influence over the state of the average company’s backend infrastructure than their CTO does.

Want to test the theory? Ask any CTO to draw out how data works in their company. You won’t find a single one that’s truly proud of what they draw. They know it’s a mess, they know it should be better, and they also know they’re fighting an uphill battle to do anything about it. 

So how does this happen in the first place? A couple of factors—one, most people think about applications first. The database comes as a bit of an afterthought, because data infrastructure is a hidden layer under the application’s UX. You can’t hide a bad user interface, so that’s what gets attention. After all, customers can’t tell if you have an elegant data backend or not - so why bother keeping it tidy? Attention is elsewhere, which opens the door for data tooling vendors with great products. Sales promises become “strategic” design decisions, and dev team experiments drive the way those decisions evolve. That combination tends to determine what your data management structure looks like…not any priority for order.

Here’s why this matters: your company’s agility suffers when your data platform is messy. Let’s say you’re having some knee pain, so you go see a specialist. Now, imagine if you had two specialists you had to see every time your knee hurt. It would be harder to make decisions, or even decide who to talk to. (Especially if it was every specialist’s job to get you to see even more specialists!) You see the problem: for every additional specialist, you become more confused and incapable of acting quickly and decisively. In the case of your company, that means dealing with the inefficiencies of tool bloat.

Consider a company like Vanguard - whose success is predicated on providing up-to-date, accurate investment recommendations to their customers. They simply cannot have bad data under the hood. So they are definitely not allowing multiple “specialists to be a part of their medical plan.” Their data is mission-critical, so they almost assuredly have a pristine data backend. Their strategy is obvious and universally applicable: if you’re trying to build great products, you need great infrastructure, which determines how well your company can execute.

“Fine, but this doesn’t help me. My company has been around for years, and our data platform is super messy - how do I clean it up?” Great question, and I’m sorry to say that there is no easy fix. It’s going to require a lot of pain and agony. Once your data platform sprawls, it’s really hard to unsprawl (see CMDS 002: The Frankenstack). Trying to clean it up will probably require bringing on new technology, which in the short term is more sprawl! And you may need to get rid of people who advocate for the old technology.. PLUS, legacy systems are just really sticky in general. Databases don’t get migrated easily or often. But don’t despair - over a long period of time, you can clean things up. It's just not going to be an easy process. It will take years, even decades. 

So how do you keep your backend from turning into a gnarled mess in the first place?

Be extremely picky about your vendors. The number one way that I can tell that a company is really good is when they have tough procurement. When Astronomer was trying to get into Apple, we had three different POCs and we were still stuck in procurement. We eventually got enough pressure from various directions to push through; it was incredibly challenging. Apple’s procurement team makes companies show all their work, and that’s part of what makes them a great company (and keeps their vendors to a minimum). 

Paradoxically, as a vendor, I am OK with the fact that future procurement teams fight Tembo hard, too. That’s because I really trust in what we’re building, and I believe in our value. We’ll give people lots of different use cases for their data, all under one open source platform (Postgres), and under one vendor (Tembo)...which will allow for elegance and efficiency on the backend. 

I’m dedicating my career to building beautiful data platform backends across all data use cases, because a beautiful backend is more strategically valuable than a beautiful frontend. I want CTOs to be proud of their company’s data platform instead of ashamed.

Thanks for following along so far. We’re taking a break for the holidays until the new year, but we’ll be back with more musings and stories in 2024.

Ry’s Weekly Resources

Want to take things further?

Tembo is the Postgres developer platform for building every data service. Our mission is to collapse the database sprawl of the modern data stack with a unified developer platform. With Tembo, developers can quickly create and deploy specialized data services using Stacks, pre-built Postgres configurations, and deployments without complex builds or additional data teams.

Join our private beta waitlist here