One problem when you build something really new is how to describe it, and we almost always do that by comparing it to something else. Airbrx is a really cool data gateway that lets you apply rules for caching and security… which sounds like "cloudfront for sql."
But then someone asks, "So you'd be sitting in front of every query we run?" and we excitedly say, "YES!" Except, in real life, even though we'd love for you to run every single query through our platform, we know caution is better, incremental change is safer…
While we say Airbrx is a switch that suddenly saves you money, you can think of it more like a volume knob – turn it up as much as you want, or turn it down if it feels too loud or too fast….
Here's what actually happens when you start using Airbrx. We give you a unique
hostname for your tenant, something like
horse-stapler-battery.gateway.airbrx.ai then you point it at your
warehouse (e.g. dbc-1234abcd-56ef.cloud.databricks.com).
dbc-1234abcd-56ef.cloud.databricks.com still exists, we don't block it,
nobody's workload has changed; you just have an alternate path now. So that pesky
tableau dashboard that's keeping your warehouse awake 24/7 can use
horse-stapler-battery.gateway.airbrx.ai and get the cached results but
finance can still hit the databricks host directly until they're comfortable they get
the results THEY want through the Airbrx endpoint.
But here's a trick, you see you don't have to set up any caching rules, and Finance could be using databricks URL OR they could use Airbrx and still get live, uncached results. We can set rules by users or groups or all sorts of variables so routing a tool or a group of users through us doesn't mean we change what it does.
Until you write a rule, queries go straight through to your warehouse unchanged and untouched, and we just log it. That gives us the data to tell you what we COULD tune, whether we tune it or not (of course, we can tell you a lot from your query history in databricks, but we can tell you more from inflight traffic).
Your teams can route through Airbrx for a few weeks or months with no caching, no security rules, no behavior change so that we can learn how to best use it.
I mean, let's be honest, big data in big enterprises mean a lot of people with their
own comfort level. Most warehouse tools end up tweaking things for EVERYONE and that
means you need buy-in from everyone… we don't force that path. Finance wants to use
dbc-123… they can still use it. Sales dashboards could be 100% cached
using horse-stapler-battery…
But, when you are ready to actually save money, the rules are as narrow as you want
them; that is, you can turn on cache for specific tables, specific users, or even very
specific queries (why DO we wake up the warehouse to
SELECT CountryName where CountryCode = 'GB'?) Leave Finance's ad-hoc
queries completely alone — that's a rule too, and it takes about a minute to write,
and a second to roll back.
Every one of those rules has its own on/off switch. Flip it off and you're back to plain passthrough. Point the hostname back at the warehouse and you're fully native again in minutes, with nothing to unwind.
So, what we suggest is start with the noisiest tool you've got… it's where the repetition lives, and the repetition is where the money is. You can decide about everything else later — or never.
Are you curious what your own traffic looks like? There's a read-only scan at airbrx.ai that reads your query history and tells you where your repeats are. It only takes a few minutes and you don't have to talk to anybody or create an account or even give us a PAT, of course, you CAN do all of those things too if you want – we want you to have full control over how you use Airbrx tools.
The volume control is at your fingertips.