Sigma Is For Engineers

I suppose I should begin by confessing some mild heresy. Before I joined Sigma, I didn’t like spreadsheets. As a software engineer, they felt imprecise, brittle, and not built for people like me. I assumed that Sigma’s product wasn’t for me either. 

I was wrong.

It turns out Sigma is for engineers, too. In using the tool, I learned that it’s actually the fastest and most efficient way to explore data—even faster than writing code—and on top of that, the end result of that exploration is more useful. 

Sigma lets you explore data faster

When I say that Sigma makes it faster and easier to explore data, I mean answering questions like:

• What are the top five reasons a customer encounters an error with feature A?
• How long does a request to endpoint X take for customers using feature Y versus feature Z?
• How many customers tried feature B and then stopped using it?

The key thing about all of these questions is that they're one-off explorations where I don't always know what I'm looking for. Most of the work will be figuring out what data we actually have and how I can translate the product-level question I want to answer into a query over that data. 

Like many companies, we've consolidated all of our data into a single cloud data warehouse. This poses both an opportunity and a challenge. On the one hand, it means that I can answer a broader range of question types - which is, obviously, a great thing. On the other hand, it also means that every day there’s more and more data to search through.

For exploratory workflows, iteration speed is everything, and Sigma is built to make it as fast as possible. As you build your query, Sigma continuously shows a live sample of your actual data, so you can instantly check: Did this formula do what I expected? What does the data look like when I filter down to this subset of rows?

Sigma also automatically optimizes SQL queries based on the data warehouse you’re using, limiting the number of rows returned so execution is much faster—especially for large tables—compared to running a naive query. Many core query-building actions, like grouping, filtering, and sorting, are single-click operations in Sigma’s UI, which eliminates entire categories of errors (like misspelled column names, incorrect data types in filters, or grouping by a column that’s not in the select clause).

And if you’re working with JSON data, Sigma is a game changer. Instead of wrestling with SQL syntax, especially when different rows have different JSON keys, you can explore and manipulate JSON directly in the UI. This makes the whole process dramatically faster and easier.

Sigma also has built-in error handling that saves you from writing extra code just to catch mistakes. If a formula is wrong in one column, it won’t break the whole query—Sigma’s compiler catches the issue and replaces it with a clear error message, while the rest of the query runs as expected. This means you can keep working and iterating even if you haven’t solved every issue yet.

Errors also show up directly in the column instead of in a separate log, so you can see them in context and fix them faster without digging through error messages elsewhere.

A Sigma workbook is a better way to share your results

Is it really data analysis if nobody asks you how you got your results? Every time I've mentioned some latency number for part of our system, the immediate response is "Can you share the analysis with me?" or “How did you get that number?” Whenever somebody else mentions usage numbers, I always want to dive in, see what exactly they measured, and explore alternatives.

If you’re working with people who don’t write code, sharing the underlying SQL is pretty much useless. Sigma’s spreadsheet-style interface changes the conversation from “I can’t read this” into “Are you sure this is the correct formula?” Even when sharing with other engineers, Sigma’s UI helps guide your audience through your analysis, whether by visualizing grouping levels or embedding results in context with dynamic text.

No matter who you’re sharing with, Sigma lets them build on your work instead of just looking at a static output. Instead of back-and-forth follow-up questions, they can tweak filters, explore different views, and iterate on the analysis themselves.

Sigma is built on SQL

Of course, sometimes SQL is the right tool for the job, and this is where Sigma’s architecture makes complex things possible. Since everything in Sigma ultimately compiles down to an SQL query that runs in your data warehouse, you can always check your workbook’s query history to see exactly what’s happening under the hood.

If you need to write your own SQL, Sigma’s SQL editor lets you seamlessly integrate custom queries with other Sigma elements. It also plays nice with other tools, since you can export any Sigma element as a SQL view in your data warehouse for continued exploration.

How I use Sigma as an engineer

Here are some real questions I’ve used Sigma to answer recently:

• Performance analysis: How many input table edits commit successfully after spending more than X minutes in the queue? This helped us prioritize improvements to input table commit performance.
• Code cleanup: Is it safe to remove this migration path for a legacy corner case? Since the last request that needed it was months ago, we knew it was safe to delete the code. A week later, I did the same for another cleanup.
• Bug diagnosis: Why did our active monitor workload flag a failure in this Kubernetes cluster? Turns out the monitor was working fine, but there was an issue reporting metrics to our service, which we then fixed.
• Customer experience: Which sales engineers have worked with this customer recently? The customer had been running into an error we had just fixed, so this helped me find the right person to update them.

Most of the time, I can go from question to shareable workbook in minutes—sometimes even before a meeting ends—so we can make data-driven decisions on the spot instead of waiting for a follow-up.

So far I've mostly focused on one-off explorations, but we also rely on Sigma for repeatable analysis that follow similar patterns.

• Our distributed tracing workbook takes a request ID and shows all of the trace spans associated with that request as it made its way through our different services, including things like span start and end times, any errors thrown, and arbitrary metadata tags (did I mention that Sigma makes it super easy to explore JSON?).
• Our input tables workbook pulls up recent interactions and any errors thrown for a given input table ID.
• A separate workbook tracks common input table errors, helping us quickly spot and prioritize fixes—one of these led me to track down the bug I mentioned above.
• We also have workbooks for debugging connections, analyzing workbooks, running end-to-end tests, tracking slow RPCs, and monitoring new releases.

If you want more examples of how we use Sigma internally, my colleague Zalak Trivedi wrote a blog post about how our product managers use Sigma.

Why I won’t go back

I've come a long way from my initial spreadsheet skepticism. I now view Sigma as a super-powered IDE for exploring data. 

Sigma takes care of the small but crucial quality-of-life details I’d expect in a great data IDE—optimizing SQL queries, showing results in real-time, preventing common errors, and allowing incremental progress. This makes iteration much faster and smoother.

Once I have a final result of my analysis, Sigma lets me share an explorable workbook with my coworkers so that they can see exactly what I did and ask their own follow-up questions, even if they don't know how to code. And since it's all SQL under the hood, Sigma can compose with other SQL tools when I really need to do something complex. 

After five years of using Sigma for nearly every major project, going back to a code-only tool would feel like writing code in TextEdit.