Hello world! from Stairwell engineering
Stairwell was born with the ambition to rewrite the rules of cybersecurity. While there are many challenges ahead of us on the path to realizing that vision, we have made giant strides with our recently introduced Stairwell platform. This is possible due to the hard work of a small but stellar engineering cast that includes a diverse mix of ex-Google/Microsoft employees as well ex-startupers. I recently joined Stairwell and have been humbled by this kick-ass team that knows how to have fun tackling some of the hardest challenges in the fields of computer science, distributed systems, and cybersecurity.
We are starting this Stairwell eng blog series to share our challenges, accomplishments, and learnings in areas ranging from technology to remote-first culture. This first one is a brief intro to the team, our habits, cultural quirks, and the technical challenges we’ve taken on. In later posts, we plan to dissect particular topics in detail.
A common perception is that engineering at Stairwell requires a security background; that you must know how to disassemble malware on esoteric architectures or be able to recite the header fields of BGP packets. That’s not the case. Instead, the common threads that bind us are: a penchant for solving difficult problems in large-scale distributed systems and building products that clearly show value to our users.
What excites us are our technical challenges. They stem from Stairwell’s promise to pre-preserve evidence for attacks (past or present). Stairwell constantly ingests new data and looks for artifacts. It figures out patterns that are unique, novel, or unexpected by developing insights based on an organization’s unique footprints. Our goal is to eliminate an attacker’s ability to evade detection and make this process mind-bogglingly fast. We’re elevating enterprises into an empowered position with truly unique context and awareness.
To build all this, we have chosen to go with a modern tech stack (Go, Kubernetes, Terraform) and build on public cloud. We are especially proud of our internal development that has continuous integration tests (i.e., Test Automation Platform (TAP)) constantly running. And we continuously deploy to production every time a change goes in and tests pass! No manual deploys at all. We believe in thoroughly testing and automating everything.
The requirements for handling very large-scale data (think multi-PBs) while maintaining high rates of processing throughput make our technical challenges intriguing. Using cloud-provided building blocks makes it relatively easy to start, but when you add requirements for scale and speed, the solutions quickly become very costly to operate and maintain. We want to use our engineering talents to craft efficient, purpose-built tools and systems necessary to solve these problems with elegance and scalability in mind. Our goal is to achieve 10X our scale and speed while reducing operating costs and satisfying our SLO/SLAs.
Our main data pattern involves very high rates of ingesting mostly immutable data, yet we need read-after-write and full transaction support. A number of database systems provide read-after-write and scale, but are not built to take advantage of mostly immutable data. While we are currently built around Postgres, we are evaluating other alternatives and asking ourselves what core capabilities we need to efficiently address our specific needs.
Another area of challenge for us is streaming. We have started with pubsub/SQS that is commonly used in distributed systems. Our data volumes are large — think multi-TBs/day — and we want the ability to do distributed aggregations easily within the streaming data (which might have duplicates) and to do so cost effectively (i.e., no new dbs etc involved). One concept we have discussed internally is related to published research about Photon from Google. We are building towards something like that, but with constructs that will work outside of Google’s walled garden.
We are working through these challenges as a distributed-first company with engineers across the US. But we play hard too. Trivia, fitness, DND games, and art are some shared interests. High standards in our eng practices and quality make us proud. Stairwell’s mission inspires us: because the world will be a better place for many if we succeed.
Curious? See our posted openings or reach out to us at [email protected].
