Announcing Streamlio

Lewis Kaneshiro

August 15, 2017
Reading time: 4 minutes

We are excited to announce Streamlio, the world’s leading enterprise-grade, unified, end-to-end real-time solution. Founded by co-creators of best-of-breed open-source technologies proven at Twitter and Yahoo, Streamlio unifies Apache Pulsar (incubating), Heron, and Apache BookKeeper to deliver the next generation of real-time applications. Enterprises across industries now need to adopt event-driven architecture, characterized by a new era of real-time action with no humans in the loop. That’s why we have founded Streamlio. Our mission is to deliver an easy-to-use, enterprise-grade, real-time solution to the masses.

We are thrilled to partner with John Vrionis and Sudip Chakrabarti from Lightspeed Venture Partners for our Series A $7.5 million financing to help accelerate our goal of delivering our innovative solution to the world’s leading enterprises.

So how did we come together as a team? When I teamed up with Karthik Ramasamy—then at Twitter and now a co-founder at Streamlio—to teach “Real-Time Analytics” at Udacity in 2014, we were excited to discover a shared belief: that the future of enterprise is all about real-time. Prior to starting Streamlio, my background was in finance. Working at Goldman Sachs and later at Falcon Investment Management, I saw firsthand how the world’s most sophisticated Hedge Fund investors used real-time data and predictive models to drive decisions. Karthik saw a different but equally business-critical use of real-time data at Twitter. He and our two other co-founders at Streamlio—Sanjeev Kulkarni and Sijie Guo—were busy leading much of the next-generation infrastructure development at Twitter that to this day processes every tweet. Real-time is part of our company DNA.

Our founding team led much of the development of real-time compute infrastructure at Twitter and Yahoo: Karthik and Sanjeev are co-creators of Heron, Sijie was tech lead of Twitter’s Apache DistributedLog and is PMC Chair of Apache BookKeeper, and Matteo Merli is the tech lead of Yahoo’s Apache Pulsar (incubating) and PMC member of Apache BookKeeper.

Accessible tools for enterprise-grade real-time data processing didn’t exist when I left Udacity to become Shazam’s first data scientist in 2015. Some were open source, some proprietary, but nothing was production ready, seamless, durable, end-to-end, or proven at scale for real-time requirements. Like most data scientists, I spent the vast majority of my time trying to solve these problems rather than creating solutions.

In 2015, Twitter announced Heron as its next-generation streaming engine and open sourced the system in 2016. Heron was developed out of the need to address operational and performance limitations of existing open-source and commercial real-time compute systems. After evaluating available options, Heron was developed from the ground up, using operational intelligence uniquely pioneered at Twitter. Heron has been proven in production with demonstrated reliability supporting all real-time compute at Twitter for the last 3+ years.

Enterprise-grade messaging system Apache Pulsar (incubating) was designed and built at Yahoo to address limitations—–including data loss—–of existing open-source and commercial messaging systems, delivering strong durability guarantees, high throughput, low latency, and multi-datacenter geo-replication with proven reliability for over 3 years. Pulsar was open sourced in late 2016 and offers the best-of-breed publish-subscribe (“pub-sub”) system with strong durability guarantees and with numerous advancements contributed back to the open source community.

Apache BookKeeper, with contributions merged from Twitter’s DistributedLog for stream storage, is Streamlio’s enterprise-grade, best-of-breed durability layer, underpinning enterprise services at Twitter and Yahoo for over 4 years. BookKeeper is the proven core stream storage solution designed to support enterprise pub-sub and stateful stream processing with strong durability guarantees.

Our team shares a mission to create the world’s leading real-time solution. We care about solutions at scale and about delivering the foundation for the real-time applications of the future, across all industries. We share a passion for open source and merging contributions back to the community to foster a growing ecosystem. We also care deeply about sharing our expertise broadly and helping others succeed through education, and we dedicate ourselves to developing free resources for online instruction.

We believe that the future of realizing value from data lies in real-time action, starting with analytics, workflows, and processes. A pragmatic investment in a real-time future begins with the foundation that Streamlio provides. Streamlio unifies “STREAM” processing, “ML” for Machine Learning, and “IO” for messaging and storage, optimizing best-of-breed systems Apache Pulsar (incubating), Heron, and Apache BookKeeper, delivered seamlessly on premises, in the cloud, hybrid, and at the edge.

If you’re interested in building next-generation applications, we’d love to talk to you. Visit us at and follow us on Twitter @streamlio.