Startups Using Kafka in Boston
Via their job posts and information submitted by startups themselves, these are the Boston Kafka startups we've found.
Interested in other technologies? Browse or search all of the built-in-boston tech stacks we've curated.
In-development platform by former Adelphic Mobile cofounder Jennifer Lum, aiming to turn “unstructured data into streams of structured information immediately usable by intelligent machines.”
Tech Stack Highlights
Apache Spark – We use Spark, Spark Streaming, and the Apache Kafka frameworks for fast in-memory compute, real-time streaming, and lambda architecture. These technologies power our cyber threat detection, remediation and visualization software.
Cassandra – Our platform relies on Apache Cassandra NoSQL database for long-term data analytics and reporting. We use Elasticsearch for real-time search and analysis and Redis for in-memory cache.
Docker – We’re built on a Docker container micro-services architecture and Ansible DevOps orchestration framework for flexible bare-metal, virtual machine & cloud deployments.
Angular.js – We use the Angular front-end framework with D3.js, and NodeJS on the backend.
Tech Stack Highlights
Spring Boot – We field a number of microservices on top of Spring Boot. Its convention-over-configuration design allows us to focus on business logic rather than plumbing. We’re particularly looking forward to the Spring team’s upcoming first-class support for Kotlin, which we’ve been gradually introducing as a safe, expressive alternative to Java 8.
React + Redux – We’ve built a highly interactive and engaging front-end using React and Redux. The resulting code is modular, easy to reason about, flexible, and composable.
Kafka – We use Kafka as our primary message bus. Unlike most “big data” technologies, Kafka has allowed us to scale without imposing a notable increase in complexity. In fact, becuase its append-only architecture allows us to view topic contents long after the message has been “consumed”, Kafka allows us to significantly improve monitoring and visibility over more traditional message buses (JMS, AMQP). We’re looking forward to experimenting with Kafka Streams as a lightweight alternative to standalone stream processing frameworks such as Spark.
Zeppelin – We use Apache Zeppelin to query, aggregate, and visualize data across a number of heterogeneous data sources, including MySQL, ElasticSearch, and S3. We write ‘notebooks’ in Scala and SQL to drive Spark in creating these visualizations. These notebooks can be ad hoc or shared, versioned, and parameterized.
NiFi – We use NiFi as an orchestration layer to manage real-time data flows in a simple scaleable way. The framework provides us with the ability to easily monitor the progress of messages as they move through the processing pipeline and to replay messages should it be necessary.