Running Kafka with a blindfold?
Maintaining Apache Kafka in production is hard. It seemed so easy at first but they you start having lots of questions that are difficult to answer:
“Are you sure the service is up?” , “How are the partitions well distributed?”, “We’re seeing performance problems on the application, can you check Kafka”, “What message format do we have on this topic?”, “Are all the replicas in sync”?
Why do you need monitoring for Kafka?
If Kafka lives up to its potential, it’s going to become one of the most critical components of your organisation. Like any distributed application, it’s going to require a deep level of instrumentation and visibility.
Without visibility, ops teams find it hard to keep up with adoption and understanding dependency on upstream and downstream applications. Furthermore, you’ll find it difficult to tune performance and plan for capacity. And of course, you’ll be completely blind when you have a serious incident.
When is the time right to have this monitoring?
You could wait for people to be constantly on your back. Or a major outage, maybe that’s why you’re here?
Or maybe you want to be proactive:
You’re thinking of the additional benefits of monitoring
You want to make your users happy
You want to see improved performance and better plan capacity
In all, these would lead to a higher chance of projects being delivered successfully and on time.
Would this make things better for you?
Monitor your Kafka Streams in Realtime
Explore your Kafka topics with SQL
Simplify getting data in and out of Kafka
Manage your data governance policies
Who else has this helped?
We speak to organisations every day of all sizes across Europe and it’s crazy how they all say the same things. Some clients who have benefited from solutions we work with propose include:
Generali, Zopa, CISION, Accenture, Sainsburys, Autoliv
Dave is chief of business relationships and co-founder. Worked with data technologies such as multivariate testing, personalisation and big data analytics. His aim is to do business with integrity, putting customers, and colleagues first without compromising on results. He’s based in London with his family and data driven dog called Spike.
Guillaume is our technical officer, looks after our marketing and is our chief French-speaker. He has had a varied career from everything including webdev, to working on a support team for enterprise software, to software consulting and presales. He can often be found in parks and canals in Paris recording Youtube demos.