This week, we open sourced a critical Frigade internal system called Trench: analytics infrastructure for tracking events, page views, and identifying users. It's powered by ClickHouse and Kafka under the hood.

Why we built Trench

When Frigade was in its early stages, we tracked events using a basic Postgres table. This setup worked initially, but as our user base expanded, the table became our largest, slowest, and most costly database component. It became clear that Postgres wasn’t built for high-scale event tracking, especially at the scale of millions of users.

To solve this, we designed Trench specifically for high-volume event tracking and real-time querying. By building on ClickHouse and Kafka, we’ve created a system that can scale efficiently and sustainably from day one with proven industry standards.

Functionality

With Trench, we’ve built an out-of-the-box, production-ready solution that addresses the common challenges of tracking millions of events. It’s designed for ease of deployment, development and scalability:

• Segment API Compatible: Trench is compliant with the Segment API, supporting Track, Group, and Identify calls.

• Quick Deployment: Go live in minutes with a single Docker image.

• High Throughput: Can process thousands of events per second on a single server.

• Real-Time Querying: Optimized for rapid, efficient data access and analytics.

• Data Connections: Trench supports throttled webhooks to connect data to other destinations.

• Open Source and MIT Licensed: Use Trench for personal or commercial use-cases without any limitations.

Trench is our solution to the challenges of scaling event tracking, and we’re thrilled to share it with other companies facing similar issues. It’s built to scale with you, whether you’re a startup or a large-scale operation.

Check it out yourself

Head over to the Trench Github and website to learn more.