FlowMesh — Open-Source Event Pipeline Platform

Context & Goals

Self-hostable alternative to Segment + Mixpanel + PagerDuty, deployed with a single Docker Compose command. FlowMesh ingests events, routes them through configurable filter/transform pipelines, delivers to multiple destinations, and alerts on anomalies — fully open source, free forever, with no vendor lock-in.

Constraints & Challenges

Every SaaS product needs event capture, routing, real-time visualisation, and alerting. Today that requires 4–6 paid tools and $2,000–$20,000/month. Segment alone costs thousands per month once you scale past the free tier. Teams in regulated industries cannot send data to third parties at all.

Solution & Approach

Polyglot microservices architecture: NestJS for all data-plane services, Go for the delivery service where concurrent I/O matters most. Two Redis instances with explicit persistence policies — ephemeral for rate limiting and cache, persistent (AOF) for idempotency keys and token blacklist. RabbitMQ over Kafka for simpler ops at the target scale. Every distributed systems pattern is implemented and documented: circuit breaker, exponential backoff, dead letter queue, saga pattern, idempotency, connection pooling.

Phase 1 ingestion service is production-ready with ~98% test coverage — schema validation, correlation ID tracking, idempotent deduplication via Redis, and RabbitMQ-backed event queue. Full pipeline (ingestion → transform → delivery → destinations) is in active development. Docker Compose deployment runs on an $11/month VPS with one command.

Technology Stack

Results & Impact

• →Replaces Segment + Mixpanel + PagerDuty with a single self-hosted deployment
• →Ingestion service complete: schema validation, idempotent deduplication, RabbitMQ queue (~98% test coverage)
• →10 distributed systems patterns implemented: rate limiting, idempotency, circuit breaker, DLQ, saga, and more
• →Polyglot architecture: NestJS for data-plane services, Go for I/O-intensive delivery
• →Open source, MIT licensed — no feature flags, no event limits, no vendor lock-in