← All work
Event PipelineServer-Side TrackingNode.jsPHP / Laravel

Built a server-side tracking pipeline delivering up to ~1M events/day to 5 ad and analytics platforms

Role
Backend Engineer
When
Nov-Dec 2020
Read
4 min read
Impact
~1M

Built with PHP / Laravel · Node.js · Socket.IO · MongoDB · Redis / Horizon · Elasticsearch

dietsquiz · socketpaymentscommerce · HTTPws-middlewareSocket.IO + HTTPeventsdedup · status 90trackingHorizon · 1/mintrackwritepoll 90MetaConversions APITwitter / XConversions APIMailchimpMarketing APIElasticsearchanalytics sinkMauticautomation
diets (socket) + payments (HTTP) → ws-middleware (dedup, status 90) → tracking → 5 platforms

Overview

I built the server-side event-tracking pipeline behind a set of high-traffic diet and fitness quiz funnels. As a user moves through the funnel the app streams events over a Socket.IO channel, while a payments service feeds in commerce events over an authenticated HTTP path; a middleware service records everything, and a separate dispatcher fans each event out, server-to-server, to a set of ad and analytics platforms.

It spans three decoupled services: a Laravel quiz app (the source), a Node.js and Socket.IO recorder (ws-middleware), and a Laravel and Horizon dispatcher (tracking), with MongoDB as the shared event store.

The problem

Conversion tracking lived in the browser, where ad-blockers, Safari's ITP, and iOS App Tracking Transparency quietly drop a large share of client-side pixel events. Ad platforms then optimize on incomplete data, so the business pays more for worse targeting. I needed reliable conversion signals sent from our own infrastructure, not the user's browser, without coupling the funnel app to any single ad platform.

What I built

The pipeline is three fully decoupled stages. The quiz app emits each event over a Socket.IO track channel, enriched at send-time with the click identifiers ad platforms need for attribution (Meta's fbp/fbc, Twitter's twclid). A stateless recorder deduplicates every event by an MD5 content checksum and writes it to a shared MongoDB events collection with a ready-to-track status flag. It forwards nothing: the durable store is the queue.

The dispatcher pulls ready events every minute, chunks them, fans each one out to the platforms, and marks them tracked. Platforms sit behind a small Strategy interface (add one = implement track() and register it), each with its own transformer and a config-driven matrix of ~50 accounts routed by domain regex, so one event can reach every matching brand pixel. Each platform hashes PII to its own spec (SDK-side for Meta, SHA-256 for Twitter, MD5 for Mailchimp), and the client event id doubles as the vendor conversion id so a server event never double-counts a surviving browser pixel.

tracking · Laravel
// TrackAllEvents.php: pull ready events, dispatch, mark done
$this->events
->readyToTrack() // status == 90
->chunk(500, function ($events) {
$this->trackingService->run($events); // fan out
foreach ($events as $event) {
$event->setStatus(Status::ALREADY_TRACKED)->save();
}
});
// Add a platform = implement one interface
interface TrackingPlatformInterface {
public static function platform(): string;
public static function events(): array;
public function track(Collection $events, string $account): void;
}
Real code · TrackAllEvents.php + TrackingPlatformInterface.php

The results

At peak season (January, when diet and fitness traffic spikes) the pipeline processed on the order of ~1M events per day. One event stream reaches five ad and analytics platforms; conversion signals survive ad-blockers and iOS privacy because they are sent server-side; events are deduplicated end to end (checksum at ingest, conversion id at dispatch); and the three stages are fully decoupled, so each scales independently and a slow platform never blocks ingestion.

~1Mevents/day at peak (Jan diet season)5platforms, one event stream21event types trackedSurvives ad-blockers & ITPDeduplicated end to endProducer / consumer decoupled
Throughput at peak season and what shipped