Skip to main content

dunnhumby Retail Analytics Data Science Platform

Problem Space

Deliver a scalable, observable data science platform for retail analytics with high cross-service reliability. Enable non-technical stakeholders to extract insights through natural-language querying. Reduce cross-team coordination overhead across frontend squads while maintaining data immutability and async message-passing guarantees.

Architecture & Patterns

  • Modular, event-driven backend using Rust and .NET with data immutability and async message-passing
  • Observability-first design with OpenTelemetry instrumentation
  • Micro-frontend architecture reducing cross-team coordination
  • Contract-driven development with Pact: 40% reduction in integration bugs
  • Reusable Terraform module library and CI/CD templates (GitHub Actions + ArgoCD)
  • LLM-based "text-to-insight" querying that translates business questions into analytics, refined with analysts through human-in-the-loop feedback
  • Model-specific prompt engineering: system prompts, few-shot exemplars, and tool-calling schemas tuned per underlying LLM (GPT-4-class, Claude, lighter open-source models), backed by a golden-question evaluation harness for per-model iteration
  • Event streaming via Kafka

Tools & Stack

Rust, .NET Core, Pact, Terraform, GitHub Actions, Kubernetes, ArgoCD, Azure Functions, OpenTelemetry, Kafka

Business Outcomes

  • 40% reduction in cross-service integration bugs via contract-driven development
  • Standardised infrastructure via reusable Terraform module library and CI/CD templates
  • Micro-frontend architecture reduced cross-team coordination overhead
  • LLM-based "text-to-insight" interfaces enabled non-technical users to query retail data