Skip to main content

dunnhumby Retail Analytics Data Science Platform

Problem Space

Deliver a scalable, observable data science platform for retail analytics with high cross-service reliability. Enable non-technical stakeholders to extract insights through natural-language querying. Reduce cross-team coordination overhead across frontend squads while maintaining data immutability and async message-passing guarantees.

Architecture & Patterns

  • Modular, event-driven backend using Rust and .NET with data immutability and async message-passing
  • Observability-first design with OpenTelemetry instrumentation
  • Micro-frontend architecture reducing cross-team coordination
  • Contract-driven development with Pact — 40% reduction in integration bugs
  • Reusable Terraform module library and CI/CD templates (GitHub Actions + ArgoCD)
  • LLM-based "text-to-insight" querying that translates business questions into analytics, refined with analysts through human-in-the-loop feedback
  • Model-specific prompt engineering — system prompts, few-shot exemplars, and tool-calling schemas tuned per underlying LLM (GPT-4-class, Claude, lighter open-source models), backed by a golden-question evaluation harness for per-model iteration
  • Event streaming via Kafka

Tools & Stack

Rust, .NET Core, Pact, Terraform, GitHub Actions, Kubernetes, ArgoCD, Azure Functions, OpenTelemetry, Kafka

Business Outcomes

  • 40% reduction in cross-service integration bugs via contract-driven development
  • Standardised infrastructure via reusable Terraform module library and CI/CD templates
  • Micro-frontend architecture reduced cross-team coordination overhead
  • LLM-based "text-to-insight" interfaces enabled non-technical users to query retail data

Reusable Narrative Snippets

Designed a modular, event-driven backend in Rust and .NET with data immutability, async message-passing, and observability-first instrumentation via OpenTelemetry.

Introduced micro-frontend architecture to reduce cross-team coordination, and contract-driven development with Pact that cut integration bugs by 40%.

Built LLM-based "text-to-insight" querying workflows with GPT agents, enabling non-technical stakeholders to extract retail analytics insights through natural language.

Treated each underlying LLM as a distinct execution target — instruction style, context budget, tool-calling format, failure modes — and authored model-specific prompts to that contract rather than reusing a generic template. Dense JSON-schema tool calls and tight few-shot exemplars for GPT-4-class models; longer, XML-tagged role framing for Claude; tighter exemplars and retries on schema violations for lighter open-source models.

Backed the prompt work with a small evaluation harness — golden questions, expected tool-call sequences, grading on whether the final answer cited the right SKUs and segments — so prompt iteration was measurable per model rather than vibes-based, and shipped templates that lifted agentic task completion and cut hallucinated SKU and segment references.

Source Notes

  • Derived from role responsibilities and achievements in config/madu_profile.json; reconciled with JobVia export (madu_alikor_export.json).
  • Confidence: high