← Back to work

MLB UAT Automation Platform

Automation · Pricing Validation · Risk Controls · Internal Tooling

What I did: Spotted a testing bottleneck and built a Python tool that validates pricing and settlement across every market before release.

Result: Testing and release time down 40%, post-release bugs down 80%, and Architecture Board funding for a 4-person team to scale it.

Executive summary

I designed and delivered a full UAT automation suite for MLB pricing, replacing manual workflows with a scalable validation engine. The platform performs complete market checks within minutes, reducing regression errors by 80% and cutting the time analysts spend on testing and releases by 40%. It now serves as the blueprint for automation across NBA, NFL, and future risk systems.

Strategic context

MLB pricing generates thousands of markets and selections per game. Historically, each release relied on:

  • Large Excel comparisons
  • Manual SQL validation
  • Inconsistent test coverage
  • No automated regression suite

The business needed a scalable, reliable, and repeatable validation framework to support rapid model updates.

The problem

Manual testing failed at scale. It was slow, inconsistent, and prone to human error. The lack of automation meant:

  • Missed behaviour differences between operators
  • Edge-case pricing issues surfaced late
  • Complexity of ladders and envelopes couldn't be validated end-to-end
  • Regression relied on memory rather than structured checks

Constraints

  • No guaranteed engineering resources, so it had to be self-built
  • Had to run from static Athena snapshots
  • Validation logic needed to mirror real model behaviour
  • Tool needed to support operator-specific differences (BETMGM, Entain)

My role

I owned the initiative end-to-end:

  • Mapped the MLB pricing lifecycle and its edge cases
  • Designed the automated rules engine
  • Built the initial Python validation tool
  • Product-managed transition to a multi-user React + SignalR web app
  • Defined coverage, KPIs, metrics, and regression standards
  • Worked daily with traders, modelling, and engineering teams

What I delivered

  • Automated rules engine covering alignment, ladders, envelopes, correlation, and settlement
  • Support for multiple operator behaviours
  • Exception reporting via CSV + Excel with automated highlighting
  • Regression test library for model release validation
  • Web-based multi-user platform for internal teams
  • Framework now adopted across sports

Impact

  • 40% less time spent testing and releasing products
  • 80% fewer missed regression errors
  • Full-market coverage, not sample-based
  • Standardised validation across MLB, NBA, and NFL
  • Lower operational risk with consistent test evidence

Where this applies

Any team that ships rule-heavy logic needs confidence that a release won't break it. The same approach fits:

  • Rules-based validation: pricing, billing, tax, payroll and insurance rating engines
  • Regression testing against real data snapshots: data platforms, analytics pipelines, ML model releases
  • Behaviour that varies by client or region: multi-tenant SaaS, multi-market compliance
  • Turning a script into a funded product: any internal tooling team

System overview

This diagram shows how pricing data flows through the automated validation engine, rules layer, and reporting pipeline.

  1. 1

    Pricing data

    Athena snapshots of live markets

  2. 2

    Validation engine

    • Market alignment checks
    • Ladder and envelope validation
    • Correlated movement detection
    • Operator-specific rules

  3. 3

    Rules layer

    Consistency, envelopes and settlement edge cases

  4. 4

    Reporting

    • CSV and Excel reports
    • Exception logs
    • Regression library