ObexMetrics
A record of a retired platform

Data science is an empirical field.

Great discoveries are preceded by extensive experimentation. ObexMetrics was a SaaS platform designed to facilitate frictionless experimentation.

Status: retired
Active: 2023 to 2025
Kind: ML experimentation SaaS
01 / The friction

The notebooks pile up, and the knowledge stays inside them.

Given the diversity of data platforms, algorithms and shapes of data, practitioners rely on ad-hoc tooling, with the Jupyter notebook playing a major part. It is a versatile tool, but after a number of experiments the users are left with a large number of notebooks. The notebooks capture various data manipulation steps, algorithms and parameters, impeding the ability to share or distill the knowledge. When a new thread of investigation is opened, it is very tedious to gather all the previous related experiments to build on top of the conclusions.

ObexMetrics was designed to solve this exact problem.

02 / What we built

The same three mechanisms, read twice.

We sold these as capabilities that save time for ML practitioners. Read today, each one is a part of the harness an LLM requires to run machine learning workflows autonomously.

THE MECHANISM
AS A PRODUCTIVITY TOOL (THEN)
AS AN AGENT HARNESS (NOW)
1) All experimental artifacts persisted and shareable
A new thread of investigation is a matter of pressing some buttons or writing a new algorithm.
The memory. The LLM can see every experiment that was tried, with its data, parameters and result.
2) Unlimited experiments in parallel
Removes the single threaded nature of experiments, which restricts the creativity of data scientists.
The hands. The LLM can launch a large number of experiments on its own and wait for none of them.
3) Workloads run on the customer's cloud account
Makes unlimited compute heavy experimentation viable for a SaaS business.
The boundary. Autonomous runs stay inside the customer's own account, data and budget.
A question we were asked often
Being a SaaS, how does it allow unlimited experiments, which are typically compute heavy, and still be a successful business ?

We built a clever mechanism, first in the ML tools business, where customers provide their cloud credentials, and we run their workloads on their cloud accounts.

03 / The timing

It was too early for the agentic world.

When we went public with the toolset, the customers were not impressed by the tooling, even when we showed how the future would look by integrating LLM capability in the demonstrations. Also, the practitioners held on to their tools and local artifacts, with some fearing the automation of their work as a threat.

The two pillars of ObexMetrics, 1) preserving and sharing of ML artifacts, 2) the flexibility to run a large number of experiments in parallel, are essential tooling harnesses if we are to move machine learning workflows to LLM driven agentic workflows. With the LLM functioning as a brain, with complete preservation of the ML artifacts, and the ability to run experiments autonomously, we can finally build goated Auto-ML algorithms.

We learnt a lot in this process.

04 / On record
The platform: experiments without notebooks
2:58
The LLM in the loop: exploratory data analysis with GPT-4o
7:55