From Lab to Field: What Building an AI Platform for Waste Management Taught Me

There is a version of machine learning that lives entirely on a benchmark. The data is clean, the labels are trustworthy, the train and test distributions match, and the metric tells you everything you need to know. That version is necessary. It is how the field measures progress. But it is not the version that ends up in the back of a waste-collection truck at five in the morning.

This essay is about the gap between those two versions, and what it took to close it for DONASO, an AI platform for waste management that I led from lab research to real-world field deployment.

The model was never the hard part

When we started, the framing was the obvious one: build a model that can classify waste streams accurately, then deploy it. The first part went roughly as expected. We had datasets, we had architectures, we had a metric moving in the right direction. On the held-out set, the model performed well.

Then we took it to the field, and the field did something that benchmarks never do: it refused to cooperate.

• The lighting was wrong: low sun, shadows, partial occlusion from the truck's hopper.
• The objects were not the objects in our training set: new packaging, broken items, partially crushed material, wet surfaces that changed how light reflected.
• The labels in our dataset, which we had treated as ground truth, turned out to reflect annotator assumptions that did not survive contact with what actually arrived on the conveyor.
• The throughput was higher than anything we tested against. The model had milliseconds, not the comfortable seconds of an offline evaluation.

None of these were model problems. They were system problems. And none of them showed up in the metric we had been optimising.

What we were actually optimising

The shift that mattered was not architectural. It was a shift in what we treated as the objective.

In the lab, the objective is accuracy on a fixed distribution. In the field, the objective is something closer to: does this system help the people operating it make better decisions, at the speed they have to make them, under the conditions they actually work in? That is a different function, and it does not optimise the same way.

We learned to read the gap between those two objectives carefully. Some of the model's errors were acceptable. They happened on cases a human operator could correct in a glance, and the system's job was to reduce their load, not replace their judgment. Other errors were quietly catastrophic. They happened on cases that looked fine to a tired operator at the end of a shift, and they compounded because no one noticed until downstream.

The engineering work was not making the model better on the benchmark. It was mapping which errors mattered, which did not, and redesigning the system so that the model's confidence was exposed and used honestly rather than collapsed into a single score.

The things that aren't in the paper

A deployed system is mostly the parts that do not make it into the paper:

• Inference latency under load. The model is only useful if it runs fast enough on the hardware that actually exists at the site, not on a GPU you can bill to a research grant.
• Failure modes the benchmark cannot see. What happens when a sensor degrades, when the network drops, when a new waste category appears that the model has never seen? The system has to degrade gracefully, and gracefully is not a metric you can put in a loss function.
• The data loop. Every deployment generated data the lab never had. Building the pipeline to capture, label, validate, and feed that data back (without poisoning the training set with the same annotator assumptions that got us in trouble the first time) was more work than the original model.
• Operating alongside people. The platform sat next to operators who had years of judgment the model did not have. Treating that judgment as noise to be eliminated, rather than as a signal the system should defer to, would have been the most expensive mistake available. And it is the default mistake of pure ML thinking.

What it taught me

I teach software engineering and machine learning, and I run a research lab. The single thing I try to get across, to students, to researchers, to anyone building with AI, is this: the benchmark is not the system. It is a measurement of the system under conditions that do not exist in production.

The engineering judgment that closes that gap is not glamorous. It is largely about honesty: being honest about what the model actually knows, being honest about what the data actually represents, being honest about the conditions under which the metric you optimised still means anything at all.

That honesty is the difference between a model that performs and a system that works. Most of my work, in the classroom, in the lab, and in the ventures I build, is about narrowing that distance.

DONASO was an applied research project taking AI for waste management from lab research to real-world field deployment. This essay is adapted from the lessons of that work.