From a room. To an environment we can test in.

CROSS-EPISODE FIELD NOTES · UPDATED 22 OCTOBER 2026

A room that looks right still needs to behave right.

I wanted an environment where I could try robot ideas without rebuilding the scene for every experiment. Across these episodes, the office changed from something to look at into something a simulated robot could act on. Each step introduced a different definition of “working”.

EP05: reconstruct appearance from a phone video.

Episode 5 began with 48.6 seconds of phone footage and no LiDAR input to the reconstruction. The selected process aligned 187 of 195 images, produced 51,475 sparse points and trained 400,000 Gaussians. Those are different outputs: sparse points describe the reconstruction’s structure; the Gaussians reproduce its appearance.

The short capture was not the total processing time. More importantly, a good-looking camera tour did not validate metric scale or surfaces the phone never saw. It gave us a visual model to inspect, with clear limits on what could be measured from it.

EP08: add geometry, objects and collision layers.

Episode 8 combined a static Ouster scan with a 13-second phone video. The scan and video were recorded four days apart. LiDAR supplied a geometric reference; video supplied appearance. Alignment had to connect those observations before the scene could become an office twin.

Labelled objects, colliders and movable parts made the environment useful for interaction. Their masses and physical parameters were estimates. This is why a splat, a collision surface and a dynamic object should not be treated as interchangeable just because they occupy the same place in a viewer.

EP09: robots find the weak spots.

In Episode 9, simulated Go2 and G1 robots brought alignment, locomotion, lighting and grasping into the same scene. A gripper closing on air made the gap between visual appearance and usable contact geometry concrete. Forcing the humanoid’s arm overhead exposed a balance failure that walking alone had not settled.

These were robot-side simulations. The released G1 fall and fallen-robot drag were reconstructed with an open velocity policy and model, rather than recovered as footage of the original run. Robot-to-robot collisions were disabled. The failures still identify useful checks, but the edit does not establish physical transfer.

EP10: return to the room and recover the alignment.

Once the real setup moved, Episode 10 had to locate the live OS0 in the earlier map and align the iPhone again. A plausible wall fit initially placed the camera incorrectly. Moving-person observations made the time offset visible alongside the geometric fit.

Segmenting my body points into a rough avatar added another layer: measured clusters, inferred skeleton and unseen anatomy. Heading flips and partial visibility showed exactly where a believable person could contain more assumptions than observations.

EP11: a task also tests the environment.

The basketball experiment put a simulated ECO65 into the desk twin. The wrong table, unstable physics and a gripper ejecting the ball all needed attention before evaluating Gemini’s throws. The final run scored four baskets in five attempts, while Gemini misclassified one made shot.

The room was now part of an experiment with an independent outcome check. That is a useful advance over a visual tour, but the result remains tied to a small, fixed simulation. The next step is checking scene dimensions, contact properties and sensor observations against physical measurements.