Waterloo Students Fold a Shirt Overnight Using a Laundry-Folding Robot

Steven Gong and his University of Waterloo engineering classmates needed a capstone that actually finished a chore companies have chased for years. Folding laundry looks simple until cloth bunches, sleeves hide, and every shirt arrives in a different mess. Their answer was a pair of follower arms that copied two leader arms driven by human hands, a pile of recorded demonstrations, and a single night of training.

Leader arms sat in front of the operators, with follower arms on the table beside them, echoing every joint angle in real time, while cameras monitored the entire workstation. Over the course of four hours, they collected approximately 200 teleoperated runs of a fairly standard sequence: a shirt is lifted out of a basket, dragged onto the table, and then, straightening out the fabric as they go, before folding each side in toward the middle, folding it in half, and finally plopping the finished rectangle down on a growing stack. The cloth was kept fast by 3D-printed grippers, albeit one of them had a little meltdown later on when the older model sent the arms swinging wildly out of control.

ACT, a transformer that tries to predict what a robot will do next, attempted to predict small bursts of motion but, despite hundreds of efforts, refused to shut the shirt. The Diffusion Policy caused motion that was too harsh for the hardware to tolerate, leaving a last resort: π0.5, Physical Intelligence’s 3.3-billion-parameter foundation model, already well-seasoned from training on large collections of robot manipulation, and effectively a work in progress to date. They fine-tuned it during their own demos before firing up a powerful external computer to cycle through all of the checkpoints.

A last-minute move to the presentation room changed the lighting and the background. Another 30 demonstrations went into the set. Training and testing went late into the night, and after roughly 30 checkpoints, that specific combination of settings worked. By 8 a.m., the following arms were operating independently, plucking a shirt from the basket, laying it out, folding the sides over, folding it in half, and stacking it, with no human intervention on the leader arm.

The motion was still a little clumsy, and the folds weren’t exactly perfect, but it got the job done, as the loop from basket to stack occurred all by itself for the first time ever. Testing later revealed that an overly ambitious start had overloaded one of the arms, which then broke mechanically, leaving only a reminder that, while foundation models and student hardware are improving, they both still have some quite obvious flaws.
[Source]