Shape-sensing sheet digitally tracks its movement as it bends and twists

MIT engineers have developed a flexible, shape-sensing sheet that digitally reconstructs its own form as it bends and twists. The sensor is similar in principle to some motion-capture garments, which track a person’s physical movements to mimic them in a virtual avatar. Those designs use rigid sensors that are stitched into a suit at strategic locations. In contrast, the team’s new design uses multiple soft optical fibers that zig-zag throughout a soft and stretchable sheet. When the sheet bends, the fibers bend in kind, changing the pattern of light that travels through them. The researchers developed an algorithm to interpret the bending light patterns on the fly, to create a digital image that moves the same way the sheet is moving. The team also showed that the design is resistant to damage: Even when a few fibers are cut or disconnected, the remaining fibers can still accurately reconstruct the sheet’s overall shape. “Our setup is targeting fully soft and stretchable sensing surfaces so it’s potentially safer for interacting with a human or a soft environment,” says Qifan Yu, a mechanical engineering graduate student at MIT. The team envisions that the new design could be fashioned into a soft and pliable garment that a user could comfortably wear to physically control a video game character or a tele-operated robot. The sheet could also be used in physical therapy settings, where it could wrap around a patient’s leg or arm to track and record their mobility and range of motion during each session. “In physical therapy, you often need to gather patient data on how they are moving and stretching their arm during rehabilitation, for example,” Yu says. “Surface shape sensing could be very useful in that way.” Yu and his colleagues present their design in a study appearing in the journal Advanced Intelligent Systems. The study’s co-authors at MIT are graduate student Nina Cao and assistant professor of mechanical engineering Kaitlyn Becker.The shape of lightThe team’s shape-sensing design is a tweak on the standard optical fiber. Also known as a waveguide, an optical fiber is designed to guide light efficiently through a fiber. Optical fibers consist of a core of transparent material, such as glass, wrapped in a dark, opaque cladding. When light is sent through one end of the fiber, the dark cladding traps the light within the glass core, which itself is extremely smooth, allowing the light to pass straight through the fiber with near-perfect efficiency. Waveguides are used in fiber optic cables for super fast, efficient data transmission and telecommunications. Scientists have also experimented with waveguides as simple shape sensors: When the fibers bend, they affect how much light can make it all the way through. The more a fiber bends, the less light comes out the other end. The amount of light that shines out, then, can be a measure of the degree the fiber bends. In this way, optical fibers have been used to sense simple curvatures. “People have used waveguides to sense the shape of a line, which they have applied to a robot arm to see how it curves,” Yu says. “But they haven’t been applied to surface shape sensing, to reconstruct the 3D shape of a surface. That’s what we’re trying to do here.”Fiber tweaksIn their new design, the researchers fabricated their own optical fibers, with some twists. Like conventional waveguides, they made their fibers from a transparent core surrounded by dark cladding. Rather than hard glass, they used a clear, flexible rubber core and a cladding made from the same rubbery material, dyed black. And instead of keeping the core completely smooth, they intentionally roughed up one side. If one side of the fiber is rougher than the other, they reasoned, then when the fiber is bent one way, any light passing through would scatter off the rough surface and affect the total amount of light that makes it out the other end. This would be a different amount than would pass through if the fiber were bent at the same angle but toward the fiber’s smooth side. In this way, the half-roughened fiber should act as a bidirectional shape sensor. The team fabricated multiple bidirectional optical fibers and looked to embed them into a soft sheet of silicone, arranged so that the fibers would reconstruct the sheet’s shape as it bends and twists. To do so, the researchers carried out simulations of sheets embedded with different optical fiber patterns, from a straightforward checkerboard to criss-crossed, zig-zag arrangements. They simulated different ways to bend or twist the sheets, and measured the output of light from each sheet’s configuration of fibers. They converted these light measurements into estimates of how much each fiber must be bending, and combined these to construct an overall 3D shape of the sheet, which they compared to the original simulated sheet shape. From these simulations, they found that a particular spacing of zig-zagging fibers was closest to recreating the sheet’s original shape.The researchers then fabricated a shape-sensing sheet with the same zig-zag pattern of optical fibers embedded into the sheet. They incorporated an LED at one end of each fiber and a light sensor at the other end, which they connected to an external circuit board to collect and amplify the light measurements. They also developed an algorithm to automatically convert the measurements from fibers into a reconstruction of the sheet’s 3D shape as a whole. In experiments, they showed that the algorithm smoothly created a virtual reconstruction of the sheet, almost in real time, as the researchers twisted it into different forms. For instance, when they folded the sheet diagonally, and then again in the opposite direction, the virtual twin mimicked the changing shapes, almost in real time. They also placed the sheet in different 3D-printed molds so they could precisely measure the difference between the digitally reconstructed sheet and the physical sheet laying over each mold. In these tests, they found the soft sensing sheet was more accurate than other designs.“We use a metric that describes the distance between the actual surface and the reconstructed surface, and from that, we found our error was less than 0.4 centimeters,” Yu says. “Existing designs, which are based on rigid sensors, have errors of around 1 to 2 centimeters. So that’s respectable, and at least on par with existing technologies.”The team will further optimize the sensing sheet, first by thinning it down. Currently, the optical fibers are 1 millimeter thick. Other fabrication processes could shave them down to tens of micrometers, which can be thinner than a strand of hair. Then, the researchers envision embedding many more fibers into a garment, to sense detailed changes in its shape and form, for instance to gauge a patient’s performance in physical therapy. “We hope to build tools that augment what a physical therapist can do and help them track and quantify their patients’ progress over time,” says Becker. “These sensing sheets could help a physical therapist to recall and compare more precisely how an evaluation went today versus two months and hundreds of appointments ago.”This research was supported, in part, by MathWorks.