Vision-assisted 4D printing of a silicone gripper with integrated sensors and high energy density actuators

Purpose Direct ink writing (DIW) enables soft robot fabrication with ultra-soft elastomers (Shore 00-10), but integrating sensors, actuators and achieving 4D morphing remains challenging. This research explores vision-guided pick-and-place methods to autonomously embed high-energy-density actuators and flexible sensors into DIW-printed grippers, enabling 4D morphing. Design/methodology/approach This study introduces an automated one-shot additive manufacturing process that embeds a strain sensor and coiled-shape memory alloy (SMA) actuator into a silicone matrix using a 4-DOF pick-and-place robot. A computer vision system identifies components and their target placement, enabling a “pause-and-go” DIW workflow without manual intervention. Finite element analysis (FEA) was carried out on a polymer strain sensor to match the flexural rigidity of a 0.18 mm steel ribbon. After embedding, the sensor provided proprioceptive and exteroceptive feedback in a gripper. The SMA actuator's dynamic response was characterized under varying currents and loads. Finally, integrating the sensor enabled closed-loop control of the gripper mounted on a humanoid robot, validating its performance in real-world tasks. Findings The vision-guided system enables automated component placement during DIW without relying on hardcoded trajectories. A strain sensor with a gauge factor of 2 enables closed-loop PI control of the SMA actuator, reducing energy consumption by 75% and cooling time by 85%. The 0.5 mm wire, 3.45 mm coil SMA actuator achieved 80% strain under a 400 g load at 3.5 A, with a blocking force of 11.76 N. Integrated into a humanoid-mounted gripper, the system demonstrated adaptive functionality across diverse object types. Originality/value This work demonstrates a novel vision-assisted DIW process that achieves one-shot integration of sensors and actuators—creating a smart, energy-efficient, high–energy-density soft robotic gripper capable of adaptive closed-loop control. Graphical abstract A diagram of robotic setup, actuation results, actuator design, and robot performing movement sequence. The illustration shows a composite image with four main sections: Top-left: It shows a labeled schematic of an experimental setup for actuator and sensor testing with a D I W 3-D printer combined with a P N P robot and an electromagnetic end effector. The frame supports a “D I W 3-D printer” head and two cameras positioned on the left to capture the actuator and sensor platform. The “Actuator and Sensor Platform” is a yellow rectangular base where testing occurs. The “Electromagnetic End Effector” on the P N P robot’s arm at the right interacts physically with the actuator on the platform. Orange dashed lines indicate the fields of view of the cameras observing the platform and end effector interaction. Right: At the top, a schematic shows a linear actuator at the left linked by a bold red arrow labeled “13 Volts, 4.5 Amperes” to a gripper at the right. The gripper illustrates an actuation angle of theta degrees using an orange arc and line. Below, test conditions are listed: “Actuation Time: 5 seconds” and “Rest Time: 20 seconds”. Below them, it presents a magenta line plot of the angle of theta degrees versus time for three repeated cycles, showing sharp rises and gradual falls. The horizontal axis is labeled “Time (seconds),” ranging from 0 to 80 with an interval of 20. The vertical axis is labeled “theta degrees,” ranging from 0 to 60 with an interval of 20. The line starts from (0, 0), attains the peaks at (5, 59), (30, 60), and (56, 63), and ends at (75, 9). At the bottom, it shows a red stair-step graph of relative resistance change versus time, with clear step increases aligned with the actuation events. The horizontal axis is labeled “Time (seconds),” ranging from 0 to 80 with an interval of 20. The vertical axis is labeled “delta R over R o,” ranging from 0 to 0.08 with an interval of 0.02. The line starts from (0, 0), attains the peaks at (5, 0.063), (30, 0.062), and (56, 0.067), and ends at (75, 0.04). Note: All the numerical data values are approximated. Middle: The left section shows two markers: a “Small Green Marker” on a yellow platform on the left and a “Blue Marker” mounted on a gray actuator arm on the right. Both markers are labeled with arrows pointing to their positions. A cartoon camera is shown positioned above and to the right, capturing the blue marker on the actuator. The illustration represents marker placement and camera perspective in an experimental setup for tracking actuator movements. The central section illustrates a linear bar sensor assembly measuring “110 millimeters” in length. The central bar has a light gray body with finely spaced grooves along its full length. At the midsection, a thin black body runs along the gray body labeled “Strain Gauge,” firmly bonded to the grooved surface. Three deep blue rectangular blocks labeled “T P U Collar” are placed symmetrically—one at the left end, one at the right end, and one exactly at the middle section—each wrapping around the bar, securing its body. The right section shows a semi-circular structure with a gray “Silicone Body” forming a curved arc that partially encloses the grooved central strip. On both sides of the arc, two rectangular, deep blue blocks labeled “S M A” are vertically aligned and embedded into the silicone structure. Dimensional arrows mark the inner span as “15 millimeters” and the outer span as “20 millimeters,” representing the thickness of the curved silicone piece. Bottom: It shows a sequence of four photographs demonstrating a humanoid robot performing a pick-and-place movement using its arm. In the first photo, the robot stands facing a table with a transparent bin and a vertical cylinder actuator placed in front. The second photo shows the robot beginning arm movement toward the bin, seemingly preparing for grasping or interaction. In the third photo, the robot’s arm is extended, grasping or lifting the actuator over the bin. In the fourth photo, the robot’s arm is fully outstretched, positioning the actuator above the bin, ready to place or drop it inside.