Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Automatic strain sensor design via active learning and data augmentation for soft machines

An Author Correction to this article was published on 11 March 2022

This article has been updated

Abstract

Emerging soft machines require high-performance strain sensors to achieve closed-loop feedback control. Machine learning is a versatile tool to uncover complex correlations between fabrication recipes and sensor performance at the device level. Here a three-stage machine learning framework was realized for a high-accuracy prediction model capable of automating the design of strain sensors. First, a support-vector machine classifier was trained by using 351 compositions of various nanomaterials. Second, through 12 active learning loops, 125 strain sensors were stagewise fabricated to enrich the multidimensional dataset. Third, to address the challenge of data scarcity, data augmentation was implemented to synthesize >10,000 virtual data points, followed by genetic algorithm-based selection to optimize the model’s prediction accuracy. Several data-driven design rules for piezoresistive nanocomposites were generalized and validated by in situ microscopic studies. As final demonstrations, model-suggested strain sensors can be integrated into/onto various soft machines to endow them with real-time strain-sensing capabilities.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: Three-stage framework for construction of a machine learning-enabled prediction model capable of automatic strain sensor design for soft machines.
Fig. 2: Boundary definition in sensor design space through an SVM.
Fig. 3: Progressive exploration of sensor design space through active learning.
Fig. 4: Model optimization through data augmentation and GA-based selection.
Fig. 5: Statistical analysis of data points collected from active learning.
Fig. 6: Automatic sensor design for soft machines.

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding authors on reasonable request. Source data are provided with this paper.

Code availability

The Python code to implement the machine learning tasks within this study are available from GitHub (https://github.com/jiali1025/Automatic_Strain_Sensor_Design) or Zenodo (https://doi.org/10.5281/zenodo.5709148)57.

Change history

References

  1. Rus, D. & Tolley, M. T. Design, fabrication and control of soft robots. Nature 521, 467–475 (2015).

    Google Scholar 

  2. Shepherd, R. F. et al. Multigait soft robot. Proc. Natl Acad. Sci. USA 108, 20400–20403 (2011).

    Google Scholar 

  3. Walsh, C. Human-in-the-loop development of soft wearable robots. Nat. Rev. Mater. 3, 78–80 (2018).

    Google Scholar 

  4. Rus, D. & Tolley, M. T. Design, fabrication and control of origami robots. Nat. Rev. Mater. 3, 101–112 (2018).

    Google Scholar 

  5. Kim, Y., Yuk, H., Zhao, R., Chester, S. A. & Zhao, X. Printing ferromagnetic domains for untethered fast-transforming soft materials. Nature 558, 274–279 (2018).

    Google Scholar 

  6. Hu, W., Lum, G. Z., Mastrangeli, M. & Sitti, M. Small-scale soft-bodied robot with multimodal locomotion. Nature 554, 81–85 (2018).

    Google Scholar 

  7. Laschi, C., Mazzolai, B. & Cianchetti, M. Soft robotics: technologies and systems pushing the boundaries of robot abilities. Sci. Rob. 1, eaah3690 (2016).

    Google Scholar 

  8. Kim, Y., Parada, G. A., Liu, S. & Zhao, X. Ferromagnetic soft continuum robots. Sci. Rob. 4, eaax7329 (2019).

    Google Scholar 

  9. Wang, M. et al. Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Nat. Electron. 3, 563–570 (2020).

    Google Scholar 

  10. Zhou, Z. et al. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nat. Electron. 3, 571–578 (2020).

    Google Scholar 

  11. Thuruthel, T. G., Shih, B., Laschi, C. & Tolley, M. T. Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot. 4, eaav1488 (2019).

    Google Scholar 

  12. Sinatra, N. R. et al. Ultragentle manipulation of delicate structures using a soft robotic gripper. Sci. Robot. 4, eaax5425 (2019).

    Google Scholar 

  13. Sundaram, S. et al. Learning the signatures of the human grasp using a scalable tactile glove. Nature 569, 698–702 (2019).

    Google Scholar 

  14. Zhang, J. et al. Robotic artificial muscles: current progress and future perspectives. IEEE Trans. Robot. 35, 761–781 (2019).

    Google Scholar 

  15. Mirvakili, S. M. & Hunter, I. W. Artificial muscles: mechanisms, applications, and challenges. Adv. Mater. 30, 1704407 (2018).

    Google Scholar 

  16. Zhao, H., O’Brien, K., Li, S. & Shepherd, R. F. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Robot. 1, eaai7529 (2016).

    Google Scholar 

  17. Cianchetti, M., Laschi, C., Menciassi, A. & Dario, P. Biomedical applications of soft robotics. Nat. Rev. Mater. 3, 143–153 (2018).

    Google Scholar 

  18. Amjadi, M., Kyung, K.-U., Park, I. & Sitti, M. Stretchable, skin-mountable, and wearable strain sensors and their potential applications: A review. Adv. Funct. Mater. 26, 1678–1698 (2016).

    Google Scholar 

  19. Qiu, A. et al. A path beyond metal and silicon:polymer/nanomaterial composites for stretchable strain sensors. Adv. Funct. Mater. 29, 1806306 (2019).

    Google Scholar 

  20. Cai, Y. et al. Stretchable Ti3C2Tx MXene/carbon nanotube composite based strain sensor with ultrahigh sensitivity and tunable sensing range. ACS Nano 12, 56–62 (2018).

    Google Scholar 

  21. Shi, X., Liu, S., Sun, Y., Liang, J. & Chen, Y. Lowering internal friction of 0D–1D–2D ternary nanocomposite-based strain sensor by fullerene to boost the sensing performance. Adv. Funct. Mater. 28, 1800850 (2018).

    Google Scholar 

  22. Wang, Y. et al. Wearable and highly sensitive graphene strain sensors for human motion monitoring. Adv. Funct. Mater. 24, 4666–4670 (2014).

    Google Scholar 

  23. Shi, X. et al. Bioinspired ultrasensitive and stretchable MXene-based strain sensor via nacre-mimetic microscale ‘brick-and-mortar’ architecture. ACS Nano 13, 649–659 (2019).

    Google Scholar 

  24. Jayathilaka, W. A. D. M. et al. Significance of nanomaterials in wearables: A review on wearable actuators and sensors. Adv. Mater. 31, 1805921 (2019).

    Google Scholar 

  25. Araromi, O. A. et al. Ultra-sensitive and resilient compliant strain gauges for soft machines. Nature 587, 219–224 (2020).

    Google Scholar 

  26. Yang, H. et al. Wireless Ti3C2Tx MXene strain sensor with ultrahigh sensitivity and designated working windows for soft exoskeletons. ACS Nano 14, 11860–11875 (2020).

    Google Scholar 

  27. Wang, H., Totaro, M. & Beccai, L. Toward perceptive soft robots: progress and challenges. Adv. Sci. 5, 1800541 (2018).

    Google Scholar 

  28. Mengüç, Y. et al. Wearable soft sensing suit for human gait measurement. Int. J. Rob. Res. 33, 1748–1764 (2014).

    Google Scholar 

  29. Lu, N. & Kim, D.-H. Flexible and stretchable electronics paving the way for soft robotics. Soft Robot. 1, 53–62 (2014).

    Google Scholar 

  30. Schmidt, J., Marques, M. R. G., Botti, S. & Marques, M. A. L. Recent advances and applications of machine learning in solid-state materials science. npj Comput. Mater. 5, 83 (2019).

  31. Xia, B. et al. Improving the actuation speed and multi-cyclic actuation characteristics of silicone/ethanol soft actuators. Actuators 9, 62 (2020).

  32. Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials science. Nature 559, 547–555 (2018).

    Google Scholar 

  33. Zahrt, A. F. et al. Prediction of higher-selectivity catalysts by computer-driven workflow and machine learning. Science 363, eaau5631 (2019).

    Google Scholar 

  34. Toyao, T. et al. Machine learning for catalysis informatics: recent applications and prospects. ACS Catal. 10, 2260–2297 (2020).

    Google Scholar 

  35. Kitchin, J. R. Machine learning in catalysis. Nat. Catal. 1, 230–232 (2018).

    Google Scholar 

  36. Vamathevan, J. et al. Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discov. 18, 463–477 (2019).

    Google Scholar 

  37. Ekins, S. et al. Exploiting machine learning for end-to-end drug discovery and development. Nat. Mater. 18, 435–441 (2019).

    Google Scholar 

  38. Voznyy, O. et al. Machine learning accelerates discovery of optimal colloidal quantum dot synthesis. ACS Nano 13, 11122–11128 (2019).

    Google Scholar 

  39. Durrer, R. et al. Automated tuning of double quantum dots into specific charge states using neural networks. Phys. Rev. Appl. 13, 054019 (2020).

    Google Scholar 

  40. Li, J. et al. AI applications through the whole life cycle of material discovery. Matter 3, 393–432 (2020).

    Google Scholar 

  41. Cole, J. M. A design-to-device pipeline for data-driven materials discovery. Acc. Chem. Res. 53, 599–610 (2020).

    Google Scholar 

  42. Cao, B. et al. How to optimize materials and devices via design of experiments and machine learning: demonstration using organic photovoltaics. ACS Nano 12, 7434–7444 (2018).

    Google Scholar 

  43. Afsarimanesh, N. et al. A review on fabrication, characterization and implementation of wearable strain sensors. Sens. Actuator A 315, 112355 (2020).

    Google Scholar 

  44. Murphey, Y. L., Guo, H. & Feldkamp, L. A. Neural learning from unbalanced data. Appl. Intell. 21, 117–128 (2004).

    MATH  Google Scholar 

  45. Hoffmann, J. et al. Machine learning in a data-limited regime: augmenting experiments with synthetic data uncovers order in crumpled sheets. Sci. Adv. 5, eaau6792 (2019).

    Google Scholar 

  46. Chen, P.-Y. et al. Multiscale graphene topographies programmed by sequential mechanical deformation. Adv. Mater. 28, 3564–3571 (2016).

    Google Scholar 

  47. Noble, W. S. What is a support vector machine? Nat. Biotechnol. 24, 1565–1567 (2006).

    Google Scholar 

  48. Whitley, D. A genetic algorithm tutorial. Stat. Comput. 4, 65–85 (1994).

    Google Scholar 

  49. Schober, P., Boer, C. & Schwarte, L. A. Correlation coefficients: appropriate use and interpretation. Anesth. Analg. 126, 1763–1768 (2018).

    Google Scholar 

  50. Štrumbelj, E. & Kononenko, I. Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41, 647–665 (2014).

    Google Scholar 

  51. Bach, S. et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, e0130140 (2015).

    Google Scholar 

  52. Zhang, S. et al. Predicting the formability of hybrid organic-inorganic perovskites via an interpretable machine learning strategy. J. Phys. Chem. Lett. 12, 7423–7430 (2021).

    Google Scholar 

  53. Low, J. H. et al. Hybrid tele-manipulation system using a sensorized 3-D-printed soft robotic gripper and a soft fabric-based haptic glove. IEEE Robot. Autom. Let. 2, 880–887 (2017).

    Google Scholar 

  54. Truong, T. V., Viswanathan, V. K., Joseph, V. S. & Alvarado, P. V. Y. Design and characterization of a fully autonomous under-actuated soft batoid-like robot. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 5826–5831 (IEEE, 2019).

  55. Alhabeb, M. et al. Guidelines for synthesis and processing of two-dimensional titanium carbide (Ti3C2Tx MXene). Chem. Mater. 29, 7633–7644 (2017).

    Google Scholar 

  56. Shenton, M. J., Lovell-Hoare, M. C. & Stevens, G. C. Adhesion enhancement of polymer surfaces by atmospheric plasma treatment. J. Phys. D 34, 2754–2760 (2001).

    Google Scholar 

  57. Li, J., Lim, K. & Yang, H. Automatic strain sensor design (v.1.0.3). Zenodo https://doi.org/10.5281/zenodo.5709148 (2021).

Download references

Acknowledgements

We thank C.-H. Yeow from the Department of Biomedical Engineering in the National University of Singapore for providing the soft gripper. We acknowledge the financial support provided by the Singapore RIE2020 Advanced Manufacturing and Engineering Programmatic Grant ‘Accelerated Materials Development for Manufacturing’ by the Agency for Science, Technology and Research under grant no. A1898b0043 (to X.W.). We acknowledge the financial support provided by the Start-Up Fund of University of Maryland, College Park (KFS no. 2957431 to P.-Y.C.). Funding for this research was provided by Maryland Industrial Partnerships under grant no. 6808 (KFS no. 4311103 to P.-Y.C.), Maryland Innovation Initiative (MII) Technology Assessment Award (KFS no. 4308302 to P.-Y.C.), and MOST-AFOSR Taiwan Topological and Nanostructured Materials Grant under grant no. FA2386-21-1-4065 (KFS no. 5284212 to P.-Y.C.).

Author information

Authors and Affiliations

Authors

Contributions

P.-Y.C., X.W., J.L. and H.Y. conceived the project ideas and designed the experiments. H.Y. and S.L. carried out the synthesis of MXene nanosheets and characterizations. H.Y., K.Z.L. and J.L. designed the machine learning framework and implemented the machine learning tasks in Python. H.Y., Q.W., J.L. and C.P. collected the experimental data. T.V.T., Q.W. and H.Y. integrated the model-suggested sensors onto soft swimmer robot and performed related tests. X.X. and H.Y. performed the FEA simulations and analyses. P.-Y.C., X.W., J.L. and H.Y. interpreted the results and co-wrote the manuscript. K.L., S.L., M.D., T.C., X.L. and Q.X. involved in the discussion and manuscript revisions. P.-Y.C, P.V.A. and X.W. supervised this project.

Corresponding authors

Correspondence to Xiaonan Wang or Po-Yen Chen.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Influential fabrication parameters for tuning strain sensor characteristics.

(a) Schematic illustration of influential fabrication parameters for tuning strain sensor characteristics, including composition, thickness, and morphology of strain sensing layer. (b) Resistance–strain profiles of various G0 sensors with the sensing layers at different MXene/SWNT/PVA ratios. All G0 sensors were with the sensing layers at the same thickness of 800 nm. (c) Resistance–strain profiles of various G0 sensors with the sensing layers at different thicknesses. All G0 sensors were with the sensing layers at the same MXene/SWNT/PVA ratio of 90/5/5. (d) Resistance–strain profiles of Gn sensors with different sensing layer morphologies. All Gn sensors were composed of ps-MXene layers either with planar texture (G0), periodic wrinkles (G1–1D) or isotropic crumples (G1–2D). The MXene/SWNT/PVA ratio and thickness of all ps-MXene layers were controlled to be 100/0/0 and 800 nm, respectively.

Source data

Extended Data Fig. 2 In situ electron microscopic studies to validate data-driven design rules.

(a) SEM images of G0 ps-MXene layers with varying thicknesses under various uniaxial strains. Crack-to-width ratios (b) and crack densities (c) in G0 ps-MXene layers with varying thicknesses. All the ps-MXene layers kept the same MXene/SWNT/PVA ratio of 90/10/0. Crack-to-width ratios (d) and crack densities (e) in G0 ps-MXene layers with varying SWNT loadings (wt.%). All the ps-MXene layers kept the same thickness of 800 nm and 0 wt.% PVA loading. FEA simulation of strain distribution and in situ SEM images of G0 (f), G1–1D (g), and G1–2D (h) ps-MXene layers under various uniaxial strains.

Source data

Supplementary information

Supplementary Information

Supplementary Notes 1–33, Figs. 1–32 and Tables 1–18.

Reporting Summary

Supplementary Video 1

A soft swimmer robot with model-suggested strain sensors for underwater exploration mission.

Supplementary Video 2

Real-time monitoring of a soft swimmer robot.

Source data

Source Data Fig. 1

Feasibility grades of detachment situations of 351 ps-MXene layers from PVDF membranes. Possibility heatmap representing the chances of successfully detaching ps-MXene layers from PVDF membranes.

Source Data Fig. 2

\({\bar L}\) and MSE values of trained navigation models by using different acquisition functions, including variance, random selection, distance and A Score. Cumulative number of fabricated Gn sensors for each loop of active learning; \({\bar{L}}\) and MSE values of evolving navigation model at different stages of active learning.

Source Data Fig. 3

Comparison of MRE values of various candidate models. Model-predicted strain labels and actual sensor characteristics of a G0 sensor. Model-predicted strain labels and actual sensor characteristics of a G1–1D sensor. Model-predicted strain labels and actual sensor characteristics of a G1–2D sensor.

Source Data Fig. 4

Classification of εmax extracted from 125 Gn sensors at different morphologies of strain-sensing layers. Spearman’s ρ statistical analyses for G1–2D, G1–1D and G0 sensors between fabrication parameters (for example, SWNT loading, PVA loading, and sensing layer thickness) and εmax. SHAP values of SWNT loading, PVA loading and sensing layer thickness for G1–2D, G1–1D and G0 sensors.

Source Data Fig. 5

GF-strain profiles of two suggested G1–1D sensors. Resistance–strain profile of Sensor 1 for monitoring a robotic task of grasping a candle. Resistance–strain profile of Sensor 1 for monitoring a robotic task of grasping a glue. GF-strain profiles of two suggested G0 sensors. Resistance–strain profile of an embedded G0 sensor (Sensor 2) in the left fin of a soft swimmer robot.

Source Data Extended Data Fig. 1

Resistance–strain profiles of various G0 sensors with the sensing layers at different MXene/SWNT/PVA ratios. Resistance–strain profiles of various G0 sensors with the sensing layers at different thicknesses. Resistance–strain profiles of Gn sensors with different sensing layer morphologies.

Source Data Extended Data Fig. 2

Crack-to-width ratios and crack densities in G0 ps-MXene layers with varying thicknesses and with varying SWNT loadings (wt%).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Li, J., Lim, K.Z. et al. Automatic strain sensor design via active learning and data augmentation for soft machines. Nat Mach Intell 4, 84–94 (2022). https://doi.org/10.1038/s42256-021-00434-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-021-00434-8

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing