Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Geometric deep learning on molecular representations

Abstract

Geometric deep learning (GDL) is based on neural network architectures that incorporate and process symmetry information. GDL bears promise for molecular modelling applications that rely on molecular representations with different symmetry properties and levels of abstraction. This Review provides a structured and harmonized overview of molecular GDL, highlighting its applications in drug discovery, chemical synthesis prediction and quantum chemistry. It contains an introduction to the principles of GDL, as well as relevant molecular representations, such as molecular graphs, grids, surfaces and strings, and their respective properties. The current challenges for GDL in the molecular sciences are discussed, and a forecast of future opportunities is attempted.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: Exemplary molecular representations for a selected molecule.
Fig. 2: Deep learning on molecular graphs.
Fig. 3: Chemical language modelling.

References

  1. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Google Scholar 

  2. Schmidhuber, J. Deep learning in neural networks: an overview. Neur. Netw. 61, 85–117 (2015).

    Google Scholar 

  3. Gawehn, E., Hiss, J. A. & Schneider, G. Deep learning in drug discovery. Mol. Inform. 35, 3–14 (2016).

    Google Scholar 

  4. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In International Conference on Machine Learning Vol. 34, 1263–1272 (2017).

  5. Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).

    Google Scholar 

  6. Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems Vol. 30, 5998–6008 (2017).

  7. Hinton, G. et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Proces. Mag. 29, 82–97 (2012).

    Google Scholar 

  8. Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems Vol. 25, 1097–1105 (2012).

  9. Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A. & Vandergheynst, P. Geometric deep learning: going beyond euclidean data. IEEE Signal Process. Mag. 34, 18–42 (2017).

    Google Scholar 

  10. Todeschini, R. & Consonni, V. Molecular Descriptors for Chemoinformatics Vols I–II (John Wiley & Sons, 2009).

  11. Townshend, R. J. et al. Geometric deep learning of RNA structure. Science 373, 1047–1051 (2021).

    Google Scholar 

  12. Gainza, P. et al. Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nat. Methods 17, 184–192 (2020).

    Google Scholar 

  13. Segler, M. H., Kogej, T., Tyrchan, C. & Waller, M. P. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Centr. Sci. 4, 120–131 (2018).

    Google Scholar 

  14. Bronstein, M. M., Bruna, J., Cohen, T. & Veličković, P. Geometric deep learning: grids, groups, graphs, geodesics, and gauges. Preprint at https://arxiv.org/abs/2104.13478 (2021).

  15. Mumford, D., Fogarty, J. & Kirwan, F. Geometric Invariant Theory Vol. 34 (Springer Science & Business Media, 1994).

  16. Cohen, T. S. & Welling, M. Group equivariant convolutional networks. In International Conference on Machine Learning Vol. 33, 2990–2999 (2016).

  17. Kondor, R. & Trivedi, S. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference on Machine Learning Vol. 35, 2747–2755 (2018).

  18. Moriguchi, I., Hirono, S., Liu, Q., Nakagome, I. & Matsushita, Y. Simple method of calculating octanol/water partition coefficient. Chem. Pharmaceut. Bull. 40, 127–130 (1992).

    Google Scholar 

  19. Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2, 303–314 (1989).

    MathSciNet  MATH  Google Scholar 

  20. Hoffmann, R. & Laszlo, P. Representation in chemistry. Angew. Chem. Int. Ed. Engl. 30, 1–16 (1991).

    Google Scholar 

  21. Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations Vol 5. (2017).

  22. Duvenaud, D. et al. Convolutional networks on graphs for learning molecular fingerprints. In International Conference on Neural Information Processing Systems, Vol. 28, 2224–2232 (2015).

  23. Monti, F. et al. Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 5115–5124 (2017).

  24. Battaglia, P. et al. Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems Vol. 29, 4502–4510 (2016).

  25. Battaglia, P. W. et al. Relational inductive biases, deep learning, and graph networks. Preprint at https://arxiv.org/abs/1806.01261 (2018).

  26. Zhou, J. et al. Graph neural networks: a review of methods and applications. AI Open 1, 57–81 (2020).

    Google Scholar 

  27. Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K. R. & Tkatchenko, A. Quantum-chemical insights from deep tensor neural networks. Nat. Commun. 8, 13890 (2017).

    Google Scholar 

  28. Klicpera, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. In International Conference on Learning Representations Vol. 8 (2020).

  29. Feinberg, E. N. et al. PotentialNet for molecular property prediction. ACS Central Science 4, 1520–1530 (2018).

    Google Scholar 

  30. Torng, W. & Altman, R. B. Graph convolutional neural networks for predicting drug-target interactions. J. Chem. Inf. Model. 59, 4131–4149 (2019).

    Google Scholar 

  31. Stokes, J. M. et al. A deep learning approach to antibiotic discovery. Cell 180, 688–702 (2020).

    Google Scholar 

  32. Somnath, V. R., Bunne, C., Coley, C. W., Krause, A. & Barzilay, R. Learning graph models for retrosynthesis prediction. In Advances in Neural Information Processing Systems Vol. 34 (2021).

  33. Coley, C. W. et al. A graph-convolutional neural network model for the prediction of chemical reactivity. Chem. Sci. 10, 370–377 (2019).

    Google Scholar 

  34. Li, J., Cai, D. & He, X. Learning graph-level representation for drug discovery. Preprint at https://arxiv.org/abs/1709.03741 (2017).

  35. Liu, K. et al. Chemi-Net: a molecular graph convolutional network for accurate drug property prediction. Int. J. Mol. Sci. 20, 3389 (2019).

    Google Scholar 

  36. Unke, O. T. & Meuwly, M. PhysNet: a neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput. 15, 3678–3693 (2019).

    Google Scholar 

  37. Schütt, K. T., Unke, O. T. & Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. Preprint at https://arxiv.org/abs/2102.03150 (2021).

  38. Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. & Müller, K.-R. SchNet–a deep learning architecture for molecules and materials. J. Chem. Phys. 148, 241722 (2018).

    Google Scholar 

  39. Schütt, K., Gastegger, M., Tkatchenko, A., Müller, K.-R. & Maurer, R. J. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nat. Commun. 10, 5024 (2019).

    Google Scholar 

  40. Bogojeski, M., Vogt-Maranto, L., Tuckerman, M. E., Müller, K.-R. & Burke, K. Quantum chemical accuracy from density functional approximations via machine learning. Nat. Commun. 11, 5223 (2020).

    Google Scholar 

  41. Yang, K. et al. Analyzing learned molecular representations for property prediction. J. Chem. Inf. Model. 59, 3370–3388 (2019).

    Google Scholar 

  42. Axelrod, S. & Gomez-Bombarelli, R. Molecular machine learning with conformer ensembles. Preprint at https://arxiv.org/abs/2012.08452 (2020).

  43. Jiménez-Luna, J., Grisoni, F. & Schneider, G. Drug discovery with explainable artificial intelligence. Nat. Mach. Intell. 2, 573–584 (2020).

    Google Scholar 

  44. Jiménez-Luna, J., Skalic, M., Weskamp, N. & Schneider, G. Coloring molecules with explainable artificial intelligence for preclinical relevance assessment. J. Chem. Inf. Model. 61, 1083–1094 (2021).

    Google Scholar 

  45. Schnake, T. et al. Xai for graphs: explaining graph neural network predictions by identifying relevant walks. Preprint at https://arxiv.org/abs/2006.03589 (2020).

  46. Sun, M., Xing, J., Wang, H., Chen, B. & Zhou, J. MoCL: data-driven molecular fingerprint via knowledge-aware contrastive learning from molecular graph. In Proc. 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 3585–3594 (Association for Computing Machinery, 2021); https://doi.org/10.1145/3447548.3467186

  47. Coley, C. W., Barzilay, R., Green, W. H., Jaakkola, T. S. & Jensen, K. F. Convolutional embedding of attributed molecular graphs for physical property prediction. J. Chem. Inf. Model. 57, 1757–1772 (2017).

    Google Scholar 

  48. Li, Y., Vinyals, O., Dyer, C., Pascanu, R. & Battaglia, P. Learning deep generative models of graphs. Preprint at https://arxiv.org/abs/1803.03324 (2018).

  49. Simonovsky, M. & Komodakis, N. GraphVAE: towards generation of small graphs using variational autoencoders. In International Conference on Artificial Neural Networks Vol. 27, 412–422 (2018).

  50. De Cao, N. & Kipf, T. MolGAN: An implicit generative model for small molecular graphs. Preprint at https://arxiv.org/abs/1805.11973 (2018).

  51. Zhou, Z., Kearnes, S., Li, L., Zare, R. N. & Riley, P. Optimization of molecules via deep reinforcement learning. Sci. Rep. 9, 10752 (2019).

    Google Scholar 

  52. Jin, W., Barzilay, R. & Jaakkola, T. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning Vol. 35, 2323–2332 (2018).

  53. You, J., Liu, B., Ying, Z., Pande, V. & Leskovec, J. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems Vol. 31, 6410–6421 (2018).

  54. Jin, W., Barzilay, R. & Jaakkola, T. Multi-objective molecule generation using interpretable substructures. In International Conference on Machine Learning Vol. 37, 4849–4859 (2020).

  55. Lei, T., Jin, W., Barzilay, R. & Jaakkola, T. Deriving neural architectures from sequence and graph kernels. In International Conference on Machine Learning Vol. 34, 2024-2033 (2017).

  56. Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How powerful are graph neural networks? Preprint at https://arxiv.org/abs/1810.00826 (2018).

  57. Chen, Z., Chen, L., Villar, S. & Bruna, J. Can graph neural networks count substructures? In Advances in Neural Information Processing Systems Vol. 33, 10383–10395 (2020).

  58. Bouritsas, G., Frasca, F., Zafeiriou, S. & Bronstein, M. M. Improving graph neural network expressivity via subgraph isomorphism counting. Preprint at https://arxiv.org/abs/2006.09252 (2020).

  59. Bodnar, C. et al. Weisfeiler and Lehman go topological: message passing simplicial networks. In International Conference on Learning Representations: Workshop on Geometrical and Topological Representation Learning (2021).

  60. Thomas, N. et al. Tensor field networks: rotation-and translation-equivariant neural networks for 3D point clouds. Preprint at https://arxiv.org/abs/1802.08219 (2018).

  61. Satorras, V. G., Hoogeboom, E. & Welling, M. E(n) equivariant graph neural networks. Preprint at https://arxiv.org/abs/2102.09844 (2021).

  62. Anderson, B., Hy, T. S. & Kondor, R. Cormorant: covariant molecular neural networks. In Advances in Neural Information Processing Systems Vol. 32, 14537–14546 (2019).

  63. Miller, B. K., Geiger, M., Smidt, T. E. & Noé, F. Relevance of rotationally equivariant convolutions for predicting molecular properties. Preprint at https://arxiv.org/abs/2008.08461 (2020).

  64. Fuchs, F., Worrall, D., Fischer, V. & Welling, M. SE(3)-transformers: 3D roto-translation equivariant attention networks. In Advances in Neural Information Processing Systems Vol. 33 (2020).

  65. Unke, O. T. et al. SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects. Preprint at https://arxiv.org/abs/2105.00304 (2021).

  66. Batzner, S. et al. SE(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Preprint at https://arxiv.org/abs/2101.03164 (2021).

  67. Unke, O. T. et al. SE(3)-equivariant prediction of molecular wavefunctions and electronic densities. Preprint at https://arxiv.org/abs/2106.02347 (2021).

  68. Hermann, J., Schätzle, Z. & Noé, F. Deep-neural-network solution of the electronic Schrödinger equation. Nat. Chem. 12, 891–897 (2020).

    Google Scholar 

  69. Pfau, D., Spencer, J. S., Matthews, A. G. & Foulkes, W. M. C. Ab initio solution of the many-electron Schrödinger equation with deep neural networks. Phys. Rev. Res. 2, 033429 (2020).

    Google Scholar 

  70. Choo, K., Mezzacapo, A. & Carleo, G. Fermionic neural-network states for ab-initio electronic structure. Nat. Commun. 11, 2368 (2020).

    Google Scholar 

  71. Rajan, K., Zielesny, A. & Steinbeck, C. Decimer: towards deep learning for chemical image recognition. J. Cheminform. 12, 65 (2020).

    Google Scholar 

  72. Cramer, R. D., Patterson, D. E. & Bunce, J. D. Comparative molecular field analysis (comfa). 1. effect of shape on binding of steroids to carrier proteins. J. Am. Chem. Soc. 110, 5959–5967 (1988).

    Google Scholar 

  73. Klebe, G. in 3D QSAR in Drug Design (eds. Kubinyi, H. et al.) 87–104 (Springer, 1998).

  74. Jiménez, J., Skalic, M., Martinez-Rosell, G. & De Fabritiis, G. KDEEP: protein–ligand absolute binding affinity prediction via 3d-convolutional neural networks. J. Chem. Inf. Model. 58, 287–296 (2018).

    Google Scholar 

  75. Ragoza, M., Hochuli, J., Idrobo, E., Sunseri, J. & Koes, D. R. Protein–ligand scoring with convolutional neural networks. J. Chem. Inf. Model. 57, 942–957 (2017).

    Google Scholar 

  76. Jiménez, J., Doerr, S., Martinez-Rosell, G., Rose, A. S. & De Fabritiis, G. DeepSite: protein-binding site predictor using 3d-convolutional neural networks. Bioinformatics 33, 3036–3042 (2017).

    Google Scholar 

  77. Ahmed, E. et al. A survey on deep learning advances on different 3d data representations. Preprint at https://arxiv.org/abs/1808.01462 (2018).

  78. Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A. & Battaglia, P. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations Vol. 8 (2020).

  79. Liu, Q. et al. OctSurf: efficient hierarchical voxel-based molecular surface representation for protein-ligand affinity prediction. J. Molec. Graph. Model. 105, 107865 (2021).

    Google Scholar 

  80. Mylonas, S. K., Axenopoulos, A. & Daras, P. DeepSurf: a surface-based deep learning approach for the prediction of ligand binding sites on proteins. Preprint at https://arxiv.org/abs/2002.05643 (2020).

  81. Wiswesser, W. J. Historic development of chemical notations. J. Chem. Inf. Comput. Sci. 25, 258–263 (1985).

    Google Scholar 

  82. Wiswesser, W. J. The Wiswesser line formula notation. Chem. Eng. News Arch. 30, 3523–3526 (1952).

    Google Scholar 

  83. Heller, S., McNaught, A., Stein, S., Tchekhovskoi, D. & Pletnev, I. InChI: the worldwide chemical structure identifier standard. J. Cheminform. 5, 7 (2013).

    Google Scholar 

  84. Weininger, D. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. J. Chem. Inf. Comput. Sci. 28, 31–36 (1988).

    Google Scholar 

  85. Öztürk, H., Özgür, A., Schwaller, P., Laino, T. & Ozkirimli, E. Exploring chemical space using natural language processing methodologies for drug discovery. Drug Discov. Today 25, 689–705 (2020).

    Google Scholar 

  86. Cadeddu, A., Wylie, E. K., Jurczak, J., Wampler-Doty, M. & Grzybowski, B. A. Organic chemistry as a language and the implications of chemical linguistics for structural and retrosynthetic analyses. Angew. Chem. Int. Ed. 53, 8108–8112 (2014).

    Google Scholar 

  87. Schwaller, P., Gaudin, T., Lanyi, D., Bekas, C. & Laino, T. Found in translation: predicting outcomes of complex organic chemistry reactions using neural sequence-to-sequence models. Chem. Sci. 9, 6091–6098 (2018).

    Google Scholar 

  88. O’Boyle, N. M. Towards a universal SMILES representation: a standard method to generate canonical SMILES based on the InChI. J. Cheminform. 4, 22 (2012).

    Google Scholar 

  89. Weininger, D., Weininger, A. & Weininger, J. L. SMILES. 2. Algorithm for generation of unique SMILES notation. J. Chem. Inf. Comput. Sci. 29, 97–101 (1989).

    MATH  Google Scholar 

  90. Gómez-Bombarelli, R. et al. Automatic chemical design using a data-driven continuous representation of molecules. ACS Centr. Sci. 4, 268–276 (2018).

    Google Scholar 

  91. O’Boyle, N. & Dalke, A. DeepSmiles: An adaptation of smiles for use in machine-learning of chemical structures. Preprint at https://doi.org/10.26434/chemrxiv.7097960.v1 (2018).

  92. Krenn, M., Häse, F., Nigam, A., Friederich, P. & Aspuru-Guzik, A. Self-referencing embedded strings (SELFIES): a 100% robust molecular string representation. Mach. Learn. 1, 045024 (2020).

    Google Scholar 

  93. Skinnider, M. A., Stacey, R. G., Wishart, D. S. & Foster, L. J. Chemical language models enable navigation in sparsely populated chemical space. Nat. Mach. Intell. 3, 759–770 (2021).

  94. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning Internal Representations by Error Propagation (eds. Rumelhart, D. E. & McClelland, J. L.) 318–362 (MIT Press, 1985).

  95. Hochreiter, S. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncert. Fuzz. Knowl. Based Syst. 6, 107–116 (1998).

    MATH  Google Scholar 

  96. Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neur. Comput. 9, 1735–1780 (1997).

    Google Scholar 

  97. Chung, J., Gulcehre, C., Cho, K. & Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. Preprint at https://arxiv.org/abs/1412.3555 (2014).

  98. Yuan, W. et al. Chemical space mimicry for drug discovery. J. Chem. Inf. Model. 57, 875–882 (2017).

    Google Scholar 

  99. Gupta, A. et al. Generative recurrent networks for de novo drug design. Mol. Inform. 37, 1700111 (2018).

    Google Scholar 

  100. Merk, D., Friedrich, L., Grisoni, F. & Schneider, G. De novo design of bioactive small molecules by artificial intelligence. Mol. Inform. 37, 1700153 (2018).

    Google Scholar 

  101. Olivecrona, M., Blaschke, T., Engkvist, O. & Chen, H. Molecular de-novo design through deep reinforcement learning. J. Cheminform. 9, 48 (2017).

    Google Scholar 

  102. Popova, M., Isayev, O. & Tropsha, A. Deep reinforcement learning for de novo drug design. Sci. Adv. 4, eaap7885 (2018).

    Google Scholar 

  103. Grisoni, F. et al. Combining generative artificial intelligence and on-chip synthesis for de novo drug design. Sci. Adv. 7, eabg3338 (2021).

    Google Scholar 

  104. Arús-Pous, J. et al. Randomized SMILES strings improve the quality of molecular generative models. J. Cheminform. 11, 71 (2019).

    Google Scholar 

  105. Bjerrum, E. J. & Threlfall, R. Molecular generation with recurrent neural networks (RNNs). Preprint at https://arxiv.org/abs/1705.04612 (2017).

  106. Grisoni, F., Moret, M., Lingwood, R. & Schneider, G. Bidirectional molecule generation with recurrent neural networks. J. Chem. Inf. Model. 60, 1175–1183 (2020).

    Google Scholar 

  107. Nagarajan, D. et al. Computational antimicrobial peptide design and evaluation against multidrug-resistant clinical isolates of bacteria. J. Biol. Chem. 293, 3492–3509 (2018).

    Google Scholar 

  108. Grisoni, F. et al. Designing anticancer peptides by constructive machine learning. ChemMedChem 13, 1300–1302 (2018).

    Google Scholar 

  109. Das, P. et al. Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations. Nat. Biomed. Eng. 5, 613–623 (2021).

    Google Scholar 

  110. Zheng, S., Li, Y., Chen, S., Xu, J. & Yang, Y. Predicting drug–protein interaction using quasi-visual question answering system. Nat. Mach. Intell. 2, 134–140 (2020).

    Google Scholar 

  111. Wang, X. et al. Optimizing pharmacokinetic property prediction based on integrated datasets and a deep learning approach. J. Chem. Inf. Model. 60, 4603–4613 (2020).

    Google Scholar 

  112. Senior, A. W. et al. Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13). Proteins 87, 1141–1148 (2019).

    Google Scholar 

  113. Tsai, S.-T., Kuo, E.-J. & Tiwary, P. Learning molecular dynamics with simple language model built upon long short-term memory neural network. Nat. Commun. 11, 5155 (2020).

    Google Scholar 

  114. Gomez-Bombarelli, R. et al. Automatic chemical design using a data-driven continuous representation of molecules. ACS Centr. Sci. 4, 268–276 (2018).

    Google Scholar 

  115. Lin, X., Quan, Z., Wang, Z.-J., Huang, H. & Zeng, X. A novel molecular representation with BiGRU neural networks for learning atom. Brief. Bioinform. 21, 2099–2111 (2019).

    Google Scholar 

  116. Preuer, K., Renz, P., Unterthiner, T., Hochreiter, S. & Klambauer, G. Fréchet ChemNet distance: a metric for generative models for molecules in drug discovery. J. Chem. Inf. Model. 58, 1736–1741 (2018).

    Google Scholar 

  117. Schwaller, P. et al. Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction. ACS Centr. Sci. 5, 1572–1583 (2019).

    Google Scholar 

  118. Schwaller, P. et al. Predicting retrosynthetic pathways using transformer-based models and a hyper-graph exploration strategy. Chemical Science 11, 3316–3325 (2020).

    Google Scholar 

  119. Pesciullesi, G., Schwaller, P., Laino, T. & Reymond, J.-L. Transfer learning enables the molecular transformer to predict regio-and stereoselective reactions on carbohydrates. Nat. Commun. 11, 4874 (2020).

    Google Scholar 

  120. Kreutter, D., Schwaller, P. & Reymond, J.-L. Predicting enzymatic reactions with a molecular transformer. Chem. Sci. 12, 8648–8659 (2021).

    Google Scholar 

  121. Schwaller, P., Vaucher, A. C., Laino, T. & Reymond, J.-L. Prediction of chemical reaction yields using deep learning. Mach. Learn. 2, 015016 (2021).

    Google Scholar 

  122. Schwaller, P. et al. Mapping the space of chemical reactions using attention-based neural networks. Nat. Mach. Intell. 3, 144–152 (2021).

  123. Morris, P., St. Clair, R., Hahn, W. E. & Barenholtz, E. Predicting binding from screening assays with transformer network embeddings. J. Chem. Inf. Model. 60, 4191–4199 (2020).

    Google Scholar 

  124. Rong, Y. et al. Self-supervised graph transformer on large-scale molecular data. In Advances in Neural Information Processing Systems Vol. 33, 12559–12571 (2020).

  125. Grechishnikova, D. Transformer neural network for protein-specific de novo drug generation as a machine translation problem. Sci. Rep. 11, 321 (2021).

    Google Scholar 

  126. Baek, M. et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science 373, 871–876 (2021).

    Google Scholar 

  127. Méndez-Lucio, O., Baillif, B., Clevert, D.-A., Rouquié, D. & Wichard, J. De novo generation of hit-like molecules from gene expression signatures using artificial intelligence. Nat. Commun. 11, 10 (2020).

    Google Scholar 

  128. Griffiths, R.-R. & Hernández-Lobato, J. M. Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chem. Sci. 11, 577–586 (2020).

    Google Scholar 

  129. Hirohara, M., Saito, Y., Koda, Y., Sato, K. & Sakakibara, Y. Convolutional neural network based on SMILES representation of compounds for detecting chemical motif. BMC Bioinform. 19, 83–94 (2018).

    Google Scholar 

  130. Kimber, T. B., Engelke, S., Tetko, I. V., Bruno, E. & Godin, G. Synergy effect between convolutional neural networks and the multiplicity of SMILES for improvement of molecular prediction. Preprint at https://arxiv.org/abs/1812.04439 (2018).

  131. Zheng, S., Yan, X., Yang, Y. & Xu, J. Identifying structure–property relationships through smiles syntax analysis with self-attention mechanism. J. Chem. Inf. Model. 59, 914–923 (2019).

    Google Scholar 

  132. ElAbd, H. et al. Amino acid encoding for deep learning applications. BMC Bioinform. 21, 12 (2020).

    Google Scholar 

  133. Satorras, V. G., Hoogeboom, E., Fuchs, F. B., Posner, I. & Welling, M. E(n) equivariant normalizing flows for molecule generation in 3d. In Advances in Neural Information Processing Systems Vol. 33 (2021).

  134. Gebauer, N. W., Gastegger, M., Hessmann, S. S., Müller, K.-R. & Schütt, K. T. Inverse design of 3d molecular structures with conditional generative neural networks. Preprint at https://arxiv.org/abs/2109.04824 (2021).

  135. Fujita, T. & Winkler, D. A. Understanding the roles of the “two QSARs”. J. Chem. Inf. Model. 56, 269–274 (2016).

    Google Scholar 

  136. Hu, W. et al. Open Graph Benchmark: datasets for machine learning on graphs. In Advances in Neural Information Processing Systems Vol. 33, 22118–22133 (2020).

  137. Wu, Z. et al. MoleculeNet: a benchmark for molecular machine learning. Chem. Sci. 9, 513–530 (2018).

    Google Scholar 

  138. Polykovskiy, D. et al. Molecular sets (MOSES): a benchmarking platform for molecular generation models. Front. Pharmacol. 11, 565644 (2020).

    Google Scholar 

  139. Brown, N., Fiscato, M., Segler, M. H. & Vaucher, A. C. GuacaMol: benchmarking models for de novo molecular design. J. Chem. Inf. Model. 59, 1096–1108 (2019).

    Google Scholar 

  140. von Lilienfeld, O. A., Müller, K.-R. & Tkatchenko, A. Exploring chemical compound space with quantum-based machine learning. Nat. Rev. Chem. 12, 945–951 (2020).

    Google Scholar 

  141. Unke, O. T. et al. Machine learning force fields. Chem. Rev. 121, 10142–10186 (2021).

    Google Scholar 

  142. Isert, C., Atz, K., Jiménez-Luna, J. & Schneider, G. QMugs: quantum mechanical properties of drug-like molecules. Preprint at https://arxiv.org/abs/2107.00367 (2021).

  143. Ramakrishnan, R., Dral, P. O., Rupp, M. & Von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 140022 (2014).

    Google Scholar 

  144. Jin, W., Coley, C., Barzilay, R. & Jaakkola, T. Predicting organic reaction outcomes with weisfeiler-lehman network. In Advances in Neural Information Processing Systems Vol. 30, 2607–2616 (2017).

  145. LeCun, Y. et al. in The Handbook of Brain Theory and Neural Networks (ed. Arbib, M. A.) 255–258 (MIT Press, 1995).

  146. Sutton, R. S. & Barto, A. G.Reinforcement Learning: An Introduction (MIT Press, 2018).

  147. Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2009).

    Google Scholar 

  148. Nguyen, L. A., He, H. & Pham-Huy, C. Chiral drugs: an overview. Int. J. Biomed. Sci. 2, 85 (2006).

    Google Scholar 

  149. Valsecchi, C., Grisoni, F., Motta, S., Bonati, L. & Ballabio, D. Nura: a curated dataset of nuclear receptor modulators. Toxicol. Appl. Pharmacol. 407, 115244 (2020).

    Google Scholar 

  150. Bemis, G. W. & Murcko, M. A. The properties of known drugs. 1. Molecular frameworks. J. Med. Chem. 39, 2887–2893 (1996).

    Google Scholar 

Download references

Acknowledgements

This research was supported by the Swiss National Science Foundation (SNSF, grant no. 205321_182176) and the ETH RETHINK initiative.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Francesca Grisoni or Gisbert Schneider.

Ethics declarations

Competing interests

G.S. declares a potential financial conflict of interest as co-founder of inSili.com LLC, Zurich, and in his role as scientific consultant to the pharmaceutical industry.

Peer review information

Nature Machine Intelligence thanks Jonathan Hirst and Oliver Wieder for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Atz, K., Grisoni, F. & Schneider, G. Geometric deep learning on molecular representations. Nat Mach Intell 3, 1023–1032 (2021). https://doi.org/10.1038/s42256-021-00418-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-021-00418-8

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing