Jump to main content
Jump to site search

Issue 24, 2021
Previous Article Next Article

Attention-based generative models for de novo molecular design

Author affiliations

Abstract

Attention mechanisms have led to many breakthroughs in sequential data modeling but have yet to be incorporated into any generative algorithms for molecular design. Here we explore the impact of adding self-attention layers to generative β-VAE models and show that those with attention are able to learn a complex “molecular grammar” while improving performance on downstream tasks such as accurately sampling from the latent space (“model memory”) or exploring novel chemistries not present in the training data. There is a notable relationship between a model's architecture, the structure of its latent memory and its performance during inference. We demonstrate that there is an unavoidable tradeoff between model exploration and validity that is a function of the complexity of the latent memory. However, novel sampling schemes may be used that optimize this tradeoff. We anticipate that attention will play an important role in future molecular design algorithms that can make efficient use of the detailed molecular substructures learned by the transformer.

Graphical abstract: Attention-based generative models for de novo molecular design

Back to tab navigation

Supplementary files

Article information


Submitted
22 Feb 2021
Accepted
14 May 2021
First published
14 May 2021

This article is Open Access
All publication charges for this article have been paid for by the Royal Society of Chemistry

Chem. Sci., 2021,12, 8362-8372
Article type
Edge Article

Attention-based generative models for de novo molecular design

O. Dollar, N. Joshi, D. A. C. Beck and J. Pfaendtner, Chem. Sci., 2021, 12, 8362
DOI: 10.1039/D1SC01050F

This article is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported Licence. Material from this article can be used in other publications provided that the correct acknowledgement is given with the reproduced material and it is not used for commercial purposes.

Reproduced material should be attributed as follows:

  • For reproduction of material from NJC:
    [Original citation] - Published by The Royal Society of Chemistry (RSC) on behalf of the Centre National de la Recherche Scientifique (CNRS) and the RSC.
  • For reproduction of material from PCCP:
    [Original citation] - Published by the PCCP Owner Societies.
  • For reproduction of material from PPS:
    [Original citation] - Published by The Royal Society of Chemistry (RSC) on behalf of the European Society for Photobiology, the European Photochemistry Association, and RSC.
  • For reproduction of material from all other RSC journals:
    [Original citation] - Published by The Royal Society of Chemistry.

Information about reproducing material from RSC articles with different licences is available on our Permission Requests page.


Social activity

Search articles by author

Spotlight

Advertisements