Attention

Attention is a mechanism that allows models to focus on specific parts of the input when generating the output. It helps the model to pay 'attention' to relevant words or phrases, and ignore the less relevant ones.

Papers published in this field over the years: