Intellification – All you want to know about Machine Learning
About me My publications and talks Privacy policy Machine Learning Terms Explained In this glossary, I will briefly list and explain terms used in the Machine Learning literature including links to more information and important publications. Whenever I come along a term I do not know or being unsure about and then doing some research to truly understand the term, I will update this glossary by this term. So expect this glossary to grow over time 🙂
Attention is the key concept in transformers . It enables the network to model dependencies between different tokens or features in a sequence. The attention mechanism was first introduced for image recognition by Volodymyr Mnih, Nicolas Heess, Alex Graves, and Koray Kavukcuoglu in 2014 on NIPS: “Recurrent models of visual attention” . The first time attention was used for language processing (machine translation) was in the paper from D. Bahdanau, K. Cho, and Yoshua Bengio in 2015: “Neural Machine Translat
Further reading:
Intellification – All you want to know about Machine Learning