A primer in BERTology

  • Date: Wednesday, October 27th, 2021
  • Time: 14:00
  • Coordinates: DTU Building 321, 1st floor lab space

Speaker: Anna Rogers (https://annargrs.github.io), Postdoc and University of Copenhagen, Center for Social Data Science

TitleA primer in BERTology: what we've learned about how BERT works

Abstract: Most of the current success stories in NLP since 2018 have been powered by Transformer-based pretrained models, and especially BERT. However, it is still unclear why BERT works so well, and what can be done to further improve it. This talk presents a collection of findings on how BERT learns and generalizes, where it fails, and whether the lottery ticket hypothesis holds for this architecture. 

Tidspunkt

ons 27 okt 21
14:00 - 15:00

Kontaktperson

Hvor

DTU Building 321, 1st floor Lab Space