In this seminar, we will discuss recent papers and methods to tackle the large compute budget required by modern AI algorithms. In this semester, we will focus on efficient transformer architectures and alternatives.