· · ·
Gemma 2: Improving Open Language Models at a Practical Size
Gemma 2 — Google DeepMind — 2024-06
Google DeepMind's open-weights model family at 2B, 9B, and 27B parameters, distilled from the Gemini line. The report covers architectural choices including local–global attention interleaving and logit soft-capping.
References
- arXiv arxiv.org/abs/2408.00118
- Org page Google DeepMind
- Released 2024-06
Credited authors (197)
Abe Friesen
Alanna Walton
Alek Andreev
Alexandre Ramé
Aliaksei Severyn
Alicia Parrish
Aliya Ahmad
Allen Hutchison
Alvin Abdagic
Amanda Carl
Amy Shen
Anand Rao
Anca Dragan
Andy Brock
Andy Coenen
Anthony Laforge
Anton Tsitsulin
Antonia Paterson
Armand Joulin
Behnam Neyshabur
Ben Bastian
Bilal Piot
Bo Wu
Bobak Shahriari
Brandon Royal
Cassidy Hardin
Charlie Chen
Charline Le Lan
Chintu Kumar
Chris Perry
Chris Welty
Christopher A. Choquette-Choo
Clement Farabet
D. Sculley
Danila Sinopalnikov
David Weinberger
Demis Hassabis
Dimple Vijaykumar
Dominika Rogozińska
Dustin Herbison
Elena Buchatskaya
Eli Collins
Elisa Bandy
Emma Wang
Eric Noland
Erica Moreira
Evan Senter
Evgenii Eltyshev
Francesco Visin
Gabriel Rasskin
Gary Wei
Glenn Cameron
Gus Martins
Hadi Hashemi
Hanna Klimczak-Plucińska
Harleen Batra
Harsh Dhand
Ivan Nardini
Jacinda Mein
Jack Zhou
James Svensson
Jean-Bastien Grill
Jeanine Banks
Jeff Dean
Jeff Stanway
Jetha Chan
Jin Peng Zhou
Joana Carrasqueira
Joana Iljazi
Jocelyn Becker
Joe Fernandez
Joelle Barral
Johan Ferret
Joost van Amersfoort
Josh Gordon
Josh Lipschultz
Josh Newlan
Ju-yeong Ji
Kareem Mohamed
Kartikeya Badola
Kat Black
Kathleen Kenealy
Katie Millican
Keelin McDonell
Kelvin Nguyen
Kiranbir Sodhia
Kish Greene
Koray Kavukcuoglu
Lars Lowe Sjoesund
Lauren Usui
Laurent Sifre
Lena Heuermann
Leticia Lago
Lilly McNealus
Livio Baldini Soares
Logan Kilpatrick
Lucas Dixon
Luciano Martins
Ludovic Peran
Léonard Hussenot
Machel Reid
Manvinder Singh
Mark Iverson
Martin Görner
Mat Velloso
Mateo Wirth
Matt Davidow
Matt Hoffman
Matt Miller
Matthew Rahtz
Matthew Watson
Meg Risdal
Mehran Kazemi
Michael Moynihan
Michelle Casbon
Ming Zhang
Minh Giang
Minsuk Kahng
Minwoo Park
Mofi Rahman
Mohit Khatwani
Morgane Riviere
Natalie Dao
Nenshad Bardoliwalla
Nesh Devanathan
Neta Dumai
Nikola Momchev
Nilay Chauhan
Nino Vieillard
Noah Fiedel
Olivier Bachem
Oriol Vinyals
Oscar Wahltinez
Pankil Botarda
Parker Barnes
Paul Barham
Paul Michel
Pengchong Jin
Peter Liu
Petko Georgiev
Phil Culliton
Phoebe Kirk
Pier Giuseppe Sessa
Piotr Stanczyk
Pouya Tafti
Pradeep Kuppala
Raia Hadsell
Ramona Comanescu
Ramona Merhej
Ravin Kumar
Reena Jana
Reza Ardeshir Rokni
Rishabh Agarwal
Robert Dadashi
Ryan Mullins
Sabela Ramos
Samaneh Saadat
Sammy Jerome
Sara Mc Carthy
Sarah Cogan
Sarah Perrin
Sebastian Borgeaud
Sebastian Krause
Sertan Girgin
Shantanu Thakoor
Shengyang Dai
Shreya Pathak
Shruti Garg
Shruti Sheth
Slav Petrov
Sue Ronstrom
Surya Bhupatiraju
Susan Chan
Sébastien M. R. Arnold
Thomas Mesnard
Timothy Jordan
Ting Yu
Tom Eccles
Tom Hennigan
Tomas Kocisky
Tris Warkentin
Tulsee Doshi
Victor Cotruta
Vihan Jain
Vikas Yadav
Vilobh Meshram
Vishal Dharmadhikari
Warren Barkley
Wei Wei
Wenming Ye
Woohyun Han
Woosuk Kwon
Xiang Xu
Zhe Shen
Zhitao Gong
Zichuan Wei
Zoubin Ghahramani