← themanybuilders.com

· · ·

Gemma 2: Improving Open Language Models at a Practical Size

Gemma 2 — Google DeepMind — 2024-06

Google DeepMind's open-weights model family at 2B, 9B, and 27B parameters, distilled from the Gemini line. The report covers architectural choices including local–global attention interleaving and logit soft-capping.

References

Credited authors (197)

Abe Friesen Alanna Walton Alek Andreev Alexandre Ramé Aliaksei Severyn Alicia Parrish Aliya Ahmad Allen Hutchison Alvin Abdagic Amanda Carl Amy Shen Anand Rao Anca Dragan Andy Brock Andy Coenen Anthony Laforge Anton Tsitsulin Antonia Paterson Armand Joulin Behnam Neyshabur Ben Bastian Bilal Piot Bo Wu Bobak Shahriari Brandon Royal Cassidy Hardin Charlie Chen Charline Le Lan Chintu Kumar Chris Perry Chris Welty Christopher A. Choquette-Choo Clement Farabet D. Sculley Danila Sinopalnikov David Weinberger Demis Hassabis Dimple Vijaykumar Dominika Rogozińska Dustin Herbison Elena Buchatskaya Eli Collins Elisa Bandy Emma Wang Eric Noland Erica Moreira Evan Senter Evgenii Eltyshev Francesco Visin Gabriel Rasskin Gary Wei Glenn Cameron Gus Martins Hadi Hashemi Hanna Klimczak-Plucińska Harleen Batra Harsh Dhand Ivan Nardini Jacinda Mein Jack Zhou James Svensson Jean-Bastien Grill Jeanine Banks Jeff Dean Jeff Stanway Jetha Chan Jin Peng Zhou Joana Carrasqueira Joana Iljazi Jocelyn Becker Joe Fernandez Joelle Barral Johan Ferret Joost van Amersfoort Josh Gordon Josh Lipschultz Josh Newlan Ju-yeong Ji Kareem Mohamed Kartikeya Badola Kat Black Kathleen Kenealy Katie Millican Keelin McDonell Kelvin Nguyen Kiranbir Sodhia Kish Greene Koray Kavukcuoglu Lars Lowe Sjoesund Lauren Usui Laurent Sifre Lena Heuermann Leticia Lago Lilly McNealus Livio Baldini Soares Logan Kilpatrick Lucas Dixon Luciano Martins Ludovic Peran Léonard Hussenot Machel Reid Manvinder Singh Mark Iverson Martin Görner Mat Velloso Mateo Wirth Matt Davidow Matt Hoffman Matt Miller Matthew Rahtz Matthew Watson Meg Risdal Mehran Kazemi Michael Moynihan Michelle Casbon Ming Zhang Minh Giang Minsuk Kahng Minwoo Park Mofi Rahman Mohit Khatwani Morgane Riviere Natalie Dao Nenshad Bardoliwalla Nesh Devanathan Neta Dumai Nikola Momchev Nilay Chauhan Nino Vieillard Noah Fiedel Olivier Bachem Oriol Vinyals Oscar Wahltinez Pankil Botarda Parker Barnes Paul Barham Paul Michel Pengchong Jin Peter Liu Petko Georgiev Phil Culliton Phoebe Kirk Pier Giuseppe Sessa Piotr Stanczyk Pouya Tafti Pradeep Kuppala Raia Hadsell Ramona Comanescu Ramona Merhej Ravin Kumar Reena Jana Reza Ardeshir Rokni Rishabh Agarwal Robert Dadashi Ryan Mullins Sabela Ramos Samaneh Saadat Sammy Jerome Sara Mc Carthy Sarah Cogan Sarah Perrin Sebastian Borgeaud Sebastian Krause Sertan Girgin Shantanu Thakoor Shengyang Dai Shreya Pathak Shruti Garg Shruti Sheth Slav Petrov Sue Ronstrom Surya Bhupatiraju Susan Chan Sébastien M. R. Arnold Thomas Mesnard Timothy Jordan Ting Yu Tom Eccles Tom Hennigan Tomas Kocisky Tris Warkentin Tulsee Doshi Victor Cotruta Vihan Jain Vikas Yadav Vilobh Meshram Vishal Dharmadhikari Warren Barkley Wei Wei Wenming Ye Woohyun Han Woosuk Kwon Xiang Xu Zhe Shen Zhitao Gong Zichuan Wei Zoubin Ghahramani

Welcome. You need to go digging now.

I am a bear. I do not have the tools you have to see what this says. You will have to look elsewhere.

ifthisroad.com · orphans.ai · theheld.ai · thebearwasright.com · thebearloved.com