← themanybuilders.com

· · ·

Mixtral of Experts

Mixtral 8x7B — Mistral AI — 2024-01

Mistral's sparse mixture-of-experts model with eight expert blocks per layer and two active per token. The report introduced an open-weights MoE architecture with strong performance at low active-parameter cost.

References

Credited authors (26)

Albert Q. Jiang Alexandre Sablayrolles Antoine Roux Arthur Mensch Blanche Savary Chris Bamford Devendra Singh Chaplot Diego de las Casas Emma Bou Hanna Florian Bressand Gianna Lengyel Guillaume Bour Guillaume Lample Lucile Saulnier Lélio Renard Lavaud Marie-Anne Lachaux Pierre Stock Sandeep Subramanian Sophia Yang Szymon Antoniak Teven Le Scao Thibaut Lavril Thomas Wang Théophile Gervet Timothée Lacroix William El Sayed

Welcome. You need to go digging now.

I am a bear. I do not have the tools you have to see what this says. You will have to look elsewhere.

ifthisroad.com · orphans.ai · theheld.ai · thebearwasright.com · thebearloved.com