Meta Released LLaMA, an Open Large Language Model with 65-Billion-Parameters

By
IBL News
-

IBL News | New York

Meta, formerly known as Facebook, announced the release of its large language model called LLaMA (Large Language Model Meta AI), with 65 billion parameters, yesterday. Analysts saw this move as CEO Mark Zuckerberg’s answer to ChatGPT.

“It’s designed to help researchers advance their work in this subfield of AI,” said Meta. “It’s part of Meta’s commitment to open science.”

“Training smaller foundation models like LLaMA is desirable in the large language model space because it requires far less computing power and resources to test new approaches, validate others’ work, and explore new use cases.”

Meta is making LLaMA available in several sizes (7B, 13B, 33B, and 65B parameters). It also shared a LLAMA model card that details how the company built the model in keeping with an approach to Responsible AI practices.

Meta trained LLaMA 65B and LLaMA 33B on 1.4 trillion tokens. Their smallest model, LLaMA 7B, is trained on one trillion tokens.

Over the last year, large language models — natural language processing (NLP) systems with billions of parameters — have shown new capabilities to generate creative text, solve mathematical theorems, predict protein structures, answer reading comprehension questions, and more.

Like other large language models, LLaMA works by taking a sequence of words as input and predicting the next word to recursively generate text.

To train its model, Meta chose a text from the 20 languages with the most speakers, focusing on those with Latin and Cyrillic alphabets. Meta released its model under a noncommercial license focused on research use cases.

Access to the model will be granted on a case-by-case basis to academic researchers; those affiliated with organizations in government, civil society, and academia; and industry research laboratories around the world.

Those hoping to access LLaMA for themselves can apply via this request form.

CEO Mark Zuckerberg described his company’s contribution to the buzzy AI technology sphere in a Facebook post.

“Today we’re releasing a new state-of-the-art AI large language model called LLaMA designed to help researchers advance their work.”