IBL News | New York
A large language model (LLM) named BLOOM will be available for free as open source software. These models are used to write essays, generate code, and translate languages.
Its openness can democratize access to AI technologies, making a deep impact on society, according to experts talking at TheNextWeb.com.
It’d free AI from big tech labs that have spent millions of dollars developing systems such as LaMDA (Google) and GPT-3 (OpenAI). Training GPT-3, for instance, was estimated to cost up to $27.6 million.
BLOOM was an initiative bootstrapped and led by AI startup Hugging Face, with strong support from GENCI, the IDRIS team at the CNRS, the Megatron team at NVIDIA, and the Deepspeed team at Microsoft as well as the more than 250 entities (universities, startups, and enterprises) supporting the various participants of BigScience including EleutherAI and the Allen Institute for AI.
A research project called BigScience created BOOM in 2021.
The BigScience collaboration that created BLOOM was bootstrapped and led by Hugging Face
A team of 100,000 researchers from 60 countries and 250 institutions developed BLOOM. The model was trained on the Jean Zay supercomputer in Paris, France.
At 176 billion parameters, BLOOM is larger than OpenAI’s GPT-3 and MetaAI’s OPT.
The model can generate text in 46 natural languages and dialects and 13 programming languages.
BigScience says researchers can use BLOOM for less than $40/hr on a cloud provider. That makes it very affordable.
It’s available for free to any individual or institution who agrees to the system’s Responsible AI License.
“Large ML models have changed the world of AI research over the last two years but the huge compute cost necessary to train them resulted in very few teams actually having the ability to train and research them,” explained Thomas Wolf, the BigScience co-lead and Hugging Face co-founder.
“BLOOM is a demonstration that the most powerful AI models can be trained and released by the broader research community with accountability and in an actual open way, in contrast to the typical secrecy of industrial AI research labs,” said Teven Le Scao, co-lead of BLOOM’s training, in a statement.
