Open-Source Initiatives Challenge Closed, Proprietary AI Systems With New LLMs

By
IBL News
-

IBL News | New York

Several startups, collectives, and academics have released a wave of new large language models (LLMs) as open source, trying to challenge the closed, proprietary AI systems such as OpenAI and Anthropic.

These private organizations, knowing that state-of-the-art LLMs require huge compute budgets — OpenAI reportedly used 10,000 Nvidia GPUs to train ChatGPT — and deep ML expertise have refused to open up their models. They rely on API distribution instead.

The data, source code, or deep learning programming, of the model weights, remain hidden from public scrutiny.

Open-source initiatives state that they are seeking to democratize access to LLMs.

Two weeks ago, Databricks announced the ChatGPT-type Dolly, which was inspired by Alpaca, another open-source LLM released by Stanford in mid-March.

Stanford’s Alpaca used the weights from Meta’s LLaMA model that was released in late February.

LLaMA was hailed for its superior performance over models such as GPT–3, despite having ten times fewer parameters.

Other open-source LLaMA-inspired models have been released in recent weeks, such as:

– Vicuna, a fine-tuned version of LLaMA that apparently matches GPT-4 performance;

– Koala, a model from Berkeley AI Research Institute;

– ColossalChat, a ChatGPT-type model part of the Colossal-AI project from UC Berkeley.

Some of these open-source models have even been optimized to run on the lowest-powered devices, from a MacBook Pro down to a Raspberry Pi and an old iPhone.

However, none of these open-source LLMs are available yet for commercial use, as the LLaMA model is not released for commercial use.

In addition, the OpenAI GPT-3.5 terms of use prohibit using the model to develop AI models that compete with OpenAI.

In March, the free-software community Mozilla announced an open-source initiative for developing AI, saying they “intend to create a decentralized AI community that can serve as a ‘counterweight’ against the large profit-focused companies.”

.