IBL News | New York
Stability AI, the startup behind the generative AI art tool Stable Diffusion, released this week an open-source LLM with 15 to 65 billion parameters to follow.
StableLM can generate text and code and is an alternative to proprietary OpenAI. “It’s a massive deal, that has the potential to change up everything in AI,” said an expert.
It can be used for commercial or research purposes, under the CC BY-SA-4.0 license.
This release is available in “alpha” on its GitHub and Hugging Spaces, a platform for hosting AI models and code.
It builds on the company’s open-sourcing record with EleutherAI, a nonprofit research hub.
In 2022, it released the image model Stable Diffusion. Earlier releases included GPT-J, GPT-NeoX, and the Pythia suite, which were trained on The Pile open-source dataset — a mix of internet-scraped text samples from websites including PubMed, StackExchange, and Wikipedia.
These open-source models, including Cerebras-GPT and Dolly-2, might demonstrate that small and efficient models can deliver performance with appropriate training, according to experts.
StableLM is trained on a new experimental dataset built on The Pile, but three times larger with 1.5 trillion tokens of content.
“The richness of this dataset gives StableLM surprisingly high performance in conversational and coding tasks, despite its small size of 3 to 7 billion parameters (by comparison, GPT-3 has 175 billion parameters),” said the company in a blog post.
Stability is also releasing a set of research models that are instruction fine-tuned. They use a combination of five recent, noncommercial, research-only, open-source datasets: Alpaca, GPT4All, Dolly, ShareGPT, and HH.
“Language models will form the backbone of our digital economy, and we want everyone to have a voice in their design,” said Stability AI.
In addition, Stability is working on a crowd-sourced RLHF program and an open-source dataset for AI assistants, Open Assistant.
Stability AI didn’t say whether StableLM hallucinates and includes offensive language and views. “Given that The Pile contains profane, lewd, and otherwise fairly abrasive language, it wouldn’t be surprising if that were the case,” wrote TechCrunch.
“This is expected to be improved with scale, better data, community feedback, and optimization,” responded Stability.
Even commercialized models like GPT-4, which have filters and human moderation teams in place, have been shown to spout toxicity.
Some researchers have criticized the release of open-source models, arguing that they could be used for unsavory purposes like creating phishing emails or aiding malware attacks.
According to Semafor, Stability AI — which raised over $100 million at a valuation of $1 billion — is under pressure to monetize its efforts. Its CEO Emad Mostaque has hinted at plans to IPO.
The company is facing legal cases that allege that it infringed on the rights of millions of artists by developing AI art tools using web-scraped, copyrighted images.
Stability just released their new LLM.
It’s open-source, has 7bparameters, and its entirely free to use commercially.
And its a MASSIVE deal, that has the potential to change up everything in AI
Here’s why: pic.twitter.com/10CONPz1R8
— Sully (@SullyOmarr) April 20, 2023
