NVIDIA Released Open Synthetic Data Generation Pipeline for Training LLMs

By
IBL News
-

IBL News | New York

NVIDIA announced Nemotron-4 340B, a family of open models developers can use to generate synthetic data for training LLMs for commercial applications across healthcare, finance, manufacturing, retail, and other industries.

Robust datasets with high-quality training data are prohibitively expensive and difficult to access. Synthetic data mimics the characteristics of real-world data.

Through a uniquely permissive open model license, Nemotron-4 340B gives developers a free, scalable way to generate synthetic data that can help build powerful LLMs.

The Nemotron-4 340B family includes base, instruct, and reward models that form a pipeline to generate synthetic data for training and refining LLMs.

The models are optimized with NVIDIA NeMo, an open-source framework for end-to-end model training, including data curation, customization, and evaluation.

They’re also optimized for inference with the open-source NVIDIA TensorRT-LLM library.

Nemotron-4 340B can be downloaded from Hugging Face.