Nvidia, Hugging Face and ServiceNow release new StarCoder2 LLM for code generation

The models, currently available in three different sizes, have been trained on more than 600 programming languages, including low-resource languages, to help enterprises in their development workflows They are developed under the open BigCode project, a joint initiative of ServiceNow and Huging Face to ensure the responsible development and use of large code language models in an open and responsible environment. They are provided free of charge under the AI ​​license.

The launch of StarCoder2 confirms the tremendous power that can come from open scientific collaboration and responsible AI practices combined with an ethical data supply chain. Harm de Vries, head of the StarCoder2 development team at ServiceNow and co-lead of BigCode, pointed out in a statement that the new open access model not only improves the previous GenAI performance, but also improves developer productivity and makes them more accessible. The benefits of code generation AI, making it easier for businesses of any size to realize their full business potential.

StarCoder2: Three models for three different needs

BigCode’s latest offering is more than just an upgrade to StarCoder LLM, it introduces three models of different sizes: 3B, 7B and 15B , and expanded the supported programming languages ​​to 619. In the new generation of products, the amount of training data for the model called Stack has increased nearly seven times compared to the previous one. This means that BigCode is constantly evolving to provide developers with more powerful and comprehensive tools and resources to help them succeed in a variety of programming tasks. This innovative spirit and attitude of continuous improvement have made BigCode the platform of choice that developers trust and rely on, providing them with a wider range of learning and application opportunities. The development of BigCode demonstrates continued investment and focus in the field of technology and programming, bringing new possibilities and opportunities to the entire industry.

The BigCode community uses the latest generation of training technology to ensure that models can understand and generate low-resource programming languages ​​such as COBOL, mathematics, and program source code. This approach is critical to helping users gain a better grasp of diverse programming languages ​​and code discussions.

The 3 billion parameter model was trained using ServiceNow’s Fast LLM framework, while the 7B model was developed based on Hugging Face’s Nantron framework. Both models are designed to provide high performance for text-to-code and text-to-workflow generation while requiring fewer computing resources.

At the same time, the largest 15 billion parameter model was trained and optimized using the end-to-end NVIDIA Nemo cloud-native framework and NVIDIA TensorRT-LLM software.

While it remains to be seen how these models perform in different encoding scenarios, the companies note that the smallest 3B model performs on par with the original 15B StarCoder LLM.

Depending on their needs, enterprise teams can use any of these models and further fine-tune them based on enterprise data for different use cases. This can be any special task, from application source code generation, Workflow generation and text summarization to code completion, advanced code summarization and code snippet retrieval.

The companies emphasized that these models are more extensively and deeply trained to provide more context-aware and accurate predictions. This highly trained model is able to better understand the context of the repository. Ultimately, these efforts pave the way to accelerate development efforts, allowing engineers and developers to focus more energy on more critical tasks.

Jonathan Cohen, vice president of applied research at Nvidia, said in a press statement: "Because every software ecosystem has a proprietary programming language, Code LLM can drive breakthroughs in efficiency and innovation in every industry."

“NVIDIA’s collaboration with ServiceNow and Huging Face introduces a safe, responsible development model and supports broader access to responsible GenAI, which we hope will benefit society around the world,” he added.

How to get started using StarCoder2?

As mentioned before, all models in the StarCoder2 series are provided under the Open Rail-M license and can be accessed and used royalty-free. Supporting code can be found in the BigCode project's GitHub repository. As an alternative, teams can also download and use all three models of Hugging Face.

That said, 15B models trained by NVIDIA will also appear on NVIDIA AI Foundation, allowing developers to experiment directly from their browser or through API endpoints.

While StarCoder is not the first entrant in the field of AI-driven code generation, the wide range of options brought by the latest generation of the project will certainly allow enterprises to leverage LLMS in application development while also saving on computation.

Other notable players in the space include OpenAI, which provides Codex, which powers the GitHub joint pilot service, and Amazon, which provides the CodeWhisper tool, as well as stiff competition from Replit and Codenium, with Replit in Hugging Face There are several small AI coding models on the market, and Codenium recently raised $65 million in Series B funding at a $500 million valuation.

The above is the detailed content of Nvidia, Hugging Face and ServiceNow release new StarCoder2 LLM for code generation. For more information, please follow other related articles on the PHP Chinese website!