The use of open source AI models in development

During the last year, a large number of tools with artificial intelligence have appeared to make the lives of users easier, whether image generation or chatbots, even scaling to tools that execute gigantic and professional processes.

I have been researching, learning and testing many of these tools from chatgpt, gemini to dall-e or midjourney, they all work very well but when I want to scale my applications with these tools I find that they do not have a free or open alternative source.

This has made me take my research a step further and I have come across stable diffusion ui (Image generation, https://github.com/AUTOMATIC1111/stable-diffusion-webui) and with *ollama *(Chatbot, https://ollama.com/), both are open source tools that allow you to run a service as an API to consume it from any of our applications, with this I have arrived at a I go further with open source alternatives, but for this to work I must keep these tools running to be consumed by our applications.

To understand how to bring this to our applications it is important to understand how these tools work, and basically what they do is use files with the "safetensors" extension that are LLM or large language models, these models being trained to perform different functions according to the needs of the person training it (Example: Image generation, translation, code development, chatbot, among others).

By understanding a little about the LLM models and the "safetensors" files, we get the following question: how to use these files in my applications, and this is where HugginFace comes in, a website/database of open source artificial intelligence models, and they have created their own library for python with 2 extremely useful components for what we want "Transformers" and "Diffusers".

*Transformers *(https://huggingface.co/docs/transformers/index) is the component that allows us to consume any specialized text model, for example converting audio to text or vice versa, chatbox as a Meta flame, among others.

import transformers

import torch

model_id = "meta-llama/Llama-3.1-8B"

pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
)

pipeline("Hey how are you doing today?")

Diffusers (https://huggingface.co/docs/diffusers/index) is the component that allows us to consume any model specialized in image generation, such as stable diffusion.

from diffusers import AutoPipelineForText2Image
import torch

pipe = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16")
pipe.to("cuda")

prompt = "A cinematic shot of a baby racoon wearing an intricate italian priest robe."

image = pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0.0).images[0]

This process is known as LLM Model Inference, and from here based on this information you can begin to apply artificial intelligence in your different applications with Python.

It should be noted that I have also tried to use model inference with another language such as nodejs and the truth is that it does not work as well as with python, but it is important to mention that powerful hardware is needed for LLM model inference so that what you can save by using the ChatGPT or Gemini APIs you can spend on purchasing suitable hardware.

This is my first article, I hope that my path to using LLM models in software development helps you skip steps on this path.

The above is the detailed content of The use of open source AI models in development. For more information, please follow other related articles on the PHP Chinese website!