To pay tribute to TempleOS, a developer has created an operating system to launch Llama 2. Netizen: An old computer with 8G memory can run it

I have to say that Llama 2’s “second creation” projects are becoming more and more hardcore and interesting.

Since Meta released the open source large model Llama 2, there have been more "secondary" projects surrounding the model.

In July this year, Andrej Karpathy, the former AI director of Tesla and returning to OpenAI, used his weekend time to make an interesting project llama2.c about Llama 2, allowing The user trains a baby Llama 2 model in PyTorch, and then uses nearly 500 lines of pure C files without any dependencies for inference.

Today, based on the Karpathy llama2.c project, another developer created a demo operating system to start Llama 2, and a user-mode assistant to call Llama 2 Kernel module for inference. The project is called Llama 2 Everywhere, or L2E for short.

However, the author also stated that the current system requirements are 512MB RAM and x86_64 (64-bit) PC, and the usability needs to be improved.

At the same time, this project also hides some Easter eggs, such as the "Doom" game, but you must solve puzzles to find it.

Project address: https://github.com/trholding/llama2.c

Andrej Karpathy also forwarded this new project, saying that it has "independent operation, binary, portable and bootable" features. He expects his llama2.c project to be a success, and this new project expands the imagination.

#L2E has been recognized by netizens from all walks of life. Some people said that this is really great. Basically it can be started on any old laptop and can be run on just the CPU, 8G of RAM is enough, 16G is better.

Some people feel that the project is very similar to TempleOS, an operating system developed for God (it was developed alone by the late genius programmer Terry Davis), but the project author said that he is just interested in Tribute to Terry and TempleOS.

L2E built an operating system using just the Linux kernel and some knowledge, a world of difference from the one Terry built from scratch.

Next, let’s learn about the original intention, demo and features of the L2E project.

L2E project introduction

The project author stated that L2E aims to ensure that it Compatibility on various devices, such as launching on Chromebooks and deploying on enterprise high-density unikernels.

In the future, this project has the potential to unlock capabilities beyond a single large-scale LLM by leveraging a large number of networked, distributed, and self-coordinating small-scale specialized LLMs with modest hardware requirements. The wisdom of crowds.

Currently, a notable use case for L2E is training small-scale models on different text sources, including textbooks, open books, and the SlimPajama corpus. These trained models are deployed using L2E to run as bootable instances on older, outdated school computers. This deployment option can be valuable in school libraries or classrooms where Internet connectivity is limited or unavailable.

The ultimate goal of the project is to create an inclusive AI ecosystem that can adapt to diverse environments and empower individuals and communities on a global scale.

Additionally by training with various hardware telemetry data, it is hoped that the model will learn to interpret sensor inputs and control actuators based on the insights gathered from the sensor inputs. This research direction is likely to play a role in areas such as automation, space, robotics and the Internet of Things, and L2E will play a key role in bridging the gap between AI and physical systems.

L2E Demo Show

The project author shows some examples of starting Llama 2.

The following is the L2E operating system of the Linux kernel. Now you can actually start and infer the Llama 2 model on the computer.

In addition, you can do cat /dev/llama and echo commands such as "Sudo make me a sandwich!" > /dev/llama , or pass a kernel parameter l2e.quest="What is the meaning of life?".

#Finally, it is possible to launch and infer a set of 1000 Virtual Baby Llama 2 models on a large enterprise server.

L2E is highly portable and can run a single executable file on any x86_64 operating system, such as Windows, GNU Linux, GNU/Systemd, NetBSD, OpenBSD, etc. . Independence is reflected in the ability to embed models and tokenizers through ZipOS (cosmocc), INCBIN, strliteral. L2E can run on CPU/GPU.

Please refer to the original project for more details.

The above is the detailed content of To pay tribute to TempleOS, a developer has created an operating system to launch Llama 2. Netizen: An old computer with 8G memory can run it. For more information, please follow other related articles on the PHP Chinese website!