I was yesterday years old when I learned that those open weight models need custom code to run.
Somehow I expected inference engines are generic LLM runtimes that can execute any weight.
So, to get this right.
Someone trains a model.
They release the weights and a reference implementation of the model architecture.
Then a provider has to host this model either by running inference via the reference implementation, an open source implementation, or build their own.
Does this mean, providers don't just differ in quantisation and configuration, but also in inference engine implementation?
This is excellent for understanding. I'm having some trouble to get into understanding - pytorch is for me the RL which is used as gym/training. There I can chose ppo, dnq and other agents to perform some predefined actions in a predefined gym/world.
The repo you are showing - I really have problems to get it into RL understanding of mine. What's the gym? What are the agents. Can it be used to train that models with pytorch?
Sorry for the noob question. Papers are overwhelming my noob brain.
Your comment is a little unclear, so it‘s hard to parse your exact question. But it seems you are conflating 3 things, PyTorch, RL and Gym/Training (?).
– PyTorch is a framework which lets you define neural network models.
– RL is a collection of methods to train neural networks (change the network parameters to improve its performance).
– An RL-Gym is a framework to apply the neural networks to some problem. This lets you collect the data necessary to later use the methods of RL to train your model.
I have been studying modern LLM architectures and started implementing them from scratch in PyTorch to better understand the design choices behind each model.
OpenArch is a collection of these implementations, including Llama, Qwen, DeepSeek, Gemma, Kimi, GPT-OSS and others.
The goal is to keep the code readable and useful as a reference when going from the paper to an actual implementation.
Would be interested in feedback from people working on model architecture and training.
hey, Building these from scratch in pure PyTorch is honestly the best way to deeply understand the paper details.
something better than simply implementing a traditional Transformer or GPT-2，As an individual maintainer, will be able to keep up with future model updates?
Somehow I expected inference engines are generic LLM runtimes that can execute any weight.
So, to get this right.
Someone trains a model.
They release the weights and a reference implementation of the model architecture.
Then a provider has to host this model either by running inference via the reference implementation, an open source implementation, or build their own.
Does this mean, providers don't just differ in quantisation and configuration, but also in inference engine implementation?
This is excellent for understanding. I'm having some trouble to get into understanding - pytorch is for me the RL which is used as gym/training. There I can chose ppo, dnq and other agents to perform some predefined actions in a predefined gym/world.
The repo you are showing - I really have problems to get it into RL understanding of mine. What's the gym? What are the agents. Can it be used to train that models with pytorch?
Sorry for the noob question. Papers are overwhelming my noob brain.
Your comment is a little unclear, so it‘s hard to parse your exact question. But it seems you are conflating 3 things, PyTorch, RL and Gym/Training (?).
– PyTorch is a framework which lets you define neural network models.
– RL is a collection of methods to train neural networks (change the network parameters to improve its performance).
– An RL-Gym is a framework to apply the neural networks to some problem. This lets you collect the data necessary to later use the methods of RL to train your model.
OpenArch is a collection of these implementations, including Llama, Qwen, DeepSeek, Gemma, Kimi, GPT-OSS and others.
The goal is to keep the code readable and useful as a reference when going from the paper to an actual implementation.
Would be interested in feedback from people working on model architecture and training.