One of the major challenges in the field of services related to large language models is how to efficiently and optimally serve these models. These challenges, which are directly linked to the extremely heavy computations required for the model’s responses, often result in significant delays and slowness when using large language models. Another challenge is the large size of the models, sometimes reaching several hundred gigabytes. At MCI-Next, we have overcome these challenges with the help of Nvidia's Triton Inference Server and TensorRT-LLM. With the efforts of the MLOps technical team, we successfully served the Llama 3.1 model with impressive results. The smaller model in this family, which has 8 billion parameters, was able to deliver 5,000 tokens per second on a single A100 GPU. And this is just the beginning of our work in serving open-source models, and we hope to serve much larger models in the future, reaching the scale of the world’s leading companies.