Inferencing with vLLM

Tue, 28 Oct 2025 00:00:00 +0000

The vLLM Inference Framework is a production-grade, high-performance inference engine designed for large-scale LLM serving.

The main characteristics of vLLM Inference Framework are:

Supports single-node deployments with one or more GPUs.
Uses Python’s native multiprocessing for multi-GPU inference.
Does not require additional frameworks, such as Ray, unless deploying across multiple nodes, which is out of scope for this benchmarking task.

In this guide you will find the necessary steps and best practices to deploy the OpenNebula vLLM appliance and perform an inference benchmarking to check its performance.

Fine-tuning AI Models on NVIDIA Slurm

Mon, 01 Jan 0001 00:00:00 +0000

This tutorial deploys the OpenNebula OneSlurm service and runs a fine-tuning job as an LDAP user. The Slurm cluster is created first. After the Controller and Worker VMs are running, the user stages the model, dataset, virtual environment, training script, and output in the shared /scratch NFS mount.

We will complete the following high-level steps:

Deploy the Service OneSlurm appliance from the OpenNebula Marketplace.
Enable LDAP identity for Slurm users.
Mount shared NFS scratch storage on the Controller and Workers.
Create or use an LDAP user and prepare the user’s scratch workspace.
Submit a GPU fine-tuning job from the Slurm Controller.

Before Starting

Before starting this tutorial, complete the AI Factory deployment with either on-premises resources or cloud resources. Use the guide that matches your available resources:

Direct AI Execution on

Inferencing with vLLM

Fine-tuning AI Models on NVIDIA Slurm

Before Starting