Getting Started with Inferno AI

Welcome! This guide will help you get Inferno AI up and running on your machine in just a few minutes.

What You’ll Learn

In this section, you’ll learn how to:

Install Inferno on your platform (macOS, Linux, Windows, or Docker)
Run your first inference with a simple command
Configure Inferno for your specific needs
Start the API server for OpenAI-compatible access

Prerequisites

Before you begin, make sure you have:

Operating System: macOS 10.15+, Linux (Ubuntu 20.04+, Debian 11+), Windows 10+, or Docker
Disk Space: At least 10GB free (models can be large)
RAM: Minimum 8GB recommended (16GB+ for larger models)
GPU (optional but recommended):
- Apple Silicon (M1/M2/M3/M4) for Metal acceleration
- NVIDIA GPU with CUDA 11.x+ support
- AMD GPU with ROCm support

Quick Start Path

Choose your preferred path to get started:

📦 Installation Guide

Detailed installation instructions for all supported platforms including Homebrew, package managers, and manual installation.

Best for: First-time users who want step-by-step guidance

⚡ Quick Start

Jump straight into running your first inference and starting the API server with minimal setup.

Best for: Experienced users who want to get started immediately

⚙️ Configuration

Learn how to configure Inferno for optimal performance, security, and your specific use case.

Best for: Users who want to customize their setup

🎯 First Steps Checklist

A simple checklist to verify your installation and run your first successful inference.

Best for: Ensuring everything is working correctly

Installation Overview

Here’s a quick overview of the installation process:

macOS (Homebrew)

# Install via Homebrew (recommended)
brew install inferno
 
# Verify installation
inferno --version

Linux (Package Manager)

# Ubuntu/Debian
wget https://github.com/ringo380/inferno/releases/latest/download/inferno-linux-amd64.deb
sudo dpkg -i inferno-linux-amd64.deb
 
# Verify installation
inferno --version

Windows

# Download the installer from GitHub releases
# Or use winget (Windows Package Manager)
winget install Inferno.InfernoAI
 
# Verify installation
inferno --version

Docker

# Pull the latest image
docker pull ringo380/inferno:latest
 
# Run a simple inference
docker run --rm ringo380/inferno:latest inferno --version

Your First Inference

Once installed, try this simple command to run your first inference:

# Download and run inference with a small model
inferno run --model llama-2-7b-chat --prompt "Hello, world! Tell me about yourself."

Inferno will:

Download the model if not already present
Load it into memory
Run the inference
Display the result

Start the API Server

To use Inferno with the OpenAI-compatible API:

# Start the server on port 8080
inferno serve --port 8080
 
# In another terminal, test with curl
curl http://localhost:8080/v1/models

Next Steps

After completing the basics:

Explore Guides - Learn production deployment and best practices
Check Examples - See real-world integration examples
Read API Reference - Understand the complete API
Try Tutorials - Follow step-by-step advanced tutorials

Need Help?

Installation Issues: Check the Troubleshooting guide
Configuration Questions: See the Configuration guide
Platform-Specific Help: Visit the Installation guide

Ready to install? Head to the Installation Guide →