Skip to Content

REST API Usage

Direct REST API examples using curl, httpie, and JavaScript fetch.

Curl Examples

Basic Chat Completion

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-2-7b-chat",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

With Authentication

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-2-7b-chat",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Streaming Response

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "llama-2-7b-chat",
    "messages": [{"role": "user", "content": "Count to 10"}],
    "stream": true
  }'

JavaScript/Node.js

Using Fetch

async function chat(message) {
  const response = await fetch('http://localhost:8080/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'llama-2-7b-chat',
      messages: [{ role: 'user', content: message }]
    })
  });
 
  const data = await response.json();
  return data.choices[0].message.content;
}
 
// Use it
chat("Hello!").then(console.log);

OpenAI SDK

import OpenAI from 'openai';
 
const client = new OpenAI({
  baseURL: 'http://localhost:8080/v1',
  apiKey: 'not-needed'
});
 
const response = await client.chat.completions.create({
  model: 'llama-2-7b-chat',
  messages: [{ role: 'user', content: 'Hello!' }]
});
 
console.log(response.choices[0].message.content);

Next Steps