vLLM

Send chat messages to RemoteGPU’s vLLM endpoint to generate a reply in your application or script. Include a model ID in every request to choose which model answers.

Send requests

Quickstart chat completion

Create a key with Token Factory access using API keys. Send it in the Authorization: Bearer <api-key> header, along with the model ID and messages you want the model to answer.

curl

curl -X POST "https://vllm.remotegpu.ai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "Qwen/Qwen3.6-27B-FP8",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this note in one concise paragraph."
      }
    ],
    "max_tokens": 256,
    "temperature": 0.2,
    "seed": 1234
  }'

JavaScript

const response = await fetch("https://vllm.remotegpu.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${process.env.REMOTEGPU_API_KEY}`,
  },
  body: JSON.stringify({
    model: "Qwen/Qwen3.6-27B-FP8",
    messages: [
      {
        role: "user",
        content: "Summarize this note in one concise paragraph.",
      },
    ],
    max_tokens: 256,
    temperature: 0.2,
    seed: 1234,
  }),
});

Python

import os
import requests

response = requests.post(
    "https://vllm.remotegpu.ai/v1/chat/completions",
    headers={
        "Content-Type": "application/json",
        "Authorization": f"Bearer {os.environ['REMOTEGPU_API_KEY']}",
    },
    json={
        "model": "Qwen/Qwen3.6-27B-FP8",
        "messages": [
            {
                "role": "user",
                "content": "Summarize this note in one concise paragraph.",
            }
        ],
        "max_tokens": 256,
        "temperature": 0.2,
        "seed": 1234,
    },
    timeout=60,
)
response.raise_for_status()
completion = response.json()
print(completion["choices"][0]["message"]["content"])

Replace YOUR_API_KEY in the curl example with your key. For JavaScript or Python, set REMOTEGPU_API_KEY in the environment where the code runs. Replace the example message with your own prompt.

A successful request returns a response like this:

{
  "id": "chatcmpl-c3b1e0e7-2f7c-4c66-bd13-97b6c2b87f1d",
  "object": "chat.completion",
  "created": 1779174000,
  "model": "Qwen/Qwen3.6-27B-FP8",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 32,
    "completion_tokens": 48,
    "total_tokens": 80
  }
}

The request waits for the model to finish, then returns an OpenAI-compatible chat.completion object. The reply is in choices[0].message.content.

How requests work

Authentication

Send a Token Factory key in the Authorization header as a Bearer token on every request. You can create or replace a key in API keys.

If the key is missing or invalid, the API returns 401. If the key is valid but does not allow inference APIs, the API returns 403.

Connect an OpenAI-compatible client

Text requests use these OpenAI-compatible endpoints:

Endpoint	Purpose
GET /v1/models	List the text models you can request
POST /v1/chat/completions	Send messages and wait for a chat.completion response

Use https://vllm.remotegpu.ai/v1 as the base URL for OpenAI-compatible text inference clients.

For a client using the OpenAI SDK, set your RemoteGPU key and base URL:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://vllm.remotegpu.ai/v1",
)

completion = client.chat.completions.create(
    model="Qwen/Qwen3.6-27B-FP8",
    messages=[
        {
            "role": "user",
            "content": "Summarize this note in one concise paragraph.",
        }
    ],
    max_tokens=256,
    temperature=0.2,
    seed=1234,
)

Choose a model first

Call GET /v1/models or GET /v1/inference/models to choose a model ID. The detailed catalog at GET /v1/inference/models also shows whether the model is ready and which request limits apply. Check those limits before choosing the number of messages or generation settings.

The model list at https://vllm.remotegpu.ai/v1/models uses your RemoteGPU API key. You can read the detailed catalog without a key at https://inference.remotegpu.ai/v1/inference/models and check model availability and parameter limits for both Text and Image.

The catalog uses these states to show whether a model can answer requests:

State	Description
ready	Ready to serve requests
starting	The model is starting
sleeping	The model needs to start before serving requests

A request to a model in sleeping state can take longer while the model starts.

Message content

For a text message, send the content as a string or as a list of text parts:

{
  "role": "user",
  "content": "Write a short summary."
}

{
  "role": "user",
  "content": [
    {
      "type": "text",
      "text": "Write a short summary."
    }
  ]
}

If the model accepts images, add an image_url part alongside the text. Encode the image as a base64 data URL and put it in image_url.url:

{
  "role": "user",
  "content": [
    {
      "type": "text",
      "text": "Describe this image."
    },
    {
      "type": "image_url",
      "image_url": {
        "url": "data:image/png;base64,BASE64_IMAGE"
      }
    }
  ]
}

Requests containing image_url content parts for models without image input support return 400 with image_input_not_supported_yet.

Image input example

To ask about an image, send it with your question to POST /v1/chat/completions. The API waits for the model's answer and returns it in the same chat.completion format as a text-only request.

curl -X POST "https://vllm.remotegpu.ai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "Qwen/Qwen3.6-27B-FP8",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Describe this image in one concise paragraph."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "data:image/png;base64,BASE64_IMAGE"
            }
          }
        ]
      }
    ],
    "max_tokens": 256,
    "temperature": 0.2,
    "seed": 1234
  }'

Image input limits:

Limit	Value
Source format	Base64 data URL such as data:image/png;base64,...
Source transport	Inline image bytes in the request body
Max images	Model-specific; Qwen/Qwen3.6-27B-FP8 accepts up to 1 image
Max source dimensions	8192px on either axis
Public request body	20MiB

Larger images can take longer to process and use more GPU memory. Choose a size that keeps the detail needed to answer your question.

If the image is rejected, the response uses the same OpenAI-style error format as other Text API errors.

Error code	What to check
image_input_not_supported_yet	Choose a model with image input support
unsupported_image_url	Use a base64 data URL in image_url.url
invalid_image	The data URL or base64 payload is invalid
parameter_out_of_range	Keep image count and generation parameters within the selected model's limits

For invalid_image, check that the base64 data decodes to an image and that neither its width nor its height exceeds the source dimension limit.

Response and timeouts

On success, POST /v1/chat/completions returns 200 OK with the chat.completion response shown in the quickstart. Read the generated text from choices[0].message.content and token counts from usage.

If the request exceeds the server timeout, the API returns 504 with an OpenAI-style error response. Check the error message and model availability before sending another request.

Common status codes

Status code	Description
400	The JSON can be read, but its values are not valid for this model or endpoint
401	Missing, invalid, revoked, or expired API key
403	API key is valid but not authorized for inference APIs
404	The selected model does not exist or is not available
422	Request validation failed, such as a missing model field
503	The model exists but cannot serve requests
504	The request timed out before a chat-completion response was available

Reference

Text models

Model	Max messages	Max input images	Max output tokens	Default parameters
Qwen/Qwen3.6-27B-FP8	64	1	8192	1024 max tokens, temperature 0.7, top-p 1.0

Request body fields

Supported fields for POST /v1/chat/completions requests:

Field	Required	Description
model	Yes	Must match a supported text model ID
messages	Yes	Chat messages for the selected model
max_tokens	No	Defaults to the selected model default; maximum is 8192
temperature	No	Defaults to 0.7; accepted range is 0.0 through 2.0
top_p	No	Defaults to 1.0; accepted range is 0.0 through 1.0
seed	No	Optional integer sampling seed for reproducible generation; minimum is 0
stop	No	Optional stop sequence or list of stop sequences
stream	No	Streaming is not accepted; send false or omit the field

Model catalog endpoint

Request the model catalog to see each model's supported parameters, defaults, and limits:

curl "https://inference.remotegpu.ai/v1/inference/models"

The response includes:

• text[].model: the model identifier to send in POST /v1/chat/completions
• text[].parameters: the public field details, including defaults and limits
• text[].runtime.state: whether the model is ready, starting, or sleeping
• text[].recent_summary_stats: recent times spent waiting, generating, and completing requests

Read these limits from the catalog so your client can use them without a code change when a model's limits change.

Read next

• Read API keys to create or rotate Token Factory keys.
• Read Token Factory overview to compare the API with hosted applications and Kubernetes.