服务调用

接口测试

curl http://localhost:8000/v1/models

输出示例：

{
   "object":"list",
   "data":[
      {
         "id":"models/gptq-Qwen2.5-7B-Instruct/",
         "object":"model",
         "created":1752157983,
         "owned_by":"vllm",
         "root":"models/gptq-Qwen2.5-7B-Instruct/",
         "parent":null,
         "max_model_len":1500,
         "permission":[
            {
               "id":"modelperm-531dc2363c5f4277bb258d23b69914bf",
               "object":"model_permission",
               "created":1752157983,
               "allow_create_engine":false,
               "allow_sampling":true,
               "allow_logprobs":true,
               "allow_search_indices":false,
               "allow_view":true,
               "allow_fine_tuning":false,
               "organization":"*",
               "group":null,
               "is_blocking":false
            }
         ]
      }
   ]
}

接口调用

注意

注意：需要替换为本地的模型路径。

另开一个窗口调用：

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "models/gptq-Qwen2.5-7B-Instruct/",
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20,
    "repetition_penalty":1.05,
    "max_tokens": 1000,
    "messages": [{"role": "user", "content": "介绍一下北京"}]
  }'

参数说明

参数名	作用	输入范围（含建议值）
model	指定要调用的模型路径/名称，需与服务端部署的模型一致	字符串（例：models/gptq-Qwen2.5-7B-Instruct/）
temperature	控制生成随机性，值越大越发散、越小越确定性	浮点数 0.0 ~ 2.0（建议：0.1~1.5）
top_p	核采样，只从累积概率≥top_p 的词中采样	浮点数 0.0 ~ 1.0（建议：0.7~0.9）
top_k	只从概率最高的前 k 个词中采样	非负整数 0 ~ ∞（建议：10~50，0表示不限制）
repetition_penalty	重复惩罚，抑制生成重复内容	浮点数 1.0 ~ 2.0（建议：1.0~1.1）
max_tokens	单次响应最大生成 token 数	正整数（建议：512~2048，≤模型最大上下文长度）

Python 调用

流式输出

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

client = OpenAI(
    # defaults to os.environ.get("OPENAI_API_KEY")
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

chat_completion = client.chat.completions.create(
    messages=[{
        "role": "system",
        "content": "You are a helpful assistant."
    }, {
        "role": "user",
        "content": "北京有哪些名胜古迹？"
    }],
    model=model,
    temperature=0.7,
    top_p=0.8,
    extra_body={
        'top_k':20,
        'repetition_penalty':1.05, # 惩罚重复,vllm默认没有加载需要添加，参考：https://modelscope.cn/models/Qwen/Qwen2.5-7B-Instruct/file/view/master?fileName=generation_config.json&status=1#L9
    },
    max_tokens=512,
    stream=True,  # 启用流式输出
)

# 处理流式响应
print("Chat response (streaming):")
for chunk in chat_completion:
    if chunk.choices:
        delta = chunk.choices[0].delta
        content = delta.content
        if content:
            print(content, end='', flush=True)
print("\n - Chat response (end) -\n")

输出示例：

Chat response (streaming):
北京是中国历史文化名城之一，拥有众多名胜古迹。其中最著名的是故宫、长城、天安门广场、天坛、颐和园、圆明园、八达岭长城、德胜门等。这些景点都是了解中国历史和文化的重要途径。
 - Chat response (end) -

非流式输出

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

client = OpenAI(
    # defaults to os.environ.get("OPENAI_API_KEY")
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

chat_completion = client.chat.completions.create(
    messages=[{
        "role": "system",
        "content": "You are a helpful assistant."
    }, {
        "role": "user",
        "content": "北京有哪些名胜古迹？"
    }],
    model=model,
    temperature=0.7,
    top_p=0.8,
    extra_body={
        'top_k':20,
        'repetition_penalty':1.05,
    },
    max_tokens=512,
    stream=False,
)

print("Chat completion results:")
print(chat_completion)