服务调用

接口测试

服务启动后，可以通过 OpenAI 兼容接口查看模型信息：

curl http://127.0.0.1:30000/v1/models

Completions 调用

curl http://127.0.0.1:30000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "北京有哪些名胜古迹？",
    "temperature": 0.7,
    "top_p": 0.8,
    "max_tokens": 100,
    "chat_template_kwargs":{"enable_thinking":false}
  }'

Chat Completions 调用

curl http://127.0.0.1:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "temperature": 0.7,
    "top_p": 0.8,
    "max_tokens": 100,
    "chat_template_kwargs":{"enable_thinking":false},
    "messages": [
      {
        "role": "user",
        "content": "北京有哪些名胜古迹？"
      }
    ]
  }'

Python 调用

流式输出

from openai import OpenAI

openai_api_key = "EMPTY"
openai_api_base = "http://127.0.0.1:30000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant.",
        },
        {
            "role": "user",
            "content": "北京有哪些名胜古迹？",
        },
    ],
    model=model,
    temperature=0.7,
    top_p=0.8,
    max_tokens=100,
    stream=True,
)

print("Chat response (streaming):")
for chunk in chat_completion:
    if chunk.choices:
        delta = chunk.choices[0].delta
        content = delta.content
        if content:
            print(content, end="", flush=True)
print("\n - Chat response (end) -\n")

非流式输出

from openai import OpenAI

openai_api_key = "EMPTY"
openai_api_base = "http://127.0.0.1:30000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant.",
        },
        {
            "role": "user",
            "content": "北京有哪些名胜古迹？",
        },
    ],
    model=model,
    temperature=0.7,
    top_p=0.8,
    max_tokens=100,
    stream=False,
)

print("Chat completion results:")
print(chat_completion)