模型性能perf测试

参考链接

创建测试配置文件：

perf_config.json

[
    {
      "model_name": "Qwen2.5-7B",
      "path": "Qwen2.5-7B-Instruct-GPTQ-Int4-MTT",
      "batchs": [1],
      "prefill_token_lens": [256,512,1024,2048],
      "decode_token_lens": [64,128]
    },
    {
      "model_name": "Qwen2.5-3B",
      "path": "Qwen2.5-3B-Instruct-GPTQ-Int4-MTT",
      "batchs": [1],
      "prefill_token_lens": [256,512,1024,2048],
      "decode_token_lens": [64,128]
    },
    {
      "model_name": "Qwen2.5-1.5B",
      "path": "Qwen2.5-1.5B-Instruct-GPTQ-Int4-MTT",
      "batchs": [1],
      "prefill_token_lens": [256,512,1024,2048],
      "decode_token_lens": [64,128]
    }
]

运行测试脚本

python -m mttransformer.perf_test perf_config.json