推理性能

模型性能数据

以下数据来自 SGLang-MUSA 0.5.10+20260907 在 M1000 AImodule 端侧环境的实测结果。均在性能模式 + compute_only 环境下测试，数值供参考。

测试环境

• 端设备：M1000 AImodule，64G 统一内存
• 系统版本：V1.4.1
• MUSA 版本：5.1.1
• 服务参数：max-total-tokens=16384
• 文本测试并发：1
• 文本测试输入 / 输出：无前缀场景为 128/100、250/100、500/100、1000/100、2000/100；带 prefix 场景为 500/100、1000/100、2000/100、4000/100、8000/100、16000/100，其中 PrefillLen=250

测试说明

测试指标说明如下：

• Prefill(tps)：Prefill 阶段吞吐，单位 token/s。
• TTFT(ms)：首 token 延迟，单位 ms。
• Decode(tps)：Decode 阶段吞吐，单位 token/s。
• TPOT(ms)：单个输出 token 平均耗时，单位 ms。
• Profile：测试 profile，格式为 prefill_len:input_len。
• Vision(fps)：视觉编码帧率，单位 frame/s。
• Precision：模型量化精度。

性能复现流程

解压得到安装包目录：

tar -zxvf 20260831_M1000_1.4.1_sglang.tar.gz
cd 20260831_M1000_1.4.1_sglang

启动服务前先预编译 TileLang kernel；如果你已经在 快速开始 里执行过这一步，这里可以直接跳过：

./scripts/ci/musa/tilelang_precompile/precompile_tilelang.sh --model qwen35,qwen3,qwen35_fq,qwen36_fq,qwen35_dflash,qwen35_fq_dflash,qwen3vl

分别启动服务并运行 benchmark。每组测试使用两个终端：启动 端运行服务，运行 benchmark 端执行对应测试。benchmark 通过 MODEL_KEY 选择对应模型配置。

启动方式和参考性能数据

Qwen3-30B-A3B-GPTQ-Int4 (MoE GPTQ，GPTQ INT4 (W4A16))

启动：

scripts/ci/musa/start_qwen3_server.sh

运行 benchmark：

MODEL_KEY=qwen3 scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Ctx(K)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Profile	Prefix(tokens)	MeasuredOutput(tokens)
Qwen3	30b_a3b	128	16	1	1	128	100	64.606	1981.228	24.531	40.765	128:128	0	100
Qwen3	30b_a3b	250	16	1	1	250	100	94.731	2639.041	23.965	41.728	250:250	0	100
Qwen3	30b_a3b	500	16	1	1	500	100	144.382	3463.043	23.311	42.898	500:500	0	100
Qwen3	30b_a3b	1000	16	1	1	1000	100	191.371	5225.453	23.028	43.426	1000:1000	0	100
Qwen3	30b_a3b	2000	16	1	1	2000	100	181.559	11015.687	22.285	44.873	2000:2000	0	100
Qwen3	30b_a3b	250	16	1	1	500	100	77.03	3245.468	23.592	42.387	250:500	250	100
Qwen3	30b_a3b	250	16	1	1	1000	100	76.303	3276.39	22.871	43.723	250:1000	750	100
Qwen3	30b_a3b	250	16	1	1	2000	100	71.801	3481.849	22.245	44.954	250:2000	1750	100
Qwen3	30b_a3b	250	16	1	1	4000	100	75.612	3306.349	20.896	47.856	250:4000	3750	100
Qwen3	30b_a3b	250	16	1	1	8000	100	62.978	3969.642	18.779	53.251	250:8000	7750	100
Qwen3	30b_a3b	250	16	1	1	16000	100	46.654	5358.567	15.662	63.85	250:16000	15750	100

Qwen3.5-35B-A3B-GPTQ-Int4 (MoE GPTQ，GPTQ INT4 (W4A16))

启动：

scripts/ci/musa/start_qwen35_server.sh

运行 benchmark：

MODEL_KEY=qwen35 scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Ctx(K)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Profile	Prefix(tokens)	MeasuredOutput(tokens)
Qwen3.5	35b_a3b	128	16	1	1	128	100	84.599	1513.026	14.819	67.48	128:128	0	100
Qwen3.5	35b_a3b	250	16	1	1	250	100	85.558	2921.984	14.647	68.271	250:250	0	100
Qwen3.5	35b_a3b	500	16	1	1	500	100	141.848	3524.893	14.699	68.032	500:500	0	100
Qwen3.5	35b_a3b	1000	16	1	1	1000	100	213.355	4687.03	14.905	67.091	1000:1000	0	100
Qwen3.5	35b_a3b	2000	16	1	1	2000	100	210.223	9513.689	14.57	68.632	2000:2000	0	100
Qwen3.5	35b_a3b	250	16	1	1	500	100	75.497	3311.373	14.578	68.596	250:500	250	100
Qwen3.5	35b_a3b	250	16	1	1	1000	100	74.451	3357.928	14.542	68.767	250:1000	750	100
Qwen3.5	35b_a3b	250	16	1	1	2000	100	73.796	3387.727	14.49	69.011	250:2000	1750	100
Qwen3.5	35b_a3b	250	16	1	1	4000	100	70.694	3536.377	14.558	68.69	250:4000	3750	100
Qwen3.5	35b_a3b	250	16	1	1	8000	100	75.132	3327.477	14.353	69.671	250:8000	7750	100
Qwen3.5	35b_a3b	250	16	1	1	16000	100	63.076	3963.49	13.827	72.324	250:16000	15750	100

Qwen3.5-35B-A3B-GPTQ-Int4 (MoE GPTQ，GPTQ INT4 (W4A16)) 开启 MTP 投机解码

启动：

QWEN35_EXTRA_SERVER_ARGS="--mem-fraction-static 0.97 --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4" scripts/ci/musa/start_qwen35_server.sh

运行 benchmark (GSM8K测试集):

MODEL_KEY=qwen35_mtp scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Prefix(tokens)	Prompt Index	accept rate((acclen-1)/(4-1))	Prediction	Ground Truth	Correct
Qwen3.5+MTP	35b_a3b	869	1	1	869	156	184.47	4710.76	20.72	48.26	0	6	0.7431	64	64	TRUE
Qwen3.5+MTP	35b_a3b	855	1	1	855	116	182.61	4682.17	23.1	43.29	0	7	0.8646	260	260	TRUE
Qwen3.5+MTP	35b_a3b	884	1	1	884	254	186.86	4730.82	21.57	46.36	0	8	0.7911	160	160	TRUE
Qwen3.5+MTP	35b_a3b	919	1	1	919	257	191.12	4808.5	23.19	43.11	0	9	0.8685	45	45	TRUE
Qwen3.5+MTP	35b_a3b	875	1	1	875	154	185.02	4729.26	22.97	43.53	0	10	0.8527	460	460	TRUE
Qwen3.5+MTP	35b_a3b	875	1	1	875	144	185.87	4707.53	23.71	42.18	0	11	0.8889	366	366	TRUE
Qwen3.5+MTP	35b_a3b	879	1	1	879	182	186.18	4721.28	23.67	42.25	0	12	0.898	694	694	TRUE
Qwen3.5+MTP	35b_a3b	882	1	1	882	179	185.90	4744.59	21.13	47.32	0	13	0.7654	12	13	FALSE
Qwen3.5+MTP	35b_a3b	872	1	1	872	376	185.32	4705.27	23.31	42.89	0	14	0.8803	18	18	TRUE
Qwen3.5+MTP	35b_a3b	867	1	1	867	223	183.73	4718.77	23.04	43.41	0	15	0.8602	60	60	TRUE
Qwen3.5+MTP	35b_a3b	909	1	1	909	123	190.23	4778.4	23.9	41.85	0	16	0.899	125	125	TRUE
Qwen3.5+MTP	35b_a3b	868	1	1	868	61	183.72	4724.65	24.32	41.11	0	17	0.9167	230	230	TRUE
Qwen3.5+MTP	35b_a3b	872	1	1	872	148	185.31	4705.53	24.37	41.04	0	18	0.9231	57500	57500	TRUE
Qwen3.5+MTP	35b_a3b	843	1	1	843	80	180.89	4660.32	21.28	47	0	19	0.7639	7	7	TRUE
Qwen3.5+MTP	35b_a3b	880	1	1	880	185	186.34	4722.51	21.91	45.64	0	20	0.8025	6	6	TRUE
Qwen3.5+MTP	35b_a3b	878	1	1	878	167	185.79	4725.69	22.7	44.06	0	21	0.844	15	15	TRUE
Qwen3.5+MTP	35b_a3b	860	1	1	860	100	183.31	4691.52	22.78	43.9	0	22	0.8452	14	14	TRUE
Qwen3.5+MTP	35b_a3b	870	1	1	870	91	184.79	4707.92	23.15	43.19	0	23	0.8667	7	7	TRUE
Qwen3.5+MTP	35b_a3b	855	1	1	855	71	182.83	4676.42	22.56	44.32	0	24	0.8333	8	8	TRUE
Qwen3.5+MTP	35b_a3b	854	1	1	854	171	182.48	4680.09	21.08	47.43	0	25	0.7564	26	26	TRUE

Qwen3.5-35B-A3B-GPTQ-Int4 (MoE GPTQ，GPTQ INT4 (W4A16)) 开启 DFLASH 投机解码

启动：

scripts/ci/musa/start_qwen35_dflash_server.sh

运行 benchmark (GSM8K测试集):

MODEL_KEY=qwen35_dflash scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Prefix(tokens)	Prediction	Ground Truth	accept len	Correct
Qwen3.5+dflash	35B-A3B	54	1	1	54	174	68.04	793.6	22.53	44.38	0	3	3	5.438	TRUE
Qwen3.5+dflash	35B-A3B	85	1	1	85	420	82.12	1035.05	27.22	36.73	0	70000	70000	6.562	TRUE
Qwen3.5+dflash	35B-A3B	63	1	1	63	253	79.11	796.32	31.77	31.47	0	540	540	7.667	TRUE
Qwen3.5+dflash	35B-A3B	139	1	1	139	274	88.55	1569.71	27.02	37.01	0	20	20	6.524	TRUE
Qwen3.5+dflash	35B-A3B	83	1	1	83	576	82.02	1011.96	24.9	40.15	0	64	64	6	TRUE
Qwen3.5+dflash	35B-A3B	69	1	1	69	323	78.12	883.22	41.85	23.89	0	260	260	10.094	TRUE
Qwen3.5+dflash	35B-A3B	98	1	1	98	368	84.96	1153.44	23.5	42.56	0	160	160	5.662	TRUE
Qwen3.5+dflash	35B-A3B	132	1	1	132	602	87.28	1512.41	30.02	33.31	0	45	45	7.253	TRUE
Qwen3.5+dflash	35B-A3B	88	1	1	88	334	82.57	1065.73	27.11	36.89	0	460	460	6.549	TRUE
Qwen3.5+dflash	35B-A3B	88	1	1	88	388	82.45	1067.26	39.06	25.6	0	366	366	9.463	TRUE
Qwen3.5+dflash	35B-A3B	93	1	1	93	445	83.48	1114.08	44.65	22.4	0	694	694	10.854	TRUE
Qwen3.5+dflash	35B-A3B	95	1	1	95	4096	82.91	1145.82	25.86	38.66	0	12	13	6.311	FALSE
Qwen3.5+dflash	35B-A3B	86	1	1	86	618	81.81	1051.22	29.77	33.59	0	18	18	7.186	TRUE
Qwen3.5+dflash	35B-A3B	80	1	1	80	571	79.99	1000.11	32.77	30.51	0	60	60	7.931	TRUE
Qwen3.5+dflash	35B-A3B	122	1	1	122	412	87.49	1394.5	29.86	33.49	0	125	125	7.228	TRUE
Qwen3.5+dflash	35B-A3B	81	1	1	81	294	80.73	1003.4	23.84	41.94	0	230	230	5.765	TRUE
Qwen3.5+dflash	35B-A3B	86	1	1	86	445	81.74	1052.08	31.71	31.54	0	57500	57500	7.672	TRUE
Qwen3.5+dflash	35B-A3B	57	1	1	57	267	76.76	742.56	29.1	34.36	0	7	7	7.026	TRUE
Qwen3.5+dflash	35B-A3B	93	1	1	93	421	83.54	1113.22	28.06	35.64	0	6	6	6.79	TRUE
Qwen3.5+dflash	35B-A3B	91	1	1	91	444	82.5	1102.99	32.24	31.01	0	15	15	7.789	TRUE
Qwen3.5+dflash	35B-A3B	74	1	1	74	279	79.06	935.98	24.05	41.58	0	14	14	5.812	TRUE
Qwen3.5+dflash	35B-A3B	83	1	1	83	221	81.32	1020.63	36.55	27.36	0	7	7	8.84	TRUE
Qwen3.5+dflash	35B-A3B	68	1	1	68	216	77.05	882.52	27.02	37.01	0	8	8	6.545	TRUE
Qwen3.5+dflash	35B-A3B	67	1	1	67	499	75.89	882.91	30.82	32.45	0	26	26	7.448	TRUE
Qwen3.5+dflash	35B-A3B	95	1	1	95	405	84.17	1128.7	31.52	31.73	0	2	2	7.642	TRUE
Qwen3.5+dflash	35B-A3B	97	1	1	97	450	84.24	1151.49	36.38	27.49	0	243	243	8.824	TRUE
Qwen3.5+dflash	35B-A3B	85	1	1	85	335	81.93	1037.42	28.27	35.38	0	16	16	6.837	TRUE
Qwen3.5+dflash	35B-A3B	78	1	1	78	263	80.03	974.66	25.91	38.6	0	25	25	6.262	TRUE
Qwen3.5+dflash	35B-A3B	97	1	1	97	343	83.48	1161.97	27.74	36.05	0	104	104	6.725	TRUE
Qwen3.5+dflash	35B-A3B	67	1	1	67	302	77.02	869.85	35.7	28.01	0	109	109	8.629	TRUE
Qwen3.5+dflash	35B-A3B	91	1	1	91	369	83.06	1095.55	33.85	29.55	0	80	80	8.2	TRUE
Qwen3.5+dflash	35B-A3B	69	1	1	69	180	77.47	890.69	24.85	40.23	0	35	35	6	TRUE

Qwen3.5_FQ-35B-A3B (MoE GPTQ，GPTQ INT4 (W4A16))

启动：

scripts/ci/musa/start_qwen35_fq_server.sh

运行 benchmark：

MODEL_KEY=qwen35_fq scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Ctx(K)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Profile	Prefix(tokens)	MeasuredOutput(tokens)
Qwen3.5_FQ	35b_a3b	128	16	1	1	128	100	82.249	1556.244	21.021	47.572	128:128	0	100
Qwen3.5_FQ	35b_a3b	250	16	1	1	250	100	81.701	3059.955	20.855	47.949	250:250	0	100
Qwen3.5_FQ	35b_a3b	500	16	1	1	500	100	131.504	3802.176	20.918	47.805	500:500	0	100
Qwen3.5_FQ	35b_a3b	1000	16	1	1	1000	100	187.131	5343.849	21.038	47.533	1000:1000	0	100
Qwen3.5_FQ	35b_a3b	2000	16	1	1	2000	100	185.97	10754.402	20.923	47.795	2000:2000	0	100
Qwen3.5_FQ	35b_a3b	250	16	1	1	500	100	71.712	3486.16	20.814	48.045	250:500	250	100
Qwen3.5_FQ	35b_a3b	250	16	1	1	1000	100	70.96	3523.1	20.439	48.926	250:1000	750	100
Qwen3.5_FQ	35b_a3b	250	16	1	1	2000	100	70.844	3528.898	20.575	48.602	250:2000	1750	100
Qwen3.5_FQ	35b_a3b	250	16	1	1	4000	100	67.604	3698.016	20.347	49.148	250:4000	3750	100
Qwen3.5_FQ	35b_a3b	250	16	1	1	8000	100	72.499	3448.316	20.255	49.371	250:8000	7750	100
Qwen3.5_FQ	35b_a3b	250	16	1	1	16000	100	61.268	4080.431	18.777	53.256	250:16000	15750	100

Qwen3.5_FQ-35B-A3B (MoE GPTQ，GPTQ INT4 (W4A16)) 开启 MTP 投机解码

启动：

QWEN35_FQ_EXTRA_SERVER_ARGS="--speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4" scripts/ci/musa/start_qwen35_fq_server.sh

运行 benchmark (GSM8K测试集):

MODEL_KEY=qwen35_mtp PORT=30139 scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Prompt Index	accept rate((acclen-1)/(4-1))	accept rate(acclen/4)	Prediction	Ground Truth	Correct
Qwen3.5_FQ	35b_a3b	869	1	1	869	340	92.04	9442	23.19	43.13	6	0.7638	0.8252	64	64	TRUE
Qwen3.5_FQ	35b_a3b	855	1	1	855	111	91.74	9319.95	25.79	38.77	7	0.8889	0.925	260	260	TRUE
Qwen3.5_FQ	35b_a3b	884	1	1	884	237	92.31	9576.49	23.65	42.27	8	0.7905	0.8464	160	160	TRUE
Qwen3.5_FQ	35b_a3b	919	1	1	919	267	92.36	9950.43	23.94	41.77	9	0.8034	0.8558	45	45	TRUE
Qwen3.5_FQ	35b_a3b	875	1	1	875	147	92.19	9490.76	24.09	41.51	10	0.7984	0.8547	460	460	TRUE
Qwen3.5_FQ	35b_a3b	875	1	1	875	140	92.23	9487.47	24.06	41.56	11	0.7967	0.8537	366	366	TRUE
Qwen3.5_FQ	35b_a3b	879	1	1	879	182	92.3	9523.19	25.57	39.11	12	0.8733	0.91	694	694	TRUE
Qwen3.5_FQ	35b_a3b	882	1	1	882	169	92.3	9555.32	21.19	47.19	13	0.6667	0.7545	12	13	FALSE
Qwen3.5_FQ	35b_a3b	872	1	1	872	387	92.27	9450.16	24.94	40.09	14	0.8471	0.8876	18	18	TRUE
Qwen3.5_FQ	35b_a3b	867	1	1	867	181	92.24	9399.15	23.6	42.37	15	0.7778	0.838	60	60	TRUE
Qwen3.5_FQ	35b_a3b	909	1	1	909	157	92.45	9831.85	23.45	42.65	16	0.773	0.8351	125	125	TRUE
Qwen3.5_FQ	35b_a3b	868	1	1	868	61	92.24	9409.8	26.83	37.27	17	0.9167	0.9531	230	230	TRUE
Qwen3.5_FQ	35b_a3b	872	1	1	872	151	92.16	9461.32	25.58	39.09	18	0.8862	0.9207	57500	57500	TRUE
Qwen3.5_FQ	35b_a3b	843	1	1	843	77	91.9	9172.96	22.34	44.77	19	0.7222	0.8021	7	7	TRUE
Qwen3.5_FQ	35b_a3b	880	1	1	880	179	92.25	9539.71	22.02	45.42	20	0.7076	0.7851	6	6	TRUE

Qwen3.5_FQ-35B-A3B(MoE GPTQ，GPTQ INT4 (W4A16)) 开启 DFLASH 投机解码

启动：

scripts/ci/musa/start_qwen35_fq_dflash_server.sh

运行 benchmark (GSM8K测试集):

MODEL_KEY=qwen35_fq_dflash scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Prefix(tokens)	Prediction	Ground Truth	accept len	Correct
Qwen3.5_FQ+dflash	35B-A3B	54	1	1	54	174	67.83	796.1	22.36	44.72	0	3	3	5.438	TRUE
Qwen3.5_FQ+dflash	35B-A3B	85	1	1	85	415	82.64	1028.62	28.49	35.1	0	70000	70000	6.917	TRUE
Qwen3.5_FQ+dflash	35B-A3B	63	1	1	63	253	79.35	793.93	32.56	30.71	0	540	540	7.906	TRUE
Qwen3.5_FQ+dflash	35B-A3B	139	1	1	139	272	89.43	1554.34	29.38	34.03	0	20	20	7.158	TRUE
Qwen3.5_FQ+dflash	35B-A3B	83	1	1	83	460	82.44	1006.82	25.94	38.55	0	64	64	6.301	TRUE
Qwen3.5_FQ+dflash	35B-A3B	69	1	1	69	322	78.65	877.28	44.16	22.65	0	260	260	10.733	TRUE
Qwen3.5_FQ+dflash	35B-A3B	98	1	1	98	374	85.69	1143.7	25.69	38.92	0	160	160	6.233	TRUE
Qwen3.5_FQ+dflash	35B-A3B	132	1	1	132	500	88.67	1488.67	28.53	35.05	0	45	45	6.944	FALSE
Qwen3.5_FQ+dflash	35B-A3B	88	1	1	88	329	83.56	1053.09	26.59	37.61	0	460	460	6.451	TRUE
Qwen3.5_FQ+dflash	35B-A3B	88	1	1	88	435	83.58	1052.91	33.13	30.18	0	366	366	8.056	TRUE
Qwen3.5_FQ+dflash	35B-A3B	93	1	1	93	445	84.33	1102.83	42.45	23.56	0	694	694	10.349	TRUE
Qwen3.5_FQ+dflash	35B-A3B	95	1	1	95	500	84.64	1122.35	22.59	44.27	0	13	13	5.495	FALSE
Qwen3.5_FQ+dflash	35B-A3B	86	1	1	86	500	82.91	1037.26	24.48	40.85	0	18	18	5.952	FALSE
Qwen3.5_FQ+dflash	35B-A3B	80	1	1	80	500	81.67	979.6	34.77	28.76	0	60	60	8.475	TRUE
Qwen3.5_FQ+dflash	35B-A3B	122	1	1	122	349	89.15	1368.5	31.19	32.06	0	125	125	7.587	TRUE
Qwen3.5_FQ+dflash	35B-A3B	81	1	1	81	343	81.99	987.9	23.18	43.15	0	230	230	5.623	TRUE
Qwen3.5_FQ+dflash	35B-A3B	86	1	1	86	472	83.06	1035.41	28.11	35.57	0	57500	57500	6.841	TRUE
Qwen3.5_FQ+dflash	35B-A3B	57	1	1	57	267	76.95	740.76	26.82	37.29	0	7	7	6.512	TRUE
Qwen3.5_FQ+dflash	35B-A3B	93	1	1	93	500	84.49	1100.73	31.06	32.2	0	6	6	7.576	TRUE
Qwen3.5_FQ+dflash	35B-A3B	91	1	1	91	444	83.64	1087.98	32.02	31.23	0	15	15	7.789	TRUE
Qwen3.5_FQ+dflash	35B-A3B	74	1	1	74	279	79.6	929.61	26.7	37.46	0	14	14	6.488	TRUE
Qwen3.5_FQ+dflash	35B-A3B	83	1	1	83	221	82.47	1006.41	39.51	25.31	0	7	7	9.609	TRUE
Qwen3.5_FQ+dflash	35B-A3B	68	1	1	68	216	77.98	871.97	26.12	38.29	0	8	8	6.353	TRUE
Qwen3.5_FQ+dflash	35B-A3B	67	1	1	67	500	77.78	861.41	30.2	33.11	0	25	25	7.353	TRUE
Qwen3.5_FQ+dflash	35B-A3B	95	1	1	95	405	84.92	1118.65	31.96	31.29	0	26	26	7.788	TRUE
Qwen3.5_FQ+dflash	35B-A3B	97	1	1	97	450	85.13	1139.41	36.84	27.14	0	27	27	9	TRUE
Qwen3.5_FQ+dflash	35B-A3B	85	1	1	85	334	82.68	1028.01	28.65	34.91	0	28	28	6.958	TRUE
Qwen3.5_FQ+dflash	35B-A3B	78	1	1	78	263	81.02	962.73	27.07	36.94	0	29	29	6.575	TRUE
Qwen3.5_FQ+dflash	35B-A3B	97	1	1	97	340	84.94	1141.97	27.39	36.51	0	30	30	6.667	TRUE
Qwen3.5_FQ+dflash	35B-A3B	67	1	1	67	338	77.83	860.88	35.65	28.05	0	31	31	8.667	TRUE

Qwen3.5-35B-A3B-GPTQ-Int4 (MoE GPTQ, GPTQ INT4 (W4A16)) 开启 INT8 KV 量化

启动:

scripts/ci/musa/start_qwen35_server.sh --kv-cache-dtype int8

运行 benchmark:

MODEL_KEY=qwen35 scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Ctx(K)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Profile	Prefix(tokens)
Qwen3.5	35b_a3b	128	16	1	1	128	100	50.1	2554.91	15.249	65.579	128:128	0
Qwen3.5	35b_a3b	250	16	1	1	250	100	79.99	3125.373	15.278	65.454	250:250	0
Qwen3.5	35b_a3b	500	16	1	1	500	100	131.015	3816.351	14.978	66.765	500:500	0
Qwen3.5	35b_a3b	1000	16	1	1	1000	100	208.798	4789.323	14.817	67.492	1000:1000	0
Qwen3.5	35b_a3b	2000	16	1	1	2000	100	219.02	9131.571	14.762	67.741	2000:2000	0
Qwen3.5	35b_a3b	250	16	1	1	250	100	80.115	3120.508	14.969	66.803	250:500	250
Qwen3.5	35b_a3b	250	16	1	1	1000	100	79.903	3128.793	14.814	67.503	250:1000	750
Qwen3.5	35b_a3b	250	16	1	1	2000	100	78.808	3172.257	14.778	67.67	250:2000	1750
Qwen3.5	35b_a3b	250	16	1	1	4000	100	69.868	3578.152	14.768	67.713	250:4000	3750
Qwen3.5	35b_a3b	250	16	1	1	8000	100	68.963	3625.127	14.705	68.002	250:8000	7750
Qwen3.5	35b_a3b	250	16	1	1	16000	100	61.807	4044.865	14.38	69.541	250:16000	15750

Qwen3.6_FQ-35B-A3B (MoE GPTQ，GPTQ INT4 (W4A16))

启动：

scripts/ci/musa/start_qwen36_fq_server.sh

运行 benchmark：

MODEL_KEY=qwen36_fq scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Ctx(K)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Profile	Prefix(tokens)	MeasuredOutput(tokens)
Qwen3.6_FQ	35b_a3b	128	16	1	1	128	100	77.586	1649.772	21.15	47.281	128:128	0	100
Qwen3.6_FQ	35b_a3b	250	16	1	1	250	100	77.685	3218.111	20.984	47.655	250:250	0	100
Qwen3.6_FQ	35b_a3b	500	16	1	1	500	100	130.304	3837.19	21.057	47.49	500:500	0	100
Qwen3.6_FQ	35b_a3b	1000	16	1	1	1000	100	199.338	5016.613	21.08	47.439	1000:1000	0	100
Qwen3.6_FQ	35b_a3b	2000	16	1	1	2000	100	200.005	9999.739	21.204	47.162	2000:2000	0	100
Qwen3.6_FQ	35b_a3b	250	16	1	1	500	100	71.765	3483.611	21.286	46.979	250:500	250	100
Qwen3.6_FQ	35b_a3b	250	16	1	1	1000	100	71.735	3485.053	21.269	47.018	250:1000	750	100
Qwen3.6_FQ	35b_a3b	250	16	1	1	2000	100	70.418	3550.218	21.156	47.268	250:2000	1750	100
Qwen3.6_FQ	35b_a3b	250	16	1	1	4000	100	67.146	3723.22	20.689	48.335	250:4000	3750	100
Qwen3.6_FQ	35b_a3b	250	16	1	1	8000	100	68.459	3651.826	20.415	48.985	250:8000	7750	100
Qwen3.6_FQ	35b_a3b	250	16	1	1	16000	100	62.79	3981.532	19.572	51.093	250:16000	15750	100

Qwen3.6_FQ-35B-A3B(MoE GPTQ，GPTQ INT4 (W4A16)) 开启 DFLASH 投机解码

启动：

scripts/ci/musa/start_qwen36_fq_dflash_server.sh

运行 benchmark (GSM8K测试集):

MODEL_KEY=qwen36_fq_dflash scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Prefix(tokens)	Prompt Index	Prediction	Ground Truth	accept len	Correct
Qwen3.6_FQ+dflash	35b_a3b	54	1	1	54	249	53.63	1006.98	31.5	31.74	0	1	3	3	6.917	TRUE
Qwen3.6_FQ+dflash	35b_a3b	85	1	1	85	599	66.36	1280.97	29.67	33.7	0	2	70000	70000	6.511	TRUE
Qwen3.6_FQ+dflash	35b_a3b	63	1	1	63	231	57.91	1087.97	30.97	32.29	0	3	540	540	6.794	TRUE
Qwen3.6_FQ+dflash	35b_a3b	139	1	1	139	353	77.32	1797.76	30.27	33.04	0	4	20	20	6.66	TRUE
Qwen3.6_FQ+dflash	35b_a3b	83	1	1	83	481	65.48	1267.62	26.08	38.34	0	5	64	64	5.726	TRUE
Qwen3.6_FQ+dflash	35b_a3b	69	1	1	69	338	60.32	1143.93	46.57	21.48	0	6	260	260	10.242	TRUE
Qwen3.6_FQ+dflash	35b_a3b	98	1	1	98	501	69.88	1402.47	27.5	36.37	0	7	160	160	6.036	TRUE
Qwen3.6_FQ+dflash	35b_a3b	132	1	1	132	649	76.03	1736.14	35.98	27.79	0	8	45	45	7.915	TRUE
Qwen3.6_FQ+dflash	35b_a3b	88	1	1	88	311	67.16	1310.38	29.54	33.86	0	9	460	460	6.479	TRUE
Qwen3.6_FQ+dflash	35b_a3b	88	1	1	88	420	79.01	1113.81	45.42	22.02	0	10	366	366	10	TRUE
Qwen3.6_FQ+dflash	35b_a3b	93	1	1	93	487	80.2	1159.64	47.07	21.24	0	11	694	694	10.362	TRUE
Qwen3.6_FQ+dflash	35b_a3b	95	1	1	95	581	69.22	1372.37	23.03	43.41	0	12	12	13	5.052	FALSE
Qwen3.6_FQ+dflash	35b_a3b	86	1	1	86	630	66.29	1297.38	36.77	27.2	0	13	18	18	8.077	TRUE
Qwen3.6_FQ+dflash	35b_a3b	80	1	1	80	656	76.79	1041.85	39.27	25.46	0	14	60	60	8.632	TRUE
Qwen3.6_FQ+dflash	35b_a3b	122	1	1	122	283	75.53	1615.23	29.28	34.16	0	15	125	125	6.432	TRUE
Qwen3.6_FQ+dflash	35b_a3b	81	1	1	81	266	64.85	1249.05	32.72	30.56	0	16	230	230	7.189	TRUE
Qwen3.6_FQ+dflash	35b_a3b	86	1	1	86	412	78.53	1095.15	41.61	24.03	0	17	57500	57500	9.156	TRUE
Qwen3.6_FQ+dflash	35b_a3b	57	1	1	57	270	54.98	1036.82	34.18	29.26	0	18	7	7	7.5	TRUE
Qwen3.6_FQ+dflash	35b_a3b	93	1	1	93	548	80.27	1158.59	33.25	30.07	0	19	6	6	7.307	TRUE
Qwen3.6_FQ+dflash	35b_a3b	91	1	1	91	431	67.99	1338.51	29.71	33.65	0	20	15	15	6.53	TRUE
Qwen3.6_FQ+dflash	35b_a3b	74	1	1	74	282	62.28	1188.19	29.15	34.31	0	21	14	14	6.409	TRUE
Qwen3.6_FQ+dflash	35b_a3b	83	1	1	83	215	77.7	1068.15	39.09	25.59	0	22	7	7	8.6	TRUE
Qwen3.6_FQ+dflash	35b_a3b	68	1	1	68	219	59.95	1134.34	43.26	23.11	0	23	8	8	9.522	TRUE
Qwen3.6_FQ+dflash	35b_a3b	67	1	1	67	500	59.5	1126	32.1	31.15	0	24	26	26	7.042	TRUE
Qwen3.6_FQ+dflash	35b_a3b	95	1	1	95	385	80.85	1175.05	35.03	28.54	0	25	2	2	7.7	TRUE
Qwen3.6_FQ+dflash	35b_a3b	97	1	1	97	578	69.71	1391.53	38.64	25.88	0	26	243	243	8.5	TRUE
Qwen3.6_FQ+dflash	35b_a3b	85	1	1	85	337	78.33	1085.22	33.35	29.98	0	27	16	16	7.326	TRUE
Qwen3.6_FQ+dflash	35b_a3b	78	1	1	78	306	63.77	1223.17	34.03	29.39	0	28	25	25	7.463	TRUE
Qwen3.6_FQ+dflash	35b_a3b	97	1	1	97	329	80.86	1199.54	29.94	33.4	0	29	104	104	6.58	TRUE
Qwen3.6_FQ+dflash	35b_a3b	67	1	1	67	300	72.54	923.67	35.91	27.84	0	30	109	109	7.895	TRUE
Qwen3.6_FQ+dflash	35b_a3b	91	1	1	91	353	79.59	1143.36	35.66	28.04	0	31	80	80	7.844	TRUE
Qwen3.6_FQ+dflash	35b_a3b	69	1	1	69	205	73.25	941.96	32.17	31.08	0	32	35	35	7.069	TRUE

Qwen3-VL-8B (VLM GPTQ，GPTQ INT4 (W4A16))

启动：

scripts/ci/musa/start_qwen3vl_server.sh

运行 benchmark：

MODEL_KEY=qwen3vl scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen(tokens)	Ctx(K)	Batch	NChip	Input(tokens)	Output(tokens)	Prefill(tps)	TTFT(ms)	Decode(tps)	TPOT(ms)	Profile	Prefix(tokens)	MeasuredOutput(tokens)
Qwen3VL	8b	128	16	1	1	128	100	95.335	1342.629	13.752	72.716	128:128	0	100
Qwen3VL	8b	250	16	1	1	250	100	162.473	1538.717	13.437	74.419	250:250	0	100
Qwen3VL	8b	500	16	1	1	500	100	252.564	1979.694	11.807	84.693	500:500	0	100
Qwen3VL	8b	1000	16	1	1	1000	100	331.686	3014.904	11.602	86.188	1000:1000	0	100
Qwen3VL	8b	2000	16	1	1	2000	100	314.092	6367.569	11.44	87.415	2000:2000	0	100
Qwen3VL	8b	250	16	1	1	500	100	132.33	1889.21	11.696	85.497	250:500	250	100
Qwen3VL	8b	250	16	1	1	1000	100	134.411	1859.97	11.395	87.76	250:1000	750	100
Qwen3VL	8b	250	16	1	1	2000	100	125.481	1992.33	11.412	87.625	250:2000	1750	100
Qwen3VL	8b	250	16	1	1	4000	100	130.614	1914.035	11.342	88.168	250:4000	3750	100
Qwen3VL	8b	250	16	1	1	8000	100	105.79	2363.174	10.733	93.174	250:8000	7750	100
Qwen3VL	8b	250	16	1	1	16000	100	76.764	3256.751	9.754	102.522	250:16000	15750	100

Model	Size	PrefillLen(k)	Ctx(k)	Batch	NChip	Input(k)	Output(k)	Prefill(tps)	Decode(tps)	Vision(fps)	Precision
Qwen3-VL	8B	0.25	16	1	1	0.25	0.05	161.499	13.463	6.004	w4a16
Qwen3-VL	8B	0.25	16	1	1	0.5	0.05	137.503	10.39	6.004	w4a16
Qwen3-VL	8B	0.25	16	1	1	1	0.05	105.347	10.856	6.004	w4a16
Qwen3-VL	8B	0.25	16	1	1	2	0.05	124.818	9.75	6.004	w4a16
Qwen3-VL	8B	0.25	16	1	1	4	0.05	130.547	9.74	6.004	w4a16
Qwen3-VL	8B	0.25	16	1	1	8	0.05	105.719	9.746	6.004	w4a16
Qwen3-VL	8B	0.25	16	1	1	16	0.05	76.728	9.752	6.004	w4a16

Qwen3.5-35B-A3B (MoE， W4A8_INT8)

启动：

MODEL_PATH=~/models/Qwen3.5-35B-A3B-QQQ-W4A8 ./scripts/ci/musa/start_qwen35_fq_server.sh --quantization qqq

运行 benchmark：

MODEL_KEY=qwen35_fq BENCH_PREFILL_SPECS='128:128, 256:256, 512:512, 1024:1024, 2048:2048' OUTPUT_TOKENS=100 bash scripts/ci/musa/run_tilelang_benchmark.sh

Model	Size	PrefillLen (tokens)	Ctx (K)	Batch	NChip	Input (tokens)	Output (tokens)	Prefill (tps)	TTFT (ms)	Decode (tps)	TPOT (ms)	Profile	Prefix (tokens)	MeasuredOutput (tokens)
Qwen3.5_FQ	35b_a3b	128	16	1	1	128	100	96.936	1320.461	22.005	45.445	128:128	0	100
Qwen3.5_FQ	35b_a3b	256	16	1	1	256	100	142.844	1792.167	22.174	45.098	256:256	0	100
Qwen3.5_FQ	35b_a3b	512	16	1	1	512	100	211.953	2415.635	22.308	44.828	512:512	0	100
Qwen3.5_FQ	35b_a3b	1024	16	1	1	1024	100	268.842	3808.933	22.19	45.065	1024:1024	0	100
Qwen3.5_FQ	35b_a3b	2048	16	1	1	2048	100	265.131	7724.489	22.092	45.266	2048:2048	0	100