
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Host: ws300
  Platform: Bare Metal
  Date: 2026-09-22 04:06
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 12 |     500 |      1 |      151 tok/s |       0.30 |           3.3s |           3.3s |        100ms |          6ms
      vLLM: 12 |     500 |      4 |      545 tok/s |       1.09 |           3.7s |           3.7s |        195ms |          7ms
      vLLM: 12 |     500 |      8 |     1041 tok/s |       2.08 |           3.8s |           3.9s |        196ms |          7ms
      vLLM: 12 |     500 |     16 |     1907 tok/s |       3.82 |           4.2s |           4.2s |        254ms |          8ms
      vLLM: 12 |     500 |     32 |     3256 tok/s |       6.51 |           4.9s |           5.0s |        458ms |          9ms
      vLLM: 12 |     500 |     64 |     3234 tok/s |       6.47 |           9.8s |          10.0s |         5.2s |          9ms
      vLLM: 12 |    4096 |      1 |      149 tok/s |       0.30 |           3.4s |           3.4s |        120ms |          6ms
      vLLM: 12 |    4096 |      4 |      538 tok/s |       1.08 |           3.7s |           3.7s |        206ms |          7ms
      vLLM: 12 |    4096 |      8 |     1011 tok/s |       2.02 |           4.0s |           4.0s |        249ms |          7ms
      vLLM: 12 |    4096 |     16 |     1781 tok/s |       3.56 |           4.5s |           4.5s |        426ms |          8ms
      vLLM: 12 |    4096 |     32 |     2925 tok/s |       5.85 |           5.5s |           5.5s |        668ms |         10ms
      vLLM: 12 |    4096 |     64 |     2977 tok/s |       5.96 |          10.7s |          10.8s |         5.8s |         10ms
      vLLM: 12 |   32768 |      1 |      139 tok/s |       0.28 |           3.6s |           4.1s |        270ms |          7ms
      vLLM: 12 |   32768 |      4 |      478 tok/s |       0.96 |           4.2s |           4.2s |        287ms |          8ms
      vLLM: 12 |   32768 |      8 |      819 tok/s |       1.64 |           4.9s |           5.0s |        376ms |          9ms
      vLLM: 12 |   32768 |     16 |     1320 tok/s |       2.64 |           6.1s |           6.2s |        623ms |         11ms
      vLLM: 12 |   32768 |     32 |     2036 tok/s |       4.08 |           7.8s |           8.1s |        773ms |         14ms
      vLLM: 12 |   32768 |     64 |     1999 tok/s |       4.00 |          15.7s |          16.2s |         8.0s |         15ms

==============================================================================
