
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Host: ws300
  Platform: Bare Metal
  Date: 2026-09-25 05:42
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 19 |     500 |      1 |      139 tok/s |       0.28 |           3.6s |           5.2s |        211ms |          7ms
      vLLM: 19 |     500 |      4 |      428 tok/s |       0.86 |           4.7s |           6.9s |        356ms |          9ms
      vLLM: 19 |     500 |      8 |      667 tok/s |       1.33 |           6.0s |           7.8s |        480ms |         11ms
      vLLM: 19 |     500 |     16 |     1116 tok/s |       2.23 |           7.2s |           7.3s |        320ms |         14ms
      vLLM: 19 |     500 |     32 |     1677 tok/s |       3.35 |           9.5s |           9.6s |        498ms |         18ms
      vLLM: 19 |     500 |     64 |     1682 tok/s |       3.36 |          18.2s |          19.1s |         9.0s |         18ms
      vLLM: 19 |    4096 |      1 |      137 tok/s |       0.27 |           3.6s |           5.0s |        211ms |          7ms
      vLLM: 19 |    4096 |      4 |      411 tok/s |       0.82 |           4.9s |           4.9s |        436ms |          9ms
      vLLM: 19 |    4096 |      8 |      626 tok/s |       1.25 |           6.3s |           6.5s |        694ms |         11ms
      vLLM: 19 |    4096 |     16 |      903 tok/s |       1.81 |           8.8s |           9.8s |        874ms |         16ms
      vLLM: 19 |    4096 |     32 |     1203 tok/s |       2.42 |          12.9s |          15.4s |         1.0s |         24ms
      vLLM: 19 |    4096 |     64 |     1212 tok/s |       2.42 |          24.3s |          28.5s |        12.2s |         24ms
      vLLM: 19 |   32768 |      1 |      113 tok/s |       0.25 |           4.1s |           6.0s |        927ms |          7ms
      vLLM: 19 |   32768 |      4 |      397 tok/s |       0.84 |           4.7s |           5.1s |        267ms |          9ms
      vLLM: 19 |   32768 |      8 |      611 tok/s |       1.28 |           6.1s |           6.6s |        407ms |         12ms
      vLLM: 19 |   32768 |     16 |      886 tok/s |       1.88 |           8.3s |           9.0s |        471ms |         17ms
      vLLM: 19 |   32768 |     32 |     1219 tok/s |       2.61 |          11.6s |          12.7s |        791ms |         23ms
      vLLM: 19 |   32768 |     64 |     1223 tok/s |       2.60 |          21.7s |          25.1s |        10.7s |         24ms

==============================================================================
