
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Host: ws300
  Platform: Bare Metal
  Date: 2026-09-25 08:12
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 19 |     500 |      1 |      141 tok/s |       0.28 |           3.5s |           3.5s |        149ms |          7ms
      vLLM: 19 |     500 |      4 |      436 tok/s |       0.87 |           4.6s |           5.2s |        285ms |          9ms
      vLLM: 19 |     500 |      8 |      701 tok/s |       1.40 |           5.7s |           7.9s |        314ms |         11ms
      vLLM: 19 |     500 |     16 |     1121 tok/s |       2.24 |           7.1s |           7.2s |        320ms |         14ms
      vLLM: 19 |     500 |     32 |     1662 tok/s |       3.32 |           9.6s |           9.7s |        483ms |         18ms
      vLLM: 19 |     500 |     64 |     1667 tok/s |       3.33 |          19.0s |          19.3s |         9.7s |         19ms
      vLLM: 19 |    4096 |      1 |      138 tok/s |       0.28 |           3.6s |           3.6s |        162ms |          7ms
      vLLM: 19 |    4096 |      4 |      410 tok/s |       0.82 |           4.9s |           4.9s |        357ms |          9ms
      vLLM: 19 |    4096 |      8 |      630 tok/s |       1.26 |           6.3s |           6.5s |        643ms |         11ms
      vLLM: 19 |    4096 |     16 |      905 tok/s |       1.82 |           8.8s |           9.1s |        765ms |         16ms
      vLLM: 19 |    4096 |     32 |     1223 tok/s |       2.45 |          12.8s |          14.4s |        853ms |         24ms
      vLLM: 19 |    4096 |     64 |     1234 tok/s |       2.47 |          25.3s |          27.5s |        13.1s |         25ms
      vLLM: 19 |   32768 |      1 |      131 tok/s |       0.28 |           3.6s |           4.5s |        369ms |          7ms
      vLLM: 19 |   32768 |      4 |      399 tok/s |       0.84 |           4.8s |           5.1s |        243ms |          9ms
      vLLM: 19 |   32768 |      8 |      617 tok/s |       1.28 |           6.2s |           6.7s |        296ms |         12ms
      vLLM: 19 |   32768 |     16 |      890 tok/s |       1.84 |           8.6s |           9.3s |        342ms |         17ms
      vLLM: 19 |   32768 |     32 |     1241 tok/s |       2.61 |          12.1s |          13.3s |        455ms |         25ms
      vLLM: 19 |   32768 |     64 |     1241 tok/s |       2.61 |          23.9s |          26.0s |        12.0s |         25ms

==============================================================================
