
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Host: ws300
  Platform: Bare Metal
  Date: 2026-09-18 08:41
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 16 |     500 |      1 |      153 tok/s |       0.31 |           3.2s |           3.1s |        329ms |          6ms
      vLLM: 16 |     500 |      4 |      450 tok/s |       0.91 |           4.4s |          16.2s |        406ms |          8ms
      vLLM: 16 |     500 |      8 |      694 tok/s |       1.40 |           5.7s |          17.3s |        481ms |         10ms
      vLLM: 16 |     500 |     16 |      833 tok/s |       1.68 |           9.5s |          28.9s |         2.3s |         15ms
      vLLM: 16 |     500 |     32 |     1480 tok/s |       2.99 |          10.6s |          31.0s |         1.5s |         18ms
      vLLM: 16 |     500 |     64 |     1621 tok/s |       3.27 |          19.2s |          40.0s |        10.3s |         18ms
      vLLM: 16 |    4096 |      1 |      150 tok/s |       0.31 |           3.2s |           3.2s |        348ms |          6ms
      vLLM: 16 |    4096 |      4 |      466 tok/s |       0.95 |           4.2s |           4.4s |        202ms |          8ms
      vLLM: 16 |    4096 |      8 |      719 tok/s |       1.47 |           5.4s |           5.8s |        234ms |         11ms
      vLLM: 16 |    4096 |     16 |     1076 tok/s |       2.19 |           7.3s |           8.0s |        240ms |         14ms
      vLLM: 16 |    4096 |     32 |     1471 tok/s |       3.00 |          10.5s |          28.2s |        799ms |         20ms
      vLLM: 16 |    4096 |     64 |     1564 tok/s |       3.19 |          19.6s |          22.1s |         9.9s |         20ms
      vLLM: 16 |   32768 |      1 |      149 tok/s |       0.33 |           3.1s |           3.8s |        329ms |          6ms
      vLLM: 16 |   32768 |      4 |      443 tok/s |       0.97 |           4.1s |           4.8s |        278ms |          8ms
      vLLM: 16 |   32768 |      8 |      671 tok/s |       1.47 |           5.4s |           6.3s |        315ms |         11ms
      vLLM: 16 |   32768 |     16 |      964 tok/s |       2.10 |           7.5s |           8.7s |        360ms |         16ms
      vLLM: 16 |   32768 |     32 |     1327 tok/s |       2.92 |          10.8s |          12.7s |        417ms |         23ms
      vLLM: 16 |   32768 |     64 |     1290 tok/s |       2.86 |          21.8s |          24.7s |        11.0s |         24ms

==============================================================================
