
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Host: ws300
  Platform: Bare Metal
  Date: 2026-09-24 03:02
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 17 |     500 |      1 |      193 tok/s |       0.39 |           2.6s |           2.6s |        146ms |          5ms
      vLLM: 17 |     500 |      4 |      620 tok/s |       1.24 |           3.2s |           3.4s |        281ms |          6ms
      vLLM: 17 |     500 |      8 |     1075 tok/s |       2.15 |           3.7s |           3.7s |        280ms |          7ms
      vLLM: 17 |     500 |     16 |     1824 tok/s |       3.65 |           4.4s |           4.4s |        283ms |          8ms
      vLLM: 17 |     500 |     32 |     2818 tok/s |       5.64 |           5.6s |           5.9s |        441ms |         10ms
      vLLM: 17 |     500 |     64 |     2940 tok/s |       5.88 |          10.8s |          11.0s |         5.7s |         10ms
      vLLM: 17 |    4096 |      1 |      190 tok/s |       0.38 |           2.6s |           2.8s |        165ms |          5ms
      vLLM: 17 |    4096 |      4 |      612 tok/s |       1.22 |           3.3s |           3.3s |        291ms |          6ms
      vLLM: 17 |    4096 |      8 |     1054 tok/s |       2.11 |           3.8s |           3.8s |        301ms |          7ms
      vLLM: 17 |    4096 |     16 |     1668 tok/s |       3.34 |           4.8s |           4.8s |        422ms |          9ms
      vLLM: 17 |    4096 |     32 |     2723 tok/s |       5.45 |           5.9s |           5.9s |        678ms |         10ms
      vLLM: 17 |    4096 |     64 |     2732 tok/s |       5.46 |          11.6s |          11.9s |         6.2s |         11ms
      vLLM: 17 |   32768 |      1 |      182 tok/s |       0.37 |           2.7s |           3.1s |        276ms |          5ms
      vLLM: 17 |   32768 |      4 |      571 tok/s |       1.14 |           3.5s |           3.5s |        483ms |          6ms
      vLLM: 17 |   32768 |      8 |      954 tok/s |       1.91 |           4.2s |           4.2s |        610ms |          7ms
      vLLM: 17 |   32768 |     16 |     1465 tok/s |       2.93 |           5.4s |           5.6s |        741ms |          9ms
      vLLM: 17 |   32768 |     32 |     2229 tok/s |       4.46 |           7.1s |           7.7s |         1.2s |         12ms
      vLLM: 17 |   32768 |     64 |     2233 tok/s |       4.47 |          14.2s |          14.8s |         7.4s |         14ms

==============================================================================
