
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Date: 2026-09-09 15:48
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 15 |     500 |      1 |      151 tok/s |       0.30 |           3.3s |           3.4s |         45ms |          7ms
      vLLM: 15 |     500 |      4 |      462 tok/s |       0.92 |           4.3s |           4.5s |         53ms |          9ms
      vLLM: 15 |     500 |      8 |      779 tok/s |       1.56 |           5.1s |           5.6s |        151ms |         10ms
      vLLM: 15 |     500 |     16 |     1338 tok/s |       2.68 |           6.0s |           6.5s |        163ms |         12ms
      vLLM: 15 |     500 |     32 |     2303 tok/s |       4.61 |           6.9s |           7.1s |        265ms |         13ms
      vLLM: 15 |    4096 |      1 |      146 tok/s |       0.29 |           3.4s |           3.5s |         53ms |          7ms
      vLLM: 15 |    4096 |      4 |      438 tok/s |       0.88 |           4.5s |           4.6s |         71ms |          9ms
      vLLM: 15 |    4096 |      8 |      755 tok/s |       1.51 |           5.3s |           5.4s |         86ms |         10ms
      vLLM: 15 |    4096 |     16 |     1224 tok/s |       2.45 |           6.5s |           6.7s |         89ms |         13ms
      vLLM: 15 |    4096 |     32 |     2074 tok/s |       4.15 |           7.7s |           7.8s |         94ms |         15ms
      vLLM: 15 |   32768 |      1 |      125 tok/s |       0.25 |           4.0s |           4.9s |        314ms |          7ms
      vLLM: 15 |   32768 |      4 |      363 tok/s |       0.73 |           5.5s |           5.6s |        143ms |         11ms
      vLLM: 15 |   32768 |      8 |      550 tok/s |       1.10 |           7.2s |           7.3s |        175ms |         14ms
      vLLM: 15 |   32768 |     16 |      787 tok/s |       1.58 |          10.1s |          10.6s |        194ms |         20ms
      vLLM: 15 |   32768 |     32 |     1221 tok/s |       2.45 |          12.8s |          13.7s |        237ms |         25ms

==============================================================================
