
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Date: 2026-09-25 12:33
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 18 |     500 |      1 |      258 tok/s |       0.52 |           1.9s |           1.9s |        115ms |          4ms
      vLLM: 18 |     500 |      4 |      830 tok/s |       1.66 |           2.4s |           2.4s |        228ms |          4ms
      vLLM: 18 |     500 |      8 |     1527 tok/s |       3.05 |           2.6s |           2.7s |        224ms |          5ms
      vLLM: 18 |     500 |     16 |     2645 tok/s |       5.29 |           3.0s |           3.0s |        229ms |          6ms
      vLLM: 18 |     500 |     32 |     4374 tok/s |       8.75 |           3.7s |           3.7s |        332ms |          7ms
      vLLM: 18 |     500 |     64 |     4386 tok/s |       8.77 |           7.3s |           7.3s |         3.8s |          7ms
      vLLM: 18 |    4096 |      1 |      254 tok/s |       0.51 |           2.0s |           2.1s |        137ms |          4ms
      vLLM: 18 |    4096 |      4 |      815 tok/s |       1.63 |           2.5s |           2.5s |        241ms |          4ms
      vLLM: 18 |    4096 |      8 |     1492 tok/s |       2.98 |           2.7s |           2.7s |        244ms |          5ms
      vLLM: 18 |    4096 |     16 |     2480 tok/s |       4.96 |           3.2s |           3.2s |        346ms |          6ms
      vLLM: 18 |    4096 |     32 |     4005 tok/s |       8.01 |           4.0s |           4.0s |        505ms |          7ms
      vLLM: 18 |    4096 |     64 |     4035 tok/s |       8.07 |           7.9s |           8.0s |         4.2s |          7ms
      vLLM: 18 |   32768 |      1 |      244 tok/s |       0.49 |           2.1s |           2.4s |        150ms |          4ms
      vLLM: 18 |   32768 |      4 |      776 tok/s |       1.55 |           2.6s |           2.6s |        150ms |          5ms
      vLLM: 18 |   32768 |      8 |     1294 tok/s |       2.59 |           3.1s |           3.1s |        160ms |          6ms
      vLLM: 18 |   32768 |     16 |     1970 tok/s |       3.94 |           4.0s |           4.1s |        150ms |          8ms
      vLLM: 18 |   32768 |     32 |     2299 tok/s |       4.60 |           6.9s |           7.1s |        425ms |         13ms
      vLLM: 18 |   32768 |     64 |     2163 tok/s |       4.33 |          14.6s |          15.1s |         7.4s |         14ms

==============================================================================
