
==============================================================================
  BENCHMARK SUMMARY
==============================================================================
  Host: ws300
  Platform: Bare Metal
  Date: 2026-09-22 08:31
==============================================================================

     Benchmark |     Ctx |   Conc |     Throughput |      Req/s |    Avg Latency |    P99 Latency |   TTFT (avg) |    ITL (avg)
  -------------+---------+--------+----------------+------------+----------------+----------------+--------------+-------------
      vLLM: 12 |     500 |      1 |      204 tok/s |       0.41 |           2.4s |           3.1s |        108ms |          5ms
      vLLM: 12 |     500 |      4 |      665 tok/s |       1.33 |           3.0s |           4.0s |        180ms |          6ms
      vLLM: 12 |     500 |      8 |     1141 tok/s |       2.28 |           3.5s |           4.6s |        158ms |          7ms
      vLLM: 12 |     500 |     16 |     1757 tok/s |       3.51 |           4.5s |           6.1s |        168ms |          9ms
      vLLM: 12 |     500 |     32 |     2422 tok/s |       4.84 |           6.6s |           8.7s |        190ms |         13ms
      vLLM: 12 |     500 |     64 |     2424 tok/s |       4.85 |          13.0s |          15.3s |         6.6s |         13ms
      vLLM: 12 |    4096 |      1 |      174 tok/s |       0.35 |           2.9s |           3.3s |        130ms |          5ms
      vLLM: 12 |    4096 |      4 |      584 tok/s |       1.17 |           3.4s |           3.9s |        155ms |          7ms
      vLLM: 12 |    4096 |      8 |      995 tok/s |       1.99 |           4.0s |           4.6s |        163ms |          8ms
      vLLM: 12 |    4096 |     16 |     1537 tok/s |       3.07 |           5.2s |           6.0s |        183ms |         10ms
      vLLM: 12 |    4096 |     32 |     2137 tok/s |       4.28 |           7.4s |           8.8s |        222ms |         14ms
      vLLM: 12 |    4096 |     64 |     2162 tok/s |       4.32 |          14.5s |          16.2s |         7.4s |         14ms
      vLLM: 12 |   32768 |      1 |      164 tok/s |       0.33 |           3.1s |           4.0s |        273ms |          6ms
      vLLM: 12 |   32768 |      4 |      559 tok/s |       1.12 |           3.6s |           4.1s |        230ms |          7ms
      vLLM: 12 |   32768 |      8 |      935 tok/s |       1.87 |           4.2s |           5.0s |        252ms |          8ms
      vLLM: 12 |   32768 |     16 |     1414 tok/s |       2.84 |           5.6s |           6.6s |        276ms |         11ms
      vLLM: 12 |   32768 |     32 |     1987 tok/s |       3.98 |           8.0s |           9.3s |        337ms |         15ms
      vLLM: 12 |   32768 |     64 |     1970 tok/s |       3.95 |          15.9s |          17.7s |         8.1s |         16ms

==============================================================================
