Infrastructure estimates
At 100 requests per second, how many are in flight?
Tools to use
Does handling 100 requests per second require 100 execution slots? It depends on residence time: a job lasting 0.01 seconds and one waiting two seconds produce different concurrent populations. This tool estimates a stable system's average, helping relate observations rather than choosing a server count.
100 per second and 0.2 seconds give an average of 20
L = λ × W
100 requests/second × 0.2 seconds = 20 requestsChoose average concurrency, rate 100 and mean residence 0.2 seconds: the result is 20. Seconds cancel, leaving requests. This is neither 100 users nor 100 CPUs. Define what counts as one request and which system boundary it occupies.
Convert 200 ms to 0.2 seconds. Entering 200 means 200 seconds and produces 20000, a thousandfold error. Likewise 100 per minute is not 100 per second. Keep the formula and units with the result before interpreting it as a performance problem.
Residence time can include waiting
For a boundary from acceptance to response, residence includes queueing and external waits. CPU-active time answers a different question. Mixing whole-API arrivals with database-only timing loses a consistent meaning. Use rates and times for the same population crossing the same boundary.
If 0.15 of the 0.2 seconds is an external wait, twenty resident requests do not imply twenty continuously busy CPUs. Async and connection handling change resource use. Separate population estimates from CPU-time and connection-pool sizing.
Slower completion increases the population at the same rate
100/s × 0.2 s = 20
100/s × 0.5 s = 50
100/s × 2.0 s = 200Keep rate 100 and try 0.5 and 2 seconds: averages become 50 and 200. Concurrency can grow without an arrival spike when completion slows. The equation explains the relationship, not the cause of the longer time; investigate the observed timing breakdown separately.
Reducing mean residence lowers average population at the same rate. But an improvement only in the fastest request is not evidence for changing the mean input. Compare like request populations and averaging methods before and after a change.
An average of 20 is not a peak or capacity guarantee
Bursts and long requests can push instantaneous counts above the average. Twenty average does not prove a limit of twenty avoids waiting. Multiplying rate by p95 or p99 latency also does not produce p95 or p99 concurrency: this is a relation between averages, not quantiles.
A rounded-up reference value is still not a guaranteed resource count. An average of 0.5 includes empty periods, not half a physical request. Choosing capacity needs waiting-time objectives, peaks, failures and load tests beyond this mean relationship.
Check that work is not accumulating indefinitely
The tool's model assumes a stable average system. Do not interpret an interval of continuously growing backlog as a sustainable steady state. Initial backlog, failures, cancellations and requests crossing the observation boundary affect aggregation. Check arrivals and completions over a comparable interval.
Report the rate, mean time and boundary: '100 per second, 0.2 seconds from acceptance to response, average 20 resident requests'. Use it to explain waiting or spot inconsistent measurements, not as a replacement for CPU or maximum-connection sizing.