Bandwidth saturated by SSD and not PCIe - Synthetic Workload Bandwidth cannot exceed value of 2GB

[Mappouras]

Hello,

I was playing with the simulator. I found it really useful but I had some questions/issues with it.

1. Saturated Bandwidth
   It seems that, for the baseline SSD configuration that is given, the maximum possible IOPS that can be achieved only corresponds to around 1GB/s bandwidth. Even if you increase the amount of workload the IOPs do no increase further.

This seems somewhat strange as the PCIe used provides up to 4GB/s. Especially, given that today we have NVMe cards with 3.5GB/s read bandwidth. I tried to reconfigure the SSD to achieve better bandwidth but I am not sure what options would be more realistic. Should I increase the frequency of the SSD channel, the number of channels, the number of chips per channel?

Do you guys have a configuration suggestion so that the SSD can fully benefit from the PCIe bandwidth and also correspond to a realistic implementation? That would be really helpful.

2. The intensity of each synthetic IO is limited to <2GB/s (2147483648). This problem comes from the definition of the "Bandwidth" parameter as an unsigned int (and not long long) and because it is measured in Bytes. Using values >=2GB causes the simulator to crash as numbers get overflowed. I tried modifying the simulator but there where other errors propagating. A workaround is to use multiple synthetic workloads under the same scenario.

[tmnvnbl]

Hi Mappouras,

I quickly tested this a little too. Without changing the baseline SSD configuration, you can reach high bandwidth by changing the workload. You want to test maximal bandwidth, so you have to define a workload that makes that possible. Sequential reading using large access granularity and a high queue depth should be the optimal scenario. This can be done by using the queue_depth synthetic trace workload, increasing the average queue depth of the workload and increasing the average request size.

A quick test using the default SSD config, and a single sequential reading workload with queue depth 128, average request size 256 sectors (which is 128kB) and a length of 10 seconds gives an output of around 17 kIOPS, which translates to 17kIOPS*128kB request size = 2176MB/s.

This is still not saturated by PCIe, so I will try to look at another use case where this happens, potentially by adding more workloads or giving more internal parallelism in the SSD design.