Testing OSv performance on EC2

Basic rules

1. Testing is done on instances with 10G networking
2. Preferred instance type is cc2.8xlarge as is runs on top of the latest Xen version used by Amazon
3. Throughput is tested using netperf application
4. Throughput is measured for two pairs of instances – PV Linux VS HVM Linux and PV Linux VS (HVM) OSv, then results compared
5. Throughput is measured between internal instance IP addresses available via EC2 management Web interface after instance launched
6. All instances reside in the same EC2 placement group during testing
7. Netperf tests being executed are TCP_STREAM (TCP RX), TCP_MAERTS (TCP TX), UDP_STREAM (UDP RX), TCP_RR (roundtrip time)
8. Tests executed for a number of netperf buffer sizes (32 bytes up to 64Kbytes)
9. Netperf invocations and results collection automated using netperf wrapper scripts from Windows KVM guest drivers repository (https://github.com/YanVugenfirer/kvm-guest-drivers-windows/tree/master/NetKVM/Tests)

Test environment creation

1. Login to EC2 management console of desired region: https://console.aws.amazon.com/ec2/v2/home?region=ap-southeast-2
2. Create a placement group: https://console.aws.amazon.com/ec2/home?region=ap-southeast-2#s=PlacementGroups
3. Create a security group allowing all packets in both directions: https://console.aws.amazon.com/ec2/home?region=ap-southeast-2#s=SecurityGroups
4. Launch 2 10G Linux instances – one PV and one HVM using EC2 launch wizard (https://console.aws.amazon.com/ec2/v2/home?region=ap-southeast-2 - LaunchInstanceWizard:), i.e.
   • Amazon Linux AMI x86_64 HVM EBS
   • Amazon Linux AMI x86_64 PV EBS choose hs1.8xlarge instance type, placement group created on step 2, security group created on step 3
5. Launch desired OSv instance of the same type (hs1.8xlarge) into the same placement and security groups, use the same instance type
6. Install netperf on Linux instances
7. Run netserver of HVM Linux instance and OSv
8. Download netperf wrapper scripts to PV Linux instance

Tests execution and results collection

1. Edit netperf_wrapper.rb script, add desired tests to the test list (TEST_CASES variable) and put internal IP address of HVM Linux instance into test_server variable
2. Run netperf_wrapper.rb
3. Edit netperf_wrapper.rb script again, put internal IP address of OSv instance into test_server variable
4. Run netperf_wrapper.rb again
5. Results will be stored in text files named test_log_.log in current working directory
6. Parse testing results with netperf_log_parser.rb, the script outputs tables with throughput measurements results per test and per buffer size