Gem5_Garnet-Sythetic_traffic-Garnet_standalone-false_outouts

[AggelosKv]

Dear gem5 Community,

The problem I encounter is related with my inability to explain the output of the metrics that heterogarnet provided.

The environment that I work in WSL(Windows Subsystem for Linux) from VSCode.

step_1)
As written clearly inside the: https://www.gem5.org/documentation/learning_gem5/part1/building/
using the command :
python3 which scons build/NULL/gem5.opt -j3

step_2)
and after that from : https://www.gem5.org/documentation/general_docs/ruby/garnet_synthetic_traffic/
using the following:
scons defconfig build/NULL build_opts/NULL
scons setconfig build/NULL RUBY_PROTOCOL_GARNET_STANDALONE=y
scons build/NULL/gem5.debug

step_3)
and I also made some changes inside "/gem5/src/mem/ruby/network/garnet/GarnetNetwork.py"
altering (the lines 47 and 48) buffers_per_data_vc and buffers_per_ctrl_vc to 8 before step_2, to change the data and ctrl buffer size.

The environment is ready.

Then with an automated script I run the command below with injection rate form 0.01 to 0.15 with both packet lengths 1024 and 120 separately (the script stops when latency is above 100 cycles):

./build/NULL/gem5.debug configs/example/garnet_synth_traffic.py
--num-cpus=64
--num-dirs=64
--network=garnet
--topology=Mesh_XY
--mesh-rows=8
--sim-cycles=100000000
--synthetic=uniform_random
--inj-vne=2
--injectionrate=$injectionrate
--router-latency=1
--link-latency=1
--vcs-per-vnet=1
--routing-algorithm=1
--sys-clock=1GHz
--ruby-clock=1GHz
--link-width-bits=$link_width_bits

the problem I encounter is that the network with 1 flit is experiencing congestion too early in contrast to the 5 flit network.
let me give an example:
for 5 flit and injection rate 0.025 I get 40474.502119 = ~40 cyclesAveragePacketLatency
for 1 flit and injection rate 0.07 I get 2687925.510492 = ~2688 cycles AveragePacketLatency (above 100 cycles == conjestion!)

why with 0.025*5 = 0.125 flits injection rate (5 flit network) the network is fine and with 0.07 flits injection rate (1 flit network) the network latency explodes?!
any commend would be quite helpful. I will also privide the metrics in excel + my scripts for anyone that wants to use them
Thank you in advance!

---

---

extract_network_stats.sh

#!/bin/bash

# Output file
OUTPUT_FILE=./my_scripts/"network_stats.txt"

injectionrate=$(grep "inj_rate=" m5out/config.ini | head -n 1 | awk -F'=' '{print $2}')

if [ ! -z "$1" ]; then
    injectionrate=$1
fi

# If the file doesn't exist, create it and add the column headers
if [ ! -f $OUTPUT_FILE ]; then
    echo "Injection Rate | Average Flit Queueing Latency | Average Flit Network Latency | Average Flit Latency  | Average Packet Queueing Latency | Average Packet Network Latency | Average Packet Latency | Flits Injected | Flits Received | Packets Injected | Packets Received | Average Hops" > $OUTPUT_FILE
fi

# Extract values from m5out/stats.txt using awk to handle varying spaces
injectionrate=$injectionrate  # Pass the injectionrate as the first argument
average_flit_queueing_latency=$(grep "average_flit_queueing_latency" ./m5out/stats.txt | awk '{print $2}')
average_flit_network_latency=$(grep "average_flit_network_latency" ./m5out/stats.txt | awk '{print $2}')
average_flit_latency=$(grep "average_flit_latency" ./m5out/stats.txt | awk '{print $2}')
average_packet_queueing_latency=$(grep "average_packet_queueing_latency" ./m5out/stats.txt | awk '{print $2}')
average_packet_network_latency=$(grep "average_packet_network_latency" ./m5out/stats.txt | awk '{print $2}')
average_packet_latency=$(grep "average_packet_latency" ./m5out/stats.txt | awk '{print $2}')
flits_injected=$(grep "flits_injected::total" ./m5out/stats.txt | awk '{print $2}')
flits_received=$(grep "flits_received::total" ./m5out/stats.txt | awk '{print $2}')
packets_injected=$(grep "system.ruby.network.packets_injected::total" ./m5out/stats.txt | awk '{print $2}')
packets_received=$(grep "packets_received::total" ./m5out/stats.txt | awk '{print $2}')
average_hops=$(grep "average_hops" ./m5out/stats.txt | awk '{print $2}')

# Append the values in a row format with | separator
echo "$injectionrate | $average_flit_queueing_latency | $average_flit_network_latency | $average_flit_latency | $average_packet_queueing_latency | $average_packet_network_latency | $average_packet_latency | $flits_injected | $flits_received  | $packets_injected | $packets_received | $average_hops" >> $OUTPUT_FILE

---

run_Garnet_Synthetic_Traffic.sh

#!/bin/bash

# Default injection rate (in case it's not passed as an argument)
injectionrate=0.25

# Default link width in bits (in case t's not passed as an argument)
link_width_bits=1024

# Default router latency (in case it's not passed as an argument)
router_latency=1

# Default link latency (in case it's not passed as an argument)
link_latency=1

# Default sending pattern (in case it's not passed as an argument)
#synthetic=neighbor
synthetic=uniform_random

# Default vsc number (in case it's not passed as an argument)
vcs_num=1

# Check if an injection rate argument was provided
if [ ! -z "$1" ]; then
    injectionrate=$1
fi

# Check if the second argument (link width in bits) was provided
if [ ! -z "$2" ]; then
    link_width_bits=$2
fi

# Check if the third argument (router latency) was provided
if [ ! -z "$3" ]; then
    router_latency=$3
fi
# Check if the fourth argument (link latency) was provided
if [ ! -z "$4" ]; then
    link_latency=$4
fi

# Check if the fourth argument (link latency) was provided
if [ ! -z "$5" ]; then
    synthetic=$5
fi

# Check if an injection rate argument was provided
if [ ! -z "$6" ]; then
    vcs_num=$6
fi

# Run the gem5 command with the specified or default injection rate
./build/NULL/gem5.debug configs/example/garnet_synth_traffic.py \
    --num-cpus=64 \
    --num-dirs=64 \
    --network=garnet \
    --topology=Mesh_XY \
    --mesh-rows=8 \
    --sim-cycles=100000000 \
    --synthetic=$synthetic \
    --inj-vne=2 \
    --injectionrate=$injectionrate \
    --router-latency=$router_latency \
    --link-latency=$link_latency \
    --vcs-per-vnet=1 \
    --routing-algorithm=1 \
    --sys-clock=0.5GHz \
    --ruby-clock=1GHz \
    --link-width-bits=$link_width_bits 



# Optional: Message when simulation finishes
echo "Simulation with injection rate $injectionrate completed!"

---

run_inj_rate_and_synth_for_loop.sh

#!/bin/bash

# Run this script from the gem5 directory

# Master output file for all the results
MASTER_OUTPUT_FILE=./my_scripts/"network_stats.txt"
# Test stats file that will store results for each traffic pattern
TEST_STATS_FILE=./my_scripts/"test_stats.txt"

# Clear the master output file before writing to it
> $MASTER_OUTPUT_FILE

# Clear or create the test_stats.txt file to store cumulative results
> $TEST_STATS_FILE

# Target value to search for in the extracted network stats (max_packet_latency)
TARGET_VALUE="100000.0"

# Define the traffic patterns
traffic_patterns=("uniform_random" "transpose" "bit_complement")

# Outer loop: Loop through each traffic pattern
for synthetic in "${traffic_patterns[@]}"; do
    echo "Running simulations with traffic pattern: $synthetic"

    # Inner loop: Loop through injection rates from 0.00005 to 0.01 with a step of 0.00005
    for injectionrate in $(seq 0.01 0.005 0.1); do
        echo "Running simulation with injection rate: $injectionrate and traffic pattern: $synthetic"

        # Run the run_garnet.sh script with the updated injection rate and traffic pattern
        ./my_scripts/run_Garnet_Synthetic_Traffic.sh $injectionrate 1024 1 1 $synthetic 1

        # Assuming the stats.txt file gets updated by the simulation run
        # Run the script that generates network_stats.txt and capture the average_packet_latency
        average_packet_latency=$(./my_scripts/extract_network_stats_return_pack_let.sh $injectionrate)

        # Check if the target packet latency has been reached
        if awk "BEGIN {exit ($average_packet_latency >= $TARGET_VALUE) ? 0 : 1}"; then
            # Copy the contents of network_stats.txt into test_stats.txt and prepend the traffic pattern
            echo "Target average packet latency ($TARGET_VALUE) reached or exceeded! Stopping further simulations for this traffic pattern."
            echo "$synthetic with 1 vc and 1 flit:" >> $TEST_STATS_FILE
            echo "Injection Rate | Average Flit Queueing Latency | Average Flit Network Latency | Average Flit Latency  | Average Packet Queueing Latency | Average Packet Network Latency | Average Packet Latency | Flits Injected | Flits Received | Packets Injected | Packets Received | Average Hops" >> $TEST_STATS_FILE
            cat $MASTER_OUTPUT_FILE >> $TEST_STATS_FILE
            echo "" >> $TEST_STATS_FILE  # Add a blank line for readability between sections
           > $MASTER_OUTPUT_FILE
            break
        else
            echo "Target average packet latency not reached, continuing to next injection rate."
        fi

    done
    
        # Inner loop: Loop through injection rates from 0.00005 to 0.01 with a step of 0.00005
    for injectionrate in $(seq 0.01 0.0025 0.1); do
        echo "Running simulation with injection rate: $injectionrate and traffic pattern: $synthetic"

        # Run the run_garnet.sh script with the updated injection rate and traffic pattern
        ./my_scripts/run_Garnet_Synthetic_Traffic.sh $injectionrate 120 1 1 $synthetic 1

        # Assuming the stats.txt file gets updated by the simulation run
        # Run the script that generates network_stats.txt and capture the average_packet_latency
        average_packet_latency=$(./my_scripts/extract_network_stats_return_pack_let.sh $injectionrate)

        # Check if the target packet latency has been reached
        if awk "BEGIN {exit ($average_packet_latency >= $TARGET_VALUE) ? 0 : 1}"; then
            # Copy the contents of network_stats.txt into test_stats.txt and prepend the traffic pattern
            echo "Target average packet latency ($TARGET_VALUE) reached or exceeded! Stopping further simulations for this traffic pattern."
            echo "$synthetic with 1 vc and 5 flits:" >> $TEST_STATS_FILE
            echo "Injection Rate | Average Flit Queueing Latency | Average Flit Network Latency | Average Flit Latency  | Average Packet Queueing Latency | Average Packet Network Latency | Average Packet Latency | Flits Injected | Flits Received | Packets Injected | Packets Received | Average Hops" >> $TEST_STATS_FILE
            cat $MASTER_OUTPUT_FILE >> $TEST_STATS_FILE
            echo "" >> $TEST_STATS_FILE  # Add a blank line for readability between sections
           > $MASTER_OUTPUT_FILE
            break
        else
            echo "Target average packet latency not reached, continuing to next injection rate."
        fi

    done

            # Inner loop: Loop through injection rates from 0.00005 to 0.01 with a step of 0.00005
    for injectionrate in $(seq 0.01 0.005 0.3); do
        echo "Running simulation with injection rate: $injectionrate and traffic pattern: $synthetic"

        # Run the run_garnet.sh script with the updated injection rate and traffic pattern
        ./my_scripts/run_Garnet_Synthetic_Traffic.sh $injectionrate 1024 1 1 $synthetic 2

        # Assuming the stats.txt file gets updated by the simulation run
        # Run the script that generates network_stats.txt and capture the average_packet_latency
        average_packet_latency=$(./my_scripts/extract_network_stats_return_pack_let.sh $injectionrate)

        # Check if the target packet latency has been reached
        if awk "BEGIN {exit ($average_packet_latency >= $TARGET_VALUE) ? 0 : 1}"; then
            # Copy the contents of network_stats.txt into test_stats.txt and prepend the traffic pattern
            echo "Target average packet latency ($TARGET_VALUE) reached or exceeded! Stopping further simulations for this traffic pattern."
            echo "$synthetic with 2vcs and 1 flit::" >> $TEST_STATS_FILE
            echo "Injection Rate | Average Flit Queueing Latency | Average Flit Network Latency | Average Flit Latency  | Average Packet Queueing Latency | Average Packet Network Latency | Average Packet Latency | Flits Injected | Flits Received | Packets Injected | Packets Received | Average Hops" >> $TEST_STATS_FILE
            cat $MASTER_OUTPUT_FILE >> $TEST_STATS_FILE
            echo "" >> $TEST_STATS_FILE  # Add a blank line for readability between sections
           > $MASTER_OUTPUT_FILE
            break
        else
            echo "Target average packet latency not reached, continuing to next injection rate."
        fi

    done

            # Inner loop: Loop through injection rates from 0.00005 to 0.01 with a step of 0.00005
    for injectionrate in $(seq 0.01 0.0025 0.1); do
        echo "Running simulation with injection rate: $injectionrate and traffic pattern: $synthetic"

        # Run the run_garnet.sh script with the updated injection rate and traffic pattern
        ./my_scripts/run_Garnet_Synthetic_Traffic.sh $injectionrate 120 1 1 $synthetic 2

        # Assuming the stats.txt file gets updated by the simulation run
        # Run the script that generates network_stats.txt and capture the average_packet_latency
        average_packet_latency=$(./my_scripts/extract_network_stats_return_pack_let.sh $injectionrate)

        # Check if the target packet latency has been reached
        if awk "BEGIN {exit ($average_packet_latency >= $TARGET_VALUE) ? 0 : 1}"; then
            # Copy the contents of network_stats.txt into test_stats.txt and prepend the traffic pattern
            echo "Target average packet latency ($TARGET_VALUE) reached or exceeded! Stopping further simulations for this traffic pattern."
            echo "$synthetic with 2vcs and 5 flits:" >> $TEST_STATS_FILE
            echo "Injection Rate | Average Flit Queueing Latency | Average Flit Network Latency | Average Flit Latency  | Average Packet Queueing Latency | Average Packet Network Latency | Average Packet Latency | Flits Injected | Flits Received | Packets Injected | Packets Received | Average Hops" >> $TEST_STATS_FILE
            cat $MASTER_OUTPUT_FILE >> $TEST_STATS_FILE
            echo "" >> $TEST_STATS_FILE  # Add a blank line for readability between sections
           > $MASTER_OUTPUT_FILE
            break
        else
            echo "Target average packet latency not reached, continuing to next injection rate."
        fi

    done

    # Optional: Message to indicate the completion of simulations for each traffic pattern
    echo "Simulations for traffic pattern $synthetic completed."
done

# Final message
echo "All simulations completed and results saved to $MASTER_OUTPUT_FILE and $TEST_STATS_FILE"