Speed improvements to RadarDataset

[abelsiqueira]

Description

Many improvements to RadarDataset.

Related issues:

Instructions to review the pull request

The following script ran on 271.5 seconds on SURF Research Cloud:

import time

from bird_cloud_gnn.radar_dataset import RadarDataset

st = time.perf_counter()
try:
    dataset = RadarDataset(
        "data/parquet",
        ["x", "y", "z"],
        "BIOLOGY",
        min_neighbours=25,
        max_distance=500.0,
        max_edge_distance=250.1,
    )
    print(dataset)
    print(dataset[0])
    print(dataset[0][0])
except ValueError as ex:
    print("Error:", ex)
print("Total time:", time.perf_counter() - st)

data/parquet contains all data converted to parquet files (114 files). The results:

Dataset("Radar", num_graphs=383288, save_path=/home/asoaressiq/.dgl/Radar)
(Graph(num_nodes=25, num_edges=487,
      ndata_schemes={'x': Scheme(shape=(3,), dtype=torch.float64)}
      edata_schemes={'a': Scheme(shape=(), dtype=torch.float64)}), tensor(0))
Graph(num_nodes=25, num_edges=487,
      ndata_schemes={'x': Scheme(shape=(3,), dtype=torch.float64)}
      edata_schemes={'a': Scheme(shape=(), dtype=torch.float64)})
Total time: 271.5795489749871

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

94.4% Coverage
0.0% Duplication

[bart1]

That looks very quick! I'm currently also playing around with it.