Using the same data across multipe tracks

[sonnyruff]

For the project I'm working on I want to have multiple tracks displaying different parts of one dataframe. This data gets requested with some parameters, but filtering on these parameters takes quite some time. Currenlty the data is requested separately for each track, which wastes a lot of time as it is basically doing the same thing multiple times. Is there an easy way of reusing data in gosling, or is this something I have to handle in the backend?

[etowahadams]

Hi Sonny,

Thanks for your question. Allowing tracks to reuse the same data source is something that is currently not supported in Gosling, but is something that we are working toward.

If you'd like to share a bit more about the particulars of the data source you are using (if its a static file such as a gff file, or higlass-server), I'd be happy to see if there might be some workarounds in the meantime.

[sonnyruff]

The data is a csv file stored locally containing locations of SNPs and SV starts and ends. The idea is to have two tracks showing their locations and one track in between showing correlation. I'm currenlty using pandas to filter on p-value and range around a specified locus and this is where the problem lies.

[etowahadams]

Thank you for sharing. If I understand correctly, you are using gos in python then?

There are two ways to filter the data: 1) using the gosling spec (example of a filter), or 2) precompute the filtered CSV and use it as the data source. For performance, I would recommend the latter (you are likely already doing this).

Do you know how many many rows and columns are in your CSV? In the future we can test the performance using comparable datasets.

[sonnyruff]

I wasn't able to use gos in python, so I'm using Gosling.js. Precomputing the filtered CSV was indeed an idea, but that runs into different problems.

The data comes from CSVs of sizes (7683791,16), (359211,2), (68006,2) and (35133,11). These filtered and joined to form a dataframe of size (12192,12). If you're interested in the specific operations, my code is available on my GitHub page. At the end of the project I'll see to it that a link to the specific datasets is available.