Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: /project/children endpoint is slow #5199

Open
MichaelAkvo opened this issue Jan 23, 2023 · 0 comments · Fixed by #5204
Open

Bug: /project/children endpoint is slow #5199

MichaelAkvo opened this issue Jan 23, 2023 · 0 comments · Fixed by #5204

Comments

@MichaelAkvo
Copy link
Contributor

What were you doing?

When accesing the hierarchies view for EUTF Africa, the program has 265 child projects and it takes about 20s to load them all. /project/children is a paged call and has to be called multiple times to get all children. Each call takes bout 2s.

What should've happened?

Each call should really only take a 10s of milliseconds not thousands.

My environment

No response

Additional context

The frontend also doesn't add the projects as they arrive, instead it waits for all calls to complete. Another bug report should probably be created.

@MichaelAkvo MichaelAkvo added this to RSR Jan 24, 2023
@MichaelAkvo MichaelAkvo moved this to Todo in RSR Jan 24, 2023
@MichaelAkvo MichaelAkvo self-assigned this Jan 24, 2023
MichaelAkvo added a commit that referenced this issue Jan 25, 2023
# Counting children

This was slow because we made a query for each project.
Now we get the children and grand children.
A simple dict with each project and a list of their children is then built in python.

**Side effect**

Paging doesn't work anymore as the queryset includes grand children.
Reimplementing it can be done another time if we really do have projects with enormous hierarchies.

# Parent retrieval

We already know who the parent is since we're getting its children --> no need to make another query.

# Locations

We weren't retrieving related objects, which we now do.
Additionally, we iterated over the list of countries twice when one iteration was enough.
@MichaelAkvo MichaelAkvo moved this from Todo to In Progress in RSR Jan 30, 2023
MichaelAkvo added a commit that referenced this issue Jan 30, 2023
# Counting children

This was slow because we made a query for each project.
Now we get the children and grand children.
A simple dict with each project and a list of their children is then built in python.

**Side effect**

Paging doesn't work anymore as the queryset includes grand children.
Reimplementing it can be done another time if we really do have projects with enormous hierarchies.

# Parent retrieval

We already know who the parent is since we're getting its children --> no need to make another query.

# Locations

We weren't retrieving related objects, which we now do.
Additionally, we iterated over the list of countries twice when one iteration was enough.
@MichaelAkvo MichaelAkvo moved this from In Progress to In Review in RSR Jan 30, 2023
zuhdil added a commit that referenced this issue Mar 10, 2023
…t-is-slow

[#5199] Optimize /project/children endpoint
@github-project-automation github-project-automation bot moved this from In Review to Done in RSR Mar 10, 2023
@MichaelAkvo MichaelAkvo moved this from Done to On Test in RSR Mar 13, 2023
@MichaelAkvo MichaelAkvo moved this from On Test to Validated on Test in RSR Mar 13, 2023
@MichaelAkvo MichaelAkvo reopened this Mar 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Validated on Test
1 participant