Optimize our GitPod setup

[noklam]

Description

Is your feature request related to a problem? A clear and concise description of what the problem is: "I'm always frustrated when ..."

Our GitPod setup has not been working for a while. #2585 is the quick fix but we can do better. In #2585 @astrojuanlu mentioned he use a custom Docker and we can do something similar for this repostory.

Context

Why is this change important to you? How would you use it? How can it benefit other users?
GitPod is quite useful for review & testing purposes, the current setup is slow and broken.

Possible Implementation

We might be able to use a custom Docker setup, but this requires some investigation into how it works and if that's the best approach.

Possible Alternatives

[noklam]

Just to add more, I tried to build docs on GitPod last week and it doesn't seem to work.

[astrojuanlu]

I've talked about this with @antonymilne a couple of times but sadly I never got the time to dive deep into why things are the way they are at the moment. I know there are some weirdnessess on how Gitpod pre-caches some things:

kedro/.gitpod.yml

Lines 4 to 13 in 64b7960

	# We want packages installed during the pre-build init steps to go to /workspace
	# rather than ~ so that they are persisted. Gitpod sets PIP_USER=yes to ensure this,
	# but pre-commit requires PIP_USER=no. Hence we set PIP_USER=no and use
	# pip install --user to install to /workspace.
	- name: kedro
	before: |
	echo PIP_USER=no >> ~/.bashrc && export PIP_USER=no
	init: |
	make sign-off
	pip install -r test_requirements.txt --user

(More info at https://www.gitpod.io/docs/configure/workspaces/tasks, https://www.gitpod.io/docs/configure/projects/prebuilds)

I'm all for simplifying the setup or having a custom Docker image as I'm doing with my workshops https://github.com/astrojuanlu/workshop-jupyter-kedro/blob/ce202c7d1164d9e5b6aa331320b32a4f1a342407/.gitpod.yml#L1-L2 but we should not do it carelessly.

At the same time, if the current setup doesn't work, it doesn't matter much if it's optimized.

[astrojuanlu]

pyenv shims unfortunately make all this even more complicated 😬

[noklam]

At this point I would favor anything that works but slow over the current setup.

For example when I open gitpod, I have to wait 10 minutes for the setup now and the pre-commit env is broken. If I run pre-commit run I get weird error I have to uninstall virtualenv and reinstall it to fix the issue. So I have to wait 10 minutes and do this manual steps often the I let GitPod hang for too long and the workspace become inactive again, it just become unusable.

The prebuild is not happening at all, I don't know if there is a easy way that we can fix it, it does requires admin permission and I don't have it.

In addition, I would like to get this https://www.gitpod.io/for/opensource if possible, I try to apply this before but never receive any follow-up.

[astrojuanlu]

@noklam you mentioned #2688 is not a complete fix, what else do you think is needed?

[noklam]

It's functional now but not optimised, spinning a gitpod instance takes 10 minutes.

It would be great if can fix the pre-built so it doesn't fetch the whole docker image every time.

[astrojuanlu]

I'm hoping that uv makes further optimisations unnecessary for now? Closing, but feel free to reopen if you disagree