Improve kedro jupyter setup with options from ipykernel install

[Lasica]

Description

Some of the additional options in installing kernels could be useful in some contexts. The ipykernel module does this well (see python -m ipykernel install --help.

Context

I encountered an issue when working with pyspark, that it needs to have environment variables correctly pointing to the correct python. Setting those envs in jupyter is possible and there is support for that via ipykernel install --env .... I thought that kedro uses it under the hood to register its own kernel with some edits, but looking at the code it uses _create_kernel custom method that also uses ipykernel install.

Possible Implementation

Pass click extra arguments to the:

kedro/kedro/framework/cli/jupyter.py

Lines 158 to 169 in 00789fa

	from ipykernel.kernelspec import install
	try:
	# Install with user=True rather than system-wide to minimise footprint and
	# ensure that we have permissions to write there. Under the hood this calls
	# jupyter_client.KernelSpecManager.install_kernel_spec, which automatically
	# removes an old kernel spec if it already exists.
	kernel_path = install(
	user=True,
	kernel_name=kernel_name,
	display_name=display_name,
	)

[astrojuanlu]

Hi @Lasica, thanks for opening this conversation! We are now emphasizing %load_ext kedro.ipython #2777, #3619 and even though the kedro jupyter setup is not going anywhere for now, I don't think we are adding new features to it in the short term.

I'm converting this to a discussion for now.