nebari-dev
Environment and Climate Change Canada integration question #1585
-
|
Suppose you had an HPC with LDAP authenticated users who would ideally have a JupyterLab instance spawn sign them in with their LDAP credentials and put them in their home directory on the system where they would develop Python notebooks for experiments and others using a variety of pip, conda and custom packages in many environments along with a Panel server that serve a library of already made Panel apps all of which have access to the local network file system. Would Nebari satisfy those needs? Bonus points if there is away to expose petabytes of geospatial data through an authenticated RESTful API, OGC APIs, MapServer along with PostgreSQL, SQLLite and Zara stores. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
|
So the core Nebari codebase is primarily designed to work on Kubernetes (on-prem or in the cloud). We have another related project that is focused more on the HPC use case qhub-hpc (we haven't got around to changing its name from Qhub to Nebari yet). qhub-hpc is reasonably feature comparable to the Nebari (different server/queue types, dashboard sharing, Dask, Environment management etc) but I don't think it would be as plug and play because every HPC system is a bit different and has different policies set by administrators. For example we use SLURM and your HPC system may use PBS or a different job submission/queueing system. qhub-hpc was primarily designed for when you are setting up a smaller HPC system from scratch and you have full control over the system. If you have a larger existing system (say maintained by a University or Lab) then it could be used as a blueprint of how to integrate the various tools. Some of the individual tools, i.e. Panel, Dask, Conda-Store etc could also be using individually on an existing HPC system. This is also the kind of thing Quansight (the company that is primarily funding Nebari right now) consults around so if that is of interest, please let me know. Overall, I think what you are trying to build is very possible, just not completely out of the box as an automated setup. |
Beta Was this translation helpful? Give feedback.
-
|
I am definitely interested Mr. Dharhas, please do refer me to someone who
can answer very technical questions in the Quansight consulting team as my
colleagues and I are evaluating a couple of options and would hate to miss
the right one.
…On Fri, Dec 2, 2022 at 3:05 PM Dharhas Pothina ***@***.***> wrote:
So the core Nebari codebase is primarily designed to work on Kubernetes
(on-prem or in the cloud). We have another related project that is focused
more on the HPC use case qhub-hpc <https://github.com/Quansight/qhub-hpc>
(we haven't got around to changing its name from Qhub to Nebari yet).
qhub-hpc is reasonable feature comparable to the Nebari but I don't think
it would be as plug and play because every HPC system is a bit different
and has different policies set by administrators. For example we use SLURM
and your HPC system may use PBS or a different job submission/queueing
system.
qhub-hpc was primarily designed for when you are setting up a smaller HPC
system from scratch and you have full control over the system. If you have
a larger existing system (say maintained by a University or Lab) then it
could be used as a blueprint of how to integrate the various tools. Some of
the individual tools, i.e. Panel, Dask, Conda-Store etc could also be using
individually on an existing HPC system.
This is also the kind of the Quansight (the company that is primarily
funding Neabri right now) consults around so if that is of interest, please
let me know. Overall, I think what you are trying to build is very
possible, just not completely out of the box as an automated setup.
—
Reply to this email directly, view it on GitHub
<#1585 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AL7R72JSUZZD5KXIORF5SS3WLJJCNANCNFSM6AAAAAASROYPPY>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Please email me at [email protected] and we can go from there. |
Beta Was this translation helpful? Give feedback.
-
|
Will do so tomorrow sir since I just left the office. Thank you very much
for your time and consideration.
…On Fri, Dec 2, 2022 at 4:54 PM Dharhas Pothina ***@***.***> wrote:
Please email me at ***@***.*** and we can go from there.
—
Reply to this email directly, view it on GitHub
<#1585 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AL7R72JVZM7G2M45NNB2H7DWLJVYHANCNFSM6AAAAAASROYPPY>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
So the core Nebari codebase is primarily designed to work on Kubernetes (on-prem or in the cloud). We have another related project that is focused more on the HPC use case qhub-hpc (we haven't got around to changing its name from Qhub to Nebari yet). qhub-hpc is reasonably feature comparable to the Nebari (different server/queue types, dashboard sharing, Dask, Environment management etc) but I don't think it would be as plug and play because every HPC system is a bit different and has different policies set by administrators. For example we use SLURM and your HPC system may use PBS or a different job submission/queueing system.
qhub-hpc was primarily designed for when you are setting up a…