Configure mscolab for sso #1818

nilupulmanodya · 2023-07-17T05:38:17Z

Configure the MSS collaboration server for SSO with testing IDP

Need to merge #1813 before merging this PR

mslib/mscolab/models.py

mslib/mscolab/server.py

ReimarBauer · 2023-07-19T14:40:04Z

add libxmlsec1 to the meta.yaml this will solve the missing library

from the description what pysaml2 uses
https://github.com/IdentityPython/pysaml2/blob/v7.4.2/README.md?plain=1#L134

and from the definition of libxmlsec1
https://github.com/conda-forge/libxmlsec1-feedstock/blob/main/recipe/meta.yaml#L8

it is the same source

…anodya/MSS into Configure-mscolab-for-SSO

Co-authored-by: Matthias Riße <[email protected]>

ReimarBauer

comment removed

mslib/mscolab/server.py

docs/conf_sso_test_msscolab.rst

mslib/mscolab/server.py

mslib/static/templates/errors/500.html

mslib/mscolab/server.py

docs/conf_sso_test_msscolab.rst

ReimarBauer

in idp_conf.py and conftest.py there is a leftover definition which writes into MSS dir. I am not sure if a file store is needed at all. conftest should add them if needed to the tmp dir and idp_conf if needed have them on the colabdata. .gitignore has also an entry

"subject_data": "./idp.subject",

Could we also move the idp/modules to the colabdata? For what are they needed? I mean why is that not just a Jinja2 template used?

interestingly didn't I find what gets mako into the environment mamba repoquery search mako finds only this module.

ReimarBauer · 2023-07-31T09:43:29Z

also in the MSS dir are two file idp.subject.dir idp.subject.dat
there are tokens in it. Is that needed?

matrss

I tried to follow the instructions and couldn't get this working. I had a hard time figuring out where to set which configuration and when trying to generate the metadata files with python mslib/mscolab/mscolab.py sso_conf --init_sso_metadata I ran into a bunch of exceptions from the xml parser due to ~/colabdata/datasso/idp.xml being empty and ~/colabdata/datasso/metadata_sp.xml containing:

<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

The output of the --init_sso_metadata command still said

idp metadata file generated succesfully
ALl necessary metadata file generated successfully

so there is something wrong with the error handling of the executed commands.

I think I am missing a step in "Enable IDP login". I just copied over mslib/mscolab/conf.py to ~/mss/mscolab_settings.py and set IDP_ENABLED = True. CONFIGURED_IDPS seemed to be correctly set already for the testing IdP. I am not sure what, if anything, to do with the yaml SP config.

mslib/mscolab/models.py

mslib/mscolab/mscolab.py

ReimarBauer · 2023-07-31T10:07:10Z

Try to avoid global as much as possible. I found one in idp_uwsgi.py

These are some ideas on that and some options. For now a ToDo refactor
before the line is ok me.

https://betterprogramming.pub/alternatives-to-using-globals-in-python-a3b2a7d5411b

http://google.github.io/styleguide/pyguide.html

Co-authored-by: Matthias Riße <[email protected]>

…lmanodya/MSS into Configure-mscolab-for-SSO

nilupulmanodya · 2023-07-31T21:49:06Z

I tried to follow the instructions and couldn't get this working. I had a hard time figuring out where to set which configuration and when trying to generate the metadata files with python mslib/mscolab/mscolab.py sso_conf --init_sso_metadata I ran into a bunch of exceptions from the xml parser due to ~/colabdata/datasso/idp.xml being empty and ~/colabdata/datasso/metadata_sp.xml containing:
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
The output of the --init_sso_metadata command still said
idp metadata file generated succesfully
ALl necessary metadata file generated successfully
so there is something wrong with the error handling of the executed commands.

I think I am missing a step in "Enable IDP login". I just copied over mslib/mscolab/conf.py to ~/mss/mscolab_settings.py and set IDP_ENABLED = True. CONFIGURED_IDPS seemed to be correctly set already for the testing IdP. I am not sure what, if anything, to do with the yaml SP config.

Thanks for figuring out that point. updated documentation and error handling. 🙂

docs/conf_sso_test_msscolab.rst

ReimarBauer

small "typos" in the documentation.

Add ToDos of the not solved comments into the codebase, e.g. #1818 (comment)

# ToDo refactor, use common lib secrets? see discussion in https://github.com/Open-MSS/MSS/pull/1818#discussion_r1270701658

nilupulmanodya · 2023-08-01T10:57:47Z

Try to avoid global as much as possible. I found one in idp_uwsgi.py

These are some ideas on that and some options. For now a ToDo refactor before the line is ok me.

https://betterprogramming.pub/alternatives-to-using-globals-in-python-a3b2a7d5411b

http://google.github.io/styleguide/pyguide.html

Added into ToDo : e3a81ba

nilupulmanodya · 2023-08-01T10:58:12Z

also in the MSS dir are two file idp.subject.dir idp.subject.dat there are tokens in it. Is that needed?

Added into Todo :e3a81ba

nilupulmanodya · 2023-08-01T10:58:48Z

in idp_conf.py and conftest.py there is a leftover definition which writes into MSS dir. I am not sure if a file store is needed at all. conftest should add them if needed to the tmp dir and idp_conf if needed have them on the colabdata. .gitignore has also an entry

"subject_data": "./idp.subject",

Could we also move the idp/modules to the colabdata? For what are they needed? I mean why is that not just a Jinja2 template used?

interestingly didn't I find what gets mako into the environment mamba repoquery search mako finds only this module.

Added into ToDo : e3a81ba

matrss

The issue I had was caused by me setting IDP_ENABLED (now USE_SAML2) in ~/mss/mscolab_settings.py instead of mslib/mscolab/conf.py. Since the former should overwrite the latter there seems to be something wrong with the loading of the configuration. Nevertheless I was able to get this working now, to the point that a login with the IdP in msui works, which is great :)

I saw a few typos in docs/conf_sso_test_msscolab.rst though, could you please proofread that?

Apart from this I think we can merge this and improve on it in the next PR.

mslib/mscolab/mscolab.py

ReimarBauer

There are a few ToDo's which will be done in the next PR

* remove global var * remove idp.subjects file dirs * remove relaystste, rndstr and use secrets * remove shell=True * correct typos * fix group order * enable flake8 for GSOC2023-NilupulManodya * fix lint * fix lint * fixes comments * resolve comments * fix comments * update doc

* remove inputs from conditions (#1808) * Setup sp and idp for the sso (#1809) * configure sp and idp * update meta.yml remove cherypy * fixes previous * update notice * update readme * regroup idp_uwsgi * regroup app.py * regroup, change wsgi server to flask * Update conf_sp_idp/README.md Co-authored-by: Matthias Riße <[email protected]> * hide secrets by config * update copy-paste-able command for creating keys and certificates * Update README.md * correct copyright lines * remove make_metadata.py file and update doc with new flow * remove idp.xml file * remove condition libxmlsec1 * Update conf_sp_idp/sp/app/conf.py Co-authored-by: Matthias Riße <[email protected]> * Update conf_sp_idp/idp/idp.py Co-authored-by: Matthias Riße <[email protected]> * remove generate_metadatascript * remove hardcoded path * recorrect copyrights --------- Co-authored-by: Matthias Riße <[email protected]> * Split conf sp idp (#1811) * split sp and idp * generate doc * remove prints idp.py * update comeponents.rst * UI changes in Qt for SSO (#1813) * ui changes in qt for sso * fixes qt UI implementation * get idp_enabled response from server * update tests for test_hello * update test utils * Update mslib/msui/mscolab.py Co-authored-by: Matthias Riße <[email protected]> * fix typo * move downed idp_enabled exception * increase height ui_mscolab_connect_dialog * resolve comments --------- Co-authored-by: Matthias Riße <[email protected]> * web browser implementation (#1814) * web browser implementation * update gitgnore * resolve comments * update docstring * Configure mscolab for sso (#1818) * db modeling * add users into id[ * backend yaml implementation * set server conf * config server for sso * qt ui implmentation * backend html templates implementation * update testcases * config qt client app * update gitignore * set yaml endpoints * update docs * update test utill, and fix error * fix test utils * remove disabled pylint * add libxmlsec1 into dep * set IDP ENabled false * Update mslib/mscolab/server.py Co-authored-by: Matthias Riße <[email protected]> * recorrect commit * update db modeling with authentication_backend for multiple idps * update conf for the multiple idps * template implementation * msui update redirect url for multiple idps * saml update for multiple idps * update mscolab server for multiple idps * update doc for multiple idps * automate CERTs generation and paths * update doc * correct typo in doc * update doc * fix typos update gitignore * fix config idp_conf * update gitignore * set one time token access * add params for cert creation * set idp token for one time validation * fix unnnescessary debug * remove duplicate imports * Update mslib/mscolab/mscolab.py Co-authored-by: Matthias Riße <[email protected]> * automate saml yaml file and improve error handling * rename IDP_ENABLED to USE_SAML2 * update error template * update doc * add todo idp_wsgi * update db models * recorrect doc * add todo refactors --------- Co-authored-by: Matthias Riße <[email protected]> * To do fixes #1818 (#1974) * remove global var * remove idp.subjects file dirs * remove relaystste, rndstr and use secrets * remove shell=True * correct typos * fix group order * enable flake8 for GSOC2023-NilupulManodya * fix lint * fix lint * fixes comments * resolve comments * fix comments * update doc * improve code for multiple Idps * conf routes for multiple conf * remove uncessary .yaml * update cmd metadata * update conf * update saml handler for multiple idps * pinning of xmlschema * pin werkzeug * disable pytests for todo refactor * disbale whole file gsoc_testing * fix conf * resolve comments * resolve comments * manual conflict resolve ui_mscolab_connect_dialog.ui file * resolve flake8 * set SSL certificate verification enablement (#2062) * ssl verification enablement for SSO * add hint * Remove testing SP (#2066) * remove testing sp * remove documentation auth_client_sp * Create documentation for SSO integration through SAML (#2064) * create documentation sso integration * added into makefile components * change dir images * resolve comments, add sample files * resolve comments * change cookies dir of web browser (#2063) * change cookies dir of web browser * Update mslib/msui/msui_web_browser.py Co-authored-by: Matthias Riße <[email protected]> --------- Co-authored-by: Matthias Riße <[email protected]> * improve accessibility saml2 urls (#2068) * improve accessibility saml2 urls * resolve comments --------- Co-authored-by: Nilupul Manodya <[email protected]> Co-authored-by: Matthias Riße <[email protected]> Co-authored-by: Matthias Riße <[email protected]> Co-authored-by: nilupulmanodya <[email protected]>

nilupulmanodya added 14 commits July 17, 2023 09:02

db modeling

8bf8603

add users into id[

d0f4a4e

backend yaml implementation

052bb51

set server conf

6d64fce

config server for sso

2891ee4

qt ui implmentation

438f473

backend html templates implementation

b33b768

update testcases

aba2d1a

config qt client app

95b9ba1

update gitignore

0b1ad9e

set yaml endpoints

650aacb

update docs

6337e5b

update test utill, and fix error

45ea586

fix test utils

8047d07

matrss reviewed Jul 18, 2023

View reviewed changes

remove disabled pylint

aa09e9c

nilupulmanodya and others added 13 commits July 19, 2023 21:13

add libxmlsec1 into dep

7f7d6b0

set IDP ENabled false

186398e

Merge branch 'GSOC2023-NilupulManodya' of https://github.com/nilupulm…

f0b7d31

…anodya/MSS into Configure-mscolab-for-SSO

Merge branch 'GSOC2023-NilupulManodya' of https://github.com/nilupulm…

ecb9161

…anodya/MSS into Configure-mscolab-for-SSO

Update mslib/mscolab/server.py

6c37956

Co-authored-by: Matthias Riße <[email protected]>

recorrect commit

5a95ba1

update db modeling with authentication_backend for multiple idps

51a50f1

update conf for the multiple idps

bb39e52

template implementation

eae3f4f

msui update redirect url for multiple idps

042f202

saml update for multiple idps

52e999f

update mscolab server for multiple idps

f013a2f

update doc for multiple idps

e788224

ReimarBauer reviewed Jul 31, 2023

View reviewed changes

ReimarBauer requested changes Jul 31, 2023

View reviewed changes

matrss reviewed Jul 31, 2023

View reviewed changes

mslib/mscolab/models.py Outdated Show resolved Hide resolved

mslib/mscolab/mscolab.py Outdated Show resolved Hide resolved

mslib/mscolab/mscolab.py Outdated Show resolved Hide resolved

mslib/mscolab/mscolab.py Outdated Show resolved Hide resolved

nilupulmanodya and others added 8 commits July 31, 2023 22:10

Update mslib/mscolab/mscolab.py

5580dc4

Co-authored-by: Matthias Riße <[email protected]>

automate saml yaml file and improve error handling

2e68633

rename IDP_ENABLED to USE_SAML2

fd8796b

update error template

0bbc3b1

update doc

616f113

Merge branch 'Configure-mscolab-for-SSO' of https://github.com/nilupu…

3d2aa31

…lmanodya/MSS into Configure-mscolab-for-SSO

add todo idp_wsgi

f97bf8d

update db models

433aeab

nilupulmanodya requested a review from ReimarBauer August 1, 2023 07:16

ReimarBauer reviewed Aug 1, 2023

View reviewed changes

docs/conf_sso_test_msscolab.rst Show resolved Hide resolved

ReimarBauer reviewed Aug 1, 2023

View reviewed changes

docs/conf_sso_test_msscolab.rst Outdated Show resolved Hide resolved

ReimarBauer requested changes Aug 1, 2023

View reviewed changes

nilupulmanodya added 2 commits August 1, 2023 16:01

recorrect doc

c957f60

add todo refactors

e3a81ba

nilupulmanodya requested a review from ReimarBauer August 1, 2023 11:01

matrss reviewed Aug 1, 2023

View reviewed changes

mslib/mscolab/mscolab.py Show resolved Hide resolved

ReimarBauer approved these changes Aug 1, 2023

View reviewed changes

ReimarBauer merged commit 4c556a3 into Open-MSS:GSOC2023-NilupulManodya Aug 1, 2023
2 checks passed

nilupulmanodya mentioned this pull request Aug 31, 2023

To do fixes #1818 #1974

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Configure mscolab for sso #1818

Configure mscolab for sso #1818

nilupulmanodya commented Jul 17, 2023

ReimarBauer commented Jul 19, 2023 •

edited

Loading

ReimarBauer left a comment •

edited

Loading

ReimarBauer left a comment •

edited

Loading

ReimarBauer commented Jul 31, 2023

matrss left a comment

ReimarBauer commented Jul 31, 2023

nilupulmanodya commented Jul 31, 2023

ReimarBauer left a comment •

edited

Loading

nilupulmanodya commented Aug 1, 2023

nilupulmanodya commented Aug 1, 2023

nilupulmanodya commented Aug 1, 2023

matrss left a comment

ReimarBauer left a comment

Configure mscolab for sso #1818

Configure mscolab for sso #1818

Conversation

nilupulmanodya commented Jul 17, 2023

ReimarBauer commented Jul 19, 2023 • edited Loading

ReimarBauer left a comment • edited Loading

Choose a reason for hiding this comment

ReimarBauer left a comment • edited Loading

Choose a reason for hiding this comment

ReimarBauer commented Jul 31, 2023

matrss left a comment

Choose a reason for hiding this comment

ReimarBauer commented Jul 31, 2023

nilupulmanodya commented Jul 31, 2023

ReimarBauer left a comment • edited Loading

Choose a reason for hiding this comment

nilupulmanodya commented Aug 1, 2023

nilupulmanodya commented Aug 1, 2023

nilupulmanodya commented Aug 1, 2023

matrss left a comment

Choose a reason for hiding this comment

ReimarBauer left a comment

Choose a reason for hiding this comment

ReimarBauer commented Jul 19, 2023 •

edited

Loading

ReimarBauer left a comment •

edited

Loading

ReimarBauer left a comment •

edited

Loading

ReimarBauer left a comment •

edited

Loading