Functional tests implementation for mscolab.py file #2065

nilupulmanodya · 2023-10-24T11:52:01Z

Purpose of PR?:

Fixes #
Functional testcases implementation for mscolab.py file

* configure sp and idp * update meta.yml remove cherypy * fixes previous * update notice * update readme * regroup idp_uwsgi * regroup app.py * regroup, change wsgi server to flask * Update conf_sp_idp/README.md Co-authored-by: Matthias Riße <[email protected]> * hide secrets by config * update copy-paste-able command for creating keys and certificates * Update README.md * correct copyright lines * remove make_metadata.py file and update doc with new flow * remove idp.xml file * remove condition libxmlsec1 * Update conf_sp_idp/sp/app/conf.py Co-authored-by: Matthias Riße <[email protected]> * Update conf_sp_idp/idp/idp.py Co-authored-by: Matthias Riße <[email protected]> * remove generate_metadatascript * remove hardcoded path * recorrect copyrights --------- Co-authored-by: Matthias Riße <[email protected]>

* split sp and idp * generate doc * remove prints idp.py * update comeponents.rst

* ui changes in qt for sso * fixes qt UI implementation * get idp_enabled response from server * update tests for test_hello * update test utils * Update mslib/msui/mscolab.py Co-authored-by: Matthias Riße <[email protected]> * fix typo * move downed idp_enabled exception * increase height ui_mscolab_connect_dialog * resolve comments --------- Co-authored-by: Matthias Riße <[email protected]>

* web browser implementation * update gitgnore * resolve comments * update docstring

* db modeling * add users into id[ * backend yaml implementation * set server conf * config server for sso * qt ui implmentation * backend html templates implementation * update testcases * config qt client app * update gitignore * set yaml endpoints * update docs * update test utill, and fix error * fix test utils * remove disabled pylint * add libxmlsec1 into dep * set IDP ENabled false * Update mslib/mscolab/server.py Co-authored-by: Matthias Riße <[email protected]> * recorrect commit * update db modeling with authentication_backend for multiple idps * update conf for the multiple idps * template implementation * msui update redirect url for multiple idps * saml update for multiple idps * update mscolab server for multiple idps * update doc for multiple idps * automate CERTs generation and paths * update doc * correct typo in doc * update doc * fix typos update gitignore * fix config idp_conf * update gitignore * set one time token access * add params for cert creation * set idp token for one time validation * fix unnnescessary debug * remove duplicate imports * Update mslib/mscolab/mscolab.py Co-authored-by: Matthias Riße <[email protected]> * automate saml yaml file and improve error handling * rename IDP_ENABLED to USE_SAML2 * update error template * update doc * add todo idp_wsgi * update db models * recorrect doc * add todo refactors --------- Co-authored-by: Matthias Riße <[email protected]>

* remove global var * remove idp.subjects file dirs * remove relaystste, rndstr and use secrets * remove shell=True * correct typos * fix group order * enable flake8 for GSOC2023-NilupulManodya * fix lint * fix lint * fixes comments * resolve comments * fix comments * update doc

…b-for-multiple-IdPs improve mscolab for configure multiple Idps

…merge_current_develop

…velop Merge current develop

* ssl verification enablement for SSO * add hint

This reverts commit a796365.

This reverts commit 456b1c4.

matrss · 2023-10-25T13:20:57Z

tests/_test_mscolab/test_mscolab.py

+
+    def test_handle_mscolab_certificate_init(self):
+        handle_mscolab_certificate_init()
+        FILE_KEY = os.path.join(mscolab_settings.MSCOLAB_SSO_DIR, 'key_mscolab.key')


I associate all-uppercase names with global and static (configuration) values, like the MSCOLAB_SSO_DIR value here. In this case FILE_KEY is just a local variable in this test function, so I think it should be lowercase file_key (or path_key or key_path, since it not really a file object yet).

In doubt you can trigger pylint, which gives also a good expression about "problems"
here is an output of "one" configuration :) When we add a .pylintrc we have to discuss which options we want to ignore, beforehand.

https://docs.pylint.org/
my local .config/pylintrc is derivated from something like this
https://www.codeac.io/documentation/pylint-configuration.html

https://pylint.readthedocs.io/en/stable/user_guide/configuration/index.html

pylint test_mscolab.py pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.Exception' ?) instead. ************* Module tests._test_mscolab.test_mscolab test_mscolab.py:43:0: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:52:0: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:68:0: C0115: Missing class docstring (missing-class-docstring) test_mscolab.py:68:0: C0103: Class name "Test_Mscolab" doesn't conform to '[A-Z_][a-zA-Z0-9]+$' pattern (invalid-name) test_mscolab.py:86:4: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:109:4: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:121:4: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:123:8: C0103: Variable name "FILE_KEY" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name) test_mscolab.py:125:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding) test_mscolab.py:129:8: C0103: Variable name "FILE_CRT" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name) test_mscolab.py:131:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding) test_mscolab.py:136:4: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:138:8: C0103: Variable name "FILE_KEY" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name) test_mscolab.py:140:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding) test_mscolab.py:144:8: C0103: Variable name "FILE_CRT" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name) test_mscolab.py:146:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding) test_mscolab.py:151:4: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:153:8: C0103: Variable name "FILE_YAML" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name) test_mscolab.py:155:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding) test_mscolab.py:160:4: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:165:4: C0116: Missing function or method docstring (missing-function-docstring) test_mscolab.py:30:0: C0411: standard import "import argparse" should be placed before "import pytest" (wrong-import-order)

Improve only new parts. For the others we can give a refactoring issue.

ReimarBauer · 2023-10-25T20:38:44Z

tests/_test_mscolab/test_mscolab.py

+        assert handle_mscolab_metadata_init(True) is True
+
+    def test_handle_local_idp_metadata_init(self):
+        handle_local_idp_certificate_init()


you want
assert handle_local_idp_certificate_init() is True

you can also debug tests, this is an example by pycharm (full version, the free version can do this too)

there are also "print" outputs which can be compared, if they are the expected ones, e.g. https://pavolkutaj.medium.com/how-to-test-printed-output-in-python-with-pytest-and-its-capsys-fixture-161010cfc5ad

ReimarBauer · 2023-10-25T20:48:05Z

tests/_test_mscolab/test_mscolab.py

+    def test_handle_mscolab_metadata_init(self):
+        handle_mscolab_certificate_init()
+        mscolab_settings.USE_SAML2 = True
+        assert handle_mscolab_metadata_init(True) is True


for using the command you assume you are in the correct dir, this can make problems for handle_mscolab_metadata_init

You can get the path of mscolab.py by importing it and looking on mscolab.__file__

Tests sometimes shows that also the code needs to become improved

ReimarBauer · 2023-11-03T10:56:31Z

mslib/msidp/idp_conf.py

@@ -46,6 +47,10 @@
 DATA_DIR = os.path.join(BASE_DIR, "colabdata")
 MSCOLAB_SSO_DIR = os.path.join(DATA_DIR, 'datasso')

+if "PYTEST_CURRENT_TEST" in os.environ:


we should move this env thing to another place. I send you an example on slack

ReimarBauer · 2023-11-03T19:34:08Z

you can run the tests locally, e.g. (mssdev):~/PycharmProjects/2023/nilupulmanodya/MSS$ pytest -k test_handle_mscolab_metadata_init

This shows to me
ERROR tests/_test_with_idp/test_auth_client.py - FileNotFoundError: [Errno 2] No such file or directory: 'mslib/auth_client_sp/key_sp.key'

nilupulmanodya · 2023-11-11T07:40:09Z

you can run the tests locally, e.g. (mssdev):~/PycharmProjects/2023/nilupulmanodya/MSS$ pytest -k test_handle_mscolab_metadata_init

This shows to me ERROR tests/_test_with_idp/test_auth_client.py - FileNotFoundError: [Errno 2] No such file or directory: 'mslib/auth_client_sp/key_sp.key'

Could you please provide me more steps to reproduce this, did you export the path ? export PYTHONPATH="pwd:$HOME/mss" or Do we not do that on tests?

May be I am not sure if you are in the correct branch.

ReimarBauer · 2023-11-11T08:20:40Z

pytest -k test_handle_mscolab_metadata_init

I can't reproduce, looks like I did something wrong.

ReimarBauer · 2023-11-11T08:21:55Z

please send the PR to develop. We merged your branch and that makes it easier to discover the features done during GSoC

…functional-tests-implementation-mscolab

…mplementation-mscolab # Conflicts: # conftest.py # mslib/mscolab/mscolab.py # mslib/msidp/idp_conf.py

nilupulmanodya and others added 30 commits June 10, 2023 10:06

remove inputs from conditions (Open-MSS#1808)

e06b396

Split conf sp idp (Open-MSS#1811)

f2b1434

* split sp and idp * generate doc * remove prints idp.py * update comeponents.rst

web browser implementation (Open-MSS#1814)

245d64e

* web browser implementation * update gitgnore * resolve comments * update docstring

improve code for multiple Idps

198cf88

conf routes for multiple conf

db82eb7

remove uncessary .yaml

90a1c62

update cmd metadata

df4ae37

update conf

8dd2c9e

update saml handler for multiple idps

05f3c2c

pinning of xmlschema

aa47a09

pin werkzeug

1912fd4

disable pytests for todo refactor

77f21d3

disbale whole file gsoc_testing

cf2a3b9

fix conf

c7ab2ae

resolve comments

34e7e33

resolve comments

b02854a

Merge pull request Open-MSS#2043 from nilupulmanodya/implement-mscola…

f94da54

…b-for-multiple-IdPs improve mscolab for configure multiple Idps

Merge branch 'develop' of https://github.com/nilupulmanodya/MSS into …

458a978

…merge_current_develop

manual conflict resolve ui_mscolab_connect_dialog.ui file

2d59aa5

resolve flake8

18a9e7c

Merge pull request Open-MSS#2061 from nilupulmanodya/merge_current_de…

ef326c2

…velop Merge current develop

set SSL certificate verification enablement (Open-MSS#2062)

3d328f9

* ssl verification enablement for SSO * add hint

functional test cases implementation mscolab.py

2fa29ae

resolove flake8

a796365

Revert "resolove flake8"

456b1c4

This reverts commit a796365.

Revert "Revert "resolove flake8""

8cc06a1

This reverts commit 456b1c4.

nilupulmanodya added 3 commits October 24, 2023 17:20

resolve flake8

b08bbdf

recorrect commit

d7b4e6b

fix flake8 test_mscolab.py

973deaf

nilupulmanodya marked this pull request as draft October 24, 2023 13:27

matrss reviewed Oct 25, 2023

View reviewed changes

ReimarBauer reviewed Oct 25, 2023

View reviewed changes

nilupulmanodya added 2 commits October 31, 2023 13:10

set fixed dir for crts keys and metadata xmls

8ee5d0a

fixes pylint

2073c9b

ReimarBauer reviewed Nov 3, 2023

View reviewed changes

nilupulmanodya changed the base branch from GSOC2023-NilupulManodya to develop November 11, 2023 08:54

nilupulmanodya added 4 commits November 11, 2023 16:29

Merge branch 'develop' of https://github.com/nilupulmanodya/MSS into …

faa974e

…functional-tests-implementation-mscolab

implement constants through envs

8d9d00c

set env through test_mscolab.py

3e56c4d

set abs path

35a1d4b

nilupulmanodya force-pushed the functional-tests-implementation-mscolab branch from 0ec5e9f to 35a1d4b Compare November 11, 2023 15:11

nilupulmanodya added 5 commits November 27, 2023 18:00

Merge remote-tracking branch 'origin/develop' into functional-tests-i…

1ce2013

…mplementation-mscolab # Conflicts: # conftest.py # mslib/mscolab/mscolab.py # mslib/msidp/idp_conf.py

resolve pull conflicts

54db2c0

resolve pull conflicts

804b36c

set env TESTING_MSCOLAB_SSO_DIR idp_conf.py

0bb9e34

test with sys.path

6868c0c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Functional tests implementation for mscolab.py file #2065

Functional tests implementation for mscolab.py file #2065

nilupulmanodya commented Oct 24, 2023

matrss Oct 25, 2023 •

edited

Loading

ReimarBauer Oct 25, 2023 •

edited

Loading

ReimarBauer Oct 25, 2023

ReimarBauer Oct 25, 2023

ReimarBauer Oct 25, 2023 •

edited

Loading

ReimarBauer Nov 3, 2023

ReimarBauer commented Nov 3, 2023 •

edited

Loading

nilupulmanodya commented Nov 11, 2023 •

edited

Loading

ReimarBauer commented Nov 11, 2023

ReimarBauer commented Nov 11, 2023

Functional tests implementation for mscolab.py file #2065

Are you sure you want to change the base?

Functional tests implementation for mscolab.py file #2065

Conversation

nilupulmanodya commented Oct 24, 2023

matrss Oct 25, 2023 • edited Loading

Choose a reason for hiding this comment

ReimarBauer Oct 25, 2023 • edited Loading

Choose a reason for hiding this comment

ReimarBauer Oct 25, 2023

Choose a reason for hiding this comment

ReimarBauer Oct 25, 2023

Choose a reason for hiding this comment

ReimarBauer Oct 25, 2023 • edited Loading

Choose a reason for hiding this comment

ReimarBauer Nov 3, 2023

Choose a reason for hiding this comment

ReimarBauer commented Nov 3, 2023 • edited Loading

nilupulmanodya commented Nov 11, 2023 • edited Loading

ReimarBauer commented Nov 11, 2023

ReimarBauer commented Nov 11, 2023

matrss Oct 25, 2023 •

edited

Loading

ReimarBauer Oct 25, 2023 •

edited

Loading

ReimarBauer Oct 25, 2023 •

edited

Loading

ReimarBauer commented Nov 3, 2023 •

edited

Loading

nilupulmanodya commented Nov 11, 2023 •

edited

Loading