Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: adding system test overview #87

Merged
merged 1 commit into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 20 additions & 11 deletions src/contrib/running-tests.rst
Original file line number Diff line number Diff line change
@@ -1,24 +1,33 @@
Running Tests
=============

SSSD is a complex piece of software with a long development history. Therefore, there are several layers of tests with different goals and using different frameworks. This page shows how to run the tests and how to add new ones or modify the existing tests.
SSSD is a complex piece of software with a long development history. Therefore, there has been several layers of tests with different goals and using different frameworks. We are currently in progress of reworking all of our tests and standardizing them using our own `SSSD Test Framework <https://tests.sssd.io/en/latest/>`_.

Existing test types
-------------------
To get started immediately you can visit `Running Tests <https://tests.sssd.io/en/latest/running-tests.html>`_ page in the `SSSD Test Framework <https://tests.sssd.io/en/latest/>`_ wiki and start following the instructions provided.

Each test is different. Sometimes, you want to test the whole system end-to-end, sometimes the test should exercise some corner case for which special input and environment must be simulated. This section should give you a better idea what kind of tests already exist in SSSD so that you can choose where to add a new test and also provides a general overview.
Overview
--------

Our system tests are written using Pytest and `Pytest Multihost Plug-in <https://github.com/next-actions/pytest-mh>`_ orchestrating the setup, management, execution and teardown of hosts and tests.

All major identity providers; LDAP, Kerberos, FreeIPA, Microsoft Active Directory, Samba Active Directory are provisioned as VMs or containers and are intended to be provisioned on each run, ensuring the machines are in a clean state. The setup code is in our `SSSD CI Containers Repository <https://github.com/sssd/sssd-ci-containers/>`_. Note, these machines are not to be used for production and should only be used in development and local testing only.

For a comprehensive test run, requires several hosts; client, nfs, dns, ipa, krb, samba, ldap and ad. Emulating several environments and scenarios needed by the tests. In local testing, all containers use podman for containerization then Vagrant and KVM for virtualization. Two networks are created for each virtual computing service and needs to be routable to one another. For simplicity, all documentation assumes that one physical host is used to run the tests.

The tests are designed to be generic, and will run against all supported topologies, but some tests are written to be run against a specific topology or topologies. Topologies also define which hosts are needed by the test. For example, AD topology will require a client host and an AD host, Samba topology will require a client and a Samba host but both can be used by the GenericAD topology group will run against both AD and Samba.

Tests are also written so they can be ran independently, any test setup that is needed by the test, is executed right before the test code and is undone or the host is refreshed for the following test. For example, 389 as an LDAP provider is destroyed and re-installed before every test that, while AD the setup is undone.

Test Types
----------

Historically we have had four different test types on different frameworks. To get started, please reference system tests, because we will be retire-ring the integration and multihost tests.

.. toctree::
:maxdepth: 0
:titlesonly:

tests/unit-tests
tests/system-tests
tests/integration-tests
tests/multihost-tests

SSSD CI Containers
------------------

Developing and testing SSSD features may require Active Directory/FreeIPA/LDAP systems that can be easily provisioned and destroyed. The `SSSD CI Containers <https://github.com/sssd/sssd-ci-containers/>`_ containers and images are intended to be used in SSSD CI and they should not be used in production. However, you can use them during SSSD local testing and development.

It creates an out of the box working container environment with 389 Directory Server, IPA and Samba Active Directory servers. It also creates an SSSD client container enrolled to those servers, ready to build and debug your code.
11 changes: 11 additions & 0 deletions src/contrib/tests/system-tests.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
.. _system-tests:

============
System Tests
============

Eventually all the integration and multihost tests will be re-worked and moved to system tests. Like the current multihost the system tests are the closest to using SSSD in the real world. The differences between multihost tests and the system tests is the approach.

To start, we are taking everything we have learned from before. The SSSD system tests now have an emphasis on documentation and coding standards. Specifying what the test is suppose to do and why. We are standardizing all the tests to use our `SSSD Test Framework <https://tests.sssd.io/en/latest/>`_ and `Pytest Multihost Plug-in <https://github.com/next-actions/pytest-mh>`_.

Unlike the multihost tests, specific provider calls are abstracted allowing us to use one test for all topologies. Greatly reducing the amount of test code and administrative overhead when maintaining these tests. To learn more, please visit `SSSD Test Framework <https://tests.sssd.io/en/latest/>`_ page.
Loading