Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust E2E tests for project configuration changes #863

Open
wants to merge 43 commits into
base: ref/project-config-ci
Choose a base branch
from

Conversation

maximilianoertel
Copy link
Collaborator

@maximilianoertel maximilianoertel commented Oct 10, 2024

Proposed changes

This PR introduces a series of changes to our E2E tests in order to mostly improve compatibility with the introduced changes related to environment configuration.

Key highlights

  • Environment variables for Cypress are now prefixed with CYPRESS_ to follow the Cypress naming practices (docs)
  • Environment variables loaded from the .env.test file are parsed by dotenvx and only variables containing the prefix are loaded into Cypress. Mimicking Cypress's native behaviour, the prefix is stripped before injecting the variables.
  • These environment variables can be directly accessed in test specs without needing to first map them inside the Cypress config file.

Additionally, this PR also improves the login command by introducing both a redirect assertion and additional validation to ensure the login was successful.

Finally, this PR removes (almost) all arbitrary timeouts and refactors both failing and flaky tests. This is mostly achieved by improving selectors, restructuring tests and introducing waitFor to await page loads in certain cases. The latter should hopefully be a temporary solution until application performance is improved.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactoring (non-breaking change that does not add functionality but makes code cleaner or more efficient)
  • Documentation Update
  • Tests (new or updated tests)
  • Style (changes to code styling)
  • CI (continuous integration changes)
  • Repository Maintenance
  • Other (please describe below)

Checklist

  • I have read the guidelines for contributing.
  • The changes in this PR are as small as they can be. They represent one and only one fix or enhancement.
  • Linting checks pass with my changes.
  • Any existing unit tests pass with my changes.
  • Any existing end-to-end tests pass with my changes.
  • I have added tests that prove my fix is effective or that my feature works.
  • If this PR fixes an existing issue, I have added a unit or end-to-end test that will detect if this issue reoccurs.
  • I have added JSDoc comments as appropriate.
  • I have added the necessary documentation to the roar-docs repository.
  • I have shared this PR on the roar-pr-reviews channel (if I have access)
  • I have linked relevant issues (if any)

Justification of missing checklist items

Further comments

Copy link

github-actions bot commented Oct 10, 2024

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 4.98% 380 / 7629
🔵 Statements 4.88% 411 / 8416
🔵 Functions 6.13% 115 / 1875
🔵 Branches 2.29% 105 / 4566
File Coverage
File Stmts % Branch % Funcs % Lines Uncovered Lines
Changed Files
src/components/AppSpinner.vue 0% 100% 0% 0% 2-5
src/components/CardAdministration.vue 0% 0% 0% 0% 174, 176, 188-189, 191, 193-230, 198-220, 203, 205-210, 213-218, 227, 232-236, 233-235, 234, 238-242, 239, 241, 244, 244-245, 245, 247-249, 248, 248, 251-253, 252, 256, 256, 259-260, 262-263, 265-270, 266-268, 267, 269, 272-277, 273-275, 274, 276, 279-282, 280-281, 284-286, 285, 288, 290-292, 294-296, 295, 298-301, 300, 303-305, 304, 304, 307-395, 309-394, 310, 312, 312-313, 313, 315, 317, 321-327, 329-345, 331, 333, 333, 335-342, 347, 351-372, 353-369, 354-364, 359-361, 360, 362, 365, 367-369, 368, 371, 380-390, 381-389, 382-388, 383-387, 384, 384-385, 385-386, 392-393, 397-398, 400-411, 400-411, 413-434, 414-415, 417-418, 420-433, 436-441, 437-440, 438-439, 7-9, 27, 29, 34, 1, 44
src/components/ListOrgs.vue 0% 0% 0% 0% 169-179, 181-186, 182-184, 183, 185, 188-193, 189-191, 190, 192, 195-196, 198-200, 202-203, 203, 205-232, 206-211, 213, 213, 215-220, 217-219, 221-224, 222-223, 225-227, 226, 228-230, 229, 231, 234-237, 236, 239, 239, 241-243, 245-247, 246, 249-251, 257-259, 262-279, 265-270, 273-278, 282-292, 283-291, 310-373, 311-372, 313, 315-323, 316-322, 325-333, 326-332, 336, 338-353, 338-353, 356, 358-363, 365-371, 375-431, 376-381, 383-388, 384-387, 390-393, 391-392, 395-428, 430, 433-446, 434, 434-445, 436-444, 448-454, 449-453, 451-452, 456-459, 457-458, 461-464, 462-463, 466-468, 467, 470-501, 471-500, 481-487, 490-496, 499, 504-507, 505, 505-506, 509-511, 510, 510, 513-515, 514, 514, 517-519, 518, 521-523, 522, 525-528, 527, 3-4, 1, 17, 22, 34, 38, 50, 72, 80, 82, 94, 97, 104, 107, 126, 1-15, 1-24, 1-40, 64-65, 1-74, 92, 102, 123
src/components/RoarDataTable.vue 0% 0% 0% 0% 693, 604-609, 607-608, 611, 628-629, 631-635, 632-634, 633, 633, 636-637, 639-655, 657-676, 660-674, 662-671, 673, 675, 678-680, 679, 682, 684-690, 685-688, 686-687, 689, 692, 696-702, 697-698, 700-701, 703, 707-716, 718-750, 719-748, 723-725, 724, 727-729, 732-734, 733, 737-740, 738-739, 742-747, 743-746, 749, 752-753, 755-758, 756, 760-774, 761-771, 773, 778-784, 781-783, 782, 785, 788-801, 803-816, 818-823, 825-836, 838-849, 851-882, 867-880, 876-879, 878, 881, 884-889, 887-888, 888, 891-896, 894-895, 895, 898-903, 901-902, 902, 905-909, 907-908, 908, 911-915, 913-914, 914, 917-921, 919-920, 920, 924-926, 925-926, 929-931, 930, 930, 933, 933-940, 935, 935-939, 937-938, 942-943, 945-948, 946-947, 950, 1, 6, 8, 20, 22, 34, 36, 68-69, 122-124, 126, 140-142, 144, 158-160, 162, 176, 178, 182, 196, 198, 200, 214, 216, 218, 296, 310, 316, 327, 337, 348, 389, 408, 418, 428, 438, 448, 550-552, 55, 63, 1-77, 1-78, 125, 143, 161, 181, 199, 217, 1-267, 1-275, 1-282, 1-290, 1-295, 1-301, 1-355, 529
src/constants/routes.js 100% 100% 100% 100%
src/pages/HomeAdministrator.vue 0% 0% 0% 0% 137-139, 141-145, 147-148, 150, 152, 154, 157-160, 158, 158-159, 162-164, 163, 163, 166-168, 167, 167, 170-172, 174, 185-195, 186, 186, 189-191, 190, 194, 201-203, 213-230, 214, 216, 216, 219, 222-228, 223, 225-227, 226, 233-310, 316-320, 317-319, 326-334, 327-333, 328, 330-332, 331, 340-344, 341-343, 342, 351-370, 352-355, 357-362, 359, 361, 363-367, 364, 366, 369, 2-3, 5-6, 1, 14-15, 17, 20-22, 41-42, 73, 1-18, 1-25, 1-44, 49
src/pages/HomeParticipant.vue 0% 0% 0% 0% 119-123, 125, 127-128, 131-135, 133, 133-134, 137-138, 140-142, 141, 141, 144-146, 145, 145, 148-149, 155-157, 163-165, 167-169, 168, 168, 171, 171, 171-172, 172, 178-180, 182-184, 186-188, 187, 190-192, 191, 194-197, 195, 195-196, 200-204, 206-216, 210-214, 211-213, 215, 218-220, 222-225, 227, 229-231, 233-271, 234, 236-237, 239, 241-257, 242, 244-251, 245, 247-250, 248-249, 253-256, 254-255, 260-266, 261-265, 262-264, 267-271, 268-270, 275-279, 281-285, 288-291, 289-290, 295-337, 296-335, 297-321, 300-302, 307, 307-318, 323-332, 325-331, 326-330, 327-329, 328, 334, 336, 339-341, 340, 340, 343-345, 344, 344, 348-357, 349-356, 352, 360-362, 361, 365-367, 366, 366, 370-376, 371, 371, 373-375, 378-401, 381-383, 382, 385-386, 386, 389-398, 391, 395-397, 396, 1, 10, 12, 27, 45-46, 65, 70, 1-33, 48, 1-49, 1-58
src/pages/SignIn.vue 0% 0% 0% 0% 155-158, 160-161, 163-183, 164-182, 165-173, 166-172, 170-171, 175-181, 176-181, 178, 180, 185-215, 186-214, 187, 189-211, 192-195, 193-194, 196-199, 197-198, 202-210, 205-206, 208-209, 213, 217, 219-227, 220-225, 222, 224, 226, 229-238, 230-237, 232-233, 235-236, 240-277, 243-276, 246-248, 247, 250-252, 251, 254-275, 257-260, 258-259, 261-264, 262-263, 266, 269-274, 271, 273, 279-282, 280-281, 284, 286, 288-291, 289-290, 293-300, 294-299, 295, 295-296, 296-297, 297-298, 298, 302-312, 303-307, 305-306, 308-311, 309-310, 314-316, 315, 5-6, 8, 1, 14-16, 22-23, 26, 35, 45, 55, 66, 68, 81, 84, 93, 104, 115, 20, 1-119, 123
src/pages/StudentReport.vue 0% 0% 0% 0% 201, 203, 205, 224, 226, 226, 228-230, 232-235, 234, 237-239, 241-243, 245-246, 248-250, 249, 252-297, 253-256, 255, 257, 257, 259-260, 263-266, 265, 267-270, 269, 271-274, 273, 275-278, 277, 280-293, 295-296, 299, 299, 299, 301-315, 302-314, 306-310, 307, 309, 312, 317-322, 318, 318, 320, 320-321, 324-327, 325, 325-326, 330-342, 331-342, 333-342, 335-342, 337-342, 339, 341, 345, 348-354, 349-350, 350, 352-353, 356-358, 357, 357, 360-364, 361-363, 362, 13-14, 17, 22, 1, 36-38, 77, 107, 109, 137, 140-141, 144, 147-148, 151, 154-155, 158, 161-162
src/router/index.js 0% 0% 0% 0% 8, 8, 12, 12, 15-395, 19, 32, 39, 46, 53, 60, 67, 74, 81, 88, 95, 102, 109, 116, 123, 130, 137, 144, 151, 158, 165, 174, 181, 188, 196, 202-203, 208, 213, 221, 227, 241-242, 249-250, 257-258, 265, 271, 277, 283, 290, 296, 302, 308, 315, 322, 329, 336, 342, 347, 352, 358, 367, 373, 380, 386, 392, 397-406, 401-402, 402-403, 403-404, 408-489, 409, 411-415, 412, 414, 417, 419-428, 430, 432-438, 433-438, 436-437, 440-447, 445-446, 450-451, 454-455, 463-479, 464-479, 470-476, 471-476, 474-475, 477-478, 482-485, 483-484, 487-488
Generated in workflow #710 for commit 8620452 by the Vitest Coverage Report Action

Copy link

cypress bot commented Oct 10, 2024

roar-dashboard-e2e    Run #8266

Run Properties:  status check failed Failed #8266  •  git commit 0aa1b0e380: Adjust E2E tests for project configuration changes (#863)
Project roar-dashboard-e2e
Run status status check failed Failed #8266
Run duration 01m 29s
Commit git commit 0aa1b0e380: Adjust E2E tests for project configuration changes (#863)
Committer Maximilian Oertel
View all properties for this run ↗︎

Test results
Tests that failed  Failures 3
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 0

Tests for review

Failed  NavBar.cy.js • 1 failed test

View Output

Test Artifacts
An uncaught error was detected outside of a test Test Replay Screenshots
Failed  slots/FilterBar.cy.js • 1 failed test

View Output

Test Artifacts
An uncaught error was detected outside of a test Test Replay Screenshots
Failed  RoarDataTable.cy.js • 1 failed test

View Output

Test Artifacts
An uncaught error was detected outside of a test Test Replay Screenshots

@maximilianoertel maximilianoertel changed the title ref/project-config-ci-e2e Adjust E2E tests for project configuration changes Oct 10, 2024
@maximilianoertel maximilianoertel self-assigned this Oct 10, 2024
@maximilianoertel maximilianoertel added the enhancement New feature or request label Oct 10, 2024
@maximilianoertel maximilianoertel marked this pull request as ready for review October 11, 2024 14:03
@maximilianoertel maximilianoertel requested a review from a team as a code owner October 11, 2024 14:03
@maximilianoertel maximilianoertel changed the base branch from ref/project-config-ci to ref/project-config October 14, 2024 23:11
@maximilianoertel maximilianoertel changed the base branch from ref/project-config to ref/project-config-ci October 14, 2024 23:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant