Feature/unit testing #1013

brianignacio5 · 2023-07-24T11:50:39Z

Description

Add Unit testing from ESP-IDF extension using PyTest from ESP-IDF.

TO-DO: Add fallback PyTest requirements.

Fixes #485

Type of change

New feature (non-breaking change which adds functionality)

Steps to test this pull request

Open an example project that has esp-idf tests as described in Unit Testing in ESP32. For example use the unit test example

Click on "Testing" icon in the Activity Bar.
The tests will be automatically found in the current workspace folder.
Next to the test there is Run button. Pressing the button will Check if Pytest is installed, install if it is not installed, copy and build a unit test project within the current project, execute PyTest and parse the results.

Expected behaviour: Tests output is shown as Task Output and the result is shown in the Testing UI (passed/ failed)
Expected output: Test execution output is shown as task output.

How has this been tested?

Manual Testing with example unit test app project.

Test Configuration:

ESP-IDF Version: 5.0
OS (Windows,Linux and macOS): macOS

Dependent components impacted by this PR:

PyTest installed from ESP-IDF

Checklist

github-actions · 2023-07-24T12:08:41Z

Download the artifacts for this pull request:

src/espIdf/unitTest/adapter.ts

src/espIdf/unitTest/testExecution.ts

radurentea

LGTM

AndriiFilippov · 2023-08-16T08:23:49Z

@brianignacio5 hi !

Tested:
OS - Windows 10
ESP-IDF: v5.1

Click "Testing" in the Activity Bar -> The tests automatically found in the current workspace folder. 👍
But when I press Run button - I do see this error:

 Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: pycparser in c:\users\andriifilippov\.espressif\python_env\idf5.1_py3.11_env\lib\site-packages 
(from cffi>=1.12->cryptography>=2.1.4->esptool~=4.5->pytest-embedded-serial-esp->-r c:\Users\AndriiFilippov\e\esp-idf-v5.1\tools\requirements\requirements.pytest.txt (line 4)) (2.21)       
Using cached pytest_embedded_serial_esp-1.3.4-py3-none-any.whl 
(5.1 kB)
Using cached pytest_embedded_idf-1.3.4-py3-none-any.whl (18 kB)Using cached pytest_embedded_jtag-1.3.4-py3-none-any.whl (4.5 kB)
Using cached pytest_embedded_qemu-1.3.4-py3-none-any.whl (6.4 kB)
Using cached pytest_rerunfailures-12.0-py3-none-any.whl (12 kB)Using cached idf_build_apps-1.1.1-py2.py3-none-any.whl (34 kB)
Using cached protobuf-4.24.0-cp310-abi3-win_amd64.whl (430 kB) 
Using cached python_can-4.2.2-py3-none-any.whl (257 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_embedded-1.3.4-py3-none-any.whl (25 kB)
Using cached pytest_embedded_serial-1.3.4-py3-none-any.whl (5.3 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached prettytable-3.8.0-py3-none-any.whl (27 kB)        
Using cached pluggy-1.2.0-py3-none-any.whl (17 kB)
Building wheels for collected packages: netifaces
  Building wheel for netifaces (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for netifaces (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      C:\Users\AndriiFilippov\AppData\Local\Temp\pip-build-env-imnumfee\overlay\Lib\site-packages\setuptools\config\setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`     
      !!
     
              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.
     
              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.      
     
              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************
     
      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running bdist_wheel
      running build
      running build_ext
      building 'netifaces' extension
      error: Microsoft Visual C++ 14.0 or greater is required. 
Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely 
not a problem with pip.
  ERROR: Failed building wheel for netifaces
Failed to build netifaces
ERROR: Could not build wheels for netifaces, which is required 
to install pyproject.toml-based projects

 *  The terminal process "C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -Command "c:\Users\AndriiFilippov\.espressif\python_env\idf5.1_py3.11_env\Scripts\python.exe" -m pip install --upgrade --no-warn-script-location -r "c:\Users\AndriiFilippov\e\esp-idf-v5.1\tools\requirements\requirements.pytest.txt"" terminated with exit code: 1.

AndriiFilippov · 2023-08-24T08:53:35Z

@brianignacio5 hi !

Tested under:
OS - Windows 10
ESP-IDF: v5.1
unit_test example

Issue still persist.

Using cached https://dl.espressif.com/pypi/ptyprocess/ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: pycparser in c:\users\andriifilippov\.espressif\python_env\idf5.1_py3.11_env\lib\site-packages (from cffi>=1.12->cryptography>=2.1.4->esptool~=4.5->pytest-embedded-serial-esp->-r c:\Users\AndriiFilippov\e\esp-idf-v5.1\tools\requirements\requirements.pytest.txt (line 4)) (2.21)
Downloading idf_build_apps-1.1.2-py2.py3-none-any.whl (35 kB)
Downloading protobuf-4.24.1-cp310-abi3-win_amd64.whl (430 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 430.4/430.4 kB 6.7 MB/s eta 0:00:00
Downloading pyecharts-2.0.4-py3-none-any.whl (147 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 147.7/147.7 kB ? eta 0:00:00
Building wheels for collected packages: netifaces
  Building wheel for netifaces (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for netifaces (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      C:\Users\AndriiFilippov\AppData\Local\Temp\pip-build-env-ykp2vs92\overlay\Lib\site-packages\setuptools\config\setupcfg.py:293: _DeprecatedConfig: Deprecated 
config in `setup.cfg`
      !!
     
              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.
     
              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.
     
              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************
     
      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running bdist_wheel
      running build
      running build_ext
      building 'netifaces' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ 
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for netifaces
Failed to build netifaces
ERROR: Could not build wheels for netifaces, which is required to install pyproject.toml-based projects

 *  The terminal process "C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -Command "c:\Users\AndriiFilippov\.espressif\python_env\idf5.1_py3.11_env\Scripts\python.exe" -m pip install --upgrade --no-warn-script-location -r "c:\Users\AndriiFilippov\e\esp-idf-v5.1\tools\requirements\requirements.pytest.txt" --extra-index-url https://dl.espressif.com/pypi" terminated with exit code: 1.

github-actions · 2023-09-09T00:16:25Z

Pull request has been marked as stale since there are no activities, and this will be closed in 5 days if there are no further activities

AndriiFilippov · 2024-01-09T12:40:16Z

@brianignacio5 hi !

Issue still persist. I can run the test but test runs forever and there is no results and any output in terminal.

AndriiFilippov · 2024-01-12T09:58:06Z

@brianignacio5 hi !

Tested under:
OS - Windows 10
ESP-IDF: v5.1.2

Able to run tests with ESP32 board connected.

output also clear.

2024-01-12 10:42:38 I (232) cpu_start: ELF file SHA256:  bcd78d807c9854e5...
2024-01-12 10:42:38 I (238) cpu_start: ESP-IDF:          v5.1.2-dirty
2024-01-12 10:42:38 I (243) cpu_start: Min chip rev:     v0.0
2024-01-12 10:42:38 I (248) cpu_start: Max chip rev:     v3.99 
2024-01-12 10:42:38 I (253) cpu_start: Chip rev:         v3.0
2024-01-12 10:42:38 I (258) heap_init: Initializing. RAM available for dynamic allocation:
2024-01-12 10:42:38 I (265) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
2024-01-12 10:42:38 I (271) heap_init: At 3FFB2A78 len 0002D588 (181 KiB): DRAM
2024-01-12 10:42:38 I (277) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
2024-01-12 10:42:38 I (284) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
2024-01-12 10:42:38 I (291) heap_init: At 4008C310 len 00013CF0 (79 KiB): IRAM
2024-01-12 10:42:38 I (298) spi_flash: detected chip: generic
2024-01-12 10:42:38 I (301) spi_flash: flash io: dio
2024-01-12 10:42:38 I (306) app_start: Starting scheduler on CPU0
2024-01-12 10:42:38 I (310) app_start: Starting scheduler on CPU1
2024-01-12 10:42:38 I (310) main_task: Started on CPU0
2024-01-12 10:42:38 I (320) main_task: Calling app_main()
2024-01-12 10:42:38
2024-01-12 10:42:38
2024-01-12 10:42:38 Press ENTER to see the list of tests.
2024-01-12 10:42:38 1
2024-01-12 10:42:38 Running Mean of an empty array is zero...
2024-01-12 10:42:38 C:/Users/AndriiFilippov/project-nameBR/components/testable/test/test_mean.c:16:Mean of an empty array is zero:PASS
2024-01-12 10:42:38 Test ran in 16ms
2024-01-12 10:42:38
2024-01-12 10:42:38 -----------------------
2024-01-12 10:42:38 1 Tests 0 Failures 0 Ignored
2024-01-12 10:42:38 OK
2024-01-12 10:42:38 Enter next test, or 'ent.

---------------------- generated xml file: C:\Users\AndriiFilippov\project-nameBR\unity-app\test.xml ---------------------- 
==================================================== 1 passed in 7.29s ====================================================

LGTM 👍

brianignacio5 added the Feature A new feature label Jul 24, 2023

brianignacio5 requested a review from radurentea July 24, 2023 11:50

brianignacio5 self-assigned this Jul 24, 2023

brianignacio5 mentioned this pull request Jul 24, 2023

[Feature Request]: Support for unit testing with esp-idf / unity (VSC-723) #485

Closed

kolipakakondal added this to the v1.7.0 milestone Jul 25, 2023

radurentea reviewed Jul 25, 2023

View reviewed changes

src/espIdf/unitTest/adapter.ts Outdated Show resolved Hide resolved

src/espIdf/unitTest/testExecution.ts Outdated Show resolved Hide resolved

src/espIdf/unitTest/testExecution.ts Outdated Show resolved Hide resolved

src/espIdf/unitTest/testExecution.ts Outdated Show resolved Hide resolved

brianignacio5 requested a review from radurentea August 4, 2023 10:41

radurentea approved these changes Aug 4, 2023

View reviewed changes

brianignacio5 requested a review from AndriiFilippov August 21, 2023 10:34

github-actions bot added the stale Stale PR or Issue label Sep 9, 2023

github-actions bot closed this Sep 14, 2023

brianignacio5 reopened this Sep 14, 2023

brianignacio5 added ongoing Ongoing Issue or PR, this label will be used for issue or PR which is to be excluded by stale bot and removed stale Stale PR or Issue labels Sep 14, 2023

brianignacio5 mentioned this pull request Dec 1, 2023

add boards filtered by idf target #1079

Merged

5 tasks

brianignacio5 force-pushed the feature/unit-testing branch from a769152 to 10d769e Compare January 9, 2024 03:00

brianignacio5 added 8 commits January 10, 2024 19:00

first attempt unit test

9b5cb21

finish initial unit test utility

bd99629

add docs refactor configure step

42ad82b

add index url pytest req install

f50ab80

add pytest install and build test app cmds

867eee9

add command to unit test doc

8ad4a17

fix lint

24ca678

fix issues on windows

873bbfb

brianignacio5 force-pushed the feature/unit-testing branch from fc5a639 to 873bbfb Compare January 10, 2024 11:01

update notificationMode

11816b0

brianignacio5 merged commit 72b7dec into master Jan 15, 2024
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/unit testing #1013

Feature/unit testing #1013

brianignacio5 commented Jul 24, 2023

github-actions bot commented Jul 24, 2023 •

edited

Loading

radurentea left a comment

AndriiFilippov commented Aug 16, 2023

AndriiFilippov commented Aug 24, 2023

github-actions bot commented Sep 9, 2023

AndriiFilippov commented Jan 9, 2024

AndriiFilippov commented Jan 12, 2024

Feature/unit testing #1013

Feature/unit testing #1013

Conversation

brianignacio5 commented Jul 24, 2023

Description

Type of change

Steps to test this pull request

How has this been tested?

Dependent components impacted by this PR:

Checklist

github-actions bot commented Jul 24, 2023 • edited Loading

radurentea left a comment

Choose a reason for hiding this comment

AndriiFilippov commented Aug 16, 2023

AndriiFilippov commented Aug 24, 2023

github-actions bot commented Sep 9, 2023

AndriiFilippov commented Jan 9, 2024

AndriiFilippov commented Jan 12, 2024

github-actions bot commented Jul 24, 2023 •

edited

Loading