ModuleNotFoundError: No module named 'pykwalify'

[RajeshRavi-brcm]

We are building Zephyr RTOS using yocto. The latest version has dependency

on the python package known as "pykwalify".

When compiled, I get following error from zephyr:

| ModuleNotFoundError: No module named 'pykwalify'

I think the pykwalify package recipe needs to be added for successful compilation.

I don't find pykwalify Yocoto recipe anywhere.

Please correct me if I'm wrong.Or please point me to pykwalify recipe, if it already exists.

Thanks & Regards,
Rajesh

[kraj]

check https://layers.openembedded.org/ and if this is not there then perhaps you need to write one

[Abhishek-brcm]

@kraj, I have added below recipe for 'pykwalify':
meta-python/recipes-devtools/python/python3-pykwalify_1.7.0.bb:

SUMMARY = "YAML/JSON validation library"
HOMEPAGE = "https://pypi.org/project/pykwalify/"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=a72ea5159364a2cd7f45c6dcbee37872"

SRC_URI[md5sum] = "58357f1d0f77de976e73dbd3660af75b"
SRC_URI[sha256sum] = "7e8b39c5a3a10bc176682b3bd9a7422c39ca247482df198b402e8015defcceb2"

PYPI_PACKAGE = "pykwalify"
inherit setuptools3 pypi

DEPENDS = "\
    python3-dateutil-native \
    python3-pyyaml-native \
"

RDEPENDS_${PN} = "\
    ${PYTHON_PN}-dateutil \
    ${PYTHON_PN}-pyyaml \
"

BBCLASSEXTEND = "native"

This recipe works fine. i.e. 'pykwalify' compile goes thorugh.

However, I see below issue during runtime as dateutil has runtime dependency on six package:

|     File "poky/build/tmp/work/*/gitAUTOINC+4bf8ed68c6_bfa48a337e_7d28a5ccba_e6c97bbc82_90d93a11ba_84cce07469_5ba575618e-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pykwalify/core.py", line 25, in <module>
|       from dateutil.parser import parse
|     File "poky/build/tmp/work/*/gitAUTOINC+4bf8ed68c6_bfa48a337e_7d28a5ccba_e6c97bbc82_90d93a11ba_84cce07469_5ba575618e-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/dateutil/parser/__init__.py", line 2, in <module>
|       from ._parser import parse, parser, parserinfo, ParserError
|     File "poky/build/tmp/work/*/gitAUTOINC+4bf8ed68c6_bfa48a337e_7d28a5ccba_e6c97bbc82_90d93a11ba_84cce07469_5ba575618e-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/dateutil/parser/_parser.py", line 42, in <module>
|       import six
|
|   ModuleNotFoundError: No module named 'six'

This issue is not seen when I add below:

diff --git a/meta-python/recipes-devtools/python/python-dateutil.inc b/meta-python/recipes-devtools/python/python-dateutil.inc
index 3c89cd62c..e5f945315 100644
--- a/meta-python/recipes-devtools/python/python-dateutil.inc
+++ b/meta-python/recipes-devtools/python/python-dateutil.inc
@@ -15,6 +15,9 @@ FILES_${PN}-zoneinfo = "${libdir}/${PYTHON_DIR}/site-packages/dateutil/zoneinfo"

 DEPENDS += "${PYTHON_PN}-setuptools-scm-native"

+RDEPENDS_${PN} = "\
+    ${PYTHON_PN}-six \
+"
 RDEPENDS_${PN}_class-target = "\
     ${PYTHON_PN}-datetime \
     ${PYTHON_PN}-numbers \

${PYTHON_PN}-six is already added under RDEPENDS_${PN}_class-target. I am not sure how is that different from RDEPENDS_${PN} as used in patch above. Can you please point to right direction here?

[kraj]

this seems ok. if you add it to RDEPENDS_${PN} then it also shows as rdep in native and nativesdk recipes, where as RDEPENDS_${PN}_class-target is only added for target recipe. It seems its needed for native and nativesdk versions too. In that case, what you have is fine but also remove it from RDEPENDS_${PN}_class-target so its not duplicated.

[RajeshRavi-brcm]

Following are the final working recipe and dependency patch from Abhishek:

A)diff --git a/meta-python/recipes-devtools/python/python-dateutil.inc b/meta-python/recipes-devtools/python/python-dateutil.inc
index 3c89cd6..40f406f 100644
--- a/meta-python/recipes-devtools/python/python-dateutil.inc
+++ b/meta-python/recipes-devtools/python/python-dateutil.inc
@@ -15,10 +15,13 @@

DEPENDS += "${PYTHON_PN}-setuptools-scm-native"

+RDEPENDS_${PN} = "\

• ${PYTHON_PN}-six
  +"

RDEPENDS_${PN}_class-target = "
${PYTHON_PN}-datetime
${PYTHON_PN}-numbers \

• ${PYTHON_PN}-six
  ${PYTHON_PN}-stringold \

---

meta-openembedded/meta-python/recipes-devtools/python/python3-pykwalify_1.7.0.bb:
B)

SUMMARY = "YAML/JSON validation library"
DESCRIPTION = "pykwalify is a schema validator for YAML and JSON"
HOMEPAGE = "https://pypi.org/project/pykwalify/"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=a72ea5159364a2cd7f45c6dcbee37872"

SRC_URI[md5sum] = "58357f1d0f77de976e73dbd3660af75b"
SRC_URI[sha256sum] = "7e8b39c5a3a10bc176682b3bd9a7422c39ca247482df198b402e8015defcceb2"

PYPI_PACKAGE = "pykwalify"
inherit setuptools3 pypi

DEPENDS = "
${PYTHON_PN}-dateutil-native
${PYTHON_PN}-pyyaml-native
"

RDEPENDS_${PN} = "
${PYTHON_PN}-dateutil
${PYTHON_PN}-pyyaml
"

BBCLASSEXTEND = "native"

[RajeshRavi-brcm]

Hi Abhishek,
Do you want to add more to this or can we close this issue.

Regards,
Rajesh

[Abhishek-brcm]

@RajeshRavi-brcm , want to keep this issue open until these patches are merged.

[kraj]

Prepare a proper patch and send either pull request or mail it to oe mailing list.

[Abhishek-brcm]

@kraj I have send in proper patches via "[email protected]".

[kraj]

thanks

[Abhishek-brcm]

@kraj Are these patches merged?
Through internal review, we found that DEPENDS may not be necessary for pykwalify. I am verifying the same. Only RDEPENDS is needed.
If not already merged, I can send v2; otherwise I will send separate patch to fix this. Please let me know.

[kraj]

No they are in master-next please send V2

[Abhishek-brcm]

Submitted v2.

they are in master-next please send V2

Does that mean you will need to revert the patch from master-next?

[kraj]

yes thats normal, master-next is meant for integrating patches, so please fix and resend.

[Abhishek-brcm]

Following error is seen by yocto autobuild with v2 patches:

ModuleNotFoundError: No module named '_sysconfigdata'

Log: http://errors.yoctoproject.org/Errors/Details/403673/
Here is the corresponding ongoing discussion on the list: https://lists.openembedded.org/g/openembedded-devel/topic/73017199

@kraj , I have created patches by referring to other recipes in the meta-python directory. I have very limited knowledge on python and yocto. Additionally, I am not able to reproduce this issue locally. In my build setup, 'pykwalify' build goes through and I am able to use it in zephyr python scripts as well. It would be great if yocto/python expert can help us on this issue to resolve it properly and quickly..

[Abhishek-brcm]

@kraj , can you please help with this issue?

[kraj]

perhaps you need something like

    if '_PYTHON_SYSCONFIGDATA_NAME' in os.environ:
        del os.environ['_PYTHON_SYSCONFIGDATA_NAME']

in do_compile maybe

[Abhishek-brcm]

@kraj I don't see any of the other python3 recipe using this. Also as I mentioned in earlier comment, I am not able to reproduce this issue in my local build machine(Ubuntu 18.04).
If you could share how to reproduce this issue, then I can try the fix suggested above and see if it resolves the issue.

[Abhishek-brcm]

@kraj I have pushed v3 with suggested fix (I did unset _PYTHON_SYSCONFIGDATA_NAME in the recipe inspired from some example in meta/ directory). Let me know if that solves the issue:
v3: https://lists.openembedded.org/g/openembedded-devel/message/83970 and
https://lists.openembedded.org/g/openembedded-devel/message/83971

[Abhishek-brcm]

@kraj Any updates?

[kraj]

merged with 1f2c4c2

[Abhishek-brcm]

@kraj , Thanks.