Socket proxy is part of the container signing feature.  It will proxy
the docker unix socket and eventually filter messages.

fix bouncer version file

the snapcraft cmake plugin doesn't copy the .git directory from
the download folder into the build directory, so git commands
may not work in the build phase.

the command does seem to work in some environments because git
"falls up" the directory structure looking for a containing
.git folder, usually finding the .git folder of snap-pelion-edge,
which obviously contains the wrong version for the part.

Docker sock was moved to /run/snap.snap-pelion-edge/var/run/.  We need
to update this script to use the same.

Use bouncer proxy socket path instead of docker socket when image
signing is enabled.

Signed-off-by: Cristian Prundeanu <[email protected]>

bouncer needs to run as a service so that it can handle the
docker-proxy.socket.  it also needs the network-bind plug so
that it can open a listening socket.

This fixes fix yq failure to read the maestro yaml
config file on startup.

This commit fixes these log errors:
maestro[]: Error: unknown command "r" for "yq"
maestro[]: Run 'yq --help' for usage.
maestro[]: Error: unknown command "r" for "yq"
maestro[]: Run 'yq --help' for usage.

The yq version was not version-locked and yq version 4.0
is a major breaking change from previous versions:
https://github.com/mikefarah/yq/releases/tag/4.0.0-alpha1

Add multiple connections support.

The version is the same, we just reference the tag name instead
of the SHA.

The meta build is at 1.15.8, but 1.15.13 has a few extra bug
fixes and security fixes.

https://golang.org/doc/devel/release#go1.15.minor

The patches had to be refreshed as well.  The main changes are
that 1) the mbed-cloud-client library was turned into a git
submodule which required some adjustment, and 2) the network
proxy functionality in mbed-cloud-client had to be totally
rewritten.

The mbed-cloud-client network proxy patch is based on this PR:
https://github.com/PelionIoT/mbed-cloud-client-internal/pull/3262/

According to the Pelion Edge 2.4 changelog, the userdata/mbed folder
should have permissions restricted to 700 and owner=root.

the tagged version of devicedb 1.10.0 didn't print the correct
version on the command line.  this is a bug in devicedb where
the old version was hardcoded and wasn't bumped to 1.10.0
during the release.

edge-core is now compiled in 3 modes:
    * factory mode
    * developer mode
    * byoc mode

https://github.com/PelionIoT/mbed-edge#configuring-edge-build

TODO:
    * add code to the launch script to check which mode to start

Now that we have multiple versions of edge-core installed in the
snap, we need to provide the user with a config option to choose
which one to start.

The new snap configuration variable is "edge-core.provision-mode"
and valid values are "factory", "developer", "byoc", and "auto".
The default is "auto".

For factory mode, the expectation is that the mcc_config folder
exists.

For developer mode, the expectation is that the developer mode
binary was built with developer credentials and installed into
the snap.

For byoc mode, the expectation is that the user has provided
a CBOR file at /var/snap/pelion-edge/common/device.cbor.  See
https://github.com/PelionIoT/mbed-edge/tree/master/edge-tool#readme
for instructions on how to create a CBOR.

The provision-mode is only useful if the device hasn't already
been provisioned, i.e., if the mcc_config folder doesn't
exist, because if the device is provisioned we just boot in
factory mode regardless of the snap config option.

To switch modes if the device is already provisioned:
    1. stop the pelion-edge snap services
    2. backup the existing mcc_config folder if desired
    3. delete the existing mcc_config folder
    4. set the snap config option edge-core.provision-mode
    5. provision credentials as in the expectations above
    6. start the pelion-edge snap services

the 'grade' flag in snapcraft.yaml is used to indicate whether
a snap is ready for release, so we can use that flag to indicate
when to include the byoc and developer variants of edge-core under
that assertion that we shouldn't include developer tools in
production builds.

This fixes a build breakage issue with the python snapcraft plugin
used for building python projects. The python package manager pip
dropped support for python2 and for python3.5 in pip version 21 and
the plugin was patched to pin the version of pip.

canonical/snapcraft#3428

edge-tool can convert Pelion Cloud developer credentials
(mbed_cloud_dev_credentials.c and update_default_resources.c)
into a CBOR file that is used for registration by edge-core
in BYOC mode.

https://github.com/PelionIoT/mbed-edge/tree/master/edge-tool

edge-tool.py v0.16.1 requires a json config file and unfortunately
it must be named .mbed_cloud_config.json which means it's a hidden
file and the home plugin doesn't allow access to the user's hidden
files.

when starting edge-core in factory mode, if the device is not
provisioned at the factory, then edge-core will still create the
mcc_config/WORKING/ folder, but its contents will be empty. this
is fixable by performing the factory provisioning step with FCCE,
but for a grade:devel snap, this breaks auto mode.

The edge-core launch code should be driven first by the user
setting, and only then fallback to auto detection.

Snapcraft supports Python 3.5 in the `core` base snap.  The
mbed-edge edge-tool recently updated one of its python dependencies,
cryptography, to version 3.3.2 which doesn't support Python 3.5 and
only supports Python 3.6+

the python package setup.py for edge-tool didn't install the
edge_tool.py main script which meant that edge-tool wasn't
runnable in the snap.

install the edge_tool.py utility under the wigwag folder so that
it is bundled in a similar directory structure as seen in the
meta-mbed-edge yocto recipe.

This reverts commit ffea43d.

Downgrade kubelet back to the version before network policies
until we add missing dependency CoreDNS which replaces CNI plugins.

The path to docker runtime files (such as docker.sock) was modified
in a previous commit to move it out of non-volatile storage to
tmpfs to fix a sudden powerloss issue, but a bug was created by
moving the persistent data dir also.  This commit moves the
data-root back to persistent storage.

Added v1.0.0 tag of pe-terminal package.

As we’re shying away for nodejs we’ve built this golang based
terminal-client for Pelion Edge which will ultimately replace
the old nodejs based relay-term but the current intention is to
disable the old one instead of removing it completely so that
we can bring it back up incase we encounter some issue with the
new one.

This very rare condition may occur during a snap-refresh if the
pelion-edge snap is also being updated.  In this case, edge-core
won't be able to write to SNAP_DATA which causes any future
firmware update to fail.

This fixes a build sequencing issue encountered when building
edge-core *after* curl.  Apparently the custom version of libcurl
isn't compiled with OPENSSL support and the build system uses
the built-in version instead if curl hasn't been staged yet.

In the normal case when building from a clean environment,
edge-core seems to be built before curl and so this issue doesn't
occur.

cmake /build/parts/edge-core/src -DCMAKE_INSTALL_PREFIX=
-DCMAKE_BUILD_TYPE=Release -DTRACE_LEVEL=WARN -DFIRMWARE_UPDATE=ON
-DDEVELOPER_MODE=OFF -DFACTORY_MODE=ON -DFOTA_ENABLE=OFF
-DMBED_CLOUD_CLIENT_CURL_DYNAMIC_LINK=OFF -DBYOC_MODE=OFF
-DTARGET_CONFIG_ROOT=/build/parts/edge-core/src/config
-DMBED_CLOUD_DEV_UPDATE_ID=ON -DNETWORK_PROXY_SUPPORT=ON
-DPARSEC_TPM_SE_SUPPORT=OFF
cmake: /build/stage/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found
(required by cmake)
Failed to run 'cmake /build/parts/edge-core/src -DCMAKE_INSTALL_PREFIX=
-DCMAKE_BUILD_TYPE=Release -DTRACE_LEVEL=WARN -DFIRMWARE_UPDATE=ON
-DDEVELOPER_MODE=OFF -DFACTORY_MODE=ON -DFOTA_ENABLE=OFF
-DMBED_CLOUD_CLIENT_CURL_DYNAMIC_LINK=OFF -DBYOC_MODE=OFF
-DTARGET_CONFIG_ROOT=/build/parts/edge-core/src/config
-DMBED_CLOUD_DEV_UPDATE_ID=ON -DNETWORK_PROXY_SUPPORT=ON
-DPARSEC_TPM_SE_SUPPORT=OFF' for 'edge-core': Exited with code 1.
Verify that the part is using the correct parameters and try again.

All tabs to 4 spaces found by pysh-check.
Add Izuma copyrights where applicable.

Fix following shellcheck findings:

snap-pelion-edge/entrypoint.sh:22:24: note: Double quote to prevent globbing and word splitting. [SC2086]
snap-pelion-edge/entrypoint.sh:24:30: note: Double quote to prevent globbing and word splitting. [SC2086]

Quotations everywhere.

v2 checkout is deprecated.

/ssd/snap-pelion-edge/snap/hooks/install:25:24: note: Double quote to prevent globbing and word splitting. [SC2086]

Fix following findings:

/snap-pelion-edge/files/edge-core-tool/edge-core-tool.sh:21:11: note: Double quote to prevent globbing and word splitting. [SC2086]
/snap-pelion-edge/files/edge-core-tool/edge-core-tool.sh:26:14: note: Double quote to prevent globbing and word splitting. [SC2086]
/snap-pelion-edge/files/edge-core-tool/edge-core-tool.sh:26:27: error: Double quote array expansions to avoid re-splitting elements. [SC2068]

/ssd/snap-pelion-edge/files/bouncer/scripts/launch-bouncer.sh:21:15: note: Double quote to prevent globbing and word splitting. [SC2086]
/ssd/snap-pelion-edge/files/bouncer/scripts/launch-bouncer.sh:21:23: note: Double quote to prevent globbing and word splitting. [SC2086]
/ssd/snap-pelion-edge/files/bouncer/scripts/launch-bouncer.sh:21:31: note: Double quote to prevent globbing and word splitting. [SC2086]
/ssd/snap-pelion-edge/files/bouncer/scripts/launch-bouncer.sh:21:61: note: Double quote to prevent globbing and word splitting. [SC2086]
/ssd/snap-pelion-edge/files/bouncer/scripts/launch-bouncer.sh:21:119: note: Double quote to prevent globbing and word splitting. [SC2086]

One instance still remains, the last line - it's just a monster.

Some quotation fixes.

Simple quotation fixes.

/snap-pelion-edge/files/fluent-bit/scripts/launch-fluent-bit.sh:3:11: note: Double quote to prevent globbing and word splitting. [SC2086]
/snap-pelion-edge/files/fluent-bit/scripts/launch-fluent-bit.sh:12:6: note: Double quote to prevent globbing and word splitting. [SC2086]
/snap-pelion-edge/files/fluent-bit/scripts/launch-fluent-bit.sh:12:46: note: Double quote to prevent globbing and word splitting. [SC2086]

Fixing most, suppressing one due to code readability reasons.

Just a bunch of quotation, really.

Update to edge-core 0.21.0 (modify patches to match, drop one).
Update to core18 (which still Python 3.6, so edge-tools will install).

Maestro still craps up.

To get rid of the libuv etc. problems.

Co-authored-by: Pete Dyer <[email protected]>

Instead of relay-term, as that has been dropped out.

As we not build it anymore, it's not there to be stopped and
that prevents the snap from starting up.

Requires multiple changes, as the snapcraft.io guys just want to break
things. Various keyword changes again and plug-in behavior changes.

golang side
- only go mod builds are supported by default.

cmake
- they dropped the default prefix which core18 used.

pe-terminal
- version info related changes as we dropped relay-term.

Seems something has been changed in the snapcraft autotools
plugin, which broke jq. This should fix that.

Same issue as fix with docker-tini.

Same fix as for docker-tini.

Moved passing the forwarding-addresses to before the optional arguments.
For some reason the go argument parser was missing the final option.

We are using the wrong variable, it should be docker_tag (in the
similar manner as for every other service, too).

We get this error in kernel logs;

pelion-edge.dockerd[134922]: /snap/pelion-edge/x1/bin/dockerd-wrapper: line 11: [: : integer expression expected

This is due to the fact variable name is wrong. It's actually,

aa_profile_reloaded="$SNAP_COMMON/profile_reloaded"

not `aa_profile_refreshed`.

Add a lot of plugs to the snapcraf.yaml to clear out apparmor complaints.
You can see them easily with snappy-debug and it even advices what you
should do to fix them. This clears out almost all of them, you will
still see a few between doing the snap install and running connect.sh,
but you should not see any after the reboot.

- Add btrfs-tooling for the Dockerfile, that's needed.

This reverts commit e80c9fb.

To enable building in potential cross-compile or without docker.

Same as mbed-edge repo. Each build is 30 minutes, our own will be a lot
faster.

- add timeout 40 minutes to build job.
- add Write -property to all files at start (golang has some files sometimes
  with Read Only and this screws up subsequent git clones etc.)
- This is just installing and removing it, actual tests cannot be run
  as we cannot reboot the GitHub runner during a job (connection gets lost).

Two (note related changes):
- build curl from sources (to allow proper cross-compilations)
- add plug process-control to processes (to close snappy-debug findings, most).

To enable the cross-compilation of it. Otherwise it fails with
multiple libs missing.

Turned out to be a bit trickier than thought due to all of the
plugs (access) that had to be set up.

You can run these via:

pelion-edge.edge-info
pelion-edge.edge-testnet

Got a build warning when doing full clean built that curl was missing
a dependency.

The syntax is likely also wrong for the prime: section for docker-tini.
- Surprisingly hard to find an example, but if we look how the other
  files are defined for it - seems we have and extra dash.
- Docs refer to same rules as with filesets, so it is then this:
  https://snapcraft.io/docs/snapcraft-filesets

This way we can probably  avoid any mismatches on the command bin/edge-x
failing. Caveat - version needs to be kept manually in place?

Edge-info uses these, so most likely the build machine should also
have these.

It's not part of the image anymore, so no point looking for it.

Maestro-shell does not seem to exist anymore, either.
The path was wrong for the pe-utils (we have the version in the
identity-tools).

Run the edge-info, version and edge-testnet as well.

It has now a dummy 0.0.1 version - it should really follow the
releases we make instead.

The file is slightly different here vs. other places. Align
to versions.json.

To be what it was exactly at 1.21-version.

https://github.com/PelionIoT/snap-pelion-edge/blob/v1.21.2/snap/snapcraft.yaml#L703C13-L704

Pull in latest release.

Look up the file under the folder and pick 1st match. The
ls command should give you just one file anyway, as we only build one file.
The name however changes per version.

No point in having an .MD-file, as it cannot do any of the env variable
replacements, which make sense for this use-case.

By checking their source-commits (and/or) tags are the same.
There is no apt package for yq available for Ubuntu 22.04... :-/

Seems our snap isn't really running in the runner.

Merge the steps into one, seems snap-pelion-edge is not
running in the next step anymore.

Take the latest pe-utils into use and also fix the version.
- Do not print component versions that do not exist (devicedb)
- Print out version of new components.

You can run these via
   pelion-edge.version
   pelion-edge.edge-info
   pelion-edge.edge-testnet

We do unfortunately (at least right now) pick up the version via
2 different means, they need to be kept in sync.

files/pe-utils/versions.json:    "version" : "2.6.0-dev"
snap/snapcraft.yaml:version: "2.6.0-dev"

It did not really work as expected. This one works at least as a
standalone bash script.

That way it has somewhere to fallback as the CI build failed with:

+ install -d /build/parts/fluent-bit/install/edge
+ git describe --tags
fatal: No names found, cannot describe anything.
Failed to run 'override-build': Exit code was 128.

Didn't see this on my local runs, though. But this should
ensure we never fail with that. If it can't find a tag, it
uses the hash OR previous tag + commits on top/hash.

Requires sudo.

Ah, the test phase does not have the code available (and thus
snap/snapcraft.yaml -file).

The checkout action apparently changes the folder as well, thus
it can't find the scripts-internal.

Linting for the yaml file to rule out most obvious mistakes.
- The rules are very relaxed in a way, since snapcraft.yaml is not exactly
  100% yaml compatible.
- The line lenghts are also going to have to be very relaxed.

That is edge-info, edge-testnet and so forth...

We have a naughty typo here, which breaks the script actually.
type is not handled correctly, as we use respone instead of $response.
Fix another typo while at it.

This fixes all known shellcheck fixes. Some of them were real
bugs, for example the

- assignment of rc with spaces (will not work)
- comparising without spaces (will not work, either)

Rest of them are more like style issues.