Template bootstrap script

[arcusfelis]

Proposed changes include:

• New script for templating
• Docs

[mongoose-im]

8139.1 / Erlang 22.0 / small_tests / fdfe643
Reports root / small

---

8139.7 / Erlang 22.0 / elasticsearch_and_cassandra_mnesia / fdfe643
Reports root/ big
OK: 328 / Failed: 0 / User-skipped: 28 / Auto-skipped: 0

[mongoose-im]

8169.1 / Erlang 22.0 / small_tests / 8da0654
Reports root / small

---

8169.2 / Erlang 22.0 / internal_mnesia / 8da0654
Reports root/ big
OK: 1412 / Failed: 0 / User-skipped: 157 / Auto-skipped: 0

---

8169.4 / Erlang 22.0 / mysql_redis / 8da0654
Reports root/ big
OK: 2636 / Failed: 0 / User-skipped: 213 / Auto-skipped: 0

---

8169.3 / Erlang 22.0 / odbc_mssql_mnesia / 8da0654
Reports root/ big
OK: 2641 / Failed: 0 / User-skipped: 208 / Auto-skipped: 0

---

8169.5 / Erlang 22.0 / riak_mnesia / 8da0654
Reports root/ big
OK: 1537 / Failed: 0 / User-skipped: 172 / Auto-skipped: 0

---

8169.7 / Erlang 22.0 / elasticsearch_and_cassandra_mnesia / 8da0654
Reports root/ big
OK: 328 / Failed: 0 / User-skipped: 28 / Auto-skipped: 0

---

8169.6 / Erlang 22.0 / ldap_mnesia / 8da0654
Reports root/ big
OK: 1366 / Failed: 0 / User-skipped: 203 / Auto-skipped: 0

---

8169.9 / Erlang 21.3 / pgsql_mnesia / 8da0654
Reports root/ big / small
OK: 2654 / Failed: 0 / User-skipped: 195 / Auto-skipped: 0

[michalwski]

Thanks for the templating script! I had some questions and concerns about the tests run after building RPM package. Also, do we want to run the tests for DEB package as well?

@sebb7 Could you also take a look at the PR? There were some changes to the Dockerfile_rpm, I'm not sure if this affects your other scripts.

[sebb7]

@michalwski @arcusfelis I reviewed the rpm Dockerfile and I like the idea of splinting the source code to TEST and SOURCE very much. CI job for building centos package failed but I managed to produce the package for centos 7 on my machine.

When it comes to the deb package the CI build debian_stretch fails so I think we need changes in Dockerfile_deb.

[sebb7]

@michalwski @arcusfelis Actually package was built but in the container on which smoke tests were run I found inside some files which have bad permissions or are placed in a wrong directory in my opinion:

[root@05b148e8be62 mongooseim]# pwd
/usr/lib/mongooseim
[root@05b148e8be62 mongooseim]# find . \! -user root -print
./etc/final_template_vars.config
./etc/demo.config

[root@05b148e8be62 mongooseim]# ls -la /usr/lib/mongooseim/etc/
total 64
drwxrwxrwx 1 root       root        4096 Apr 15 20:51 .
drwxr-xr-x 1 root       root        4096 Apr 15 20:51 ..
-rw-rw-rw- 1 root       root        2885 Apr 15 20:48 app.config.example
-rw-r--r-- 1 mongooseim mongooseim   185 Apr 15 20:51 demo.config
-rw-rw-r-- 1 mongooseim mongooseim   126 Apr 15 20:51 final_template_vars.config
-rw-rw-rw- 1 root       root       28826 Apr 15 20:48 mongooseim.cfg.example
-rw-rw-rw- 1 root       root          79 Apr 15 20:48 templates.ini
-rw-rw-rw- 1 root       root         652 Apr 15 20:48 vm.args.example
-rw-rw-rw- 1 root       root        1493 Apr 15 20:48 vm.dist.args.example

It concerns testing rpm package which was fixed not to have any non-root files in a /usr/lib directory. Shouldn't above files be placed in a /etc/mongooseim directory with the root permissions?

[sebb7]

Ok. I think that I got it now. My previous comments were written when I was checking only what happened to the package and its files after the installation.

So the files I mentioned earlier (demo.config and final_template_vars.config) which belong to the mongooseim user will be later loaded by the bootstrap escript right?

So without these chmod's after package installation and trying to run the package right after the installation we get:

[root@d5670302ba42 test]# ls
mongooseim_3.6.2-1~centos~7_amd64.rpm
[root@d5670302ba42 test]# yum install -y ./mongooseim_3.6.2-1~centos~7_amd64.rpm
[...]
[root@d5670302ba42 test]# mongooseimctl bootstrap
Execute //usr/lib/mongooseim/scripts/bootstrap01-hello.sh
Hello from //usr/lib/mongooseim/scripts/bootstrap01-hello.sh script.
MongooseIM is installed into //usr/lib/mongooseim
Execute //usr/lib/mongooseim/scripts/bootstrap20-template.escript
Template script started
MIM_DIR is set to "//usr/lib/mongooseim"
Found 1 env variables
InputFiles ["demo.config"]
Result file written "/usr/lib/mongooseim/etc/demo.config": {error,eacces}
escript: exception error: no match of right hand side value {error,enoent}
  in function  'bootstrap20-template_escript__escript__1586__988269__522524__2':render_file/3 (//usr/lib/mongooseim/scripts/bootstrap20-template.escript, line 67)
  in call from 'bootstrap20-template_escript__escript__1586__988269__522524__2':'-init_and_template/3-lc$^1/1-1-'/4 (//usr/lib/mongooseim/scripts/bootstrap20-template.escript, line 46)
  in call from 'bootstrap20-template_escript__escript__1586__988269__522524__2':init_and_template/3 (//usr/lib/mongooseim/scripts/bootstrap20-template.escript, line 47)
  in call from escript:run/2 (escript.erl, line 758)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1
  in call from init:do_boot/3

But it was mentioned already that this will not work without changing permissions.

Do we want to allow the user to run mongooseimctl bootstrap just after package installation? If yes I think that we should have dedicated directory for the given user. Maybe solutions similar to this? But we already have the configuration files which belong to the root so when we want to create and save the configuration files for the mongooseim user i think that we need something like user-specific configuration files. I found this question and user-specific configuration directory for given app as ~/.config/MyApp sounds good to me. I know that this solution would probably require a lot of changes and work.

Sorry if I missed something and if I do not completely understand this issue :D Please let me know what do you think about it.

[mongoose-im]

8178.1 / Erlang 22.0 / small_tests / d7436ab
Reports root / small

---

8178.2 / Erlang 22.0 / internal_mnesia / d7436ab
Reports root/ big
OK: 1427 / Failed: 0 / User-skipped: 157 / Auto-skipped: 0

---

8178.7 / Erlang 22.0 / elasticsearch_and_cassandra_mnesia / d7436ab
Reports root/ big
OK: 328 / Failed: 0 / User-skipped: 28 / Auto-skipped: 0

---

8178.5 / Erlang 22.0 / riak_mnesia / d7436ab
Reports root/ big
OK: 1552 / Failed: 0 / User-skipped: 172 / Auto-skipped: 0

---

8178.6 / Erlang 22.0 / ldap_mnesia / d7436ab
Reports root/ big
OK: 1366 / Failed: 0 / User-skipped: 218 / Auto-skipped: 0

---

8178.9 / Erlang 21.3 / pgsql_mnesia / d7436ab
Reports root/ big / small
OK: 2669 / Failed: 0 / User-skipped: 195 / Auto-skipped: 0

[mongoose-im]

8189.1 / Erlang 22.0 / small_tests / 05f8a56
Reports root / small

---

8189.2 / Erlang 22.0 / internal_mnesia / 05f8a56
Reports root/ big
OK: 1427 / Failed: 0 / User-skipped: 157 / Auto-skipped: 0

---

8189.3 / Erlang 22.0 / odbc_mssql_mnesia / 05f8a56
Reports root/ big
OK: 2656 / Failed: 0 / User-skipped: 208 / Auto-skipped: 0

---

8189.4 / Erlang 22.0 / mysql_redis / 05f8a56
Reports root/ big
OK: 2651 / Failed: 0 / User-skipped: 213 / Auto-skipped: 0

---

8189.5 / Erlang 22.0 / riak_mnesia / 05f8a56
Reports root/ big
OK: 1552 / Failed: 0 / User-skipped: 172 / Auto-skipped: 0

---

8189.6 / Erlang 22.0 / ldap_mnesia / 05f8a56
Reports root/ big
OK: 1366 / Failed: 0 / User-skipped: 218 / Auto-skipped: 0

---

8189.7 / Erlang 22.0 / elasticsearch_and_cassandra_mnesia / 05f8a56
Reports root/ big
OK: 328 / Failed: 0 / User-skipped: 28 / Auto-skipped: 0

---

8189.9 / Erlang 21.3 / pgsql_mnesia / 05f8a56
Reports root/ big / small
OK: 2669 / Failed: 0 / User-skipped: 195 / Auto-skipped: 0

[codecov]

Codecov Report

Merging #2712 into master will decrease coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2712      +/-   ##
==========================================
- Coverage   79.43%   79.38%   -0.05%     
==========================================
  Files         358      358              
  Lines       30152    30152              
==========================================
- Hits        23950    23936      -14     
- Misses       6202     6216      +14     

Impacted Files	Coverage Δ
..._distrib/mod_global_distrib_outgoing_conns_sup.erl	91.30% <0.00%> (-8.70%)	⬇️
...system_metrics/service_mongoose_system_metrics.erl	71.92% <0.00%> (-7.02%)	⬇️
src/event_pusher/mod_event_pusher_sns.erl	85.13% <0.00%> (-5.41%)	⬇️
src/mam/mod_mam_rdbms_prefs.erl	92.59% <0.00%> (-3.71%)	⬇️
src/cassandra/mongoose_cassandra_worker.erl	75.36% <0.00%> (-2.47%)	⬇️
src/rdbms/mongoose_rdbms.erl	70.40% <0.00%> (-2.05%)	⬇️
src/mam/mod_mam_utils.erl	83.49% <0.00%> (-0.67%)	⬇️
src/mod_muc_log.erl	77.69% <0.00%> (ø)
src/ejabberd_c2s.erl	89.47% <0.00%> (ø)
src/pubsub/mod_pubsub.erl	72.22% <0.00%> (+0.11%)	⬆️
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3271941...0618f54. Read the comment docs.