Apply public firewalld rules immediately

Previously, firewalld rules were applied to configuration, then
firewalld reloaded to pick up all the new rules. Reloading firewalld can
be disruptive because it sets all chains to a DROP policy while building
up its firewall rules, breaking open connections.

This change switches to applying rules both permanently (to config) and
immediately, such that no reload is required.

Change-Id: I8e48b7827b33bdd2061d0e89c905bea8e29f60e8
(cherry picked from commit 2fbb067)

ansible/roles/haproxy-config/tasks/main.yml

@@ -24,8 +24,9 @@
 
 - name: "Configuring firewall for {{ project_name }}"
   firewalld:
-    offline: "yes"
-    permanent: "yes"
+    immediate: true
+    offline: true
+    permanent: true
     port: "{{ item.value.port }}/tcp"
     state: "enabled"
     zone: "{{ external_api_firewalld_zone }}"
@@ -38,5 +39,3 @@
     - enable_external_api_firewalld | bool
     - kolla_action != "config"
   with_dict: "{{ project_services | extract_haproxy_services }}"
-  notify:
-    - "Reload firewalld"

ansible/roles/loadbalancer/handlers/main.yml

@@ -1,10 +1,4 @@
 ---
-- name: Reload firewalld
-  become: True
-  service:
-    name: "firewalld"
-    state: reloaded
-
 # NOTE(yoctozepto): this handler dance is to ensure we delay restarting master
 # keepalived and haproxy which control VIP address until we have working backups.
 # This could be improved by checking if backup keepalived do not report FAULT state.

releasenotes/notes/firewalld-immediate-c2abf09977c455a9.yaml

@@ -0,0 +1,7 @@
+---
+features:
+  - |
+    Modifies public API firewalld rules to be applied immediately to a running
+    firewalld service. This requires firewalld to be running, but avoids
+    reloading firewalld, which is disruptive due to the way in which firewalld
+    builds its firewall chains.