- 0. How to install Ansible Galaxy collection to correct path:
- 1. How the NetBox objects are expressed as IaC, using the py script in dir IaC to create phy devices with mgmt IP, mgmt interface, rack position, site and more
- 2. How to add all physical appliances with management IPs
- 3. Add phy interfaces
- 4. Add VMs and IP plan
- 5. How to integrate NAPALM with Containerized NetBox, to allow for real-time collecting of facts in NetBox, from its network devices objects, based on netbox-docker guide
- How to configure the NetBox with credentials for the Network OS devices it will NAPALM calls to:
- How to assign NAPALM driver to a NetBox device:
- How to check if a NAPALM driver is installed in NetBox:
- How to rebuild image from the netboxcommunity/netbox:latest in Docker Hub, while adding napalm-driver in Dockerfile:
- 6. How to backup the PostgreSQL database of NetBox and the uploaded files (like images) and to restore, per netbox-docker official wiki
- 7. How to clear all NetBox database
- 8. How to get NetBox access over HTTPS
- [Done by placing a Reverse-Proxy in front of it, like Caddy, per netbox-docker guide:](#done-by-placing-a-reverse-proxy-in-front-of-it-like-caddy-per-netbox-docker-guide)
- create a server certificate (like from Vault PKI) with SANs: DNS Name=netbox.dnszone, IP Address=192.168.x.x, IP Address=127.0.0.1
- configure the Caddy Reverse Proxy
- to troubleshoot check Caddy RP container logs:
- how to connect to NetBox (with HTTP to HTTPS forwarding):
- open SSH tunnel and forward 192.168.x.x:443 to localhost:xyz (
ssh -i ~/.ssh/id_rsa user@bastion -L xyz:192.168.x.x:443), then use https://localhost:xyz/ or https://127.0.0.1:xyz/ in browser - OR open dynamic SOCKS5 tunnel (
ssh -i ~/.ssh/id_rsa user@bastion -D zyx) and set SOCKS5 proxy 127.0.0.1:zyx and pass DNS through it in browser (Mozilla Firefox), then use https://netbox.tooling.neo/ or https://192.168.x.x/ in browser
- open SSH tunnel and forward 192.168.x.x:443 to localhost:xyz (
- 9. How to add a visual Topology view in NetBox Organization > Sites:
- Clone NextBox UI Plugin repo, where a Dockerfile exists for building a custom netbox Docker image containing the plugin
- Ensure the proper Proxy is set for the container, so that the Docker image can be build with packages from Internet:
- set the custom netbox-ui-plugin image to be created from a previous image (if specific packages like NAPALM were installed on that image)
- Edit and assign NetBox devices with tags of type icon_router, icon_switch, icon_server, per official NextBox UI Plugin doc, so that a representative icon is available for them in Topology view (those without get question mark ?)
Ansible playbooks usage for Netbox automation, based on Galaxy collection.
boburciu@WX-5CG020BDT2:~$ ansible-config list | grep COLLECTIONS_PATHS -C1 # verify default location for the collections, so that the new modules can be read by Ansible cfg
Ansible version
COLLECTIONS_PATHS:
default: ~/.ansible/collections:/usr/share/ansible/collections
--
env:
- name: ANSIBLE_COLLECTIONS_PATHS
- name: ANSIBLE_COLLECTIONS_PATH
boburciu@WX-5CG020BDT2:~$
boburciu@WX-5CG020BDT2:~$
boburciu@WX-5CG020BDT2:~$ ansible-galaxy collection install netbox.netbox --collections-path ~/.ansible/collections # installing the collection of roles in proper location
1. How the NetBox objects are expressed as IaC, using the py script in dir IaC to create phy devices with mgmt IP, mgmt interface, rack position, site and more
boburciu@WX-5CG020BDT2: $ cd ~/netbox-ansible-automation
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$ cat external_vars.yml
url_var: http://192.168.200.23:8001/
token_var: ***********************
## ======== variables used for NetBox WebUI Organization tab objects ======== ##
tag_name: ' '
tag_description: ' '
tag_color: FFA500 # RGB color in hexadecimal, use https://htmlcolorcodes.com/
#
region_name: ' '
#
site_name: ' '
site_status: Planned
site_region: EMEA
site_facility: Hyperscale DC
site_asn: ' ' # found by trial and error that needs to be integer
site_time_zone: ' '
site_description: ' '
site_physical_address: ' '
site_shipping_address: ' '
site_latitude: ' ' # found by trial and error the need to ensure there are no more than 8 digits in total
site_longitude: ' ' # found by trial and error the need to ensure there are no more than 9 digits in total
site_contact_name: ' '
site_contact_phone: ' '
site_contact_email: ' '
#
tenant_group_name: ' '
tenant_name: ' ' # found by trial and error that does not support underscore, for VRF to be referenced to its name
tenant_description: ' '
# tenant_comments: "SDN backend"
tenant_tag: oiaas
#
rack_group_name: ' '
rack_site: ' '
rack_role_name: Hyperscale infra
rack_role_color: DF2E08
rack_name: ' '
rack_desc_units: no # rack units will be numbered top-to-bottom, yes or no
rack_outer_unit: Millimeters # whether the rack unit is in Millimeters or Inches and is required if outer_width/outer_depth is specified
rack_outer_width: 605 # per https://www.ibm.com/support/pages/overview-ibm-42u-and-47u-1200-mm-deep-static-and-deep-dynamic-rack
rack_outer_depth: 1200
rack_type: 4-post cabinet # Choices: 2-post frame, 4-post frame, 4-post cabinet, Wall-mounted frame, Wall-mounted cabinet
rack_u_height: 42
rack_width: 23 # the rail-to-rail width, choices: 10, 19, 21, 23
rack_serial: 00000000
rack_facility_id: ' ' # found by trial and error that fields group & facility_id must make a unique set for a rack
rack_comments: ' '
## ======== variables used for NetBox WebUI Devices tab objects ======== ##
device_manufacturer_name: ' '
device_model: ' '
device_part_number: ' '
device_u_height: ' ' # found by trial and error that it needs to be 0 if parent type is "child"
device_is_full_depth: no # whether or not the device consumes both front and rear rack faces, yer or no
device_subdevice_role: ' ' # whether the device type is parent (which has device bays), child (which must be installed within a device bay), or neither
device_hw_set_id: ' ' # just a tag for the set of hardware params above; found by trial and error that it should not be part of model of manufacturer, i.e. CE6851 is not accepted
#
device_role_name: ' '
device_role_color: ' '
#
device_hostname: ' '
device_site: ' '
device_face: front # front/rear, found by trial and error that needs not be used if parent type is "child"
device_primary_ip4: ' '
device_serial: ' '
device_rack_name: ' '
device_position_in_rack: ' ' # found by trial and error that needs not be used if parent type is "child"
device_tenant: ' '
device_comments: ' '
device_tag: ' '
#
device_bay_chassis: Chassis1 # device the device bay will be associated to. The device type must be "parent".
device_bay_slot: 1
device_bay_blade: ' ' # device that will be installed into the bay. The device type must be "child" and 0U
#
interface_device: ' '
interface_name: ' '
interface_mac_address: ' '
interface_enabled: true
interface_type: 1000BASE-T
interface_mtu: 1500
interface_mgmt_only: yes # interface is used only for out-of-band management, yes or no
interface_description: Mgmt interface
#
cable_end_a_host: SWH-OoB-R2
cable_end_a_if: GE1/0/1
cable_end_b_host: ' '
cable_end_b_if: iBMC
cable_type: mmf-om4
## ======== variables used for NetBox WebUI IPAM tab objects ======== ##
vrf_name: OOB
vrf_rd: ' '
vrf_description: OOB for underlay switches and firewalls and IPMI network of servers
vrf_tag: oiaas
#
ip_addr_address: ' '
ip_addr_vrf: OOB
ip_addr_tenant: ' '
ip_addr_status: Reserved
ip_addr_role: VIP # the role of the IP address: Loopback, Secondary, Anycast, VIP, VRRP, HSRP, GLBP, CARP
ip_addr_description: Mgmt interface
ip_addr_interface_name: ' '
ip_addr_interface_device: ' '
Taking info from an .xls file, the py script creates and populates yaml files with the values of a template overriten
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$ python3 IaC/xls2iac.py
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$ ls -lt IaC/
total 28
-rw-rw-rw- 1 boburciu boburciu 10253 Apr 5 21:08 'Physical appliances 4 IaC.xlsx'
-rw-rw-rw- 1 boburciu boburciu 4077 Feb 10 18:44 xls2iac.py
-rwxrw-rw- 1 boburciu boburciu 4238 Feb 10 10:31 run_ansible_optimized.sh
-rwxrw-rw- 1 boburciu boburciu 1576 Feb 8 21:22 get_iac_unique_var.sh
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$
boburciu@WX-5CG020BDT2: /netbox-ansible-automation$ cd ../parse_excel_servers/; python3 xls2iac.py; cd ../netbox-ansible-automation/
boburciu@WX-5CG020BDT2: /netbox-ansible-automation$ ls -lt ~/parse_excel_servers/ | grep external_vars_
-rw-rw-rw- 1 boburciu boburciu 4589 Feb 7 01:42 external_vars_******.yml
-rw-rw-rw- 1 boburciu boburciu 4585 Feb 7 01:42 external_vars_*******.yml
-rw-rw-rw- 1 boburciu boburciu 4635 Feb 7 01:42 external_vars_********.yml
-rw-rw-rw- 1 boburciu boburciu 4631 Feb 7 01:42 external_vars_*********.yml
:
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$ for i inls -lt ~/parse_excel_servers/ | grep external_vars_ | awk '{print $9}'; do echo ""; echo ""; echo "***** running playbook for variables in $i *****"; echo ""; echo ""; echo ""; ansible-playbook -i ./hosts create_device_wMgmtIntIP_inRack_inTenant_inRack_inSite.yml -e "external_vars='../parse_excel_servers/$i' ansible_python_interpreter='/usr/bin/python3'" -v; done
./IaC/run_ansible_optimized.sh -d ~/parse_excel_servers/
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$ ls -lX /home/boburciu/parse_excel_servers/external_vars* | awk '{print $9}' | cat -n | tail -36
51 /home/boburciu/parse_excel_servers/external_vars_******.yml
:
86 /home/boburciu/parse_excel_servers/external_vars_******.yml
boburciu@WX-5CG020BDT2: /netbox-ansible-automation$ for x in ls -lX /home/boburciu/parse_excel_servers/external_vars* | awk '{print $9}' | cat | tail -36 ; do echo ""; echo ""; echo "@@@@@ running Device creation playbooks for extra-vars in $x @@@@@"; echo ""; echo ""; echo ""; ansible-playbook -i ./hosts -v add_device_w_mgmt.yml -e "external_vars=$x"; done
boburciu@WX-5CG020BDT2: /netbox-ansible-automation$ ansible-playbook -i ./hosts -v create_device_wMgmtIntIP_inRack_inTenant_inRack_inSite.yml -e "external_vars=/home/boburciu/parse_excel_servers/external_vars_******.yml"
To configure data-plane interfaces of phy device, we can use NAPALM py library to connect to NOS devices (ToR switches) and parse live their configuration and using ansible-runner py library, call Ansible playbooks to update devices (with LAG ID, trunk/access mode, VLANs) by a py3 script - in dir NAPALM/
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$ ls -lt NAPALM/
total 68
-rw-rw-rw- 1 boburciu boburciu 3542 Apr 5 21:27 napalm_calls_4_py_shell.py
-rw-rw-rw- 1 boburciu boburciu 20291 Apr 5 21:26 huawei-napalm2netbox.py
-rw-rw-rw- 1 boburciu boburciu 23519 Apr 5 21:25 dell-napalm2netbox.py
-rwxrwxrwx 1 boburciu boburciu 20395 Apr 5 21:23 README.md
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$
To configure VMs and IP prefixes and addresses for virtual interfaces (VM interfaces and VLANifs), we can use the py3 script getting input from an .xls file - in dir IPplan/
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$ ls -lt IPplan/
total 380
-rw-rw-rw- 1 boburciu boburciu 9552 Apr 5 21:51 ip-plan2netbox.py
-rw-rw-rw- 1 boburciu boburciu 376320 Apr 5 21:47 IP_plan_4_NetBox.xls
boburciu@WX-5CG020BDT2:~/netbox-ansible-automation$
5. How to integrate NAPALM with Containerized NetBox, to allow for real-time collecting of facts in NetBox, from its network devices objects, based on netbox-docker guide
[root@NetboX netbox-docker]# vi /root/projects/netbox-docker/env/netbox.env
[root@NetboX netbox-docker]# cat /root/projects/netbox-docker/env/netbox.env | grep NAPALM
NAPALM_USERNAME=user
NAPALM_PASSWORD=pass
NAPALM_TIMEOUT=10
[root@NetboX netbox-docker]#
[root@NetboX netbox-docker]# pip3 install napalm
[root@NetboX netbox-docker]# pip3 install napalm-ce
[root@NetboX netbox-docker]# docker-compose restart
Restarting netbox-docker_nginx_1 ... done
Restarting netbox-docker_netbox_1 ... done
Restarting netbox-docker_netbox-worker_1 ... done
Restarting netbox-docker_redis-cache_1 ... done
Restarting netbox-docker_postgres_1 ... done
Restarting netbox-docker_redis_1 ... done
[root@NetboX netbox-docker]#
1. Create a platform where the driver section is filled with "junos" for the default Juniper NAPALM driver or "ce" for the napalm-ce driver for Huawei devices
[root@NetboX netbox-docker]# docker-compose run --rm --entrypoint /bin/bash netbox
Creating netbox-docker_netbox_run ... done
bash-5.0$ ./manage.py nbshell
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
### NetBox interactive shell (d9fceee54b53)
### Python 3.9.1 | Django 3.1.3 | NetBox 2.10.3
### lsmodels() will show available models. Use help(<model>) for more info.
>>>
>>>
>>> import napalm
>>> driver=napalm.get_network_driver("ce")
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.9/site-packages/napalm/base/__init__.py", line 97, in get_network_driver
raise ModuleImportError(
napalm.base.exceptions.ModuleImportError: Cannot import "ce". Is the library installed?
>>> exit()
bash-5.0$ exit
exit
[root@NetboX netbox-docker]#
How to rebuild image from the netboxcommunity/netbox:latest in Docker Hub, while adding napalm-driver in Dockerfile:
[root@NetboX netbox-docker]# vi Dockerfile
[root@NetboX netbox-docker]# diff Dockerfile Dockerfile.bkp
26d25
< napalm-ce \
79c78
< RUN chmod -R g+w static media
---
> RUN mkdir static && chmod -R g+w static media
[root@NetboX netbox-docker]#
[root@NetboX netbox-docker]# mkdir .netbox
[root@NetboX netbox-docker]# touch .netbox/requirements.txt
[root@NetboX netbox-docker]# cat .netbox/requirements.txt
napalm==3.2.0
napalm-ce==0.2.0
ruamel.yaml==0.16.13
django-auth-ldap==2.3.0
django-storages[azure,boto3,dropbox,google,libcloud,sftp]==1.11.1
[root@NetboX netbox-docker]#
[root@NetboX netbox-docker]# docker image build . --tag netbox:napalm --build-arg FROM=netboxcommunity/netbox:latest --build-arg NETBOX_PATH=".netbox"
Sending build context to Docker daemon 158.7kB
Step 1/30 : ARG FROM
:
Successfully built 3ab20b99381c
Successfully tagged netbox:napalm
[root@NetboX netbox-docker]# [root@NetboX netbox-docker]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
netbox napalm 3ab20b99381c 45 seconds ago 336MB
postgres 12-alpine 7ec50832fed0 8 weeks ago 159MB
redis 6-alpine 933c79ea2511 2 months ago 31.6MB
netboxcommunity/netbox latest 8e452c542927 2 months ago 229MB
nginx 1.19-alpine 629df02b47c8 3 months ago 22.3MB
[root@NetboX netbox-docker]#
[root@NetboX netbox-docker]# cp docker-compose.yml docker-compose.yml.bkp
[root@NetboX netbox-docker]# vi docker-compose.yml
[root@NetboX netbox-docker]# diff docker-compose.yml docker-compose.yml.bkp
4c4
< image: netbox:napalm
---
> image: netboxcommunity/netbox:${VERSION-latest}
[root@NetboX netbox-docker]#
[root@NetboX netbox-docker]# docker-compose ps
Name Command State Ports
------------------------------
[root@NetboX netbox-docker]#
[root@NetboX netbox-docker]# docker-compose up
Creating network "netbox-docker_default" with the default driver
Creating netbox-docker_redis-cache_1 ... done
Creating netbox-docker_postgres_1 ... done
Creating netbox-docker_redis_1 ... done
Creating netbox-docker_netbox-worker_1 ... done
Creating netbox-docker_netbox_1 ... done
Creating netbox-docker_nginx_1 ... done
Attaching to netbox-docker_postgres_1, netbox-docker_redis_1, netbox-docker_redis-cache_1, netbox-docker_netbox-worker_1, netbox-docker_netbox_1, netbox-docker_nginx_1
[root@NetboX netbox-docker]# docker-compose ps
Name Command State Ports
------------------------------------------------------------------------------------------------------------------------------
netbox-docker_netbox-worker_1 python3 /opt/netbox/netbox ... Up
netbox-docker_netbox_1 /opt/netbox/docker-entrypo ... Up
netbox-docker_nginx_1 /docker-entrypoint.sh ngin ... Up 80/tcp,
0.0.0.0:8001->8080/tcp,0.0.0.0:49154->8080/tcp
netbox-docker_postgres_1 docker-entrypoint.sh postgres Up 5432/tcp
netbox-docker_redis-cache_1 docker-entrypoint.sh sh -c ... Up 6379/tcp
netbox-docker_redis_1 docker-entrypoint.sh sh -c ... Up 6379/tcp
[root@NetboX netbox-docker]#
[root@NetboX netbox-docker]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
055cd02cd6a0 nginx:1.19-alpine "/docker-entrypoint.…" 3 minutes ago Up 3 minutes 80/tcp, 0.0.0.0:8001->8080/tcp, 0.0.0.0:49154->8080/tcp netbox-docker_nginx_1
e541d6a00f8e netbox:napalm "/opt/netbox/docker-…" 3 minutes ago Up 3 minutes netbox-docker_netbox_1
441f071d94dd netbox:napalm "python3 /opt/netbox…" 3 minutes ago Up 3 minutes netbox-docker_netbox-worker_1
4c3c5d0de41d redis:6-alpine "docker-entrypoint.s…" 3 minutes ago Up 3 minutes 6379/tcp netbox-docker_redis_1
8bea76d9a0dd postgres:12-alpine "docker-entrypoint.s…" 3 minutes ago Up 3 minutes 5432/tcp netbox-docker_postgres_1
5fff97d26517 redis:6-alpine "docker-entrypoint.s…" 3 minutes ago Up 3 minutes 6379/tcp netbox-docker_redis-cache_1
[root@NetboX netbox-docker]#
6. How to backup the PostgreSQL database of NetBox and the uploaded files (like images) and to restore, per netbox-docker official wiki
Access the database:
docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'Take a database backup
docker-compose exec -T postgres sh -c 'pg_dump -cU $POSTGRES_USER $POSTGRES_DB' | gzip > db_dump.sql.gzRestore that database:
# Stop all NetBox instances that access the db
docker-compose stop netbox netbox-worker
# Restore the DB dump
gunzip -c db_dump.sql.gz | docker-compose exec -T postgres sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'Backup of the media directory, which contains uploaded images.
docker-compose exec -T netbox tar c -jf - -C /opt/netbox/netbox/media ./ > media-backup.tar.bz2Restore of the media directory:
⚠️ This may overwrite files in the media directory!
docker-compose exec -T netbox tar x -jvf - -C /opt/netbox/netbox/media < media-backup.tar.bz2 [root@gitlab-runner-and-netbox netbox-docker]# cat docker-compose.yml
version: '3.4'
services:
:
# postgres
postgres:
image: postgres:12-alpine
env_file: env/postgres.env
volumes:
- netbox-postgres-data:/var/lib/postgresql/data
:
volumes:
:
netbox-postgres-data:
driver: local
:
[root@gitlab-runner-and-netbox netbox-docker]# docker volume ls
DRIVER VOLUME NAME
:
local netbox-docker_netbox-postgres-data
:
[root@gitlab-runner-and-netbox netbox-docker]# docker volume inspect netbox-docker_netbox-postgres-data
[
{
"CreatedAt": "2021-08-19T12:19:36+03:00",
"Driver": "local",
"Labels": {
"com.docker.compose.project": "netbox-docker",
"com.docker.compose.version": "1.27.4",
"com.docker.compose.volume": "netbox-postgres-data"
},
"Mountpoint": "/var/lib/docker/volumes/netbox-docker_netbox-postgres-data/_data",
"Name": "netbox-docker_netbox-postgres-data",
"Options": null,
"Scope": "local"
}
]
[root@gitlab-runner-and-netbox netbox-docker]#
[root@gitlab-runner-and-netbox netbox-docker]# ls -lt /var/lib/docker/volumes/netbox-docker_netbox-postgres-data/_data
total 60
drwx------. 2 70 70 63 Aug 19 12:35 pg_stat_tmp
drwx------. 4 70 70 68 Aug 19 12:24 pg_logical
drwx------. 2 70 70 4096 Aug 19 12:19 global
drwx------. 2 70 70 6 Aug 19 12:19 pg_stat
-rw-------. 1 70 70 94 Aug 19 12:19 postmaster.pid
-rw-------. 1 70 70 24 Aug 19 12:19 postmaster.opts
drwx------. 2 70 70 18 Aug 19 12:19 pg_notify
drwx------. 3 70 70 92 Apr 2 18:09 pg_wal
drwx------. 6 70 70 54 Jan 25 2021 base
-rw-------. 1 70 70 4782 Jan 25 2021 pg_hba.conf
drwx------. 2 70 70 18 Jan 25 2021 pg_subtrans
drwx------. 2 70 70 18 Jan 25 2021 pg_xact
-rw-------. 1 70 70 1636 Jan 25 2021 pg_ident.conf
-rw-------. 1 70 70 88 Jan 25 2021 postgresql.auto.conf
-rw-------. 1 70 70 26580 Jan 25 2021 postgresql.conf
drwx------. 2 70 70 6 Jan 25 2021 pg_commit_ts
drwx------. 2 70 70 6 Jan 25 2021 pg_dynshmem
drwx------. 4 70 70 36 Jan 25 2021 pg_multixact
drwx------. 2 70 70 6 Jan 25 2021 pg_replslot
drwx------. 2 70 70 6 Jan 25 2021 pg_serial
drwx------. 2 70 70 6 Jan 25 2021 pg_snapshots
drwx------. 2 70 70 6 Jan 25 2021 pg_tblspc
drwx------. 2 70 70 6 Jan 25 2021 pg_twophase
-rw-------. 1 70 70 3 Jan 25 2021 PG_VERSION
[root@gitlab-runner-and-netbox netbox-docker]#
[root@gitlab-runner-and-netbox netbox-docker]# rm -rf /var/lib/docker/volumes/netbox-docker_netbox-postgres-data/_data
[root@gitlab-runner-and-netbox netbox-docker]# ls -lt /var/lib/docker/volumes/netbox-docker_netbox-postgres-data/_data
ls: cannot access /var/lib/docker/volumes/netbox-docker_netbox-postgres-data/_data: No such file or directory
[root@gitlab-runner-and-netbox netbox-docker]#
[root@gitlab-runner-and-netbox netbox-docker]# mkdir /var/lib/docker/volumes/netbox-docker_netbox-postgres-data/_data
[root@gitlab-runner-and-netbox netbox-docker]# docker-compose down
[root@gitlab-runner-and-netbox netbox-docker]# docker-compose up
netbox-docker_redis-cache_1 is up-to-date
netbox-docker_redis_1 is up-to-date
netbox-docker_netbox-worker_1 is up-to-date
Creating netbox-docker_postgres_1 ... done
Attaching to netbox-docker_redis-cache_1, netbox-docker_redis_1, netbox-docker_netbox-worker_1, netbox-docker_postgres_1, netbox-docker_netbox_1, netbox-docker_nginx_1
:
postgres_1 | CREATE DATABASE
postgres_1 |
:
postgres_1 | PostgreSQL init process complete; ready for start up.
:
netbox_1 | ✅ Initialisation is done.
netbox_1 | [2021-08-19 09:42:47 +0000] [1] [INFO] Starting gunicorn 20.0.4
netbox_1 | [2021-08-19 09:42:47 +0000] [1] [INFO] Listening at: http://0.0.0.0:8001 (1)
[root@gitlab-runner-and-netbox netbox-docker]# docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER -d $POSTGRES_DB -V'
psql (PostgreSQL) 12.5
[root@gitlab-runner-and-netbox netbox-docker]#
Done by placing a Reverse-Proxy in front of it, like Caddy, per netbox-docker guide:
create a server certificate (like from Vault PKI) with SANs: DNS Name=netbox.dnszone, IP Address=192.168.x.x, IP Address=127.0.0.1
ubuntu@netbox-vm:/vault$ /vault$ sudo mv vault_netbox_key_1.key ../netbox-docker/vault_netbox_key_ipsan.key
ubuntu@netbox-vm:sudo mv vault_netbox_cert_1.crt ../netbox-docker/vault_netbox_cert_ipsan.crt
configure the Caddy Reverse Proxy
ubuntu@netbox-vm:~/netbox-docker$ cat Caddyfile
# ./Caddyfile
{
default_sni 192.168.x.x # will use this IP as TLS SNI for all ClientHello s that come without one (like when IP address is used in URL instead of hostname); Caddy needs to see SNI, otherwise it returns TLS Alert
}
netbox.dnszone, localhost, 192.168.x.x, 127.0.0.1 {
reverse_proxy netbox:8080
encode gzip zstd
file_server
tls /etc/ssl/private/cert_ipsan.crt /etc/ssl/private/key_ipsan.key
# or:
# tls /etc/ssl/private/cert.pem
log {
level error
}
}
ubuntu@netbox-vm:~/netbox-docker$ cat docker-compose.override.yml
services:
tls:
image: caddy:2-alpine
depends_on:
- netbox
volumes:
- /home/ubuntu/netbox-docker/vault_netbox_cert_ipsan.crt:/etc/ssl/private/cert_ipsan.crt:ro,z
- /home/ubuntu/netbox-docker/vault_netbox_key_ipsan.key:/etc/ssl/private/key_ipsan.key:ro,z
- /home/ubuntu/netbox-docker/Caddyfile:/etc/caddy/Caddyfile:ro
ports:
- 80:80 # Allows for http redirection
- 443:443
ubuntu@netbox-vm:/netbox-docker$ /netbox-docker$ sudo docker-compose down
ubuntu@netbox-vm:sudo docker-compose up&
ubuntu@netbox-vm:~/netbox-docker$ sudo docker-compose logs tls
Attaching to netbox-docker_tls_1
:
tls_1 | {"level":"info","ts":1620325857.0710692,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"netbox.dnszone","server_name":"srv0"}
tls_1 | {"level":"info","ts":1620325857.0712597,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"192.168.x.x","server_name":"srv0"}
tls_1 | {"level":"info","ts":1620325857.0714242,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"127.0.0.1","server_name":"srv0"}
tls_1 | {"level":"info","ts":1620325857.0714638,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
:
ubuntu@netbox-vm:~/netbox-docker$
open SSH tunnel and forward 192.168.x.x:443 to localhost:xyz ( ssh -i ~/.ssh/id_rsa user@bastion -L xyz:192.168.x.x:443), then use https://localhost:xyz/ or https://127.0.0.1:xyz/ in browser
OR open dynamic SOCKS5 tunnel (ssh -i ~/.ssh/id_rsa user@bastion -D zyx) and set SOCKS5 proxy 127.0.0.1:zyx and pass DNS through it in browser (Mozilla Firefox), then use https://netbox.tooling.neo/ or https://192.168.x.x/ in browser
Clone NextBox UI Plugin repo, where a Dockerfile exists for building a custom netbox Docker image containing the plugin
[root@gitlab-runner-and-netbox projects]# git clone https://github.com/iDebugAll/nextbox-ui-plugin
[root@gitlab-runner-and-netbox projects]#
Ensure the proper Proxy is set for the container, so that the Docker image can be build with packages from Internet:
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# vim /etc/systemd/system/docker.service.d/http-proxy.conf
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# cat /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://192.168.201.130:8080"
Environment="HTTPS_PROXY=http://192.168.201.130:8080"
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# systemctl daemon-reload
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# systemctl restart docker
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# systemctl show --property=Environment docker
Environment=HTTP_PROXY=http://192.168.201.130:8080 HTTPS_PROXY=http://192.168.201.130:8080
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# cat ~/.docker/config.json
{
"proxies":
{
"default":
{
"httpProxy": "http://192.168.201.130:8080",
"httpsProxy": "http://192.168.201.130:8080"
}
}
}
set the custom netbox-ui-plugin image to be created from a previous image (if specific packages like NAPALM were installed on that image)
[root@gitlab-runner-and-netbox projects]# vim nextbox-ui-plugin/Dockerfile
[root@gitlab-runner-and-netbox projects]# cat nextbox-ui-plugin/Dockerfile
ARG FROM_IMAGE=netbox
ARG FROM_TAG=napalm-huawei-fortinet
ARG FROM=${FROM_IMAGE}:${FROM_TAG}
FROM ${FROM}
ENV VIRTUAL_ENV=/opt/netbox/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
COPY ./nextbox_ui_plugin /source/nextbox-ui-plugin/nextbox_ui_plugin/
COPY ./setup.py /source/nextbox-ui-plugin/
COPY ./MANIFEST.in /source/nextbox-ui-plugin/
COPY ./README.md /source/nextbox-ui-plugin/
COPY --chown=1000:1000 --chmod=644 ./nextbox_ui_plugin/static/nextbox_ui_plugin /opt/netbox/netbox/static/nextbox_ui_plugin
RUN pip3 install --no-cache-dir nextbox-ui-plugin
[root@gitlab-runner-and-netbox projects]# cd nextbox-ui-plugin
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# docker build -t netbox-custom .
Sending build context to Docker daemon 10.92MB
Step 1/12 : ARG FROM_IMAGE=netbox
Step 2/12 : ARG FROM_TAG=napalm-huawei-fortinet
Step 3/12 : ARG FROM=${FROM_IMAGE}:${FROM_TAG}
Step 4/12 : FROM ${FROM}
:
Successfully tagged netbox-custom:latest
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
netbox-custom latest 8fdf1515e906 7 seconds ago 370MB
<none> <none> 35c29aca8441 16 hours ago 432MB
netboxcommunity/netbox latest-ldap 4dbbdeaf05a4 7 days ago 409MB
netbox napalm-huawei-fortinet dfbe342cff44 7 weeks ago 340MB
alpine latest d4ff818577bc 2 months ago 5.6MB
gitlab/gitlab-runner latest 6965be4d8032 3 months ago 819MB
netbox napalm 3ab20b99381c 5 months ago 336MB
postgres 12-alpine 7ec50832fed0 7 months ago 159MB
redis 6-alpine 933c79ea2511 7 months ago 31.6MB
netboxcommunity/netbox latest 8e452c542927 7 months ago 229MB
nginx 1.19-alpine 629df02b47c8 8 months ago 22.3MB
[root@gitlab-runner-and-netbox nextbox-ui-plugin]#
[root@gitlab-runner-and-netbox nextbox-ui-plugin]# cd ../netbox-docker_Feper/
[root@gitlab-runner-and-netbox netbox-docker_Feper]# cat -n docker-compose.yml | head
1 version: '3.4'
2 services:
3 netbox: &netbox
4 image: netbox:napalm-huawei-fortinet
5 depends_on:
6 - postgres
7 - redis
8 - redis-cache
9 - netbox-worker
10 env_file: env/netbox.env
[root@gitlab-runner-and-netbox netbox-docker_Feper]# vim +4 docker-compose.yml
[root@gitlab-runner-and-netbox netbox-docker_Feper]# cat -n docker-compose.yml | head
1 version: '3.4'
2 services:
3 netbox: &netbox
4 image: netbox-custom:latest
5 depends_on:
6 - postgres
7 - redis
8 - redis-cache
9 - netbox-worker
10 env_file: env/netbox.env
[root@gitlab-runner-and-netbox netbox-docker_Feper]#
[root@gitlab-runner-and-netbox netbox-docker_Feper]# cat -n configuration/configuration.py | grep PLUGIN
192 PLUGINS = []
196 PLUGINS_CONFIG = {
[root@gitlab-runner-and-netbox netbox-docker_Feper]# vim +192 configuration/configuration.py
[root@gitlab-runner-and-netbox netbox-docker_Feper]# cat -n configuration/configuration.py | grep PLUGIN
192 PLUGINS = ['nextbox_ui_plugin']
196 PLUGINS_CONFIG = {
[root@gitlab-runner-and-netbox netbox-docker_Feper]# docker-compose down
Stopping netbox-docker_feper_nginx_1 ...
Stopping netbox-docker_feper_netbox_1 ...
Stopping netbox-docker_feper_netbox-worker_1 ...
Stopping netbox-docker_feper_redis_1 ...
Stopping netbox-docker_feper_postgres_1 ...
Stopping netbox-docker_feper_redis-cache_1 ...
[root@gitlab-runner-and-netbox netbox-docker_Feper]# docker-compose up -d &