Improve the passwords tool behavior in distributed environments #1854

s-ocando · 2022-09-20T13:20:25Z

Wazuh version	Platform
4.3.7	Ubuntu 20.04

The Wazuh password tool is not working as expected in a distributed environment. In a Wazuh dashboard node, the tool is not updating the password in the Wazuh dashboard keystore nor updating the Wazuh manager API password in the wazuh.yml configuration file as it should.

In a Wazuh server node, the tool is not updating the corresponding password in the Filebeat keystore.

Below, I'll describe the process of changing the default passwords in a Wazuh distributed environment and include comments and suggestions along the way.

Wazuh indexer node

As a first step, I generated a file with random passwords using the option -gf, --generate-file <wazuh-passwords.txt>.

/usr/share/wazuh-indexer/plugins/opensearch-security/tools/wazuh-passwords-tool.sh -gf wazuh-passwords.txt

wazuh-passwords.txt

# Admin user for the web user interface and Wazuh indexer. Use this user to log in to Wazuh dashboard
  indexer_username: 'admin'
  indexer_password: 'xMzqfVE3x0K2Wc6Rzw64?0zhI0yiW+Pm'

# Wazuh dashboard user for establishing the connection with Wazuh indexer
  indexer_username: 'kibanaserver'
  indexer_password: '**mgzA46F10dE5S7IhrHot3YKy5FQW3k'

# Regular Dashboard user, only has read permissions to all indices and all permissions on the .kibana index
  indexer_username: 'kibanaro'
  indexer_password: 'PT8aSPYRorZ?NnypCmLKC3bIdS71.K+S'

# Filebeat user for CRUD operations on Wazuh indices
  indexer_username: 'logstash'
  indexer_password: 'qMqJOndasCC2e+eBvygwW1q+5.JAzji+'

# User with READ access to all indices
  indexer_username: 'readall'
  indexer_password: '9t69Uej1Wq?ytS6lcGAX.Ee0hsa8f?lK'

# User with permissions to perform snapshot and restore operations
  indexer_username: 'snapshotrestore'
  indexer_password: '8gTKyzy*a?VH69c?2?9*x4e5OMxIAy1d'

# Password for wazuh API user
  api_username: 'wazuh'
  api_password: 'juB9FVXyPi.76LyIhBO2fAHMMPJ?CgvN'

# Password for wazuh-wui API user
  api_username: 'wazuh-wui'
  api_password: '*X+888Lc2MiDtUYenaeT+LYsUoZndbHk'

Next, I used the following command to change the Wazuh indexer passwords.

/usr/share/wazuh-indexer/plugins/opensearch-security/tools/wazuh-passwords-tool.sh --change-all  -f wazuh-passwords.txt

20/09/2022 09:45:06 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.
20/09/2022 09:45:18 INFO: The password for user admin is xMzqfVE3x0K2Wc6Rzw64?0zhI0yiW+Pm
20/09/2022 09:45:18 INFO: The password for user kibanaserver is **mgzA46F10dE5S7IhrHot3YKy5FQW3k
20/09/2022 09:45:18 INFO: The password for user kibanaro is PT8aSPYRorZ?NnypCmLKC3bIdS71.K+S
20/09/2022 09:45:18 INFO: The password for user logstash is qMqJOndasCC2e+eBvygwW1q+5.JAzji+
20/09/2022 09:45:18 INFO: The password for user readall is 9t69Uej1Wq?ytS6lcGAX.Ee0hsa8f?lK
20/09/2022 09:45:18 INFO: The password for user snapshotrestore is 8gTKyzy*a?VH69c?2?9*x4e5OMxIAy1d
20/09/2022 09:45:18 WARNING: Wazuh indexer passwords changed. Remember to update the password in the Wazuh dashboard and Filebeat nodes if necessary, and restart the services.

The output message states INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed. suggesting that the API credential should be provided. If they are provided, the output states:

ERROR: The Wazuh API user wazuh does not exist
ERROR: The Wazuh API user wazuh-wui does not exist

This message is confusing as the users do exist, the problem is that there's not a Wazuh manager API available on this node. An additional check verifying if the Wazuh manager is installed could be added to improve the output messages.

Consider talking about the "Wazuh manager API" instead of just the "Wazuh API" to avoid possible confusion with the Wazuh indexer API.

Wazuh server node

I ran the following command in a Wazuh server node.

bash wazuh-passwords-tool.sh --change-all --admin-user wazuh --admin-password wazuh -f wazuh-passwords.txt

20/09/2022 10:00:46 INFO: The password for Wazuh API user wazuh is juB9FVXyPi.76LyIhBO2fAHMMPJ?CgvN
20/09/2022 10:00:46 INFO: The password for Wazuh API user wazuh-wui is *X+888Lc2MiDtUYenaeT+LYsUoZndbHk

The password tool changed the Wazuh manager API passwords but did not update the admin user password in Filebeat as it should.

# filebeat test output
elasticsearch: https://192.168.56.4:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.56.4
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.2
    dial up... OK
  talk to server... ERROR 401 Unauthorized: Unauthorized

After updating the password manually and restarting Filebeat the communication is restored.

echo xMzqfVE3x0K2Wc6Rzw64?0zhI0yiW+Pm | filebeat keystore add password --stdin --force

Regarding the passwords tool output message, it should indicate that as the Wazuh manager API passwords have been changed, the user should update the wazuh.yml configuration file in the Wazuh dashboard node.

Wazuh dashboard node

Finally, I ran the passwords tool in the Wazuh dashboard node.

bash wazuh-passwords-tool.sh --change-all -f wazuh-passwords.txt

20/09/2022 10:43:38 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.

In this case, the passwords were not updated either in the Wazuh dashboard keystore nor in the wazuh.yml configuration file.

I changed the wazuh-wui password manually in the /usr/share/wazuh-dashboard/data/wazuh/config/wazuh.yml configuration file, updated the kibanaserver user password in the Wazuh dashboard keystore and restarted the Wazuh dashboard service.

echo **mgzA46F10dE5S7IhrHot3YKy5FQW3k | /usr/share/wazuh-dashboard/bin/opensearch-dashboards-keystore --allow-root add -f --stdin opensearch.password

After updating the passwords and restarting, everything is working as expected.

As a final remark, consider adding a check to verify if the Wazuh manager API user set in the wazuh.yml configuration file is indeed wazuh-wui and consider the case where there's more than one API configured.

The text was updated successfully, but these errors were encountered:

teddytpc1 · 2024-05-14T13:33:50Z

@Enaraque. We need to test if we can reproduce these behaviors to define if they are already fixed.
Please, use 4.8.0 scripts to test it.

Enaraque · 2024-05-20T12:26:07Z

For now I have reproduced these behaviours for the indexer and the manager. They still seem to have the same problem. When I generate the passwords in the indexer and change them in both the indexer and the manager, the passwords for Filebeat are not changed in the manager.
Here is a step by step description of what I have been doing so far:

Wazuh Indexer

I generated the password file with:
bash wazuh-passwords-tool.sh -gf wazuh-passwords.txt

This command generated this password file:

wazuh-password.txt

# Admin user for the web user interface and Wazuh indexer. Use this user to log in to Wazuh dashboard
  indexer_username: 'admin'
  indexer_password: 'bFU*pv7Q343NR8loPSv8pnIQgR*cAjfH'

# Wazuh dashboard user for establishing the connection with Wazuh indexer
  indexer_username: 'kibanaserver'
  indexer_password: 'tOg1+ViA1*22CHyrmqyj*GaQ?RBGfd2Q'

# Regular Dashboard user, only has read permissions to all indices and all permissions on the .kibana index
  indexer_username: 'kibanaro'
  indexer_password: '8SAp29zMp+blOC+8xwN9cafM210T*KBU'

# Filebeat user for CRUD operations on Wazuh indices
  indexer_username: 'logstash'
  indexer_password: 'P5voNn*rlL7ag7nG2A6K6FlPCQuFFRgM'

# User with READ access to all indices
  indexer_username: 'readall'
  indexer_password: 'fLQ67uOs??hmwSvpL2+ejWpSP2?oY+cj'

# User with permissions to perform snapshot and restore operations
  indexer_username: 'snapshotrestore'
  indexer_password: 'iRoohS8Hrr1?.deGnjRheMp7mhWdK5mV'

# Password for wazuh API user
  api_username: 'wazuh'
  api_password: '4S5ObanUaz9HlGw939oQBVg*wo7FWAa.'

# Password for wazuh-wui API user
  api_username: 'wazuh-wui'
  api_password: 'Wg27.m.vwtShDh?3MaeBSetjBR.4TxfQ'

Next, I changed the wazuh indexer password:
bash wazuh-passwords-tool.sh --change-all -f wazuh-passwords.txt

20/05/2024 11:16:51 INFO: Updating the internal users.
20/05/2024 11:17:13 INFO: A backup of the internal users has been saved in the /etc/wazuh-indexer/internalusers-backup folder.
20/05/2024 11:17:13 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.
20/05/2024 11:17:55 INFO: The password for user admin is bFU*pv7Q343NR8loPSv8pnIQgR*cAjfH
20/05/2024 11:17:55 INFO: The password for user kibanaserver is tOg1+ViA1*22CHyrmqyj*GaQ?RBGfd2Q
20/05/2024 11:17:55 INFO: The password for user kibanaro is 8SAp29zMp+blOC+8xwN9cafM210T*KBU
20/05/2024 11:17:55 INFO: The password for user logstash is P5voNn*rlL7ag7nG2A6K6FlPCQuFFRgM
20/05/2024 11:17:55 INFO: The password for user readall is fLQ67uOs??hmwSvpL2+ejWpSP2?oY+cj
20/05/2024 11:17:55 INFO: The password for user snapshotrestore is iRoohS8Hrr1?.deGnjRheMp7mhWdK5mV
20/05/2024 11:17:55 WARNING: Wazuh indexer passwords changed. Remember to update the password in the Wazuh dashboard, Wazuh server, and Filebeat nodes if necessary, and restart the services.

The message INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed. keeps popping up with "wazuh API" name which is still confusing.

Wazuh manager

When I change the passwords in the manager, only the API passwords are changed.

[root@localhost vagrant]# bash wazuh-passwords-tool.sh --change-all --admin-user wazuh --admin-password wazuh -f wazuh-passwords.txt
20/05/2024 11:26:05 INFO: The password for Wazuh API user wazuh is 4S5ObanUaz9HlGw939oQBVg*wo7FWAa.
20/05/2024 11:26:06 INFO: The password for Wazuh API user wazuh-wui is Wg27.m.vwtShDh?3MaeBSetjBR.4TxfQ

The Filebeat password remains unchanged, as when testing the connection, the error "Unauthorized" is displayed.

[root@localhost vagrant]# filebeat test output
elasticsearch: https://192.168.56.13:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.56.13
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... ERROR 401 Unauthorized: Unauthorized

Enaraque · 2024-05-21T08:51:59Z

If I update the Filebeat password manually in the wazuh manager, it seems to work as expected.

echo bFU*pv7Q343NR8loPSv8pnIQgR*cAjfH | filebeat keystore add password --stdin --force

Wazuh dashboard

If I change the passwords on the wazuh dashboard neither the wazuh-wui API user password nor the kibanaserver password are automatically updated.

Because of this, I have to manually do the update of the wazuh-wui password in /usr/share/wazuh-dashboard/data/wazuh/config/wazuh.yml and the kibanaserver password with

tOg1+ViA1*22CHyrmqyj*GaQ?RBGfd2Q /usr/share/wazuh-dashboard/bin/opensearch-dashboards-keystore --allow-root add -f --stdin opensearch.password

Conclusion

This shows that the bugs are still present in the 4.8.0 scripts.

Enaraque · 2024-05-24T12:20:40Z

Issue Update

During these days we have been working on solving this issue.

Wazuh Indexer

Regarding the wazuh indexer, we have managed to fix the problem when changing passwords where the message INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed. could be confusing. Now this message does not appear when it is not necessary.

Wazuh manager

Regarding the wazuh manager and the filebeat password change problem, we have been working on the following aspects:

Now every time a new password file is generated, a password for filebeat is added in the API section. With this, every time the user wants to change the filebeat password with the password file, he will be able to do it.
We are working on the automatic change of the filebeat password with the password file. So far it seems to be working correctly, although more tests are needed.

Testing

Here we can see how in the indexer the admin password has been changed so the manager generates an authorisation error.

[root@localhost passwords_good]# filebeat test output
elasticsearch: https://192.168.56.13:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.56.13
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... ERROR 401 Unauthorized: Unauthorized

Once the script is executed you can see how the password is successfully changed.

[root@localhost passwords_good]# bash wazuh-passwords-tool.sh --change-all --admin-user wazuh --admin-password 4S5ObanUaz9HlGw939oQBVg*wo7FWAa. -f wazuh-passwords2.txt
24/05/2024 12:03:17 INFO: The password for Wazuh API user wazuh is 4S5ObanUaz9HlGw939oQBVg*wo7FWAa.
24/05/2024 12:03:18 INFO: The password for Wazuh API user wazuh-wui is nSJrzudtj0.n1xln1cBS8iDNxJb0PAA0
24/05/2024 12:04:08 INFO: The new password for Filebeat is RKO?OMAnvD+Nk6+S4a9I+F9LAGpkg9?3
[root@localhost passwords_good]# filebeat test output
elasticsearch: https://192.168.56.13:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.56.13
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2

Enaraque · 2024-05-27T12:08:29Z

Issue update

Wazuh server

Added the option if you want to change the filebeat password by giving the --user and --password instead of the --changeall option.

Tests

If we do the filebeat test after changing the admin password in the indexer, we get Unauthorized:

Filebeat with wrong passworrd in the wazuh server

[root@localhost passwords_good]# bash wazuh-passwords-tool.sh --api --admin-user wazuh --admin-password 4S5ObanUaz9HlGw939oQBVg*wo7FWAa. --user filebeat --password g2ou7A4PZsYLIh?HEF.2PCf5*Yj4XfCs
27/05/2024 09:56:40 INFO: The new password for Filebeat is g2ou7A4PZsYLIh?HEF.2PCf5*Yj4XfCs
[root@localhost passwords_good]# filebeat test output
elasticsearch: https://192.168.56.13:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.56.13
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... ERROR 401 Unauthorized: Unauthorized

Then if we change the filebeat password on the server, giving the user instead of the --changeall option, we can see that it is also changed correctly:

change filebeat password with user option

[root@localhost passwords_good]# bash wazuh-passwords-tool.sh --api --admin-user wazuh --admin-password 4S5ObanUaz9HlGw939oQBVg*wo7FWAa. --user filebeat --password HnRSRU6.TIDbv9kGPfEuEhRBIhKdJ?Nt
27/05/2024 09:57:43 INFO: The new password for Filebeat is HnRSRU6.TIDbv9kGPfEuEhRBIhKdJ?Nt

Filebeat test output run succesfully

[root@localhost passwords_good]# filebeat test output
elasticsearch: https://192.168.56.13:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.56.13
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2

Wazuh dashboard

The necessary options have been added to be able to:

Give the file and be able to change the wazuh-wui password with the passwords file and the --changeall.
Be able to change the wazuh-wui password with the --user and --password option.

Tests

This part is still being tested.

Enaraque · 2024-05-28T12:25:36Z

Issue Update

Wazuh dashboard

Everything that corresponds to the kibanaserver user has been updated. This includes changing the password for this with the --changeall option as well as using --user and --password.

Tests

When changing the wazuh-wui password in the wazuh server if we open the dashboard we have this:

Now if we updated the corresponding passwords in the dashboard, it works properly:

root@debian9:/home/vagrant/passwords_good# bash wazuh-passwords-tool.sh --change-all  -f wazuh-passwords_new.txt
28/05/2024 12:21:00 INFO: Updated wazuh-wui user password in wazuh dashboard to 'LGJr4ejgeoBEq5ATyp+hu3DK63M2CFTV'. Remember to restart the service.
28/05/2024 12:21:08 INFO: The password for the kibanaserver user has been updated to G79KSfTTvyj6?6*MvV96.uFp3gkxeEnr

Also we can do it we the --user and --password option:

root@debian9:/home/vagrant/passwords_good# bash wazuh-passwords-tool.sh --user wazuh-wui --password LGJr4ejgeoBEq5ATyp+hu3DK63M2CFTV
28/05/2024 12:22:52 INFO: Updated wazuh-wui user password in wazuh dashboard to 'LGJr4ejgeoBEq5ATyp+hu3DK63M2CFTV'. Remember to restart the service.
root@debian9:/home/vagrant/passwords_good# bash wazuh-passwords-tool.sh --user kibanaserver --password G79KSfTTvyj6?6*MvV96.uFp3gkxeEnr
28/05/2024 12:23:55 INFO: The password for the kibanaserver user has been updated to G79KSfTTvyj6?6*MvV96.uFp3gkxeEnr

This was referenced Sep 26, 2022

Wazuh indexer passwords change rework wazuh/wazuh-documentation#5543

Open

Improve the "Securing your Wazuh installation" section in the step-by-step installation guide wazuh/wazuh-documentation#5635

Merged

snaow added this to the Release 4.5.0 milestone Nov 16, 2022

snaow removed this from the Release 4.5.0 milestone Dec 21, 2022

alberpilot added type/feature New feature request component: password-tool labels Jan 16, 2023

teddytpc1 added level/task Subtask issue type/bug Bug issue labels May 22, 2023

teddytpc1 assigned Enaraque May 14, 2024

teddytpc1 removed type/feature New feature request component: password-tool labels May 21, 2024

Enaraque mentioned this issue May 29, 2024

Solve bugs when changing passwords in the manager, indexer and dashboard services #2968

Merged

Enaraque linked a pull request May 29, 2024 that will close this issue

Solve bugs when changing passwords in the manager, indexer and dashboard services #2968

Merged

davidcr01 closed this as completed in #2968 Jun 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve the passwords tool behavior in distributed environments #1854

Improve the passwords tool behavior in distributed environments #1854

s-ocando commented Sep 20, 2022 •

edited

Loading

teddytpc1 commented May 14, 2024

Enaraque commented May 20, 2024

Enaraque commented May 21, 2024

Enaraque commented May 24, 2024

Enaraque commented May 27, 2024

Enaraque commented May 28, 2024

Improve the passwords tool behavior in distributed environments #1854

Improve the passwords tool behavior in distributed environments #1854

Comments

s-ocando commented Sep 20, 2022 • edited Loading

Wazuh indexer node

Wazuh server node

Wazuh dashboard node

teddytpc1 commented May 14, 2024

Enaraque commented May 20, 2024

Wazuh Indexer

Wazuh manager

Enaraque commented May 21, 2024

Wazuh dashboard

Conclusion

Enaraque commented May 24, 2024

Issue Update

Wazuh Indexer

Wazuh manager

Testing

Enaraque commented May 27, 2024

Issue update

Wazuh server

Tests

Wazuh dashboard

Tests

Enaraque commented May 28, 2024

Issue Update

Wazuh dashboard

Tests

s-ocando commented Sep 20, 2022 •

edited

Loading