community.vmware.vmware_guest_file_operation: Fetch throw error "msg": "local file write exception : 'NoneType' object has no attribute 'read'", #485

kpatelsp · 2020-11-06T15:24:06Z

~~Please raise issues via the new interface~~

SUMMARY

I am using community.vmware.vmware_guest_file_operation: fetch option getting error

The full traceback is:
File "/tmp/ansible_community.vmware.vmware_guest_file_operation_payload_7oyn5t3f/ansible_community.vmware.vmware_guest_file_operation_payload.zip/ansible_collections/community/vmware/plugins/modules/vmware_guest_file_operation.py", line 355, in fetch
fatal: [localhost]: FAILED! ={
"changed": false,
"invocation": {
"module_args": {
"cluster": "clusetter",
"copy": null,
"datacenter": "datacenter",
"directory": null,
"fetch": {
"dest": "/1402.txt",
"src": "/ketan/1402.txt"
},
"folder": "/vm",
"hostname": "vcenter.domain.local",
"password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"port": 443,
"proxy_host": null,
"proxy_port": null,
"username": "[[email protected]](mailto:[email protected])",
"validate_certs": false,
"vm_id": "Redhat-Test",
"vm_id_type": "vm_name",
"vm_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"vm_username": "root"
}
},
"msg": "local file write exception : 'NoneType' object has no attribute 'read'",
"uuid": "4234a023-3a97-2de7-5dc3-e322eaecf7c3"
}

I already change vmware_guest_file_operation.py line 355 to as per
Line 341 hostname = self.module.params['hostname']
Line 350-352

url = fileTransferInfo.url
url = url.replace("*", self.params['hostname'])
resp, info = urls.fetch_url(self.module, url, method="GET")

Still getting same error
I found old solution that suggest to change above line. however no luck.

ISSUE TYPE

Bug Report

COMPONENT NAME

vmware_guest_file_operation

ANSIBLE VERSION


ansible 2.9.15
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.8 (default, Aug 18 2020, 08:33:21) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

CONFIGURATION

GALAXY_IGNORE_CERTS(/etc/ansible/ansible.cfg) = True
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False

OS / ENVIRONMENT

Red Hat Enterprise Linux Server release 7.8 (Maipo)

STEPS TO REPRODUCE

  community.vmware.vmware_guest_file_operation:
    hostname: "{{ vcenter_server }}"
    username: "{{ vcenter_user }}"
    password: "{{ vcenter_pass }}"
    datacenter: "{{ datacenter_name }}"
    cluster: "{{datacenter_cluster}}"
    folder: "{{vcenter_folder}}"
    validate_certs: no
    vm_id: "{{vm_name}}"
    vm_username: "{{ guest_username }}"
    vm_password: "{{ guest_userpassword }}"
    fetch:
       src: "/folder/1402.txt"
       dest: "/1402.txt"
  delegate_to: localhost
  register: facts
- debug:
    var: facts

EXPECTED RESULTS

ACTUAL RESULTS


The full traceback is:
  File "/tmp/ansible_community.vmware.vmware_guest_file_operation_payload_7oyn5t3f/ansible_community.vmware.vmware_guest_file_operation_payload.zip/ansible_collections/community/vmware/plugins/modules/vmware_guest_file_operation.py", line 355, in fetch
fatal: [localhost]: FAILED! ={
    "changed": false,
    "invocation": {
        "module_args": {
            "cluster": "clusetter",
            "copy": null,
            "datacenter": "datacenter",
            "directory": null,
            "fetch": {
                "dest": "/1402.txt",
                "src": "/ketan/1402.txt"
            },
            "folder": "/vm",
            "hostname": "vcenter.domain.local",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": 443,
            "proxy_host": null,
            "proxy_port": null,
            "username": "[email protected]",
            "validate_certs": false,
            "vm_id": "Redhat-Test",
            "vm_id_type": "vm_name",
            "vm_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "vm_username": "root"
        }
    },
    "msg": "local file write exception : 'NoneType' object has no attribute 'read'",
    "uuid": "4234a023-3a97-2de7-5dc3-e322eaecf7c3"
}"

Copied from original issue: ansible/ansible#72521

The text was updated successfully, but these errors were encountered:

kpatelsp · 2020-11-06T15:26:14Z

I already change vmware_guest_file_operation.py line 351 to

            url = fileTransferInfo.url
            url = url.replace("*", self.params['hostname'])
            resp, info = urls.fetch_url(self.module, url, method="GET")

Still getting the same error

"msg": "local file write exception : 'NoneType' object has no attribute 'read'",

sky-joker · 2020-11-07T13:21:51Z

Thank you @kpatelsp for reporting this issue.

Haven't a problem of connection reachable to an ESXi host from Ansible host?
The vmware_guest_file_operation module connects to ESXi when it fetches a file from a virtual machine.
For example, haven't the ping packet doesn't reach ESXi from Ansible host or ESXi hostname doesn't resolve?

Akasurde · 2020-11-09T07:04:07Z

@kpatelsp Please use the issue template. It helps to gather information about the issue. A simple one-liner in the issue is not going to take us anywhere.

kpatelsp · 2020-11-09T14:25:57Z

I did the open issues with the template but Bot closed it #72521.
I am able to create/delete the directory that works. Only Copy and Fetch fail.
I am able to ping my ESXi host from my ansible
My VSphere version 6.7.0.40000
Esxi Version 6.7.0, 15160138
When it connects to Esxi what account is used to connect.
Thanks

sky-joker · 2020-11-09T16:18:32Z

Connect to ESXi doesn't use a user.
When a file fetches, use a generated temporary URL(include temp token) from vCenter.
So the module connects to ESXi where the VM exists using a temporary URL.

Temporary URL example

https://ESXi IP or HostName:443/guestFile?id={N}&token={Temp Token}

So the below error happens a lot when the module can't connect to ESXi.

"msg": "local file write exception : 'NoneType' object has no attribute 'read'",

Akasurde · 2020-11-11T12:51:26Z

@kpatelsp Can you please make the following change and let us know the result?

diff --git a/plugins/modules/vmware_guest_file_operation.py b/plugins/modules/vmware_guest_file_operation.py
index 6eb20fa..707af31 100644
--- a/plugins/modules/vmware_guest_file_operation.py
+++ b/plugins/modules/vmware_guest_file_operation.py
@@ -349,6 +349,8 @@ class VmwareGuestFileManager(PyVmomi):
             url = fileTransferInfo.url
             url = url.replace("*", hostname)
             resp, info = urls.fetch_url(self.module, url, method="GET")
+            if info['status'] != 200 or not resp:
+                self.module.fail_json(msg="Failed to fetch file : %s" % info['msg'], body=info['body'])
             try:
                 with open(dest, "wb") as local_file:
                     local_file.write(resp.read())

I am able to reproduce this kind of error when I specify a invalid file name.

    - community.vmware.vmware_guest_file_operation:
        datacenter: Asia-Datacenter1
        cluster: Asia-Cluster1
        folder: "/Asia-Datacenter1/vm/prod"
        validate_certs: no
        vm_id: "{{ vm_name }}"
        vm_username: "{{ guest_username }}"
        vm_password: "{{ guest_password }}"
        fetch:
          src: "../etc/passwd"
          dest: "./1402.txt"

Error looks like this -

fatal: [localhost]: FAILED! => {"body": "<error>\n   <errorCode>22</errorCode>\n   <errorInfo>The file name is not valid</errorInfo>\n</error>", "changed": false, "msg": "Failed to fetch file : HTTP Error 500: Internal Server Error"}

Return useful error message when exception occurs while file operation. Partially fixes: ansible-collections#485 Signed-off-by: Abhijeet Kasurde <[email protected]>

kpatelsp · 2020-11-11T15:52:51Z

@Akasurde
Add those line in vmware_guest_file_operation.py

Error:

<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1605109618.4800053-154341-15475861582465 `" && echo ansible-tmp-1605109618.4800053-154341-15475861582465="` echo /root/.ansible/tmp/ansible-tmp-1605109618.4800053-154341-15475861582465 `" ) && sleep 0'
Using module file /root/collections/ansible_collections/community/vmware/plugins/modules/vmware_guest_file_operation.py
<localhost> PUT `/root/.ansible/tmp/ansible-local-154328wes7onz3/tmp825a4bp3` TO /root/.ansible/tmp/ansible-tmp-1605109618.4800053-154341-15475861582465/AnsiballZ_vmware_guest_file_operation.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1605109618.4800053-154341-15475861582465/ /root/.ansible/tmp/ansible-tmp-1605109618.4800053-154341-15475861582465/AnsiballZ_vmware_guest_file_operation.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/libexec/platform-python /root/.ansible/tmp/ansible-tmp-1605109618.4800053-154341-15475861582465/AnsiballZ_vmware_guest_file_operation.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1605109618.4800053-154341-15475861582465/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_community.vmware.vmware_guest_file_operation_payload_3qzke76a/ansible_community.vmware.vmware_guest_file_operation_payload.zip/ansible_collections/community/vmware/plugins/modules/vmware_guest_file_operation.py", line 354, in fetch
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "cluster": "cluster",
            "copy": null,
            "datacenter": "datacenter",
            "directory": null,
            "fetch": {
                "dest": "/tmp/1402.txt",
                "src": "/tmp/1402.txt"
            },
            "folder": "/vm",
            "hostname": "vcenter.domain.local",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": 443,
            "proxy_host": null,
            "proxy_port": null,
            "username": "[email protected]",
            "validate_certs": false,
            "vm_id": "Redhat-Test",
            "vm_id_type": "vm_name",
            "vm_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "vm_username": "root"
        }
    },
    "msg": "Failed to Fetch file from Vm VMware exception : 'body'",
    "uuid": "4234a023-3a97-2de7-5dc3-e322eaecf7c3"

Akasurde · 2020-11-12T04:17:30Z

@kpatelsp

How about this -

diff --git a/plugins/modules/vmware_guest_file_operation.py b/plugins/modules/vmware_guest_file_operation.py
index 6eb20fa..58d5fb4 100644
--- a/plugins/modules/vmware_guest_file_operation.py
+++ b/plugins/modules/vmware_guest_file_operation.py
@@ -349,6 +349,8 @@ class VmwareGuestFileManager(PyVmomi):
             url = fileTransferInfo.url
             url = url.replace("*", hostname)
             resp, info = urls.fetch_url(self.module, url, method="GET")
+            if info.get('status') != 200 or not resp:
+                self.module.fail_json(msg="Failed to fetch file : %s" % info.get('msg', ''), body=info.get('body', ''))
             try:
                 with open(dest, "wb") as local_file:
                     local_file.write(resp.read())

kpatelsp · 2020-11-12T15:00:33Z

@Akasurde
New Error:
"msg": "Failed to fetch file : Request failed: <urlopen error [Errno 0] Error>"

kpatelsp · 2020-11-28T15:33:46Z

@Akasurde
What info you need.

Akasurde · 2020-11-30T04:36:05Z

I do not know what is 'urlopen error 0' and not able to reproduce this at my end. @sky-joker Do you know anything about this error?

sky-joker · 2020-11-30T12:21:28Z

I do not know what is 'urlopen error 0' and not able to reproduce this at my end. @sky-joker Do you know anything about this error?

Umm...
Sorry, I don't know.
I never look at the error.

I seem the cause of the error is an environmental dependence.
I'm wondering that anything is existing between ansible host and ESXi.(For example, a proxy server and so on)

My lab doesn't the error occurs again...

kpatelsp · 2020-12-09T14:58:38Z

Issue Resolved.

It was a proxy issue. I did have an entire management network to the environment file added

no_proxy="*.domain.local,x.x.x.x/24"

This did not work. I have to add each host IP address.

Thanks for all your help.

mariolenz · 2020-12-09T17:57:47Z

Issue Resolved.

It was a proxy issue. I did have an entire management network to the environment file added

no_proxy="*.domain.local,x.x.x.x/24"

This did not work. I have to add each host IP address.

Akasurde mentioned this issue Nov 11, 2020

vmware_guest_file_operation: provide error message #491

Merged

Akasurde added bug This issue/PR relates to a bug module module needs_info This issue requires further information. Please answer any outstanding questions labels Nov 12, 2020

mariolenz closed this as completed Dec 9, 2020

Akasurde assigned sky-joker Dec 10, 2020

Akasurde added this to the v1.6.0 milestone Dec 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

community.vmware.vmware_guest_file_operation: Fetch throw error "msg": "local file write exception : 'NoneType' object has no attribute 'read'", #485

community.vmware.vmware_guest_file_operation: Fetch throw error "msg": "local file write exception : 'NoneType' object has no attribute 'read'", #485

kpatelsp commented Nov 6, 2020 •

edited by sky-joker

Loading

kpatelsp commented Nov 6, 2020 •

edited by Akasurde

Loading

sky-joker commented Nov 7, 2020 •

edited

Loading

Akasurde commented Nov 9, 2020

kpatelsp commented Nov 9, 2020

sky-joker commented Nov 9, 2020

Akasurde commented Nov 11, 2020

kpatelsp commented Nov 11, 2020 •

edited by Akasurde

Loading

Akasurde commented Nov 12, 2020

kpatelsp commented Nov 12, 2020 •

edited

Loading

kpatelsp commented Nov 28, 2020

Akasurde commented Nov 30, 2020

sky-joker commented Nov 30, 2020

kpatelsp commented Dec 9, 2020

mariolenz commented Dec 9, 2020

community.vmware.vmware_guest_file_operation: Fetch throw error "msg": "local file write exception : 'NoneType' object has no attribute 'read'", #485

community.vmware.vmware_guest_file_operation: Fetch throw error "msg": "local file write exception : 'NoneType' object has no attribute 'read'", #485

Comments

kpatelsp commented Nov 6, 2020 • edited by sky-joker Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

kpatelsp commented Nov 6, 2020 • edited by Akasurde Loading

sky-joker commented Nov 7, 2020 • edited Loading

Akasurde commented Nov 9, 2020

kpatelsp commented Nov 9, 2020

sky-joker commented Nov 9, 2020

Akasurde commented Nov 11, 2020

kpatelsp commented Nov 11, 2020 • edited by Akasurde Loading

Akasurde commented Nov 12, 2020

kpatelsp commented Nov 12, 2020 • edited Loading

kpatelsp commented Nov 28, 2020

Akasurde commented Nov 30, 2020

sky-joker commented Nov 30, 2020

kpatelsp commented Dec 9, 2020

mariolenz commented Dec 9, 2020

kpatelsp commented Nov 6, 2020 •

edited by sky-joker

Loading

kpatelsp commented Nov 6, 2020 •

edited by Akasurde

Loading

sky-joker commented Nov 7, 2020 •

edited

Loading

kpatelsp commented Nov 11, 2020 •

edited by Akasurde

Loading

kpatelsp commented Nov 12, 2020 •

edited

Loading