Making a HTTPoison POST multipart request with more data other than the file

[Bestulo]

I'm trying to build a function to send a file through a POST request in multipart format, using this as guide, but HTTPoison keeps giving me a two errors no matter what changes I make to the form. They are all

HTTPoison.post("https://api.telegram.org/myCredentials", {:multipart, form}, headers)

and the three versions of my form and the errors are the following (whether I use headers or not):

1st and 2nd Version (same error for both):

form = [{"photo", [{"name", "myphoto.jpg"}, {:file, "files/aphoto.jpg"}]}, {"chat_id", 237799110}]
-----
form = [photo: [{"name", "myphoto.jpg"}, {:file, "files/aphoto.jpg"}], chat_id: 237799110]
Which give me this error:

** (FunctionClauseError) no function clause matching in anonymous fn/2 in :hackney_multipart.len_mp_stream/2
      (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_multipart.erl:159: anonymous fn({"photo", [{"name", "myphoto.jpg"}, {:file, "files/aphoto.jpg"}]}, 0) in :hackney_multipart.len_mp_stream/2
       (stdlib) lists.erl:1263: :lists.foldl/3
      (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_multipart.erl:159: :hackney_multipart.len_mp_stream/2
      (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_request.erl:319: :hackney_request.handle_body/4
      (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_request.erl:81: :hackney_request.perform/2
      (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney.erl:373: :hackney.send_request/2
    (httpoison) lib/httpoison/base.ex:432: HTTPoison.Base.request/9

And the third version:

form = [chat_id: 237799110, photo: [{"name", "myphoto.jpg"}, {:file, "files/aphoto.jpg"}]]

Which gives me the following error:

** (ArgumentError) argument error
              :erlang.byte_size(:chat_id)
    (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_multipart.erl:255: :hackney_multipart.mp_data_header/2
    (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_multipart.erl:180: anonymous fn/3 in :hackney_multipart.len_mp_stream/2
     (stdlib) lists.erl:1263: :lists.foldl/3
    (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_multipart.erl:159: :hackney_multipart.len_mp_stream/2
    (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_request.erl:319: :hackney_request.handle_body/4
    (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney_request.erl:81: :hackney_request.perform/2
    (hackney) c:/Users/venta/projects/elixir/wrapper/deps/hackney/src/hackney.erl:373: :hackney.send_request/2

Is this a problem by HTTPoison? Am I doing something wrong?

[edgurgel]

I think the problem here is that chat_id is not a binary/String.t. Keywords use atoms as keys

[Bestulo]

As you can see there are three versions of the form. The atom one is the third posted (first tried), and it gave me that error ([chat_id: 23779911, ...).

Then I tried making it a string and got the 1st error (second tried) where it passes the whole form as first argument to Hackney, and then since it doesn't read it in first level, gives it an accSize of 0 as second argument and Hackney just doesn't know what to do with that.

What I don't know is how to pass this to Elixir so that it will pass it to Hackney in a proper format (I know no Erlang at all) for it to be processed effectively. And, as far as I know (and as far as I've tried and asked around) this issue may be due to a problem in HTTPoison.

[Spyes]

+1 I'm also having trouble getting this to work. I need to POST an image file as well as a text argument (subscription_key for example), but no matter what I try I can't get HTTPoison to recognize anything past the :file ....

[Bestulo]

See this post on Elixir Forum that I made, I explore the topic and get to the point where I think this is an issue with the module.

To sum up, the message one sends has to have a field for the file, say content-disposition: form-data; name="registrationPicture";, and that's not possible to do with a file because it can either have ONLY a file, or it can have both a file and a name value, and no other parameters can be set.

[edgurgel]

Have you checked the docs of the multipart API? https://github.com/benoitc/hackney#send-a-body

Example:

iex(12)> form = [{:file, "/tmp/a"}, {"my_key", "my_value"}]
[{:file, "/tmp/a"}, {"my_key", "my_value"}]
iex(13)> HTTPoison.post!("http://httpbin.org/post", {:multipart, form}, []).body |> IO.puts
{
  "args": {},
  "data": "",
  "files": {
    "file": "content of file /tmp/a\n"
  },
  "form": {
    "my_key": "my_value"
  },
  "headers": {
    "Connection": "close",
    "Content-Length": "407",
    "Content-Type": "multipart/form-data; boundary=---------------------------skuticrcfxwnfsqb",
    "Host": "httpbin.org",
    "User-Agent": "hackney/1.7.1"
  },
  "json": null,
  "origin": "219.88.238.223",
  "url": "http://httpbin.org/post"
}

I saw that your keys are nested but AFAIK there's no support for this. What kind of request are you expecting? Can you show using curl for example?

[cblavier]

Same issue here, have you found a way to upload a file along with json data?

[Grafikart]

@edgurgel I have the same problem but like you stated the problem could come from hackney

This is the CURL I'm trying to translate into httpoison

curl -X POST -F filedata=@/some/path/video.mp4 -H "AccessToken: XXX" https://api.vid.me/video/upload

The file need to be named filedata instead of the default file used by hackney but apparently this use case can work but my understanding of erlang is too limited to figure this out :( (benoitc/hackney#292)

[Grafikart]

I digged deeper and found the solution :

curl -X POST -F filedata=@/some/path/video.mp4 -H "AccessToken: XXX" https://api.vid.me/video/upload

For this use case you need to use :multipart body

file = "/some/path/video.mp4"
HTTPoison.post(
      "https://api.vid.me/video/upload",
      {:multipart, [{:file, file, {"form-data", [name: "filedata", filename: Path.basename(file)]}, []}]},
      ["AccessToken": "XXXXX"]
)

[DmitryKK]

You should use multipart body like:

form = {:multipart, [{"chat_id", "237799110"}, {:file, file_path, {"form-data", [{:name, "photo"}, {:filename, Path.basename(file_path)}]}, []}]}

file_path - path to file
{:name, "photo"} - is tuple for name of field. You need "photo" for sending photo
{"chat_id", "237799110"} - is tuple for extra fields. All fields must be strings

Check that :)

[alex88]

@DmitryKK in my case with code

form = {:multipart, [{"player", Poison.encode!(player)}, {:file, path, {"form-data", [{:name, "image"}, {:filename, Path.basename(path)}]}}]}

I get

** (FunctionClauseError) no function clause matching in :lists.map/2

    The following arguments were given to :lists.map/2:

        # 1
        #Function<3.94787237/1 in :hackney_multipart.mp_file_header/2>

        # 2
        {"form-data", [name: "image", filename: "briefly-793993-614189-5"]}

    (stdlib) lists.erl:1238: :lists.map/2
    (hackney) uploader/deps/hackney/src/hackney_multipart.erl:235: :hackney_multipart.mp_file_header/2
    (hackney) uploader/deps/hackney/src/hackney_multipart.erl:164: anonymous fn/3 in :hackney_multipart.len_mp_stream/2
    (stdlib) lists.erl:1263: :lists.foldl/3
    (hackney) uploader/deps/hackney/src/hackney_multipart.erl:159: :hackney_multipart.len_mp_stream/2
    (hackney) uploader/deps/hackney/src/hackney_request.erl:317: :hackney_request.handle_body/4
    (hackney) uploader/deps/hackney/src/hackney_request.erl:81: :hackney_request.perform/2
    (hackney) uploader/deps/hackney/src/hackney.erl:358: :hackney.send_request/2

Update: nvm sorry forgot to add the empty list after the file name

[cigzigwon]

@DmitryKK Passing body params in this way does not show up on our receiving end but end end but our files do. I'm passing files from a new API endpoint for the sake of utilizing an old one to do the heavy lifting for GDrive. This shim for an API upgrade/re-write. You are correct about passing atoms as it will trigger an error. But when passing them as strings its as if they never get picked up. Hmmm... I've tried everything and I'm wondering if this has to do with how Keyword lists and Tuple based lists get processed.

[cigzigwon]

@DmitryKK Nevermind! Of course it's not a problem w/Hackney! The problem is that NodeJS Express multer lib does not seem to understand the spec that is being send or I require additional config on that end. This would be our v1 API. This just re-enforces that it's good to move off Express.

[cigzigwon]

@DmitryKK there is an issue w/content-type when you make requests against against express/multer and you have to build the request like is being advised. The request you make is actually being incorrectly stamped with application/octet-stream which it is NOT. It is text/plain. You should probably update your code either way. @edgurgel Here's a full example which should help the greater community gain some insight.

files =
dir
|> File.ls!()
|> Enum.map(fn filename ->
{:file, @root_dir <> "#{id}/" <> filename, {"form-data", [name: "files[]", filename: filename]}, []}
end)

case HTTPoison.post(
		 apiv1_host <> storage_path,
		 {:multipart, files ++ [{"id", id, ["content-type": "text/plain", "content-disposition": ~s(form-data; name="id")]}, {"title", opp_name, ["content-type": "text/plain", "content-disposition": ~s(form-data; name="title")]}]}
	) do
{:ok, res} ->
Logger.info("#{inspect(res.status_code)}")
:ok

{:error, error} ->
Logger.error("#{inspect(error)}")
:error
end