Josh Thompson     about     archive     turing     office hours

HTTParty and to_json

I was having some trouble debugging an HTTParty POST request.

A few tools that were useful to me:

  1. post DEBUG info to STDOUT
  2. netcat to listen to HTTP requests locally

I had this code:

options = {
  headers: {
    "Content-Type": "application/json",
    authorization: "Bearer #{our_token}",
  },
  query: { data: true },
  body: { token: their_token },
  debug_output: STDOUT
}

And when I posted it:

HTTParty.post("#{BASE_URL}/endpoint", options)

I kept getting something like this:

opening connection to externalservice.net:443...
opened
starting SSL for externalservice.net:443...
SSL established
<- "POST /endpoint?data=true HTTP/1.1\r\nContent-Type: application/json\r\nAuthorization: Bearer alksdjflkajsdf\r\nHost: externalservice.net\r\nContent-Length: 1234\r\n\r\n"
<- "token=aslsdjfhasiudyfkajn"
-> "HTTP/1.1 400 Bad Request\r\n"
-> "Date: Mon, 11 Mar 2019 16:58:13 GMT\r\n"
-> "Content-Type: text/plain\r\n"
-> "Content-Length: 28\r\n"
-> "\r\n"
reading 28 bytes...
-> "Invalid json line 1 column 7"
read 28 bytes
Conn keep-alive
=> "Invalid json line 1 column 7"

Invalid json? The body was being passed into HTTParty as a hash, I had been assuming this would convert it to JSON.

.
body: { token: their_token },
.

So, I fired up netcat on localhost, to try making different POST requests and watch the formatting a little closer:

nc -l -k localhost 4000

I was seeing requests like so:

looks OK, right?

what was the solution?

My coworker John Livingston was chatting with me about a PR he had open, and asked for a review on an update to a Slack notification bot. I saw him use a suspicious method inside of his HTTParty request.

response = HTTParty.post(url, {
        body: payload.to_json,
        headers: {'Content-Type' => 'application/json'}
      })

AHHHHHHHH

AHHHHHHHH

Because I’d also been working on a different post request, where the Content-Type was application/x-www-form-urlencoded, I wasn’t actually looking to make sure the body was putting out JSON. Of course this isn’t JSON.

Not json:

token=slkdjflkj

json:

{"token":"slkdjflkj"}

The fix? add .to_json on the request body:

options = {
  headers: {
    "Content-Type": "application/json",
    authorization: "Bearer #{our_token}",
  },
  query: { data: true },
  body: { token: their_token }.to_json
  # I just added .to_json      ^^^^^^^
}

I’m writing this whole thing out because in hindsight it’s blindingly obvious what the problem was, but at the time I was really stuck on why the endpoint couldn’t parse the body as json.

Now we all know.

Get occasional emails

If I've written anything new, you'll get an email with summaries on Friday.