Load Testing your app with Siege

Last time, I dug into using Apache Benchmark to do performance testing on a page that requires authentication to access.

Today, we’ll figure out how to use siege to visit many unique URLs on our page, and to get benchmarks on that process. I’ll next figure out performance profiling in Datadog, and with these three tools put together, we should be ready to make some meaningful improvements to our application.

This series of posts is a direct result of Nate Berkopec’s The Complete Guide to Rails Performance, with aid from his screencasts, as well as Steve Grossi’s Load Testing Rails Apps with Apache Bench, Siege, and JMeter. Armed with these resources, and standing on shoulders of giants, off we go.

Siege’s description:

Siege is an http load testing and benchmarking utility. It was designed to let web developers measure their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP, HTTPS and FTP protocols. It lets its user hit a server with a configurable number of simulated clients. Those clients place the server “under siege.”

You can get siege with brew install siege.

I’m using it because it can run a list of URLs you give it. Imagine your app is a store, and it lists a few thousand products. Each product should have a unique URL, something like www.mystore.com/product-name, or maybe www.mystore.com/productguid-product-name. That product-guid makes sure that you can have unique URLs, even if there are two items with the same product name.

Knowing what’s in your database, you can easily concat product-guid and product-name, stick it to the end of www.mystore.com, and come up with a list of a hundred or a thousand or ten thousand unique product URLs in your application. If you saved these to a text file and had Siege visit every single one of those pages as quickly as possible… this might look like some sort of good stress test, huh?

Dumping unique URLs into a text file

You’ll probably start working in a rails console session, to figure out how to access the URL scheme just right.

I fired up the console, and entered:

File.open("all_campaign_urls.txt", "w") do |file|
  # this opens the file in write mode; will over-write contents of file if it exists
  Campaign.where(account_id: 4887).find_each do |campaign|
    puts "writing " + "http://localhost:3000/account/campaigns/" + campaign.to_param
    file.puts "http://localhost:3000/account/campaigns/" + campaign.to_param
  end
end

Benchmarking a page protected by a login with Apache Benchmark

I’ve been slowly working through The Complete Guide to Rails Performance. I’m taking the ideas and concepts from Nate’s book and working on applying the lessons to the app I work on in my day job.

I had a chance to attend Nate’s workshop in Denver a few days ago, as well; while there, we fired up our apps in production-like mode, and used wrk, a HTTP benchmarking tool, to see how many pages our app could serve in a given amount of time. (wrk docs).

A normal wrk run might look like so:

$ wrk -t12 -c400 -d30s http://127.0.0.1:8080/index.html

wrk will run for a [t]hread count of 12, with 400 [c]onnections, for a [d]uration of 30 seconds, against 127.0.0.1:8080/index.html

This is fine for testing unprotected home pages, but the app I work on requires a login before getting to anything interesting. So, I wondered

how does one use a HTTP benchmarking tool against an application with a login page?

And I found the answer.

Apache Benchmark

Apache Benchmark describes itself as [an] Apache HTTP server benchmarking tool. Similar to wrk, but a bit more feature-rich.

You can use it very similarly to wrk - give it a thread count, connection count, duration, and address, and it’ll hammer that page and serve up all sorts of good results.

The “rule of thumb” for benchmarking “protected” pages is:

Whatever page you can access locally in an incognito browser is what your benchmarking tool can hit without any special authentication.

In other words, when I visit http://localhost:3000/ locally, I get redirected to http://localhost:3000/users/sign_in. This is fine for apache bench, if I want to test how quickly our sign-in page loads. I can run:

$ ab -t 10 http://127.0.0.1:3000/

which returns:

Deliberate Practice in Programming with Avdi Grimm and the Rake gem

I’ve had the concept of Deliberate Practice stuck in my head for a while.

I want to improve at things (all the things!) in general, but writing and reading code, specifically. Writing and reading code is germane to my primary occupation (software developer) and drives most of my effectiveness on my team.

If there are two people, one who knows more about something, and one who knows less, but the person who knows less is learning new things faster than the other person, in short order, they’ll switch places.

This concept comes from John Ousterhout, and his talk at Stanford about how the slope is more important than y-intercept.

I know how to write the code I know how to write; I don’t know how to write better code than that. So, I’m taking advantage of code written by people who are really good at what they do, and I’m modeling my code after them.

Deliberate Practice

I’m trying to apply the principles of deliberate practice to writing code.

Eric Anders coined the term Deliberate Practice, and it boils down to four components:

1. You must be motivated to attend to the task and exert effort to improve your performance.
2. The design of the task should take into account your pre-existing knowledge so that the task can be correctly understood after a brief period of instruction.
3. You should receive immediate informative feedback and knowledge of results of your performance.
4. You should repeatedly perform the same or similar tasks.

Turing Prep Chapter 4: Arrays, Hashes, and Nested Collections

Index of this series:

• Turing Prep Chapter 1: Make Mod 1 Easier Than It Otherwise Would Be
• Turing Prep Chapter 2: Your first passing tests!
• Turing Prep Chapter 3: Video walk-through of a few of the mythical creatures, like unicorn.rb, dragon.rb, and hobbit.rb
• Turing Prep Chapter 4: Arrays, Hashes, and Nested Collections (This is what you’re reading right now)
• Turing Prep appendix: Troubleshooting Errors

So far, we’ve covered some string manipulation and did a few mythical creatures.

You totally don’t have to look through the screencasts in order. I just wanted to call out array/hash manipulation on its own page.

You can also look through the whole playlist.

Array Manipulation Exercises

These are from ruby-exercises/data-types/collections/arrays.rb

Exploring source code via Griddler and Griddler-Mailgun

Proofpoint had a two-day “hack day” recently. My coworker John and I teamed up on a cool little feature. I’ll give some context in a moment, but this post isn’t about the hack day, or email - it’s about exploring source code.

Here’s the context:

In my day-to-day, I work on a simulated phishing tool; it lets our customers send simulated phishing attacks to their employees

We then gather and report data on things like:

• who opened the email
• who clicked the phishing link (or opened the attachment)
• who read the subsequent training page
• and more

One strong benefit of using this tool is our customers can send their employees very realistic, very tricky phish, and educate them how to avoid falling for those tricky phish “in the wild”.

The more realistic the phish, the higher-quality the training.

So, we wanted to set up an email inbox that our customers could forward real-life phish to, and our staff could look through all the submitted phish, preview the email, and decide to convert the real phish into a simulated phishing template available in our “phishing template library”.

Basically, if you got a really sneaky phish, you could forward it to [email protected], and we could quickly decide if we wanted to make this phishing template available to all our customers.

I’d heard this idea discussed before, always as a “nice-to-have”, but the feature ticket never got written, and we never prioritized it. John and I work closely with the Director of Support, and others, and when digging into their pain-points, they also said this would be a nice feature.

So, we decided to build it!

We still have to set up some SMTP stuff in Mailgun, and do a few other bits of configuration, but the actual rails app is functioning as expected, and can receive mail passed along from Mailgun.

I’m not writing about receiving mail in a Rails app, though - that’ll be another post. But at one point, John and I got pretty bogged down with some unexpected errors.

It wasn’t until we started exploring the source code of the gem generating the errors that we found the problem.

Had we been quicker to jump into the gem source code, we would have saved ourselves three hours, and maybe would have gotten the entire feature built and up for testing in the time-frame we had.

Recently I’ve been making it a habit to explore stack traces when I get them, because when they pop up in my terminal, it’s super easy to view the source of the problem.

So, here’s the error we were getting: