The poor man’s memory profiling

While working on www.mes-courses.fr, a background scheduled task that was running fine on heroku started to fail with out of memory errors. After searching a bit, I discovered that the inputs had changed, and that the memory consumption of my task was linearly correlated to the size of the inputs.

So I tried to setup an automatic test to verify that the memory consumption of my task would remain small enough for it to run on heroku. This is what I wanted to do :

• write a unit test for this
• run the task once to warm up the memory
• run the task once for some small sample input and note the peak memory usage
• run the task once for some large sample input and note the peak memory usage
• check that the memory usages are very close, whatever the size of the inputs

Everything there is quite straightforward, appart from “note the peak memory usage”. Here is what I came up with

• note the initial memory usage
• start a thread that garbage collects and notes the memory usage every 10 ms
• process the data
• tell the thread to stop
• memory usage is the difference between the maximum and initial memory usages

Here is the code in ruby, but it can be easily translated to any language (I did it for C# once)

Unfortunately in ruby, memory usage is not directly available without patching and rebuilding the interpreter, but allocated objects count is available, and it’s actually enough for our purpose.

Stay up to date with content and upcoming products

I usually write about 15 minutes worth of reading per month. I won't transfer your email. No Spam, unsubscribe whenever you want.

As a gift for subscribing, you'll receive an illustrated mini-ebook "How to start a team coding dojo"!