Fixing a full disk on WebPageTest

In a previous post I described how I determined that my private WebPageTest server hosted on Amazon EC2 had a full disk. Here I walk you through what I did to create more disk space so I could run web page tests again.

Disk Space on Amazon EC2

I was using a m3.medium instance type which only has 15GB volume attached (though only 8 GB usable)

Since we’re on Amazon’s elastic platform, it should be easy to scale and add more disk space, right? After all, we’re in “the cloud” and things look cheap. I could add another 100Gb for only $10/month.

Seems like something I’ll have to consider, but let’s consider all of our options.

Enabling archiving

I thought about enabling the archiving feature of WebPageTest. You specify an archive directory (or Amazon S3 bucket) and then test will be zipped and stored elsewhere. This sounded like good solution, until I ran a quick command to zip one of the tests:

[root]# zip -r QN
[root]# du -sh *
916K QN

As you can see, it only saved 20% of each test. That’s a temporarily solution, not one that’s going to be sustainable in the long run. Let’s plan on enlarging the disk itself.

Enlarging the server’s drive

I next attempted to enlarge the existing instance’s disk space by following this tutorial: Expanding the Storage Space of a Volume

I got close to completing this, but then I ran into an error:

[root]$ sudo resize2fs /dev/xvda1
resize2fs 1.41.12 (17-May-2010)
The filesystem is already 2098482 blocks long. Nothing to do!

According to the documenation I was going to have to follow another huge set of steps in this tutorial: Expanding a Linux Partition

I give up, there has to be a better way.

Attaching another disk volume

What if instead of expanding and repartition the existing drive, I just add another drive to the server? If I can mount it directly to the /var/www/webpagetest/results directory then I wouldn’t even have to move the location of the test results!

Here’s the tutorial that shows you how to add another disk to your EC2 instance: Add a Volume to Your Instance

I renamed the current results directory to ‘results_old’, then once the new drive was mounted I copied everything from results_old to results. The permissions were off, so in order to make them accessible and writeable by apache you just need to change the owner:

sudo chown -R apache:apache results

And just like that, we’re back in business with a new large place to store our test results.

Still not completely solved

Even though we added more space, I noticed I’d never be able to add more than 1000GB, and with each test taking up about 1 MB (I had video screenshots enabled), I’d reach my limit in a couple months (I was running a lot of tests!). I needed a more long term solution than that.

I ended up also deleting really old tests every night so that I could make room for new tests. Do I really need to keep a test around that ran 6 months ago? The site has most likely changed drammatically! Read about deleting old webpagetest results, then your server will be maintenance free!