Why is my app using more memory than I have available and slowing down?

Resolution

We're continuing to iterate how we monitor and report memory usage and especially swap. Memory reporting is a complicated problem for multi-tenant architectures. What you're seeing here is likely a result of "swapping", or writing RAM to disk to free up space for other memory in RAM, at the OS level. Our runtimes are based on Ubuntu, so the Linux kernel is making a lot of decisions about what to swap and when based on activity across multiple apps. For that reason you will sometimes see swapping even when you haven't used all your available RAM. As long as your total swap usage is staying well below your total active memory, you shouldn't be experiencing noticeable performance problems. Continually increasing swap usage is much more indicative of an issue, but that's not present here.

If you see spikes in other metrics (specifically, overall response time) that correlate with spikes in swap, definitely let us know. Otherwise, you will likely see some swapping happening on 1X and 2X dynos, but low levels shouldn't be cause for concern.