Use Instance Caches to Save Money: Latency == $$$

In the post Using memcache might be free, but it’s costing you money with instance pricing! Use instance caches if possible made on the Google App Engine group, Santiago Lema brings up an oldie but a goody of an idea that was once used to improve performance, but now it’s used to save money:

  • Santiago’s GAE application went from about $9 to about $177 per month.
  • Memcache is slow enough that under higher loads extra instances are created help to consolidate payday loans by the scheduler to handle the load.
  • For static or semi-static data, a way around the cost of the extra instances, is to keep a cache in the instance so requests can be served out of local memory rather than going to memcache or the database. A simple hashtable makes a good in-memory cache.
  • This solution made his app affordable again by reducing the number of instances back to 1 (sometimes 2).

Where have we seen this before?