This is a quick overview of some of the big-picture lessons we’ve helped our customers with in understanding how to best take advantage of Redis. This week marks our ninth month of running production-ready Redis servers for our customers. In that time we’ve found ourselves giving some common advice to many developers who already know Redis, but want to get more out of it. Enjoy!
Pick the right structure
Redis data structures are simple - none of them are likely to be a perfect match for the problem you’re trying to solve. But if you pick the right initial structure for your data, Redis commands can guide you toward efficient ways to get what you need.
Here’s our standard reference table for Redis datatypes, their most common uses,
and their most common misuses. We’ll have follow-up posts with more details,
specific use-cases (and code), but this is a handy reference:
|Strings||Caches, counters, and clever bit-operations: strings values are extremely versatile and have many novel use-cases.||Be careful about extremely large values, the GETting or SETting of which can introduce hard-to-find failures in certain networks and Redis clients. It is easy to hurt yourself by using SETNX to build a locking system.|
|Lists||Queues, stacks and cyclical lists.||Many, many developers try to do random access on extremely large lists. Don’t try to do random access on large lists.|
|Hashes||Blacklists and rate-limiting; space-efficient storage for many small values.||Just like Redis itself, one of the largest problems users encounter with Hashes is key accounting. Be sure you have a system in place to ensure you don’t fill up your hashes with keys and values you don’t need.|
|Sets||Folksonomies, buckets for time-series data.||Intersections and unions must be used with care - keep your sets small.|
|Sorted Sets||Scoreboards, random-access lists||Zsets are one of the most versatile structures, and also one of the most expensive. Keep a close eye on your big O!|
|Pubsub||Publish/Subscribe pattern||Many developers try to use pubsub where they need a queue. Pubsub is a way to broadcast information about events, not to pass instructions to someone. Remember that if you send a message on a pubsub channel and no one is listening, it is discarded.|
Use Your Big O
Misusing a data structure’s access patterns is like misusing a dictionary: don’t be the person flipping through every page in the Oxford English Dictionary to look up a word! The time complexity of every Redis command is documented on redis.io. If you’re unfamiliar with big O, a quick intro or review might be helpful - check the sidebar.
Good system design takes more than just basic algorithmic analysis, but it’s worthwhile to brush up on the basics from time to time, and think about the total complexity of data access in your app.
Keep Operations Small
This seems to cause the most consistent pain for new Redis users. Redis is single-threaded, which makes it easy to reason about but also can also cause some surprise to people only familiar with multithreaded data access. In brief, long-running operations in Redis can be much more damaging than with other databases: since Redis only does one thing at a time, long-running operations can cause system-wide backups.
Keeping data access operations very granular is the key to good performance with Redis. 100,000 O(1) operations in Redis may well be preferable to 1 O(N) operation on a 100,000-element collection, even though you’ll be paying a penalty due to network and protocol parsing.
How can you find out the pain-points in your system? Check SLOWLOG regularly (or if you’re a RedisGreen customer, just look at the “Slow Queries” tab on your dashboard). The longer a single operation runs, the more it slows down everything else your system is trying to do, so keep it short!
This post is mostly theory - the big picture on how to approach Redis as a developer. Next week we’ll go over some specific problems that we’ve helped our customers with, and the solutions that you might be able to use in your own code.