This is a quick overview of some of the big-picture lessons we’ve helped our customers with in understanding how to best take advantage of Redis. We’ve found ourselves giving some common advice to many developers who already know Redis, but want to get more out of it. Enjoy!
Redis data structures are simple — none of them are likely to be a perfect match for the problem you’re trying to solve. But if you pick the right initial structure for your data, Redis commands can guide you toward efficient ways to get what you need.
Here’s our standard reference table for Redis datatypes, their most common uses, and their most common misuses. We’ll have follow-up posts with more details, specific use-cases (and code), but this is a handy reference:
|Strings||Caches, counters, and clever bit-operations: strings values are extremely versatile and have many novel use-cases.||Be careful about extremely large values, the GETting or SETting of which can introduce hard-to-find failures in certain networks and Redis clients. It is easy to hurt yourself by using SETNX to build a locking system.|
|Lists||Queues, stacks and cyclical lists.||Many, many developers try to do random access on extremely large lists. Don't try to do random access on large lists.|
|Hashes||Blacklists and rate-limiting; space-efficient storage for many small values.||Just like Redis itself, one of the largest problems users encounter with Hashes is key accounting. Be sure you have a system in place to ensure you don't fill up your hashes with keys and values you don't need.|
|Sets||Folksonomies, buckets for time-series data.||Intersections and unions must be used with care - keep your sets small. With the addition of HyperLogLog, sets are no longer the best way to count uniques. Use PFADD/PFCOUNT instead!|
|Sorted Sets||Scoreboards, lexicographical searches, random-access lists||Zsets are one of the most versatile structures, and also one of the most expensive. Keep a close eye on your big O! If you're using sorted sets for autocomplete, you might investigate ZRANGEBYLEX, which handles lexicographical searching (but be careful with UTF-8!)|
|HyperLogLog||Counting unique things||Merging multiple HLLs together can be costly. Be sure to have a cap on the number of parameters you specify in a PFMERGE.|
|Pubsub||Publish/Subscribe pattern||Many developers try to use pubsub where they need a queue. Pubsub is a way to broadcast information about events, not to pass instructions to someone. Remember that if you send a message on a pubsub channel and no one is listening, it is discarded.|
Misusing a data structure’s access patterns is like misusing a dictionary: don’t be the person flipping through every page in the Oxford English Dictionary to look up a word! The time complexity of every Redis command is documented on redis.io. If you’re unfamiliar with big O, a quick intro or review might be helpful - check the sidebar.
Good system design takes more than just basic algorithmic analysis, but it’s worthwhile to brush up on the basics from time to time, and think about the total complexity of data access in your app.
This seems to cause the most consistent pain for new Redis users. Redis is single-threaded, which makes it easy to reason about but also can also cause some surprise to people only familiar with multithreaded data access. In brief, long-running operations in Redis can be much more damaging than with other databases: since Redis only does one thing at a time, long-running operations can cause system-wide backups.
Keeping data access operations very granular is the key to good performance with Redis. 100,000 O(1) operations in Redis may well be preferable to 1 O(N) operation on a 100,000-element collection, even though you’ll be paying a penalty due to network and protocol parsing.
How can you find out the pain-points in your system? Check SLOWLOG regularly (or if you’re a RedisGreen customer, just look at the “Slow Queries” tab on your dashboard). The longer a single operation runs, the more it slows down everything else your system is trying to do, so keep it short!