You’ve heard that Redis has an embedded scripting language, but haven’t given it a try yet? Here’s a tour of what you need to understand to use the power of Lua with your Redis server.
Our first Redis Lua script just returns a value without actually interacting with Redis in any meaningful way:
This is as simple as it gets. The first line sets up a local variable with our
message, and the second line returns that value from the Redis server to the
client. Save this file locally as
hello.lua and run it like so:
Running this will print “Hello, world!”. The first argument of
EVAL is the complete lua script — here we’re using the
command to read the script from a file. The second argument is the number of Redis
keys that the script will access. Our simple “Hello World” script doesn’t
access any keys, so we use
Accessing Keys and Arguments
Suppose we’re building a URL-shortener. Each time a URL comes in we want to store it and return a unique number that can be used to access the URL later.
We’ll use a Lua script to get a unique ID from Redis using
immediately store the URL in a hash that is keyed by the unique ID:
We’re accessing Redis for the first time here, using the
call()’s arguments are the commands to send to Redis: first we
HSET <key> <field> <value>. These two commands will run sequentially
— Redis won’t do anything else while this script executes, and it will run
We’re accessing two Lua tables,
ARGV. Tables are associative
arrays, and Lua’s only mechanism for structuring data. For our
purposes you can think of them as the equivalent of an array in whatever
language you’re most comfortable with, but note these two Lua-isms that trip up
folks new to the language:
Tables are one-based, that is, indexing starts at 1. So the first element in
mytable, the second is
Tables cannot hold nil values. If an operation would yield a table of
[ 1, nil, 3, 4 ], the result will instead be
[ 1 ]— the table is truncated at the first nil value.
When we invoke this script, we need to also pass along the values for the
This time when calling
EVAL, after the script we provide
2 as the number of
KEYS that will be accessed, then we list our
KEYS, and finally we provide
ARGV. Normally when we build apps with Redis Lua scripts, the Redis
client library will take care of specifying the number of keys, but we’re
showing the actual command sent to Redis here for completeness.
Just to make things clear, here’s our original script again, this time with
When writing Lua scripts for Redis, every key that is
accessed should be accessed only by the
KEYS table. The
ARGV table is used for parameter-passing — here it’s just the value of
the URL we want to store.
Conditional Logic: increx and hincrex
Our example above saves the link for our URL-shortener, but we also need to track the number of times a URL has been accessed. To do that we’ll keep a counter in a hash in Redis. When a user comes along with a link identifier, we’ll check to see if it exists, and increment our counter for it if it does:
Each time someone clicks on a shortlink, we run this script to track that the
link was shared again. We invoke the script using
EVAL and pass in
links:visits for our single key and the link identifier returned from our
previous script as the single argument.
The script would look almost the same without hashes. Here’s a script which increments a standard Redis key only if it exists:
SCRIPT LOAD and EVALSHA
Remember that when Redis is running a Lua script, it will not run anything else. The best scripts simply extend the existing Redis vocabulary of small atomic data operations with the smallest bit of logic necessary. Bugs in Lua scripts can lock up a Redis server altogether — best to keep things short and easy to debug.
Even though they’re usually quite short, we need not specify the full Lua script each time we want to run one. In a real application you’ll instead register each of your Lua scripts with Redis when your application boots (or when you deploy), then call the scripts later by their unique SHA-1 identifier.
An explicit call to
SCRIPT LOAD is usually unnecessary in a live application
EVAL implicitly loads the script that is passed to it. An application
can attempt to
EVALSHA optimistically and fall back to
EVAL only if the script
is not found.
If you’re a Ruby programmer, take a look at Shopify’s Wolverine, which simplifies the loading and storing of Lua scripts for Ruby apps. For PHP programmers, Predis supports adding Lua scripts to be called just as though they were normal Redis commands. If you use these or other tools to standardize your interaction with Lua, let me know — I’d be interested to find out what else is out there.
When to use Lua?
Redis support for Lua overlaps somewhat with
blocks, which group operations so they are executed together. So how do you
choose to use one over the other? Each operation in a
MULTI block needs to be
independent, but with Lua, later operations can
depend on the results of earlier operations. Using Lua scripts can also avoid
race conditions that can starve slow clients when
WATCH is used.
From what we’ve seen at RedisGreen, most apps that use Lua will also use MULTI/EXEC, but not vice versa. Most successful Lua scripts are tiny, and just implement a single feature that your app needs but isn’t a part of the Redis vocabulary.
Visiting the Library
The Redis Lua interpreter loads seven libraries: base, table, string, math, debug, cjson, and cmsgpack. The first several are standard libraries that allow you to do the basic operations you’d expect from any language. The last two let Redis understand JSON and MessagePack — this is an extremely useful feature, and I keep wondering why I don’t see it used more often.
Web apps with public APIs tend to have JSON lying around all over. So maybe you have a bunch of JSON blobs stored in normal Redis keys and you want to access some particular values inside of them, as though you had stored them as a hash. With Redis JSON support, that’s easy:
Here we check to see if the key exists and quickly return nil if not. Then we
get the JSON value out of Redis, parse it with
cjson.decode(), and return the
Loading this script into your Redis server lets you treat JSON values stored in Redis as though they were hashes. If your objects are reasonably small, this is actually quite fast, even though we have to parse the value on each access.
If you’re working on an internal API for a system that demands performance, you’re likely to choose MessagePack over JSON, as it’s smaller and faster. Luckily with Redis (as in most places), MessagePack is pretty much a drop-in replacement for JSON:
Doing the Numbers
Lua and Redis have different type systems, so it’s important to understand how values may change when crossing the Redis-Lua border. When a number comes from Lua back to a Redis client, it becomes an integer — any digits past the decimal point are dropped:
When you run this script, Redis will return an integer of 3 — you lose the interesting pieces of pi. Seems simple enough, but things get a bit more tricky when you start interacting with Redis in the middle of the script. An example:
The resulting value here is a
"3.2" Why? Redis doesn’t have a dedicated numeric type. When we first
SET the value, Redis saves it as a string, losing all record of the fact that Lua initially thought of the value as a float. When we pull the value out later, it’s still a string.
Values in Redis that are accessed with
SET should be thought of as strings except
when numeric operations like
DECR are run against them. These
special numeric operations will actually return integer replies (and
manipulate the stored value according to mathematical rules), but the
“type” of the value stored in Redis is still a string value.
Gotchas: A Summary
These are the most common errors that we see when working with Lua in Redis:
Tables are one-based in Lua, unlike most popular languages. The first element in the KEYS table is
KEYS, the second is
A nil value terminates a table in Lua. So
[ 1, 2, nil, 3 ]will automatically become
[1, 2]. Don’t use nil values in tables.
redis.callwill raise exception-style Lua errors, while
redis.pcallwill automatically trap any errors and return them as tables that can be inspected.
Lua numbers are converted to integers when being sent to Redis — everything past the decimal point is lost. Convert any floating point numbers to strings before returning them.
Be sure to specify all the keys you use in your Lua scripts in the
KEYStable, otherwise your scripts will probably break in future versions of Redis.
Lua scripts are just like any other operation in Redis: nothing else runs while they’re being executed. Think of scripts as a way to expand the vocabulary of the Redis server — keep them short and to-the-point.
There are lots of great resources for Lua and Redis online — here are a few I use: