Time to Redis!

Redis is a – take a deep breath - nosql key-value in-memory storage system, also called memcache. It is written in ANSI C and works in most POSIX operating systems. Its features are numerous: high perf, on disk persistence, replication, pub/sub and TTL (i.e. Time To Live) to name a few.

The community around Redis is very active, resulting a lot of client for many languages: Java, Node, Ruby, PHP, etc. On this post, after a brief “How to set up Redis”, I will focus on one of the most active Java API, Jedis.

First steps with Redis

To familiarize with, as first approach, you can try the excellent online interactive tutorial.

Once you are ready to go further, simply run these commands on a shell to install Redis:

wget http://redis.googlecode.com/files/redis-2.2.11.tar.gz
tar xzf redis-2.2.11.tar.gz
cd redis-2.2.11
make

Then, you can execute the Redis server with:

src/redis-server

And finally, interact with it by running the Redis client:

src/redis-cli

Throughout your tests, you will probably generate a lot of keys. Here is a useful Redis command to delete all keys, once.

flushdb

Jedis, the Java Redis API

The Jedis API is an open project hosted on Github. It supports most of Redis features, including connection handling & pooling, transactions, pipelining or publish/subscribe. Please refer to the Related links part below to get some pointers about Jedis.

Jedis contributors done things well. They created a class that enable the API to work in a multithreaded environment. Here is a quotation from their wiki page.

A single Jedis instance is not threadsafe! To avoid these problems, you should use JedisPool, which is a threadsafe pool of network connections. You can use the pool to reliably create several Jedis instances, given you return the Jedis instance to the pool when done.

You can store the pool somewhere statically, it is thread-safe.

No sooner said than done! We can write a simple RedisManager to handle our pool of jedis instances. Here is a way to do so:

import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

public class RedisManager {
	private static final RedisManager instance = new RedisManager();
	private static JedisPool pool;
	private RedisManager() {}
    public final static RedisManager getInstance() {
        return instance;
    }
	public void connect() {
		// Create and set a JedisPoolConfig
		JedisPoolConfig poolConfig = new JedisPoolConfig();
		// Maximum active connections to Redis instance
		poolConfig.setMaxActive(10);
		// Tests whether connection is dead when connection
		// retrieval method is called
		poolConfig.setTestOnBorrow(true);
		/* Some extra configuration */
		// Tests whether connection is dead when returning a
		// connection to the pool
		poolConfig.setTestOnReturn(true);
		// Number of connections to Redis that just sit there
		// and do nothing
		poolConfig.setMaxIdle(5);
		// Minimum number of idle connections to Redis
		// These can be seen as always open and ready to serve
		poolConfig.setMinIdle(1);
		// Tests whether connections are dead during idle periods
		poolConfig.setTestWhileIdle(true);
		// Maximum number of connections to test in each idle check
		poolConfig.setNumTestsPerEvictionRun(10);
		// Idle connection checking period
		poolConfig.setTimeBetweenEvictionRunsMillis(60000);
		// Create the jedisPool
		pool = new JedisPool(poolConfig, "localhost", 6379);
	}
	public void release() {
		pool.destroy();
	}
	public Jedis getJedis() {
		return pool.getResource();
	}
	public void returnJedis(Jedis jedis) {
		pool.returnResource(jedis);
	}
}

Note that JedisPool class depends on the Apache commons-pool library. You can download this lib from the main page of the project.

The token problem

This problem is about transaction. Redis offers some native atomic functions like INCR or DECR, but also a transactional mechanism, also called “check-and-set”. This is what we will use below.

The problem I had is quite simple: I have to set keys that expire (TTL). Some Threads scan asynchronously for expired keys. If a thread detects that a key is absent, it executes some http requests which is time expensive (and a blocking io). We have to avoid that two or more threads executes the same request!

This problem can be solved by using a token. This is how I made it work:

  • The thread test the presence of a key. If the key exists, it ignores it and search for another key. This test has to be atomic.
  • If the key does not exist, the thread set a provisional value for this key that does not expire: It the the token. Other threads that will ask for this key in the future will show that it is already set (or in progress).
  • After the execution, the worker thread, that get responses to requests (or not), set the final value, that expire after a fixed time.

Here is the code:

	// Get Jedis instance from pool
	Jedis jedis = redisManager.getJedis();
	jedis.watch("key:"+myKey);
	String value = jedis.get("key:"+myKey);
	if (type == null) {
		// No value for the key, we set the token atomically
		Transaction t = jedis.multi();
		Response<String> response = t.set("key:"+myKey, "Searching");
		t.exec();
		if (response != null) {
			// The key was correctly set
			// You can do stuff here...
			// And finally update the key
			jedis.setex("key:"+myKey, TIMEOUT, myValue);
		}
	}
	// Return the jedis instance to the pool
	jedis.unwatch();
	redisManager.returnJedis(jedis);

Related links

  • To learn more on how Redis handles transactions, you can read the official related topic
  • The page presenting all available client APIs
  • Some extra links about Jedis:
    • The download page, where you can get latest Jars
    • The wiki page, where concept and usage are explained
    • Some extra documentation
  1. OCTO talks ! » Le push web vu par Diffusion – Partie 2 - pingback on July 22, 2011 at 09:31
  2. More fun with Redis | Nicolas Colomer - pingback on July 27, 2011 at 10:48

Leave a Comment

Trackbacks and Pingbacks: