What is a Rate Limiter?
Rate limiters are vital to help improve the performance, scalability, and availability of your applications. But what is a rate limiter exactly, and how does a rate limiter work?
What are rate limiters?
A rate limiter is a tool that helps you control the frequency of a program's execution, or the amount of traffic that it receives.
When websites and applications receive surges in traffic, the demand becomes so great that it can result in delays, outages, and crashes. Rate limiters prevent this by only accepting requests at a rate that the website or program can handle. This helps reduce the risk of overuse and/or misuse by malicious actors.
There are many scenarios in which a rate limiter would come in handy:
- Your website or application has received an unexpected amount of attention.
- A user's machine is malfunctioning and sending too many requests.
- A distributed denial of service (DDoS) attack is targeting your website or application.
- Your system is experiencing performance issues, and you want to limit the number of requests until the issues are resolved.
How do rate limiters work?
The RateLimiter class is included in Guava, an open-source set of libraries developed by Google for the Java programming language. In the Guava implementation, the RateLimiter is responsible for distributing permits, which are authorizations to access a given resource. The primary attribute of a RateLimiter is the rate at which it allows users to request a permit.
Rate limiters have multiple use cases, and can be applied to any high-concurrency system in which scalability and availability are a concern. For example, a rate limiter could help you restrict the number of tasks executed per second, or the maximum number of bytes per second in a data stream.
The concept of a rate limiter is similar to that of a semaphore, another tool that restricts access to a given resource. However, there's one key difference: semaphores restrict access to a limited number of threads or users, while rate limiters restrict access to a defined number of requests within a given period of time. As such, the permits issued by rate limiters don't need to be released, like they do with semaphores.
Note that rate limiters aren't the best choice for every situation - for example, real-time events in which limiting requests may change the data that users receive. In this case, scaling the system by adding more infrastructure is likely a better idea.
Rate limiters in Redis
Rate limiters are also useful for databases, preventing users from sending too many queries per second. That's why rate limiters can be built with Redis, an open-source, in-memory data structure store used to build key-value databases and caches.
If you want to build a rate limiter in Redis, you'll need to use the Redis commands INCR and EXPIRE. The INCR command increments the value stored at a given key by 1, and the EXPIRE command sets a time limit after which a given key will be deleted.
The idea is that for each time period, you create a new key in the Redis database. To avoid clogging the database, the key will expire at the end of this time period and another key will be created. When a user requests access to the database (e.g. via an API), the value corresponding to that key will be incremented by 1. If the value is already at the limit, the user is denied access and told to wait.
Redisson implements the concept of a rate limiter with the RRateLimiter interface, which makes it easier than having to code your own rate limiter with INCR and EXPIRE in the base version of Redis. Below is an example of how to use RRateLimiter in Redisson. This code sets a limit of 5 permits for every 2 seconds, and then attempts to acquire 3 permits from the rate limiter:
RRateLimiter limiter = redisson.getRateLimiter("myLimiter");
// Initialization required only once.
// 5 permits per 2 seconds
limiter.trySetRate(RateType.OVERALL, 5, 2, RateIntervalUnit.SECONDS);
// acquire 3 permits, or block until they became available
limiter.acquire(3);