Redis: Show database size/size for keys

So my solution to my own problem: After playing around with redis-cli a bit longer I found out that DEBUG OBJECT <key> reveals something like the serializedlength of key, which was in fact something I was looking for...

For a whole database you need to aggregate all values for KEYS * which shouldn't be too difficult with a scripting language of your choice...

The bad thing is that redis.io doesn't really have a lot of information about DEBUG OBJECT.

The solution from the comments deserves its own answer:

redis-cli --bigkeys

MEMORY USAGE key command gives you the number of bytes that a key and its value require to be stored in RAM.

The reported usage is the total of memory allocations for data and administrative overheads that a key its value require (source redis documentation)

Take a look at this project it outputs some interesting stats about keyspaces based on regexs and prefixes. It uses the DEBUG OBJECT command and scans the db, identifying groups of keys and estimating the percentage of space they're taking up.

https://github.com/snmaynard/redis-audit

Output looks like this:

Summary  

---------------------------------------------------+--------------+-------------------+---------------------------------------------------  
Key                                                | Memory Usage | Expiry Proportion | Last Access Time                                    
---------------------------------------------------+--------------+-------------------+---------------------------------------------------  
notification_3109439                               | 88.14%       | 0.0%              | 2 minutes                               
user_profile_3897016                               | 11.86%       | 99.98%            | 20 seconds  
---------------------------------------------------+--------------+-------------------+---------------------------------------------------  

Or this this one: https://github.com/sripathikrishnan/redis-rdb-tools which does a full analysis on the entire keyspace by analyzing a dump.rdb file offline. This one works well also. It can give you the avg/min/max size for the entries in your db, and will even do it based on a prefix.

You might find it very useful to sample Redis keys and group them by type. Salvatore has written a tool called redis-sampler that issues about 10000 RANDOMKEY commands followed by a TYPE on retrieved keys. In a matter of seconds, or minutes, you should get a fairly accurate view of the distribution of key types.

I've written an extension (unfortunately not anywhere open-source because it's work related), that adds a bit of introspection of key names via regexs that give you an idea of what kinds of application keys (according to whatever naming structure you're using), are stored in Redis. Combined with the more general output of redis-sampler, this should give you an extremely good idea of what's going on.

Perhaps you can do some introspection on the db file. The protocol is relatively simple (yet not well documented), so you could write a parser for it to determine which individual keys are taking up a lot of space.

New suggestions:

Have you tried using MONITOR to see what is being written, live? Perhaps you can find the issue with the data in motion.

I usually prefer the key sampling method to troubleshoot such scenarios.

redis-cli -p 6379 -n db_number --bigkeys

Eg:-

redis-cli -p 6370 -n 0 --bigkeys

Quick and dirty if you know the prefix you can sum the set of keys:

echo "keys userfeed*" | redis-cli -h 10.168.229.48 | xargs -I{} echo "debug object {}" | redis-cli -h 10.168.229.48 | awk '{print $5}' | cut -
d: -f2 | awk '{s+=$1} END {print s}'

This give the size in bytes. Remember this is "serialized length" not size in memory.

To get size in memory in bytes:

echo "keys op*" | redis-cli -h 10.168.229.48 | xargs -I{} echo "memory usage {} samples 0" | redis-cli -h 10.168.229.48 | awk '{s+=$1} END {pr
int s}'

You can use .net application https://github.com/abhiyx/RedisSizeCalculator to calculate the size of redis key,

Please feel free to give your feedback for the same

You can also check the INFO command in redis to view the memory usage

$ redis-cli
127.0.0.1:6379> INFO memory

lua one-liner

eval "local sum = 0; local matches = redis.call('KEYS', '<pattern>'); for _,key in ipairs(matches) do local val = redis.call('memory', 'usage', key); sum = sum + tonumber(val) end return sum" 0

One-liner. All redis keys and corresponding used memory.

redis-cli keys "*" | while read line; do echo -n "$line: "; redis-cli memory usage $line; done

How about

redis-cli get KEYNAME | wc -c