'Redis::TimeoutError: Connection timed out Error -Rails cache

We are using redis(Elasticache) as our cache store.We are experiencing crashes in our app due to redis connection timeout.We have three app servers and use phusion passenger.At the time of crashes there are about 200 active connections.

/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:55:in `rescue in _read_from_socket'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:48:in `_read_from_socket'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:41:in `gets'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/connection/ruby.rb:273:in `read'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:248:in `block in read'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:236:in `io'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:247:in `read'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:112:in `block in call'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:217:in `block (2 levels) in process'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:353:in `ensure_connected'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:207:in `block in process'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:292:in `logging'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:206:in `process'
/home/chillr/deploy/chillr-api/shared/vendor/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:112:in `call'

SlOWLOG commands 

1) 1) (integer) 6867375
2) (integer) 1486701507
3) (integer) 62008
4) 1) "eval"
   2) "for i, name in ipairs(redis.call('KEYS', 'cache:user_transaction_logs:*:9008245678')) do redis.call('DEL', name); end"
   3) "0"
 2) 1) (integer) 6867374
    2) (integer) 1486701507
    3) (integer) 61989
    4) 1) "KEYS"
       2) "cache:user_transaction_logs:*:9008245678"
 3) 1) (integer) 6867373
    2) (integer) 1486701507
    3) (integer) 61026
    4) 1) "eval"
       2) "for i, name in ipairs(redis.call('KEYS', 'cache:user_transaction_logs:*:8888662136')) do redis.call('DEL', name); end"
       3) "0"
 4) 1) (integer) 6867372
    2) (integer) 1486701507
    3) (integer) 61006
    4) 1) "KEYS"
       2) "cache:user_transaction_logs:*:8888662136"
 5) 1) (integer) 6867371
    2) (integer) 1486701507
    3) (integer) 63070
    4) 1) "eval"

We use

  • Elasticache cache.
  • t2.medium Phusion Passenger

No of connections at time of Crash 175..200

GEM redis (3.2.1) redis-rails (5.0.1)

What could be done

ruby-on-railsrubyredisamazon-elasticache


Solution 1:[1]

Monitor current resource usage for your redis server. Temporally increase your connection timeout redis connection.

Try this check list first

https://redis.io/topics/latency

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Keval Gohil

Related Questions