Kirill Zonov (aka graffzon) blog

Fast async hit counter for your rails app

Hi all!

I hav not written anything for a long time. And now I want to show you solution to implement hit counter in your rails app. Certanly you can just increase field of your model inside your controller, but if you have high attendance of your site, it will cause perfomance problem. 

So my decision is using redis with cron. 

Let’s take a look on what we have.

Model Event with field visits; controller EventsController with show action.

First you should install redis on your machine. It’s great describet here http://redis.io/topics/quickstart Next you should check that redis installed.

Try to call redis-server inside your console. With default params it will start redis server insite that window. In another console window you can run redis-cli and play a little. Redis is really greate DB and it has really many commands which you can learn on redis website.

So, redis installed, what next. Go into your gemfile and add redis gem there. Now last is 3.0.1

gem 'redis', '~> 3.0.1'

Make bundle install. Next if you want (and you want) to connect to redis in start of your rails server you should create

config/initializers/redis.rb

file with:

$redis = Redis.new(:host => 'localhost', :port => 6379)

This will connect to redis db, started at your machine on default port. Now you can start rails console and play a little. Redis-rb commands fully identical with manual redis commands. So you can just try (don’t forget to start redis-server):

1.9.2p290 :001 > $redis.set('foo', 'bar') 

=> "OK" 

1.9.2p290 :002 > $redis.get('foo') 

=> "bar" 

1.9.2p290 :003 > $redis.hset('chunky', 'backon', 1) 

=> true 

1.9.2p290 :004 > $redis.hget('chunky', 'backon') 

=> "1" 

1.9.2p290 :005 > $redis.hincrby('chunky', 'backon', 2) 

=> 3 

1.9.2p290 :006 > $redis.hget('chunky', 'backon') 

=> "3" 

Last four commands works with redis Hashes. I said that Redis is fucking cool?

So it’s all nice, but let’s return to our business task. What we want to do:

1) User visit our event page

2) We check his session (or cookie if you want)

3) If he has appropriate key we do nothing (his visit already counted)

4) If not - set that key for him and add his visit to redis

5) We should have cron task which in some interval (f.e. 1 minute) check if redis have some event visits

6) If it is - cron should call Event’s increase method. Let’s continue. Here is what we have in EventsController:

 def show

   increase_visits_if_needed

 end

 private

 def increase_visits_if_needed

   unless session[visit_event_key]

     session[visit_event_key] = true

     $redis.hincrby("event_visits", resource.id, 1)

   end

 end

 def visit_event_key

   (resource.url + '-visited').to_sym

 end

I think, there is all clear. Redis method hincrby can increase unpresent key too.

So visit added and next we should have cron task. For that I recommend to use Whenever gem. It gives nice interface to add your ruby tasks in cron.

So add it into your gemfile and make bundle install:

 gem 'whenever' 

Next make

 wheneverize . 

inside your console. It will create

config/shedule.rb

file for you. There we can add our ruby code.

 every 1.minute do

   runner "Event.async_increase_visits"

 end 

Also you can specify some options, which you can find on https://github.com/javan/whenever/

Next check that everything works all right (in terminal):

 master ~/projects/present_work/my_project $ whenever

* * * * * /bin/bash -l -c 'cd /Users/kirillzonov/projects/present_work/my_project && script/rails runner -e production '\''Event.async_increase_visits'\'''

## [message] Above is your schedule file converted to cron syntax; your crontab file was not updated.

## [message] Run `whenever --help' for more options.

This shows you that your task interpreted successfully.

Next add that method in your Event model:

 def self.async_increase_visits

   $redis.hkeys('event_visits').each do |id|

     self.find(id).increase_visit($redis.hget('event_visits', id))

     $redis.hdel('event_visits', id)

   end

 end

 def increase_visit(visits)

   visits.to_i.times { self.increment!(:visits) }

 end

Here we go through each redis key for event visits and increase event’s visits by given number of visits. After that we clear that keys. So that’s all. We have method for async increasing your visits. You can check how it works in your rails console:

 1.9.2p290 :018 > Event.last

 => #<Event id: 29, created_at: "2012-06-07 09:42:15", updated_at: "2012-06-18 23:32:02", visits: 0> 

1.9.2p290 :019 > $redis.hincrby("event_visits", 29, 1)

 => 1

1.9.2p290 :020 > Event.async_increase_visits

 => ["29"] 

1.9.2p290 :021 > Event.last

 => #<Event id: 29, created_at: "2012-06-07 09:42:15", updated_at: "2012-06-21 10:49:04", visits: 1> 

1.9.2p290 :022 > Event.async_increase_visits

 => []

Yeah, that’s cool. Now you need just compile and run your cron task:

 whenever --update-crontab my_project 

That’s all! You have working cron task for async visits increasing. If you change some of your shedule.rb code, you should just run the same command.

If you want to specify evnironment (f.e. development or staging) you should run:

 whenever --set environment=development --update-crontab my_project 

Try and check. All should works perfect.

Last what you need - is correctly deploy. First you should add

/etc/redis.conf

file with daemonized option (in development env I use non daemonized redis server, but you can do what you want) in your production server:

 daemonize yes

 pidfile your_project_folder/shared/pids/redis.pid

 port 6379

 timeout 300

 databases 1

 dbfilename dump.rdb

 dir your_project_folders/shared/

 appendfsync everysec

 vm-enabled no

You also can make your custom conf file, just look at http://redis.io Last, what I made for my deploy.rb (capistrano):

 before :deploy do

   run "/usr/bin/redis-server /etc/redis.conf"

 end

 namespace :deploy do

   task :restart_redis do

     run "/usr/bin/redis-server /etc/redis.conf"

     run "cd #{deploy_to}/current; script/rails runner -e production 'Event.async_increase_visits'"

     run "/usr/bin/redis-cli shutdown"

     run "/usr/bin/redis-server /etc/redis.conf"

   end

 end

 after :deploy do

   run "cd #{deploy_to}/current; bundle exec whenever --update-crontab tixis"

 end

Anyway, whenever has own cap task, but it doesn’t work for me.

Here I start redis-server before deploy (if it’s already started - nothing will happen). And after deploy I update cap tasks. For redis restart I made separate cap task because we don’t needed to restart it each time.

That’s all, now You have working async hit counting!