Ruby & Rails | Dmytro Shteflyuk's Home

Advanced Capistrano usage

Dmytro Shteflyuk — Wed, 23 Jun 2010 19:32:06 +0000

One of the most important parts of a development process is an application deployment. There are many tools developed to make this process easy and painless: from the simple inploy to a complex all-in-one chef-based solutions. My tool of choice is Capistrano, simple and incredibly flexible piece of software. Today I’m going to talk about some advanced Capistrano usage scenarios.

1. Graceful Passenger restarts

Passenger user guide contains a simple Capistrano recipe for application server restarts. It works pretty well in almost all the cases, but there is a huge problem when you use a multi-server setup: it restarts all Passengers at the same time, so all client requests will hang (or even drop) during the time needed to start your application. The simplest solution is to restart Passengers one by one with some shift in time (for example, 15 seconds — choose this value based on how long it take to get your application up and running), so at any given moment only one of your application servers will be unavailable. In this case Haproxy (you use it, don’t you?) won’t send any requests to the restarting server, and most of your users will continue their work without any troubles.

Let me show you how we could achieve this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

namespace :deploy do
desc <<-EOF
Graceful passengers restarts. By default, it restarts \
passengers on servers with a 15 interval, but \
this delay could be changed with the smart_restart_delay \
variable (in seconds). If you specify 0, the restart will be \
performed on all your servers immediately.

cap production deploy:smart_restart

Yet another way to restart passenger immediately everywhere is \
to specify NOW environment variable:

NOW=1 cap production deploy:smart_restart
EOF
task :smart_restart, :roles => :app do
delay = fetch(:smart_restart_delay, 15).to_i
delay = 0 if ENV['NOW']

if delay <= 0
logger.debug "Restarting passenger"
run "touch #{shared_path}/restart.txt"
else
logger.debug "Greaseful passengers restart with #{delay} seconds delay"
parallel(:roles => :app, :pty => true, :shell => false) do |session|
find_servers(:roles => :app).each_with_index do |server, idx|
# Calculating restart delay for this server
sleep_time = idx * delay
time_window = sleep_time > 0 ? "after #{sleep_time} seconds delay" : 'immediately'

# Restart command sleeps a given number of seconds and the touches the restart.txt file
touch_cmd = sleep_time > 0 ? "sleep #{sleep_time} && " : ''
touch_cmd << "touch #{shared_path}/restart.txt && echo [`date`] Restarted Passenger #{time_window}"
restart_cmd = "nohup sh -c '(#{touch_cmd}) &' 2>&1 >> #{current_release}/log/restart.log"

# Run restart command on a given server
session.when "server.host == '#{server.host}'", restart_cmd
end
end
end
end
end

The trickiest part is at the lines 25-26. There we use the parallel method to run all our commands in parallel, but it has a great limitation: there is no way to substitute command parts on the fly based on server where the command is going to be executed. So instead we are building a condition for each server in the :app role, and calculate time shift based on its index.

Sometimes it’s necessary to perform an immediate restart (for example, a database migration breaks old code). We use an environment variable to do this: cap production deploy:restart NOW=1

2. Generating deployment stages on the fly in multi-stage environments

In Scribd we use a single QA box for testing, with multiple configured applications on it. The only difference between corresponding deployment scripts is an application path (e.g. /var/www/apps/qa/01, /var/www/apps/qa/02, etc.) So how do we keep them DRY? First we have created a single deployment stage called qa, and deployed with cap qa deploy QAID=1. Works, but smells bad. Today’s version is much more elegant, but it took some effort to implement:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

(1..10).each do |idx|
qid = '%02d' % idx
name = "qa#{qid}"
stages << name

desc "Set the target stage to `#{name}'."
task(name) do
location = fetch(:stage_dir, "config/deploy")
set :stage, :qa
set :qa_id, qid
load "#{location}/qa"
end
end
# This is a tricky part. We need to re-define [cci]multistage:ensure[/cci] callback
# (which is simply raises an exception), so it will not be executed for our newly
# defined stages.
if callbacks[:start]
idx = callbacks[:start].index { |callback| callback.source == 'multistage:ensure' }
callbacks[:start].delete_at(idx)
on :start, 'multistage:ensure', :except => stages + ['multistage:prepare']
end

In the qa stage script we set the :deploy_to variable from :qa_id. Now we can deploy using cap qa01 deploy. I leave the implementation of cap qa deploy, which selects a free QA box and then performs deploy there, up to you (check the Hint 4: Deploy locks explaining how to prevent stealing QA boxes by overwriting deployments using a simple locks technique).

3. Campfire notifications

This is the most straightforward and easy to implement feature:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

begin
gem 'tinder', '>= 1.4.0'
require 'tinder'
rescue Gem::LoadError => e
puts "Load error: #{e}"
abort "Please update tinder, your version is out of date: 'gem install tinder -v 1.4.0'"
end

namespace :campfire do
desc "Send a message to the campfire chat room"
task :snitch do
campfire = Tinder::Campfire.new 'SUBDOMAIN', :ssl => true, :token => 'YOUR_TOKEN'
room = campfire.find_room_by_name 'YOUR ROOM'
snitch_message = fetch(:snitch_message) { ENV['MESSAGE'] || abort('Capfire snitch message is missing. Use set :snitch_message, "Your message"') }
room.speak(snitch_message)
end

desc "Send a message to the campfire chat room about the deploy start"
task :snitch_begin do
set :snitch_message, "BEGIN DEPLOY [#{stage.upcase}]: #{ENV['USER']}, #{branch}/#{real_revision[0, 7]} to #{deploy_to}"
snitch
end

desc "Send a message to the campfire chat room about the deploy end"
task :snitch_end do
set :snitch_message, "END DEPLOY [#{stage.upcase}]: #{ENV['USER']}, #{branch}/#{real_revision[0, 7]} to #{deploy_to}"
snitch
end

desc "Send a message to the campfire chat roob about the rollback"
task :snitch_rollback do
set :snitch_message, "ROLLBACK [#{stage.upcase}]: #{ENV['USER']}, #{latest_revision[0, 7]} to #{previous_revision[0, 7]} on #{deploy_to}"
snitch
end
end

#############################################################
# Hooks
#############################################################

before :deploy do
campfire.snitch_begin unless ENV['QUIET'].to_i > 0
end

after :deploy do
campfire.snitch_end unless ENV['QUIET'].to_i > 0
end

before 'deploy:rollback', 'campfire:snitch_rollback'

To deploy without notifications use cap production deploy QUIET=1 (but be careful, usually it’s not a good idea).

4. Deploy locks

Sometimes it’s useful to lock deploys to a specific stage. The most common reason is that you pushed a heavy migration to the master and want to run it yourself, before the actual deploy, or performing some production servers maintenance and want to be sure nobody will interfere with your work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

namespace :deploy do
desc "Prevent other people from deploying to this environment"
task :lock, :roles => :web do
check_lock
msg = ENV['MESSAGE'] || ENV['MSG'] ||
fetch(:lock_message, 'Default lock message. Use MSG=msg to customize it')
timestamp = Time.now.strftime("%m/%d/%Y %H:%M:%S %Z")
lock_message = "Deploys locked by #{ENV['USER']} at #{timestamp}: #{msg}"
put lock_message, "#{shared_path}/system/lock.txt", :mode => 0644
end

desc "Check if deploys are OK here or if someone has locked down deploys"
task :check_lock, :roles => :web do
# We use echo in the end to reset exit code when lock file is missing
# (without it deployment will fail on this command — not exactly what we expected)
data = capture("cat #{shared_path}/system/lock.txt 2>/dev/null;echo").to_s.strip

if data != '' and !(data =~ /^Deploys locked by #{ENV['USER']}/)
logger.info "\e[0;31;1mATTENTION:\e[0m #{data}"
if ENV['FORCE']
logger.info "\e[0;33;1mWARNING:\e[0m You have forced the deploy"
else
abort 'Deploys are locked on this machine'
end
end
end

desc "Remove the deploy lock"
task :unlock, :roles => :web do
run "rm -f #{shared_path}/system/lock.txt"
end
end

before :deploy, :roles => :web do
deploy.check_lock
end

Now use can use cap production deploy:lock MSG="Running heavy migrations".

5. Generating servers list on the fly

Another interesting and sometimes pretty useful task is to fetch the list of servers for a deploy from some external service. For example, you have an application cloud, and do not want to change your deployment script every time you add, remove, or disable a node. Well, I have a good news for you: it’s easy!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

namespace :deploy do
task :set_nodes_from_remote_resource do
# Here you will fetch the list of servers from somewhere
nodes = %w(app01 app02 app03)

# Clear servers lists of :app and :db roles
roles[:app].clear
roles[:db].clear

# Fill :app role servers lists
nodes.each do |node|
parent.role :app, node
end

# First server in list is a primary node and db node (to run migrations)
primary = roles[:app].first
primary.options[:primary] = true
roles[:db].push(primary)

# Show information in log about where we are going to deploy to
nodes_to_deploy = roles[:app].servers.map do |server|
opts = server.options[:primary] ? ' (primary, db)' : ''
"#{server.host}#{opts}"
end.join(', ')

logger.info "Deploying to #{nodes_to_deploy}"
end
end

on :start, 'deploy:set_nodes_from_remote_resource'

When you run cap production deploy, something like this will be printed to your console:

1
2
3

triggering start callbacks for `deploy'
* executing `deploy:set_nodes_from_remote_resource'
** Deploying to app01 (primary, db), app02, app03

That’s all for today. Deployment automation could be a really tricky task, but with a right tool it turns out to be a pleasure. Do you have any questions, suggestions, or some other example deployment recipes? Do me a favor, put them in a comment! Also I have (surprise!) a Twitter account @kpumuk, and you simply must follow me there. No excuses!

The post Advanced Capistrano usage first appeared on Dmytro Shteflyuk's Home.

Submitting a patch to the Open Source project: composite_primary_keys

Dmytro Shteflyuk — Thu, 17 Dec 2009 10:28:12 +0000

Not so far ago I have found a weird bug in the Open Source Ruby gem called composite_primary_keys, occurred when you specify :primary_key option for has_one or has_many association. There are two ways to get it fixed: submit an issue and wait till someone will work out this problem or fix it by yourself and then pull request to get the patch merged into the core. This is a great library and I use it in almost all my project, so I decided to help the author and fix this bug by myself. Here I will show you how to do that.

I have discovered, that composite_primary_keys breaks my SQL queries when :primary_key option specified both for has_many and has_one associations.

Step 0. Reproducing the bug

First of all we need to reproduce a bug. Please note: if you know where the problem is, you can skip this step. Let’s start from a simple example:

1	rails cpk_bug && cd cpk_bug

Now we will add dependencies to the config/environment.rb:

1	config.gem 'composite_primary_keys'

and to the config/environments/test.rb:

1
2
3

config.gem 'rspec', :lib => 'spec'
config.gem 'rspec-rails', :lib => false
config.gem 'factory_girl'

Okay. Now we are ready to start. Let’s generate some migrations:

1
2
3
4
5

script/generate rspec_model document upload_request_id:integer title:string description:text
script/generate rspec_model upload_request filename:string state:integer
script/generate rspec_model copyright_request upload_request_id:integer explanation:text
rm -rf spec/fixtures
rake db:migrate && rake db:test:clone

Let’s create our factories:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Factory.define :copyright_request do |cr|
cr.explanation "This document is copyrighted by O'Reilly"
end

Factory.define :document do |d|
d.sequence(:title) { |n| "Document #{n}" }
d.description { |a| "The perfect description for the document '#{a.title}'" }
d.association :upload_request
end

Factory.define :upload_request do |ur|
ur.sequence(:filename) { |n| "file#{'%03d' % n}.pdf" }
ur.state 0
end

And our spec which should fail:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

require 'spec/spec_helper'

describe Document do
context 'when has copyright requests' do
before :each do
# Fake upload request used to desynchronise document and
# upload request IDs
Factory(:upload_request)
end

it 'should not have any copyright requests when just created' do
@document = Factory(:document)
@document.copyright_requests.should == []
end

it 'should return a list of copyright requests from #copyright_requests' do
@document = Factory(:document)
@request1 = Factory(:copyright_request, :upload_request => @document.upload_request)

@document.copyright_requests.should == [@request1]
end
end
end

Ok, it fails because we haven’t defined our associations on models. Let’s do it:

1
2
3
4
5
6
7
8
9
10
11
12

class Document < ActiveRecord::Base
belongs_to :upload_request

has_many :copyright_requests, :primary_key => :upload_request_id, :foreign_key => :upload_request_id
end

class UploadRequest < ActiveRecord::Base
end

class CopyrightRequest < ActiveRecord::Base
belongs_to :upload_request
end

Woohoo! Specs are failing with an error we’re trying to reproduce:

1
2
3
4
5
6
7
8
9
10
11
12

~/cpk_bug$ spec spec
..F.

1)
'Document when has copyright requests should return a list of copyright requests from #copyright_requests' FAILED
expected: [#],
got: [] (using ==)
/Users/kpumuk/cpk_bug/spec/models/document_spec.rb:20:

Finished in 0.155708 seconds

4 examples, 1 failure

Step 1. Setting up an environment for composite_primary_keys gem

In general to submit a patch to the Open Source project hosted by GitHub you have to perform the following steps: fork the repository on GitHub, write tests which fail, write a patch, ensure it works, push your changes to your fork repository, and submit a pull request. Let’s do just that!

To fork the repository I will use a perfect github gem by Dr Nic (BTW, he is the author of composite_primary_keys!)

1
2
3
4

sudo gem install github
gh clone drnic/composite_primary_keys
cd composite_primary_keys
gh fork

Now let’s configure our test environment and run tests:

1
2
3

rake local:setup
rake mysql:build_databases
rake test_mysql

You should see something like this:

1
2
3
4
5
6

Using native MySQL
Started
...................................................................................
Finished in 0.487679 seconds.

83 tests, 262 assertions, 0 failures, 0 errors

If you have any problems, check the test/README_tests.txt file for help.

Step 2. Reproducing failing tests inside composite_primary_keys test suite

We are doing TDD, right? So before any fixes we have to write a failing test first. Gem we’re hacking has a powerful test suite with many database tables created, so all we need is just to add associations to one of models, which will cover our issue.

First, add this to the test/fixtures/membership.rb model:

1 2	has_many :readings, :primary_key => :user_id, :foreign_key => :user_id has_one :reading, :primary_key => :user_id, :foreign_key => :user_id, :order => 'id DESC'

And this tests set goes to the test/test_associations.rb:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

def test_has_many_with_primary_key
@membership = Membership.find([1, 1])

assert_equal 2, @membership.readings.size
end

def test_has_one_with_primary_key
@membership = Membership.find([1, 1])

assert_equal 2, @membership.reading.id
end

def test_joins_has_many_with_primary_key
@membership = Membership.find(:first, :joins => :readings, :conditions => { :readings => { :id => 1 } })

assert_equal [1, 1], @membership.id
end

def test_joins_has_one_with_primary_key
@membership = Membership.find(:first, :joins => :reading, :conditions => { :readings => { :id => 2 } })

assert_equal [1, 1], @membership.id
end

Now rake test_mysql produces following error (there are 4 of them, I will show only the first one):

1
2
3
4

1) Error:
test_has_many_with_primary_key(TestAssociations):
ActiveRecord::StatementInvalid: Mysql::Error: Operand should contain 1 column(s): SELECT * FROM `readings` WHERE (`readings`.`user_id` = 1,1)
...

Well, that are the errors we are working on. Time to fix them!

Step 3. Fixing the bug

I will not explain how I fixed that, you can check my commit for details. Here is the diff:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

diff --git a/lib/composite_primary_keys/associations.rb b/lib/composite_primary_keys/associations.rb
index 6b63664..9a9e173 100644
--- a/lib/composite_primary_keys/associations.rb
+++ b/lib/composite_primary_keys/associations.rb
@@ -180,11 +180,12 @@ module ActiveRecord::Associations::ClassMethods
raise AssociationNotSupported, "Polymorphic joins not supported for composite keys"
else
foreign_key = options[:foreign_key] || reflection.active_record.name.foreign_key
+ primary_key = options[:primary_key] || parent.primary_key
" LEFT OUTER JOIN %s ON %s " % [
table_name_and_alias,
composite_join_clause(
full_keys(aliased_table_name, foreign_key),
- full_keys(parent.aliased_table_name, parent.primary_key)),
+ full_keys(parent.aliased_table_name, primary_key)),
]
end
when :belongs_to
@@ -338,7 +339,7 @@ module ActiveRecord::Associations
@finder_sql << " AND (#{conditions})" if conditions

else
- @finder_sql = full_columns_equals(@reflection.klass.table_name, @reflection.primary_key_name, @owner.quoted_id)
+ @finder_sql = full_columns_equals(@reflection.klass.table_name, @reflection.primary_key_name, owner_quoted_id)
@finder_sql << " AND (#{conditions})" if conditions
end

@@ -386,7 +387,7 @@ module ActiveRecord::Associations
"#{@reflection.klass.quoted_table_name}.#{@reflection.options[:as]}_id = #{@owner.quoted_id} AND " +
"#{@reflection.klass.quoted_table_name}.#{@reflection.options[:as]}_type = #{@owner.class.quote_value(@owner.class.base_class.name.to_s)}"
else
- @finder_sql = full_columns_equals(@reflection.klass.table_name, @reflection.primary_key_name, @owner.quoted_id)
+ @finder_sql = full_columns_equals(@reflection.klass.table_name, @reflection.primary_key_name, owner_quoted_id)
end

@finder_sql << " AND (#{conditions})" if conditions

Run tests to get the following output:

1
2
3
4
5
6
7

~/cpk_bug/composite_primary_keys (master)$ rake test_mysql
Using native MySQL
Started
.......................................................................................
Finished in 0.511129 seconds.

87 tests, 266 assertions, 0 failures, 0 errors

We are done for now!

Step 4. Committing changes and pulling request

It’s time to commit our changes now:

1
2
3
4
5
6
7
8
9
10
11
12
13

~/cpk_bug/composite_primary_keys (master)$ git add .
~/cpk_bug/composite_primary_keys (master)$ git commit -m 'Fixed several bugs in has_one and has_many associations when :primary_key specified'
[master 3e29891] Fixed several bugs in has_one and has_many associations when :primary_key specified
3 files changed, 31 insertions(+), 3 deletions(-)
~/cpk_bug/composite_primary_keys (master)$ git push
Counting objects: 16, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 2.14 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To git@github.com:kpumuk/composite_primary_keys.git
050d832..3e29891 HEAD -> master
~/cpk_bug/composite_primary_keys (master)$ gh home

Okay, the next step is a pull request. Last command opened a browser window with your fork. Navigate to the latest commit and press the “Pull Request” button (at the time of writing this article gh pull-request didn’t worked, and you can try to fix by yourself to understand the workflow):

That’s all, we have contributed to the community! Today Darrin Holst merged my commit into the core, and you can find it here. Not all things went smooth (I forgot to add a fixture, so tests were failing on first run), but he helped me a lot to get it working. That’s how Open Source works: we help each other to develop high quality software.

Credits

First of all, thanks to Dr Nic for the great plugin, one of the best piece of functionality I can’t imagine life without. Thanks to Darrin Holst for his patience and great help in debugging tests problem, and also for merging my commit into the composite_primary_keys core. Thanks to GitHub for the great Open Source code hosting solution, which makes working on Open Source projects so exciting.

Do you have comments or suggestions? You are welcome! Also, I will be happy if you follow me in Twitter.

The post Submitting a patch to the Open Source project: composite_primary_keys first appeared on Dmytro Shteflyuk's Home.

Simplifying your Ruby on Rails code: Presenter pattern, cells plugin

Dmytro Shteflyuk — Wed, 09 Sep 2009 05:41:14 +0000

Today we will talk about code organization in Ruby on Rails projects. As everybody knows, Ruby on Rails is a conventional framework, which means you should follow framework architects’ decisions (put your controllers inside app/controllers, move all your logic into models, etc.) But there are many open questions around those conventions. In this write-up I will try to summarize my personal experience and show how I usually solve these problems.

Here is the list of questions we will talk about:

You have some logic in your view, which uses your models extensively. There are no places in other views with such logic. The classic recommendation is to move this code into a model, but after a short time your models become bloated with stupid one-off helper methods. The solution: pattern Presenter.
Your constructor contains a lot of code to retrieve some values for your views from the database or another storage. You have a lot of fragment_exist? calls to ensure no of your data is loaded when corresponding fragment is already in cache. It’s really hard to test a particular action because of it’s size. The solution: pattern Presenter.
You have a partial, used everywhere on the site. It accepts a lot of parameters to configure how rendered code should look like. The header of this partial, which initializes default values of parameters, becomes larger and larger. The solution: cells plugin.

Please note: sample application is available on GitHub.

Presenter Pattern

Okay, you have an idea when to use this patterns. Let’s look at the example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

class HomeController < ApplicationController
def show
unless fragment_exist?('home/top_videos')
@top_videos = Video.top.all(:limit => 10)
end

unless fragment_exist?('home/categories')
@categories = Category.all(:order => 'name DESC')
end

unless fragment_exist?('home/featured_videos')
@featured_videos = Video.featured.all(:limit => 5)
end

unless fragment_exist?('home/latest_videos')
@latest_videos = Video.latest.all(:limit => 5)
end
end
end

And the view:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

<h1>Home page</h1>

<div id="top_videos">
<h2>Top videos</h2>
<% cache('home/top_videos') do %>
<%= render 'videos', :videos => @top_videos, :hide_description => true %>
<% end %>
</div>

<div class="tabs">
<ul id="taxonomy">
<li><a href="#" id="categories" class="current">Categories</a></li>
</ul>
<div class="categories_panel">
<h2>Categories</h2>
<% cache('home/categories') do %>
<%= render 'categories' %>
<% end %>
</div>
</div>

<div class="box">
<div id="latest">
<h2>Latest videos</h2>
<% cache('home/latest_videos') do %>
<%= render 'videos', :videos => @latest_videos, :hide_thumbnail => true %>
<% end %>
</div>
<div id="featured">
<h2>Featured videos</h2>
<% cache('home/featured_videos') do %>
<%= render 'videos', :videos => @featured_videos, :hide_thumbnail => true %>
<% end %>
</div>
</div>

Note: this code is available in the first commit of my presenter example project.

Scary code, isn’t it? So let’s refactor it using Presenter pattern. I prefer to put presenters into a separate folder app/presenters, so first we should add it to Rails load path. Add this line to your config/environment.rb:

1
2
3

config.load_paths += %W(
#{Rails.root}/app/presenters
)

Now we are ready to write our presenter (app/presenters/home_presenters/show_presenter.rb):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

module HomePresenters
class ShowPresenter
def top_videos
@top_videos ||= Video.top.all(:limit => 10)
end

def categories
@categories ||= Category.all(:order => 'name DESC')
end

def featured_videos
@featured_videos ||= Video.featured.all(:limit => 5)
end

def latest_videos
@latest_videos ||= Video.latest.all(:limit => 5)
end
end
end

Sometimes presenters depend on parameters, so feel free to add an initialize method. It could accept particular params or whole params collection:

1
2
3

def initialize(video_id)
@video_id = video_id
end

Now let’s refactor our controller:

1
2
3
4
5

class HomeController < ApplicationController
def show
@presenter = HomePresenters::ShowPresenter.new
end
end

Whoa, that’s nice! View now is little different:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

<h1>Home page</h1>

<div id="top_videos">
<h2>Top videos</h2>
<% cache('home/top_videos') do %>
<%= render 'videos', :videos => @presenter.top_videos, :hide_description => true %>
<% end %>
</div>

<div class="tabs">
<ul id="taxonomy">
<li><a href="#" id="categories" class="current">Categories</a></li>
</ul>
<div class="categories_panel">
<h2>Categories</h2>
<% cache('home/categories') do %>
<%= render 'categories' %>
<% end %>
</div>
</div>

<div class="box">
<div id="latest">
<h2>Latest videos</h2>
<% cache('home/latest_videos') do %>
<%= render 'videos', :videos => @presenter.latest_videos, :hide_thumbnail => true %>
<% end %>
</div>
<div id="featured">
<h2>Featured videos</h2>
<% cache('home/featured_videos') do %>
<%= render 'videos', :videos => @presenter.featured_videos, :hide_thumbnail => true %>
<% end %>
</div>
</div>

Presenters testing is much easier than testing of bloated controllers:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

describe HomePresenters::ShowPresenter do
before :each do
@presenter = HomePresenters::ShowPresenter.new
end

it 'should respond to :top_videos' do
expect { @presenter.top_videos }.to_not raise_error
end

it 'should respond to :categories' do
expect { @presenter.categories }.to_not raise_error
end

it 'should respond to :featured_videos' do
expect { @presenter.featured_videos }.to_not raise_error
end

it 'should respond to :latest_videos' do
expect { @presenter.latest_videos }.to_not raise_error
end
end

Please note: this code is available in the second commit of my presenter example project.

Please note: you should not do any manipulations on models in presenters. They only decorate models with helper methods to be used inside controllers or views, nothing else. There are several articles describing a Conductor pattern as a presenter, do not repeat their mistakes. See the first link in the list below to get an idea about the differences.

Cells Plugin

Okay, now we have a clean controller. But what about views? Let’s take a look at the videos partial:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

<%
hide_thumbnail = hide_thumbnail === true;
hide_description = hide_description === true;
css_class ||= 'videos'
style ||= :div
case style.to_sym
when :section
parent_tag = 'section'
child_tag = 'div'
when :list
parent_tag = 'ul'
child_tag = 'li'
else
parent_tag = 'div'
child_tag = 'div'
end
%>

<% content_tag parent_tag, :class => css_class do %>
<% videos.each do |video| %>
<% content_tag child_tag do %>
<h3><%= h video.title %></h3>
<%= image_tag(video.thumbnail_url, :class => 'thumb') unless hide_thumbnail %>
<%= '

%s</p>' % h(video.description) unless hide_description %>
<% end %>
<% end %>
<% end %>

So, what the heck? Is this a view or a controller? Remember old PHP days, with all this spaghetti code? That is it. It’s hard to test, it looks scary, it bad. So here cells plugin comes to the stage.

First, we need to install the plugin:

1	script/plugin install git://github.com/apotonick/cells.git

Now let’s generate a cell:

1	script/generate cell Video videos

And write some code (app/cells/video.rb):

1
2
3
4
5
6
7
8
9
10
11
12
13

class VideoCell < Cell::Base
def videos
@videos = @opts[:videos]
@hide_thumbnail = @opts[:hide_thumbnail] === true;
@hide_description = @opts[:hide_description] === true;
@css_class = @opts[:css_class] || 'videos'

view = (@opts[:style] || :div).to_sym
view = :div unless [:section, :list].include?(view)

render :view => "videos_#{view}"
end
end

app/cells/video/videos_section.html.erb:

1
2
3
4
5
6
7

class="<%= @css_class %>">
<% @videos.each do |video| %>
<div>
<%= render :partial => 'video', :locals => { :video => video } %>
</div>
<% end %>
</section>

app/cells/video/videos_list.html.erb:

1
2
3
4
5
6
7

<ul class="<%= @css_class %>">
<% @videos.each do |video| %>
<li>
<%= render :partial => 'video', :locals => { :video => video } %>
</li>
<% end %>
</ul>

app/cells/video/videos_div.html.erb:

1
2
3
4
5
6
7

<div class="<%= @css_class %>">
<% @videos.each do |video| %>
<div>
<%= render :partial => 'video', :locals => { :video => video } %>
</div>
<% end %>
</div>

app/cells/video/_video.html.erb:

1
2
3

<h3><%= h video.title %></h3>
<%= image_tag(video.thumbnail_url, :class => 'thumb') unless @hide_thumbnail %>
<%= '

%s</p>' % h(video.description) unless @hide_description %>

And the view:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

<h1>Home page</h1>

<div id="top_videos">
<h2>Top videos</h2>
<% cache('home/top_videos') do %>
<%= render_cell :video, :videos, :videos => @presenter.top_videos, :hide_description => true %>
<% end %>
</div>

<div class="tabs">
<ul id="taxonomy">
<li><a href="#" id="categories" class="current">Categories</a></li>
</ul>
<div class="categories_panel">
<h2>Categories</h2>
<% cache('home/categories') do %>
<%= render 'categories' %>
<% end %>
</div>
</div>

<div class="box">
<div id="latest">
<h2>Latest videos</h2>
<% cache('home/latest_videos') do %>
<%= render_cell :video, :videos, :videos => @presenter.latest_videos, :hide_thumbnail => true %>
<% end %>
</div>
<div id="featured">
<h2>Featured videos</h2>
<% cache('home/featured_videos') do %>
<%= render_cell :video, :videos, :videos => @presenter.featured_videos, :hide_thumbnail => true %>
<% end %>
</div>
</div>

Wow! That’s pretty easy to read and modify. All the logic is in the code now, all the views are easy to read, and moreover: it’s more than easy to test now! I have a little plugin called rspec-cells, and I have committed a patch yesterday to get it working with the latest RSpec. Here is how you spec could look like:

1
2
3
4
5
6
7
8
9

describe VideoCell do
context '.videos' do
it 'should initialize :videos variable' do
videos = mock('Videos')
render_cell :videos, :videos => videos
assigns[:videos].should be(videos)
end
end
end

So it looks almost like a classic Ruby on Rails controller spec. I hope to review the code in nearest feature and will send a pull request to the cells plugin author. Of course, if you
found a bug, feel free to contact me.

Please note: this code is available in the third commit of my presenter example project.

Creating a simple but powerful profiler for Ruby on Rails

Dmytro Shteflyuk — Wed, 26 Aug 2009 21:23:15 +0000

You are developing a large Web application. Controllers are full of complex data retrieving logic, views contain tons of blocks, partials, loops. One day you will receive an email with user complaints about some of your pages slowness. There are many profiling tools, some of them are easy (ruby-prof), others are large and complex (newrelic), but regardless of this it’s really hard to find the particular place where you have a real bottleneck. So we created really simple, but über-useful tool for ruby code profiling.

First of all, we need to decide what features we need from this tool. Don’t know about you, but all I need is to measure execution time of particular ruby code block. Here is what I mean:

1
2
3
4
5
6
7
8
9
10
11
12
13

[home#index] debug: Logged in user home page
[home#index] progress: 0.7002 s [find top videos]
[home#index] progress: 0.0452 s [build categories list]
[home#index] progress: 0.0019 s [build tag cloud]
[home#index] progress: 0.0032 s [find featured videos]
[home#index] progress: 0.0324 s [find latest videos]
[home#index] debug: VIEW STARTED
[home#index] progress: 0.0649 s [top videos render]
[home#index] progress: 0.0014 s [categories render]
[home#index] progress: 2.5887 s [tag cloud render]
[home#index] progress: 0.0488 s [latest videos render]
[home#index] progress: 0.1053 s [featured video render]
[home#index] results: 3.592 seconds

So what do we see from this output? There are two slow blocks: top videos retrieving and tag cloud rendering. Now we just know what to do to make this page faster.

Let’s write the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

module EasyProfiler
class Profile
@@profile_results = {}

cattr_accessor :enable_profiling
@@enable_profiling = false

cattr_accessor :print_limit
@@print_limit = 0.01

def self.start(name, options = {})
options[:enabled] ||= @@enable_profiling
options[:limit] ||= @@print_limit
return NoProfileInstance.new unless options[:enabled]

if @@profile_results[name]
puts "EasyProfiler::Profile.start() collision! '#{name}' is already started!"
return NoProfileInstance.new
end

@@profile_results[name] = ProfileInstance.new(name, options)
end

def self.stop(name, options = {})
options[:enabled] ||= @@enable_profiling
options[:limit] ||= @@print_limit
return unless options[:enabled]

unless @@profile_results[name]
puts "EasyProfiler::Profile.stop() error! '#{name}' is not started yet!"
return false
end

total = @@profile_results[name].total

if total > options[:limit]
@@profile_results[name].buffer_checkpoint("results: %0.4f seconds" % total)
@@profile_results[name].dump_results
end

@@profile_results.delete(name)
end
end

class ProfileInstance
def initialize(name, options = {})
@name = name
@start = @progress = Time.now.to_f
@buffer = []
end

def progress(message)
progress = (now = Time.now.to_f) - @progress
@progress = now
buffer_checkpoint("progress: %0.4f seconds [#{message}]" % progress)
end

def debug(message)
@progress = Time.now.to_f
buffer_checkpoint("debug: #{message}")
end

def total
Time.now.to_f - @start
end

def buffer_checkpoint(message)
@buffer << message
end

def dump_results
profile_logger.info("[#{@name}] Benchmark results:")
@buffer.each do |message|
profile_logger.info("[#{@name}] #{message}")
end
end

def profile_logger
root = Object.const_defined?(:RAILS_ROOT) ? "#{RAILS_ROOT}/log" : File.dirname(__FILE__)
@profile_logger ||= Logger.new(root + '/profile.log')
end
end

class NoProfileInstance
def progress(message)
end

def debug(message)
end
end
end

We have defined two class attributes: EasyProfiler::Profile.enable_profiling (to be able to disable or enable profiler globally) and EasyProfiler::Profile.print_limit (to filter out from log code blocks that are fast enough).

Then we defined two methods, which accept name of profile session (for example, “home#index”), and hash of options. Possible options are :enabled (to enable profiling of particular block) and :limit (limit in seconds to filter out fast code fragments).

Method start returns an instance of profiler, which will be used to print check points. It contains two useful methods: debug (to display custom message) and progress (to display a message along with time spent since last checkpoint). Both methods define a new checkpoint.

To simplify usage, let’s create a helper:

1
2
3
4
5
6
7

module Kernel
def easy_profiler(name, options = {})
yield EasyProfiler::Profile.start(name, options)
ensure
EasyProfiler::Profile.stop(name, options)
end
end

And now example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

class HomeController < ApplicationController
def index
easy_profiler('home#index', :enabled => profile_request?, :limit => 2) do |p|
p.progress 'logged in user home page'

@top_videos = Video.top(:limit => 10)
p.progress 'find top videos'

@categories = Category.all(:order => 'name DESC')
p.progress 'build categories list'

@tag_cloud = Tag.tag_cloud(:limit => 200)
p.progress 'build tag cloud'

@featured_videos = Video.featured(limit => 5)
p.progress 'find featured videos'

@latest_videos = Video.latest(:limit => 5)
p.progress 'find latest videos'

@profiler = p
p.debug 'VIEW STARTED'
end
end

private

# Method returns +true+ if current request should ouput profiling information
def profile_request?
params['_with_profiling'] == 'yes'
end
end

and view:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

<div id="top_videos">
<%= render :partial => 'top_videos' %>
<% @profiler.progress 'top videos render' %>
</div>

<div class="tabs">
<ul id="taxonomy">
<li><a href="#" id="categories" class="current">Categories</a></li>
<li><a href="#" id="tags">Tags</a></li>
</ul>
<div class="categories_panel">
<%= render :partial => 'categories' %>
<% @profiler.progress 'categories render' %>
</div>
<div class="categories_panel hidden">
<%= render :partial => 'tag_cloud' %>
<% @profiler.progress 'tag cloud render' %>
</div>
</div>

<div class="box">
<div id="latest">
<%= render :partial => 'videos', :videos => @latest_videos %>
<% @profiler.progress 'latest videos render' %>
</div>
<div id="featured">
<%= render :partial => 'videos', :videos => @featured_videos %>
<% @profiler.progress 'featured video render' %>
</div>
</div>

As you can see from this example, profiler will be enabled only when you pass a _with_profiling parameter with value yes: http://example.com/home?_with_profiling=yes.

That’s all. If you have any question, feel free to post a comment or contact me.

Update: I have created a Rails plugin called easy-prof, which is hosted on GitHub. It’s more powerful and feature complete, so feel free to grab sources and play with it by yourself (check the RDoc documentation at rdoc.info). Do not forget to drop me a line about your feelings.

The post Creating a simple but powerful profiler for Ruby on Rails first appeared on Dmytro Shteflyuk's Home.

Memo #6: Using named routes and url_for outside the controller in Ruby on Rails

Dmytro Shteflyuk — Thu, 16 Jul 2009 12:11:50 +0000

Sometimes we need to write small console scripts which do some background processing on our models. In Scribd we are using Loops to run these tasks (check Alexey Kovyrin’s introduction post). One of our scripts supposed to generate some sort of html reports, and we need to generate links to our site there.

In Ruby on Rails we are using routes to generate any sort of links. So let’s include routing mechanism into our own script or class.

First, you need to ensure that Rails core is loaded (if you haven’t done this earlier; for example, in script/console you should not do this). I’m assuming you’re creating a script under /script folder:

1
2
3

ENV['RAILS_ENV'] ||= 'production'
require File.dirname(__FILE__) + '/../config/boot'
require RAILS_ROOT + '/config/environment'

Now you need to include ActionController::UrlWriter module, which allows to write URLs from arbitrary places in your codebase, and configure default_url_options[:host]:

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# this is slow because all routes and resources being calculated now
include ActionController::UrlWriter
default_url_options[:host] = 'www.example.com'

# map.connect ':controller/:action/:id'
url_for(:controller => 'folders', :action => 'show', :id => Folder.first)
# => "http://www.example.com/folders/2"

# map.resources :folders
folders_url
# => "http://www.example.com/folders"
folder_url(Folder.first)
# => "http://www.example.com/folders/2"
edit_folder_url(Folder.first)
# => "http://www.example.com/folders/2/edit"

# you can use relative paths too
folders_path
# => "/folders"

Easy and helpful technique. Enjoy!

The post Memo #6: Using named routes and url_for outside the controller in Ruby on Rails first appeared on Dmytro Shteflyuk's Home.