Development | Dmytro Shteflyuk's Home

Advanced Capistrano usage

Dmytro Shteflyuk — Wed, 23 Jun 2010 19:32:06 +0000

One of the most important parts of a development process is an application deployment. There are many tools developed to make this process easy and painless: from the simple inploy to a complex all-in-one chef-based solutions. My tool of choice is Capistrano, simple and incredibly flexible piece of software. Today I’m going to talk about some advanced Capistrano usage scenarios.

1. Graceful Passenger restarts

Passenger user guide contains a simple Capistrano recipe for application server restarts. It works pretty well in almost all the cases, but there is a huge problem when you use a multi-server setup: it restarts all Passengers at the same time, so all client requests will hang (or even drop) during the time needed to start your application. The simplest solution is to restart Passengers one by one with some shift in time (for example, 15 seconds — choose this value based on how long it take to get your application up and running), so at any given moment only one of your application servers will be unavailable. In this case Haproxy (you use it, don’t you?) won’t send any requests to the restarting server, and most of your users will continue their work without any troubles.

Let me show you how we could achieve this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

namespace :deploy do
desc <<-EOF
Graceful passengers restarts. By default, it restarts \
passengers on servers with a 15 interval, but \
this delay could be changed with the smart_restart_delay \
variable (in seconds). If you specify 0, the restart will be \
performed on all your servers immediately.

cap production deploy:smart_restart

Yet another way to restart passenger immediately everywhere is \
to specify NOW environment variable:

NOW=1 cap production deploy:smart_restart
EOF
task :smart_restart, :roles => :app do
delay = fetch(:smart_restart_delay, 15).to_i
delay = 0 if ENV['NOW']

if delay <= 0
logger.debug "Restarting passenger"
run "touch #{shared_path}/restart.txt"
else
logger.debug "Greaseful passengers restart with #{delay} seconds delay"
parallel(:roles => :app, :pty => true, :shell => false) do |session|
find_servers(:roles => :app).each_with_index do |server, idx|
# Calculating restart delay for this server
sleep_time = idx * delay
time_window = sleep_time > 0 ? "after #{sleep_time} seconds delay" : 'immediately'

# Restart command sleeps a given number of seconds and the touches the restart.txt file
touch_cmd = sleep_time > 0 ? "sleep #{sleep_time} && " : ''
touch_cmd << "touch #{shared_path}/restart.txt && echo [`date`] Restarted Passenger #{time_window}"
restart_cmd = "nohup sh -c '(#{touch_cmd}) &' 2>&1 >> #{current_release}/log/restart.log"

# Run restart command on a given server
session.when "server.host == '#{server.host}'", restart_cmd
end
end
end
end
end

The trickiest part is at the lines 25-26. There we use the parallel method to run all our commands in parallel, but it has a great limitation: there is no way to substitute command parts on the fly based on server where the command is going to be executed. So instead we are building a condition for each server in the :app role, and calculate time shift based on its index.

Sometimes it’s necessary to perform an immediate restart (for example, a database migration breaks old code). We use an environment variable to do this: cap production deploy:restart NOW=1

2. Generating deployment stages on the fly in multi-stage environments

In Scribd we use a single QA box for testing, with multiple configured applications on it. The only difference between corresponding deployment scripts is an application path (e.g. /var/www/apps/qa/01, /var/www/apps/qa/02, etc.) So how do we keep them DRY? First we have created a single deployment stage called qa, and deployed with cap qa deploy QAID=1. Works, but smells bad. Today’s version is much more elegant, but it took some effort to implement:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

(1..10).each do |idx|
qid = '%02d' % idx
name = "qa#{qid}"
stages << name

desc "Set the target stage to `#{name}'."
task(name) do
location = fetch(:stage_dir, "config/deploy")
set :stage, :qa
set :qa_id, qid
load "#{location}/qa"
end
end
# This is a tricky part. We need to re-define [cci]multistage:ensure[/cci] callback
# (which is simply raises an exception), so it will not be executed for our newly
# defined stages.
if callbacks[:start]
idx = callbacks[:start].index { |callback| callback.source == 'multistage:ensure' }
callbacks[:start].delete_at(idx)
on :start, 'multistage:ensure', :except => stages + ['multistage:prepare']
end

In the qa stage script we set the :deploy_to variable from :qa_id. Now we can deploy using cap qa01 deploy. I leave the implementation of cap qa deploy, which selects a free QA box and then performs deploy there, up to you (check the Hint 4: Deploy locks explaining how to prevent stealing QA boxes by overwriting deployments using a simple locks technique).

3. Campfire notifications

This is the most straightforward and easy to implement feature:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

begin
gem 'tinder', '>= 1.4.0'
require 'tinder'
rescue Gem::LoadError => e
puts "Load error: #{e}"
abort "Please update tinder, your version is out of date: 'gem install tinder -v 1.4.0'"
end

namespace :campfire do
desc "Send a message to the campfire chat room"
task :snitch do
campfire = Tinder::Campfire.new 'SUBDOMAIN', :ssl => true, :token => 'YOUR_TOKEN'
room = campfire.find_room_by_name 'YOUR ROOM'
snitch_message = fetch(:snitch_message) { ENV['MESSAGE'] || abort('Capfire snitch message is missing. Use set :snitch_message, "Your message"') }
room.speak(snitch_message)
end

desc "Send a message to the campfire chat room about the deploy start"
task :snitch_begin do
set :snitch_message, "BEGIN DEPLOY [#{stage.upcase}]: #{ENV['USER']}, #{branch}/#{real_revision[0, 7]} to #{deploy_to}"
snitch
end

desc "Send a message to the campfire chat room about the deploy end"
task :snitch_end do
set :snitch_message, "END DEPLOY [#{stage.upcase}]: #{ENV['USER']}, #{branch}/#{real_revision[0, 7]} to #{deploy_to}"
snitch
end

desc "Send a message to the campfire chat roob about the rollback"
task :snitch_rollback do
set :snitch_message, "ROLLBACK [#{stage.upcase}]: #{ENV['USER']}, #{latest_revision[0, 7]} to #{previous_revision[0, 7]} on #{deploy_to}"
snitch
end
end

#############################################################
# Hooks
#############################################################

before :deploy do
campfire.snitch_begin unless ENV['QUIET'].to_i > 0
end

after :deploy do
campfire.snitch_end unless ENV['QUIET'].to_i > 0
end

before 'deploy:rollback', 'campfire:snitch_rollback'

To deploy without notifications use cap production deploy QUIET=1 (but be careful, usually it’s not a good idea).

4. Deploy locks

Sometimes it’s useful to lock deploys to a specific stage. The most common reason is that you pushed a heavy migration to the master and want to run it yourself, before the actual deploy, or performing some production servers maintenance and want to be sure nobody will interfere with your work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

namespace :deploy do
desc "Prevent other people from deploying to this environment"
task :lock, :roles => :web do
check_lock
msg = ENV['MESSAGE'] || ENV['MSG'] ||
fetch(:lock_message, 'Default lock message. Use MSG=msg to customize it')
timestamp = Time.now.strftime("%m/%d/%Y %H:%M:%S %Z")
lock_message = "Deploys locked by #{ENV['USER']} at #{timestamp}: #{msg}"
put lock_message, "#{shared_path}/system/lock.txt", :mode => 0644
end

desc "Check if deploys are OK here or if someone has locked down deploys"
task :check_lock, :roles => :web do
# We use echo in the end to reset exit code when lock file is missing
# (without it deployment will fail on this command — not exactly what we expected)
data = capture("cat #{shared_path}/system/lock.txt 2>/dev/null;echo").to_s.strip

if data != '' and !(data =~ /^Deploys locked by #{ENV['USER']}/)
logger.info "\e[0;31;1mATTENTION:\e[0m #{data}"
if ENV['FORCE']
logger.info "\e[0;33;1mWARNING:\e[0m You have forced the deploy"
else
abort 'Deploys are locked on this machine'
end
end
end

desc "Remove the deploy lock"
task :unlock, :roles => :web do
run "rm -f #{shared_path}/system/lock.txt"
end
end

before :deploy, :roles => :web do
deploy.check_lock
end

Now use can use cap production deploy:lock MSG="Running heavy migrations".

5. Generating servers list on the fly

Another interesting and sometimes pretty useful task is to fetch the list of servers for a deploy from some external service. For example, you have an application cloud, and do not want to change your deployment script every time you add, remove, or disable a node. Well, I have a good news for you: it’s easy!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

namespace :deploy do
task :set_nodes_from_remote_resource do
# Here you will fetch the list of servers from somewhere
nodes = %w(app01 app02 app03)

# Clear servers lists of :app and :db roles
roles[:app].clear
roles[:db].clear

# Fill :app role servers lists
nodes.each do |node|
parent.role :app, node
end

# First server in list is a primary node and db node (to run migrations)
primary = roles[:app].first
primary.options[:primary] = true
roles[:db].push(primary)

# Show information in log about where we are going to deploy to
nodes_to_deploy = roles[:app].servers.map do |server|
opts = server.options[:primary] ? ' (primary, db)' : ''
"#{server.host}#{opts}"
end.join(', ')

logger.info "Deploying to #{nodes_to_deploy}"
end
end

on :start, 'deploy:set_nodes_from_remote_resource'

When you run cap production deploy, something like this will be printed to your console:

1
2
3

triggering start callbacks for `deploy'
* executing `deploy:set_nodes_from_remote_resource'
** Deploying to app01 (primary, db), app02, app03

That’s all for today. Deployment automation could be a really tricky task, but with a right tool it turns out to be a pleasure. Do you have any questions, suggestions, or some other example deployment recipes? Do me a favor, put them in a comment! Also I have (surprise!) a Twitter account @kpumuk, and you simply must follow me there. No excuses!

The post Advanced Capistrano usage first appeared on Dmytro Shteflyuk's Home.

Scribd open source projects

Dmytro Shteflyuk — Tue, 08 Sep 2009 02:06:05 +0000

It’s time to summarize what we have done for the Open Source community. Scribd is pretty open company, we release a lot of code into the public after a time (sometimes it is short, sometimes it is not). Here I want to mention all the code we have opensourced. Please take into account that time is moving on, so we are publishing more and more code. I will update this post periodically, so stay tuned. Follow me on Twitter to get instant updates.

Here is the list of our projects in alphabetical order:

bounces-handler — Email Bounces Processing System with Rails plugin to prevent Rails mailers from sending any messages to a blocked addresses.
db-charmer — ActiveRecord Connections Magic (slaves, multiple connections, etc).
easy-prof — Simple and easy to use Ruby code profiler, which could be used as a Rails plugin.
Fast Sessions — Sessions class for ActiveRecord sessions store created to work fast (really fast).
loops — Simple background loops framework for Ruby on Rails and Merb.
magic-enum — Method used to define ENUM-like attributes in your model (int fields actually).
rlibsphinxclient — A Ruby wrapper for pure C searchd client API library.
rscribd — Ruby client library for the Scribd API.
Rspec Cells — A library for testing applications that are using Cells in RSpec.
Scribd Desktop Uploader — A fully native Cocoa Macintosh uploader app for the Scribd.com website.

bounces-handler

Bounces-handler package is a simple set of scripts to automatically process email bounces and ISP’s feedback loops emails, maintain your mailing blacklists and a Ruby on Rails plugin to use those blacklists in your RoR applications.

This piece of software has been developed as a part of more global work on mailing quality improvement in Scribd.com, but it was one of the most critical steps after setting up reverse DNS records, DKIM and SPF.

Links: Project Home Page on GitHub | Introduction Blog Post | RDoc Documentation.

db-charmer

DbCharmer is a simple yet powerful plugin for ActiveRecord that does a few things:

Allows you to easily manage AR models’ connections (switch_connection_to method)
Allows you to switch AR models’ default connections to a separate servers/databases
Allows you to easily choose where your query should go (Model.on_db methods)
Allows you to automatically send read queries to your slaves while masters would handle all the updates.
Adds multiple databases migrations to ActiveRecord

It requires Ruby on Rails version 2.3 or later. The main purpose of this plugin is to put all the databases-related code we have been using in Scribd for a while into a single easy-to use package.

Links: Project Home Page on GitHub | Test Rails Application on GitHub | RDoc Documentation.

easy-prof

Simple and easy to use Ruby code profiler, which could be used as a Rails plugin. The main idea behind the easy-prof is creating check points and your code and measuring time needed to execute code blocks. Here is the example of easy-prof output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

[home#index] Benchmark results:
[home#index] debug: Logged in user home page
[home#index] progress: 0.7002 s [find top videos]
[home#index] progress: 0.0452 s [build categories list]
[home#index] progress: 0.0019 s [build tag cloud]
[home#index] progress: 0.0032 s [find featured videos]
[home#index] progress: 0.0324 s [find latest videos]
[home#index] debug: VIEW STARTED
[home#index] progress: 0.0649 s [top videos render]
[home#index] progress: 0.0014 s [categories render]
[home#index] progress: 2.5887 s [tag cloud render]
[home#index] progress: 0.0488 s [latest videos render]
[home#index] progress: 0.1053 s [featured video render]
[home#index] results: 3.592 s

From this output you can see what checkpoints takes longer to reach, and what code fragments are pretty fast.

Links: Project Home Page on GitHub | Introduction Blog Post | RDoc Documentation.

Fast Sessions

FastSessions is a sessions class for ActiveRecord sessions store created to work fast (really fast). It uses some techniques which are not so widely known in developers’ community and only when they cause huge problems, performance consultants are trying to help with them.

FastSessions plugin was born as a hack created for Scribd.com (large RoR-based web project), which was suffering from InnoDB auto-increment table-level locks on sessions table.

So, first of all, we removed id field from the table. Next step was to make lookups faster and we’ve used a following technique: instead of using (session_id) as a lookup key, we started using (CRC32(session_id), session_id) — two-columns key which really helps MySQL to find sessions faster because key cardinality is higher (so, mysql is able to find a record earlier w/o checking a lots of index rows). We’ve benchmarked this approach and it shows 10–15% performance gain on large sessions tables.

And last, but most powerful change we’ve tried to make was to not create database records for empty sessions and to not save sessions data back to database if this data has not been changed during current request processing. With this change we basically reduce inserts number by 50-90% (depends 0n application).

All of these changes were implemented and you can use them automatically after a simple plugin installation.

There is a fork patched by mudge for full compatibility with Ruby on Rails version 2.3 or later.

Links: Project Home Page on Google Code | Introduction Blog Post | Fork on GitHub Compatible with Rails 2.3 and Later.

loops

loops is a small and lightweight library for Ruby on Rails, Merb and other frameworks created to support simple background loops in your application which are usually used to do some background data processing on your servers (queue workers, batch tasks processors, etc).

Originally loops plugin was created to make our own loops code more organized. We used to have tens of different modules with methods that were called with script/runner and then used with nohup and other not so convenient backgrounding techniques. When you have such a number of loops/workers to run in background it becomes a nightmare to manage them on a regular basis (restarts, code upgrades, status/health checking, etc).

After a short time of writing our loops in more organized ways we were able to generalize most of the loops code so now our loops look like a classes with a single mandatory public method called run. Everything else (spawning many workers, managing them, logging, backgrounding, pid-files management, etc) is handled by the plugin itself.

Links: Project Home Page on GitHub | Introduction Blog Post | RDoc Documentation.

magic-enum

Method used to define ENUM-like attributes in your model (int fields actually). It’s easier to show what it does in code rather than to explain in plain English:

1
2
3
4
5
6
7

Statuses = {
:unknown => 0,
:draft => 1,
:published => 2,
:approved => 3
}
define_enum :status, :default => 1, :raise_on_invalid => true, :simple_accessors => true

is identical to

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

Statuses = {
:unknown => 0,
:draft => 1,
:published => 2,
:approved => 3
}
StatusesInverted = Statuses.invert

def status
StatusesInverted[self[:status].to_i] || StatusesInverted[1]
end

def status=(value)
raise ArgumentError, "Invalid value "#{value}" for :status attribute of the #{self.class} model" if
Statuses[value].nil?
self[:status] = Statuses[value]
end

def unknown?
status == :unknown
end

def draft?
status == :draft
end

def published?
status == :published
end

def approved?
status == :approved
end

This plugin was originally developed for Best Tech Videosand later was cleaned up in Scribd repository and released to the public.

Links: Project Home Page on GitHub | RDoc Documentation.

rlibsphinxclient

A Ruby wrapper for pure C searchd client API library. It works much faster than any Ruby client for Sphinx, so you can check it to ensure you application works as fast as possible.

Please note: this is *highly experimental* library so use it at your own risk.

Links: Project Home Page on GitHub | RDoc Documentation.

rscribd

Ruby client library for the Scribd API. This gem provides a simple and powerful library for the Scribd API, allowing you to write Ruby applications or Ruby on Rails websites that upload, convert, display, search, and control documents in many formats. For more information on the Scribd platform, visit the Scribd Platform Documentation page.

The main features are:

Upload your documents to Scribd’s servers and access them using the gem
Upload local files or from remote web sites
Search, tag, and organize documents
Associate documents with your users’ accounts

Links: Project Home Page on GitHub | Scribd Platform Documentation | RDoc Documentation.

Rspec Cells

This plugin allows you to test your cells easily using RSpec. Basically, it adds an example group especially for cells, with several helpers to perform cells rendering.

If you are not sure what is cells, please visit its home page.

Spec for a regular cell could look like:

1
2
3
4
5
6
7
8
9
10
11
12
13

describe VideoCell do
integrate_views

context '.videos' do
it 'should initialize :videos variable' do
params[:id] = 10
session[:user_id] = 20
opts[:opt] = 'value'
result = render_cell :videos, { :videos => [] }, :slug => 'hello'
result.should have_tag('div', :class => :videos)
end
end
end

Links: Project Home Page on GitHub | Cells Home Page | Cells Home Page on GitHub.

Scribd Desktop Uploader

A fully native Cocoa Macintosh uploader app for the Scribd.com website. Supports following features:

Upload many files at once from your desktop.
Edit titles, tags, and other metadata before uploading.
Quickly and easily manage bulk uploads, straight from your desktop.
Right-click to start uploading files directly to Scribd (Windows only).

Links: Project Home Page on GitHub | Home Page on Scribd.com.

Changelog

September 9, 2009
- Fixed a link to GitHub homepage of the rspec-cells plugin.
September 8, 2009
- Added rspec-cells plugin.

The post Scribd open source projects first appeared on Dmytro Shteflyuk's Home.

10 recommendations on using HTML5 today (aka Homo-Adminus Blog 2.0 HTML5ified)

Dmytro Shteflyuk — Mon, 20 Jul 2009 14:00:14 +0000

There was a lot of articles about HTML5 last days, so when Alexey Kovyrin asked me to help him with his new blog design I saw no other choice but using HTML5. There are a lot of new features added since HTML4, and some of them could be used today, like new elements >, >, >, >, >, etc. I think this is a nice addition to the HTML, because these elements add more sense to an unstructured markup. There is a buzzword “semantic” exists to describe this, but I don’t like buzzwords, so I would call it “sense”. So what features we could get from HTML5, that are supported by all modern browsers?

Ok, there is not so much of a difference between HTML4 and HTML5 from web-developer standpoint, just some <div> elements are replaced with more meaningful ones (I’m talking about markup now, not about new APIs. Here is what I mean:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

lang="en">
>
>Homo-Adminus Blog>
rel="stylesheet" href="style.css" type="text/css" media="screen" />

>
>
id="container">
id="page-header">
id="menu">
>
> href="#">Home>>
> href="#">Resume/CV>>
> href="#">Projects>>
> href="#">Photos>>
> href="#">Contact>>
>
>
>
id="page-content">
id="content">
>
>
> href="#">Loops plugin for rails and merb released>>
class="meta">
Posted by Scoundrel under
href="#">Development>,
href="#">My Projects>,
href="#">Networks>
>
>
>
datetime="2008-08-20">Sun 20 />Aug />2008>
>
class="text">
Article content goes here.
>
>
>
Tags:
href="#" rel="tag">github>,
href="#" rel="tag">plugin>
>
>
href="#comments">1 Comment> |
href="#">Bookmark on del.icio.us>
>
>
>
>

id="page-footer">
Copyright © 2009 href="#">Alexey Kovyrin> |
Designed by href="#">Tatiana Kovyrina>
>
>
id="sidebar">
id="categories" class="left">
>>Categories>>
>
> href="#">Admin-tips> (26)>
> href="#">Blog> (2)>
> href="#">Databases> (33)>
>
>
>
>
>
>

In a few words: the only difference is that we use new HTML elements instead of descriptive CSS class names and IDs. I’m not going to describe all new features of HTML5 here, the good introduction could be found here or here.

Here are my recommendations on using HTML5 today:

To enable new HTML5 elements in IE you should use document.createElement — see lines 6-8, explanation is here.
Ensure that new HTML5 elements are block level:

1
2
3
4
5
header, footer, section, nav, article, aside {
margin: 0px;
padding: 0px;
display: block;
}
Use > element if you need to add a block of specific content, like categories list (lines 54-61). Use good old <div> to add a markup element with no specific sense (for example, <div id="container"> on line 11).
Use > element to create any sort of navigation panes (top-level menu, sidebar, footer links). Do not use > to create sidebars — this is wrong!
> and > elements are good candidates to wrap any heading (logo, top level navigation, and almost everything that visually is under the main content) and bottom (article metadata, page footer, bottom links, and almost everything that visually is below the main content) information. You can use these elements inside other elements like >, >, or >, so use them.
New element > contains a block of information inside the main content area. It could be an article or blog post, a comment, a particular book description in bookstore, etc. Be creative and use this exciting element to fill your pages with a sense.
The most controversial element is >. Lots of articles about HTML5 in the Internet suggest to use it for the sidebar navigation, but that is absolutely wrong. Use this tag to take some information out of primary content. Good candidates are footnotes or side notes, quotes, comment author’s avatar, etc. Yes, I’ll repeat: be creative.
Almost everything you do in HTML4 is valid for HTML5: use unordered lists <ul> of list items where order does not matter, use ordered lists <ol> otherwise. Use tables (yes, <table>) for tabular data structures (and not for layout). Paragraphs, emphasizes, quotes, and much more elements should take their own place in your markup.
Use validator to ensure you are doing it right.
Read the code. The good place to start from is HTML5 Gallery.

You can start using HTML5 right now. It’s our future, and we need to know, how would it look like.

PS. Did you notice the nice code syntax highlighting in this blog? I’m using CodeColorer — the best syntax highlighting plugin for WordPress. Try it and you will stay with it forever!

The post 10 recommendations on using HTML5 today (aka Homo-Adminus Blog 2.0 HTML5ified) first appeared on Dmytro Shteflyuk's Home.

Memo #2: Useful Git tricks with remote branches

Dmytro Shteflyuk — Mon, 22 Dec 2008 22:59:15 +0000

Here in Scribd we are using Git as primary version control system. We have tons of branches created, merged and destroyed every day. Someday I hope to describe our workflow with Git, but today I want to write about some useful techniques of working with this incredible tool.

It’s implied that you know what is Git itself and how to work with it. Below you can find some advanced tricks, that should be helpful for you (at least they were helpful for me).

As you may know, to remove local branch git branch -d branch_name command is used. But if this branch was pushed to the origin repo, it will never be deleted there. To remove a remote branch use the following command:

1	git push origin :branch_name

where branch_name — the name of your branch.

When you are working in a team, everybody can create and push their own branches, which will be retrieved from the Git server during git pull or git fetch. If a branch being removed from the server, it will remain in your local repo forever. To clean these stale branches up, use this:

1	git remote prune origin

Also don’t forget to take a look at the git_remote_branch command-line tool, which makes work with remote branches as easy as it is possible.

Have any questions? Ask me and I will answer you in following Memo– posts.

The post Memo #2: Useful Git tricks with remote branches first appeared on Dmytro Shteflyuk's Home.

Sorting RSS-feed by date using XSLT

Dmytro Shteflyuk — Thu, 06 Apr 2006 06:22:15 +0000

In my previous post I’ve described how to display RSS-feed in browser using XSLT. But sometimes It’s necessary to change order of items in feed, for example sort them by date. XSLT 1.1 allows sorting by complex data types, but XSLT 1.0 does not and we need extract separate date parts.

First I want to demonstrate simplicity of sorting by date in XSLT 1.1. We need just define dc namespace:

1
2
3
4

version="1.1"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:digg="http://digg.com//docs/diggrss/"
xmlns:dc="http://purl.org/dc/elements/1.1/">

Now we are ready to sort:

1
2
3

select="item">
select="pubDate" data-type="dc:date" order="ascending"/>
>

As I said early in XSLT 1.0 we need to split date to get day, month and so on. To convert month to number I will use following XML-tree:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

>
>Jan>
>Feb>
>Mar>
>Apr>
>May>
>Jun>
>Jul>
>Aug>
>Sep>
>Oct>
>Nov>
>Dec>
>

It’s time to create sorting code:

1
2
3
4
5
6
7
8
9
10
11

name="vMonths" select="document('')/*/months:*"/>

select="item">
select="substring(substring-after(substring-after(substring-after(pubDate, ' '), ' '), ' '), 1, 4)" order="ascending"/>
select="count($vMonths/name[
.=substring(substring-after(substring-after(current()/pubDate, ' '), ' '), 1, 3)]
/preceding-sibling::name)"
data-type="number"
order="ascending"/>
select="substring(substring-after(pubDate, ' '), 1, 2)" data-type="number" order="ascending"/>
select="substring(substring-after(substring-after(substring-after(substring-after(pubDate, ' '), ' '), ' '), ' '), 1, 8)" data-type="text" order="ascending"/>
>

That’s all. Feed items will be sorted by date in ascending order.

You can view results here or download full sources here.

Books recommended

The post Sorting RSS-feed by date using XSLT first appeared on Dmytro Shteflyuk's Home.

Creating browser-friendly RSS feeds

Dmytro Shteflyuk — Wed, 05 Apr 2006 20:54:03 +0000

RSS-feeds become the most popular content format in the web todays. There are tons of RSS-readers, aggregators, desktop and online tools for viewing feeds in the world. But what about Web-browsers? Every day I make same mistake: I click “RSS-feed” link and my browser displays XML-source of the feed. Why I can’t examine feed directly in my browser?

In this article I’ll try to create feed which will be displayed pretty well both in browser and RSS-reader. I’ll use XSLT 1.0 technology to do it (because my Firefox does not support XSLT 1.1).

First we need RSS-feed. I’ve saved digg.com RSS-feed under “programming” category (you can look at this feed here). This feed contains some additional fields in the item section (see RSS 2.0 Specification), this will be useful to show how you can extend my sample.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

>
>Transparent PNG Images in IE>
>http://digg.com/programming/Transparent_PNG_Images_in_IE>
>Fellow digg user patrickweber commented w/ this URL. A
javascript fix that passes off a transparent PNG to DirectX to
render so IE can render PNG's. You can also do it with CSS via:
filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(enabled=true,
sizingMethod=image src='imgurlhere');
Wed, 5 Apr 2006 02:26:44 GMT
http://digg.com/programming/Transparent_PNG_Images_in_IE
650

kefs
/userimages/kefs/medium.jpg

kefs
programming
90

HTML presentation for RSS-feed must contain buttons for add this feed to different online-readers. Brief instructions about using feed necessary too. I like CSS-based design therefor page layout will be placed in external CSS style sheet.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

version="1.0" encoding="UTF-8"?>
version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

method="html"/>

match="/">
>
>
>

select="/rss/channel/title"/>>

rel="stylesheet" href="digg.css" type="text/css"/>
>

select="rss/channel"/>
>

>

match="channel">
>
class="topbox">
>This is a >RSS-feed> from the

select="title"/>
website. RSS feeds allow you to stay up to date with the latest news and
features you want from

select="title"/>. To subscribe to it,
you will need a News Reader or other similar device.>

class="contbox">
class="header">
class="fltl">
class="subhead">Blog for: >
>
href="#" class="item">
height="15" hspace="5" vspace="0" border="0" width="32" alt="RSS" src="rss_feed.gif" title="RSS " align="left"/>

select="title"/>
>

class="fltclear"/>
>
class="mainbox">
class="itembox">
>

select="item">>

>
>
class="subscrbox">
class="padrhsbox">
>Subscribe to this feed>
>You can subscribe to this RSS feed in a number of ways, including
the following:>
>
>Drag the orange RSS button into your News Reader>
>Drag the URL of the RSS feed into your News Reader>
>Cut and paste the URL of the RSS feed into your News Reader>
>
>One-click subscriptions>
>If you use one of the following web-based News Readers,
click on the appropriate button to subscribe to the RSS feed.
>
href="#" onClick="window.location='http://add.my.yahoo.com/rss?url=' + window.location;return false;">
height="17" width="91" vspace="3" border="0" alt="my yahoo" src="myyahoo.gif"/>
>/>
href="#" onClick="window.location='http://www.bloglines.com/sub/'+ window.location;return false;">
height="18" width="91" vspace="3" border="0" alt="bloglines" src="bloglines.gif"/>
>/>
href="#" onClick="window.location='http://www.newsgator.com/ngs/subscriber/subext.aspx?url='+ window.location;return false;">
height="17" width="91" vspace="3" border="0" alt="newsgator" src="newsgator.gif"/>
>/>
href="#" onClick="window.location='http://client.pluck.com/pluckit/prompt.aspx?GCID=C12286x053&a=' + window.location + '&t={title}';return false;">
src="pluspluck.png" vspace="3" border="0" alt="Subscribe with Pluck RSS reader"/>
>/>
href="#" onClick="window.location='http://www.rojo.com/add-subscription?resource=' + window.location;return false;">
src="add-to-rojo.gif" vspace="3" border="0" alt="Subscribe in Rojo"/>
>/>
href="#" onClick="window.location='http://fusion.google.com/add?feedurl=' + window.location;return false;">
src="add-to-google-plus.gif" vspace="3" border="0" width="104" height="17" alt="Add to Google"/>
>
>
>
>
>
>
>

match="item">
>
href="{link}">

select="title"/>
>

> —

select="pubDate"/>>

/>
class="item_desc">

select="description"/>
>

>
>
>

It’s simple to integrate our XSLT-sheet with the RSS-feed. Just add following line after the line:

1 2	version="1.0"?> title="XSL_formatting" type="text/xsl" href="digg.xsl"?>

In the beginning I made a promise to show how you can use custom digg fields. In the xsl:stylesheet we need to add digg namespace:

1
2
3

version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:digg="http://digg.com//docs/diggrss/">

Please note that URL contains two slashes. Now we are ready to use custom digg attributes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

match="item">
>
href="{link}">

select="title"/>
>

>
by >

select="digg:submitter/digg:username"/>>

—

select="pubDate"/>
(

select="digg:diggCount"/> diggs)
>

/>
class="item_desc">

select="description"/>
>

>
>

You can look results here. Full sources are available for download.

Books recommended

The post Creating browser-friendly RSS feeds first appeared on Dmytro Shteflyuk's Home.

Development | Dmytro Shteflyuk's Home

Advanced Capistrano usage

1. Graceful Passenger restarts

2. Generating deployment stages on the fly in multi-stage environments

3. Campfire notifications

4. Deploy locks

5. Generating servers list on the fly

Scribd open source projects

Table of Contents

Changelog

10 recommendations on using HTML5 today (aka Homo-Adminus Blog 2.0 HTML5ified)

Memo #2: Useful Git tricks with remote branches

Sorting RSS-feed by date using XSLT

Books recommended

Creating browser-friendly RSS feeds

Books recommended