script | Dmytro Shteflyuk's Home

Do I need to buy additional memory?

Dmytro Shteflyuk — Tue, 29 May 2007 14:07:41 +0000

Today I spent almost full day for reflection on to buy or not additional memory for my computer. As a result new script was born, and he saved my brain:

If you have questions like this — ask the script, maybe it would help you too :-)

The post Do I need to buy additional memory? first appeared on Dmytro Shteflyuk's Home.

Colorizing console Ruby-script output

Dmytro Shteflyuk — Fri, 23 Mar 2007 16:28:06 +0000

Very often I have to implement console scripts (because of my laziness, for boring processes optimization). Many of them write some information to the output, show process status or display results of work. Anyway, it’s very wearisome action to read script output, and I want to highlight most important things: errors in red, successfully finished steps in green color, etc. And it is a case when ANSI escape sequences could help. They are supported by the most terminals, including VT100 (btw, Windows NT family console does not support it, but I will back to this issue later).

For the beginning, let’s examine ANSI escape sequence structure. It starts with ESC symbol (ASCII-code 27), following by left square bracket [. There are one or more numbers separated by semicolon ; with a letter at the end appear after it.

I will not describe all possible codes, anybody who wishes could find them in Wikipedia. The sequence with m letter at the end is used to change foreground and background colors. In general situation it looks like: ESC[31m, where 31 sets red color as foreground. Here the table with codes which supported by most terminals:

Code	Effect
0	Turn off all attributes
1	Set bright mode
4	Set underline mode
5	Set blink mode
7	Exchange foreground and background colors
8	Hide text (foreground color would be the same as background)
30	Black text
31	Red text
32	Green text
33	Yellow text
34	Blue text
35	Magenta text
36	Cyan text
37	White text
39	Default text color
40	Black background
41	Red background
42	Green background
43	Yellow background
44	Blue background
45	Magenta background
46	Cyan background
47	White background
49	Default background color

As you could see from the table, you are able to set foreground and background colors separately, also you could combine them into one sequence (for example, ESC[1;33;44m – bright yellow text on blue background).

Attention: Don’t forget to turn off all attributes before exit, otherwise all following text would be displayed with your attributes.

That’s enough of the theory, let’s examine example:

1
2
3
4

# Actual work
puts "Importing categories [ e[32mDONEe[0m ]"
# Actual work
puts "Importing tags [e[31mFAILEDe[0m]"

As the result you will see something like following:

All my life I used codes just like shown in the previous example, but not so long ago I found simple helpers when delving in the RSpec sources:

1
2
3
4
5
6
7
8
9
10
11

def colorize(text, color_code)
"#{color_code}#{text}e[0m"
end

def red(text); colorize(text, "e[31m"); end
def green(text); colorize(text, "e[32m"); end

# Actual work
puts 'Importing categories [ ' + green('DONE') + ' ]'
# Actual work
puts 'Importing tags [' + red('FAILED') + ']'

It’s a good idea, and now I’m using it. And recommend it to you :-)

Now about sorrowful things. Windows XP (and as far as I remember, Windows 2000 too) does not support ANSI escape sequences. If you are love perversions, look at the Command Interpreter Ansi Support article. Others could stay here and look, how to solve problem using Ruby facilities.

You should install win32console first:

1	gem install win32console

Now add following lines at the beginning of your script (and again, I found them in RSpec):

1
2
3
4
5

begin
require 'Win32/Console/ANSI' if RUBY_PLATFORM =~ /win32/
rescue LoadError
raise 'You must gem install win32console to use color on Windows'
end

Script output will be colorized both on windows and Unix systems.

And in the end I will show full table of different codes, which you could use in your scripts:

It has been obtained using following script:

1
2
3
4
5
6
7
8
9
10
11
12

#!/usr/bin/ruby

[0, 1, 4, 5, 7].each do |attr|
puts '----------------------------------------------------------------'
puts "ESC[#{attr};Foreground;Background"
30.upto(37) do |fg|
40.upto(47) do |bg|
print "\033[#{attr};#{fg};#{bg}m #{fg};#{bg} "
end
puts "\033[0m"
end
end

Updated 06/10/2010: Replaced PLATFORM constant with the RUBY_PLATFORM (thanks to Ian Alexander Wood).

The post Colorizing console Ruby-script output first appeared on Dmytro Shteflyuk's Home.

Encoding media files in Ruby using ffmpeg/mencoder with progress tracking

Dmytro Shteflyuk — Wed, 11 Oct 2006 04:28:00 +0000

In my current project I need to encode media files from any format to several predefined. Furthermore I need to track progress and show it for the customer. I don’t want to describe wich media formats I need and what troubles with converting (maybe it will be my future posts, if anybody interested), instead I will describe common idea how to implement encoder scripts and how to track progress.

First, I need to decide how will be progress shown. I can’t find anything simpler then plain text output to standard output PROGRESS: 56, where 56 is progress in percent, and ERROR on error. Every output format will be handled by separate script (for example, to produce video for iPod it will be ipod.rb).

There are two encoding software I need to use – mencoder and ffmpeg (I don’t count additional tools like flvtool2 or something else, because they took much lower process time). It means that all I need is to find common method of executing these programs and interpret theirs output (which already contains information enough to calculate progress).

Let’s get started. First we will run mencoder:

1	mencoder input.avi -o output.avi -oac lavc -ovc lavc -lavcopts vcodec=xvid:acodec=mp3 > output.txt

This is very simple command line and I hope you will not use it in real projects :-) As you can see, output.txt contains lines ended with r:

1
2
3
4
5

--skipped--
Pos: 0.1s 3f ( 1%) 0.00fps Trem: 0min 0mb A-V:0.008 [0:0]
Pos: 0.2s 4f ( 2%) 0.00fps Trem: 0min 0mb A-V:0.012 [0:0]
Pos: 0.2s 5f ( 3%) 0.00fps Trem: 0min 0mb A-V:0.016 [0:0]
--skipped--

I will use IO.popen Ruby’s method to parse:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

class MediaFormatException < StandardError
end

def execute_mencoder(command)
progress = nil
IO.popen(command) do |pipe|
pipe.each("r") do |line|
if line =~ /Pos:[^(]*(s*(d+)%)/
p = $1.to_i
p = 100 if p > 100
if progress != p
progress = p
print "PROGRESS: #{progress}n"
$defout.flush
end
end
end
end
raise MediaFormatException if $?.exitstatus != 0
end

First I defined class MediaFormatException which will be used to raise exceptions to inform caller about errors (we will talk about it later). Then you can see method execute_mencoder, which accepts command line. Progress information will be show on standard output, and no progress status will be shown twice.

Let’s continue with ffmpeg:

1	ffmpeg -y -i input.avi -vcodec xvid -acodec mp3 -ab 96 output.avi > output.txt

Weird! It produses output information to standard error!

1	ffmpeg -y -i input.avi -vcodec xvid -acodec mp3 -ab 96 output.avi &> output.txt

As we can see, it shows current frame and time, but not percents. But in the beginning of output it produces Duration: 00:00:24.9, therefor we can calculate progress ourself:

1
2
3
4
5
6
7

--skipped--
Duration: 00:00:24.9, start: 0.000000, bitrate: 331 kb/s
--skipped--
frame= 41 q=7.0 size= 116kB time=1.6 bitrate= 579.7kbits/s
frame= 78 q=12.0 size= 189kB time=3.1 bitrate= 497.2kbits/s
frame= 115 q=13.0 size= 254kB time=4.6 bitrate= 452.3kbits/s
--skipped--

Let’s do it!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

def execute_ffmpeg(command)
progress = nil
IO.popen(command) do |pipe|
pipe.each("r") do |line|
if line =~ /Duration: (d{2}):(d{2}):(d{2}).(d{1})/
duration = (($1.to_i * 60 + $2.to_i) * 60 + $3.to_i) * 10 + $4.to_i
end
if line =~ /time=(d+).(d+)/
if not duration.nil? and duration != 0
p = ($1.to_i * 10 + $2.to_i) * 100 / duration
else
p = 0
end
p = 100 if p > 100
if progress != p
progress = p
print "PROGRESS: #{progress}n"
$defout.flush
end
end
end
end
raise MediaFormatException if $?.exitstatus != 0
end

Here we are using the same exception class MediaFormatException. We have methods now and ready to test them.

1
2
3
4
5
6
7
8
9
10

command_mencoder = "mencoder input.avi -o output.avi -oac lavc -ovc lavc -lavcopts vcodec=xvid:acodec=mp3"
command_ffmpeg = "ffmpeg -y -i input.avi -vcodec xvid -acodec mp3 -ab 96 output.avi 2>&1"

begin
execute_mencoder(command_mencoder)
execute_ffmpeg(command_ffmpeg)
rescue
print "ERRORn"
exit 1
end

Please note, that we are redirected standard error output to standard output to handle progress of ffmpeg.

Looks not so good because we need to handle exception in every script. Let’s create method, which will do this instead of us:

1
2
3
4
5
6

def safe_execute
yield
rescue
print "ERRORn"
exit 1
end

And here is example of using:

1
2
3
4
5
6
7

command_mencoder = "mencoder input.avi -o output.avi -oac lavc -ovc lavc -lavcopts vcodec=xvid:acodec=mp3"
command_ffmpeg = "ffmpeg -y -i input.avi -vcodec xvid -acodec mp3 -ab 96 output.avi 2>&1"

safe_execute do
execute_mencoder(command_mencoder)
execute_ffmpeg(command_ffmpeg)
end

It’s simple, right? Now we have plain common progress statistics on standard output and don’t need to handle different outputs from different encoders. Now we can use this script as a part of largest process, which can catch progress status and show it to user.

The post Encoding media files in Ruby using ffmpeg/mencoder with progress tracking first appeared on Dmytro Shteflyuk's Home.