Wednesday, August 02, 2006

» Simple batch processing +

Ok, so I wanted a way to backup my Anime in mpeg4. Simple, just use transcode or mencoder. But what about chapter splitting and such? Well, I could always use my trusty bash history -- up-arrow, edit, return. But that is rather tedious, and oddly enough don't work when I'm sleeping. It is especially irksome if your encoding something like 12 Kingdoms, with 5 episodes per disk. So I made a simple batch process dispatcher in Ruby. All it does is take file with a list of programs and run them in sequence (with a little bit of fluff, like starting from a process number, or running only a single process). Here it is, if anyone might find it useful.

#!/usr/bin/ruby

# = batch - Simple batch file processing module
#
# == Format of batch file
#
# - A # is a comment delimiter, ignore everything on the line.
# - Anything else is command to run - no line continuations!
#
# You can basically just use a list of commands understood
# by the shell.
#
# === Example batch file:
#
# echo "Cmd 1" # this is a comment
# df -h
# # this is a comment too
# ls -lh
#
# echo "Cmd 4"
#
# Simple, eh?
#
# == Version information
#
# * ver 1.1 - slight cleanup
# * ver 1.2 - remove interactive jobs (causing some bugs),
# rename variables to be more intuitive, add comments
#
# If you really want to have interactive jobs, you could use
# something like this bit of ugliness (all on one line!):
#
# read -t 30 -p "Run command? [Y] "; if [ -z ${REPLY} ] ||
# ([ ${REPLY:0:1} != "N" ] && [ ${REPLY:0:1} != "n" ]); then
# echo "Cmd 3"; fi
#
# * ver 1.3 - minor cleanup

$KCODE='U'

class SimpleBatchFile

# Read jobs from batch file, fill jobs and jobcount instance vars
def initialize(batchfile)
unless (File.file?(batchfile))
puts("Cannot find batch file: #{batchfile}")
exit(1)
end
@file = batchfile
data = File.open(batchfile, 'rb') { |f| f.readlines() }
@jobs = []
data.length.times { |i|
item = data[i].strip()
## skip comments and empty lines
next if (item[0,1] == '#' or item == '')
## format for jobs is (command, jobnum, linenum)
@jobs.push([item, @jobs.length + 1, i + 1])
}
@jobcount = @jobs.length
end

# Execute a command in a subshell, die on errors
def exec(command, jobnum, linenum, batchfile)
puts(" Starting job: #{jobnum}\n" +
"Executing command: #{command}\n")
unless (system(command))
puts("Problem running job...\n" +
"File: #{batchfile}\n" +
" Job: #{jobnum}\n" +
"Line: #{linenum}\n" +
'...dieing!')
unless ($?.exitstatus.nil?)
exit($?.exitstatus)
else
exit(1)
end
end
end

private :exec

# List jobs to stdout
def list()
@jobcount.times { |i|
if (i == 0)
pad = ''
else
pad = "========\n"
end
puts("#{pad} Job: #{@jobs[i][1]}\n" +
" Line: #{@jobs[i][2]}\n" +
"Command: #{@jobs[i][0]}")
}
end

# Run a job or jobs
#
# * to start from a given job use n, e.g., start=3
# * or to run a range of jobs use n..n, e.g., start=3, stop=5
# * or for a single job only use n..n, e.g., start=2, stop=2
# * no args means all jobs in batch are run
def run(start=1, stop=nil)
## some bounds checking and logics
stop = @jobcount unless (stop)
start = 1 if (start < 1)
start = @jobcount if (start > @jobcount)
stop = @jobcount if (stop > @jobcount)
stop = start if (stop < start)
(start - 1).upto(stop - 1) { |i|
exec(@jobs[i][0], @jobs[i][1], @jobs[i][2], @file)
}
end

end ## class SimpleBatchFile

if (__FILE__ == $0)
## standalone script

## get batch file name from argv
bfnm = ARGV[0]
## no file given, print usage and exit
if (bfnm.nil?)
me = File.basename($0)
puts("Usage: #{me} batch_file [start_job_number]")
exit(1)
end
## instantiate class
sbf = SimpleBatchFile.new(bfnm)
## get range from argv
scmd = ARGV[1]
if (scmd.nil?)
## run all jobs
sbf.run()
elsif (['-l', '--'].include?(scmd[0,2]))
## show job list
sbf.list()
elsif (scmd.include?('..'))
## run range of jobs
scmd, ecmd = scmd.split('..')
unless (ecmd)
ecmd = 0
end
sbf.run(scmd.to_i, ecmd.to_i)
else
## run all jobs including and after
sbf.run(scmd.to_i)
end

end

This way I can do things like this (rubyconf05.batch):

wget -c http://yhrhosting.com/rubyconf/AslakHellesoy320.mov
wget -c http://yhrhosting.com/rubyconf/AustinZiegler320.mov
wget -c http://yhrhosting.com/rubyconf/BrentRoman240S.mov
wget -c http://yhrhosting.com/rubyconf/DHH320.mov
wget -c http://yhrhosting.com/rubyconf/FowlerAndWeirich320.mov
wget -c http://yhrhosting.com/rubyconf/JimWeirich320.mov
wget -c http://yhrhosting.com/rubyconf/KarlinFox320.mov
wget -c http://yhrhosting.com/rubyconf/MatzKeynote320.mov
wget -c http://yhrhosting.com/rubyconf/NathanielTalbott320.mov
wget -c http://yhrhosting.com/rubyconf/OpenSpace320.mov
wget -c http://yhrhosting.com/rubyconf/RyanDavis320.mov
wget -c http://yhrhosting.com/rubyconf/rc01-fri-morning-david_black2.mp3
wget -c http://yhrhosting.com/rubyconf/rc02-fri-morning-francis_hwang.mp3
wget -c http://yhrhosting.com/rubyconf/rc03-fri-morning-akira_tanaka.mp3
wget -c http://yhrhosting.com/rubyconf/rc045-fri-aftnoon-koichi_sasada.mp3
wget -c http://yhrhosting.com/rubyconf/rc04-fri-aftnoon-charles_nutter.mp3
wget -c http://yhrhosting.com/rubyconf/rc05-fri-aftnoon-eric_hodel.mp3
wget -c http://yhrhosting.com/rubyconf/rc06-fri-eve-matz.mp3
wget -c http://yhrhosting.com/rubyconf/rc07-sat-morning-david_black.mp3
wget -c http://yhrhosting.com/rubyconf/rc08-sat-morn-kevin_baird.mp3
wget -c http://yhrhosting.com/rubyconf/rc09-sat-morn-brent_roman.mp3
wget -c http://yhrhosting.com/rubyconf/rc10-sat-morn-austin_ziegler.mp3
wget -c http://yhrhosting.com/rubyconf/rc11-sat-aftnoon-ryan_davis.mp3
wget -c http://yhrhosting.com/rubyconf/rc12-sat-aftnoon-jim_weirich.mp3
wget -c http://yhrhosting.com/rubyconf/rc13-sat-aftnoon-karlin_fox.mp3
wget -c http://yhrhosting.com/rubyconf/rc15-sat-eve-matz.mp3
wget -c http://yhrhosting.com/rubyconf/rc16-sun-morn-david-hh.mp3
wget -c http://yhrhosting.com/rubyconf/rc17-sat-morn-nathaniel_talbott.mp3
wget -c http://yhrhosting.com/rubyconf/rc18-sun-morn-aslak_hellessoy.mp3

# batch rubyconf05.batch

Crossing the Ruby-con. :)

Of course, I could also just sh rubyconf05.batch, but then that wouldn't been very Ruby-ish, now would it? Plus I couldn't very well start where I left off if my ethernet barfed when downloading Matz' keynote presentation by doing it that way now could I? But this way I'd just batch rubyconf05.batch 8 and go on about my buisness. Or if the other files got downloaded but only Matz' speach didn't, cause the server was having trouble, then it's batch rubyconf05.batch 8..8 (run only job #8 and quit). No editing files, no looking through shell history and so on.

0 Comments:

Post a Comment

Links to this post:

Create a Link

<< Home