Long-Running Tasks In Rails

Without Much Effort

Andy Stewart
April 2008
Not me
Still not me
Why?
User starts
something
lengthy
Delayed
task
Regular
task
Example
Synchronous link to Campaign Monitor
10
BackgroundJob (BJ) Spawn
Sparrow
Delayed Job (DJ)
AM4R
Conveyor
BackgrounDRb
Beanstalkd &
async-observer
Workling
BackgroundFu
Factors
 Durability
Progress reporting
Scheduling
Serial vs. Parallel
Process management
Execution environment
Error handling
Learning curve
Installation burden
Constraints on task code
Durability
 Durable Hopeful

BackgrounDRb
Background Job
Beanstalk
BackgroundFu
Spawn
Delayed Job
Sparrow
Workling
Workling
0% 50% 100%
 Tool Method
Background Job Process’s stats
BackgroundFu Incremental
BackgrounDRb Ask workers
Workling DIY “return store”
Scheduling
 BackgrounDRb

cron + {rake, script/runner,

whatever}
Delaying
(on purpose)

Delayed Job
Beanstalk
 Process
Management
 No Problem Hassle

Background Job
Everything else
Spawn
Examples
 BackgrounDRb
# Lots of configuration
...

# Your worker code

class BillingWorker < BackgrounDRb::MetaWorker
set_worker_name :billing_worker

# Called when worker is loaded for the first time

def create(args = nil)
end

# The lengthy task.

def charge_customer(customer_id = nil)
# ... do stuff ...
end
end

# Your invocation code

class CustomersController < ApplicationController
def upgrade_account
# ...
MiddleMan.worker(:billing_worker).charge_customer(@customer.id)
end
end
 Beanstalk
# Some configuration
...

# Comments controller
def create
@comment = Comment.new(params[:comment])
if @comment.save
BEANSTALK.yput({:type => "comment", :id => @comment.id}) rescue nil
# Then redirect and return
end
end

# Worker - Rake task

loop do
job = BEANSTALK.reserve
job_hash = job.ybody # ybody deserializes the job
case job_hash[:type]
when "comment"
if Comment.check_for_spam(job_hash[:id])
job.delete
else
job.bury
end
else
puts "Don't know what type of job this is: #{job_hash.inspect}"
end
end

Source: nubyonrails.com
 Async Observer
# Some configuration
...

# Start some workers

$ ./vendor/plugins/async_observer/bin/worker

# Your code
class Person < ActiveRecord::Base
async_after_create do |person|
SiteStats.increment_members()
end

def befriend(other_person)
Friendship.async_send(:create, self, other_person)
end
end
 Delayed Job
# Small config - db migration

# A job is a Ruby object with #perform

class NewsletterJob < Struct.new(:text, :emails)
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end

# Stick in queue
Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...',
Customers.find(:all).collect(&:email))

# Delayed job
BatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)

# Running tasks
$ rake jobs:work <CTRL-C to cancel>
 Background Job
# Command line

$ Bj.submit "./jobs/background_job_to_run"

$ Bj.submit "./script/runner ./jobs/background_job_to_run"

# Within Rails

def upload_to_s3
Bj.submit "./script/runner ./jobs/s3_uploader.rb #{self.id}"
end

Source: slackworks.com
1
 Background Job
Two-line installation
Zero configuration
No job-code constraints
Automatic* process management
Durable, prioritised, taggable jobs
Comprehensive job results
Clustering
* Manual if you want
Feed my spaniel
Buy my PeepCode PDF
 Photo Credits:
1. BBC
2. Forever in Song (CD cover)
3. Me
4. Me

My Photos’ Licence:
Creative Commons Attribution-Noncommercial-No Derivative Works 2.0
UK: England & Wales Licence

Text’s Licence:
Creative Commons Attribution-Share Alike 2.0 UK: England & Wales Licence