Ruby as service


#1

大概就是 … Rails 本身並不是萬能的,圍繞在 Rails 外面應該要有一票輔助用的 server 才是,而這邊來寫如何把一支 pure Ruby 的程式當作 service 跑起來,並做 loop 的持續性的服務

這篇如果要串 ActiveRecord 請先看這篇 ActiveRecord in pure Ruby(+ API server),這邊只做 initialize

先大略解釋,這支程式應該叫做 video converter,裡面使用了 bundler / logger / loop / ActiveRecord / yaml 完成,一支簡單的 single thread loop(multi thread請另外實作) & 這支是demo code,我並沒實際 run 過就是了

(其實這邊是砲火展示??…嘛…我平常的code的量是這個的 10 倍以上唄)

###[ project/Gemfile ]

gem 'god'
gem 'activerecord'
gem 'mysql2'
gem 'redis'
gem 'awesome_print'

###[ project/video.god ]

workers = [ :god_video ]
workers.each do |worker|
  God.watch do |w|
    w.dir = "./"
    w.name = "#{worker}"
    w.group = 'god_workers'
    w.interval = 30.seconds
    w.start = "ruby video.rb"
    w.log = "./log/god_log.log"
    w.err_log = "./log/god_err.log"

    # restart if memory gets too high
    w.transition(:up, :restart) do |on|
      on.condition(:memory_usage) do |c|
        c.above = 800.megabytes
        c.times = 2
      end
    end

    w.start_if do |start|
      start.condition(:process_running) do |c|
        c.running = false
      end
    end
  end
end

###[ project/config.yml ]

:database:
  adapter: mysql2
  encoding: utf8
  database: project_video
  username: imroot
  password: imroot
:redis:
  :host: 127.0.0.1
  :port: 6379
  :thread_safe: true

###[ project/video.rb ]

#bundle check
require 'rubygems'
require 'bundler/setup'

#require std libs

require 'active_record'
require 'yaml'
require 'thread'
require 'open3' #ruby std lib內的 call system 神器
require 'logger'
require 'ostruct' #logger用的

#initialize

$sys_config = YAML.load_file('./config.yml')
$redis = Redis.new($sys_config[:redis])
ActiveRecord::Base.establish_connection($sys_config[:database])

# test connection

begin
  $redis.keys 'temptemp'
rescue
  Log.error.error "fail : redis : ==:#{$!}\n#{$@.join("\n")}"
  abort
end
Log.info.info "[redis init okay]"
begin
  ActiveRecord::Base.connection.execute "SELECT 1"
rescue"fail : ActiveRecord :==:#{$!}\n#{$@.join("\n")}"
  abort
end
Log.info.info "[active_record init okay]"

#require main lib

require_relative 'lib/logger.rb'
require_relative 'lib/models.rb'
require_relative 'lib/main.rb'

#start

Video.init!

###[ project/lib/logger.rb ]

class Logger
  def format_message(level, time, progname, msg)
    #\e[0m = reset color(text) , \e[0;32m = Yellow(text)
    "\e[0;32m#{level} #{time.strftime("%Y-%m-%d %H:%M:%S.%2N")}-- #{msg}\e[0m\n"
  end
end
class MultiLogger
  def initialize(*targets)
    @targets = targets
  end
  def write(*args)
    @targets.each{|t|t.write(*args)}
  end
  def close
    @targets.each(&:close)
  end
end
begin
  Log = OpenStruct.new
  Log.recoder = [:info , :error]
  Log.recoder.each do |name|
    file = File.open("log/#{name}.log" , 'a')
    file.sync = true
    if $sys_config[:env] != 'Production'
      Log.send("#{name}=" , Logger.new(MultiLogger.new(STDOUT,file)))
    else
      Log.send("#{name}=" , Logger.new(file))
    end
    Log.send(name).datetime_format = '%Y-%m-%d %H:%M:%S'
  end
rescue Exception => e ; puts "==:#{e.message}\n#{e.backtrace.join("\n  ==> ")}"
  abort("\n Can't open logger , maybe logger Permission denied \n")
end

###[ project/lib/models.rb ]

class User < ActiveRecord::Base
  has_many :video
end
class Video < ActiveRecord::Base
  STATUS = [['新增' , 0],['轉檔中' , 1],['完成' , 2]]
  belongs_to :user
end

###[ project/lib/main.rb ]

class Video
  SLEEP_TIMER = 20.0 # per run
  def self.init!
    loop do
      @@now = Time.now
      @@now_int = @@now.to_i

      go!
      
      sleep_timer = SLEEP_TIMER - (Time.now.to_i - @@now_int)
      sleep(sleep_timer) if sleep_timer > 0
    end
  rescue Exception => e ; Log.error.error "==:#{e.message}\n#{e.backtrace.join("\n")}"
    # do nothing
  end

  def self.go!
    videos = Video.where(['status = 0 OR (status = 1 AND convert_at < ?)' , @@now_int - 86400]).to_a
    unless videos.empty?
      video_id_set = videos.map{|i|i.id}.join(',')
      Log.info.info "start convert video : #{video_id_set}"
      Video.where("id IN (#{video_id_set})").update_all(['status = 1 , timestamp = ?' , @@now])
      videos.each do |video|
        begin
          #do convert ... maybe call system
          ...
          #finished
          video.update_attribute('status' , 2)
        rescue
          Log.error.error "==:#{$!}\n#{$@.join("\n")}"
          Log.info.info "Convert fail : #{video.attributes}"
        end
      end
    end
  rescue Exception => e ; Log.error.error "==:#{e.message}\n#{e.backtrace.join("\n")}"
    # do nothing
  end
end

go & run

> mkdir log #沒有log沒地方存
> bundle install
> ruby video.rb
> rerun video.rb # 方便開發使用,請見 gem : rerun
> god -c video.god # 開成god,請見 gem : god
> god terminate

這支程式我不解釋太深(…因為解釋不完|||),首先有 bundle 支援,可以用 Gemfile 來限定 gem 的版本,有 logger 支援還 multilogger ,用 info 和 error 來分離成兩個 log 檔,有 ActiveRecord 支援,YAML 讀外部的 config.yml 讓設定值可以提到外面去,Redis 雖然沒用到不過 init 已經做好了,大概就這樣而已

如果要做到無法開啟相同的兩台 server 來做交易防止 race condition,請服用我丟深入技術文件的 blog 的其中一篇 Ruby File Lock For Check Duplicate Running

anyway 看會了這篇的 code 你就可以把一個有穩定性的輔助用的 server 跑起來才是,以上 :slight_smile:


Ruby / Rails 武器的選擇
#2

JC大了火力總是這麼猛==