20120217-fix-errors-check-threads-problem-oddb_org

<< Masa.20120220-update-crawler_pattern-drop-pointer-link-sbsm-test-heavy-access-oddb_org | Index | Masa.20120216-update-interaction-algorithm-fix-float-domain-err-add-migel-interactions-mobile-check-threads-oddb_org >>

1. Fix errors oddb.org
2. Check when a new thread is generated in oddb.org
3. Stop threads when the same session has many requests

Commits

1. Deleted thread status information from status log (oddb.org)
2. Fix MoMethodError undefined method code for nil:NilClass in interaction result view (oddb.org)
3. Skip NoMethodError undefined method values for lt:String in src/model/sequence.rb:53:in active_package_count (oddb.org)
4. Skip NoMethodError undefined method values for %2C:String in src/model/sequence.rb:243:in public_package_count (oddb.org)
5. Skip NoMethodError undefined method any? for %:String in src/model/sequence.rb:181:in has_public_packages? (oddb.org)
6. Fix NoMethodError undefined method [] for nil:NilClass at src/state/page_facade.rb:66:in page (oddb.org)
7. Skip NoMethodError undefined method [] for 1.0:Float at src/model/sequence.rb:219:in package (oddb.org)
8. Skip NoMethodError undefined method collect for =\:String at src/model/package.rb:123:in commercial_forms (oddb.org)
9. Skip NoMethodError undefined method limitation_text for ":String at src/model/package.rb:270:in limitation_text (oddb.org)
10. Skip NoMethodError undefined method de for H2:String at src/view/dataformat.rb:84:in name_base (oddb.org)
11. Skip NoMethodError undefined method prices for /^triple-?/:Regexp at src/state/drugs/price_history.rb:35:in init (oddb.org)
12. Skip NoMethodError undefined method collect for %:String at src/model/composition.rb:63:in substances (oddb.org)
13. Skip NoMethodError undefined method galenic_form for ([^ a-zA-Z0-9_.-]+):String at src/model/sequence.rb:174:in block in galenic_forms (oddb.org)

Fix errors oddb.org

Refer to http://dev.ywesee.com/uploads/$PageName//ch.oddb_crash_17.2.2012_05.29.txt

Commits

• Deleted thread status information from status log (oddb.org)
• Fix MoMethodError undefined method code for nil:NilClass in interaction result view (oddb.org)
• Skip NoMethodError undefined method values for lt:String in src/model/sequence.rb:53:in active_package_count (oddb.org)

Refer to http://dev.ywesee.com/uploads/$PageName//ch.oddb_crash_17.2.2012_03.17.txt

Commits

• Skip NoMethodError undefined method values for %2C:String in src/model/sequence.rb:243:in public_package_count (oddb.org)
• Skip NoMethodError undefined method any? for %:String in src/model/sequence.rb:181:in has_public_packages? (oddb.org)
• Fix NoMethodError undefined method [] for nil:NilClass at src/state/page_facade.rb:66:in page (oddb.org)

Refer to http://dev.ywesee.com/uploads/$PageName//ch.oddb_crash_16.2.2012_19.49.txt

Commits

• Skip NoMethodError undefined method [] for 1.0:Float at src/model/sequence.rb:219:in package (oddb.org)
• Skip NoMethodError undefined method collect for =\:String at src/model/package.rb:123:in commercial_forms (oddb.org)
• Skip NoMethodError undefined method limitation_text for ":String at src/model/package.rb:270:in limitation_text (oddb.org)

Refer to http://dev.ywesee.com/uploads/$PageName//ch.oddb_crash_16.2.2012_21.19.txt

Commits

• Skip NoMethodError undefined method de for H2:String at src/view/dataformat.rb:84:in name_base (oddb.org)
• Skip NoMethodError undefined method prices for /^triple-?/:Regexp at src/state/drugs/price_history.rb:35:in init (oddb.org)
• Skip NoMethodError undefined method collect for %:String at src/model/composition.rb:63:in substances (oddb.org)
• Skip NoMethodError undefined method galenic_form for ([^ a-zA-Z0-9_.-]+):String at src/model/sequence.rb:174:in block in galenic_forms (oddb.org)

Check when a new thread is generated in oddb.org

Note

• A new thread is created in /usr/local/lib/ruby/1.9.1/drb/drb.rb#main_loop by Thread.start(@protocol.accept)
• When a request comes, Thread.start(@protocol.accept) creates a new thread
• This drb (threads) belongs to one oddb application process
• Apache object, which is provieded by mod_ruby, in sbsm is shared between processes
• Even if a oddb application process is switched in doc/index.rbx, the same Apache.request is sent to all the processes
• It seems that same Apache.request object is used several times
• Point: How to sort Apache.requests to each oddb application process

Experiment

• doc/index.rbx

#!/usr/bin/env ruby
# index.rbx -- oddb -- hwyss@ywesee.com

require 'sbsm/request'
require 'util/oddbconfig'

DRb.start_service('druby://localhost:0')

begin
    request0 = SBSM::Request.new('druby://localhost:10000')
    request1 = SBSM::Request.new('druby://localhost:98765')
    if Time.now.sec % 2 == 0
warn "A"
      request0.server_thread_count
      request0.process
    else
warn "B"
      request1.server_thread_count
      request1.process
    end
rescue Exception => e
  $stderr << "ODDB-Client-Error: " << e.message << "\n"
  $stderr << e.class << "\n"
  $stderr << e.backtrace.join("\n") << "\n"
end

• src/util/oddbapp.rb

  def thread_count
    Thread.list.length
  end

• /usr/local/lib/ruby/gems/1.9.1/gems/sbsm-1.0.7/lib/sbsm/request.rb

    def server_thread_count
      @proxy ||= begin
        args = {
          'database_manager'  =>  CGI::Session::DRbSession,
          'drbsession_uri'    =>  @drb_uri,
          'session_path'      =>  '/',
        }
        if(is_crawler?)
          sleep 0.5
          sid = [ENV['DEFAULT_FLAVOR'], @cgi.params['language'], @cgi.user_agent].join('-')
          args.store('session_id', sid)
        end
        session = CGI::Session.new(@cgi, args)
        session[:proxy]
                 end
      @proxy.thread_count
    end

• /usr/local/lib/ruby/1.9.1/drb/drb.rb

    def main_loop
      Thread.start(@protocol.accept) do |client|
        @grp.add Thread.current
p " #{Time.now}: @grp = #{@grp},@grp.list.length = #{@grp.list.length}, Thread.list.length = #{Thread.list.length}"

Run

• ywesee/oddb.org/bin/oddbd
• work/oddb.org/bin/oddbd with SERVER_URI = "druby://localhost:98765"

Benchmark

 $ /usr/sbin/ab -n 100 -c 100 http://oddb.masa.org/

Result

• server1 (druby://localhost:10000)

 ....
 " 2012-02-17 16:19:45 +0100: @grp = #<ThreadGroup:0x0000000073a2f8>,@grp.list.length = 32, Thread.list.length = 38"
 " 2012-02-17 16:19:45 +0100: @grp = #<ThreadGroup:0x0000000073a2f8>,@grp.list.length = 33, Thread.list.length = 40"
 " 2012-02-17 16:19:45 +0100: @grp = #<ThreadGroup:0x0000000073a2f8>,@grp.list.length = 34, Thread.list.length = 40"
 " 2012-02-17 16:19:45 +0100: @grp = #<ThreadGroup:0x0000000073a2f8>,@grp.list.length = 35, Thread.list.length = 41"
 " 2012-02-17 16:19:45 +0100: @grp = #<ThreadGroup:0x0000000073a2f8>,@grp.list.length = 36, Thread.list.length = 42"

• server2 (druby://localhost:98765)

 ....
 " 2012-02-17 16:20:00 +0100: @grp = #<ThreadGroup:0x00000001cd2908>,@grp.list.length = 64, Thread.list.length = 70"
 " 2012-02-17 16:20:01 +0100: @grp = #<ThreadGroup:0x00000001cd2908>,@grp.list.length = 64, Thread.list.length = 70"
 " 2012-02-17 16:20:03 +0100: @grp = #<ThreadGroup:0x00000001cd2908>,@grp.list.length = 64, Thread.list.length = 70"
 " 2012-02-17 16:20:08 +0100: @grp = #<ThreadGroup:0x00000001cd2908>,@grp.list.length = 64, Thread.list.length = 70"

Note

• Usually it goes over 100 threads if only one oddb application runs
• The requests are sorted to each server

suspend

Stop threads when the same session has many requests