20110711-process-develop-drb-server

<< | Index | >>

1. Debug swissindex plugin
2. Delete recheck deletions process swissmedic plugin
3. Separate the update_swissmedic to a different process from oddbd

Goal/Estimate/Evaluation

• Develop swissmedic updater drb server / 60% / 30%

Milestones

• move methods from ODDB::SwissindexPlugin to ODDB::Swissinex::Updater
  1. get_latest_file
  2. initialize_export_registrations
  3. keep_active_registrations_praeparateliste
  4. update_* methods suspend after the reboot oddbd (first, check if @app.update is possible or not via DRb server)
  5. set_all_export_flag_false suspend
  6. sanity_check_deletions
  7. delete @@suspendææ
  8. recheck_deletions
  9. deactivate suspend

Summary

Commits

• Updated swissindex plugin log output with more information regarding the updates (oddb.org)
• Do not consider Preaparateliste_mit_WS.xls when setting the 'deaktiviert am' date. (oddb.org)

Debug swissindex plugin

Email

• ch.ODDB.org Report - Swissindex Pharma - 07/2011

 Checked 23943 packages
 Updated in trade     (out_of_trade:false): 0 packages
 Updated out of trade (out_of_trade:true) : 12945 packages
 Updated pharmacode: 0 packages
 Deleted pharmacode: 13168 packages

Note

• These numbers look wrong

Run locally and check log

Resolved

• After reboot siwssindex_pharmad
• We do not know the reason

Commit

• Updated swissindex plugin log output with more information regarding the updates

Delete recheck deletions process swissmedic plugin

Problem

• We used to check the @diff.deletions (registrations and sequences) if they are still in Praeparateliste_mit_WS.xls

Commit

• Do not consider Preaparateliste_mit_WS.xls when setting the 'deaktiviert am' date. (oddb.org)

Separate the update_swissmedic to a different process from oddbd

• ext/swissmedic/bin/swissmedicd

#!/usr/bin/env ruby
# Swissmedicd -- oddb.org -- 08.07.2011 -- mhatakeyama@ywesee.com

$: << File.expand_path("../src", File.dirname(__FILE__))

require 'drb/drb'
require 'swissmedic'

uri = ODDB::Swissmedic::URI
DRb.start_service(uri, ODDB::Swissmedic)
DRb.thread.join

• ext/swissmedic/src/swissmedic.rb

#!/usr/bin/ruby
# encoding: utf-8
# ODDB::Swissmedic::Updater -- 08.07.2011 -- mhatakeyama@ywesee.com

require 'rubygems'
require 'drb'


module ODDB
  module Swissmedic
    URI = 'druby://localhost:60001'
    def Swissmedic.session
      yield(ODDB::Swissmedic::Updater.new)
    end

class Updater
  include DRb::DRbUndumped
  def test
    'test'
  end
end

  end # Swissmedic
end # ODDB

• src/util/updater.rb

require 'plugin/swissmedic_updater'

• src/plugin/swissmedic_updater.rb

#!/usr/bin/env ruby
# ODDB::SwissmedicUpdaterPlugin -- oddb.org -- 11.07.2011 -- mhatakeyama@ywesee.com

require 'plugin/plugin'
require 'drb'
require 'ext/swissmedic/src/swissmedic'

module ODDB
  class SwissmedicUpdaterPlugin < Plugin
    SWISSMEDIC_SERVER = DRbObject.new(nil, ODDB::Swissmedic::URI)
    def test
      SWISSMEDIC_SERVER.session do |server|
        server.test
      end
    end
    def report
      lines = [
        nil
      ]
      lines.join("\n")
    end
  end
end # ODDB

Run

• bin/oddbd
• ext/swissmedic/bin/swissmedicd
• bin/admin

ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> test

Note

• Good

Next

• move the original methods one by one to the drb server class
  1. get_latest_file
  2. initialize_export_registrations

Note

• Mechanize failed

etc/swissmedic/src/swissmedic.rb#test

  def test
    agent = Mechanize.new
    page = agent.get "http://ywesee.com"
  end

Run

• bin/oddbd
• ext/swissmedic/bin/swissmedicd
• bin/admin

ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> /usr/lib64/ruby/1.8/drb/drb.rb:570:in `load': connection closed

Experiment

• etc/swissmedic/src/swissmedic.rb#test

  def test
    agent = Mechanize.new.extend(DRb::DRbUndumped)
    page = agent.get("http://ywesee.com")
  end

Result

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> /usr/lib64/ruby/1.8/drb/drb.rb:570:in `load': connection closed
ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> /usr/lib64/ruby/1.8/drb/drb.rb:570:in `load': connection closed
ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> #<Mechanize::Page:0x7ff0f83fb178>
ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> #<Mechanize::Page:0x7ff0f83e9a40>
ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> #<Mechanize::Page:0x7ff0f83d8308>

Note

• It fails in the connection everytime 2 times
• Then, it succeedes
• suspend this probelm

Experiment

• ext/swissmedic/src/swissmedic.rb

  def test
    start_time = Time.new
    agent = Mechanize.new
    target = get_latest_file(agent, 'Packungen')
    if(target)
      initialize_export_registrations(agent)
      keep_active_registrations_praeparateliste
      diff target, @latest, [:atc_class, :sequence_date]
    end
  end

Run

• bin/admin

ch.oddb> ODDB::SwissmedicUpdaterPlugin.new(self).test
-> (druby://localhost:60001) /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.5.0/lib/spreadsheet.rb:68:in `initialize': No such file or directory - /home/masa/work/Packungen-latest.xls

Note

• This error happens because Packungen-latest.xls is not found but
• Why does this error not happen when the old swissmedic plugin is executed?
• suspend this problem