20110505-swissindex_pharma_plugin

<< | Index | >>

1. SwissindexPharmaPlugin
2. SwissindexNonPharmaPlugin
3. Check duplicated pharmacodes
4. Update package out_of_trade flag

Goal/Estimate/Evaluation

• SwissindexPlugin / 100% / 70%

Milestones

• Log result yesterday
• Update swissindex
• Add NonPharma ext module
• Refactoring

Summary

• http://masay.ch/2011/05/05/swissindex_pharma_plugin/

Commits

SwissindexPharmaPlugin

Log (yesterday)

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Updater.new(self).check_ean
(druby://localhost:10000) /home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1418:in `_admin': (druby://localhost:9485) /usr/lib64/ruby/1.8/timeout.rb:60:in `rbuf_fill': execution expired (Timeout::Error)
        from (druby://localhost:10000) /usr/lib64/ruby/1.8/drb/drb.rb:1555:in `join'
        from bin/admin:97

Experiment (if it is possible to get pharma information by 9 first digit of eancode)

require 'drb'
uri = 'druby://localhost:9485'
SWISSINDEX = DRbObject.new(nil, uri)

#p SWISSINDEX.search('7680473220308')
p SWISSINDEX.search('768047322')

Result

 nil

Note

• The eancode must be full 11 digit to search data by swissindex

Check all the eancode by swissindex

Next task

• Check all the eancode in oddb.org by the swissindex
• If there is not a eancode found in the swissindex, output the eancode list to a file

src/plugin/swissindex.rb

    def check_ean_pharma
      out = open('/home/masa/work/out.dat', "w")
      result = nil
      @app.each_package do |pack|
        if ean = pack.barcode.to_s
          result = nil
          SWISSINDEX_PHARMA_SERVER.session do |swissindex|
            result = swissindex.search(ean)
          end
          unless result
            out.print ean, "\n"
          end
        end
      end
      out.close
      out.path
    end

Run

• oddbd
• ext/swissindex/bin/swissindex_pharmad
• check_ean_pharma (from bin/admin)

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Updater.new(self).check_ean_pharma

SwissindexNonPharmaPlugin

ToDo

• ext/swissindex/bin/swissindex_nonpharmade
• ext/swissindex/src/swissindex_nonpharma.rb
• src/util/updater.rb#new_method
• src/plugin/swissindex.rb#new_method

Memo (vim, multi-window)

Reference (Japanese)

• http://kaworu.jpn.org/kaworu/2007-07-26-1.php
• http://www.nurs.or.jp/~sugi/vi.htm

 F1 (:sp file, ctl-w s) split windows
 F2 move window
 ctl-w q close window

Check duplicated pharmacodes

Problem

• Email Doppelte Pharmacodes im oddb.csv
• There are some duplicated pharmacodes (but eancodes are unique)

Script

• Attach:check_dup_pharma.20110505.rb
• Attach:check_dup_pharma_result.20110505.txt

Update package out_of_trade flag

Task

• If the eancode in oddb.org is not found in swissindex, out_of_trade flag of the package should be turned on (true)

check_swissindex_ean.rb (just check it by single file)

# 20110505 masa
# check ean code exists or not in swissindex

$: << File.expand_path("../../src", File.dirname(__FILE__))
$: << File.expand_path("src", File.dirname(__FILE__))

require 'swissindex'

eancode_file = '/home/masa/work/all_ean_oddb.dat'
output_file     = '/home/masa/work/out.dat'
eancode_list = File.readlines(eancode_file).to_a.map{|line| line.chomp}
eancode_list.delete("")

swissindex = ODDB::Swissindex::SwissindexPharma.new
#p swissindex.search('7680473220308')
#p swissindex.search('7680611920015')

total = eancode_list.length
count = 1
start_time = Time.now
open(output_file, "w") do |out|
  eancode_list.each_with_index do |ean, i|
    if i % 10 == 0 and i > 0
      current_time = Time.now - start_time
      estimate = current_time * (total/i.to_f)
      puts
      print "Estimate end of time: ", estimate, " [s], ", estimate/60, " [m], ", estimate/60/60, " [h]\n"
      puts
    end
    print i, " / ", total, " (", i.to_f/total*100, " %)\t", ean, "\n"
    puts
    unless swiss_phar = swissindex.search(ean)
      out.print count, "\t", ean, "\n"
      puts
      print "*"*50, " HIT ", "*"*50, "\n"
      if count % 10 == 0 and i > 0
        puts
        print "Hit Percentage: ", count.to_f/i*100, " % (", count, " / ", i, ")\n"
        puts
      end
      count += 1
    end
  end
end

Result

...

************************************************** HIT **************************************************

Hit Percentage: 57.6923076923077 % (30 / 52)

53 / 23617 (0.224414616589745 %)        7680611980026

...

Estimate: 10649.06471475 [s], 177.4844119125 [m], 2.958073531875 [h]

80 / 23617 (0.338739043909049 %)        7680592880087
...

Note

• The total number of packages in oddb.org is 23617
• A rough estimation time for the calculation is 3 hours (or more probably)

Add the method in swissindex plugin

src/plugin/swissindex.rb

    def update_package_trade_status
      out_of_trade_true_list  = []
      out_of_trade_false_list = []
      @app.each_package do |pack|
        SWISSINDEX_PHARMA_SERVER.session do |swissindex|
          if swissindex.search(pack.barcode.to_s)
            # not change flag
            # but just save the list
            out_of_trade_false_list << pack
          else
            out_of_trade_true_list << pack
          end
        end
      end

      #
      # change flag
      #

      #out_of_trade_true_list << @app.package('883519')
      #out_of_trade_false_list << @app.package('883519')
      out_of_trade_true_list.each do |pack|
        unless pack.out_of_trade
          @app.update(pack.pointer, {:out_of_trade => true}, :refdata)
        end
      end
      out_of_trade_false_list.each do |pack|
        if pack.out_of_trade
          @app.update(pack.pointer, {:out_of_trade => false, :refdata_override => false}, :refdata)
        end
      end

      # for debug
      open('/home/masa/work/result_update_swissindex_package_trade_status.dat', 'w') do |out|
        out.print "out_of_trade true list\n"
        out.print out_of_trade_true_list.join("\n"), "\n"
        out.print "\n"
        out.print "out_of_trade false list\n"
        out.print out_of_trade_false_list.join("\n"), "\n"
      end

    end

src/util/updater.rb

    public
    def update_package_trade_status_by_swissindex
      plug = SwissindexPlugin.new(@app)
      plug.update_package_trade_status
    end

Run

• bin/oddbd
• ext/swissindex/bin/swissindex_pharmad
• bin/admin

Updater.new(self).update_package_trade_status_by_swissindex

Result

 check it tomorrow