20110811-migel-search-import-oddb_org

<< | Index | >>

1. Make reverse sort function
2. Another link
3. Implement import function
4. Design Item class for migel data

Goal/Estimate/Evaluation

• Migel sort function / 100% / 100%
• Migel import function / 80% / 30%

Milestones

• reverse function

Summary

• http://masay.ch/2011/08/11/develop-search-and-import-functions-of-migel-data-in-oddb-org/

Commits

ToDo Tomorrow

• develop ODDB::Migel::Item class and save the instance in the database via ODBA
• import function
• save link information in the database (z.B. in ODDB::Migel::Product)
• diactivate links

Make reverse sort function

Experiment

• src/state/migel/searched_table.rb

#!/usr/bin/env ruby
#  ODDB::State::Migel:Test -- oddb.org -- 09.08.2011 -- mhatakeyama@ywesee.com

require 'state/global_predefine'
require 'view/migel/searched_table'

module ODDB
  module State
    module Migel
class SearchedTable < State::Migel::Global
  VIEW = ODDB::View::Migel::SearchedTable
  def sort
    get_sortby!
    @model.sort! { |a, b| compare_entries(a, b) }
    @model.reverse! if(@sort_reverse)
    self
  end
end
    end
  end
end

• src/model/migel/searched_table.rb

  def reverse!
    @table.reverse!
  end

Run

• bin/oddbd
• ext/swissindex/bin/swissindex_nonpharmad
• bin/currencyd
• bin/yusd

Access

• http://oddb.masa.org/de/gcc/migel_search/migel_code/151003011

Result

First click on 'Artikelname' (sort by dictionary order)

Second click (reverse)

Another link

Experiment

• vim src/view/migel/product.rb#migel_code

class ProductInnerComposite < HtmlGrid::Composite
  def migel_code(model)
    link = PointerLink.new(:to_s, model, @session, self)
    link.value = model.migel_code
    link.href = @lookandfeel._event_url(:migel_search, {:migel_code => model.migel_code.gsub(/\./, '')})
    link
  end

Result

Implement import function

Experiment

• ext/swissindex/src/swissindx.rb#search_migel_info

  def search_migel_info(code, query_key = 'Pharmacode')
    # 'MiGelCode' is also available for query_key
    agent = Mechanize.new
    try_time = 3
    begin
      agent.get(@base_url + query_key + '=' + code)
      line  = []
      agent.page.search('td').each_with_index do |td, i|
        text = td.inner_text.chomp.strip
        if text.is_a?(String)
          text = text.split(/\n/)[1] || text.split(/\n/)[0]
          text = text.gsub(/\302\240/, '').strip if text
          line << text
        end
        break if i > 10
      end
      info = if line[10]
               {
                 :migel_code   => line[6],
                 :limitation   => (line[7] == 'L'),
                 :product_text => line[8],
                 :unit         => line[9],
                 :price        => line[10]
               }
             else
               nil
             end
    rescue StandardError, Timeout::Error => err
      if try_time > 0
        puts err
        puts err.backtrace
        puts
        puts "retry"
        sleep 10
        agent = Mechanize.new
        try_time -= 1
        retry
      else
        return nil
      end
    end
  end

• src/plugin/swissindex.rb#search_migel_info

    def search_migel_info(migel_code)
      info_list = nil
      SWISSINDEX_NONPHARMA_SERVER.session(ODDB::Swissindex::SwissindexNonpharma) do |swissindex|
        info_list = swissindex.search_migel_info(migel_code, 'MiGelCode')
      end
      info_list
    end

Run

• bin/admin

ch.oddb> open('/home/masa/work/test1.dat','w'){|f| f.print ODDB::SwissindexNonpharmaPlugin.new(self).search_migel_info('151003011').pretty_inspect}
ch.oddb> open('/home/masa/work/test2.dat','w'){|f| f.print ODDB::SwissindexNonpharmaPlugin.new(self).search_migel_info('15100301').pretty_inspect}

Result

masa@masa ~/work $ cat test1.dat 
{:migel_code=>"151003011",
 :product_text=>
  "Einmalblasenkatheter Ready to use mit Infektionsschutzh\303\274lse, mit Nelaton- oder Tiemannspitze",
 :limitation=>true,
 :unit=>"St\303\274ck",
 :price=>"9.90"}
masa@masa ~/work $ cat test2.dat 
nil

Note

• I misunderstood
• I do not have to update migel product data (group, subgroup neither)
• Just import nonpharma products related to a migel code

Next

• Desgin Item (migel data) class (State, View class too)

Design Item class for migel data

Experiment

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> migel_count
-> 571
ch.oddb> open('/home/masa/work/test.dat','w'){|f| each_migel_product{|product| f.print product.migel_code, "\n"}}
-> Hash

Note

• The total number of migel product is 571

Experiment

• src/model/migel/item.rb

#!/usr/bin/env ruby
# ODDB::Migel::Item -- oddb.org -- 11.08.2011 -- mhatakeyama@ywesee.com

require 'util/language'
require 'model/text'

module ODDB
  module Migel
    class Item
      include SimpleLanguage
      attr_reader :pharmacode, :product
      def initialize(pharmacode, product)
        @pharmacode = pharmacode
        @product = product
      end
    end
  end
end

• src/model/migel/product.rb

      attr_reader :code, :accessories, :products, :product_text, :items
      def initialize(code)
        @items = []

...

      def add_item(pharmacode)
        @items ||= []
        item = ODDB::Migel::Item.new(pharmacode, self)
        @items.push(item)
        item
      end

• src/util/oddbapp.rb

  def masa
    product = search_migel_products('151002011', 'de')[0]
    pointer = product.pointer + [:item]
    item = product.items[0]
    update_values = {
      :pharmacode => item.pharmacode,
    }
    self.update(pointer.creator, update_values, :masa)
  end

Run

• bin/admin

ch.oddb> search_migel_products('151002011', 'de')[0].items[0].class
-> undefined method `[]' for nil:NilClass
ch.oddb> search_migel_products('151002011', 'de')[0].add_item('123')
-> 
ch.oddb> search_migel_products('151002011', 'de')[0].items[0].pharmacode
-> 123
ch.oddb> masa
-> undefined method `diff' for #<ODDB::Persistence::InvalidPathError:0x7f4ef4b50c58>

Log

failsafe rescued ODDB::Persistence::InvalidPathError < StandardError
:!migel_group,15!subgroup,10!product,02%.01%.1!. -> undefined Method ODDB::Migel::Product::item()
/home/masa/ywesee/oddb.org/src/util/persistence.rb:313:in `resolve'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:296:in `each'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:296:in `resolve'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:228:in `issue_create'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:99:in `create'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:98:in `create'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:306:in `send'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:306:in `resolve'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:296:in `each'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:296:in `resolve'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:274:in `issue_update'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:119:in `update'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:118:in `update'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1274:in `masa'

Consideration

• The saved data does not have the Item instance, that is why undefined Method ODDB::Migel::Product::item() is comming.
• I guess, I should update the ODDB::Migel::Product first before the Item instance is saved

Next

• How to save a new instance in the database
• How to add a new data structure to an existing class