view · edit · sidebar · attach · print · history

20110616-update-polling-process-globo_xmlconv_bbmb_ch-update-swissmedicPlugin-ch_oddb

<< | Index | >>


  1. Create setup.globo.xmlconv.bbmb.ch.sh
  2. Check polling process
  3. Read conversion code
  4. Review the new task

Goal/Estimate/Evaluation
  • update polling process / 70% / 50% suspend
  • update praeparate.xls process / 70% / 80%
Milestones
  • Check polling process
  • Replacing ean-code
  • Check task
  • Update method
Summary
Commits
Next

Create setup.globo.xmlconv.bbmb.ch.sh

Check polling process

Refer: xmlconv/lib/xmlconv/util/transaction.rb#execute

convert_test.rb

require 'lib/conversion/propharma_bdd'
require 'lib/conversion/bdd_csv'

reader = XmlConv::Conversion::ProPharmaBdd
input  = reader.parse(File.read('input.txt'))
model  = reader.convert(input)
writer = XmlConv::Conversion::BddCsv
output = writer.convert(model)
p output.to_s

Data

Result

masa@masa ~/ywesee/globo.xmlconv.bbmb.ch $ ruby -I lib test.rb 
"380013,,16062011,,02333332,,,10,,113047,\n"

convert_test2.rb

require 'lib/conversion/propharma_bdd'
require 'lib/conversion/bdd_csv'
require 'lib/conversion/bdd_i2'


reader = XmlConv::Conversion::ProPharmaBdd
input  = reader.parse(File.read('input.txt'))
model  = reader.convert(input)
#writer = XmlConv::Conversion::BddCsv
writer = XmlConv::Conversion::BddI2
output = writer.convert(model)
print output.to_s

Result

masa@masa ~/ywesee/globopharm.xmlconv.bbmb.ch $ ruby -I lib test.rb 
001:7601001000681
002:ORDERX
003:220
010:Masapotheke_00113047.dat
100:YWESEE
101:113047
201:CU
237:61
238:1
250:ADE
251:113047
300:4
301:20110616
500:1
501:
502:02333332
520:10
521:PCE

Note

  • There is not '202' record
  • lib/conversion/bdd_i2.rb is not found in the git repository but it is there in online server
  • lib/conversion/propharma_bdd.rb is different from the one of online server
masa@masa ~/ywesee/globopharm.xmlconv.bbmb.ch/lib/conversion $ diff propharma_bdd.rb.online propharma_bdd.rb.git 
62c62,64
<     customer.add_id('ACC', id)
---
>     ## Propharma sends the Supplier-side Customer-ID, not the public EAN13 as it 
>     #  is the case in Propharma -> Globopharm
>     customer.add_id('supplier', id)
69c71,73
<     customer.add_id('ACC', _named_data(:cid, ast))
---
>     ## Propharma sends the Supplier-side Customer-ID, not the public EAN13 as it 
>     #  is the case in Propharma -> Globopharm
>     customer.add_id('customer', _named_data(:cid, ast))

convert_test_propharma_bdd_i2.rb (by using propharma_bdd.rb of online server)

require 'lib/conversion/propharma_bdd'
require 'lib/conversion/pharmacieplus_bdd'
require 'lib/conversion/bdd_csv'
require 'lib/conversion/bdd_i2'


reader = XmlConv::Conversion::ProPharmaBdd
#reader = XmlConv::Conversion::PharmaciePlusBdd
input  = reader.parse(File.read('input.txt'))
#input  = reader.parse(File.read('input.xml'))
model  = reader.convert(input)
#writer = XmlConv::Conversion::BddCsv
writer = XmlConv::Conversion::BddI2
output = writer.convert(model)
print output.to_s

Result

masa@masa ~/ywesee/globopharm.xmlconv.bbmb.ch $ ruby -I lib convert_test_propharma_bdd_i2.rb 
001:7601001000681
002:ORDERX
003:220
010:Masapotheke_00113047.dat
100:YWESEE
101:113047
201:CU
202:380083
237:61
238:1
250:ADE
251:113047
300:4
301:20110616
500:1
501:
502:02333332
520:10
521:PCE

Note

  • 202 record is outputted (but this is not ean-code)
  • 'pharmacieplus_bdd.rb', 'wbmb_bdd.rb' are also different
masa@masa ~/ywesee/globopharm.xmlconv.bbmb.ch/lib/conversion $ diff pharmacieplus_bdd.rb.online pharmacieplus_bdd.rb.git 
109c109
<     item.qty = _latin1(xml_item.attributes['qte-livraison'])
---
>     item.qty = _latin1(xml_item.attributes['qte-facture'])
masa@masa ~/ywesee/globopharm.xmlconv.bbmb.ch/lib/conversion $ diff wbmb_bdd.rb.online wbmb_bdd.rb.git
65a66
>         customer.add_id('ACC', value)
  • the number of conversion libraries is different

Local

masa@masa ~/ywesee/globopharm.xmlconv.bbmb.ch/lib/conversion $ ls
bdd_csv.rb  pharmacieplus_bdd.rb  propharma_bdd.rb  sunstore_bdd.rb  wbmb_bdd.rb  xundart_bdd.rb

Online

ywesee@server /var/www/globopharm.xmlconv.bbmb.ch/lib/conversion $ ls
bdd_i2.rb  pharmacieplus_bdd.rb  propharma_bdd.rb  wbmb_bdd.rb

Read conversion code

  • propharma_bdd.rb, bdd_i2.rb (online)

Experiment (xmlconv/lib/xmlconv/i2/address.rb#to_s)

      def to_s
 print "@party_id = "
 p @party_id
        output = []
        numerals = [ 201, 202, 220, 221, 222, 223, 224, 225, 226 ]
        [
          I2_ADDR_CODES[ @code],
          @party_id, @name1, @name2, @street1,
          @city, @country, @zip_code, @street2,
        ].each_with_index { |value, idx|
          unless(value.nil?)
            output << "#{numerals[idx]}:#{value}"
          end
        }
        output.join("\n") << "\n"
      end

Result

masa@masa ~/ywesee/globopharm.xmlconv.bbmb.ch $ ruby -I lib convert_test_propharma_bdd_i2.rb 
@party_id = "380083"
001:7601001000681
002:ORDERX
003:220
010:Masapotheke_00113047.dat
100:YWESEE
101:113047
201:CU
202:380083
237:61
238:1
250:ADE
251:113047
300:4
301:20110616
500:1
501:
502:02333332
520:10
521:PCE

Note

  • @party_id is the Kundennr
  • This value comes from 'idt' in the input file

Review the new task

Email

Note

  • 'Präparateliste.xls' has been used in initialize_export_registrations method of SwissmedicPlugin class
  • The column position if the xls file is defined as 'PREPARATIONS_COLUMNS'

Compare the column list

  • The new file has column E 'H=Hauptzul., E=Export'
  • The column N 'Anz. Packungen Seq.' and O 'Packungsgrösse' disappear in the new file
  • The column N 'Anz. Packungen Seq.' (:package_count) is used in the source code
  • The 'package_count' is used to judge the 'export_flag' true or false
    • if 'package_count' != 0, 'export_flag' becomes true

Experiment

module ODDB
  class SwissmedicPlugin < Plugin
=begin
    PREPARATIONS_COLUMNS = [ :iksnr, :seqnr, :name_base, :company,
      :index_therapeuticus, :atc_class, :production_science,
      :sequence_ikscat, :ikscat, :registration_date, :sequence_date,
      :expiry_date, :substances, :package_count, :package_sizes, :composition,
      :indication_registration, :indication_sequence ]
=end
    PREPARATIONS_COLUMNS = [ :iksnr, :seqnr, :name_base, :company, :export_flag,
      :index_therapeuticus, :atc_class, :production_science,
      :sequence_ikscat, :ikscat, :registration_date, :sequence_date,
      :expiry_date, :substances, :composition, :indication_registration, :indication_sequence ]
    def initialize_export_registrations(agent)
      #latest_name = File.join @archive, "Präparateliste-latest.xls"
      latest_name = File.join @archive, "Praeparate_mit_WS-latest.xls"
#=begin
      #if target = get_latest_file(agent, 'Präparateliste')
      if target = get_latest_file(agent, 'Praeparate_mit_WS')
        FileUtils.cp target, latest_name
      end
#=end
      reg_indices = {}
      #[ :iksnr, :package_count, :expiry_date,
        #:registration_date ].each do |key|
      [ :iksnr, :export_flag, :expiry_date, :registration_date ].each do |key|
        reg_indices.store key, PREPARATIONS_COLUMNS.index(key)
      end
      seq_indices = {}
      [ :seqnr, :name_base, :atc_class ].each do |key|
        seq_indices.store key, PREPARATIONS_COLUMNS.index(key)
      end
      Spreadsheet.open(latest_name) do |workbook|
        iksnr_idx = reg_indices.delete(:iksnr)
        seqnr_idx = seq_indices.delete(:seqnr)
        #count_idx = reg_indices.delete(:package_count)
        export_flag_idx = reg_indices.delete(:export_flag)
        workbook.worksheet(0).each(3) do |row|
          iksnr = row[iksnr_idx]
          seqnr = row[seqnr_idx]
          #count = row[count_idx].to_i
          export = row[export_flag_idx]
          #if count == 0
          if export == 'E'
            data = {}
            seq_indices.each do |key, idx|
              data.store key, row[idx]
            end
            @export_sequences[[iksnr, seqnr]] = data

          if aggregate = @export_registrations[iksnr]
#            aggregate[:package_count] += count
          else
            data = {}
            reg_indices.each do |key, idx|
              data.store key, row[idx]
            end
#            data[:package_count] = count
            @export_registrations.store iksnr, data
          end
          end
        end
      end
#      @export_registrations.delete_if do |iksnr, data|
#        data[:package_count] != 0
#      end
      @export_registrations
    end
    def update_export_registrations export_registrations
      export_registrations.delete_if do |iksnr, data|
        if reg = @app.registration(iksnr)
          @known_export_registrations += 1 if reg.export_flag
#          data.delete :package_count
          data.update :export_flag => true, :inactive_date => nil
          @app.update reg.pointer, data, :swissmedic
          false
        else
          true
        end
      end
    end

Run

  • bin/oddbd
  • update_swissmedic (bin/admin)
masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Updater.new(self).update_swissmedic

Result

Plugin: ODDB::SwissmedicPlugin
Error: RuntimeError
Message: could not identify url to Praeparate_mit_WS.xls
Backtrace:
/home/masa/ywesee/oddb.org/src/plugin/swissmedic.rb:204:in `get_latest_file'
/home/masa/ywesee/oddb.org/src/plugin/swissmedic.rb:226:in `initialize_export_registrations'
/home/masa/ywesee/oddb.org/src/plugin/swissmedic.rb:43:in `update'
/home/masa/ywesee/oddb.org/src/util/updater.rb:346:in `update_swissmedic'
/home/masa/ywesee/oddb.org/src/util/updater.rb:473:in `call'
/home/masa/ywesee/oddb.org/src/util/updater.rb:473:in `wrap_update'
/home/masa/ywesee/oddb.org/src/util/updater.rb:344:in `update_swissmedic'
(eval):1:in `_admin'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1426:in `_admin'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1424:in `initialize'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1424:in `new'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1424:in `_admin'
/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `__send__'
/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'
/usr/lib64/ruby/1.8/drb/drb.rb:1515:in `perform'
/usr/lib64/ruby/1.8/drb/drb.rb:1589:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1430:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1347:in `initialize'
/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `new'
/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `start_service'
bin/oddbd:38

Note

  • 'get_latest_file' method could not download the file

Experiment

    def initialize_export_registrations(agent)
      #latest_name = File.join @archive, "Präparateliste-latest.xls"
      latest_name = File.join @archive, "Praeparate_mit_WS-latest.xls"
#=begin
      if target = get_latest_file(agent, 'Präparateliste')
      #if target = get_latest_file(agent, 'Praeparate_mit_WS')
        FileUtils.cp target, latest_name
      end
#=end

Run

  • bin/oddbd
  • update_swissmedic (bin/admin)
masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Updater.new(self).update_swissmedic

Result

ODDB::SwissmedicPlugin - Report 16.06.2011
Created Packages: 373
Updated Packages: 18265
Deleted Packages: 81 (0 Replaced)
Deleted Sequences: 224
Deactivated Registrations: 466
Updated new Export-Registrations: 141
Updated existing Export-Registrations: 405
Updated new Export-Sequences: 70
Updated existing Export-Sequences: 569
Total Sequences without ATC-Class: 21
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,57441!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,57525!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,58550!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,59080!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,59290!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,59338!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,60378!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,60379!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,60380!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,60381!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,60413!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61207!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61408!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61436!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61471!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61519!sequence,01.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61519!sequence,02.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61519!sequence,03.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61519!sequence,04.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61519!sequence,05.
http://oddb.masa.org/de/gcc/resolve/pointer/:!registration,61540!sequence,01.

Note

  • Looks not bad

Commit

view · edit · sidebar · attach · print · history
Page last modified on June 17, 2011, at 07:38 AM