view · edit · sidebar · attach · print · history

< Niklaus.20171206-refdata-import | Index | Niklaus.20171204-fi-parse-error >>

20171205-refdata-import

Summary

  • Use refdata to import doctors
  • remove feedback from migel
  • Keep in Mind

Commits

Index

Use refdata to import doctors, hospitals, pharmacies and companies

Checking address-comparision for companies and doctors. Added code to disable parsing the details via medreg. Comparing the addresses looks okay for doctors now.

Running import for 10 doctors and companies.

The refdata_partner import handles the addresses correctly (e.g. the mail contains

7601001367753: {:name=>"Vorstadt Apotheke => GaleniCare AG", :narcotics=>"6011 Verzeichnis a/b/c BetmVV-EDI => ", "addresses"=>"-GaleniCare AG,Vorstadt 30/32,8200 Schaffhausen,fon,fax\n+Vorstadt Apotheke,Vorstadt  30/32,8200 Schaffhausen,fon,fax\n"}

but the narcotics fiels get set to nil, because the details were not fetched. This must be fixed.

Importing the doctors had another problem which must be fixed, too. Fixed by using ODDB::Doctor.new instead of mocking with OpenStruct.new.

Adapting test_plugin/refdata_partner.rb to avoid use of ODDB::App.new, but my mocking the few calls used. Finishes now in < 1 second instead of 25.

Rerunning the import (limited to 3).

As it took way too long I debugged and found out that @app.companies.odba_store took over 70 seconds times it was called. Runninig this code therefore only at the end of the import. Not the import finishes a lot faster, but at the end we got the following error

Error: TypeError
Message: no implicit conversion of String into Array
Backtrace:
/var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/stub.rb:146:in `+'
/var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/stub.rb:146:in `+'
/var/www/oddb.org/src/model/address.rb:147:in `to_diffable'
/var/www/oddb.org/src/model/address.rb:140:in `diff'
/var/www/oddb.org/src/plugin/refdata_partner.rb:292:in `store_company'
/var/www/oddb.org/src/plugin/refdata_partner.rb:380:in `block in parse_xml'
/var/www/oddb.org/src/plugin/refdata_partner.rb:322:in `each'
/var/www/oddb.org/src/plugin/refdata_partner.rb:322:in `parse_xml'
/var/www/oddb.org/src/plugin/refdata_partner.rb:131:in `update'
/var/www/oddb.org/src/util/updater.rb:258:in `block in update_refdata_partners'
/var/www/oddb.org/src/util/updater.rb:531:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:256:in `update_refdata_partners'
jobs/import_refdata_partners:14:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_refdata_partners:12:in `<module:Util>'
jobs/import_refdata_partners:11:in `<module:ODDB>'
jobs/import_refdata_partners:10:in `<main>'

Reloading the database and rerunning the import. Moving odba_store to the end for refdata_doctors, too. To speed up reworked parsing the XML using OX.load(xml, mode: hash_no_attrs).

The to_diff error stems from the fact that we have addresses in the database where the fon/fax-fields are strings not arrays. Must reload the database to check the import again.

Updating the partner took around 8 minutes. Running import of doctor now. Adding code for missing fields language and firstname, and fixing address.name, too.

Remarked that after running the import I did not find Coop Vitality Apotheke Kriens when searching for a pharmacy using "Kriens". Why?

Dropping and reloading the database to start the import doctors again. Checking manually the reported changes. Many changes look fine, but when there is no address at all for a doctor, the import removes the old address or it reports a change even when the new address is missing, too.

remove feedback from migel

This should not be too difficult.

view · edit · sidebar · attach · print · history
Page last modified on December 05, 2017, at 05:15 PM