view · edit · sidebar · attach · print · history

Index>

20160217-fi-parser-use-refdata-names

Summary

  • Adapt FI-parser to use refdata names for packages

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Order of entering search type and value should not matter. Both should show long URL with search
  • Remove parser for minifi (but keep the minifi)

Adapt FI-parser to use refdata names for packages

Continuing work of yesterday. PI-Square appears now in #de/gcc/search/zone/drugs/search_query/43788/search_type/st_registration?#best_result, but the link points to /de/gcc/patinfo/reg/43788/seq/01, which is not correct, as it should point to the package. Added a Watir test for it.

Now the link points to http://oddb-ci2.dyndns.org/de/gcc/patinfo/reg/43788/seq/01/pack/086, but it does not resolve correctly on oddb-ci2, but it worked for ch.oddb.org.

Fixing it changing src/state/global.rb and src/view/drugs/patinfo.rb. Now the patinfo display, but has no print link and clicking into a chapter does not work. Should we change displaying the patinfo and use its OID? No. But somehow the patinfo.pointer.skeleton returns :create instead of :patinfo. This must be fixed.

Found 257 entries via bin/admin patinfos.values.find_all{|x| x.pointer.skeleton == [:create] }.size. Cleaning my database by issuing in bin/admin each_registration { |reg| reg.packages.each { |pack| next if !pack or !pack.patinfo or pack.patinfo.pointer.skeleton != [:create]; delete_patinfo(pack.patinfo.oid); pack.patinfo = nil; pack.odba_store} }

Fixed this problems. Now things look a lot better. Running all Watir tests before checking in.

In oddb-ci2 mails found 

Plugin: ODDB::TextInfoPlugin
Error: ArgumentError
Message: invalid byte sequence in US-ASCII
Backtrace:
/var/www/oddb.org/src/plugin/text_info.rb:1443:in `split'
/var/www/oddb.org/src/plugin/text_info.rb:1443:in `parse_aips_download'
/var/www/oddb.org/src/plugin/text_info.rb:1504:in `import_swissmedicinfo'
/var/www/oddb.org/src/util/updater.rb:573:in `block in update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:549:in `call'
/var/www/oddb.org/src/util/updater.rb:549:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:567:in `update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:344:in `update_textinfo_swissmedicinfo'
/var/www/oddb.org/src/util/updater.rb:198:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

This must be fixed, too. Ran import_daily, but did not see the error here.

Pushed the commit, running reparse_needed.sh on thinpower and a jobs/update_textinfo_swissmedicinfo --reparse --target=both (just for the fun) locally.

Running on thinpower fails, because of missing rights. Moving generated files under log/oddb. Make output less verbose.

This looks good now:

Import all failed at IKSNR 36360 Problem was: 

2016-02-17 16:05:48 +0100: /var/www/oddb.org/src/plugin/text_info.rb:178:in `ensure_correct_atc_code': ensure_correct_atc_code iksnr 36360 atcFromFI is nil
failsafe rescued ArgumentError < StandardError
wrong number of arguments (5 for 1)
/var/www/oddb.org/src/util/oddbapp.rb:369:in `create_atc_class'
/var/www/oddb.org/src/util/persistence.rb:234:in `issue_create'
/var/www/oddb.org/src/util/oddbapp.rb:117:in `block in create'
/var/www/oddb.org/src/util/failsafe.rb:10:in `call'
/var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe'
/var/www/oddb.org/src/util/oddbapp.rb:116:in `create'
/var/www/oddb.org/src/command/create.rb:11:in `execute'
/var/www/oddb.org/src/util/oddbapp.rb:745:in `execute_command'
/var/www/oddb.org/src/util/oddbapp.rb:1556:in `create'
/var/www/oddb.org/src/plugin/text_info.rb:218:in `ensure_correct_atc_code'
/var/www/oddb.org/src/plugin/text_info.rb:248:in `block in update_fachinfo_lang'
/var/www/oddb.org/src/plugin/text_info.rb:237:in `each'
/var/www/oddb.org/src/plugin/text_info.rb:237:in `update_fachinfo_lang'
/var/www/oddb.org/src/plugin/text_info.rb:1345:in `parse_fachinfo'
/var/www/oddb.org/src/plugin/text_info.rb:1510:in `block in import_swissmedicinfo'
/var/www/oddb.org/src/plugin/text_info.rb:1506:in `each'
/var/www/oddb.org/src/plugin/text_info.rb:1506:in `import_swissmedicinfo'
/var/www/oddb.org/src/util/updater.rb:573:in `block in update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:549:in `call'
/var/www/oddb.org/src/util/updater.rb:549:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:567:in `update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:344:in `update_textinfo_swissmedicinfo'
jobs/update_textinfo_swissmedicinfo:39:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/update_textinfo_swissmedicinfo:12:in `<module:Util>'
jobs/update_textinfo_swissmedicinfo:11:in `<module:ODDB>'
jobs/update_textinfo_swissmedicinfo:10:in `<main>'

It choked because I got two ATC-Codes in the same FI, eg. 

<medicalInformations>
  <medicalInformation type="fi" version="1" lang="de" safetyRelevant="false" informationUpdate="03.2006">
    <title>Kalium Hausmann®</title>
    <authHolder>Vifor SA</authHolder>
    <atcCode>A12BA01, A12BA30</atcCode>
    <substances>Kalium</substances>
    <authNrs>21278, 36360</authNrs>

There are quite a few of them. Here is the list 

    <atcCode>B02BC30, lokale Hämostatika</atcCode>
    <atcCode>J01DF01, Monobactam-Antibiotikum</atcCode>
    <atcCode>N02BE01 Analgetikum, Antipyretikum</atcCode>
    <atcCode>D07AD02, D07XD</atcCode>
    <atcCode>D07AC01, D07BC01</atcCode>
    <atcCode>J05AB15, antivirale Mittel</atcCode>
    <atcCode>L01XC21, agents anti-n&amp;eacute;oplasiques, anticorp</atcCode>
    <atcCode>C05AD04 Cinchocain, D08AE02 Policresulen</atcCode>
    <atcCode>P01AB01, G01AF01</atcCode>
    <atcCode>D06AX01 (Crème/Salbe), D09AA02 (Gaze)</atcCode>
    <atcCode>B05BB02, Elektrolyte mit Kohlenhydraten</atcCode>
    <atcCode>B05BA03, Kohlenhydrate</atcCode>
    <atcCode>B05BA03, hydrate de carbone</atcCode>
    <atcCode>Actrapid A10AB01, Insulatard A10AC01</atcCode>
    <atcCode>A10AB01, A10AC01, A10AD01</atcCode>
    <atcCode>J01DH03 (carbap&amp;eacute;n&amp;egrave;me, ertap&amp;eacute;n</atcCode>
    <atcCode>A12BA01, A12BA30</atcCode>
    <atcCode>P01AB01, G01AF01</atcCode>
    <atcCode>C10AA03, HMG-CoA-Reduktasehemmer (Statin)</atcCode>
    <atcCode>B05BB01, Elektrolyte</atcCode>
    <atcCode>B05BB01, Electrolytes</atcCode>
    <atcCode>B05BA10, Lösungen zur parenteralen Ernährung, Komb</atcCode>
    <atcCode>B05BA10, Lösungen zur parenteralen Ernährung, Komb</atcCode>
    <atcCode>C09CA (Angiotensin-II-Rezeptor-Antagonisten, rein)</atcCode>
    <atcCode>A06AG04, Klysmen, Glycerol</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, HMG-CoA-Reduktasehemmer (Statin)</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>B05BB01, Elektrolyte</atcCode>
    <atcCode>Ringerlactate B05BB01, Ringerlactate + Glucose 5% </atcCode>
    <atcCode>J05AF11, substance antivirale &amp;agrave; usage syst&amp;</atcCode>
    <atcCode>C10AA03, HMG-CoA-Reduktasehemmer (Statin)</atcCode>
    <atcCode>B02BC30 (Lokale Hämostatika, Kombinationen)</atcCode>
    <atcCode>P01AB03, J01XD03</atcCode>
    <atcCode>B02BC30, lokale Hämostatika</atcCode>
    <atcCode>J01DF01, Monobactam-Antibiotikum</atcCode>
    <atcCode>N02BE01 Analgetikum, Antipyretikum</atcCode>
    <atcCode>D07AD02, D07XD</atcCode>
    <atcCode>D07AC01, D07BC01</atcCode>
    <atcCode>J05AB15, antivirale Mittel</atcCode>
    <atcCode>L01XC21, agents anti-n&amp;eacute;oplasiques, anticorp</atcCode>
    <atcCode>C05AD04 Cinchocain, D08AE02 Policresulen</atcCode>
    <atcCode>P01AB01, G01AF01</atcCode>
    <atcCode>D06AX01 (Crème/Salbe), D09AA02 (Gaze)</atcCode>
    <atcCode>B05BB02, Elektrolyte mit Kohlenhydraten</atcCode>
    <atcCode>B05BA03, Kohlenhydrate</atcCode>
    <atcCode>B05BA03, hydrate de carbone</atcCode>
    <atcCode>Actrapid A10AB01, Insulatard A10AC01</atcCode>
    <atcCode>A10AB01, A10AC01, A10AD01</atcCode>
    <atcCode>J01DH03 (carbap&amp;eacute;n&amp;egrave;me, ertap&amp;eacute;n</atcCode>
    <atcCode>P01AB01, G01AF01</atcCode>
    <atcCode>C10AA03, HMG-CoA-Reduktasehemmer (Statin)</atcCode>
    <atcCode>B05BB01, Elektrolyte</atcCode>
    <atcCode>B05BB01, Electrolytes</atcCode>
    <atcCode>B05BA10, Lösungen zur parenteralen Ernährung, Komb</atcCode>
    <atcCode>B05BA10, Lösungen zur parenteralen Ernährung, Komb</atcCode>
    <atcCode>C09CA (Angiotensin-II-Rezeptor-Antagonisten, rein)</atcCode>
    <atcCode>A12BA01, A12BA30</atcCode>
    <atcCode>A06AG04, Klysmen, Glycerol</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, HMG-CoA-Reduktasehemmer (Statin)</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>C10AA03, Inhibiteur de la HMG-CoA r&amp;eacute;ductase</atcCode>
    <atcCode>B05BB01, Elektrolyte</atcCode>
    <atcCode>Ringerlactate B05BB01, Ringerlactate + Glucose 5% </atcCode>
    <atcCode>J05AF11, substance antivirale &amp;agrave; usage syst&amp;</atcCode>
    <atcCode>C10AA03, HMG-CoA-Reduktasehemmer (Statin)</atcCode>
    <atcCode>B02BC30 (Lokale Hämostatika, Kombinationen)</atcCode>
    <atcCode>P01AB03, J01XD03</atcCode>

Fixed my problem, when duplicate ATC-Codes are given. But ignoring atcCodes like Ringerlactate B05BB01, Ringerlactate + Glucose 5%. Waiting for pushing my changes till reparse all finishes (which will take many hours to complete).

Pushed commit

view · edit · sidebar · attach · print · history
Page last modified on February 19, 2016, at 10:59 AM