view · edit · sidebar · attach · print · history

< Niklaus.20160112-fix-migel-import | Index | 2015 >>

20160111-missing-cool-in-vaccines

Summary

  • Column M Kühlkette missing in vaccines from SwissMedic
  • Fix import fachinfo
  • Migel imports don't work anymore
  • Why do we have so many results for Adenosin?

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Order of entering search type and value should not matter. Both should show long URL with search
  • Remove parser for minifi (but keep the minifi)

Column M Kühlkette missing in vaccines from SwissMedic

End of december Swissmedic dropped support for marking vaccines needed a special cooling conditions. Their argument is

"Der Grund für das nicht mehr publizieren dieser Spalte liegt darin, dass die Angaben nicht zu logistischen Zwecken gemäss GDP Anforderungen verwendbar sind und das hat zu Missverständnissen geführt, deshalb mussten wir entscheiden, diese Informationen nicht mehr zu publizieren. Die genauen Angaben zu Lagerung der Produkte sind in den jeweiligen Arzneimitteltexten und auf den Packmitteln aufgeführt. Erklärung: In der von Ihnen erwähnten Spalte kann nur erwähnt werden, ob Kühlkettenpflichtig oder nicht; es trifft aber nicht immer zu, wenn ein Produkt in der Spalte als „nicht Kühlkettenpflichtig“ eingestuft wird, auch bei Raumtemperatur transportiert werden darf. Die Lagerungsbedingungen sind z.T. sehr differenziert, deshalb ist es eher verwirrend wenn Swissmedic diese Angaben so publiziert."

This problem must be fixed.

It is easy to remove the COOL from parsing the the fridge.xlsx. It make however afterwards no sense to parse it all. zurrose_transfer.dat contains the field CLAG with the following definition

Feld
CLAG
0 = keine Einschränkung
1 = Lagerung im Kühlschrank
2 = Lagerung im Gefrierraum

Until now it is only used when we have also an entry from the fridge.xlsx. This should probably changed, too.

Also I just remarked that in my spec tests I expect 7680586180018 to be an article having the COOL attribute and it does not show up in the zurrose_transfer.dat. The Fachinfo contains in "Sonstige Hinweise" "Im Tiefkühlschrank bei <-20 °C lagern. Die Kühlkette bis zur Anwendung nicht unterbrechen."

After a discussion with Zeno we agreed on the following:

  • Drop importing the file bloodproducts and vaccines
  • Mark all bloodproducts and vaccines having COOL=1 (No harm, if some could be stored at normal temperature)

Pushed commits

Migel imports don't work anymore

One of my commits broke running the Migel imports. E.g

NoMethodError
undefined method `session' for ODDB::Swissindex:Module
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/invokemethod.rb:14:in `perform_with_block'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1506:in `perform'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
/var/www/migel/lib/migel/plugin/swissindex.rb:36:in `block in save_all_products'
/usr/local/lib/ruby/1.9.1/csv.rb:1367:in `open'
/var/www/migel/lib/migel/plugin/swissindex.rb:35:in `save_all_products'
/var/www/migel/lib/migel/util/importer.rb:350:in `save_all_products'
/var/www/migel/lib/migel/util/importer.rb:234:in `reported_save_all_products'
/var/www/migel/lib/migel/util/importer.rb:224:in `block in save_all_products_all_languages'
/var/www/migel/lib/migel/util/importer.rb:221:in `each'
/var/www/migel/lib/migel/util/importer.rb:221:in `save_all_products_all_languages'
jobs/update_migel_products_with_report:13:in `block in <module:Util>'
/var/www/migel/lib/migel/util/job.rb:22:in `call'
/var/www/migel/lib/migel/util/job.rb:22:in `run'
jobs/update_migel_products_with_report:12:in `<module:Util>'
jobs/update_migel_products_with_report:11:in `<module:Migel>'
jobs/update_migel_products_with_report:10:in `<main>'

The migel import do not work anymore, because some classes were renamed and not tested.

Fach fachinfo import

Must fix the following error

Plugin: ODDB::TextInfoPlugin
Error: NoMethodError
Message: undefined method `oid' for #<ArgumentError: wrong number of arguments (0 for 1)>
Backtrace:
Plugin: ODDB::TextInfoPlugin
Error: NoMethodError
Message: undefined method `oid' for #<ArgumentError: wrong number of arguments (0 for 1)>
Backtrace:
/var/www/oddb.org/src/plugin/text_info.rb:209:in `block in ensure_correct_atc_code'
/var/www/oddb.org/src/plugin/text_info.rb:207:in `each'
/var/www/oddb.org/src/plugin/text_info.rb:207:in `ensure_correct_atc_code'
/var/www/oddb.org/src/plugin/text_info.rb:267:in `block in update_fachinfo'
/var/www/oddb.org/src/plugin/text_info.rb:258:in `each'
/var/www/oddb.org/src/plugin/text_info.rb:258:in `update_fachinfo'
/var/www/oddb.org/src/plugin/text_info.rb:1203:in `parse_and_update'
/var/www/oddb.org/src/plugin/text_info.rb:1213:in `block (3 levels) in import_info'
/var/www/oddb.org/src/plugin/text_info.rb:1212:in `each'
/var/www/oddb.org/src/plugin/text_info.rb:1212:in `block (2 levels) in import_info'
/var/www/oddb.org/src/plugin/text_info.rb:1211:in `each_pair'
/var/www/oddb.org/src/plugin/text_info.rb:1211:in `block in import_info'
/var/www/oddb.org/src/plugin/text_info.rb:1208:in `each_pair'
/var/www/oddb.org/src/plugin/text_info.rb:1208:in `import_info'
/var/www/oddb.org/src/plugin/text_info.rb:1416:in `block in import_swissmedicinfo_by_index'
/var/www/oddb.org/src/plugin/text_info.rb:1415:in `each_pair'
/var/www/oddb.org/src/plugin/text_info.rb:1415:in `import_swissmedicinfo_by_index'
/var/www/oddb.org/src/plugin/text_info.rb:1477:in `import_swissmedicinfo'
/var/www/oddb.org/src/util/updater.rb:573:in `block in update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:549:in `call'
/var/www/oddb.org/src/util/updater.rb:549:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:567:in `update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:344:in `update_textinfo_swissmedicinfo'
/var/www/oddb.org/src/util/updater.rb:197:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

Running locally I got

Error: NoMethodError
Message: undefined method `to_date' for nil:NilClass
Backtrace:
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:323:in `_comparable'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:265:in `block in rows_diff'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:262:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:262:in `each_with_index'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:262:in `rows_diff'
/var/www/oddb.org/src/plugin/swissmedic.rb:615:in `rows_diff'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:179:in `block in diff'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:399:in `block in each_valid_row'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyXL-3.3.1/lib/rubyXL/worksheet.rb:23:in `block in each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyXL-3.3.1/lib/rubyXL/worksheet.rb:23:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyXL-3.3.1/lib/rubyXL/worksheet.rb:23:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:386:in `each_valid_row'
/usr/local/lib/ruby/gems/1.9.1/gems/swissmedic-diff-0.2.1/lib/swissmedic-diff.rb:153:in `diff'
/var/www/oddb.org/src/plugin/swissmedic.rb:201:in `update'
/var/www/oddb.org/src/util/updater.rb:444:in `block in update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:549:in `call'
/var/www/oddb.org/src/util/updater.rb:549:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:442:in `update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:201:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'

It looks to me that both errors stem from the fact that we have a very peculiar fachinfo Erythrozytenkonzentrat (EK) leukozytendepletiert, in additiver Lösung which contains no IKSNR. Dropping and reloading the database to see how we react to my changes.

With my local database (not yet having Erythrozytenkonzentrat) the FI import passed fine. I got in the e-mail


1 FIs without an ATC-code
00000

Now I got the T_NODE error on oddb-ci2, too

 und Patienteninfo Updates (swissmedicinfo.ch) - 01/2016 size 2326 with 0 attachments. Plugin: ODDB::TextInfoPlugin
Error: NotImplementedError
Message: method `method_missing' called on unexpected T_NODE object (0x0000000b8000d8 flags=0x23c klass=0x0)
Backtrace:
/var/www/oddb.org/src/model/registration_observer.rb:36:in `iksnrs'

T_NODE looks like a very deeply embedded type of Ruby, eg. debugger/ruby_core_source/ruby-1.9.3-p194/vm_core.h:#define RUBY_VM_IFUNC_P(ptr) (BUILTIN_TYPE(ptr) == T_NODE)

Now the import completed without an error. Loading thinpowers database and trying again. Rescue the problem over the whole method item_to_html, not just a small part of it.

Got the swissmedic-diff error. Adding some debug output and running jobs/import_swissmedic again. Made some changes to swissmedic-diff to ignore empty date fiels (as they are present with the wrong iksnr 0 registration. Now the import_swissmedic proceeeds a lot longer. Waiting for it finish before pushing the changes.

Pushed the following commits to swissmedic-diffs

Pushed commits

Why do we have so many results for Adenosin?

When search via price we get 46 results. Eg. why does IKNSR 52701 Grosse Apotheke Dr. G. Bichsel AG - Aqua ad injectabilia "Bichsel", sterile Lösung appear?

Digging via bin/admin

ch.oddb> $result_substances = search_exact_substance('Adenosin')
-> #<ODDB::SearchResult:0x000000082c6610>
ch.oddb> $result_substances.atc_classes
-> [Andere Diagnostika, Adenosin, Lösungs- und Verdünnungsmittel, inkl. Spüllösungen]
ch.oddb> $result_substances.atc_classes.first
-> Andere Diagnostika
ch.oddb> $result_substances.atc_classes.first.code
-> V04CX
ch.oddb> $result_substances.atc_classes.last.code
-> V07AB
ch.oddb> registration('52107').atc_class.code
-> V07AB
ch.oddb> registration('51567').name_base
-> Krenosin
ch.oddb> registration('51567').atc_class
-> Adenosin
ch.oddb> registration('51567').atc_class.code
-> C01EB10

Okay. Now why do we find V07AB Lösungs- und Verdünnungsmittel, inkl. Spüllösungen when looking for Adenosin?

ch.oddb> $sequences = ODBA.cache.retrieve_from_index('substance_index_sequence', 'Adenosin')
-> [Krenosin diagnostic, solution injectable, solution injectable, Krenosin, solution injectable, Viaspan, Lösung zur Konservierung von Organtransplantaten]
ch.oddb> $sequences.collect{|x| x.iksnr}
-> ["53566", "51567", "57387"]
ch.oddb> registration('53566').atc_class
-> Andere Diagnostika
ch.oddb> registration('53566').atc_class.code
-> V04CX
ch.oddb> registration('51567').atc_class.code
-> C01EB10
ch.oddb> registration('57387').atc_class.code
-> V07AB
ch.oddb> registration('57387').active?
-> false

Here is my result. The substance_index_sequence-index considers also non active registrations. And in the old registration 57387 there was a wrong ATC-Code.

view · edit · sidebar · attach · print · history
Page last modified on January 11, 2016, at 05:34 PM