view · edit · sidebar · attach · print · history

Index>

20151009-fix-swissmedic-import

Summary

  • Fix nil error in swissmedic-import
  • Non-SL medis must be last for evidentia

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • Create a history of FI changes and add a nice diff (colored) to compare any two versions. Look at gems like diffy or differ
  • Ofev (IKSNR 65330) has a not yet WHO-Approved ATC-Code L01XE31. The job atc_less should add the new ATC-Code with an empty name and flag it in its report for human intervention (e.g. extracting the substance name from the FI)
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Get ATC-Code from FI and compare it to the XML and Packungen.xlsx

Non-SL medis must be last for evidentia

When looking at http://evidentia.oddb-ci2.dyndns.org/de/evidentia/search/zone/drugs/search_query/keppra/search_type/st_oddb?#best_result then IKNSR 62616 evetiracetam Desitin 500 mg/5 mL Levetiracetamum Infusionskonzentrat 500mg / 5ml 10 Durchstechflasche(n) must be after all SL entries. Why is it before Levetiracetam Desitin 250 mg Levetiracetamum Minipacks mit Mini-Filmtabletten 250 mg 30 Tablette(n) 13.49 27.80 10% Desitin Pharma GmbH B / SL (IKSNR 62069)

Neither spec/evidentia_spec.rb nor test/test_util/resultsort.rb catch this error. Why?

Used gem transpec to transform all rspec2 tests to rspec3. Added a new test for evidentia Avoid NoMethodError for active agents after Using new rspec3 expect

Added another test in test/test_util/resultsort.rb.

But it looks as if the database is not consistent. E.g bin/admin reports 

ch.oddb> registration('61848').packages.collect{|x| [x.sl_generic_type,x.generic_type,x.sl_entry.class]}[0..3]
-> [[nil, :generic, NilClass], [:generic, :generic, ODDB::SlEntry]]
ch.oddb> registration('62616').packages.collect{|x| [x.sl_generic_type,x.generic_type,x.sl_entry.class]}[0..3]
-> [[:generic, nil, NilClass], [nil, nil, NilClass]]
ch.oddb> registration('62069').packages.collect{|x| [x.sl_generic_type,x.generic_type,x.sl_entry.class]}[0..3]
-> [[nil, :unknown, ODDB::SlEntry], [nil, :unknown, ODDB::SlEntry], [nil, :unknown, ODDB::SlEntry], [nil, :unknown, ODDB::SlEntry]]
ch.oddb> registration('62069').packages.collect{|x| [x.sl_generic_type,x.generic_type,x.sl_entry.class]}[4..7]
-> [[nil, :unknown, ODDB::SlEntry], [nil, :unknown, ODDB::SlEntry], [nil, :unknown, ODDB::SlEntry]]

Looks like the BSV-Import did not update correctly all packages. Trying a fix locally, as the code analysis showed that the XML-Tag OrgGenCode comes from the registration, but is not updated in each package. Will think about whether we should not better move the sl_generic_type to the registration and update the packages correspondingly. However I don't know exactly how I can migrate the database.

Fix nil error in swissmedic-import

Must fix the following error when running swissimport 

Plugin: ODDB::SwissmedicPlugin
Error: NoMethodError
Message: undefined method `find' for nil:NilClass
Backtrace:
/var/www/oddb.org/src/model/composition.rb:48:in `inactive_agent'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in
`method_missing'
/var/www/oddb.org/src/plugin/swissmedic.rb:623:in `update_active_agent'
/var/www/oddb.org/src/plugin/swissmedic.rb:769:in `block (2 levels) in
update_compositions'
/var/www/oddb.org/src/plugin/swissmedic.rb:746:in `each'
/var/www/oddb.org/src/plugin/swissmedic.rb:746:in `each_with_index'
/var/www/oddb.org/src/plugin/swissmedic.rb:746:in `block in update_compositions'
/var/www/oddb.org/src/plugin/swissmedic.rb:739:in `each'
/var/www/oddb.org/src/plugin/swissmedic.rb:739:in `each_with_index'
/var/www/oddb.org/src/plugin/swissmedic.rb:739:in `update_compositions'
/var/www/oddb.org/src/plugin/swissmedic.rb:1049:in `update_all_sequence_info'
/var/www/oddb.org/src/plugin/swissmedic.rb:1073:in `block in
update_registrations'
/var/www/oddb.org/src/plugin/swissmedic.rb:1060:in `each'
/var/www/oddb.org/src/plugin/swissmedic.rb:1060:in `update_registrations'
/var/www/oddb.org/src/plugin/swissmedic.rb:167:in `update'
/var/www/oddb.org/src/util/updater.rb:444:in `block in update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:549:in `call'
/var/www/oddb.org/src/util/updater.rb:549:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:442:in `update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:201:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

Looking at log/debug/oddb/2015/10.log I find it concerns the registration 57514 

2015-10-19 07:33:47 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:1067 update iksnr 57514 seqnr 01 true opts {}
2015-10-19 07:33:48 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:743 update_compositions iksnr 57514 seqnr 01 comp_idx 0 parsed_comp #<ParseComposition:0x000001503446e8>
2015-10-19 07:33:48 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 753 57514/01 using oid 9435288 source ginseng extractum ethanolicum siccum 220 mg corresp
2015-10-19 07:33:48 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:611 update_active_agent 57514/01 Ginseng Extractum Ethanolicum Siccum  true dose 220 mg
2015-10-19 07:33:48 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 753 57514/01 using oid 9435288 source ginseng extractum ethanolicum siccum 220 mg corresp
2015-10-19 07:33:48 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:611 update_active_agent 57514/01 DER: 10-15:1  false dose 0

With my DB I get the following error in bin/admin 

ch.oddb> registration('57514').compositions.first.cleanup_old_active_agent
-> undefined local variable or method `reg' for Kapseln: Ginseng Extractum Ethanolicum Siccum 220 mg:ODDB::Composition

After fixing this problem with a commit I was able to run the bin/admin command and did see on oddbd output cleanup_old_active_agent. Skipping inactive registration 57514 expiration #<Date: 2015-06-27 ((2457201j,0s,0n),+0s,2299161j)>.

Running now import_daily. Got errors import patinfo, swissmedic and sl-update. But not the same as on thinpower. Dropping and reloading database of thinpower.

Got now another error with registration 57514. 

Plugin: ODDB::SwissmedicPlugin
Error: NoMethodError
Message: undefined method `oid' for #<NoMethodError: undefined method `push' for nil:NilClass>
Backtrace:
/var/www/oddb.org/src/plugin/swissmedic.rb:646:in `update_active_agent'
/var/www/oddb.org/src/plugin/swissmedic.rb:769:in `block (2 levels) in update_compositions'
/var/www/oddb.org/src/plugin/swissmedic.rb:746:in `each'
/var/www/oddb.org/src/plugin/swissmedic.rb:746:in `each_with_index'
/var/www/oddb.org/src/plugin/swissmedic.rb:746:in `block in update_compositions'
/var/www/oddb.org/src/plugin/swissmedic.rb:739:in `each'
/var/www/oddb.org/src/plugin/swissmedic.rb:739:in `each_with_index'
/var/www/oddb.org/src/plugin/swissmedic.rb:739:in `update_compositions'
/var/www/oddb.org/src/plugin/swissmedic.rb:1049:in `update_all_sequence_info'
/var/www/oddb.org/src/plugin/swissmedic.rb:1073:in `block in update_registrations'
/var/www/oddb.org/src/plugin/swissmedic.rb:1060:in `each'
/var/www/oddb.org/src/plugin/swissmedic.rb:1060:in `update_registrations'
/var/www/oddb.org/src/plugin/swissmedic.rb:167:in `update'
/var/www/oddb.org/src/util/updater.rb:444:in `block in update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:549:in `call'
/var/www/oddb.org/src/util/updater.rb:549:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:442:in `update_swissmedic'
jobs/import_swissmedic:14:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_swissmedic:12:in `<module:Util>'
jobs/import_swissmedic:11:in `<module:ODDB>'
jobs/import_swissmedic:10:in `<main>'

log/debug/oddb/2015/10.log contains 

2015-10-19 10:46:25 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:1067 update iksnr 57514 seqnr 01 true opts {}                                                                                                                                               
2015-10-19 10:46:26 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:743 update_compositions iksnr 57514 seqnr 01 comp_idx 0 parsed_comp #<ParseComposition:0x000000937f9d40>                                                              
2015-10-19 10:46:26 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 753 57514/01 using oid 9435288 source ginseng extractum ethanolicum siccum 220 mg corresp                                                                                                  
2015-10-19 10:46:26 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:611 update_active_agent 57514/01 Ginseng Extractum Ethanolicum Siccum  true dose 220 mg                                                                                                  
2015-10-19 10:46:26 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 753 57514/01 using oid 9435288 source ginseng extractum ethanolicum siccum 220 mg corresp                                                                                                  
2015-10-19 10:46:27 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb:611 update_active_agent 57514/01 DER: 10-15:1  false dose 0

Okay it looks as we cannot parse correctly the substance for 57514. Its name is nil. Why. Parsing ginseng extractum ethanolicum siccum 220 mg seems to work in oddb2xml. No the problem is caused by wrong debug message. Removing it and the import swissmedic continues.

Pushed commits

view · edit · sidebar · attach · print · history
Page last modified on October 19, 2015, at 05:41 PM