<< | Index | >>
---
Import-Daily job of started 44 hours (consumed 19,5 hours of CPU time) ago still working on my VM. Found 17707 idx (rows of Packungen) in my log.
Find the following output on my console:
SBSM::InvalidDataError: e_invalid_ean_code when updating index 'oddb_package_name_with_size_company_name_and_ean13' with a ODDB::Package ["/var/www/oddb.org/src/model/ean13.rb:12:in `initialize'", "/var/www/oddb.org/src/model/ean13.rb:26:in `new'", "/var/www/oddb.org/src/model/ean13.rb:26:in `new_unchecked'", "/var/www/oddb.org/src/model/package.rb:163:in `barcode'"] [...] ODBA::Stub was unable to replace #706979 from ODDB::Package:#220866 ODBA::Stub was unable to replace #699155 from ODDB::Package:#226011 ODBA::Stub was unable to replace #18640734 from ODDB::Package:#18249535
Looking via Markenname for "MabThera" (54378) on my VM (oddb-ci.dyndns.org) I get SBSM::InvalidDataError e_invalid_ean_code. MabThera appeared on top of the fachinfo.rss feed. I get the same error when looking for Levex (61816). Looking at the two registration via bin/admin.
ch.oddb-> registration('61816').name_base
-> Levex i.v. 250
ch.oddb> registration('61816').compositions
-> [#<ODBA::Stub:61271160#26782749 @odba_class=ODDB::Composition @odba_container=61250060#26782747>, #<ODBA::Stub:60414140#26782766 @odba_class=ODDB::Composition @odba_container=60407800#26782764>]
ch.oddb> registration('61816').compositions.first
-> : Levofloxacinum 250 mg
ch.oddb> registration('61816').packages.first.barcode
-> 7680618160018
ch.oddb> registration('54378').packages.first.barcode
-> 7680543780176
Why have both registration a valid barcode (ean13) for their first package? Must I iterate over all to find problems? I get
ch.oddb> registration('54378').packages[0].barcode
-> 7680543780176
ch.oddb> registration('54378').packages[1].barcode
-> 7680543780251
ch.oddb> registration('61816').packages.size
-> 4
ch.oddb> registration('61816').packages[0].barcode
-> 7680618160018
ch.oddb> registration('61816').packages[1].barcode
-> 7680618160025
ch.oddb> registration('61816').packages[2].barcode
-> 7680618160032
ch.oddb> registration('61816').packages[3].barcode
-> 7680618160049
ch.oddb> registration('61816').packages[4].class
-> NilClass
Looks okay for me.
Looking for iksnr 62630 (Xeljanz) under oddb-ci does not find any result.
After treating my console-log I can compare a good and a row which raises the ean13 error (row 340 of the new Packungen.xls):
(good) Data_ngng:[12537.0, 2.0, "Kamillosan, Liquidum", "MEDA Pharma GmbH", "10.08.0.", "D03AX", "Phytotherapeutika", 16761.0, 36942.0, 42458.0, 77.0, "250", "ml", "D", "matricariae extractum ethanolicum liquidum", "matricariae extractum ethanolicum liquidum corresp. matricariae aetheroleum 0.5-1.5 mg corresp. levomenolum 50-150 µg et apigenini-7 glucosidum 1.75-3.25 mg, ratio: 1:4-4.5, excipiens ad solutionem pro 1 g corresp. ethanolum 43 % V/V.", "Bei kleineren Hautverletzungen", nil, 0]> (provokes error) Data_ngng:[12548.0, 1.0, "Dul-X Classic, Emulsion", "Melisana AG", "07.10.4.", "M02AX10", "Phytotherapeutika", 16771.0, 16771.0, 42199.0, 60.0, "125", "ml", "D", "levomentholum, anisi stellati aetheroleum, eucalypti aetheroleum, gaultheriae aetheroleum, citronellae aetheroleum, menthae piperitae aetheroleum, rosmarini aetheroleum, arnicae floris extractum oleosum", nil, "Einreibemittel bei Muskelschmerzen", nil, 0]> SBSM::InvalidDataError: e_invalid_ean_code when updating index 'oddb_package_name_with_size_company_name_ean13_fi' with a ODDB::Package
I remarked that the ordering of Verpackungsid for (12537) "Kamillosan, Liquidum": was 077, 042, 034, now it is 034, 042, 077. Does this influence our algorithm? In any case http://oddb-ci.dyndns.org/de/gcc/drug/reg/12537/seq/02 does not find any packages after running import_daily. That is not good.
For IKSNR 00635 (Luivac) I find still two packages for the given sequence.
Tried to fall into a pry session when getting the error. This fails with an Ruby exception, see Attach:import_daily_8.txt
Running swissmedic-diff for packungen.xls of June-25 and Oktobre-7 returns 13438 lines! E.g. + 00274: Cardio-Pulmo-Rénal Sérocytol, suppositoire - 274.0: Cardio-Pulmo-Rénal Sérocytol, suppositoire Therefore we have to fix this gem.
This was quite easy. First pushed a commit Updated Gem for move to github and travis integration.Then copied a recent Packungen.xls to test/data. Removed most of the content to have just a few differences. Made the tests fail and now they are green. Pushed commit Auto-adapt to new format of swissmedicinfo
In src/plugin/swissmedic.rb there are various places where we use the magic constant '3' to skip the first 4 lines of Packungen.xls. Also setting iksnr to a 5 digit string must be performed various times. I think I must come up with a better solution to add a new utility method, eg. each_iksnr_row in swissmedic-diff.
Refactored with this commit Refactored. Added utility each_valid_row. Verified that now the difference look a lot better for the packages of June and Octobre. But we still have over 7'000 changes, but only 37 new ones and about 80 deletions.
Now refactoring the oddb.org/src/plugin/swissmedic.rb. Must be refactoring even more, as Präparateliste has also now 4 lines to skip!
Refactoring oddb.org/src/plugin/swissmedic.rb complicates the corresponding tests even further. Adding some skips and running the import_daily on my VM. Will see result tomorrow morning.
Results of running (time 36 minutes). Logging ist too verbost (530 MB). On line 139 of the log I see that the diff is calculated. Fails updating the 39-th registration. Stack is
Plugin: ODDB::SwissmedicPlugin Error: NoMethodError Message: undefined method `gsub' for nil:NilClass Backtrace: /var/www/oddb.org/src/plugin/swissmedic.rb:565:in `update_compositions' /var/www/oddb.org/src/plugin/swissmedic.rb:800:in `block in update_registrations' /var/www/oddb.org/src/plugin/swissmedic.rb:796:in `each' /var/www/oddb.org/src/plugin/swissmedic.rb:796:in `update_registrations' /var/www/oddb.org/src/plugin/swissmedic.rb:53:in `update' /var/www/oddb.org/src/util/updater.rb:387:in `block in update_swissmedic' /var/www/oddb.org/src/util/updater.rb:488:in `call' /var/www/oddb.org/src/util/updater.rb:488:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:385:in `update_swissmedic' /var/www/oddb.org/src/util/updater.rb:195:in `run' /var/www/oddb.org/jobs/import_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' /var/www/oddb.org/jobs/import_daily:12:in `<module:Util>' /var/www/oddb.org/jobs/import_daily:11:in `<module:ODDB>' /var/www/oddb.org/jobs/import_daily:10:in `<main>'