Fixed FI importing when running jobs/update_textinfo_swissmedicinfo --reparse --target=both 65383 65460. Running import_daily and the test-suite before committing and pushing.
Running the import daily showed, that it did not create a new FI for 66000 correctly. I find in the email Fachinfo - DE - Pregabalin-Mepha Kapseln, Lösung zum Einnehmen["65850", "66000"] but now FI was created. This must be fixed. Packungen.xlsx of February 2016 contains 66000. Found the culprit. parse_textinfo did not check correctly whether the demanded FI/PI already existed or not. Unified searching for the concerned textinfo element.
Things look brighter now. E.g. http://oddb-ci2.dyndns.org/de/gcc/patinfo/reg/53322/seq/01/pack/028 is okay, whereas it is not shown under ch.oddb.org. But http://oddb-ci2.dyndns.org/de/gcc/fachinfo/reg/65460 still displays the FI in french. Why? We will reparse it an show, whether the problem still exists.
Pushed commit Fix parse FI/PI, Did not create correctly FI/PI
After running the import daily many (or all?) of the new patinfo point to ebixa, which should belong to the registratin 55829. But the registration registration('55829') does not have any sequences anymore.
Reverting to yesterday db to check whether this registration as wrong already before. Now bin/admin reports
ch.oddb> registration('55829').sequences.size
-> 0
The same as on thinpower. Okay I must handle this situation, too. Checking how many registrations have the same problem. On thinpower and odb-ci I found 82 registrations where no sequences where present, when looking via bin/admin
ch.oddb> $xx = registrations.values.find_all{|reg| reg.sequences.size == 0}
-> Array
ch.oddb> $xx.size
-> 82
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[0..19]
-> ["09231", "14517", "15339", "17499", "18084", "19881", "20777", "22570", "24274", "24656", "29088", "29334", "32331", "37300", "38363", "38943", "39479", "39496", "41635", "41636"]
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[20..39]
-> ["42009", "42158", "43534", "43643", "44461", "44957", "46242", "46802", "47349", "47351", "48316", "48445", "48474", "48689", "49630", "49992", "50518", "50521", "50525", "50526"]
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[40..59]
-> ["50530", "50531", "50534", "50535", "50536", "50763", "51242", "51289", "51816", "51818", "51819", "51821", "51826", "51829", "53766", "53787", "53811", "53849", "54119", "54369"]
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[60..99]
-> ["54696", "54717", "54739", "54889", "54935", "55019", "55522", "55680", "55753", "55829", "56337", "56692", "57025", "57304", "57305", "57306", "57307", "57321", "57422", "58232", "58233", "90000"]
$xx.find_all{|reg| reg.active?}.collect{|reg| reg.iksnr}.sort
-> ["55829", "57422"]
ch.oddb> registration('55829').revision_date
->
ch.oddb> registration('57422').revision_date
->
But we have only 2 active packages which have this brain damaged structure and both have a nil revision_date.
And I cannot find a package 57422 in Packungen.xlsx. Therefore I have to deal only with 55829 Ebixa which has two entries in Packungen.xlsx.
Also I must move the authNrs.each outside the parse_textinfo to avoid skipping parsing when more than one authNr is given.
The patinfo is reused during the import_daily. Breaking before each parse_textinfo and looking via bin/admin before the third IKSNR gets:
ch.oddb> registration('63166').sequences.values.first.patinfo.oid
-> 34563734
ch.oddb> registration('48561').sequences.values.first.patinfo.oid
-> 34563734
ch.oddb> registration('65427').sequences.values.first.patinfo.oid
-> undefined method `oid' for nil:NilClass
I get new oid if I call @app.create_patinfo. But this is not how the old text_info plugin worked.
Looked via bin/admin ch.oddb> $x = registrations.values.find_all{|reg| reg.active? && reg.sequences.values.first && reg.sequences.values.first.patinfo && reg.sequences.values.first.patinfo.oid == 34563734}; $x.size and found 255 occurrences of the same oid.
After replacing Persistence::Pointer.new(:patinfo).creator by @app.create_patinfo, the patinfo looks fine inside bin/admin but cannot be displayed because of the following error
NoMethodError undefined method `skeleton' for nil:NilClass /var/www/oddb.org/src/view/drugs/patinfo.rb:81:in `init' /usr/local/lib/ruby/gems/1.9.1/gems/htmlgrid-1.0.6/lib/htmlgrid/component.rb:139:in `initialize' /var/www/oddb.org/src/view/drugs/patinfo.rb:194:in `new' /var/www/oddb.org/src/view/drugs/patinfo.rb:194:in `chapter_chooser' /usr/local/lib/ruby/gems/1.9.1/gems/htmlgrid-1.0.6/lib/htmlgrid/composite.rb:68:in `create' error in SBSM::Session#to_html: /de/gcc/patinfo/reg/66000/seq/01
Looks like I need to to both things, as it displayed (albeit with some errors, eg. a missing image) after patching it manually.
The modified loop was wrong as it passed twice through each item. Simplyfing it again. Reloading database.
The problem with the loop was not the IKSNR, but that it did not check whether the given language is already present. Fixing it, makes the import_daily importing all italian PI.
The import-daily is still broken. Modified my patch to exclude italian PIs for the moment. Cannot push and find a good solution for the momen. Current state see Attach:patch_fix_pi.txt
Created /var/www/oddb.org/log/bad_pi.lst on thin_power mit bin/admin
$x = registrations.values.find_all{|reg| reg.active? && reg.sequences.values.first && reg.sequences.values.first.patinfo \
&& reg.sequences.values.first.patinfo.oid == 34563734}; $x.size
File.open("log/bad_pi.lst", 'w+'){|f| $x.each{|reg| f.puts(reg.iksnr)} }
To speed up things, I test earlier, whether reparsing is demanded or not see commit Check reparsing demanded earlier
Pushed commits
Working on adding the needed regular expression to recognize the italian chapters.