view · edit · sidebar · attach · print · history

< Niklaus.20160314-wrong-pi-dokus | Index | Niklaus.20160301-fix-import-daily >>

20160302-fix-import-daily

Summary

  • FI and changelog of newly imported FI has wrong language
  • FI changelog was not completely translated into french
  • Allow parsing italian patinfos

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Order of entering search type and value should not matter. Both should show long URL with search
  • Remove parser for minifi (but keep the minifi)
  • BSV-Plugin does not send mail. Is it sending two mails at once?

FI and changelog of newly imported FI has wrong language

Fixed FI importing when running jobs/update_textinfo_swissmedicinfo --reparse --target=both 65383 65460. Running import_daily and the test-suite before committing and pushing.

Running the import daily showed, that it did not create a new FI for 66000 correctly. I find in the email Fachinfo - DE - Pregabalin-Mepha Kapseln, Lösung zum Einnehmen["65850", "66000"] but now FI was created. This must be fixed. Packungen.xlsx of February 2016 contains 66000. Found the culprit. parse_textinfo did not check correctly whether the demanded FI/PI already existed or not. Unified searching for the concerned textinfo element.

Things look brighter now. E.g. http://oddb-ci2.dyndns.org/de/gcc/patinfo/reg/53322/seq/01/pack/028 is okay, whereas it is not shown under ch.oddb.org. But http://oddb-ci2.dyndns.org/de/gcc/fachinfo/reg/65460 still displays the FI in french. Why? We will reparse it an show, whether the problem still exists.

Pushed commit Fix parse FI/PI, Did not create correctly FI/PI

After running the import daily many (or all?) of the new patinfo point to ebixa, which should belong to the registratin 55829. But the registration registration('55829') does not have any sequences anymore.

Reverting to yesterday db to check whether this registration as wrong already before. Now bin/admin reports

ch.oddb> registration('55829').sequences.size
-> 0

The same as on thinpower. Okay I must handle this situation, too. Checking how many registrations have the same problem. On thinpower and odb-ci I found 82 registrations where no sequences where present, when looking via bin/admin

ch.oddb> $xx = registrations.values.find_all{|reg| reg.sequences.size == 0}
-> Array
ch.oddb> $xx.size
-> 82
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[0..19]
-> ["09231", "14517", "15339", "17499", "18084", "19881", "20777", "22570", "24274", "24656", "29088", "29334", "32331", "37300", "38363", "38943", "39479", "39496", "41635", "41636"]
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[20..39]
-> ["42009", "42158", "43534", "43643", "44461", "44957", "46242", "46802", "47349", "47351", "48316", "48445", "48474", "48689", "49630", "49992", "50518", "50521", "50525", "50526"]
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[40..59]
-> ["50530", "50531", "50534", "50535", "50536", "50763", "51242", "51289", "51816", "51818", "51819", "51821", "51826", "51829", "53766", "53787", "53811", "53849", "54119", "54369"]
ch.oddb> $xx.collect{|reg| reg.iksnr}.sort[60..99]
-> ["54696", "54717", "54739", "54889", "54935", "55019", "55522", "55680", "55753", "55829", "56337", "56692", "57025", "57304", "57305", "57306", "57307", "57321", "57422", "58232", "58233", "90000"]
$xx.find_all{|reg| reg.active?}.collect{|reg| reg.iksnr}.sort
-> ["55829", "57422"]
ch.oddb> registration('55829').revision_date
-> 
ch.oddb> registration('57422').revision_date
-> 

But we have only 2 active packages which have this brain damaged structure and both have a nil revision_date.

And I cannot find a package 57422 in Packungen.xlsx. Therefore I have to deal only with 55829 Ebixa which has two entries in Packungen.xlsx.

Also I must move the authNrs.each outside the parse_textinfo to avoid skipping parsing when more than one authNr is given.

The patinfo is reused during the import_daily. Breaking before each parse_textinfo and looking via bin/admin before the third IKSNR gets:

ch.oddb> registration('63166').sequences.values.first.patinfo.oid
-> 34563734
ch.oddb> registration('48561').sequences.values.first.patinfo.oid
-> 34563734
ch.oddb> registration('65427').sequences.values.first.patinfo.oid
-> undefined method `oid' for nil:NilClass

I get new oid if I call @app.create_patinfo. But this is not how the old text_info plugin worked.

Looked via bin/admin ch.oddb> $x = registrations.values.find_all{|reg| reg.active? && reg.sequences.values.first && reg.sequences.values.first.patinfo && reg.sequences.values.first.patinfo.oid == 34563734}; $x.size and found 255 occurrences of the same oid.

After replacing Persistence::Pointer.new(:patinfo).creator by @app.create_patinfo, the patinfo looks fine inside bin/admin but cannot be displayed because of the following error

NoMethodError
undefined method `skeleton' for nil:NilClass
/var/www/oddb.org/src/view/drugs/patinfo.rb:81:in `init'
/usr/local/lib/ruby/gems/1.9.1/gems/htmlgrid-1.0.6/lib/htmlgrid/component.rb:139:in `initialize'
/var/www/oddb.org/src/view/drugs/patinfo.rb:194:in `new'
/var/www/oddb.org/src/view/drugs/patinfo.rb:194:in `chapter_chooser'
/usr/local/lib/ruby/gems/1.9.1/gems/htmlgrid-1.0.6/lib/htmlgrid/composite.rb:68:in `create'
error in SBSM::Session#to_html: /de/gcc/patinfo/reg/66000/seq/01

Looks like I need to to both things, as it displayed (albeit with some errors, eg. a missing image) after patching it manually.

The modified loop was wrong as it passed twice through each item. Simplyfing it again. Reloading database.

The problem with the loop was not the IKSNR, but that it did not check whether the given language is already present. Fixing it, makes the import_daily importing all italian PI.

The import-daily is still broken. Modified my patch to exclude italian PIs for the moment. Cannot push and find a good solution for the momen. Current state see Attach:patch_fix_pi.txt

Created /var/www/oddb.org/log/bad_pi.lst on thin_power mit bin/admin

$x = registrations.values.find_all{|reg| reg.active? && reg.sequences.values.first && reg.sequences.values.first.patinfo \
&& reg.sequences.values.first.patinfo.oid == 34563734}; $x.size
File.open("log/bad_pi.lst", 'w+'){|f| $x.each{|reg| f.puts(reg.iksnr)} }

To speed up things, I test earlier, whether reparsing is demanded or not see commit Check reparsing demanded earlier

FI changelog was not completely translated into french

Pushed commits

Allow parsing italian patinfos

Working on adding the needed regular expression to recognize the italian chapters.

view · edit · sidebar · attach · print · history
Page last modified on March 04, 2016, at 04:57 PM