<< | Index | >>
undefined method `[]' for nil:NilClass
--
Plugin: ODDB::SwissindexPharmaPlugin Error: NoMethodError Message: undefined method `barcode' for nil:NilClass Backtrace: /var/www/oddb.org/src/plugin/swissindex.rb:189:in `block (2 levels) in update_out_of_trade' /var/www/oddb.org/src/plugin/swissindex.rb:189:in `map' /var/www/oddb.org/src/plugin/swissindex.rb:189:in `block in update_out_of_trade' /var/www/oddb.org/src/plugin/swissindex.rb:29:in `start' /var/www/oddb.org/src/plugin/swissindex.rb:185:in `update_out_of_trade' /var/www/oddb.org/src/plugin/swissindex.rb:164:in `update_package_trade_status' /var/www/oddb.org/src/util/updater.rb:515:in `block in update_notify_simple' /var/www/oddb.org/src/util/updater.rb:491:in `call' /var/www/oddb.org/src/util/updater.rb:491:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:509:in `update_notify_simple' /var/www/oddb.org/src/util/updater.rb:426:in `update_package_trade_status_by_swissindex' jobs/update_swissindex_pharma:16:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/update_swissindex_pharma:13:in `<module:Util>' jobs/update_swissindex_pharma:11:in `<module:ODDB>' jobs/update_swissindex_pharma:10:in `<main>'
We have some really weird stuff in the database, e.g.
ch.oddb> registration('274.0').iksnr
-> 274.0
ch.oddb> registration('274.0').name_base
-> Cardio-Pulmo-Rénal Sérocytol
This can also be seen via http://ch.oddb.org/de/gcc/search/zone/drugs/search_query/Cardio-Pulmo-R%C3%A9nal/search_type/st_sequence#best_result
Pushed commit Conservative programming to avoid errors in update_swissindex_pharma
I think we should call manually in bin/admin delete_registration('274.0') and update the indices afterwards.
continuing work started yesterday
Checking again via bin/admin.
ch.oddb> $suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.sequences['00'] != nil}; $suche.sort!; $suche.size
-> 36
ch.oddb> $suche[0..18]
-> ["00828", "10754", "12498", "17644", "29300", "31919", "33709", "42180", "52265", "57487", "57515", "58458", "62438", "62453", "62470", "62566", "62583", "62612", "62630"]
ch.oddb> $suche[19..36]
-> ["62658", "62672", "62711", "62785", "62786", "62904", "62920", "62976", "63029", "63030", "63200", "63257", "65040", "67769", "69870", "72736", "Zulassungs-Nummer"]
ch.oddb> $suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.sequences['00'] != nil and reg.active?}; $suche.sort!; $suche.size
-> 14
ch.oddb> $suche[0..18]
-> ["62453", "62566", "62612", "62630", "62658", "62672", "62785", "62786", "62904", "62920", "62976", "63029", "63030", "65040"]
ch.oddb> $suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.sequences['00'] != nil and not reg.active?}; $suche.sort!; $suche.size
-> 22
ch.oddb> $suche[0..20]
-> ["00828", "10754", "12498", "17644", "29300", "31919", "33709", "42180", "52265", "57487", "57515", "58458", "62438", "62470", "62583", "62711", "63200", "63257", "67769", "69870", "72736"]
ch.oddb> $suche[20..22]
-> ["72736", "Zulassungs-Nummer"]
ch.oddb> registration('00828').name_base
-> Diencéphale/Zwischenhirn Sérocytol®
ch.oddb> registration('00828').inactive_date
-> 2013-10-24
ch.oddb> registration('00828').active?
-> false
ch.oddb> registration('00828').inactive?
-> true
The IKSNR 00828 (Diencéphale/Zwischenhirn Sérocytol®) has a "Widerruf" for the 24.10.2013. The sequences['00'] must be deleted by running a bin/admin statement like
ch.oddb> $x={}; registrations.each{ |iksnr, reg| if reg.sequences['00'] != nil and not reg.active? then reg.sequences.each{ |id,seq| $x["#{reg.iksnr}.#{id}"]=seq } end }; $x.size
-> 22
ch.oddb> registrations.each{ |iksnr, reg| if reg.sequences['00'] != nil and not reg.active? then reg.sequences.each{ |id,seq| delete seq.pointer; update seq.pointer, { :pointer => nil} } end }
-> Hash
ch.oddb> $x={}; registrations.each{ |iksnr, reg| if reg.sequences['00'] != nil and not reg.active? then reg.sequences.each{ |id,seq| $x["#{reg.iksnr}.#{id}"]=seq } end }; $x.size
-> 0
ch.oddb> $x=[]; registrations.each{ |iksnr, reg| if reg.sequences['00'] != nil then reg.sequences.each{ |id,seq| $x <<iksnr } end }; $x.sort!; $x.size
-> 36
ch.oddb> $x[0..18]
-> ["62453", "62453", "62453", "62453", "62453", "62566", "62566", "62612", "62612", "62612", "62612", "62630", "62630", "62630", "62658", "62658", "62672", "62672", "62785"]
ch.oddb> $x[19..36]
-> ["62785", "62785", "62785", "62786", "62786", "62904", "62904", "62904", "62920", "62920", "62976", "63029", "63029", "63030", "63030", "65040", "65040"]
Now we have 36 new entries. This looks okay. Now I have to see, why the update_swissmedic in updater.rb did not correctly handle these situations.
Created a jobs/fix_sequences script. Adding some debug output to see, whether we can use.
Looked at 00556 "Vivotif L, Doppelbeutel" which cannot be found anymore at swissmedinfo.ch but which is in Packungen.xls.
For bisecting the problems I want to know when the duplicate registration (e.g. '556.0') were created.
ch.oddb> registration('556.0').sequences.first[1].package('001').swissmedic_source.first
-> [:import_date, #<Date: 2013-10-14 ((2456580j,0s,0n),+0s,2299161j)>]
ch.oddb> registration('00556').sequences.first[1].package('001').swissmedic_source.first
-> [:import_date, #<Date: 2013-07-06 ((2456480j,0s,0n),+0s,2299161j)>]
Answer is my birthday!
As in the registrations of the form '<y>.0' we don't find a sequence('00') they were probably created when the unchanged algorithm read the new Packungen.xls format. Also Packungen.xls contains exactly 244 lines starting with '00', which is the number of registrations that I found yesterday (see the end of the chapter http://dev.ywesee.com/Niklaus/20131028-swissindex-update#error-export-yaml). This is the double of the 112 $toDelete-fachfinfos. (Each fachinfo produces usually 2 entries in the fachinfo.yaml. One for german and one for french.)
I first test to remove the sequence '00' showed that there quite a few (about 120) registration where we have to delete sequences, e.g
/var/www/oddb.org/src/plugin/swissmedic.rb: 800 2013-10-29 21:34:07 +0100: update_registrations iksnr '00450' seqnr 02 pack 001 /var/www/oddb.org/src/plugin/swissmedic.rb: 830 2013-10-29 21:34:07 +0100: delete packung 001 in seqnr 01
Added some debug output to see where the problem stems from.
Made a simple patch to handle the situation where a pointer is nil. See Avoid accessing a nil pointer while running update_swissindex_pharma
Also split the test/util/oddbapp.rb into two files, as minitest seems to have a restriction to 200 test-cases.
undefined method `[]' for nil:NilClassThe problem is that we have a lot (> 900) invalid fachinfos which have no iksnrs attached. After discussing the problem with Zeno we agreed to make a quick patch to just skip exporting fachinfos which have no valid iksnrs.
In my opinion we should delete all fachinfo where the localized name is empty and no iksnrs are defined. Running in bin/admin
ch.oddb> $toDelete = []; fachinfos.each{ |iksnr, fi| $toDelete << fi.pointer if fi.iksnrs.size == 0 and fi.localized_name == nil}; $toDelete.size
> 112
ch.oddb> $toDelete = []; fachinfos.each{ |iksnr, fi| delete fi.pointer if fi.iksnrs.size == 0 and fi.localized_name == nil}; $toDelete.size
> 112
Now an export of the fachinfo completes without any error.
I'm wondering whether we have problems with other FI having no iksnrs. Printing them via fachinfos.each{ |iksnr, fi| $stdout.puts fi.localized_name if fi.iksnrs.size == 0 }; gives Attach:fi_with_no_iksnrs.txt with 978 entries.
Looking at two examples are
fachinfos.each{ |iksnr, fi| puts "xx #{fi.localized_name} ik #{fi.iksnrs.inspect}" if fi.localized_name.eql?('Levocetirizin-Mepha®') }; returns 5 fis, 4 with no iksrns, one with 62519
fachinfos.each{ |iksnr, fi| puts "xx #{fi.localized_name} ik #{fi.iksnrs.inspect}" if fi.localized_name.eql?('Famvir®') }; returns 4 fis, all with no iksrns. The IKSNR should be 52693.
I am running an import for 52693 and 62519. Situations is unchanged. Now I delete the pointers with fachinfos.each{ |iksnr, fi| delete fi.pointer if fi.localized_name.eql?('Famvir®') }; and run the import script again.
I am concerned that fachinfos are not correctly updated after this import.
ch.oddb> $y=[]; fachinfos.each{ |iksnr, fi| $y << "a1 #{fi.localized_name} ik #{fi.iksnrs.inspect}" if fi.localized_name.index('Levocetirizin') }; $y
-> ["a1 Levocetirizin Sandoz® ik []", "a1 Levocetirizin Sandoz® ik []", "a1 Levocetirizin Spirig HC® ik [\"62920\"]", "a1 Levocetirizin Sandoz® ik [\"62522\"]"]
ch.oddb> fi = registration('62519').fachinfo; $y = "a3 #{fi.localized_name} ik #{fi.iksnrs.inspect}" if fi.localized_name.index('Levocetirizin'); $y
-> a3 Levocetirizin-Mepha® ik ["62519"]
The namebase of a fachinfo can be nil if this no longer active. But in this case iksnrs must still contain at least one entry.
I think we should delete all fachinfos where the name_base == nil and iksnrs.size == 0. But first we must verify that we have not active registration where the fachinfo.name_base is nil.
ch.oddb> $y=[];registrations.each{ |iksnr, reg| $y << reg if reg.fachinfo and reg.active? and reg.fachinfo.name_base == nil}; $y.size
-> 0
ch.oddb> $y=[];registrations.each{ |iksnr, reg| $y << reg if reg.fachinfo and reg.fachinfo.name_base == nil}; $y.size
-> 32
ch.oddb> $y[0].iksnr
-> 42009
ch.oddb> $y[0].fachinfo.iksnrs
-> ["42009"]
ch.oddb> $y=[];registrations.each{ |iksnr, reg| $y << reg if reg.fachinfo and reg.fachinfo.iksnrs.size == 0}; $y.size
-> 0
I think we should delete all fachinfos where the name_base == nil. But first we must verify that we have the fachinfo.oid of all registration can be found via fachinfo(oid).
ch.oddb> $y=[];registrations.each{ |iksnr, reg| $y << reg if reg.fachinfo and fachinfo(reg.fachinfo.oid) == nil}; $y.size
-> 0
ch.oddb> $y=[];registrations.each{ |iksnr, reg| $y << reg if reg.fachinfo and reg.fachinfo.iksnrs.size == 0}; $y.size
-> 0
ch.oddb> $toDelete=[];fachinfos.each{ |iksnr, fi| $toDelete << fi if fi.iksnrs.size == 0}; $toDelete.size
-> 966
ch.oddb> fachinfos.each{ |iksnr, fi| delete fi.pointer if fi.iksnrs.size == 0};
Deleting takes quite a while. Now I would be very grateful if I had a test suite which would allow me to verify that I did not break something vital!
These fachinfo are not part of the orphanded_fachinfos, as demonstrated by this script
ch.oddb> $all=[]; orphaned_fachinfos.each{ |oid, fi| $all << oid if fachinfo(oid) }; $all.size
-> 0