view · edit · sidebar · attach · print · history

< Niklaus.20160323-textinfo-pi | Index | Niklaus.20160321-fix-export-errors >>

20160322-fix-export-error

Summary

  • Fix errors in export, where some packages are nil

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Order of entering search type and value should not matter. Both should show long URL with search
  • Remove parser for minifi (but keep the minifi)
  • BSV-Plugin does not send mail. Is it sending two mails at once?
  • Ebixa: bin/admin returns nil for registration('55829').sequences.values.first.iksnr.
  • PI/FI update not correct.

Fix errors in export, where some packages are nil

Corrected rescue in src/view/drugs/csv_result.rb to cover more fields. Rerunning import_bsv_followers. This is not enough. Creating a rescue block where for the whole line being emitted. Rerunning import_bsv_followers.

Looks better now, but we still have an error in Generikaliste. Analysing this one, too.

The following 7 packages have problems

niklaus@ng-tr /o/s/oddb.org> grep missing oddb*.csv 
oddb.csv:#Medi;00610;011;00610011;7680006100114;;;Gen H-B-Vax 5;Suspension;5 µg;1;1;;;Sanofi Pasteur MSD AG;B;Ja;missing sl_entry package ean code=7680006100114;Ja;;Für Hepatitis-B-Reihenimpfungen von Jugendlichen im Alter von 11 - 15 Jahren nach dem 2-Dosenschema 10 mcg (s. Art. 12 Bst. k Ziffer 2 KLV) gilt der Preis von Fr. 22.50 für eine Dosis unter der Voraussetzung einer minimalen Bestellung von 100 Dosen an eine Lieferadresse.;Nein;;30.12.2006;;;"";;Nein;10%;Ja;;;;Nein;Ja;Nein
oddb.csv:#Medi;55645;008;55645008;7680556450080;;;Iscador M 5 mg spez.;Injektionslösung;5 mg;8 Ampulle(n);8;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450080;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;57.16.10.;;Nein;Nein;Nein
oddb.csv:#Medi;55645;006;55645006;7680556450066;;;Iscador M 2 mg spez.;Injektionslösung;2 mg;8 Ampulle(n);8;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450066;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;57.16.10.;;Nein;Nein;Nein
oddb.csv:#Medi;55645;004;55645004;7680556450042;;;Iscador M 1 mg spez.;Injektionslösung;1 mg;8 Ampulle(n);8;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450042;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;57.16.10.;;Nein;Nein;Nein
oddb.csv:#Medi;55645;028;55645028;7680556450288;;;Iscador Qu 5 mg spez.;Injektionslösung;5 mg;8 Ampulle(n);8;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450288;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;57.16.10.;;Nein;Nein;Nein
oddb.csv:#Medi;55645;026;55645026;7680556450264;;;Iscador Qu 2 mg spez.;Injektionslösung;2 mg;8 Ampulle(n);8;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450264;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;57.16.10.;;Nein;Nein;Nein
oddb.csv:#Medi;55645;024;55645024;7680556450240;;;Iscador Qu 1 mg spez.;Injektionslösung;1 mg;8 Ampulle(n);8;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450240;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;57.16.10.;;Nein;Nein;Nein
oddb2.csv:#Medi;00610;011;00610011;7680006100114;;;Gen H-B-Vax 5;Suspension;Suspension;5 µg;1;0;;;Sanofi Pasteur MSD AG;B;Ja;missing sl_entry package ean code=7680006100114;Ja;;Für Hepatitis-B-Reihenimpfungen von Jugendlichen im Alter von 11 - 15 Jahren nach dem 2-Dosenschema 10 mcg (s. Art. 12 Bst. k Ziffer 2 KLV) gilt der Preis von Fr. 22.50 für eine Dosis unter der Voraussetzung einer minimalen Bestellung von 100 Dosen an eine Lieferadresse.;Nein;;30.12.2006;;;"";;Nein;10%;Ja;;;unbekannt;unbekannt
oddb2.csv:#Medi;55645;008;55645008;7680556450080;;;Iscador M 5 mg spez.;Injektionslösung;Solution injectable;5 mg;8 Ampulle(n);0;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450080;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;P;Injektion/Infusion;Injection/Infusion
oddb2.csv:#Medi;55645;006;55645006;7680556450066;;;Iscador M 2 mg spez.;Injektionslösung;Solution injectable;2 mg;8 Ampulle(n);0;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450066;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;P;Injektion/Infusion;Injection/Infusion
oddb2.csv:#Medi;55645;004;55645004;7680556450042;;;Iscador M 1 mg spez.;Injektionslösung;Solution injectable;1 mg;8 Ampulle(n);0;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450042;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;P;Injektion/Infusion;Injection/Infusion
oddb2.csv:#Medi;55645;028;55645028;7680556450288;;;Iscador Qu 5 mg spez.;Injektionslösung;Solution injectable;5 mg;8 Ampulle(n);0;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450288;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;P;Injektion/Infusion;Injection/Infusion
oddb2.csv:#Medi;55645;026;55645026;7680556450264;;;Iscador Qu 2 mg spez.;Injektionslösung;Solution injectable;2 mg;8 Ampulle(n);0;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450264;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;P;Injektion/Infusion;Injection/Infusion
oddb2.csv:#Medi;55645;024;55645024;7680556450240;;;Iscador Qu 1 mg spez.;Injektionslösung;Solution injectable;1 mg;8 Ampulle(n);0;;52.60;Weleda AG;A;Ja;missing sl_entry package ean code=7680556450240;;;;Nein;13.11.1996;;01.12.2005;;"";;Nein;10%;Ja;A;P;Injektion/Infusion;Injection/Infusion

They belong to IKSNR 55645 (6) and 00610. IKSNR 55645 has "Widerruf der Zulassung 01.12.2005". Therefore it should be no longer active. And IKSRN 00610 is not int the Packungen.xlsx and has "Gültig bis 30.12.2006". Looking at bin/admin I see the following discrepancy between the status active? of the registration of the package

ch.oddb> registration('55645').packages.first
-> #<ODDB::Package:0x00000005838038>
ch.oddb> registration('55645').packages.collect{|x| x.active?}
-> [true, true, true, true, true, true]

Where does it stem from?

Found via bin/admin 8 packages where the registration is an Array via

 $x= []; each_package{|pac| $x << pac if pac.registration && pac.registration.is_a?(Array)}
ch.oddb> $x.first.pointer
-> :!registration,49703!sequence,01!package,101.
ch.oddb> $x[1].pointer
-> :!registration,49703!sequence,01!package,102.
ch.oddb> $x[2].pointer
-> :!registration,49703!sequence,02!package,099.
ch.oddb> $x[3].pointer
-> :!registration,49703!sequence,02!package,100.
ch.oddb> $x[4].pointer
-> :!registration,49703!sequence,03!package,097.
ch.oddb> $x[5].pointer
-> :!registration,49703!sequence,03!package,098.
ch.oddb> $x[6].pointer
-> :!registration,50473!sequence,02!package,031.
ch.oddb> $x[7].pointer
-> :!registration,50473!sequence,02!package,058.

Pushed commits

Fix error ebixa

http://oddb-ci2.dyndns.org/de/gcc/drug/reg/55829/seq/01 displays NoMethodError undefined method `invoiceable?' for nil:NilClass.

Analysing the situation with bin/admin shows

ch.oddb> registration('55829').sequences.values.first.registration.class
-> NilClass
## Is it the only one?
ch.oddb> $z = registrations.values.find_all{|x| x.active? && x.sequences.values.first && x.sequences.values.first.registration.kind_of?(NilClass)}
ch.oddb> $z.size
-> 1
## yes
ch.oddb> $z.first.iksnr
-> 55829
ch.oddb> $z.first.name_base
-> Ebixa

Fixing it with the following bin/admin command

registration('55829').sequences.values.first.registration = registration('55829')
registration('55829').odba_store

Still unclear, what produced this wrong entry. Runnig the same command on thinpower. http://ch.oddb.org/de/gcc/drug/reg/55829/seq/01 works now again.

Now the search via trademark ebixa still fails, because of an nil composition in a package with Pack-Number 888. Do we have such a package in Packungen.xlsx? Yes. We have Ebixa sequence 3 with 15 mg, Filmtabletten. But we cannot find this sequence via bin/admin. And it has the comment "Tablette(n) nur als Kombipackung: siehe Seq.-Nr.06", but there is no sequence 06!

Fixed the problem with the bin/admin statements

registration('55828').sequence('03').package('888').sequence = registration('55828').sequence('03');
registration('55828').sequence('03').odba_store

Looks like my insert of a new package was not correct. As I found in 03.log 2016-03-16 17:06:16 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb:990:in `update_package': create 55828/03/888 ptr :!create,:!registration,55828!sequence,03!package,888.. package #<ODDB::Package:0x0000031ea900d0>. Analysing my code again.

Using create_package, create_part and create_sequence from src/model in plugin/textinfo.rb and plugin/swissmedic.rb. When running PI-Import I got the following exception

/usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:411: [BUG] Segmentation fault
ruby 1.9.3p545 (2014-02-24 revision 45159) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0022 p:---- s:0078 b:0078 l:000077 d:000077 CFUNC  :===
c:0021 p:0035 s:0074 b:0073 l:000062 d:000072 BLOCK  /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:411
c:0020 p:---- s:0068 b:0068 l:000067 d:000067 FINISH
c:0019 p:---- s:0066 b:0066 l:000065 d:000065 CFUNC  :each
c:0018 p:0053 s:0063 b:0063 l:000062 d:000062 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:404
c:0017 p:0034 s:0057 b:0057 l:000046 d:000056 BLOCK  /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:98
c:0016 p:---- s:0052 b:0052 l:000051 d:000051 FINISH
c:0015 p:---- s:0050 b:0050 l:000049 d:000049 CFUNC  :delete_if
c:0014 p:0073 s:0047 b:0047 l:000046 d:000046 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:96
c:0013 p:0049 s:0042 b:0042 l:000028 d:000041 BLOCK  /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:105
c:0012 p:---- s:0039 b:0039 l:000038 d:000038 FINISH
c:0011 p:---- s:0037 b:0037 l:000036 d:000036 CFUNC  :each_value
c:0010 p:0011 s:0034 b:0034 l:000028 d:000033 BLOCK  /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:102
c:0009 p:0019 s:0032 b:0032 l:000031 d:000031 METHOD <internal:prelude>:10
c:0008 p:0050 s:0029 b:0029 l:000028 d:000028 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:101
c:0007 p:0139 s:0021 b:0021 l:000020 d:000020 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:81
c:0006 p:0030 s:0014 b:0014 l:0009c8 d:000013 BLOCK  /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:490
c:0005 p:---- s:0011 b:0011 l:000010 d:000010 FINISH
c:0004 p:---- s:0009 b:0009 l:000008 d:000008 CFUNC  :loop
c:0003 p:0040 s:0006 b:0006 l:0009c8 d:000005 BLOCK  /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:487
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:---- s:0002 b:0002 l:000001 d:000001 TOP   

Also remarked that it tried to recreate a registration 10754 "Magnesia S. Pellegrino®", but not with the correct ikscd 099, but with 00.

Comment of masa in segmentation fault is "Note: It is difficult to reproduce segmentation fault". I agree with this statement, as I tried several times to run the same action but never got a consistent Segmentation fault.

Dropping and reloading database. Trying to fix the error above and adding a break into pry for it.

Looks like attaching different patinfos does not work correctly in all cases. E.g. for IKSRN 54534 Tebokan we have two different Patinfo for 80 and 120. But looking in the log it looks as if we are always using the same

2016-03-22 17:06:25 +0100: /var/www/oddb.org/src/plugin/text_info.rb:265:in `store_patinfo_for_one_packages': store_patinfo_for_one_packages 54534 de 1105 :!patinfo,1105. ["Tebokan® 80", "Was ist Tebokan 80 und wann wird es angewendet?"]
store_patinfo_for_one_packages 54534 de 1105 :!patinfo,1105. ["Tebokan® 80", "Was ist Tebokan 80 und wann wird es angewendet?"]
2016-03-22 17:06:25 +0100: /var/www/oddb.org/src/plugin/text_info.rb:265:in `store_patinfo_for_one_packages': store_patinfo_for_one_packages 54534 de 1105 :!patinfo,1105. ["Tebokan® 80", "Was ist Tebokan 80 und wann wird es angewendet?"]
store_patinfo_for_one_packages 54534 de 1105 :!patinfo,1105. ["Tebokan® 80", "Was ist Tebokan 80 und wann wird es angewendet?"]
2016-03-22 17:06:25 +0100: /var/www/oddb.org/src/plugin/text_info.rb:265:in `store_patinfo_for_one_packages': store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 80", "Qu’est-ce que le Tebokan 80 et quand doit-il être utilisé?"]
store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 80", "Qu’est-ce que le Tebokan 80 et quand doit-il être utilisé?"]
2016-03-22 17:06:25 +0100: /var/www/oddb.org/src/plugin/text_info.rb:265:in `store_patinfo_for_one_packages': store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 80", "Qu’est-ce que le Tebokan 80 et quand doit-il être utilisé?"]
store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 80", "Qu’est-ce que le Tebokan 80 et quand doit-il être utilisé?"]
2016-03-22 17:06:26 +0100: /var/www/oddb.org/src/plugin/text_info.rb:265:in `store_patinfo_for_one_packages': store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 120", "Qu’est-ce que Tebokan 120 et quand doit-il être utilisé?"]
store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 120", "Qu’est-ce que Tebokan 120 et quand doit-il être utilisé?"]
2016-03-22 17:06:26 +0100: /var/www/oddb.org/src/plugin/text_info.rb:265:in `store_patinfo_for_one_packages': store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 120", "Qu’est-ce que Tebokan 120 et quand doit-il être utilisé?"]
store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 120", "Qu’est-ce que Tebokan 120 et quand doit-il être utilisé?"]
2016-03-22 17:06:26 +0100: /var/www/oddb.org/src/plugin/text_info.rb:265:in `store_patinfo_for_one_packages': store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 120", "Qu’est-ce que Tebokan 120 et quand doit-il être utilisé?"]
store_patinfo_for_one_packages 54534 fr 1105 :!patinfo,1105. ["Tebokan® 120", "Qu’est-ce que Tebokan 120 et quand doit-il être utilisé?"]

Also the german and french are identical. Why?

ch.oddb> registration('63166').packages.first.patinfo.fr.to_s.eql?(registration('63166').packages.first.patinfo.de.to_s)
-> true
ch.oddb> registration('63166').packages.first.patinfo.fr.to_s[0..150]
-> Actemra®
Qu’est-ce que l’Actemra et quand est-il utilisé?
Actemra contient le principe actif tocilizumab, un anticorps monoclonal qui inhibe l'action d

But looking at the output of the log file we see that we decoded correctly the de (after the fr)

2016-03-22 17:08:52 +0100: /var/www/oddb.org/src/plugin/text_info.rb:1204:in `parse_textinfo': parse_textinfo 1204: must create textinfo for pi fr 63166 of ["63166"]
2016-03-22 17:08:52 +0100: /var/www/oddb.org/src/plugin/text_info.rb:313:in `update_patinfo_lang': update_patinfo_lang #<struct Struct::SwissmedicMetaInfo iksnr="63166", authNrs=["63166"], atcCode=nil, title="Actemra®", authHolder="Roche Pharma (Schweiz) AG", substances="Tocilizumab", type="pi", lang="fr", informationUpdate="09.2014", refdata=nil, xml_file="/var/www/oddb.org/data/details/63166_pi_fr_Actemra__problem.xml", same_content_as_xml_file=true> ["fr"]
2016-03-22 17:08:52 +0100: /var/www/oddb.org/src/plugin/text_info.rb:297:in `store_patinfo_for_all_packages': store_patinfo :!patinfo,34666790. none for reg.iksnr 63166 new oid 34666790 ["Actemra®", "Qu’est-ce que l’Actemra et quand est-il utilisé?", "Actemra contient le principe actif tocilizumab, un anticorps monoclonal qui inhibe l'action d'une certaine protéine (une cytokine) appelée interleukine-6. Cette protéine joue un rôle dans les processus inflammatoires qui se déroulent dans l'organisme et son blocage peut diminuer l'activité inflammatoire dans votre corps."]
2016-03-22 17:08:53 +0100: /var/www/oddb.org/src/plugin/text_info.rb:1204:in `parse_textinfo': parse_textinfo 1204: must create textinfo for pi de 63166 of ["63166"]
2016-03-22 17:08:53 +0100: /var/www/oddb.org/src/plugin/text_info.rb:313:in `update_patinfo_lang': update_patinfo_lang #<struct Struct::SwissmedicMetaInfo iksnr="63166", authNrs=["63166"], atcCode=nil, title="Actemra®", authHolder="Roche Pharma (Schweiz) AG", substances="Tocilizumab", type="pi", lang="de", informationUpdate="09.2014", refdata=nil, xml_file="/var/www/oddb.org/data/details/63166_pi_de_Actemra__problem.xml", same_content_as_xml_file=true> ["de"]
2016-03-22 17:08:53 +0100: /var/www/oddb.org/src/plugin/text_info.rb:284:in `store_patinfo_for_all_packages': store_patinfo update for reg.iksnr 63166 lang de existing oid 34666790 []
2016-03-22 17:08:53 +0100: /var/www/oddb.org/src/plugin/text_info.rb:285:in `store_patinfo_for_all_packages': store_patinfo update for reg.iksnr 63166 lang de new ["Actemra®", "Was ist Actemra und wann wird es angewendet?"]

Runnning jobs/update_textinfo_swissmedicinfo --reparse --target=pi 63166 and breaking into pry at the bug location. Calling patinfo.odba_store there fixed the problem. Now looking at the problem, why we did not create several patinfos for 54534 Tebokan. Finding the problem. Trying a fix via jobs/update_textinfo_swissmedicinfo --reparse --target=pi 54534

Things look better, but there is still a confusion and the patinfo are not correctly attributed to the sequence/ikcsd

ch.oddb> registration('54534').packages.collect{|x| [x.seqnr, x.patinfo.oid, x.patinfo.de.name ] }
-> [["02", 34664741, Tebokan® 80], ["02", 34664744, Tebokan® 80], ["03", 34664747, Tebokan® 120], ["03", 34664750, Tebokan® 120], ["03", 34664753, Tebokan® 120]]
ch.oddb> registration('54534').packages.collect{|x| [x.seqnr, x.patinfo.oid, x.ikscd, x.patinfo.de.name ] }
-> [["02", 34664741, "035", Tebokan® 80], ["02", 34664744, "043", Tebokan® 80], ["03", 34664747, "045", Tebokan® 120], ["03", 34664750, "046", Tebokan® 120], ["03", 34664753, "047", Tebokan® 120]]
ch.oddb> registration('54534').sequences.values.collect{|x| [x.seqnr, x.active?]}
-> [["01", true], ["02", true], ["03", true]]

Looks like we have a problem as the sequence 02 is no longer active.

view · edit · sidebar · attach · print · history
Page last modified on March 22, 2016, at 07:37 PM