view · edit · sidebar · attach · print · history

20131028-swissindex-update

<< | Index | >>


Summary

  • Error when running wissindex-update via import_dail
  • sprintf error while running update_swissindex_pharma

Commits

Index

---

running jobs/export_fachinfo_yaml produces undefined method `[]' for nil:NilClass

On thinpower we got the following error

Error: NoMethodError
Message: undefined method `[]' for nil:NilClass
Backtrace:
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in
`method_missing'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:131:in `[]'
/var/www/oddb.org/src/plugin/yaml.rb:87:in `block (3 levels) in
export_fachinfos'
/var/www/oddb.org/src/plugin/yaml.rb:86:in `each'
/var/www/oddb.org/src/plugin/yaml.rb:86:in `block (2 levels) in
export_fachinfos'
/var/www/oddb.org/src/plugin/yaml.rb:85:in `each'
/var/www/oddb.org/src/plugin/yaml.rb:85:in `block in export_fachinfos'
/var/www/oddb.org/src/plugin/yaml.rb:32:in `call'
/var/www/oddb.org/src/plugin/yaml.rb:32:in `check_infos'
/var/www/oddb.org/src/plugin/yaml.rb:84:in `export_fachinfos'
/var/www/oddb.org/src/util/exporter.rb:307:in `block in export_fachinfo_yaml'
/var/www/oddb.org/src/util/exporter.rb:425:in `call'
/var/www/oddb.org/src/util/exporter.rb:425:in `safe_export'
/var/www/oddb.org/src/util/exporter.rb:306:in `export_fachinfo_yaml'
/var/www/oddb.org/jobs/export_fachinfo_yaml:18:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
/var/www/oddb.org/jobs/export_fachinfo_yaml:13:in `<module:Util>'
/var/www/oddb.org/jobs/export_fachinfo_yaml:12:in `<module:ODDB>'
/var/www/oddb.org/jobs/export_fachinfo_yaml:11:in `<main>'

Fix was easy. Testing it on my vm. Fix did not work, adding debug output and running import again.

Had to call to_yaml with a rescue clause (needed by exactly 2 fachinfos). Also 224 (of 12886) fachinfos do not have a localized_name and stem probably from unsuccessfull import. May be should delete them. bin/admin reports 6443 as fachinfos.size. (each with :de and :fr). Also fachinfo.to_yaml is very slow! I took 135 minutes to complete.

composition for sequence 01 not updated from Packungen.xls

For Xeljanz (Iksnr 62630) we still have a sequence 00 after the update from Packungen.xls. This must be corrected. Looking for all sequence['00'] via bin/admin.

$suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.sequences['00'] != nil}; $suche.sizeHash
ch.oddb> 
-> 
ch.oddb> $suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.sequences['00'] != nil}; $suche.size
-> 31
ch.oddb> $suche[0..10]
-> ["00828", "10754", "12498", "17644", "29300", "31919", "33709", "42180", "52265", "57487", "57515"]
-> ["58458", "62438", "62470", "62566", "62583", "62612", "62630", "62658", "62711", "62785", "62786"]
ch.oddb> $suche[22..31]
-> ["62920", "62976", "63029", "63030", "63200", "63257", "67769", "69870", "72736"]

Looking at http://ch.oddb.org/de/gcc/show/reg/00828 we find that we have created a sequence '00' even when we see a "Widerruf der Zulassung 24.10.2013". For unknown reasons http://oddb-ci.dyndns.org/de/gcc/show/reg/00828 does not display. "Widerruf der Zulassung 24.10.2013". Therefore I will drop in reimport the actual database from oddb.org. Then I will try to fix this situation via a bin/admin command, to see how many errors will remain afterwards.

$suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.sequences['00'] != nil and reg.inactive?}; $suche.size
ch.oddb> $suche[0..10]
-> ["Zulassungs-Nummer", "17644", "52265", "62470", "29300", "57487", "67769", "00828", "31919", "57515", "62583"]
ch.oddb> $suche[11..21]
-> ["69870", "10754", "33709", "58458", "72736", "12498", "42180", "62438", "62711", "63257", "63200"]
$deleted = []; registrations.each{ |iksnr, reg| if reg.sequences['00'] != nil and reg.inactive? then $deleted <<iksnr; delete reg.sequence('00').pointer; update(reg.pointer, :sequences => {}); end }; $deleted.size
-> 22
ch.oddb> $suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.sequences['00'] != nil }; $suche.size
-> 14
ch.oddb> $suche[0..14]
-> ["62976", "62658", "63030", "62566", "62785", "62786", "62612", "62920", "62630", "63029", "62672", "65040", "62904", "62453"]

Now looking at

  • http://ch.oddb.org/de/gcc/show/reg/62976 has only sequence '00'
    • Conclusion: As Packungen.xls does not containt an entry for 62976 this is okay.
  • http://ch.oddb.org/de/gcc/show/reg/62658 has sequence '00' and '01'.
    • Sequence '01' has correct package '001'.
    • Sequence '00' has no package
    • Conclusion: Import Packungen.xls did not not delete Sequence '00'
  • http://ch.oddb.org/de/gcc/show/reg/63030. Has sequence '00' and '01', but not
    • Sequence '00' has correct packages '001', '002' and '003'
    • Sequence '01' has no packages
    • Conclusion: Import Packungen.xls created 3 package in Sequence '00' instead of '01'.
  • http://ch.oddb.org/de/gcc/show/reg/62566 has sequence '00' and '01'.
    • Sequence '00' has no packages
    • Sequence '00' has correct packages '001' and '002'.
    • Conclusion: Import Packungen.xls did not not delete Sequence '00'
  • http://ch.oddb.org/de/gcc/show/reg/62785 has sequence '00', '01', '02' and '03'.
    • Sequence '00' has packages '001' and '002'.
    • Sequence '01' has packages '003' and '004' (Should have '001' and '002')
    • Sequence '02' has no package. (Should have '003' and '004')
    • Sequence '03' has packages '005' and '006'. Correct
    • Conclusion: Import Packungen.xls did not not delete Sequence '00' and placed packages into wrong sequences!

Error beim Import von allen FIs für AstraZeneca

On thinpower we got the following error while running sudo -u apache jobs/update_textinfo_swissmedicinfo --no-download --target=both AstraZeneca --reparse.

Plugin: ODDB::TextInfoPlugin
Error: NoMethodError
Message: undefined method `create_package' for nil:NilClass
Backtrace:
/var/www/oddb.org/src/plugin/text_info.rb:582:in
`create_dummy_package_if_necessary'
/var/www/oddb.org/src/plugin/text_info.rb:1365:in `block in
import_swissmedicinfo_by_iksnrs'
/var/www/oddb.org/src/plugin/text_info.rb:1362:in `each'
/var/www/oddb.org/src/plugin/text_info.rb:1362:in
`import_swissmedicinfo_by_iksnrs'
/var/www/oddb.org/src/plugin/text_info.rb:1383:in
`import_swissmedicinfo_by_companies'
/var/www/oddb.org/src/plugin/text_info.rb:1397:in `import_swissmedicinfo'
/var/www/oddb.org/src/util/updater.rb:512:in `block in update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:488:in `call'
/var/www/oddb.org/src/util/updater.rb:488:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:506:in `update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:303:in `update_textinfo_swissmedicinfo'
jobs/update_textinfo_swissmedicinfo:39:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/update_textinfo_swissmedicinfo:12:in `<module:Util>'
jobs/update_textinfo_swissmedicinfo:11:in `<module:ODDB>'
jobs/update_textinfo_swissmedicinfo:10:in `<main>'

Trying to reproduce the problem here on my VM. Could reproduce the problem here. Situation for IKSNR 52110 'Beloc Zok' is that there exist 4 sequences but no package at all. Adapted handling the code to handle this situation and rerunning the import.

I pose myself the question, how we got into this situation and why parsing the Packungen.xls this situation was not fixed. Running registrations.each{ |iksnr, reg| $stdout.puts "iksrn #{iksnr} reg #{reg.inspect}" if reg.packages.size == 0 and not reg.expired?} in bin/admin returns quite a few packages, eg. 00709 which is not found in Packungen.xls

Found 388 registrations where noc packages are defined. Used bin/admin

ch.oddb> $suche = []; registrations.each{ |iksnr, reg| $suche << iksnr if reg.packages.size == 0 and reg.active?}; $suche.size
-> 388

Solution is: create_dummy_package_if_necessary must exclude the following registrations

  1. having at least one package (only condition checked till now)
  2. having the inactive? flag set
  3. having the export flag set
  4. expiration_date < actual date
  5. having at least one sequence

Rename the method to create_sequence_and_package_if_necessary to better reflect the actual work done. Now the import worked correctly. Pushed commit Fix importing expired registrations

We cannot find anything when searching by indications

Plugin: ODDB::SwissindexPharmaPlugin
Error: ArgumentError
Message: invalid value for Integer(): "7680274.00017"
Backtrace:
/var/www/oddb.org/src/plugin/swissindex.rb:234:in `sprintf'
/var/www/oddb.org/src/plugin/swissindex.rb:234:in `block in report'
/var/www/oddb.org/src/plugin/swissindex.rb:233:in `each'
/var/www/oddb.org/src/plugin/swissindex.rb:233:in `report'
/var/www/oddb.org/src/plugin/plugin.rb:54:in `block in log_info'
/var/www/oddb.org/src/plugin/plugin.rb:53:in `each'
/var/www/oddb.org/src/plugin/plugin.rb:53:in `inject'
/var/www/oddb.org/src/plugin/plugin.rb:53:in `log_info'
/var/www/oddb.org/src/util/updater.rb:144:in `log_info'
/var/www/oddb.org/src/util/updater.rb:514:in `block in update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:488:in `call'
/var/www/oddb.org/src/util/updater.rb:488:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:506:in `update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:423:in
`update_package_trade_status_by_swissindex'
jobs/update_swissindex_pharma:16:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/update_swissindex_pharma:13:in `<module:Util>'
jobs/update_swissindex_pharma:11:in `<module:ODDB>'
jobs/update_swissindex_pharma:10:in `<main>'

Adding a debug printf to see which reg is updated.

Question is, wherefrom does the pack.barcode get the value "7680274.00017"? Adding also a printf when the @out_of_trade_true_list is set.

Tried to run the import, but got an Message: druby://oddb-ci.dyndns.org:45291 - #<Errno::ETIMEDOUT: Connection timed out - connect(2)> error. Runing oddbd in a separate screen and rerunning the import. This time I got druby://oddb-ci.dyndns.org:48310 - #<Errno::ETIMEDOUT: Connection timed out - connect(2)>. Running jobs /var/www/oddb.org/ext/swissindex/bin/swissindex_nonpharmad and /var/www/oddb.org/ext/swissindex/bin/swissindex_pharmad in separate screens.

I thinks I have a problem with my network configuration. Unfortunately http://www.funtoo.org/ answers with nginx: 502 Bad Gateway. Attacking therefore another problem.

Error when running swissindex-update via import_daily

Once each week we get the following error:

Plugin: ODDB::SwissindexPharmaPlugin
Error: ArgumentError
Message: wrong number of arguments (2 for 1)
Backtrace:
/var/www/oddb.org/src/plugin/plugin.rb:40:in `initialize'
/var/www/oddb.org/src/util/updater.rb:508:in `new'
/var/www/oddb.org/src/util/updater.rb:508:in `block in update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:488:in `call'
/var/www/oddb.org/src/util/updater.rb:488:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:506:in `update_notify_simple'
/var/www/oddb.org/src/util/updater.rb:423:in
`update_package_trade_status_by_swissindex'
/var/www/oddb.org/src/util/updater.rb:398:in `update_swissmedic_followers'
/var/www/oddb.org/src/util/updater.rb:196:in `run'
/var/www/oddb.org/jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
/var/www/oddb.org/jobs/import_daily:12:in `<module:Util>'
/var/www/oddb.org/jobs/import_daily:11:in `<module:ODDB>'
/var/www/oddb.org/jobs/import_daily:10:in `<main>'

Added some changes in src/util/updater.rb and src/plugin/plugin.rb to avoid this error. Running import_daily in a console to see whether everything is okay. Seems to be okay. Pushed commit Fix option handling for import_daily

view · edit · sidebar · attach · print · history
Page last modified on October 29, 2013, at 02:03 PM