view · edit · sidebar · attach · print · history

Index>

20151021-cleanup-bsv-import

Summary

  • Fix BSV import (generic-type, remove pharmacode)
  • Fix Problem with text_info of Remodulin

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • Create a history of FI changes and add a nice diff (colored) to compare any two versions. Look at gems like

diffy or differ

  • Ofev (IKSNR 65330) has a not yet WHO-Approved ATC-Code L01XE31. The job atc_less should add the new ATC-Code with an empty name and flag it in its report for human intervention (e.g. extracting the substance name from the FI)
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Get ATC-Code from FI and compare it to the XML and Packungen.xlsx

Swissmedic-Update must delete all sequences, which are absent

Analysing http://ch.oddb.org/de/gcc/drug/reg/00485/seq/26 we remarked that this is a sequence which no longer exists in the swissmedic packungen.xlsx but was never updated. Therefore the substances included Neuraminidasum Inactivatum (Virus-Stamm A/california/7/2009 (H1N1)-Like: Reassortant Virus nymc X-181).

The http://ch.oddb.org/de/gcc/drug/reg/00485/seq/28 Influvac is corect.

Fix Problem with text_info of Remodulin

Got an Out-Of-Memory error when parsing the textinfo for remodulin. Remarked that its yaml file is big -rw-r--r-- 1 apache apache 1363014 21. Okt 02:04 data/html/fachinfo/de/Remodulin__swissmedicinfo.yaml. See Attach:remodulin_yaml.txt. The HTML file 1.3 MB long and file reports Remodulin__swissmedicinfo.html: HTML document, UTF-8 Unicode text, with very long lines. Looking via swissindex at its content I see that we some mathematical formulas and at the end some long tables.

In log/oddb/debug/2015/10.log I found the following lines 

2015-10-21 02:04:38 +0200: /var/www/oddb.org/src/plugin/text_info.rb:1156:in `block in parse_and_update': parse_and_update: calls parse_fachinfo reparse dist /var/www/oddb.org/data/html/fachinfo/de/Remodulin__swissmedicinfo.html false iksnrs_from_xml "56120"] Remodulin__swissmedicinfo.html, name Remodulin® de title Remodulin® 20
2015-10-21 05:27:11 CESTlog notify Error: Fach- und Patienteninfo Updates (swissmedicinfo.ch): start outgoing process ["log"]. Must attach 0 files and 0 parts.
Error: NoMemoryError 
Message: failed to allocate memory
Backtrace: 
/var/www/oddb.org/src/model/text.rb:396:in `block in wrap' 
/var/www/oddb.org/src/model/text.rb:377:in `each' 
/var/www/oddb.org/src/model/text.rb:377:in `wrap'
/var/www/oddb.org/src/model/text.rb:351:in `block (2 levels) in to_s' 
/var/www/oddb.org/src/model/text.rb:350:in `collect'
/var/www/oddb.org/src/model/text.rb:350:in `block in to_s'
/var/www/oddb.org/src/model/text.rb:345:in `collect'
/var/www/oddb.org/src/model/text.rb:345:in `to_s'
/var/www/oddb.org/src/model/text.rb:432:in `block in to_s'
/var/www/oddb.org/src/model/text.rb:432:in `collect'
/var/www/oddb.org/src/model/text.rb:432:in `to_s' 
/var/www/oddb.org/src/model/text.rb:491:in `block in to_s' 
/var/www/oddb.org/src/model/text.rb:491:in `collect'
/var/www/oddb.org/src/model/text.rb:491:in `to_s' 
/var/www/oddb.org/src/plugin/text_info.rb:1166:in `inspect'
/var/www/oddb.org/src/plugin/text_info.rb:1166:in `inspect' 
/var/www/oddb.org/src/plugin/text_info.rb:1166:in `inspect'
/var/www/oddb.org/src/plugin/text_info.rb:1166:in `parse_and_update'
/var/www/oddb.org/src/plugin/text_info.rb:1185:in `block (3 levels) in import_info' 
/var/www/oddb.org/src/plugin/text_info.rb:1184:in `each'
/var/www/oddb.org/src/plugin/text_info.rb:1184:in `block (2 levels) in import_info' 
/var/www/oddb.org/src/plugin/text_info.rb:1183:in `each_pair' 
/var/www/oddb.org/src/plugin/text_info.rb:1183:in `block in import_info' 
/var/www/oddb.org/src/plugin/text_info.rb:1180:in `each_pair' 
/var/www/oddb.org/src/plugin/text_info.rb:1180:in `import_info' 
/var/www/oddb.org/src/plugin/text_info.rb:1388:in `block in import_swissmedicinfo_by_index' 
/var/www/oddb.org/src/plugin/text_info.rb:1387:in `each_pair' 
/var/www/oddb.org/src/plugin/text_info.rb:1387:in `import_swissmedicinfo_by_index'
/var/www/oddb.org/src/plugin/text_info.rb:1449:in `import_swissmedicinfo' 
/var/www/oddb.org/src/util/updater.rb:573:in `block in update_notify_simple'

Comparing the content of Swissmedic and ch.oddb.org I found that state of information changed from May 2014 to September 2015 and it introduced a quite complicated formula. See

Trying to add a unit test for this case.

Running jobs/update_textinfo_swissmedicinfo --target=fi --reparse 56120 to see, whether we have a problem with just this FI. This one completed (After a change in the debug message at line 1166). Starting import_daily again.

Pushed commit Fix sort order of desitin

After some longer discussion with zeno I think I understood his ideas correctly. Pushed commit Another fix for resultsort evidentia.

Fix BSV import (generic-type, remove pharmacode)

Continuing work of yesterday. Current state is Attach:patch_cleanup_bsv_patch.txt.

Pushed commit Remove pharmacode from bsv-import

view · edit · sidebar · attach · print · history
Page last modified on October 21, 2015, at 05:57 PM