Index>

20150407-fix-atc-less

Summary

• Fix error in atcless ArgumentError: invalid byte sequence in US-ASCII
• Improve oddb2xml for iscobar

Commits

• https://github.com/ngiger/oddb.org/commit/967c7b8d8e6a41a9b946b4af73ef0da90d9a1b9d Force UTF-8 encoding for atc_less

Index

• Fix error in atcless ArgumentError: invalid byte sequence in US-ASCII
• Improve oddb2xml for iscador

Keep in Mind for work to do

• Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
• I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
• The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
• creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
• Import via data/medreg_companies.yaml
• Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe

Improve oddb2xml for iscador

Iscador (IKSNR 56829) should look like this Viscum Album (Mali) Recens 0.001 mg/ml and not as

         <SUBSTANCE_NAME>Extractum Aquosum Liquidum Fermentatum
0.05 Mg Ex Viscum Album (mali) Recens</SUBSTANCE_NAME>
            <IS_ACTIVE_AGENT>false</IS_ACTIVE_AGENT>
            <QTY>0.01</QTY>
            <UNIT>mg/ml</UNIT>

See also http://scm.ywesee.com/?p=oddb.org/.git;a=commit;h=3d24ea905342c755819973fcdeacfc3c941b0a8e. Extract from an e-mail from Good, June 7, 2011 is Extractum aquosum liquidum fermentatum 0.05 mg ex viscum album (mali) recens 0.01 mg, natrii chloridum, aqua q.s. ad solutionem pro 1 ml.

Ausgedeutscht heisst das:
Der Extrakt ist ein Auszug aus der Frischpflanze im Verhältnis 1:5, also ‚extractum 0.05 mg’ entspricht 0.01 mg frischem Mistelkraut.
Es handelt sich um EINEN Wirkstoff, also „in Kombination“ ist falsch formuliert.
Die HILFSSTOFFE sind Aqua ad iniectabilia und Natrii chloridum.

Redefined parsing the lines. But the codes gets worse and worse. I think it might be better to create a real parser (eg. using Parslet or Treetop) as we have to split each line and finding

• Labels with descriptions, e.g I) DTPa-IPV-Komponente (Suspension):
• Preprations, e.g. Praeparatio cryodesiccata: virus rabiei inactivatum
• Substances:
  • with or without a dose
  • with or without a correspnding chemical substance (and optional dose)
• handle coresp.
• handle residui:
• handle stuff like pro/et/ex

probably all combinations of above is found once or more.

And adapting the various regular expressions gets more and more time consuming. I think I will invest 4 hours to see whether I can get parslet work with my example. Was able to add a simple example which parses '2 mg' and create a correct dose, see Attach:dose.txt

Fix error in atcless ArgumentError: invalid byte sequence in US-ASCII

Received the following error message

Plugin: ODDB::Atc_lessPlugin
Error: ArgumentError
Message: invalid byte sequence in US-ASCII
Backtrace:
/var/www/oddb.org/src/plugin/atc_less.rb:71:in `sub'
/var/www/oddb.org/src/plugin/atc_less.rb:71:in `parse_swissindex_xml'
/var/www/oddb.org/src/plugin/atc_less.rb:98:in `update_atc_codes'
/var/www/oddb.org/src/util/updater.rb:558:in `update_immediate'
/var/www/oddb.org/src/util/updater.rb:453:in `update_atc_less'
/var/www/oddb.org/src/util/updater.rb:456:in `update_swissmedic_followers'
/var/www/oddb.org/src/util/updater.rb:201:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

Trying to reproduce the problem locally running jobs/update_atc_less. It worked without a problem on oddb-ci2. Committed Force UTF-8 encoding for atc_less which should resolve the problem on thinpower. It worked fine on oddb-ci2.

Pushing & pulling the changes and starting the import job on thinpower. There it completed successfully, too.