<< | Index | >>
Fixed a bug (:fi -> :pi) which resulted in not reparsing patinfo when company names given in jobs/update_textinfo_swissmedicinfo. Details see commit
Thinpower gets the following error when restarting /service/ch.oddb-exporter and then calls jobs/export_index_therapeuticus_csv (index_terapeuticus.csv)
Calling the jobs manually is no problem, but using a crontab we get the error
From: <admin@ywesee.com> Date: 2013/7/16 Subject: ch.ODDB.org Report - Error Export: index_therapeuticus - 07/2013 To: zdavatz@ywesee.com Error: DRb::DRbConnError Message: druby://localhost:10005 - #<Errno::ECONNREFUSED: Connection refused - connect(2)> Backtrace: /usr/local/lib/ruby/1.9.1/drb/drb.rb:736:in `rescue in block in open' /usr/local/lib/ruby/1.9.1/drb/drb.rb:730:in `block in open' /usr/local/lib/ruby/1.9.1/drb/drb.rb:729:in `each' /usr/local/lib/ruby/1.9.1/drb/drb.rb:729:in `open' /usr/local/lib/ruby/1.9.1/drb/drb.rb:1191:in `initialize' /usr/local/lib/ruby/1.9.1/drb/drb.rb:1171:in `new' /usr/local/lib/ruby/1.9.1/drb/drb.rb:1171:in `open' /usr/local/lib/ruby/1.9.1/drb/drb.rb:1087:in `block in method_missing' /usr/local/lib/ruby/1.9.1/drb/drb.rb:1105:in `with_friend' /usr/local/lib/ruby/1.9.1/drb/drb.rb:1086:in `method_missing' /var/www/oddb.org/src/plugin/csv_export.rb:160:in `export_index_therapeuticus' /var/www/oddb.org/src/util/exporter.rb:161:in `block in export_index_therapeuticus_csv' /var/www/oddb.org/src/util/exporter.rb:435:in `call' /var/www/oddb.org/src/util/exporter.rb:435:in `safe_export' /var/www/oddb.org/src/util/exporter.rb:159:in `export_index_therapeuticus_csv' /var/www/oddb.org/src/util/exporter.rb:59:in `run' /var/www/oddb.org/jobs/export_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' /var/www/oddb.org/jobs/export_daily:12:in `<module:Util>' /var/www/oddb.org/jobs/export_daily:11:in `<module:ODDB>' /var/www/oddb.org/jobs/export_daily:10:in `<main>'
Reason? Will look first at src/util/exporter.rb
To reproduce the error the following steps were executed
svc -h /var/lib/service/ch.oddb-export/ ruby /var/www/oddb.org/jobs/mail_index_therapeuticus_csv
Added support for the crontab in puppet (Using crontab /etc/cronta
). Adjusted launch time for therapeuticus to now+3 minutes. Ignored two batch jobs not yet present.
After some time I received ch.ODDB.org Report - index_therapeuticus.csv - 07/2013
. But the containing idx_th.csv was empty.
I will let the VM run to see, whether the crontab jobs work. sudo crontab -l displays
# DO NOT EDIT THIS FILE - edit the master and reinstall. # (/etc/crontab installed on Wed Jul 17 16:07:49 2013) # (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $) # # $Header: /var/cvsroot/gentoo-x86/sys-process/vixie-cron/files/crontab-3.0.1-r4,v 1.1 2005/03/04 23:59:48 ciaranm Exp $ # # # Global variables SHELL=/bin/bash #PATH=/sbin:/bin:/usr/sbin:/usr/bin PATH=/usr/local/bin:/usr/bin:/bin MAILTO=root HOME=/ LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 # check scripts in cron.hourly, cron.daily, cron.weekly and cron.monthly 0 * * * * rm -f /var/spool/cron/lastrun/cron.hourly 1 3 * * * rm -f /var/spool/cron/lastrun/cron.daily 15 4 * * 6 rm -f /var/spool/cron/lastrun/cron.weekly 30 5 1 * * rm -f /var/spool/cron/lastrun/cron.monthly */10 * * * * test -x /usr/sbin/run-crons && /usr/sbin/run-crons # update webalizer stats for virtual hosts # ngngng: missing */10 * * * * root /usr/local/bin/update_vhost_stats >/dev/null 2>&1 # backup to backup-disk # ngngng: missing 0 22 * * * root /usr/bin/nice /usr/local/sbin/ywesee-backup > /dev/null # compress older apache-logs # ngngng: missing 0 5 1 1 * /usr/local/bin/compress_old_logs # run ch.oddb.org updates 1 9 * * * /var/www/oddb.org/jobs/import_daily >/dev/null 2>&1 1 6 28 * * /var/www/oddb.org/jobs/export_fachinfo_yaml 1 6 27 * * /var/www/oddb.org/jobs/export_patinfo_yaml 12 16 * * * /var/www/oddb.org/jobs/export_daily 14 16 * * * /var/www/oddb.org/jobs/mail_index_therapeuticus_csv #http://en.wikipedia.org/wiki/Cron#Format # run ch.oddb migel-products updates 1st of January and 1st of June (run the BAG update via NovaCantica manually) # <minute> <hour> <day of month> <month> <day of week> <command> 1 0 1 6 * /var/www/migel/jobs/update_migel_products_with_report >/dev/null 2>&1 1 0 1 1 * /var/www/migel/jobs/update_migel_products_with_report >/dev/null 2>&1
Until now x-wmf is not yet supported. Created test case for this problem. Fix is quite simple.But the result is ugly. See
Pushed the simple fix Accept x-wmf images
Analysis of HTML code
Swissmedic Style s5 is .s5{font-family:Times New Roman;font-size:11pt;}
definition from definition:
<style><!-- p{margin-top:0pt;margin-right:0pt;margin-bottom:0pt;margin-left:0pt;}table{border-spacing:0pt;border-collapse:collapse;} table td{vertical-align:top;}.s2{font-family:Arial;font-size:16pt;font-weight:bold;}.s3{font-family:Arial;font-size:12.8pt;font-weight:bold;}.s4{text-indent:-50.4pt;line-height:150%;margin-top:18pt;margin-left:50.4pt;}.s5{font-family:Times New Roman;font-size:11pt;}.s6{line-height:150%;text-align:justify;margin-top:6pt;}.s7{font-family:Arial;font-size:11pt;font-weight:bold;}.s8{font-family:Times New Roman;font-size:11pt;font-style:italic;}.s9{font-family:Times New Roman;font-size:11pt;text-decoration:none;color:#000000;}.s10{font-family:Times New Roman;font-size:8.8pt;}.s11{line-height:150%;margin-top:18pt;}.s12{line-height:150%;text-align:justify;}.s13{font-family:Times New Roman;font-size:12pt;}.s14{font-family:Times New Roman;font-size:11pt;text-decoration:line-through;}.s15{font-family:Arial;font-size:11pt;font-style:italic;font-weight:normal;}.s16{font-family:Arial;font-size:8.8pt;font-style:italic;font-weight:normal;}.s17{line-height:150%;margin-top:2pt;}.s18{font-family:Arial;font-size:11pt;}.s19{line-height:150%;text-align:justify;margin-top:1pt;}.s20{line-height:150%;text-align:center;}.s21{font-family:Arial;font-size:11pt;font-weight:normal;}.s22{font-size:11pt;line-height:150%;text-align:justify;margin-top:1pt;}.s23{font-size:11pt;line-height:150%;margin-top:2pt;}.s24{font-family:Symbol;font-size:11pt;font-weight:normal;}.s25{font-family:Symbol;}.s26{font-family:Times New Roman;font-size:11pt;color:#00b0f0;} --></style>
Firmenlogo has this definition in swissmedic
<span class="s5"><span> Firmenlogo</span></span><span class="s5"><span>s</span></span> <span class="s5"><span> (</span></span><span class="s5"><img src="data:image/x-wmf;base64,iVBORw0KGgoAAAANSUhEUgAAAA8AAAAUCAYAAABSx2cSAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAGLSURBVDhPnZShq8JAHMfXbGu2JZPFJDJsJpsWHyJY5n8wEGZbU5AhaBBcMGiymNdWBiZhiGhZMdusA/299/uNu3fnPd7Ag4N9f/f9fG/73THt9XrBp1P7DwRh/OWTYPQyEz7PZjPQNA2KxSLFvAdwGBeDIBA3g+VySXC9Xs+HF4sF6LrOA06nE8GWZeXD6ECz53lkPp/PpHu9Xj6cJAmZy+Uy3x217/v58Gg0IrjVaknwarWSesEaRw1jA0Gcm81Gqu33e9KXywXG4zF/Cw7P53MCcYoD9fV65aXBYACNRoM0dzLQdV2ejA/NZhMOh4MS6DhOBm+3W2VX9jntdpuDrGYYRubHFdu2SVSrVamrKCaTiVJDH4eHwyEJ0zQV4/sxoaFWq/3CYRhKry3e7zRNlcBCoZD52XdUKhUq7HY7xSyGHY9HvpF0zv1+nxbE7or3II5jWi+VStlRialYuN/v0O12YTqdwu12I9Pj8YD1eg2drw5EUcS7r/wMpAP9ERj+fD6lsnQ9P/0NfQOeIvETcTl7/gAAAABJRU5ErkJggg==" style="height:15.3pt;width:11.2pt;" /></span> <span class="s5"><span>) unterh</span></span><span class="s5"><span>alb eines schwarzen Streifens.</span></span></p>
On oddb we show it like this
<BR><SPAN style="padding-bottom: 4px; white-space: normal; line-height: 1.4em;">Durchsichtige orange Kapseln mit einem weissen Pulver. Aufdruck des Produktcodes GPL50 in schwarz oberhalb eines schwarzen Streifens, Aufdruck des Firmenlogo s (</SPAN><BR></P> <P style="font-size: 13px; margin-top: 4px; line-height: 1.4em"> <SPAN style="font-style: italic"></SPAN> <P style="padding-bottom: 4px; white-space: normal; line-height: 1.4em;"><IMG src="/resources/images/fachinfo/de/Seebri_Breezhaler_files/1.x-wmf" style="height:15.3pt;width:11.2pt;"> </P> </P> <P style="font-size: 13px; margin-top: 4px; line-height: 1.4em"> <SPAN style="font-style: italic"></SPAN> <SPAN style="padding-bottom: 4px; white-space: normal; line-height: 1.4em;">) unterh alb eines schwarzen Streifens.</SPAN> <BR>
Fixed with commit Flag non conforming elements
Created following report:
Searched for Stored 2 Fachinfos Ignored 0 Pseudo-Fachinfos Ignored 0 up-to-date Fachinfo-Texts Stored 0 Patinfos Ignored 0 up-to-date Patinfo-Texts Checked 0 companies Unknown Iks-Numbers: 0 Fachinfos without iksnrs: 0 Session failures: 0 Download errors: 0 Parse Errors: 0 New/Updates FI from swissmedicinfo.ch ISKNR : Fachinfo - DE - Seebri Breezhaler - ["62580"] ISKNR : Fachinfo - FR - Seebri Breezhaler - ["62580"] Skipped FI form swissmedicinfo.ch Invalid FI from swissmedicXML Not found FI in swissmedicXML Non conforming contents: 1 62580: 'Seebri Breezhaler' with non conforming data:image/x-wmf;base64 element
This was a simple one liner fix Mention companies in report. Tested on my local VM.