20120525-import-daily-patinfo2csv

<< | Index | >>

Summary

• Updated patinfo2csv.
  • added multi line text section support.
• Debug import_daily via cron job.
• Added drugbank link to search result.

Commits

patinfo2csv

• Added multi lines text section support

ch.oddb.org

• Added drugbank link to result list

Index

• Update patinfo2csv
• Debug import_daily job
• Added Drug bank link

Update patinfo2csv

Problem

patinfo.yaml has line-break in preformatted text: section.

  2–4 Jahre        (12–17 kg)        4–6 Tropfen      <br />5–7 Jahre        (18–25 kg)        7–9 Tropfen      <br />8–11 Jahre       (26–35 kg)        12–18 Tropfen   

patinfo.yaml

167           text: "Alter            K\xC3\xB6rpergewicht     Dosierung        \n\
168             ----------------------------------------------------\n\
169             2\xE2\x80\x934 Jahre        (12\xE2\x80\x9317 kg)        4\xE2\x80\x936 Tropfen      \n\
170             5\xE2\x80\x937 Jahre        (18\xE2\x80\x9325 kg)        7\xE2\x80\x939 Tropfen      \n\
171             8\xE2\x80\x9311 Jahre       (26\xE2\x80\x9335 kg)        12\xE2\x80\x9318 Tropfen    \n"

patinfo

• http://oddb.yasuhiro.org/de/gcc/patinfo/reg/36736/seq/01
• http://oddb.yasuhiro.org/de/gcc/patinfo/reg/41925/seq/02

Added regular expression for multi lines text section.

Commit

• Added multi lines text section support

Debug import_daily job

Problem

notification mail does not come from import_daily job via cron.

Refs

• http://dev.ywesee.com/Masa/20101027-debug-bsvFollower-autorun
• http://dev.ywesee.com/Masa/20101028-debug-bsvFollower-autorun
• http://dev.ywesee.com/Masa/20101029-debug-bsvFollower-autorun

Check flow

in src/util/updater.rb

• Updater#run
  • update_swissmedic
    • update_swissmedic_followers
  • update_bsv
    • update_bsv_follewers
  • update_interactions (1st/month)

check environment, setting

checked following functions of updater via cron.

• output to log file (into /var/www/oddb.org/log/oddb/debug/2012/05.log)
• send notification mail (Exporter#mail_swissmedic_notifications)

works fine.

check dependencies

current dependencies import_daily

• import_daily
  • update_swissmedic
  • update_swissmedic_followers
    • swissindex_pharmad (via swissindexpharma plugin)
    • swissregd (via swissreg plugin)
    • meddatad (via medwin plugin)
    • exportd (via xls_export plugin)
  • update_bsv
    • meddatad (via bsv_xml plugin)
  • update_bsv_followers
    • swissindex_pharmad (via swissindexpharma plugin)
    • exportd (via xls_export plugin)
    • exportd (via csv_export plugin)
    • exportd (via ouwerkerk plugin)
  • update_interactions

NOTE

duplicate update methods (update_swissmedic_followers and update_bsv_followers)

• update_package_trade_status_by_swissindex
• export_generic_xls
• update_lppv

Refs

• http://masaywesee.wordpress.com/2010/07/12/run-import_bsv-only-read-code-relations/ (2010)

check production log

recent (updated) log in production server

2012-05-05 09:01:21 CEST getin update_bsv
2012-05-05 09:01:29 CEST getin BsvXmlPlugin.update
2012-05-05 09:01:29 CEST target_url = http://bag.e-mediat.net/SL2007.Web.External/File.axd?file=XMLPublications.zip
2012-05-05 09:01:29 CEST save_dir   = /var/www/oddb.org/data/xml
2012-05-05 09:01:29 CEST getin download_file
2012-05-05 09:01:31 CEST save_file   = /var/www/oddb.org/data/xml/XMLPublications-2012.05.05.zip
2012-05-05 09:01:31 CEST latest_file = /var/www/oddb.org/data/xml/XMLPublications-latest.zip
2012-05-05 09:01:31 CEST File.exists?(/var/www/oddb.org/data/xml/XMLPublications-latest.zip) = true
2012-05-05 09:01:31 CEST FileUtils.compare_file(/tmp/foo20120505-14836-1rp7847, /var/www/oddb.org/data/xml/XMLPublications-latest.zip) = false
2012-05-05 09:01:31 CEST path = "/var/www/oddb.org/data/xml/XMLPublications-2012.05.05.zip"
2012-05-05 09:01:31 CEST entry.name = Gestrichene_Packungen_Emballages_radies.xls
2012-05-05 09:01:31 CEST entry.name = PR120501.txt
2012-05-05 09:01:31 CEST entry.name = GL_Diff_SB.xml
2012-05-05 09:01:31 CEST entry.name = ItCodes.xml
2012-05-05 09:03:37 CEST entry.name = Preparations.xml

debug file download

Last night, I ran import_daily via cron at 03:00.
It still running (17:00).

checked get_latest_file

in src/plugin/swissmedic.rb

   def get_latest_file(agent, keyword='Packungen')
      page = agent.get @index_url
      links = page.links.select do |link|
        ptrn = keyword.gsub /[^A-Za-z]/u, '.' 
        /#{ptrn}/iu.match link.attributes['title']
      end 
      link = links.first or raise "could not identify url to #{keyword}.xls"
      file = agent.get(link.href)
      download = file.body
      ...
   end

Refs

• http://www.swissmedic.ch/daten/00080/00251/index.html?lang=de

Continue

to check result of import_dail job via cron.

Added Drug bank link

in result list.

NOTE

At Result list, use AtcFacade class.

• AtcClass (in src/mode/atcclass.rb)
• AtcFacade (in src/mode/search_result.rb)

Commit

Added drugbank link to result list

Refs

• http://www.drugbank.ca/search?utf8=✓&query=N05BA12&commit=Search