view · edit · sidebar · attach · print · history

20120525-import-daily-patinfo2csv

<< | Index | >>


Summary

  • Updated patinfo2csv.
    • added multi line text section support.
  • Debug import_daily via cron job.
  • Added drugbank link to search result.

Commits

patinfo2csv
ch.oddb.org

Index


Update patinfo2csv

Problem

patinfo.yaml has line-break in preformatted text: section.

  2€“4 Jahre        (12€“17 kg)        4€“6 Tropfen      <br />5€“7 Jahre        (18€“25 kg)        7€“9 Tropfen      <br />8€“11 Jahre       (26€“35 kg)        12€“18 Tropfen   
patinfo.yaml
167           text: "Alter            K\xC3\xB6rpergewicht     Dosierung        \n\
168             ----------------------------------------------------\n\
169             2\xE2\x80\x934 Jahre        (12\xE2\x80\x9317 kg)        4\xE2\x80\x936 Tropfen      \n\
170             5\xE2\x80\x937 Jahre        (18\xE2\x80\x9325 kg)        7\xE2\x80\x939 Tropfen      \n\
171             8\xE2\x80\x9311 Jahre       (26\xE2\x80\x9335 kg)        12\xE2\x80\x9318 Tropfen    \n"
patinfo

Added regular expression for multi lines text section.

Commit

Debug import_daily job

Problem

notification mail does not come from import_daily job via cron.

Refs

Check flow

in src/util/updater.rb

  • Updater#run
    • update_swissmedic
      • update_swissmedic_followers
    • update_bsv
      • update_bsv_follewers
    • update_interactions (1st/month)

check environment, setting

checked following functions of updater via cron.

  • output to log file (into /var/www/oddb.org/log/oddb/debug/2012/05.log)
  • send notification mail (Exporter#mail_swissmedic_notifications)

works fine.

check dependencies

current dependencies import_daily

  • import_daily
    • update_swissmedic
    • update_swissmedic_followers
      • swissindex_pharmad (via swissindexpharma plugin)
      • swissregd (via swissreg plugin)
      • meddatad (via medwin plugin)
      • exportd (via xls_export plugin)
    • update_bsv
      • meddatad (via bsv_xml plugin)
    • update_bsv_followers
      • swissindex_pharmad (via swissindexpharma plugin)
      • exportd (via xls_export plugin)
      • exportd (via csv_export plugin)
      • exportd (via ouwerkerk plugin)
    • update_interactions
NOTE

duplicate update methods (update_swissmedic_followers and update_bsv_followers)

  • update_package_trade_status_by_swissindex
  • export_generic_xls
  • update_lppv

Refs

check production log

recent (updated) log in production server

2012-05-05 09:01:21 CEST getin update_bsv
2012-05-05 09:01:29 CEST getin BsvXmlPlugin.update
2012-05-05 09:01:29 CEST target_url = http://bag.e-mediat.net/SL2007.Web.External/File.axd?file=XMLPublications.zip
2012-05-05 09:01:29 CEST save_dir   = /var/www/oddb.org/data/xml
2012-05-05 09:01:29 CEST getin download_file
2012-05-05 09:01:31 CEST save_file   = /var/www/oddb.org/data/xml/XMLPublications-2012.05.05.zip
2012-05-05 09:01:31 CEST latest_file = /var/www/oddb.org/data/xml/XMLPublications-latest.zip
2012-05-05 09:01:31 CEST File.exists?(/var/www/oddb.org/data/xml/XMLPublications-latest.zip) = true
2012-05-05 09:01:31 CEST FileUtils.compare_file(/tmp/foo20120505-14836-1rp7847, /var/www/oddb.org/data/xml/XMLPublications-latest.zip) = false
2012-05-05 09:01:31 CEST path = "/var/www/oddb.org/data/xml/XMLPublications-2012.05.05.zip"
2012-05-05 09:01:31 CEST entry.name = Gestrichene_Packungen_Emballages_radies.xls
2012-05-05 09:01:31 CEST entry.name = PR120501.txt
2012-05-05 09:01:31 CEST entry.name = GL_Diff_SB.xml
2012-05-05 09:01:31 CEST entry.name = ItCodes.xml
2012-05-05 09:03:37 CEST entry.name = Preparations.xml

debug file download

Last night, I ran import_daily via cron at 03:00.
It still running (17:00).

checked get_latest_file

in src/plugin/swissmedic.rb

   def get_latest_file(agent, keyword='Packungen')
      page = agent.get @index_url
      links = page.links.select do |link|
        ptrn = keyword.gsub /[^A-Za-z]/u, '.' 
        /#{ptrn}/iu.match link.attributes['title']
      end 
      link = links.first or raise "could not identify url to #{keyword}.xls"
      file = agent.get(link.href)
      download = file.body
      ...
   end

Refs

Continue

to check result of import_dail job via cron.


Added Drug bank link

in result list.

NOTE

At Result list, use AtcFacade class.

  • AtcClass (in src/mode/atcclass.rb)
  • AtcFacade (in src/mode/search_result.rb)
Commit

Added drugbank link to result list

Refs

view · edit · sidebar · attach · print · history
Page last modified on May 25, 2012, at 12:52 PM