20101104-update-decorators-for-ebooks

<< Masa.20101105-create-patinfo-ebook | 2010 | Masa.20101103-decorators-for-ebooks >>

1. Understand the decorator structure
2. Update ebps library to be able to create a link on title
3. Set up E-shop in my local suspend
4. Put a drug name before ean code (ebps)

Goal

• Update decorators for Ebooks / 70%

Milestones

1. Experiments of decorator locally for understanding 14:00
2. Update ebps library
3. Put drug name before ean code
4. Set up E-shop in my local

Summary

• http://masay.ch/2010/11/04/update-decorators-for-ebooks/

Commits

• link_drug_brand_name flag can be set to make a link of a page title (ebps)

ToDo Tomorrow

Keep in Mind

• emerge --sync
• Rake http://www2s.biglobe.ne.jp/~idesaku/sss/tech/rake/
• ch.ODDB.org Report - Error Export: oddbdat - 11/2010 (fwd)
• Mon Nov 1 04:09:08 2010: ODDB::Export::Xls::ComparisonDeCh
• Update price feeds did not run automatically
• Replace get_latest_file (update_swissmedic) method by download_file method
• If update_bsv goes well on 1st Dec., let's think about deleting debug print code
• delete-first in update_bsv
• arrange files and back up script oldie
• arrange ydpm job list
• xls2odat argument files
• Search a fundamental cause of export_chde_xls bug
• adding a prefix for the pharmacode field in xls2odat
• Delete all SL-Entries before importing the new flags.
• 20100922-trace-limitation-data-flow-in-bsvXml?action=edit
  • 20100922 Options for LimitationNiveau
• 20100908 rpdf2txt GPLv3
• Tokyo Tyrant + ODBA
• admin tool hack
• Organize the information of yus README
• Organize the information of de.oddb.org README
• block sample http://q.hatena.ne.jp/1217573703 (japanese)
• Debug ODDB::CsvExportPlugin Done

Attached Files

Understand the decorator structure

Check the calculate time to run meddrugs.ch_de_firefox

pre-process: 0'0'0
input-process: 0'1'56
decorate-process: 0'0'0
export-process: 0'2'21
post-process: 0'0'0

In total

• 4 mins 17 secs

Count the number times of calling decorate method

just_medical_de.rb

module EBPS
  module Decorator
    module JustMedical
      def self.decorate model
        @count ||= 0
        p @count+=1, " "
        if codes = model.metadata['article_codes']
          ean13s = codes.collect do |hash| hash[:article_ean13] end.compact
          unless ean13s.empty?
            chapter = Text::Chapter.new
            chapter.heading << 'Preisvergleich'
            ean13s.each do |ean13|
              href = "http://just-medical.oddb.org/de/just-medical/compare/ean13/#{ean13}"
              chapter.add_paragraph Text::LinkedParagraph.new(href, ean13)
            end
            model.add_chapter chapter
          end
        end
      end
    end
  end
end

Run

masa@masa ~/work $ ls
just_medical_de.rb  meddrugs.ch_de_firefox  meddrugs.ch_de_firefox.yml
masa@masa ~/work $ ruby1.9 meddrugs.ch_de_firefox config="meddrugs.ch_de_firefox.yml" 

Result

...
4100 4101 end decorate-process
...

Notes

• def self.decorate method is called 4101 times.

Analysis

meddrugs.ch_de_firefox

....
checkTime :begin, "start decorate-process"

  EBPS.config.decorate.each do |path, name|
    require path
    decorator = identify_module(EBPS::Decorator, name)
    model.each do |doc|
      decorator.decorate(doc)
    end
    log.push sprintf('Decorate: %s.decorate', name)
  end

checkTime :end, "end decorate-process"
....

Notes

• It means that model.length = 4101
• model.class=Array
• model[0].class=EBPS::Text::Document
• self.decorate method in just_medical_de.rb is called for every line

Consider

• I guess metadata method in just_medical_de.rb comes from EBPS::Text::Document class

Check

lib/ebps/text/document.rb

require 'ebps/text/chapter'

module EBPS
  module Text
    class Document
      attr_reader :chapters, :metadata
      attr_accessor :title
      def initialize
        @chapters = []
        @metadata = {}
        @title = ''
      end

Notes

• Bingo: Document class has 'metadata' (hash) as an access method

Check EBPS::Text::Document class structure

See

• Classes under EBPS::Text module

Experiment (Title modification test)

/var/ebps/bin/decorators/just_medical_de.rb

module EBPS
  module Decorator
    module JustMedical
      def self.decorate model
        if model.title =~ /Omeprazol/
        p model.title
        model.title = "Masa" + model.title + "Hata"
        end
...

Run

ruby1.9 meddrugs.ch_de_firefox config="meddrugs.ch_de_firefox.yml"

Result

• It works

Experiment /var/ebps/bin/decorators/just_medical_de.rb

module EBPS
  module Decorator
    module JustMedical
      def self.decorate model
        if model.title =~ /Omeprazol/
        p model.title
        model.title = "<a href=''>" + model.title + "</a>"
        end
...

Result

• It does not work
• I guess I have to use Build library as lib/ebps/conversion/oebps.rb#export

Check export process

/var/ebps/bin/meddrugs.ch_de_firefox

  
...
exporter = identify_module EBPS::Conversion, export_to
...
target = EBPS.config.target
exporter.export model, target
...

/etc/ebps/meddrugs.ch_de_firefox.yml

...
export_to: epub
...

lib/ebps/conversion/epub.rb#export

      def self.export docs, target, override_tmpdir=nil
        Oebps.export docs, target, override_tmpdir do |tmpdir, name|
          compile_epub target, tmpdir, name
        end
      end

Notes

• This method delegates the process to Oebps.export method
• 'docs' is Array of EBPS::Text::Documentation objects ('model' in just_medical_de.rb)

Refer

• Builder RDoc http://builder.rubyforge.org/
• Builder (Japanese) http://d.hatena.ne.jp/Kazzz/20080605/p1
• EBPS::Text::LinkedParagraph class

Experiment

ebps/lib/ebps/conversion/oebps.rb

      class HtmlFactory < Factory
        def body
          @builder.h1 @subject.title, 'id' => @uid
if @subject.title =~ /Omeprazol/
  p @builder
  exit
end
...

Run

masa@masa ~/work $ ruby1.9 -I ebps/lib meddrugs.ch_de_firefox config="meddrugs.ch_de_firefox.yml"

Result

#<Builder::XmlMarkup:0x0000001274e0e8 @indent=0, @level=2, @target="<?xml version=\"1.0\" encoding=\"UTF-8\"?><!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\"><html xmlns=\"http://www.w3.org/1999/xhtml\"><head><meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\"/><title>Omeprazol - 1 A Pharma</title><link type=\"text/css\" rel=\"stylesheet\" href=\"oddb.css\"/></head><body><h1 id=\"id0720facbb3b35a26776c3ea8796f9df1\">Omeprazol - 1 A Pharma</h1>">

Experiment

ebps/lib/ebps/conversion/oebps.rb

        def body
#          @builder.h1 @subject.title, 'id' => @uid
if @subject.title =~ /Omeprazol Actavis/
  @builder.h1 'id' => @uid do |xml|
    xml.a @subject.title, 'href' => 'http://www.ywesee.com'
  end
else
  @builder.h1 @subject.title, 'id' => @uid
end
....

Result

• It works

Notes

• I have found that it is possible to make a link of title by the modification of oebps.rb

Problem

• This solution will influence on the other scripts, since the ebps library changes
• Let's think the way to modify the title link WITHOUT changing ebps library, only by using decorator script suspend

Update ebps library to be able to create a link on title

Next task ebps: link_drug_brand_name - adjust through the /etc/ebs/job_config.yml

• If we set 'link_drug_brand_name: true' in a configuration yaml file, the system creates a link on page title

Test

meddrugs.ch_de_firefox.yml

...
link_drug_brand_name: true
...

ebps/lib/ebps/conversion/oebps.rb

module EBPS
  module Conversion
    module Oebps
      class Factory
        attr_reader :uid
        def initialize subject, ids, tmpdir
          @ids = ids
          @indent = EBPS.config.xml_indent
          @builder = Builder::XmlMarkup.new :indent => @indent
          @builder.instruct! :xml, :version=>"1.0", :encoding=>"UTF-8"
          @subject = subject
          @tmpdir = tmpdir
@link_drug_brand_name = EBPS.config.link_drug_brand_name
        end
      end
....
      class HtmlFactory < Factory
        def body
#          @builder.h1 @subject.title, 'id' => @uid
if @link_drug_brand_name
  search_url = ["http://just-medical.oddb.org/de/just-medical/search/zone/drugs/search_query/",
                @subject.title.delete('®'),
                "/search_type/st_sequence#best_result"].join
  @builder.h1 'id' => @uid do |xml|
    xml.a @subject.title, 'href' => search_url
  end
else
  @builder.h1 @subject.title, 'id' => @uid
end
....

Result

• Success

Notes

• We can also set the option without the modification of yml file as follows:

masa@masa ~/work $ ruby1.9 config="meddrugs.ch_de_firefox.yml" link_drug_brand_name=true

Commit

• link_drug_brand_name flag can be set to make a link of a page title (ebps)

Set up E-shop in my local

Email IP-Address, E-shop Solution: Version 2.0

Put a drug name before ean code (ebps)

just_medical_de.rb

module EBPS
  module Decorator
    module JustMedical
      def self.decorate model
        if codes = model.metadata['article_codes']
          ean13s = codes.collect do |hash| hash[:article_ean13] end.compact
          unless ean13s.empty?
            chapter = Text::Chapter.new
            chapter.heading << 'Preisvergleich'
            ean13s.each do |ean13|
              href = "http://just-medical.oddb.org/de/just-medical/compare/ean13/#{ean13}"
              #chapter.add_paragraph Text::LinkedParagraph.new(href, ean13)

chapter.add_paragraph Text::LinkedParagraph.new(href, model.title + " " + ean13)

            end
            model.add_chapter chapter
          end
        end
      end
    end
  end
end

Result

• Success and uploaded