20120626-anwendung-search-dosing-link-test

<< | Index | >>

Summary

• created Anwendung search
  • I had to update some invalid data via bin/admin
  • update model to prevent error by unexcpeted values.
• created test-case for dosing link updater job.

Commits

• Updated indication search
• Added test-case of dosing link updater

Index

• Anwendung search
• Testcase of dosing link updater
• Debug text-info updater, fiparsed

Anwendung search

I have to fixd data.
I removed this unexpected indication.(It may also data-corruption.)

ch.oddb> indications.select {|i| i.registrations.is_a? ODDB::Migel::Item }.length
-> 1
ch.oddb> indications.select {|i| i.registrations.is_a? ODDB::Migel::Item }.first.class
-> ODDB::Indication

restored data from production server.

Problem

Registration class has some unexcpected value as atc_classes.
follownig code caused error.

in src/model/indication.rb

      def atc_classes
        ...
        atcs = @registrations.inject([]) do |memo, reg| 
          memo.concat reg.atc_classes
        end
        ...

ch.oddb> indications.select {|i| !i.registrations.is_a? Array }.length
-> 1
ch.oddb> indications.select {|i| !i.registrations.is_a? Array }.first.registrations
-> #<ODDB::Migel::Item:0x0000000c504720>
ch.oddb> indications.select {|i| !i.registrations.is_a? Array }.first.registrations.odba_delete
-> #<ODDB::Migel::Item:0x0000000c504720>

ch.oddb> indications.select {|i| i.registrations.nil? }.length
-> 1
ch.oddb> indications.select {|i| i.registrations.nil? }.first
-> Neuroleptika
ch.oddb> indications.select {|i| i.registrations.nil? }.first.odba_id
-> 27218815

• Registration has nil as sequence
• Indication has MiGel Item as registration

Updated indication search (:st_indication).

commit

• Updated indication search

z.B.

• http://ch.oddb.org/de/gcc/search/zone/drugs/search_query/Beta/search_type/st_indication#best_result

NOTE

trans_handler needs Apache reboot.

[Tue Jun 26 14:00:39 2012] [error] mod_ruby: error in ruby
[Tue Jun 26 14:00:39 2012] [error] mod_ruby: /var/www/ch.oddb.org/src/util/trans_handler.rb:13:in `handle_shortcut': incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string) (Encoding::CompatibilityError)
[Tue Jun 26 14:00:39 2012] [error] mod_ruby:   from /path/to/ruby/lib/ruby/gems/1.9.1/gems/sbsm-1.1.8/lib/sbsm/trans_handler.rb:99:in `translate_uri'

Testcase of dosing link updater

I created tast-case for dosing link updater-job

Loaded suite test/test_plugin/dosing
Started
....

Finished in 0.003275636 seconds.

4 tests, 28 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

1221.14 tests/s, 8547.96 assertions/s

commit

• Added test-case of dosing link updater

Debug text-info updater, fiparsed

Problem

update_company_textinfos updater

Plugin: ODDB::TextInfoPlugin
Error: ArgumentError
Message: invalid byte sequence in US-ASCII
Backtrace:
(druby://localhost:10002)
/usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.4/lib/hpricot/builder.rb:9:in
`gsub'
(druby://localhost:10002)
/usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.4/lib/hpricot/builder.rb:9:in
`uxs'
(druby://localhost:10002)
/usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.4/lib/hpricot/tag.rb:139:in
`to_s'
(druby://localhost:10002)
/usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.4/lib/hpricot/traverse.rb:160:in
`block in inner_text'
(druby://localhost:10002)
/usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.4/lib/hpricot/traverse.rb:160:in
`map'
(druby://localhost:10002)
/usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.4/lib/hpricot/traverse.rb:160:in
`inner_text'
(druby://localhost:10002)
/var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:121:in `text'
(druby://localhost:10002)
/var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:36:in `extract'
(druby://localhost:10002)
/var/www/oddb.org/ext/fiparse/src/fiparse.rb:55:in
`parse_fachinfo_html'

Experiment

in ext/fiparse/src/textinfo_hpricot.rb

  def text(elem)
->  p elem.class
    str = elem.inner_text || elem.to_s
->  if str.encoding.to_s.downcase != 'utf-8'
->    p str.encoding
->    p str 
->  end 
    target_encoding(str.gsub(/( |\s)+/u, ' ').gsub(/[■]/u, '').strip)
  end

• Hpricot::Text
• Hpricot::Elem

following methods use inner_text of hpricot.

• preformatted_text (for th, td)
• text

hpricot

in /path/to/gems/hplicot-0.8.6/lib/hpricot/elements.rb

   class Elements < Array
    ...
    # Returns an string containing the text contents of each element in this list.
    # All HTML tags are removed.
    def inner_text
      map { |x| x.inner_text }.join
    end
    alias_method :text, :inner_text

in /path/to/gems/hplicot-0.8.6/lib/hpricot/builder.rb

  # XML unescape
  def self.uxs(str)
    str.to_s.
        gsub(/\&(\w+);/) { [NamedCharacters[$1] || 63].pack("U*") }. # 63 = ?? (query char)
        gsub(/\&\#(\d+);/) { [$1.to_i].pack("U*") }.
        gsub(/\&\#x([0-9a-fA-F]+);/) { [$1.to_i(16)].pack("U*") }
  end

in hpricot-0.8.6/lib/hpricot/tag.rb

  class Text
    def initialize content; self.content = content end 
    def pathname; "text()" end 
    def to_s
      Hpricot.uxs(content)
    end 
    alias_method :inner_text, :to_s
    ...

NOTE

• hpricot is over