view · edit · sidebar · attach · print · history

Masa.20101021-debug-export_chde_xls

Check the result of experiment yesterday
Backtrace the error message one by one
Check data on production server
Try to catch and skip error
Backtrace the error message again suspend
Create a reporting mail process for the error data

Goal

debug export_chde_xls de.oddb.org / 70 %

Milestones

~~Check the result of experiment yesterday again~~ 8:30
~~Trace the error message one by one~~ 10:00
~~Check the data on production server~~ 11:00
~~Try to catch the error and skip the error data~~ 11:40
- If it is possible, then create a report mail ~~suspend~~ -> 6.
~~Trace the error more in order to understand the comparison process~~ suspend 14:30
Create a reporting process for error data
- Grab the error data and create a link
- Create a reporting process, probably refer to the other export processes

Summary

http://masay.ch/2010/10/21/debug-export_chde_xls/

Commits
ToDo Tomorrow
Keep in Mind

emerge --sync
Search a fundamental cause of imoprt_chde_xls bug
Tokyo Tyrant + ODBA
adding a prefix for the pharmacode field in xls2odat
look for the way to delete the product de.oddb.org import_gkv
Organize the information of yus README
Organize the information of de.oddb.org README
Delete all SL-Entries before importing the new flags.
Dimdi-Import-Error.
20100922-trace-limitation-data-flow-in-bsvXml?action=edit
- 20100922 Options for LimitationNiveau
20100908 rpdf2txt GPLv3
block sample http://q.hatena.ne.jp/1217573703 (japanese)
debug ydpm (I cannot create and modify a task detail from my local)
check ydpm priorities

Attached Files

Check the result of experiment yesterday

Experiment (in my local)

oddb.org/src/model/part.rb

    def active_agents
print "@composition.class="
p @composition.class
if @composition.class == ODDB::Composition
print "@compostion.active_agents="
p @compostion.active_agents
end
      @composition ? @composition.active_agents : []
    end

Run export_chde_xls

Result

I got a similar error to the production server EMail Thu Oct 21 07:44:47 2010: ODDB::Export::Xls::ComparisonDeCh

Thu Oct 21 07:44:47 2010: ODDB::Export::Xls::ComparisonDeCh#export
NoMethodError
undefined method `active_agents' for nil:NilClass
(druby://localhost:10013) /home/masa/ywesee/oddb.org/src/model/part.rb:36:in `active_agents'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:141:in `send'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:141:in `method_missing'
(druby://localhost:10013) /home/masa/ywesee/oddb.org/src/model/package.rb:68:in `active_agents'
(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/drb.rb:395:in `inject'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:141:in `each'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:141:in `inject'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:141:in `send'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:141:in `method_missing'
(druby://localhost:10013) /home/masa/ywesee/oddb.org/src/model/package.rb:68:in `active_agents'
(druby://masa:60740) /home/masa/ywesee/de.oddb.org/lib/oddb/remote/drugs/package.rb:29:in `active_agents'
(druby://masa:60740) /home/masa/ywesee/de.oddb.org/lib/oddb/remote/drugs/package.rb:77:in `local_comparables'
(druby://masa:60740) /home/masa/ywesee/de.oddb.org/lib/oddb/export/xls.rb:27:in `collect_comparables'
(druby://masa:60740) /usr/lib64/ruby/1.8/drb/drb.rb:1556:in `call'
(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/invokemethod.rb:10:in `block_yield'
(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/invokemethod.rb:17:in `perform_with_block'
(druby://localhost:10013) /home/masa/ywesee/oddb.org/ext/readonly/src/readonly_server.rb:40:in `call'
(druby://localhost:10013) /home/masa/ywesee/oddb.org/ext/readonly/src/readonly_server.rb:40:in `remote_each_package'
(druby://localhost:10013) /home/masa/ywesee/oddb.org/ext/readonly/src/readonly_server.rb:38:in `each'
(druby://localhost:10013) /home/masa/ywesee/oddb.org/ext/readonly/src/readonly_server.rb:38:in `remote_each_package'
/home/masa/ywesee/de.oddb.org/lib/oddb/export/xls.rb:22:in `collect_comparables'
/home/masa/ywesee/de.oddb.org/lib/oddb/export/xls.rb:15:in `export'
/home/masa/ywesee/de.oddb.org/lib/oddb/export/server.rb:57:in `safe_export'
/usr/lib64/ruby/1.8/tempfile.rb:172:in `open'
/home/masa/ywesee/de.oddb.org/lib/oddb/export/server.rb:54:in `safe_export'
/home/masa/ywesee/de.oddb.org/lib/oddb/export/server.rb:25:in `export_chde_xls'
jobs/export_chde_xls:17
/home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `call'
/home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `run'
jobs/export_chde_xls:16

Experiment

oddb.org/src/model/part.rb

    def active_agents
print "@composition.class="
p @composition.class
print "@compostion="
p @compostion
      @composition ? @composition.active_agents : []
    end

Result

@composition.class=ODDB::Composition
@compostion=nil
@composition.class=NilClass
@compostion=nil
@composition.class=ODDB::Composition
@compostion=nil
@composition.class=ODDB::Composition
@compostion=nil
@composition.class=ODDB::Composition
@compostion=nil
@composition.class=ODDB::Composition
@compostion=nil
@composition.class=ODDB::Composition
@compostion=nil
...

Notes

In spite of the fact that the @compostion.class == ODDB::Compostion, @compostion became nil
The cause of the error mail above is @composition == nil

Backtrace the error message one by one

Error report Wed Oct 20 08:20:00 2010: ODDB::Export::Xls::ComparisonDeCh

Wed Oct 20 08:20:00 2010: ODDB::Export::Xls::ComparisonDeCh#export
NoMethodError
undefined method `active_agents' for nil:NilClass
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'
(druby://localhost:10013) /var/www/oddb.org/src/model/part.rb:31:in `active_agents'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'
(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'
(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'
(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/drb.rb:395:in `inject'
(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `each'
(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `inject'
(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'
(druby://production server:46243) /var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:27:in `active_agents'
(druby://production server:46243) /var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:75:in `local_comparables'
(druby://production server:46243) /var/www/de.oddb.org/lib/oddb/export/xls.rb:27:in `collect_comparables'
(druby://production server:46243) /usr/lib64/ruby/1.8/drb/drb.rb:1555:in `call'
(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/invokemethod.rb:10:in `block_yield'
(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/invokemethod.rb:17:in `perform_with_block'
(druby://localhost:10013) /var/www/oddb.org/ext/readonly/src/readonly_server.rb:40:in `call'
(druby://localhost:10013) /var/www/oddb.org/ext/readonly/src/readonly_server.rb:40:in `remote_each_package'
(druby://localhost:10013) /var/www/oddb.org/ext/readonly/src/readonly_server.rb:38:in `each'
(druby://localhost:10013) /var/www/oddb.org/ext/readonly/src/readonly_server.rb:38:in `remote_each_package'
/var/www/de.oddb.org/lib/oddb/export/xls.rb:22:in `collect_comparables'
/var/www/de.oddb.org/lib/oddb/export/xls.rb:15:in `export'
/var/www/de.oddb.org/lib/oddb/export/server.rb:57:in `safe_export'
/usr/lib64/ruby/1.8/tempfile.rb:172:in `open'
/var/www/de.oddb.org/lib/oddb/export/server.rb:54:in `safe_export'
/var/www/de.oddb.org/lib/oddb/export/server.rb:25:in `export_chde_xls'
jobs/export_chde_xls:17
/var/www/de.oddb.org/lib/oddb/util/job.rb:16:in `call'
/var/www/de.oddb.org/lib/oddb/util/job.rb:16:in `run'
jobs/export_chde_xls:16

jobs/export_chde_xls:16

module ODDB
  module Util
    Job.run :readonly => true do
      Export::Server.export_chde_xls
    end
  end
end

Notes

This just calles Export::Server.export_chde_xls method

de.oddb.org/lib/oddb/export/server.rb:25

      def Server.export_chde_xls
        if uri = ODDB.config.remote_databases.first
          safe_export Export::Xls::ComparisonDeCh, 'chde.xls', uri
        end
      end

Notes

I confirmed that uri == "druby://localhost:10013"
safe_export method is called with the three arguments, Export::Xls::ComparisonDeCh, "chde.xls", uri
- Export::Xls::ComparisonDeCh is a class name
- chde.xls is (probably) a output file name
- uri is the DRb connextion uri to oddb.org (readonlyd)

lib/oddb/export/server.rb:54

      def Server.safe_export(exporter_class, name, *args, &block)
        dir = ODDB.config.export_dir or raise "Please configure 'export_dir'"
        FileUtils.mkdir_p(dir)
        Tempfile.open(name, dir) { |fh|
          exporter = exporter_class.new
          args.push fh
          exporter.export *args
          fh.close
          newpath = File.join(dir, name)
          FileUtils.mv(fh.path, newpath)
          FileUtils.chmod(0644, newpath)
          compress(dir, name)
        }
        name

Notes

I confirmed that dir == "/home/masa/ywesee/de.oddb.org/var/downloads"
Export::Xls::ComparisonDeCh instance is substituted in exporter
args == ["druby://localhost:10013", #<File:/home/masa/ywesee/de.oddb.org/var/downloads/chde.xls.4553.0>]
Mainly Export::Xls::ComparisonDeCh#export method is called with uri as an arguments

lib/oddb/export/xls.rb:15

  def export(drb_uri, io)
    write_xls(io, collect_comparables(drb_uri))
  end

Notes

collect_comparalbles method is called first
write_xls method is called next

drb_uri == "druby://localhost:10013"
io == #<File:/home/masa/ywesee/de.oddb.org/var/downloads/chde.xls.4526.0

lib/oddb/export/xls.rb:22 (collect_comparables method)

  def collect_comparables(drb_uri)
    data = []
    DRb::DRbObject.new(nil, drb_uri).remote_each_package { |remote|
      package = Remote::Drugs::Package.new(drb_uri, remote,
                                           1.0 / currency_rate,
                                           tax_factor)
      if(package.price(:public) \
         && (comparable = package.local_comparables.select { |pac|
            pac.price(:public)
          }.sort_by { |pac|
            pac.price(:public)
          }.first))
        data.push [comparable, package]
      end
      nil # don't return data from the block across drb
    }
    data
  end

Notes

I guess this part is the main part of the comparison process
write_xls method is just writing the data into a file
I guess It is possible to trace the package which cause an error by inserting 'p' here
Data.length == 2435
The number of remote packages == 19237

Check data on production server

Experiment (on production server)

de.oddb.org/lib/oddb/export/xls.rb

  def collect_comparables(drb_uri)
    data = []
count = 0
    DRb::DRbObject.new(nil, drb_uri).remote_each_package { |remote|
count += 1
      package = Remote::Drugs::Package.new(drb_uri, remote,
                                           1.0 / currency_rate,
                                           tax_factor)
print count, " "
p package.name.de
      if(package.price(:public) \
         && (comparable = package.local_comparables.select { |pac|
            pac.price(:public)
          }.sort_by { |pac|
            pac.price(:public)
          }.first))

        data.push [comparable, package]
      end
p "-"
      nil # don't return data from the block across drb
    }
    data
  end

Run export_chde_xls

Result Attach:export_chde_xls_test_result.txt

....
10070 "Jodoplex"
"-"
10071 "Accupro 10"
"-"
10072 "Topamax 50 mg"
"-"
10073 "Bupivacain Sintetica 0,25 %"
"-"
10074 "Travatan 40 ug"
"-"
10075 "Bupivacain Sintetica 0,5 %"
"-"
10076 "Fludara"

(split error happens here)

Notes

It is certain that the error comes in 'if' statement of collect_comparables method
The error happens after the 10076 'Fludara' oddb.org (ch) data

Solution

Is it possible to catch the OdbaError and skip the data?
Trace the code more and understand the comparison process

Try to catch and skip error

Experiment (on production server)

de.oddb.org/lib/oddb/export/xls.rb

  def collect_comparables(drb_uri)
    data = []

count = 0
err_count = 0
    DRb::DRbObject.new(nil, drb_uri).remote_each_package { |remote|
count += 1
err_count += 1
      package = Remote::Drugs::Package.new(drb_uri, remote,
                                           1.0 / currency_rate,
                                           tax_factor)
print count, " "
p package.name.de
begin
      if(package.price(:public) \
         && (comparable = package.local_comparables.select { |pac|
            pac.price(:public)
          }.sort_by { |pac|
            pac.price(:public)
          }.first))

        data.push [comparable, package]
      end
rescue NoMethodError => e
  p e
  p $@
end
      nil # don't return data from the block across drb
    }
exit
    data
  end

Run export_chde_xls

Result

....

10076 "Travatan 40 ug"
10077 "Bupivacain Sintetica 0,5 %"
10078 "Fludara"
#<NoMethodError: undefined method `active_agents' for nil:NilClass>
1
["(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/part.rb:31:in `active_agents'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'", "(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/drb.rb:395:in `inject'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `each'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `inject'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'", "/var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:27:in `active_agents'", "/var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:75:in `local_comparables'", "/var/www/de.oddb.org/lib/oddb/export/xls.rb:33:in `collect_comparables'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `call'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `__send__'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'", "/usr/lib64/ruby/1.8/drb/drb.rb:1515:in `perform'", "/usr/lib64/ruby/1.8/drb/drb.rb:1589:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `start'", "/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1430:in `run'", "/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `start'", "/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `run'", "/usr/lib64/ruby/1.8/drb/drb.rb:1347:in `initialize'", "/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `new'", "/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `start_service'", "/var/www/de.oddb.org/lib/oddb/util/job.rb:13:in `run'", "jobs/export_chde_xls:16"]

....

16766 "Fludarabin Actavis"
#<NoMethodError: undefined method `active_agents' for nil:NilClass>
2
["(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/part.rb:31:in `active_agents'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'", "(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/drb.rb:395:in `inject'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `each'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `inject'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'", "/var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:27:in `active_agents'", "/var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:75:in `local_comparables'", "/var/www/de.oddb.org/lib/oddb/export/xls.rb:33:in `collect_comparables'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `call'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `__send__'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'", "/usr/lib64/ruby/1.8/drb/drb.rb:1515:in `perform'", "/usr/lib64/ruby/1.8/drb/drb.rb:1589:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `start'", "/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1430:in `run'", "/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `start'", "/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `run'", "/usr/lib64/ruby/1.8/drb/drb.rb:1347:in `initialize'", "/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `new'", "/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `start_service'", "/var/www/de.oddb.org/lib/oddb/util/job.rb:13:in `run'", "jobs/export_chde_xls:16"]

....

18417 "Fludarabin Actavis"
#<NoMethodError: undefined method `active_agents' for nil:NilClass>
3
["(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/part.rb:31:in `active_agents'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'", "(druby://localhost:10013) /usr/lib64/ruby/1.8/drb/drb.rb:395:in `inject'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `each'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `inject'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `send'", "(druby://localhost:10013) /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:112:in `method_missing'", "(druby://localhost:10013) /var/www/oddb.org/src/model/package.rb:68:in `active_agents'", "/var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:27:in `active_agents'", "/var/www/de.oddb.org/lib/oddb/remote/drugs/package.rb:75:in `local_comparables'", "/var/www/de.oddb.org/lib/oddb/export/xls.rb:33:in `collect_comparables'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `call'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `__send__'", "/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'", "/usr/lib64/ruby/1.8/drb/drb.rb:1515:in `perform'", "/usr/lib64/ruby/1.8/drb/drb.rb:1589:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `start'", "/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `main_loop'", "/usr/lib64/ruby/1.8/drb/drb.rb:1430:in `run'", "/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `start'", "/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `run'", "/usr/lib64/ruby/1.8/drb/drb.rb:1347:in `initialize'", "/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `new'", "/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `start_service'", "/var/www/de.oddb.org/lib/oddb/util/job.rb:13:in `run'", "jobs/export_chde_xls:16"]

Notes

It is possible to catch and skip the error
The number of the error data is 3

Backtrace the error message again

Start point

lib/oddb/export/xls.rb:22 (collect_comparables method)

  def collect_comparables(drb_uri)
    data = []
    DRb::DRbObject.new(nil, drb_uri).remote_each_package { |remote|
      package = Remote::Drugs::Package.new(drb_uri, remote,
                                           1.0 / currency_rate,
                                           tax_factor)
      if(package.price(:public) \
         && (comparable = package.local_comparables.select { |pac|
            pac.price(:public)
          }.sort_by { |pac|
            pac.price(:public)
          }.first))
        data.push [comparable, package]
      end
      nil # don't return data from the block across drb
    }
    data
  end

package(ODDB::Remote::Drugs::Package).local_comparables

de.oddb.org/lib/oddb/remote/drugs/package.rb:77

  def local_comparables
    comparables = []
    doses = active_agents.collect { |act| act.dose }

    if(doses.size == 1 \
       && (atc = self.atc && ODDB::Drugs::Atc.find_by_code(self.atc.code)))

      descriptions = galenic_forms.collect { |form| form.description.de }
      groupnames = galenic_forms.collect { |form| form.groupname }
      range = (size*0.75)..(size*1.25)

      atc.products.each { |prod|
        prod.sequences.each { |seq|
          if(seq.doses == doses)
            descs = []
            names = []
            seq.galenic_forms.each { |form|
              descs.push form.description
              if grp = form.group
                names.push grp.name
              end
            }
            if(descs == descriptions || groupnames.all? { |name|
               names.any? { |other| other == name } })

              comparables.concat seq.packages.select { |pac|
                range.include?(pac.size)
              }

            end
          end
        }
      }
    end
    comparables
  end

Notes

The 'active_agents' method is doubtful...

Experiment

de.oddb.org/lib/oddb/remote/drugs/package.rb (in my local)

  def active_agents
p @remote.class
exit
    @active_agents ||= @remote.active_agents.collect { |act|
      Remote::Drugs::ActiveAgent.new(@source, act)
    }
  end

suspend

Create a reporting mail process for the error data

ToDo

I have to know how to grab the error data information
I have to know how to create a reporting process (this is probably easy by referring to the other export processes)

How to grab an error data

Experiment

de.oddb.org/lib/oddb/export/xls.rb

  def collect_comparables(drb_uri)
    data = []

    DRb::DRbObject.new(nil, drb_uri).remote_each_package { |remote|
      package = Remote::Drugs::Package.new(drb_uri, remote,
                                           1.0 / currency_rate,
                                           tax_factor)
p package.name.de
p package.code(:ean).value
exit
      if(package.price(:public) \
         && (comparable = package.local_comparables.select { |pac|
            pac.price(:public)
          }.sort_by { |pac|
            pac.price(:public)
          }.first))

        data.push [comparable, package]
      end
      nil # don't return data from the block across drb
    }
    data
  end

Run export_chde_xls

Result

"Viagra 25 mg"
"7680546420246"

Notes

This data can be accessed by the following url http://ch.oddb.org/en/gcc/compare/ean13/7680546420246

How to send a report mail

Refer to de.oddb.org/lib/oddb/export/server.rb

      def Server.safe_export(exporter_class, name, *args, &block)
        dir = ODDB.config.export_dir or raise "Please configure 'export_dir'"
        FileUtils.mkdir_p(dir)
        Tempfile.open(name, dir) { |fh|
          exporter = exporter_class.new
          args.push fh
          exporter.export *args
          fh.close
          newpath = File.join(dir, name)
          FileUtils.mv(fh.path, newpath)
          FileUtils.chmod(0644, newpath)
          compress(dir, name)
        }
        name
      rescue StandardError => err
        subject = sprintf("%s: %s",
                          Time.now.strftime('%c'), exporter_class)
        lines = [
          sprintf("%s: %s#export",
                  Time.now.strftime('%c'), exporter_class)
        ]
        lines.push(err.class, err.message, *err.backtrace)
        Util::Mail.notify_admins(subject, lines)
      end

Experiment (in my local)

de.oddb.org/lib/oddb/export/xls.rb

class ComparisonDeCh

attr_reader :error_data

  def export(drb_uri, io)
    write_xls(io, collect_comparables(drb_uri))
  end
  def adjust_price(price)
    price * currency_rate * tax_factor
  end
  def collect_comparables(drb_uri)
    data = []

@error_data = []
count = 0
    DRb::DRbObject.new(nil, drb_uri).remote_each_package { |remote|
count += 1
      package = Remote::Drugs::Package.new(drb_uri, remote,
                                           1.0 / currency_rate,
                                           tax_factor)
begin
if count == 1 or count == 2
  raise NoMethodError
end
      if(package.price(:public) \
         && (comparable = package.local_comparables.select { |pac|
            pac.price(:public)
          }.sort_by { |pac|
            pac.price(:public)
          }.first))

        data.push [comparable, package]
      end
rescue NoMethodError => err
  @error_data.push("http://ch.oddb.org/en/gcc/compare/ean13/" + package.code(:ean).value)
end

de.oddb.org/lib/oddb/export/server.rb

      def Server.safe_export(exporter_class, name, *args, &block)
        dir = ODDB.config.export_dir or raise "Please configure 'export_dir'"
        FileUtils.mkdir_p(dir)
        Tempfile.open(name, dir) { |fh|
          exporter = exporter_class.new
          args.push fh
          exporter.export *args
          fh.close
          newpath = File.join(dir, name)
          FileUtils.mv(fh.path, newpath)
          FileUtils.chmod(0644, newpath)
          compress(dir, name)

if(exporter_class ==  Export::Xls::ComparisonDeCh)
  unless(exporter.error_data.empty?)
    message = "The following data was not able to be compared due to NoMethodError:\n"
    raise NoMethodError, message + exporter.error_data.join("\n").to_s
  end
end

Result

EMail Thu Oct 21 16:23:18 2010: ODDB::Export::Xls::ComparisonDeCh

Thu Oct 21 16:23:18 2010: ODDB::Export::Xls::ComparisonDeCh#export
NoMethodError
The following data was not able to be compared due to NoMethodError:
http://ch.oddb.org/en/gcc/compare/ean13/7680546420246
http://ch.oddb.org/en/gcc/compare/ean13/7680546420321
/home/masa/ywesee/de.oddb.org/lib/oddb/export/server.rb:67:in `safe_export'
/usr/lib64/ruby/1.8/tempfile.rb:172:in `open'
/home/masa/ywesee/de.oddb.org/lib/oddb/export/server.rb:54:in `safe_export'
/home/masa/ywesee/de.oddb.org/lib/oddb/export/server.rb:25:in `export_chde_xls'
jobs/export_chde_xls:17
/home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `call'
/home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `run'
jobs/export_chde_xls:16

Notes

Not so smart solution but it can report it
Let's think about the other solution
The orignal error backtrace information also should be added in the report mail

Experiment

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

Masa.20101021-debug-export_chde_xls

Check the result of experiment yesterday

Backtrace the error message one by one

Check data on production server

Try to catch and skip error

Backtrace the error message again

Create a reporting mail process for the error data

How to grab an error data

How to send a report mail

ywesee Developer-Wiki Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

Masa.20101021-debug-export_chde_xls

Check the result of experiment yesterday

Backtrace the error message one by one

Check data on production server

Try to catch and skip error

Backtrace the error message again

Create a reporting mail process for the error data

How to grab an error data

How to send a report mail

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler