view · edit · sidebar · attach · print · history

20110202-debug-oddbdat-oddb_org

<< | Index | >>


  1. Check all the test-cases oddb.org suspend
  2. Debug oddbdat error
  3. Temporal solution (Skipping nil data)
  4. Dig into more deeper
  5. Update export_oddbdat with reporting

Goal
  • Update test-cases oddb.org / 70%
Milestones
  1. Check all the test-cases
  2. Debug oddbdat error
    • Confirm the same error in local 9:15
    • Traceback error 15:00
    • Update with report process
    • Test-cases
    • Commit
Summary
Commits
ToDo Tomorrow
Keep in Mind
  1. Also probably Dose error ch.ODDB.org Report - Error: Generikaliste - 02/2011 ch.ODDB.org Report - Error: oddb.csv - 02/2011
  2. Add my email address to SL-Update recipients
  3. Debug bsv_followers autorun
  4. oddb.org test-case 100%
  5. Some concerns about FI news job
  6. oddbbat error
  7. Check rdbi instead of dbi for ODBA
  8. Encoding woes (from Davatz-san)
  9. Feedback: This option indicates that the regular expression is parsed as 'UTF8' (from Davatz-san)
  10. pg on Ubuntu - see http://dev.ywesee.com/wiki.php/Gem/Pg (from Davatz-san)
  11. On Ice
  12. emerge --sync

Check all the test-cases oddb.org

masa@masa ~/ywesee/oddb.org $ ruby test/suite.rb 
...
1006 tests, 2351 assertions, 7 failures, 43 errors

Note

  • (7+43)/1006 = 95% success, but actually this does not become a validation
  • since a test-case sometimes stops after an error

Debug oddbdat error

Email

Error: NoMethodError
Message: undefined method `qty' for nil:NilClass
Backtrace:
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:408:in `structure'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:269:in `initialize'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:292:in `initialize'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:53:in `new'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:53:in `lines'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:175:in `export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:174:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:174:in `export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:172:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:172:in `export_oddbdat'
/var/www/oddb.org/src/plugin/oddbdat_export.rb:51:in `export'
/var/www/oddb.org/src/util/exporter.rb:152:in `export_oddbdat'
/var/www/oddb.org/src/util/exporter.rb:289:in `call'
/var/www/oddb.org/src/util/exporter.rb:289:in `safe_export'
/var/www/oddb.org/src/util/exporter.rb:150:in `export_oddbdat'
/var/www/oddb.org/src/util/exporter.rb:51:in `run'
/var/www/oddb.org/jobs/export_daily:13
/var/www/oddb.org/src/util/job.rb:17:in `call'
/var/www/oddb.org/src/util/job.rb:17:in `run'
/var/www/oddb.org/jobs/export_daily:12

Confirm the same error in local

Run

  • oddb.org/bin/oddbd
  • currency/bin/currencyd
  • oddb.org/ext/export/bin/exportd

Run export_oddbdat

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Exporter.new(self).export_oddbdat

Result

Error: NoMethodError
Message: undefined method `qty' for nil:NilClass
Backtrace:
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:408:in `structure'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:269:in `initialize'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:292:in `initialize'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:53:in `new'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:53:in `lines'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:175:in `export_oddbdat'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:174:in `each'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:174:in `export_oddbdat'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:172:in `each'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:172:in `export_oddbdat'
/home/masa/ywesee/oddb.org/src/plugin/oddbdat_export.rb:51:in `export'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:153:in `export_oddbdat'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:297:in `call'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:297:in `safe_export'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:149:in `export_oddbdat'
(eval):1:in `_admin'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1399:in `_admin'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1397:in `initialize'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1397:in `new'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1397:in `_admin'
/usr/lib64/ruby/1.8/drb/drb.rb:1556:in `__send__'
/usr/lib64/ruby/1.8/drb/drb.rb:1556:in `perform_without_block'
/usr/lib64/ruby/1.8/drb/drb.rb:1516:in `perform'
/usr/lib64/ruby/1.8/drb/drb.rb:1590:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1586:in `loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1586:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1582:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1582:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1431:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1428:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1428:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1348:in `initialize'
/usr/lib64/ruby/1.8/drb/drb.rb:1628:in `new'
/usr/lib64/ruby/1.8/drb/drb.rb:1628:in `start_service'
bin/oddbd:38

Note

  • It is the same error

Next

  • Traceback the error message

Experiment

src/plugin/oddbdat_export.rb#export

    def export
        files = []
p "A"
        # package tables
        package_tables = [
            AcTable, AccompTable, AcLimTable, AcmedTable, AcnamTable,
            AcOddbTable, AcpricealgTable, AcscTable, LimitationTable,
            LimTxtTable, EanTable
        ]
        ids = []
        @app.each_package { |pac| ids.push(pac.odba_id) }
p "A2"
        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, package_tables)
p "B"
        # codes table
        ids = @app.atc_classes.values.collect { |atc| atc.odba_id }
p "B2"
        @app.each_galenic_form { |galform| ids.push(galform.odba_id) }
p "B3"
        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, [CodesTable])
p "C"
        # substances table
        ids = @app.substances.collect { |subs| subs.odba_id }
p "C2"
        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, [ScTable])
p "D"
        # companies table
        ids = @app.companies.collect { |oid, comp| comp.odba_id }
p "D2"
        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, [CompTable])
p "E"
        # readme
        files += EXPORT_SERVER.export_oddbdat(nil, EXPORT_DIR, [Readme])
p "F"
        # compress
        EXPORT_SERVER.compress_many(EXPORT_DIR, 'oddbdat', files)
    end

Run again

Result

"getin export_oddbdat"
"1"
"2"
"A"
"A2"

Note

  • The following part does not run
 files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, package_tables)

Next

  • Check EXPORT_SERVER.export_oddbdat

Experiment

ext/export/src/odba_exporter.rb#export_oddbdat

        def OdbaExporter.export_oddbdat(odba_ids, dir, klasses)
            FileUtils.mkdir_p(dir)
p "AA"
            files = klasses.collect { |klass| 
                table = klass.new
                file = Tempfile.new(table.filename, dir)
                [file, table]
            }
p "BB"
            if(odba_ids.nil?)
p "CC"
                files.each { |file, table|
                    file.puts table.lines
                }
            else
p "DD"
                odba_ids.each { |odba_id|
                    item = ODBA.cache.fetch(odba_id, nil)
                    files.each { |file, table|
                        file.puts table.lines(item)
                    }
                    #ODBA.cache.clear
                }
            end
p "EE"
            files.each { |file, table|
                path = File.join(dir, table.filename)
                FileUtils.mv(file.path, path)
            }
p "FF"
            files.collect { |file, table| table.filename }
        ensure
            if(files)
                files.each { |file, table| file.close! }
            end
        end

Run again

Result

"AA"
"BB"
"DD"

Next ext/export/src/odba_exporter.rb#export_oddbdat

        def OdbaExporter.export_oddbdat(odba_ids, dir, klasses)
            FileUtils.mkdir_p(dir)
p "AA"
            files = klasses.collect { |klass|
                table = klass.new
                file = Tempfile.new(table.filename, dir)
                [file, table]
            }
p "BB"
            if(odba_ids.nil?)
p "CC"
                files.each { |file, table|
                    file.puts table.lines
                }
            else
p "DD"
                odba_ids.each { |odba_id|
print "odba_id="
p odba_id
                    item = ODBA.cache.fetch(odba_id, nil)
                    files.each { |file, table|
print "file="
p file
print "table.class="
p table.class
                        file.puts table.lines(item)
                    }
                    #ODBA.cache.clear
                }
            end
p "EE"
            files.each { |file, table|
                path = File.join(dir, table.filename)
                FileUtils.mv(file.path, path)
            }
p "FF"
            files.collect { |file, table| table.filename }
        ensure
            if(files)
                files.each { |file, table| file.close! }
            end
        end

Result

...
odba_id=218092
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s01x.4036.0>
table.class=ODDB::OdbaExporter::AcTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s19x.4036.0>
table.class=ODDB::OdbaExporter::AccompTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s09x.4036.0>
table.class=ODDB::OdbaExporter::AcLimTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s02x.4036.0>
table.class=ODDB::OdbaExporter::AcmedTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s03x.4036.0>
table.class=ODDB::OdbaExporter::AcnamTable

Keep in mind

  • 'odba_id=218092'

Next

  • Check ODDB::OdbaExporter::AcnamTable

src/ext/export/src/oddbdat.rb

        class AcnamTable < Table
            FILENAME = 's03x'
            def lines(package)
                [AcnamLine.new(package)]
            end
        end

Note

  • The argument looks 'Package' class

Next

  • Check AcnamLine.new
        class AcnamLine < PackageLine
...

Next

  • Check PackageLine.new
        class PackageLine < Line
            def initialize(package, *args)
                @package = package
                super
            end
        end

Next

  • Check Line.new
        class Line
            def initialize(*args)
                @date = Date.today.strftime("%Y%m%d%H%M%S")
                @structure = structure
            end

Next

  • AcnamLine#structure

ext/export/src/oddbdat.rb::AcnamLine#structure

        class AcnamLine < PackageLine
            LENGTH = 26
            def structure
                seq = @package.sequence
                galform = if(gf = seq.galenic_forms.first)
                    gf.to_s
                end
                conc, unit = if((dose = seq.dose) && dose.is_a?(Quanty))
                    [dose.qty, dose.unit]
                end
        measures = @package.parts.collect { |part| part.measure }.compact
                measure, munit = unless(measures.empty?)
                           ms = measures.inject { |a, b| a + b }
                           [ms.qty, ms.unit] if ms.is_a?(Quanty)
                         end
        comform = @package.commercial_forms.first
                qty, qty_unit = if(munit && !munit.empty?)
                    [measure, munit]
                else
                    [@package.comparable_size.qty, comform]          # <= HERE
                end

Note

  • @package probably comes from ODBA.cache.fetch(218092)

Experiment ext/export/src/oddbdat.rb::AcnamLine#structure

        class AcnamLine < PackageLine
            LENGTH = 26
            def structure
                seq = @package.sequence
                galform = if(gf = seq.galenic_forms.first)
                    gf.to_s
                end
                conc, unit = if((dose = seq.dose) && dose.is_a?(Quanty))
                    [dose.qty, dose.unit]
                end
        measures = @package.parts.collect { |part| part.measure }.compact
                measure, munit = unless(measures.empty?)
                           ms = measures.inject { |a, b| a + b }
                           [ms.qty, ms.unit] if ms.is_a?(Quanty)
                         end
        comform = @package.commercial_forms.first
                qty, qty_unit = if(munit && !munit.empty?)
                    [measure, munit]
                else
                    [@package.comparable_size.qty, comform]          # <= HERE
                end

Experiment

        class AcnamLine < PackageLine
            LENGTH = 26
            def structure
                seq = @package.sequence
                galform = if(gf = seq.galenic_forms.first)
                    gf.to_s
                end
                conc, unit = if((dose = seq.dose) && dose.is_a?(Quanty))
                    [dose.qty, dose.unit]
                end
        measures = @package.parts.collect { |part| part.measure }.compact
                measure, munit = unless(measures.empty?)
                           ms = measures.inject { |a, b| a + b }
                           [ms.qty, ms.unit] if ms.is_a?(Quanty)
                         end
        comform = @package.commercial_forms.first
                qty, qty_unit = if(munit && !munit.empty?)
                    [measure, munit]
                else
print "@package="
p @package
                    [@package.comparable_size.qty, comform]          # <= HERE
                end

Result

...
odba_id=218092
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s01x.4811.0>
table.class=ODDB::OdbaExporter::AcTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s19x.4811.0>
table.class=ODDB::OdbaExporter::AccompTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s09x.4811.0>
table.class=ODDB::OdbaExporter::AcLimTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s02x.4811.0>
table.class=ODDB::OdbaExporter::AcmedTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s03x.4811.0>
table.class=ODDB::OdbaExporter::AcnamTable
@package.class=ODDB::Package

Next

  • Check ODDB::Package#comparable_size method

Experiment

src/model/package.rb#comparable_size

    def comparable_size
print "@parts="
p @parts
      @parts.collect { |part| part.comparable_size }.inject { |a, b| a + b }
    rescue RuntimeError
      @parts.inject(Dose.new(0)) { |comp, part|
        ODDB::Dose.new(comp.qty + part.comparable_size.qty)
      } rescue nil
    end

Note

  • 'ODDB::Package#comparable_size' means the sum of its @parts comparable_sizes
  • But ODDB::Package#comparable_size should have 'qty' method too

Result

...
odba_id=218092
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s01x.5495.0>
table.class=ODDB::OdbaExporter::AcTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s19x.5495.0>
table.class=ODDB::OdbaExporter::AccompTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s09x.5495.0>
table.class=ODDB::OdbaExporter::AcLimTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s02x.5495.0>
table.class=ODDB::OdbaExporter::AcmedTable
file=#<File:/home/masa/ywesee/oddb.org/data/downloads/s03x.5495.0>
table.class=ODDB::OdbaExporter::AcnamTable
@package.class=ODDB::Package
@parts=[]

Note

  • There is no @parts element

Next

  • Check ODDB::Package@parts

src/model/package.rb#create_part

    def create_part
      part = Part.new
      part.package = self
      @parts.push part
      part
    end

Note

  • @parts is an Array of Part instance

Next

  • Check Part class

src/model/part.rb

  class Part
    include Persistence
    include SizeParser

Next

  • Module SizeParser

src/model/package.rb::SizeParser#comparable_size

    def comparable_size
      ODDB::Dose.from_quanty(@comparable_size)
    end

Note

  • Namely, ODDB::Package#comparable_size comes from ODDB::SizeParser#comparable_size
  • But this comes from ODDB::Dose#from_quanty method (and ODDB::SizeParser@comparable_size)
  • The ODDB::SizeParser@comparable_size is set in the following ODDB::SizeParser#set_comparable_size!

src/model/package.rb::SizeParser#set_comparable_size!

    def set_comparable_size!
      measure = (@measure.nil? || @measure == UNIT) \
              ? _composition_scale \
              : @measure
      measure ||= UNIT
      scale = @scale || UNIT
      @comparable_size = multiplier * measure / scale
    end

Note

  • ODDB::SizeParser@comparable_size should be float variable (probably not an integer)

Experiment

ext/export/src/oddbdat.rb::AcnamLine#structure

        class AcnamLine < PackageLine
            LENGTH = 26
            def structure
                seq = @package.sequence
                galform = if(gf = seq.galenic_forms.first)
                    gf.to_s
                end
                conc, unit = if((dose = seq.dose) && dose.is_a?(Quanty))
                    [dose.qty, dose.unit]
                end
        measures = @package.parts.collect { |part| part.measure }.compact
                measure, munit = unless(measures.empty?)
                           ms = measures.inject { |a, b| a + b }
                           [ms.qty, ms.unit] if ms.is_a?(Quanty)
                         end
        comform = @package.commercial_forms.first
                qty, qty_unit = if(munit && !munit.empty?)
                    [measure, munit]
                else
print "@package.class="
p @package.class
if sss = @package.comparable_size
  print "@package.comparable_size.class="
  p @package.comparable_size.class
end

                    [@package.comparable_size.qty, comform]          # <= HERE
                end

Result

...
@package.comparable_size.class=ODDB::Dose
...

Note

  • So, the 'qty' is a method of ODDB::Dose

Consideration

  • The problem is [].inject becomes nil
  • I guess the following modification will resolve the error
  • but this is not fundamental understanding of mine

src/model/package.rb#comparable_size

    def comparable_size
#      @parts.collect { |part| part.comparable_size }.inject { |a, b| a + b } 
      @parts.collect { |part| part.comparable_size }.inject { |a, b| a + b } or raise RuntimeError
    rescue RuntimeError
      @parts.inject(Dose.new(0)) { |comp, part|
        ODDB::Dose.new(comp.qty + part.comparable_size.qty)
      } rescue nil
    end

Result

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> ODBA.cache.fetch('218092', nil).name
-> Phytomed Knoblauch/Weissdorn, Kapseln
ch.oddb> ODBA.cache.fetch('218092', nil).comparable_size
-> 0 

Test

Reboot

  • oddb.org/bin/oddbd
  • oddb.org/ext/export/bin/exportd

Run

  • currency/bin/currencyd

Run export_oddbdat

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Exporter.new(self).export_oddbdat

Result

Error: NoMethodError
Message: undefined method `multi' for nil:NilClass
Backtrace:
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:423:in `structure'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:269:in `initialize'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:292:in `initialize'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:53:in `new'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/oddbdat.rb:53:in `lines'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:189:in `export_oddbdat'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:184:in `each'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:184:in `export_oddbdat'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:176:in `each'
(druby://localhost:10005) /home/masa/ywesee/oddb.org/ext/export/src/odba_exporter.rb:176:in `export_oddbdat'
/home/masa/ywesee/oddb.org/src/plugin/oddbdat_export.rb:52:in `export'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:153:in `export_oddbdat'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:297:in `call'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:297:in `safe_export'
/home/masa/ywesee/oddb.org/src/util/exporter.rb:149:in `export_oddbdat'
(eval):1:in `_admin'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1399:in `_admin'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1397:in `initialize'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1397:in `new'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1397:in `_admin'
/usr/lib64/ruby/1.8/drb/drb.rb:1556:in `__send__'
/usr/lib64/ruby/1.8/drb/drb.rb:1556:in `perform_without_block'
/usr/lib64/ruby/1.8/drb/drb.rb:1516:in `perform'
/usr/lib64/ruby/1.8/drb/drb.rb:1590:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1586:in `loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1586:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1582:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1582:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1431:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1428:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1428:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1348:in `initialize'
/usr/lib64/ruby/1.8/drb/drb.rb:1628:in `new'
/usr/lib64/ruby/1.8/drb/drb.rb:1628:in `start_service'
bin/oddbd:38

Note

  • This is a different error but it looks coming from the similar part
  • This error comes from
        class AcnamLine < PackageLine
            LENGTH = 26
            def structure
                seq = @package.sequence
                galform = if(gf = seq.galenic_forms.first)
                    gf.to_s
                end
                conc, unit = if((dose = seq.dose) && dose.is_a?(Quanty))
                    [dose.qty, dose.unit]
                end
        measures = @package.parts.collect { |part| part.measure }.compact
                measure, munit = unless(measures.empty?)
                           ms = measures.inject { |a, b| a + b }
                           [ms.qty, ms.unit] if ms.is_a?(Quanty)
                         end
        comform = @package.commercial_forms.first
                qty, qty_unit = if(munit && !munit.empty?)
                    [measure, munit]
                else
                    [@package.comparable_size.qty, comform]
                end
                {
                    1       =>  '03',
                    2       =>  @date,
                    3       =>  '1',
                    4       =>  @package.oid,
                    5       =>  'D',
                    6       =>  '4',
                    7       =>  seq.name,
                    8       =>  seq.name_base,
                    9       =>  seq.name_descr,
                    11  =>  galform,
                    12  =>  conc,
                    13  =>  unit,
                    16  =>  @package.parts.first.multi,     #<= HERE
                    17  =>  comform,
                    18  =>  qty,
                    19  =>  qty_unit,
                }
            end
        end

Note

  • This coding indicates that
    • it seems that ODDB::Package instance should have @parts (an Array of Part instances) elements, at least one.

Temporal solution (Skipping nil data)

Experiment (Skip nil data)

src/model/package.rb#comparable_size

    def comparable_size
#      @parts.collect { |part| part.comparable_size }.inject { |a, b| a + b } 
      @parts.collect { |part| part.comparable_size }.inject { |a, b| a + b } or raise RuntimeError
    rescue RuntimeError
      @parts.inject(Dose.new(0)) { |comp, part|
        ODDB::Dose.new(comp.qty + part.comparable_size.qty)
      } rescue nil
    end

ext/export/src/oddbdat.rb::AcnamLine#structure

...
                {
                    1       =>  '03',
                    2       =>  @date,
                    3       =>  '1',
                    4       =>  @package.oid,
                    5       =>  'D',
                    6       =>  '4',
                    7       =>  seq.name,
                    8       =>  seq.name_base,
                    9       =>  seq.name_descr,
                    11  =>  galform,
                    12  =>  conc,
                    13  =>  unit,
                    #16 =>  @package.parts.first.multi,
                    16  =>  @package.parts.first ? @package.parts.first.multi : 'masa',
                    17  =>  comform,
                    18  =>  qty,
                    19  =>  qty_unit,
                }
            end

Run

  • oddb.org/bin/oddbd
  • currency/bin/currencyd
  • oddb.org/ext/export/bin/exportd

Run export_oddbdat

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Exporter.new(self).export_oddbdat

Result

  • No email comes
  • New files are created
masa@masa ~/ywesee/oddb.org $ ls data/downloads/ -al
insgesamt 19000
drwxr-xr-x 2 masa masa    4096  2. Feb 13:55 .
drwxr-xr-x 8 masa masa      64  2. Feb 13:47 ..
-rw-r--r-- 1 masa masa 2129601  2. Feb 13:55 oddbdat.tar.gz
-rw-r--r-- 1 masa masa 2114956  2. Feb 13:55 oddbdat.zip
-rw------- 1 masa masa    1564  2. Feb 13:55 README
-rw------- 1 masa masa 2267400  2. Feb 13:54 s01x
-rw------- 1 masa masa 1490904  2. Feb 13:54 s02x
-rw------- 1 masa masa 3723469  2. Feb 13:54 s03x
-rw------- 1 masa masa 1094490  2. Feb 13:54 s06x
-rw------- 1 masa masa  905903  2. Feb 13:54 s07x
-rw------- 1 masa masa  111740  2. Feb 13:54 s09x
-rw------- 1 masa masa 1430231  2. Feb 13:54 s10x
-rw------- 1 masa masa  416422  2. Feb 13:55 s11x
-rw------- 1 masa masa   73769  2. Feb 13:55 s12x
-rw------- 1 masa masa  105999  2. Feb 13:54 s16x
-rw------- 1 masa masa  827293  2. Feb 13:54 s19x
-rw------- 1 masa masa  484995  2. Feb 13:55 s40x
-rw------- 1 masa masa 2023634  2. Feb 13:54 s41x
-rw------- 1 masa masa  208172  2. Feb 13:54 s99x

Note

  • This is not fundamental solution, since I do not understand why ODDB::Package@parts becomes empty
  • This is just skipping the nil data that raises an error

Dig into more deeper

Question

  • Why (How) does ODDB::Package@parts becomes [] (emptiy)?
    • This is the cause of the error above

Next

  • When is the Package@parts set?

src/model/package.rb#create_parts

    def create_part
      part = Part.new
      part.package = self
      @parts.push part
      part
    end

Note

  • But 'parts' is an Array and is set as 'attr_reader', this indicates that @parts might be changed outside of the class

Memo

  • The ODDB::Package.parts (Dose instances) are corresponding to the following interface

Note

  • That is why ODDB::Package@parts (Array) is empty

Solution

  • Skipping the empty (nil) data but reporting the data

Update export_oddbdat with reporting

Question

  • Where should I catch and throw the error?

Candidates

  1. src/model/package.rb (Package#comparable_size) (core part, this is related to many objects)
  2. ext/export/src/odba_exporter.rb (OdbaExporter.export_oddbdat, checking it with odba_id) (middle, only for export_oddbdat job)
  3. ext/export/src/oddbdat.rb (AcnamLine#structure, package.comparable_size is nil) (surface, there may be many other places regarding qty)

Conditions

  • We do not want to stop the export_oddbdat job
  • We want an report mail when the Dose data is missing

Consideration

  • 1. is the best position to be caught, but it is difficult to send an report
  • 2. is better than 3., but still difficult to report an email and at the same time to resume the process

Memo

  • The link to a package will be as follows
 http://ch.oddb.org/de/gcc/resolve/pointer/%3A!registration%2C(reg.no.)!sequence%2C(seq.no.)!package%2C(pac.no.).

Experiment

src/model/package.rb#comparable_size

    def comparable_size
      @parts.collect { |part| part.comparable_size }.inject{ |a, b| a + b } or raise RuntimeError
    rescue RuntimeError
      @parts.inject(Dose.new(0)) { |comp, part|
        ODDB::Dose.new(comp.qty + part.comparable_size.qty)
      } rescue nil
    end

ext/export/src/oddbdat.rb::AcnamLine#structure

                {
                    1       =>  '03',
                    2       =>  @date,
                    3       =>  '1',
                    4       =>  @package.oid,
                    5       =>  'D',
                    6       =>  '4',
                    7       =>  seq.name,
                    8       =>  seq.name_base,
                    9       =>  seq.name_descr,
                    11  =>  galform,
                    12  =>  conc,
                    13  =>  unit,
                    #16 =>  @package.parts.first.multi,
                    16  =>  @package.parts.first ? @package.parts.first.multi : '',
                    17  =>  comform,
                    18  =>  qty,
                    19  =>  qty_unit,
                }
            end

src/plugin/oddbdat_export.rb#export

    def export
        files = []
        # package tables
        package_tables = [
            AcTable, AccompTable, AcLimTable, AcmedTable, AcnamTable,
            AcOddbTable, AcpricealgTable, AcscTable, LimitationTable,
            LimTxtTable, EanTable
        ]
        ids = []
        dose_missing_list = []
        @app.each_package { |pac|
          ids.push(pac.odba_id)
          if pac.parts.empty?
            dose_missing_list.push([pac.basename, pac.iksnr, pac.sequence.seqnr, pac.ikscd])
          end
        }

        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, package_tables)
        # codes table
        ids = @app.atc_classes.values.collect { |atc| atc.odba_id }
        @app.each_galenic_form { |galform| ids.push(galform.odba_id) }
        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, [CodesTable])
        # substances table
        ids = @app.substances.collect { |subs| subs.odba_id }
        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, [ScTable])
        # companies table
        ids = @app.companies.collect { |oid, comp| comp.odba_id }
        files += EXPORT_SERVER.export_oddbdat(ids, EXPORT_DIR, [CompTable])
        # readme
        files += EXPORT_SERVER.export_oddbdat(nil, EXPORT_DIR, [Readme])
        # compress
        EXPORT_SERVER.compress_many(EXPORT_DIR, 'oddbdat', files)
        # warning
        return dose_missing_list
    end

src/util/exporter.rb#export_oddbdat

        def export_oddbdat
          dose_missing_list = []
      safe_export 'oddbdat' do
        exporter = OdbaExporter::OddbDatExport.new(@app)
        dose_missing_list = exporter.export
        EXPORT_SERVER.clear
        sleep(30)
        run_on_weekday(1) {
          exporter.export_fachinfos
          EXPORT_SERVER.clear
          sleep(30)
        }

        # here to raise warning if package.parts is empty
        if !dose_missing_list.empty?
            log = Log.new(@@today)
            log.report = [
                "Warning: Dose data is missing",
                "Message: export_oddbdat succeeded but the following package(s) do not have Dose data",
                "Package(s):",
                dose_missing_list.collect do |list|
                  list[0].to_s + ", " + \
                  "http://ch.oddb.org/de/gcc/resolve/pointer/%3A!registration%2C" + list[1].to_s + \
                  "!sequence%2C" + list[2].to_s + "!package%2C" + list[3].to_s + ".\n"
                end
            ].join("\n")
            log.notify("Warning Export: oddbdat")
        end
      end

Run

  • oddb.org/bin/oddbd
  • currency/bin/currencyd
  • oddb.org/export/bin/exportd

Run export_oddbdat

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> Exporter.new(self).export_oddbdat

Result

Warning: Dose data (ODDB::Package.parts, Array of ODDB::Dose instances) is empty.
Message: export_oddbdat succeeded but the following package(s) do not have Dose data.
Package(s):
Phytomed Knoblauch/Weissdorn, http://oddb.masa.org/de/gcc/resolve/pointer/%3A!registration%2C46799!sequence%2C02!package%2C011.

Note

Commit

view · edit · sidebar · attach · print · history
Page last modified on February 03, 2011, at 08:40 AM