20110608-fix-galenic_form-estcase-plugin-analysis-swissreg-ch_oddb

<< | Index | >>

1. Check the algorithm, product name, description, and galenic form
2. Fix update_compositions method
3. Testcases

Goal/Estimate/Evaluation

• Fix the search result view / 80% / 90%

Milestones

• Check galenic form process
• Test run locally

Summary

• http://masay.ch/2011/06/08/fix-galenic_form-estcase-plugin-analysis-swissreg-ch_oddb/

Commits

• Updated update_sequence and update_galenic_form methods in swissmedic plugin in order to update galenic form correctly
• Updated testcases, plugin/analysis.rb, swissreg.rb (ch.oddb)

Check the algorithm, product name, description, and galenic form

Test

# new algorithm
puts "New algorithm"
name  = 'Iscador M 0,0001 mg, Injektionslösung, anthroposophisches Arzneimittel'
#name  = 'Helixor A 0,01 mg Injektionslösung, anthrop. Arzneimittel'
parts = name.split(/\s*,(?!\d|[^(]+\))\s*/u)
base = parts.shift
descr = parts.join(', ')
=begin
descr = unless parts.empty?
          parts.pop
        else
          nil
        end
## some names have dosage data after the galenic form
if /[\d\s][m]?[glL]\b/.match(descr) && parts.size > 1
  descr = parts.join(', ') << ', ' << descr
end
=end

parts = descr.split(/\s*,(?!\d|[^(]+\))\s*/u)
gal = parts.first[/[^\d]{3,}/].strip

print "name(Column C)                  = '", name, "'\n"
print "base(:name_base, Product name)  = '", base, "'\n"
print "descr(:name_descr, Description) = '", descr, "'\n"
print "galenic_form                    = '", gal, "'\n"

puts

# old algorithm
puts "Old algorithm"
name  = 'Iscador M 0,0001 mg, Injektionslösung, anthroposophisches Arzneimittel'
#name  = 'Helixor A 0,01 mg Injektionslösung, anthrop. Arzneimittel'
parts = name.split(/\s*,(?!\d|[^(]+\))\s*/u)
descr = parts.pop
if /[\d\s][m]?[glL]\b/.match(descr) && parts.size > 1
  descr = parts.pop << ', ' << descr
end
base = parts.join(', ')
base, descr = descr, nil if base.empty?

parts = descr.split(/\s*,(?!\d|[^(]+\))\s*/u)
gal = parts.first[/[^\d]{3,}/].strip

print "name(Column C)                  = '", name, "'\n"
print "base(:name_base, Product name)  = '", base, "'\n"
print "descr(:name_descr, Description) = '", descr, "'\n"
print "galenic_form                    = '", gal, "'\n"

Result

New algorithm
name(Column C)                  = 'Iscador M 0,0001 mg, Injektionslösung, anthroposophisches Arzneimittel'
base(:name_base, Product name)  = 'Iscador M 0,0001 mg'
descr(:name_descr, Description) = 'Injektionslösung, anthroposophisches Arzneimittel'
galenic_form                    = 'Injektionslösung'

Old algorithm
name(Column C)                  = 'Iscador M 0,0001 mg, Injektionslösung, anthroposophisches Arzneimittel'
base(:name_base, Product name)  = 'Iscador M 0,0001 mg, Injektionslösung'
descr(:name_descr, Description) = 'anthroposophisches Arzneimittel'
galenic_form                    = 'anthroposophisches Arzneimittel'

Note

• This is the correct result

Fix update_compositions method

Experiment (src/plugin/swissmedic.rb#update_compositions)

    def update_sequence(registration, row, opts={:create_only => false})
      seqnr = "%02i" % cell(row, column(:seqnr)).to_i
      ptr = if(sequence = registration.sequence(seqnr))
              return sequence if opts[:create_only]
              sequence.pointer
            else
              (registration.pointer + [:sequence, seqnr]).creator
            end
      ## some names use commas for dosage
      parts = cell(row, column(:name_base)).split(/\s*,(?!\d|[^(]+\))\s*/u)
      base = parts.shift
      descr = parts.join(', ')
 =begin
      descr = unless parts.empty?
                parts.pop
              else
                nil
              end
      ## some names have dosage data after the galenic form
      if /[\d\s][m]?[glL]\b/.match(descr) && parts.size > 1
        descr = parts.join(', ') << ', ' << descr
      end
 =end
      if ctext = cell(row, column(:composition))
        ctext = ctext.gsub(/\r\n?/u, "\n")
      end
 ...

Run

• put the Packungen-latest.xls in data/xls directory
• fix_packages from bin/admin

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> ODDB::SwissmedicPlugin.new(self).fix_packages({:fix_galenic_form => true})

Result

NoMethodError when fixing {:seqnr=>"01",
 :indication_sequence=>nil,
 :ikscat=>"B",
 :unit=>"Tablette(n)",
 :sequence_date=>Thu, 16 Nov 2006,
 :import_date=>Wed, 08 Jun 2011,
 :production_science=>"Synthetika human",
 :composition=>
  "ondansetronum 4 mg ut ondansetroni hydrochloridum dihydricum, vanillinum, color.: E 104, conserv.: E 200, excipiens pro compresso obducto.",
 :name_base=>"Ondansetron-Teva, 4mg, Filmtabletten",
 :substances=>"ondansetronum",
 :atc_class=>"A04AA01",
 :iksnr=>"57202",
 :expiry_date=>Tue, 15 Nov 2011,
 :index_therapeuticus=>"01.09.0.",
 :size=>"10",
 :ikscd=>"004",
 :registration_date=>Thu, 16 Nov 2006,
 :indication_registration=>"Antiemetikum",
 :company=>"Teva Pharma AG"}
undefined method `strip' for nil:NilClass
/home/masa/ywesee/oddb.org/src/plugin/swissmedic.rb:527:in `_update_galenic_form'

Note

• There is the following case

 Ondansetron-Teva, 4mg, Filmtabletten

Test

• Attach:check_algorithm.rb.20110806.txt

Result

name(Column C)                  = 'Iscador M 0,0001 mg, Injektionslösung, anthroposophisches Arzneimittel'
base(:name_base, Product name)  = 'Iscador M 0,0001 mg'
descr(:name_descr, Description) = 'Injektionslösung, anthroposophisches Arzneimittel'
galenic_form                    = 'Injektionslösung'

name(Column C)                  = 'Helixor A 0,01 mg Injektionslösung, anthrop. Arzneimittel'
base(:name_base, Product name)  = 'Helixor A 0,01 mg Injektionslösung'
descr(:name_descr, Description) = 'anthrop. Arzneimittel'
galenic_form                    = 'anthrop. Arzneimittel'

name(Column C)                  = 'Ondansetron-Teva, 4mg, Filmtabletten'
base(:name_base, Product name)  = 'Ondansetron-Teva, 4mg'
descr(:name_descr, Description) = 'Filmtabletten'
galenic_form                    = 'Filmtabletten'

Experiment (src/plugin/swissmedic.rb#update_sequence)

    def update_sequence(registration, row, opts={:create_only => false})
      seqnr = "%02i" % cell(row, column(:seqnr)).to_i
      ptr = if(sequence = registration.sequence(seqnr))
              return sequence if opts[:create_only]
              sequence.pointer
            else
              (registration.pointer + [:sequence, seqnr]).creator
            end
      ## some names use commas for dosage
      parts = cell(row, column(:name_base)).split(/\s*,(?!\d|[^(]+\))\s*/u)
      base = parts.shift
      ## some names have dosage data before the galenic form
      # ex. 'Ondansetron-Teva, 4mg, Filmtabletten'
      if /[\d\s][m]?[glL]\b/.match(parts.first)
        base << ', ' << parts.shift unless parts.empty?
      end
      descr = parts.join(', ') unless parts.empty?

 =begin
      descr = unless parts.empty?
                parts.pop
              else
                nil
              end
      ## some names have dosage data after the galenic form
      if /[\d\s][m]?[glL]\b/.match(descr) && parts.size > 1
        descr = parts.join(', ') << ', ' << descr
      end
 =end
      if ctext = cell(row, column(:composition))
        ctext = ctext.gsub(/\r\n?/u, "\n")
      end
 ...

src/plugin/swissmedic.rb#update_galenic_form

    def _update_galenic_form(comp, lang, name)
      # remove counts and doses from the name - this is assuming name looks
      # (in the worst case) something like this: "10 Filmtabletten"
      # or: "Infusionsemulsion, 1875ml"
      parts = name.split(/\s*,(?!\d|[^(]+\))\s*/u)
      #name = parts.first[/[^\d]{3,}/].strip
      unless name = parts.first[/[^\d]{3,}/]
       name = parts.last[/[^\d]{3,}/]
      end
      name.strip! if name

Run

• put the Packungen-latest.xls in data/xls directory
• fix_packages from bin/admin

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> ODDB::SwissmedicPlugin.new(self).fix_packages({:fix_galenic_form => true})

Result

• Half success

Note

• Some of galenic forms are updated correctly but some are not

Commit

• Updated update_sequence and update_galenic_form methods in swissmedic plugin in order to update galenic form correctly

Experiment

ch.oddb> ODDB::Package.find_by_pharmacode('3767452').galenic_forms
-> Heilmittel auf Grundlage anthroposophischer Erkenntnis
ch.oddb> update(ODDB::Package.find_by_pharmacode('3767452').sequence.compositions[0].pointer, {:galenic_form => 'Tabletten'}, :swissmedic)
-> Tabletten: Viscum Album (Pini) Recens 50 mg/ml

Result

Note

• Also after the reboot, it remains

But!

Experiment

ch.oddb> update(ODDB::Package.find_by_pharmacode('3767452').sequence.compositions[0].pointer, {:galenic_form => 'anthrop. Arzneimittel'}, :swissmedic)
-> Heilmittel auf Grundlage anthroposophischer Erkenntnis: Viscum Album (Pini) Recens 50 mg/ml

Result

Consideration

• Namely, 'anthrop. Arzneimittel' is somehow linked to 'Heilmittel auf Grundlage anthroposophischer Erkenntnis'
• Probably, galenic form data set is saved in the cache (database)

Check all the galenic group and galenic form

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> open("/home/masa/work/test.dat","w"){|f| galenic_groups.values.each_with_index{|gg,i| f.print "\n\n", i, ", ", gg, "(",gg.oid,")\n"; gg.each_galenic_form{|g| f.print "\t", g, "(", g.oid, "), "}}}

Result

• Attach:galenic_group_form.20110608.txt

Experiment

masa@masa ~/ywesee/oddb.org $ bin/admin
ch.oddb> galenic_group(1).delete_galenic_form(278)
-> Heilmittel auf Grundlage anthroposophischer Erkenntnis

ch.oddb> ODDB::SwissmedicPlugin.new(self).fix_packages({:fix_galenic_form => true})

Result

• Looks good

Testcases

1. src/plugin/analysis.rb (coverage: 100%)
2. src/plugin/swissreg.rb (coverage: 100%)
3. src/state/substances/substance.rb (coverage: 81.82%)
4. src/state/admin/companyuser.rb (coverage: 63.27%)
5. src/view/user/plugin.rb (coverage: 38.38%)
6. src/view/admin/orphaned_fachinfo_assign.rb (coverage: 81.44%)

Commit

• Updated testcases, plugin/analysis.rb, swissreg.rb (ch.oddb)