view · edit · sidebar · attach · print · history

20110127-check-migrate-ramaze_oddb

<< | Index | >>

  1. Continue checking migrate script suspend
  2. Focus on the migration of company data suspend
  3. Test run migrate for ramaze
  4. Draw class diagram
Goal
  • Understand migrate / 80%
Milestones
  • Read migrate script
  • Try migrate_company
  • BraSt
  • Test run migrate for ramaze anyway
Summary
Commits
ToDo Tomorrow
  • Continue checking migrate script
  • Debug TypeError first
  • Focus on company object next
Keep in Mind
  1. Check drug data structure (class diagram) in ch.oddb.org, de.oddb.org, and ramaze oddb
  2. Check rdbi instead of dbi for ODBA
  3. Encoding woes (from Davatz-san)
  4. Feedback: This option indicates that the regular expression is parsed as 'UTF8' (from Davatz-san)
  5. pg on Ubuntu - see http://dev.ywesee.com/wiki.php/Gem/Pg (from Davatz-san)
  6. On Ice
  7. emerge --sync

Continue checking migrate script

Confirm the last error 

masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate
masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate 
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_pkey" for table "object"
NOTICE:  CREATE TABLE / UNIQUE will create implicit index "object_name_key" for table "object"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_connection_pkey" for table "object_connection"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "collection_pkey" for table "collection"
TypeError: can't convert String into Integer when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `[]'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `block in current_ids'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:299:in `collect'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:299:in `current_ids'"]
[...]

TypeError: can't convert String into Integer when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `[]'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `block in current_ids'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:299:in `collect'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:299:in `current_ids'"]
[...]

TypeError: can't convert String into Integer when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `[]'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `block in current_ids'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:299:in `collect'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:299:in `current_ids'"]
[...]

...

Experiment

/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb 

    def current_ids(rows, id_name)
print "id_name="
p id_name
      rows.collect { |row| 
        [
          row[id_name], 
          @resolve_search_term.keys.collect { |key|
            [key.to_s, row[key]] }.sort,
        ]
      }
    end

Result 

masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate 
...

id_name="origin_id"
id_name="origin_id"
id_name="origin_id"
id_name="origin_id"
id_name="origin_id"
id_name="origin_id"
id_name="origin_id"
TypeError: can't convert String into Integer when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:303:in `[]'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:303:in `block in current_ids'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `collect'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `current_ids'"]
[...]

...

Note

  • Where is 'origin_id' coming from?

Experiment

/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb 

    def current_ids(rows, id_name)
print "id_name="
p id_name
print caller(0).pretty_inspect
puts
      rows.collect { |row| 
        [
          row[id_name], 
          @resolve_search_term.keys.collect { |key|
            [key.to_s, row[key]] }.sort,
        ]
      }
    end

Result 

masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate 
...
id_name="origin_id"
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:301:in `current_ids'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:314:in `current_origin_ids'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:198:in `update_target'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:549:in `each'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:549:in `update_indices'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:475:in `store'",
 "/usr/lib64/ruby/site_ruby/1.9.1/odba/persistable.rb:267:in `odba_isolated_store'",
 "/home/masa/ywesee/ch.oddb.org/lib/oddb/persistence/odba/model.rb:29:in `save'",
 "/home/masa/ywesee/ch.oddb.org/lib/oddb/model.rb:52:in `block in belongs_to'",
 "bin/migrate:127:in `block in migrate_atc'",
 "/usr/lib64/ruby/1.9.1/drb/drb.rb:1558:in `call'",
 "/usr/lib64/ruby/1.9.1/drb/drb.rb:1558:in `perform_without_block'",
 "/usr/lib64/ruby/1.9.1/drb/drb.rb:1518:in `perform'",
 "/usr/lib64/ruby/1.9.1/drb/drb.rb:1592:in `block (2 levels) in main_loop'",
 "/usr/lib64/ruby/1.9.1/drb/drb.rb:1588:in `loop'",
 "/usr/lib64/ruby/1.9.1/drb/drb.rb:1588:in `block in main_loop'"]
...

Next

  • Trace this flow

Consideration

  • /usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb 
    def current_ids(rows, id_name)
      rows.collect { |row|
        [
          row[id_name],
  • There are two possibilities,
    • 'row' instance may be wrong
    • 'id_name' may be wrong

Experiment

/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb#current_ids 

    def current_ids(rows, id_name)
print "id_name="
p id_name
print caller(0).pretty_inspect
puts
unless rows[0]
print "rows[0].class="
p rows[0].class
print "rows[0]="
p rows[0]
if rows[0]!=nil
  exit
end
end
      rows.collect { |row|
        [
          row[id_name],
          @resolve_search_term.keys.collect { |key|
            [key.to_s, row[key]] }.sort,
        ]
      }
    end

Result 

masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate
...

rows[0].class=NilClass
rows[0]=nil
"End"

Notes

  • 'row' is always 'nil'
  • I have also confirmed 'rows.length' is '0', 'rows' is NilClass

Experiment (skip this process)

/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb#current_ids 

    def current_ids(rows, id_name)
if rows.length > 0
      rows.collect { |row|
        [
          row[id_name],
          @resolve_search_term.keys.collect { |key|
            [key.to_s, row[key]] }.sort,
        ]
      }
end
    end

Result 

masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate 
...
NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:549:in `each'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:549:in `each'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:549:in `each'"]
[...]

...

Note

  • NoMethodError

Check /usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb#update_target 

        def update_target(target) # :nodoc:
      target_id = target.odba_id
      current = current_origin_ids(target_id)
      old_terms = current.collect { |row|      # <= HERE
        [row[0], row[1]]
      }

Note

  • It seems that the update_target method does not include Exception process
  • This means the author expects 'current' object must be something other than nil there

Consideration

  • It looks something strange
    • I am not sure which is wrong, data or the process
    • Or mismatching between the source data structure (oddb.org) and the data structure of importing process (migrate script, ch.oddb.org)

suspend

Focus on the migration of company data

Experiment

ch.oddb.org/bin/migrate 

total = server.company_count.to_f
count = 0
=begin
server.remote_each_atc_class do |atc|
  unless atc.en.to_s.empty?
    migrate_atc ODDB::Remote::Drugs::Atc.new url, atc
  end
end
=end
server.remote_each_company do |remote|
  company = ODDB::Remote::Business::Company.new(url, remote)
  unless company.products.empty?
    migrate_company company
  end
  count += 1
  print (" " * 8) \
    << ("\b" * 8) \
    << sprintf('%7.3f%%', count*100/total) \
    << ("\b" * 8)
  $stdout.flush
end

Refresh database 

sudo -u postgres dropdb oddb
sudo -u postgres createdb -E UTF8 -T template0 oddb

Run oddb.org/bin/oddbd

Run ch.oddb.org/bin/migrate

Result 

masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate 
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_pkey" for table "object"
NOTICE:  CREATE TABLE / UNIQUE will create implicit index "object_name_key" for table "object"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_connection_pkey" for table "object_connection"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "collection_pkey" for table "collection"
(druby://localhost:10000) /usr/lib64/ruby/1.8/delegate.rb:158:in `method_missing': undefined method `remote_each_company' for #<ODDB::App:0x7f814c4c19c0> (NoMethodError)
        from bin/migrate:454:in `<main>'

Note

  • becase there is no 'remote_each_company' method definition on the oddb.org server

Experiment

oddb.org/src/util/oddbapp.rb 

    def remote_each_company(&block)
        @companies.extend(DRb::DRbUndumped).each_value(&block)
    end

Result 

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_galenicform_code' with a ODDB::Drugs::GalenicForm
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_substance_code' with a ODDB::Drugs::Substance
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_galenicform_code' with a ODDB::Drugs::GalenicForm
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_substance_code' with a ODDB::Drugs::Substance
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_substance_code' with a ODDB::Drugs::Substance
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_substance_code' with a ODDB::Drugs::Substance
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

NoMethodError: undefined method `collect' for nil:NilClass when updating index 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
["/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:201:in `update_target'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/index.rb:161:in `update'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/stub.rb:112:in `method_missing'", "/usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:550:in `block in update_indices'"]
[...]

Notes

  • Errors happen in several parts
    • 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
    • 'oddb_drugs_sequence_code' with a ODDB::Drugs::Sequence
    • 'oddb_drugs_package_code' with a ODDB::Drugs::Package
    • 'oddb_drugs_substance_code' with a ODDB::Drugs::Substance
    • 'oddb_drugs_galenicform_code' with a ODDB::Drugs::GalenicForm
    • 'oddb_drugs_atc_level_and_code' with a ODDB::Drugs::Atc
  • These error are same to 'migrate_atc'

Next

  • I should consider what to do

BraSt

  • This 'migrate' script is for the ramaze version of ch.oddb.org, the database table name is 'ch_oddb_org'
  • The source data should be 'oddb.org', which is the database table name of the project 'oddb.org'
  • I have to understand what kind of data structure is necessary for the ramaze version of ch.oddb.org
    • Namely, the data structure of 'migrate ' script must be the same structure to the ramaze oddb, 'ch.oddb.org'
    • It may be better first to check the data structure between ramaze version (ch.oddb.org) and the old oddb version (oddb.org)
    • I think this is more straightforward and steady strategy rather than debug the error of migrate script
    • since it seems that the migrate script is not completely implemented

Question

  • How (why) did this run when this was committed?

Test run migrate for ramaze

Experiment (Try to run the ramaze version after the 'migrate' script runs)

Confirm Ramaze version of ch.oddb.org runs

  • Ok

Clear the database 

$ sudo -u postgres dropdb ch_oddb_org; sudo -u postgres createdb -E UTF8 -T template0 ch_oddb_org

Run oddb.org/bin/oddbd

Set ch.oddb.org/lib/oddb/config.rb 

    #'db_name'             => 'oddb',
    'db_name'             => 'ch_oddb_org',
    #'db_user'             => 'oddb',
    'db_user'             => 'postgres',
    #'db_auth'             => 'oddb',
    'db_auth'             => '',

Run ch.oddb.org/bin/migrate 

masa@masa ~/ywesee/ch.oddb.org $ ruby1.9 bin/migrate .
...
(Many NoMethod Error)
...
(druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:488:in `store': can't add a new key into hash during iteration (RuntimeError)
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:488:in `block in store_cache_entry'
        from (druby://localhost:54642) <internal:prelude>:10:in `synchronize'
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:482:in `store_cache_entry'
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/cache.rb:474:in `store'
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/persistable.rb:267:in `odba_isolated_store'
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/persistable.rb:384:in `block in odba_store_unsaved'
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/persistable.rb:381:in `each'
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/persistable.rb:381:in `odba_store_unsaved'
        from (druby://localhost:54642) /usr/lib64/ruby/site_ruby/1.9.1/odba/persistable.rb:369:in `odba_store'
        from (druby://localhost:54642) /home/masa/ywesee/ch.oddb.org/lib/oddb/persistence/odba/model.rb:32:in `block in save'
        from (druby://localhost:54642) /home/masa/ywesee/ch.oddb.org/lib/oddb/persistence/odba/model.rb:30:in `each'
        from (druby://localhost:54642) /home/masa/ywesee/ch.oddb.org/lib/oddb/persistence/odba/model.rb:30:in `save'
        from (druby://localhost:54642) /home/masa/ywesee/ch.oddb.org/lib/oddb/model.rb:52:in `block in belongs_to'
        from (druby://localhost:54642) bin/migrate:357:in `migrate_product'
        from (druby://localhost:54642) bin/migrate:144:in `block in migrate_company'
        from (druby://localhost:54642) bin/migrate:143:in `each'
        from (druby://localhost:54642) bin/migrate:143:in `migrate_company'
        from (druby://localhost:54642) bin/migrate:455:in `block in <main>'
        from (druby://localhost:54642) /usr/lib64/ruby/1.9.1/drb/drb.rb:1558:in `call'
        from (druby://localhost:54642) /usr/lib64/ruby/1.9.1/drb/drb.rb:1558:in `perform_without_block'
        from (druby://localhost:54642) /usr/lib64/ruby/1.9.1/drb/drb.rb:1518:in `perform'
        from (druby://localhost:54642) /usr/lib64/ruby/1.9.1/drb/drb.rb:1592:in `block (2 levels) in main_loop'
        from (druby://localhost:54642) /usr/lib64/ruby/1.9.1/drb/drb.rb:1588:in `loop'
        from (druby://localhost:54642) /usr/lib64/ruby/1.9.1/drb/drb.rb:1588:in `block in main_loop'
        from (druby://localhost:10000) /usr/lib64/ruby/1.8/drb/invokemethod.rb:10:in `block_yield'
        from (druby://localhost:10000) /usr/lib64/ruby/1.8/drb/invokemethod.rb:17:in `perform_with_block'
        from (druby://localhost:10000) /home/masa/ywesee/oddb.org/src/util/oddbapp.rb:594:in `each_value'
        from (druby://localhost:10000) /home/masa/ywesee/oddb.org/src/util/oddbapp.rb:594:in `remote_each_company'
        from bin/migrate:452:in `<main>'

Note

  • Stop by error

Confirm the ramaze runs

  • start.rb can run and some data is imported (1487 Arzneimittel A-Z)
  • default (ramaze.ch.oddb.org online): 18360 Arzneimittel A-Z
  • ch.oddb.org online (oddb.org): 18825 Arzneimittel A-Z
  • The number of errors: total 19814 (migrate_atc(about 10000), migrate_company(about 10000))

Next

  • Draw class structure (diagram) (first, only Drugs) to understand the data structure in old ch.oddb.org and new ch.oddb.org (ramaze version), also de.oddb.org

Draw class diagram

Install

  • Dia
  • Dia2code

Emerge dia failed 

!!! /var/tmp/portage/app-office/dia-0.97.1/work/dia-0.97.1/config.log
 * ERROR: app-office/dia-0.97.1 failed:
 *   econf failed
 * 
 * Call stack:
 *     ebuild.sh, line   56:  Called src_configure
 *   environment, line 5780:  Called gnome2_src_configure
 *   environment, line 3306:  Called econf '--without-cairo' '--with-python' '--disable-db2html' '--disable-gnome' '--disable-libemf' '--without-swig' '--without-hardbooks' '--disable-static' '--docdir=/usr/share/doc/dia-0.97.1' '--disable-gtk-doc'
 *     ebuild.sh, line  552:  Called die
 * The specific snippet of code:
 *              die "econf failed"
 * 
 * If you need support, post the output of 'emerge --info =app-office/dia-0.97.1',
 * the complete build log and the output of 'emerge -pqv =app-office/dia-0.97.1'.
 * The complete build log is located at '/var/tmp/portage/app-office/dia-0.97.1/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/app-office/dia-0.97.1/temp/environment'.
 * S: '/var/tmp/portage/app-office/dia-0.97.1/work/dia-0.97.1'

>>> Failed to emerge app-office/dia-0.97.1, Log file:

>>>  '/var/tmp/portage/app-office/dia-0.97.1/temp/build.log'

 * Messages for package app-office/dia-0.97.1:

 * ERROR: app-office/dia-0.97.1 failed:
 *   econf failed
 * 
 * Call stack:
 *     ebuild.sh, line   56:  Called src_configure
 *   environment, line 5780:  Called gnome2_src_configure
 *   environment, line 3306:  Called econf '--without-cairo' '--with-python' '--disable-db2html' '--disable-gnome' '--disable-libemf' '--without-swig' '--without-hardbooks' '--disable-static' '--docdir=/usr/share/doc/dia-0.97.1' '--disable-gtk-doc'
 *     ebuild.sh, line  552:  Called die
 * The specific snippet of code:
 *              die "econf failed"
 * 
 * If you need support, post the output of 'emerge --info =app-office/dia-0.97.1',
 * the complete build log and the output of 'emerge -pqv =app-office/dia-0.97.1'.
 * The complete build log is located at '/var/tmp/portage/app-office/dia-0.97.1/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/app-office/dia-0.97.1/temp/environment'.
 * S: '/var/tmp/portage/app-office/dia-0.97.1/work/dia-0.97.1'
view · edit · sidebar · attach · print · history
Page last modified on January 27, 2011, at 04:55 PM