20110908-debug-importer-testcases-migel

<< | Index | >>

1. Debug importer migel
2. Testcases migel

Goal/Estimate/Evaluation:

• Debug importer migel / 80% / 80%

Milestones

1. debug importer
2. testcases migel

Summary

• http://masaywesee.wordpress.com/2011/09/08/debug-importer-spec-migel/

ToDo

• save test
• Migel.logger test
• require 'migel/util/m10l_document' problem in multilingual.rb
• require 'oddb/util/m10l_document' problem in model_super.rb
• 'could not sort' warning -> this is caused by probably to_s method
• RangeError 0x3fd07d5c4ee0 is recycled object

Debug importer migel

Segmentation fault (while init_products (creating a fulltext index table for products)

migel> init_products
-> connection closed

/usr/lib64/ruby/site_ruby/1.8/odba/cache_entry.rb:60: [BUG] Segmentation fault
ruby 1.8.6 (2009-06-08) [x86_64-linux]

Abgebrochen

Note

• saving time is increasing with the number of products
• This is related to 'odba_store' method

Experiment

• lib/migel/util/server.rb

      def init_products(estimate = false)
        pharmacode_list = ODBA.cache.index_keys('migel_model_product_pharmacode')
        total = pharmacode_list.length
        start_time = Time.now
        #ODBA.cache.index_keys('migel_model_product_pharmacode').each do |pharmacode|
        pharmacode_list.each_with_index do |pharmacode, i|
          #add_product Migel::Model::Product.find_by_pharmacode(pharmacode)
          products.store(pharmacode, Migel::Model::Product.find_by_pharmacode(pharmacode))
          puts estimate_time(start_time, total, i+1) if estimate
        end
        products.odba_store
      end

Point

• 'odba_store' method is called only once at the end of the method

Experiment

• bin/migeld
• bin/admin

migel> clear_products
->
migel> init_products true
-> Hash

Log

..
30741 / 30742   Estimate total: 0.60 [m] It will be done in: 0.00 [m]
30742 / 30742   Estimate total: 0.60 [m] It will be done in: -0.00 [m]

Note

• it takes only a minute
• The most of the time is taken by 'odba_store', which means saving the data in the database

Next (build fulltext index tables up)

• bin/admin

migel> rebuild_fulltext_index_tables
-> Array

Run

• bin/oddbd
• bin/currencyd
• (bin/yusd)
• bin/migeld

Result

Note

• It is strange that all the migelids have 'Limitation text', 'L' is shown
• Because I resolved this problem before.
• This is related to Multilingual class (limitation_text instance)
• It is possible to show appropriately if DRbUndumped module is included in Multilingual class
• but this makes an error for the importer. See http://dev.ywesee.com/wiki.php/Masa/20110826-searchByCompanyname-migel-drb-client-oddbOrg#DRbDump

Experiment

• lib/migel/model/migel.rb, group.rb, subgroup.rb

    def limitation_text(update = false)
      if update
        @limitation_text ||= Migel::Util::Multilingual.new
      else
        if @limitation_text
          ODBA::DRbWrapper.new(@limitation_text)
        end
      end
    end
    def en
      name.de
    end
    def de
      name.de
    end
    def fr
      name.fr
    end

Note

• the method definitions of limitation_text, de, en, fr, are necessary for each class, Group, Subgroup, Migelid
• It seems impossible to move them in Model_Super class

Result

Note

• Only once the segumentation fault happened while importing de csv file.
• 'save' method probably related to this segumentation fault -> less time of saving may avoid this error

Testcases migel

migel server side

1. util
   1. swissindex.rb (coverage: 100%)
   2. server.rb (coverage: 96.43%)
   3. multilingual.rb pass because it is similar to the multilingual.rb of de.oddb.org
   4. importer.rb (coverage: 100%)

oddb side (client side)

1. model
   1. migel/group.rb
   2. migel/items.rb