view · edit · sidebar · attach · print · history

20110907-update-importer-testcase-spreadsheet-testcases-migel

<< Masa.20110908-debug-importer-testcases-migel | 2011 | Masa.20110906-update-importer-testcases-migel >>


  1. emerge --sync error
  2. Update importer migel (products)
  3. Update a testcase of spreadsheet
  4. Testcases migel

Goal/Estimate/Evaluation:

  • Update importer migel / 100% / 90%
Milestones
  1. update importer
  2. testcases spreadsheet
  3. testcases migel
Summary
Commits
ToDo
  • Testcases oddb.org, migel
    • at the same time, refactoring Group, Subgroup, Migelid because some code in the classes are totally same
  • refactor rebuild_fulltext_index_tables
  • delete_migelids in lib/migel/util/server.rb
  • 'could not sort' warning -> this is caused by probably to_s method
  • RangeError 0x3fd07d5c4ee0 is recycled object

emerge --sync error

Reference

My make.conf

Update importer migel (products)

Log

...
569 / 571       Estimate total: 11.60 [h] It will be done in: 2.44 [m]
570 / 571       Estimate total: 11.58 [h] It will be done in: 1.22 [m]
571 / 571       Estimate total: 11.58 [h] It will be done in: -0.00 [m]

Run

  • bin/migeld
  • bin/admin
migel> Migel::Importer.new.import_all_products_from_csv('migel_product_de.csv', 'de', true)
migel> Migel::Importer.new.import_all_products_from_csv('migel_product_fr.csv', 'fr', true)

Log

...
11606 / 30742   Estimate total: 1.37 [h] It will be done in: 51.22 [m]
11607 / 30742   Estimate total: 1.37 [h] It will be done in: 51.31 [m]
11608 / 30742   Estimate total: 1.38 [h] It will be done in: 51.38 [m]
...

Result

  • Error
21986 / 30742   Estimate total: 4.03 [h] It will be done in: 1.15 [h]
21987 / 30742   Estimate total: 4.03 [h] It will be done in: 1.15 [h]
/usr/lib64/ruby/site_ruby/1.8/odba/cache_entry.rb:42: [BUG] Segmentation fault
ruby 1.8.6 (2009-06-08) [x86_64-linux]

Abgebrochen
...
30741 / 30742   Estimate total: 8.90 [m] It will be done in: 0.00 [m]
30742 / 30742   Estimate total: 8.90 [m] It will be done in: -0.00 [m]

Note

  • But there is only one error.

Setup index tables

  • bin/admin
migel> init_migelids
migel> init_products
migel> rebuild_fulltext_index_tables

Attach:import_produt.20110907.jpg Δ

Update a testcase of spreadsheet

Email

Reference

Task

  • Pass the following testase failure by using flexmock, not necessary to update the spreadsheet code
masa@masa ~/work/dblock-spreadsheet-164dcfb $ ruby test/integration.rb 
Loaded suite test/integration
Started
E
Finished in 0.008582 seconds.

  1) Error:
test_write_frozen_string(Spreadsheet::TestIntegration):
Iconv::InvalidCharacter: "g"
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/encodings.rb:32:in `iconv'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/encodings.rb:32:in `internal'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/biff8.rb:43:in `_unicode_string'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/workbook.rb:508:in `_write_sst'
    /usr/lib64/ruby/1.8/fileutils.rb:243:in `each_with_index'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/workbook.rb:502:in `each'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/workbook.rb:502:in `each_with_index'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/workbook.rb:502:in `_write_sst'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/workbook.rb:489:in `write_sst'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/workbook.rb:439:in `write_from_scratch'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/excel/writer/workbook.rb:620:in `write_workbook'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/writer.rb:15:in `write'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/writer.rb:14:in `open'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/writer.rb:14:in `write'
    /home/masa/work/dblock-spreadsheet-164dcfb/lib/spreadsheet/workbook.rb:106:in `write'
    test/integration.rb:1309:in `test_write_frozen_string'

1 tests, 0 assertions, 0 failures, 1 errors

Note

  • Ruby 1.9 passses this testcase
  • Ruby 1.8 fails

Experiment

  • test.rb
require 'iconv'
string = 'Frozen String'
client = 'UTF-16LE'
iconv = Iconv.new('UTF-16LE', client)
p iconv.iconv(string)

Result

masa@masa ~/work/dblock-spreadsheet-164dcfb $ ruby test.rb 
test.rb:6:in `iconv': "g" (Iconv::InvalidCharacter)
        from test.rb:6

Note

  • It is the same error

Experiment

 require 'iconv'
 string = 'Frozen String.'
 iconv = Iconv.new('UTF-16LE', 'UTF-16LE')
 p iconv.iconv(string)

Run

masa@masa ~/work/dblock-spreadsheet-164dcfb $ ruby test.rb 
"Frozen String."

Note

  • It works but I do not know why

Experiment

  • test/integration.rb
    def test_write_frozen_string
      Spreadsheet.client_encoding = 'UTF-16LE'
      book = Spreadsheet::Workbook.new
      path = File.join @var, 'test_write_workbook.xls'
      sheet1 = book.create_worksheet
      str1 = "Frozen String.".freeze
      sheet1[0,0] = str1
      sheet1.row(0).push str1
      book.write path
    end

Run

masa@masa ~/work/dblock-spreadsheet-164dcfb $ ruby -I lib test/integration.rb 
Loaded suite test/integration
Started
E
Finished in 0.008138 seconds.

  1) Error:
test_write_frozen_string(Spreadsheet::TestIntegration):
TypeError: can't modify frozen string
    ./lib/spreadsheet/excel/writer/biff8.rb:66:in `<<'
    ./lib/spreadsheet/excel/writer/biff8.rb:66:in `_unicode_string'
    ./lib/spreadsheet/excel/writer/workbook.rb:508:in `_write_sst'
    /usr/lib64/ruby/1.8/fileutils.rb:243:in `each_with_index'
    ./lib/spreadsheet/excel/writer/workbook.rb:502:in `each'
    ./lib/spreadsheet/excel/writer/workbook.rb:502:in `each_with_index'
    ./lib/spreadsheet/excel/writer/workbook.rb:502:in `_write_sst'
    ./lib/spreadsheet/excel/writer/workbook.rb:489:in `write_sst'
    ./lib/spreadsheet/excel/writer/workbook.rb:439:in `write_from_scratch'
    ./lib/spreadsheet/excel/writer/workbook.rb:620:in `write_workbook'
    ./lib/spreadsheet/writer.rb:15:in `write'
    ./lib/spreadsheet/writer.rb:14:in `open'
    ./lib/spreadsheet/writer.rb:14:in `write'
    ./lib/spreadsheet/workbook.rb:106:in `write'
    test/integration.rb:1309:in `test_write_frozen_string'

1 tests, 0 assertions, 0 failures, 1 errors

Note

  • There is no assertion in the testcase
  • The testcase should be as follows
    def test_write_frozen_string
      Spreadsheet.client_encoding = 'UTF-16LE'
      book = Spreadsheet::Workbook.new
      path = File.join @var, 'test_write_workbook.xls'
      sheet1 = book.create_worksheet
      str1 = "Frozen String.".freeze
      sheet1[0,0] = str1
      sheet1.row(0).push str1
      assert_nothing_raised do
        book.write path
      end
    end

Run

masa@masa ~/work/dblock-spreadsheet-164dcfb/test $ ruby18 -rubygems -I ../lib integration.rb 
Loaded suite integration
Started
F
Finished in 0.010687 seconds.

  1) Failure:
test_write_frozen_string(Spreadsheet::TestIntegration) [integration.rb:1309]:
Exception raised:
Class: <TypeError>
Message: <"can't modify frozen string">
---Backtrace---
../lib/spreadsheet/excel/writer/biff8.rb:66:in `<<'
../lib/spreadsheet/excel/writer/biff8.rb:66:in `_unicode_string'
../lib/spreadsheet/excel/writer/workbook.rb:508:in `_write_sst'
/usr/lib64/ruby/1.8/fileutils.rb:243:in `each_with_index'
../lib/spreadsheet/excel/writer/workbook.rb:502:in `each'
../lib/spreadsheet/excel/writer/workbook.rb:502:in `each_with_index'
../lib/spreadsheet/excel/writer/workbook.rb:502:in `_write_sst'
../lib/spreadsheet/excel/writer/workbook.rb:489:in `write_sst'
../lib/spreadsheet/excel/writer/workbook.rb:439:in `write_from_scratch'
../lib/spreadsheet/excel/writer/workbook.rb:620:in `write_workbook'
../lib/spreadsheet/writer.rb:15:in `write'
../lib/spreadsheet/writer.rb:14:in `open'
../lib/spreadsheet/writer.rb:14:in `write'
../lib/spreadsheet/workbook.rb:106:in `write'
integration.rb:1310:in `test_write_frozen_string'
integration.rb:1309:in `test_write_frozen_string'
---------------

1 tests, 1 assertions, 1 failures, 0 errors

Experiment

  • lib/spreadsheet/encoding.rb module Encodings
    if RUBY_VERSION >= '1.9'
...
    else
      require 'iconv'
      @@iconvs = {}
      def client string, internal='UTF-16LE'
        string = string.dup
        key = [Spreadsheet.client_encoding, internal]
        iconv = @@iconvs[key] ||= Iconv.new(Spreadsheet.client_encoding, internal)
        iconv.iconv string
      end
      def internal string, client=Spreadsheet.client_encoding
        string = string.dup
        key = ['UTF-16LE', client]
        iconv = @@iconvs[key] ||= Iconv.new('UTF-16LE', client)
        iconv.iconv string
      end
      def utf8 string, client=Spreadsheet.client_encoding
        string = string.dup
        key = ['UTF-8', client]
        iconv = @@iconvs[key] ||= Iconv.new('UTF-8', client)
        iconv.iconv string
      end
    end

Run

masa@masa ~/work/dblock-spreadsheet-164dcfb $ ruby -rubygems -I lib test/integration.rb 
Loaded suite test/integration
Started
..........................
Finished in 1.162569 seconds.

26 tests, 807 assertions, 0 failures, 0 errors
masa@masa ~/work/dblock-spreadsheet-164dcfb $ ruby1.9 -rubygems -I lib test/integration.rb 
Loaded suite test/integration
Started
..........................

Finished in 0.83873682 seconds.

26 tests, 807 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

Patch

Commit

Testcases migel

  1. util
    1. swissindex.rb (coverage: 100%)
    2. server.rb (coverage: 88.41%)
    3. multilingual.rb pass because it is similar to the multilingual.rb of de.oddb.org

Reference

view · edit · sidebar · attach · print · history
Page last modified on March 12, 2013, at 02:08 PM