view · edit · sidebar · attach · print · history

20110211-testcases-oddb_org

<< | Index | >>

Make test-cases for ext scripts

Goal/Estimate

test-cases ext scripts / 70%

Milestones

~~Check the current status ext~~ 8:45
fiparse 11 Failures 94 Errors 59%
analysisparse 3 Failures 82%
chapterparse 6 Failures 62%
comarketing 1 Failure 91%
docdata 1 Error 25%
export 2 Failures 1 Error 48%
fipdf Error (stop)
fulltext (no test-case)
meddata Error (stop)
readonly (no test-case)
swissreg 2 Errors 64%

Summary

http://masay.ch/2011/02/11/testcases-oddb_org-2/

Commits
ToDo Tomorrow
Keep in Mind

no more test_minifi.rb
On Ice

Make test-cases for ext scripts

Confirm each test-case in ext directory

 * analysisparse  3  Failures           82%
 * chapterparse   6  Failures           62%
 * comarketing    1  Failure            91%
 * docdata                     1  Error 25%
 * export         2  Failures  1 Error  48% 
 * fiparse        11 Failures 94 Errors 59%        
 * fipdf          Error (stop)
 * fulltext       (no test-case)
 * meddata        Error (stop)
 * readonly       (no test-case)
 * swissreg                    2 Errors 64%

one by one

Update test-case fiparse

Check each test-case file

 * test_fachinfo_doc_parser.rb  89 Errors
 * test_fachinfo_hpricot.rb      2 Failures
 * test_fachinfo_pdf.rb            Error (stop)
 * test_fiparse.rb               1 Error
 * test_indications.rb           1 Error
 * test_minifi.rb                7 Failures 1 Error We do not need this anymore
 * test_patinfo_hpricot.rb       1 Failure

Update test_fachinfo_doc_parser.rb

masa@masa ~/ywesee/oddb.org/ext/fiparse/test $ ruby test_fachinfo_doc_parser.rb
...
 89) Error:
test_registration_owner9(TestFachinfoDocParser9):
NoMethodError: undefined method `create_parser' for Rwv2:Module
    test_fachinfo_doc_parser.rb:1048:in `setup'

89 tests, 0 assertions, 0 failures, 89 errors

Note

Ever error is 'undefined method `create_parser' for Rwv2:Module'
So, it is 'rwv2' problem

test.rb

require 'rwv2'
require 'rwv2/handlers'

class TextHandler < Rwv2::TextHandler
  def run_of_text(text, character_properties)
    puts text
  end
end

parser = Rwv2.create_parser('test.doc')
parser.set_text_handler(TextHandler.new)
parser.parse

Note

This is from http://rwv2.rubyforge.org/

Result

masa@masa ~/work $ ruby test.rb
/usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- rwv2 (LoadError)
        from /usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
        from test.rb:3

Experiment

/usr/lib64/ruby/gems/1.8/gems/rwv2-0.6.0/lib/rwv2/rwv2.rb

#require 'rwv2'

test.rb

require 'rwv2/rwv2'
require 'rwv2/handlers'

class TextHandler < Rwv2::TextHandler
  def run_of_text(text, character_properties)
    puts text
  end
end

parser = Rwv2.create_parser('test.doc')
parser.set_text_handler(TextHandler.new)
parser.parse

Result

masa@masa ~/work $ ruby test.rb
/usr/lib64/ruby/gems/1.8/gems/rwv2-0.6.0/lib/rwv2/rwv2.rb:29: warning: already initialized constant VERSION
test.rb:13: undefined method `create_parser' for Rwv2:Module (NoMethodError)

Note

I got the same error as test_fachinfo_doc_parser.rb

Memo

rwv2 is a wrapper library for wv2
but wv2 is not installed yet in my PC

Install wv2

$ sudo emerge wv2

Compile rwv2 extended Ruby library

masa@masa ~/ywesee/rwv2/ext/rwv2 $ ls
extconf.rb  include  MANIFEST  mkmf.log  rwv2.cpp  rwv2_handlers.cpp  rwv2_parser.cpp  rwv2_properties.cpp
masa@masa ~/ywesee/rwv2/ext/rwv2 $ ruby extconf.rb 
checking for main() in -lwv2... yes
creating Makefile
masa@masa ~/ywesee/rwv2/ext/rwv2 $ make
g++ -I. -I. -I/usr/lib64/ruby/1.8/x86_64-linux -I. -I./include   -fPIC -march=core2 -O2 -pipe -fno-strict-aliasing  -fPIC   -c rwv2.cpp
g++ -I. -I. -I/usr/lib64/ruby/1.8/x86_64-linux -I. -I./include   -fPIC -march=core2 -O2 -pipe -fno-strict-aliasing  -fPIC   -c rwv2_handlers.cpp
g++ -I. -I. -I/usr/lib64/ruby/1.8/x86_64-linux -I. -I./include   -fPIC -march=core2 -O2 -pipe -fno-strict-aliasing  -fPIC   -c rwv2_parser.cpp
g++ -I. -I. -I/usr/lib64/ruby/1.8/x86_64-linux -I. -I./include   -fPIC -march=core2 -O2 -pipe -fno-strict-aliasing  -fPIC   -c rwv2_properties.cpp
x86_64-pc-linux-gnu-gcc -shared -o rwv2.so rwv2.o rwv2_handlers.o rwv2_parser.o rwv2_properties.o -L. -L/usr/lib64 -Wl,-R/usr/lib64 -L./lib -L. -Wl,-O1 -Wl,--sort-common -Wl,--as-needed -rdynamic -Wl,-export-dynamic    -Wl,-R -Wl,/usr/lib64 -L/usr/lib64 -lruby18 -lwv2 -lstdc++  -lrt -ldl -lcrypt -lm   -lc
masa@masa ~/ywesee/rwv2/ext/rwv2 $ ls
extconf.rb  Makefile  mkmf.log  rwv2_handlers.cpp  rwv2.o           rwv2_parser.o        rwv2_properties.o
include     MANIFEST  rwv2.cpp  rwv2_handlers.o    rwv2_parser.cpp  rwv2_properties.cpp  rwv2.so

Note

I got 'rwv2.so' that is the ruby extended library

Install

masa@masa ~/ywesee/rwv2 $ sudo ruby install.rb config
install.rb: entering config phase...
---> lib
---> lib/rwv2
<--- lib/rwv2
<--- lib
---> ext
---> ext/rwv2
/usr/bin/ruby18 /home/masa/ywesee/rwv2/ext/rwv2/extconf.rb 
checking for main() in -lwv2... yes
creating Makefile
---> ext/rwv2/include
<--- ext/rwv2/include
<--- ext/rwv2
<--- ext
install.rb: config done.
masa@masa ~/ywesee/rwv2 $ sudo ruby install.rb setup
install.rb: entering setup phase...
---> lib
---> lib/rwv2
<--- lib/rwv2
<--- lib
---> ext
---> ext/rwv2
make 
x86_64-pc-linux-gnu-gcc -shared -o rwv2.so rwv2.o rwv2_handlers.o rwv2_parser.o rwv2_properties.o -L. -L/usr/lib64 -Wl,-R/usr/lib64 -L./lib -L. -Wl,-O1 -Wl,--sort-common -Wl,--as-needed -rdynamic -Wl,-export-dynamic    -Wl,-R -Wl,/usr/lib64 -L/usr/lib64 -lruby18 -lwv2 -lstdc++  -lrt -ldl -lcrypt -lm   -lc
---> ext/rwv2/include
<--- ext/rwv2/include
<--- ext/rwv2
<--- ext
install.rb: setup done.
masa@masa ~/ywesee/rwv2 $ sudo ruby install.rb install
install.rb: entering install phase...
---> lib
mkdir -p /usr/lib64/ruby/site_ruby/1.8/
---> lib/rwv2
mkdir -p /usr/lib64/ruby/site_ruby/1.8/rwv2
install handlers.rb /usr/lib64/ruby/site_ruby/1.8/rwv2
install rwv2.rb /usr/lib64/ruby/site_ruby/1.8/rwv2
<--- lib/rwv2
<--- lib
---> ext
---> ext/rwv2
mkdir -p /usr/lib64/ruby/site_ruby/1.8/x86_64-linux/.
install rwv2.so /usr/lib64/ruby/site_ruby/1.8/x86_64-linux/.
---> ext/rwv2/include
<--- ext/rwv2/include
<--- ext/rwv2
<--- ext
install.rb: install done.

Experiment

test.rb

require 'rwv2'
require 'rwv2/handlers'

class TextHandler < Rwv2::TextHandler
  def run_of_text(text, character_properties)
    puts text
  end
end

parser = Rwv2.create_parser('test.doc')
parser.set_text_handler(TextHandler.new)
parser.parse

masa@masa ~/work $ ls
test.doc  test.rb

Attach:test.0110211.doc

Result

masa@masa ~/work $ ruby test.rb
hello

Note

Good

Run test_fachinfo_doc_parser.rb

masa@masa ~/ywesee/oddb.org/ext/fiparse/test $ ruby test_fachinfo_doc_parser.rb 
/usr/lib64/ruby/site_ruby/1.8/rwv2/rwv2.rb:28: warning: already initialized constant VERSION
Loaded suite test_fachinfo_doc_parser
Started
......FF.FFFFFFFF...F..FFFFFF.F.FFFF>------------------------------------------------------------------------
                   Disktest (10 &#65533;g<
>------------------------------------------------------------------------
                   Disktest (10 �<
F.F.F.FFF..FF..F..F....FF.FF..FFF..F.FF.FF.F.FFFFFFtest_fachinfo_doc_parser.rb:1048: [BUG] Segmentation fault
ruby 1.8.6 (2009-06-08) [x86_64-linux]

Abgebrochen

Note

Error has gone, but many failures and Abort!!

Trace test-case one by one

Interim report

Attach:test_fachinfo_doc_parser.rb.20110211_1.txt

Result

masa@masa ~/ywesee/oddb.org/ext/fiparse/test $ rcov test_fachinfo_doc_parser.rb -t
/usr/lib64/ruby/site_ruby/1.8/rwv2/rwv2.rb:28: warning: already initialized constant VERSION
Loaded suite /usr/bin/rcov
Started
...........
>------------------------------------------------------------------------
                   Disktest (10 &#65533;g<
>------------------------------------------------------------------------
                   Disktest (10 �<
...........F.
Finished in 1.823073 seconds.

  1) Failure:
test_company5(TestFachinfoDocParser5) [./test_fachinfo_doc_parser.rb:401]:
<"Salmon Pharma"> expected but was
<"Salmon Pharma\n\n\n\nZusammensetzung">.

24 tests, 120 assertions, 1 failures, 0 errors
54.2%   37 file(s)   9596 Lines   7452 LOC

Updated

Attach:test_fachinfo_doc_parser.rb.20110211_2.txt

Result

masa@masa ~/ywesee/oddb.org/ext/fiparse/test $ ruby  test_fachinfo_doc_parser.rb
....
87 tests, 359 assertions, 17 failures, 1 errors

Note

Some failures are similar, which are probably related to a same cause
The following part are commented out since it causes a segumentation fault.
ext/fiparse/test/test_fachinfo_doc_parser.rb

class TestFachinfoDocParser9 < Test::Unit::TestCase
...

  1) Failure:
test_composition10(TestFachinfoDocParser10) [test_fachinfo_doc_parser.rb:1083]:
<nil> expected to be an instance of
<ODDB::Text::Chapter> but was
<NilClass>.

Problem

'ODDB::FiParse::FachinfoTextHandler.writers.first.composition.class' does not become not ODDB::Text::Chapter but NilClass

Memo

I should understand the algorithm and data structure of fiparse library
I guess fiparse library is improved on rwv2 library
I should first understand the code structure of rwv2

Code structure (pattern) of rwv2

This is a kind of 'template method' pattern used
The main flow is probably defined in Rwv2.create_parser#parse
But this is implemented in C++ rwv2/ext/rwv2/rwv2.cpp
Basic method to use the rwv2 library is as follows:
1. Define a handler class that inherits Rwv2::**Handler class
2. Overwrite the methods which are defined in Rwv2::**Handler class
3. Create a parser instance by Rwv2#create_parser method with the handler instance
4. Call 'parse' method of the parser
5. Then the each method that is overwritten in the handler class will run

The sample is below

require 'rwv2'
require 'rwv2/handlers'
#require '/usr/lib64/ruby/gems/1.8/gems/rwv2-0.6.0/lib/rwv2/rwv2.rb'

class TextHandler < Rwv2::TextHandler
  def run_of_text(text, character_properties)
    puts text
  end
end

parser = Rwv2.create_parser('test.doc')
parser.set_text_handler(TextHandler.new)
parser.parse

Note

In the 'parser.parse' method, the TextHandler#run_of_text is called
The arguments of the run_of_text method have already meanings (functions)
The run_of_text method is declared in Rwv2::TextHandler class but nothing is defined
- See rwv2/lib/rwv2/rwv2_handler.rb
I guess Rwv2 can parse a word file (.doc) as a default
Probably FachinfoTextHandler is defined as it can parse the fachinfo doc file and analyze the file format

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

20110211-testcases-oddb_org

Make test-cases for ext scripts

Confirm each test-case in ext directory

Update test-case fiparse

Update test_fachinfo_doc_parser.rb

ywesee Developer-Wiki Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

20110211-testcases-oddb_org

Make test-cases for ext scripts

Confirm each test-case in ext directory

Update test-case fiparse

Update test_fachinfo_doc_parser.rb

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler