<< | Index | >>
Currently we have two errors when running test/test_util/validator.rb
1) Error:
test_search_query(TestOddbValidator):
NoMethodError: undefined method `u' for #<ODDB::Validator:0x00000002cf8120>
/opt/src/oddb.org/src/util/validator.rb:578:in `search_query'
/home/niklaus/.rvm/gems/ruby-1.9.3-p448/gems/sbsm-1.2.3/lib/sbsm/validator.rb:146:in `perform_validation'
/home/niklaus/.rvm/gems/ruby-1.9.3-p448/gems/sbsm-1.2.3/lib/sbsm/validator.rb:119:in `validate'
/opt/src/oddb.org/test/test_util/validator.rb:72:in `test_search_query'
2) Error:
test_search_query_shorter_than_3(TestOddbValidator):
NoMethodError: undefined method `u' for #<ODDB::Validator:0x00000002d2c948>
/opt/src/oddb.org/src/util/validator.rb:578:in `search_query'
/home/niklaus/.rvm/gems/ruby-1.9.3-p448/gems/sbsm-1.2.3/lib/sbsm/validator.rb:146:in `perform_validation'
/home/niklaus/.rvm/gems/ruby-1.9.3-p448/gems/sbsm-1.2.3/lib/sbsm/validator.rb:119:in `validate'
/opt/src/oddb.org/test/test_util/validator.rb:75:in `test_search_query_shorter_than_3'
39 tests, 63 assertions, 0 failures, 2 errors, 0 skips
This small patch fixes the problem
index 7f67e73..2ad4f73 100644
--- a/src/util/validator.rb
+++ b/src/util/validator.rb
@@ -575,7 +575,7 @@ module ODDB
value = @@utf8.iconv(value)
result = @@utf8.iconv(result)
end
- result = u result
+ result = result.force_encoding('utf-8')
if(result.length > 2)
result
else
I tried to reproduce the problemm on oddb-ci.dyndns.org by Executing a search with 'po', but got the @@Es ist ein Fehler aufgetreten: Ihr Such-Stichwort ergibt ein sehr grosses Resultat. Bitte verwenden Sie mindestens 3 Buchstaben.@@
Apply the patch and rerunning the same query. No problem.
Found a few definitions for a method u on my VM via
grep -rw "def u" ~/.rvm/
/home/vagrant/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/matrix/lup_decomposition.rb: def u
/home/vagrant/.rvm/src/ruby-1.9.3-p0/test/ruby/test_m17n_comb.rb: def u(str) str.dup.force_encoding("UTF-8") end
/home/vagrant/.rvm/src/ruby-1.9.3-p0/test/ruby/test_m17n.rb: def u(str) str.dup.force_encoding("UTF-8") end
/home/vagrant/.rvm/src/ruby-1.9.3-p0/test/ruby/test_string.rb: def u(str)
/home/vagrant/.rvm/src/ruby-1.9.3-p0/lib/matrix/lup_decomposition.rb: def u
/home/vagrant/.rvm/gems/ruby-1.9.3-p0/gems/character-encodings-0.4.1/lib/encoding/character/utf-8.rb: def u(str)
/home/vagrant/.rvm/gems/ruby-1.9.3-p0/gems/odba-1.1.0/lib/odba/18_19_loading_compatibility.rb: def u str
Inside my /home/niklaus/.rvm/rubies/ruby-1.9.3-p448/bin/ruby I found it only once /home/niklaus/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/matrix/lup_decomposition.rb: def u
Therefore I think my patch is correct and compatible with newer version of Ruby. But I wondering why in our test-suite the 18_19_loading_compatibility.rb is not loaded. Anyway in 18_19_loading_compatibility.rb the method "u" returns an unchanged string. Therefore I could simplify my patch to remove the line completely.
Pushed commit Remove obsolete call to u
Running with ruby 1.9.3-p0 on my VM the job sudo -u apache /usr/local/rvm/rubies/ruby-1.9.3-p0/bin/ruby /var/www/oddb.org/jobs/mail_index_therapeuticus_csv 2>&1 | tee /vagrant/mail_index_therapeuticus_csv.log
I don't get any errors but at least 1902 warnings /usr/local/rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/net/protocol.rb:313: warning: regexp match /.../n against to UTF-8 string which might help
With the following command I see the status of each test-suite by parsing its log
2013-09-02 19:13:53 +0200: Now testing /opt/src/oddb.org/test/../ext/suite.rb true 239 tests, 593 assertions, 17 failures, 21 errors, 0 pendings, 0 omissions, 0 notifications 2013-09-02 19:14:33 +0200: Now testing /opt/src/oddb.org/test/test_state/suite.rb true 796 tests, 816 assertions, 54 failures, 83 errors, 0 skips 2013-09-02 19:14:43 +0200: Now testing /opt/src/oddb.org/test/test_view/suite.rb true 271 tests, 299 assertions, 33 failures, 23 errors, 1 skips 2013-09-02 19:14:48 +0200: Now testing /opt/src/oddb.org/test/test_command/suite.rb true 3 tests, 3 assertions, 0 failures, 0 errors, 0 skips 2013-09-02 19:14:55 +0200: Now testing /opt/src/oddb.org/test/test_remote/suite.rb true 6 tests, 6 assertions, 1 failures, 0 errors, 0 skips 2013-09-02 19:14:58 +0200: Now testing /opt/src/oddb.org/test/test_custom/suite.rb true 43 tests, 43 assertions, 10 failures, 0 errors, 0 skips 2013-09-02 19:15:02 +0200: Now testing /opt/src/oddb.org/test/test_plugin/suite.rb true 446 tests, 658 assertions, 14 failures, 72 errors, 0 skips 2013-09-02 19:15:45 +0200: Now testing /opt/src/oddb.org/test/test_model/suite.rb true 661 tests, 1380 assertions, 22 failures, 1 errors, 0 skips 2013-09-02 19:15:52 +0200: Now testing /opt/src/oddb.org/test/test_util/suite.rb true 312 tests, 457 assertions, 17 failures, 6 errors, 0 skips
Will clean-up the unit test using the following strategy.
Pushed a few commits
Found one place where we still had a non-UTF-8 string in the source code. Fixed it with commit Fixed UTF-8 error in ydim.rb and unit-tests. I think our problem when exporting the doctors.csv could stem from another occurrence of the same problem. Is there maybe a header or a database column still defined with a non-UTF-8 string?