Branch master
Branch saxParser
---
Plugin: ODDB::TextInfoPlugin Error: NoMethodError Message: undefined method `<<' for nil:NilClass Backtrace: (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:244:in `handle_element' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:290:in `block in handle_all_children' (druby://localhost:10002) /usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.6/lib/hpricot/traverse.rb:499:in `each' (druby://localhost:10002) /usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.6/lib/hpricot/traverse.rb:499:in `each_child' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:289:in `handle_all_children' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:236:in `handle_element' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:290:in `block in handle_all_children' (druby://localhost:10002) /usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.6/lib/hpricot/traverse.rb:499:in `each' (druby://localhost:10002) /usr/local/lib/ruby/gems/1.9.1/gems/hpricot-0.8.6/lib/hpricot/traverse.rb:499:in `each_child' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:289:in `handle_all_children' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:232:in `handle_element' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:43:in `chapter' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:89:in `block in extract' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:87:in `each' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/textinfo_hpricot.rb:87:in `extract' (druby://localhost:10002) /var/www/oddb.org/ext/fiparse/src/fiparse.rb:201:in `parse_fachinfo_html' (druby://localhost:10002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block' (druby://localhost:10002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1508:in `perform' (druby://localhost:10002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop' (druby://localhost:10002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop' (druby://localhost:10002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop' /var/www/oddb.org/src/plugin/text_info.rb:112:in `parse_fachinfo' /var/www/oddb.org/src/plugin/text_info.rb:1059:in `parse_and_update' /var/www/oddb.org/src/plugin/text_info.rb:1316:in `block in import_stuff' /var/www/oddb.org/src/plugin/text_info.rb:1311:in `each' /var/www/oddb.org/src/plugin/text_info.rb:1311:in `import_stuff' /var/www/oddb.org/src/plugin/text_info.rb:1365:in `import_swissmedicinfo_by_index' /var/www/oddb.org/src/plugin/text_info.rb:1433:in `import_swissmedicinfo' /var/www/oddb.org/src/util/updater.rb:525:in `block in update_notify_simple' /var/www/oddb.org/src/util/updater.rb:501:in `call' /var/www/oddb.org/src/util/updater.rb:501:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:519:in `update_notify_simple' /var/www/oddb.org/src/util/updater.rb:308:in `update_textinfo_swissmedicinfo' /var/www/oddb.org/src/util/updater.rb:193:in `run' jobs/import_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_daily:12:in `<module:Util>' jobs/import_daily:11:in `<module:ODDB>' jobs/import_daily:10:in `<main>'
Why did we get this error? Unfortunately. I do not see any trace entry to see which fachinfo was parsed. Adding such an entry in new branch saxParser via commit Log parsing pat/fachinfo After running it for some time I get the following log entry
2014-06-18 12:32:08 +0200: parse_and_update 1060: de calls parse_fachinfo dist /var/www/oddb.org/data/html/fachinfo/de/Sevoflurane Baxter_swissmedicinfo.html 57867 bytes name Sevoflurane Baxter info.title Sevoflurane Baxter 2014-06-18 12:32:08 +0200: /var/www/oddb.org/src/plugin/text_info.rb:112:in `parse_fachinfo': parse_fachinfo html path /var/www/oddb.org/data/html/fachinfo/de/Sevoflurane Baxter_swissmedicinfo.html format swissmedicinfo Sevoflurane Baxter 2014-06-18 12:32:08 +0200: parse_and_update done iksnr #<struct Struct::Stuff2Import iksnr="57671", lang="de", title="Sevoflurane Baxter", type=:fi, date=nil> Util.log_and_deliver_mail to=["ngiger@ywesee.com"] subject ch.ODDB.org Report - Error: Fach- und Patienteninfo Updates (swissmedicinfo.ch) - 06/2014 size Plugin: ODDB::TextInfoPlugin Error: NoMethodError
Tranforming it into a test-case via commit Fixed error while parsing Sevoflurane
Added two commits to run tests easily under Ruby 1.9.3 and Ruby 2.1.2 Added monkey patching CSV::Cell and Corrected dbi-version for 1.9.3
Restarting import_daily on oddb_ci2.
When looking for speed up I think the easiest possibility (but consuming about 1 GB of RAM) would be to cache the HTML-content of the fach/patinfo with the metadata, as the method extract_matched_content takes between 20 and 50 seconds for each of the 180 items to import, which amound to a big fraction of the used CPU-time.
As Zeno does not want use the saxParser branch until it is as fast as the master, I backported the changes to the master branch with commit Fixed UTF-8 and nil pointer access in textinfo_hpricot
Fixed problem that http access did not work by removing word executable bit in doc/index.rbx. Now I the home page displays, but I see the following error in bin/oddbd
initialized: 9.060264524 error in SBSM::Session#process: / NoMethodError undefined method `[]' for #<DRb::DRbUnknown:0x007f1ddbede388> /usr/local/lib/ruby/gems/2.1.0/gems/sbsm-1.2.5/lib/sbsm/session.rb:208:in `import_cookies' /usr/local/lib/ruby/gems/2.1.0/gems/sbsm-1.2.5/lib/sbsm/session.rb:360:in `process' /var/www/oddb.org/src/util/session.rb:125:in `process' /usr/local/lib/ruby/gems/2.1.0/gems/sbsm-1.2.5/lib/sbsm/session.rb:172:in `block in drb_process' /usr/local/lib/ruby/gems/2.1.0/gems/sbsm-1.2.5/lib/sbsm/session.rb:171:in `synchronize'
Reverted my manual change to /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache_entry.rb. I must investigate and fix the error
/var/www/oddb.org/src/util/oddbapp.rb:1459:in `initialize'
bin/oddbd:42:in `new'
bin/oddbd:42:in `<main>'
/usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache_entry.rb:53:in `block in odba_add_reference': undefined method `odba_id' for module `Kernel' (NameError)
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache_entry.rb:52:in `synchronize'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache_entry.rb:52:in `odba_add_reference'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:322:in `block (3 levels) in fetch_or_restore'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:313:in `call'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:313:in `fetch_or_do'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:320:in `block (2 levels) in fetch_or_restore'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:319:in `synchronize'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:319:in `block in fetch_or_restore'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:313:in `call'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:313:in `fetch_or_do'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:317:in `fetch_or_restore'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:304:in `block in fetch_named'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:313:in `call'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:313:in `fetch_or_do'
from /usr/local/lib/ruby/gems/2.1.0/gems/odba-1.1.0/lib/odba/cache.rb:296:in `fetch_named'
from /var/www/oddb.org/src/util/oddbapp.rb:1459:in `initialize'
from bin/oddbd:42:in `new'
from bin/oddbd:42:in `<main>'
Adding a trigger in the same file for ruby 1.9.3 to anaylize the stuff. The variable odba_id was set got '2' in cache_entry.db for ODDB::App. After continuing the system intitialized, too and I more or less the same error for 1.9 when acessing http://oddb-ci2.dyndns.org/
From: /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb @ line 50 ODBA::CacheEntry#odba_add_reference:
47: def odba_add_reference(object)
48: if object.class == ODDB::App
49: puts caller
=> 50: require 'pry'; binding.pry
51: end
52: @cache_entry_mutex.synchronize do
53: @accessed_by.store(object.object_id, object.odba_id)
54: end
55: object
56: end
[1] pry(#<ODBA::CacheEntry>)> odba_id
=> 2
2] pry(#<ODBA::CacheEntry>)> continue
init system
<..>
initialized: 39.078981212
error in SBSM::Session#process: /
NoMethodError
undefined method `[]' for #<DRb::DRbUnknown:0x007f269642bf58>
/var/www/.gem/ruby/1.9.1/gems/sbsm-1.2.5/lib/sbsm/session.rb:208:in `import_cookies'
/var/www/.gem/ruby/1.9.1/gems/sbsm-1.2.5/lib/sbsm/session.rb:360:in `process'
/var/www/oddb.org/src/util/session.rb:125:in `process'
/var/www/.gem/ruby/1.9.1/gems/sbsm-1.2.5/lib/sbsm/session.rb:172:in `block in drb_process'
<internal:prelude>:10:in `synchronize'
Therefore my changes for Ruby 2.1 have broken some stuff in Ruby 1.9. But what? Added a tag for easier comparing git tag started_ruby_21x c72b5862c4338cd1bb4506f1c786a9403a8e59e2 and analysed all changes in src since tag started_ruby_21x.
Looking at the request which fails
[3] pry(#<ODDB::Session>)> request => #<DRb::DRbObject:0x007f59d80143f0 @ref=69845763075840, @uri="druby://localhost:43422"> [4] pry(#<ODDB::Session>)> request.unparsed_uri => "/"
Bisecting the changes. The following changeset still have the problem in 1.9.3 1cdf6b18add3b51e38fb5bdd3212d591130fba18, edd656b6aa100549629e51e453c16e969eaf1286, 98141c4599528ac6454d2224bba551a619892f1e. Even going back to the maste branch, I have the problem. Dropping and reinstalling the database.
Finally I could fix it by changing the line 208 in /usr/local/lib/ruby/gems/1.9.1/gems/sbsm-1.2.3/lib/sbsm/session.rb to if RUBY_VERSION < "1.9" && (cuki = request.cookies[self::class::PERSISTENT_COOKIE_NAME]), which just uncomments it completely.
But the commit http://scm.ywesee.com/?p=sbsm/.git;a=commitdiff;h=9684a6869ef58ffbdf92029c46a64d739108cd3e;hp=162fcbd620a071f5167c67a915f18e17317530bc and Yasus blog entry http://dev.ywesee.com/Yasu/20120926-debug-fachinfo-chapter-modification-prescription-search-style-chooser seems to indicate, that this code is necessary. And is it okay to have different version for sbsm in bin/oddbd and modruby? E.g. /etc/apache2/sites-enabled/000-default: conatins RubyAddPath /var/lib/gems/1.9.1/gems/sbsm-1.2.3/lib.