---
Error: TypeError Message: can't iterate from Time Backtrace: /var/www/oddb.org/src/plugin/download_invoicer.rb:37:in `each' /var/www/oddb.org/src/plugin/download_invoicer.rb:37:in `include?' /var/www/oddb.org/src/plugin/download_invoicer.rb:37:in `include?' /var/www/oddb.org/src/plugin/download_invoicer.rb:37:in `block (2 levels) in filter_paid' /var/www/oddb.org/src/plugin/download_invoicer.rb:36:in `each_value' /var/www/oddb.org/src/plugin/download_invoicer.rb:36:in `block in filter_paid' /var/www/oddb.org/src/plugin/download_invoicer.rb:35:in `each_value' /var/www/oddb.org/src/plugin/download_invoicer.rb:35:in `filter_paid' /var/www/oddb.org/src/plugin/download_invoicer.rb:13:in `run' /var/www/oddb.org/src/util/exporter.rb:331:in `block in mail_download_invoices' /var/www/oddb.org/src/util/exporter.rb:431:in `call' /var/www/oddb.org/src/util/exporter.rb:431:in `safe_export' /var/www/oddb.org/src/util/exporter.rb:330:in `mail_download_invoices' /var/www/oddb.org/src/util/exporter.rb:59:in `block in run' /var/www/oddb.org/src/util/schedule.rb:10:in `call' /var/www/oddb.org/src/util/schedule.rb:10:in `run_on_monthday' /var/www/oddb.org/src/util/exporter.rb:58:in `run' jobs/export_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/export_daily:12:in `<module:Util>' jobs/export_daily:11:in `<module:ODDB>' jobs/export_daily:10:in `<main>'
Tests under test/test_plugin/download_invoicer.rb run fine and do not produce an error. Same for test/test_util/updater.rb and test/test_util/exporter.rb.
Patching src/util/exporter.rb to run only mail_download_invoices. Starting export_daily to try to reproduce the problem locally. Cannot reproduce it locally as I probable do not have any invoices. Therefore looking at the code to see whether I have a good idea on howto fix this problem without breaking anything else. No. This is no good idea, I must have a closer look at the data. Therefore getting yesterday's dump from thinpower.
Got the following error message today
Plugin: ODDB::BsvXmlPlugin Error: NoMethodError Message: undefined method `force_encoding' for #<Zip::InputStream:0x000000653486a8> Backtrace: /var/www/oddb.org/src/plugin/bsv_xml.rb:999:in `update_it_codes' /var/www/oddb.org/src/plugin/bsv_xml.rb:660:in `block (2 levels) in _update' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry.rb:500:in `get_input_stream' /var/www/oddb.org/src/plugin/bsv_xml.rb:660:in `block in _update' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `call' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `block in each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/central_directory.rb:182:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:139:in `block in foreach' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:99:in `open' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:138:in `foreach' /var/www/oddb.org/src/plugin/bsv_xml.rb:655:in `_update' /var/www/oddb.org/src/plugin/bsv_xml.rb:650:in `update' /var/www/oddb.org/src/util/updater.rb:246:in `block in update_bsv' /var/www/oddb.org/src/util/updater.rb:496:in `call' /var/www/oddb.org/src/util/updater.rb:496:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:244:in `update_bsv' /var/www/oddb.org/src/util/updater.rb:198:in `run' jobs/import_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_daily:12:in `<module:Util>' jobs/import_daily:11:in `<module:ODDB>' jobs/import_daily:10:in `<main>'
Running jobs/import_bsv to see whether I can reproduce the problem locally. Before each restart I manually copied data/xml/XMLPublications-2014.08.26.zip data/xmlXMLPublications-latest.zip. Yes I am lucky and it takes just about 15 seconds to reproduce the problem. Nice!
It looks like that my fix as described in http://dev.ywesee.com/Niklaus/20140811-Fix-invalid-byte-in-epha-interactions was wrong. ItCodes should be an XML files (if you look at it heading). And one cannot change the encoding of file inside ZipFile. Therefore we must patch the listener.
Okay. Converting into a StringIO seems to work. Using now as_utf_8 = StringIO.new(io.read.force_encoding('utf-8')) instead of as_utf_8 = io.force_encoding('utf-8'). Waiting for import to finish before pushing commit. Import-BSV used quite some CPU time till it accumlated about 3 minutes. Afterwards CPU load drops to around 3% and no visible progess is seen. Why does it take so long?
Now I see the following error
failsafe rescued TypeError < StandardError 0 is not a symbol /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:122:in `respond_to?' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:122:in `respond_to?' /var/www/oddb.org/src/util/persistence.rb:284:in `block in resolve' /var/www/oddb.org/src/util/persistence.rb:278:in `each' /var/www/oddb.org/src/util/persistence.rb:278:in `resolve' /var/www/oddb.org/src/util/persistence.rb:208:in `issue_create' /var/www/oddb.org/src/util/oddbapp.rb:110:in `block in create' /var/www/oddb.org/src/util/failsafe.rb:10:in `call' /var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe' /var/www/oddb.org/src/util/oddbapp.rb:109:in `create' /var/www/oddb.org/src/util/persistence.rb:288:in `block in resolve' /var/www/oddb.org/src/util/persistence.rb:278:in `each' /var/www/oddb.org/src/util/persistence.rb:278:in `resolve' /var/www/oddb.org/src/util/persistence.rb:256:in `issue_update' /var/www/oddb.org/src/util/oddbapp.rb:131:in `block in update' /var/www/oddb.org/src/util/failsafe.rb:10:in `call' /var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe' /var/www/oddb.org/src/util/oddbapp.rb:129:in `update' /var/www/oddb.org/src/util/oddbapp.rb:1537:in `update' /var/www/oddb.org/src/plugin/bsv_xml.rb:426:in `block in tag_end' /var/www/oddb.org/src/plugin/bsv_xml.rb:399:in `each' /var/www/oddb.org/src/plugin/bsv_xml.rb:399:in `tag_end' /usr/local/lib/ruby/1.9.1/rexml/parsers/streamparser.rb:26:in `parse' /usr/local/lib/ruby/1.9.1/rexml/document.rb:205:in `parse_stream' /var/www/oddb.org/src/plugin/bsv_xml.rb:1015:in `update_preparations' /var/www/oddb.org/src/plugin/bsv_xml.rb:660:in `block (2 levels) in _update' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry.rb:500:in `get_input_stream' /var/www/oddb.org/src/plugin/bsv_xml.rb:660:in `block in _update' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `call' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `block in each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/central_directory.rb:182:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:139:in `block in foreach' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:99:in `open' /usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:138:in `foreach' /var/www/oddb.org/src/plugin/bsv_xml.rb:655:in `_update' /var/www/oddb.org/src/plugin/bsv_xml.rb:650:in `update' /var/www/oddb.org/src/util/updater.rb:246:in `block in update_bsv' /var/www/oddb.org/src/util/updater.rb:496:in `call' /var/www/oddb.org/src/util/updater.rb:496:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:244:in `update_bsv' jobs/import_bsv:14:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_bsv:12:in `<module:Util>' jobs/import_bsv:11:in `<module:ODDB>' jobs/import_bsv:10:in `<main>'
Looks like we have problem updating a limitation_text. Adding debug output. Import db_dump from thinpower, then restarting the import.
Now it looks that the import finished. Seen the following error (probably missconfiguration of oddb-ci2)
til.log_and_deliver_mail to=["ngiger@ywesee.com"] subject ch.ODDB.org Report - Error: SL-Update (XML) - 09/2014 size Plugin: ODDB::BsvXmlPlugin Error: NoMethodError Message: undefined method `index' for nil:NilClass Backtrace: /var/www/oddb.org/src/util/mail.rb:50:in `block (2 levels) in get_mailing_list_anrede' /var/www/oddb.org/src/util/mail.rb:50:in `each' /var/www/oddb.org/src/util/mail.rb:50:in `block in get_mailing_list_anrede' /var/www/oddb.org/src/util/mail.rb:49:in `each' /var/www/oddb.org/src/util/mail.rb:49:in `get_mailing_list_anrede' /var/www/oddb.org/src/plugin/bsv_xml.rb:912:in `report_bsv' /var/www/oddb.org/src/plugin/bsv_xml.rb:832:in `log_info_bsv' /var/www/oddb.org/src/util/updater.rb:144:in `log_info' /var/www/oddb.org/src/util/updater.rb:473:in `log_notify_bsv' /var/www/oddb.org/src/util/updater.rb:251:in `block in update_bsv' /var/www/oddb.org/src/util/updater.rb:496:in `call' /var/www/oddb.org/src/util/updater.rb:496:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:244:in `update_bsv' jobs/import_bsv:14:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_bsv:12:in `<module:Util>' jobs/import_bsv:11:in `<module:ODDB>' jobs/import_bsv:10:in `<main>'
Mails came in and job completed successfully. Therefore pushed commit Fix non-UTF-8 encoding for bsv.xml. Pulled on thinpower and running import_daily under screen import_daily.
I tried already once to add a watir test for downloading a CSV. One of the problems is, that depending on the preferences a browser will download to a different location (or even open the document differently). And changing these settings is different for each browser and did not work for Chrome when I tried it a few months ago.
In the watir test I will simply assume that chrome will download the files to $HOME/Downloads. Before exporting I will simply delete all files there to be able to detect a download using a wrong name. No. It is simpler to just glob all existing files in that directory and compare the files before and after downloads. This worked fine and now I committed Added watir test for download of example.csv
Created a watir test to login, search for Aspirin, click proceed_payment, click checkout_invoice. Now it fails because the URL http://oddb-ci2.dyndns.org/resources/errors/appdown.html matches errors. No trace about an error on oddbd nor in log/oddb/debug/2014/09.log. No trace in apache/error.log neither. No output in the java-script console neither. Using Firefox Extension HttpFox to follow the I see when submitinggnt checkout_invoice
checkout_invoice Zu Rechnung hinzufügen flavor gcc language de event checkout state_id 70003414376460 pointer :!create,:!invoice.. zone drugs
About 800 ms afterward I get the redirect to GET /resources/errors/appdown.html HTTP/1.1 Adding debug output in oddbapp/create_invoice and src/plugin/invoicer.rb. Looking for testcase. Found errors because of my debug output overwrote the return value. Correcting debug output.
Added some spec tests for searching doctors, hospitals, substances. Searching via analysis fails. Should be corrected in the next few day.
Pushed commit Added some watir test