view · edit · sidebar · attach · print · history

Index>

20140826-debug-import-daily

Summary

  • Debug import-daily
  • Fix more errors for prescription
  • Make extended fachinfo search work again.

Commits

Index

Keep in Mind
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • One unit-test for searchbar fails and might be a clue why searching does not work correctly.
  • Added two skip in test/test_plugin/rss.rb. Why does the mocking not work there anymore?
  • Make ext/swissindex/test/test_swissindex.rb and skipping tests in migel-gem pass.

---

Debug import-daily

With my local changes from yesterday, import_daily hangs without any output after

2014-08-25 17:19:29 +0200: update_fachinfo Konakion® MM iksnr 48112 store_fachinfo {}

Is using 100% of CPU of two CPU's. Must find out why. How can I debug this stuff?

http://spin.atomicobject.com/2013/08/19/debug-ruby-processes/. May be I will add the macros from there into my ~/.gdbinit.

  • Attach to process: sudo gdb -p 19500 /usr/local/bin/ruby
  • Add some gdb macro for ruby debugging
define evalr
  call(rb_p(rb_eval_string_protect($arg0,(int*)0)))
end
document evalr
   Evaluate an arbitrary Ruby expression from current gdb context.
end
  • evalr "puts caller.join('\n')", but it does not print anything. After Ctrl-C I got
[Switching to Thread 0x7fe09fca4700 (LWP 19500)]
warning: Unable to restore previously selected frame.
The program stopped in another thread while making a function call from GDB.
Evaluation of the expression containing the function

Therefore adding the macros to my .gdbinit and restarting import_daily and attaching early to try to see, whether debugging with gdb is possible or not.

Created a symbolic link from my ~/.gdbinit to /root to have it evaluated when running gdb as root, too. Now executing evalr "$stdout.puts caller[0..].join('\n')" prints the stack but also crashes ruby with

/usr/local/lib/ruby/1.9.1/net/protocol.rb:141: [BUG] object allocation during garbage collection phase
ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]

and about 500 lines of stack-trace. I do not attach it here, because I am not interested in this stack trace. But I will restart import_daily now and wait whether it will also go into an eternal loop. Today I seem not to have any problem as the import has already passed the point where it choked yesterday.

But just a few minutes late bin/oddbd crashe with the following stack trace. See Attach:stacktrace.txt

And in the import_daily I did see quite a few lines which point to empty fachinfos.

ODBA::Stub was unable to replace ODDB::Text::Chapter#31095013 from ODDB::FachinfoDocument2001:#31095007
ODBA::Stub was unable to replace ODDB::SimpleLanguage::Descriptions#30593148 from ODDB::Fachinfo:#30593146
ODBA::Stub was unable to replace ODDB::Text::Chapter#31154072 from ODDB::FachinfoDocument2001:#31154055
ODBA::Stub was unable to replace ODDB::Text::Chapter#31153793 from ODDB::FachinfoDocument2001:#31153776
ODBA::Stub was unable to replace ODDB::Text::Chapter#31140320 from ODDB::FachinfoDocument2001:#31140303
ODBA::Stub was unable to replace ODDB::Text::Chapter#31109237 from ODDB::FachinfoDocument2001:#31109230
ODBA::Stub was unable to replace ODDB::Text::Chapter#31103782 from ODDB::FachinfoDocument2001:#31103775
ODBA::Stub was unable to replace ODDB::Text::Chapter#31074697 from ODDB::FachinfoDocument2001:#31074690
ODBA::Stub was unable to replace ODDB::Text::Chapter#31074236 from ODDB::FachinfoDocument2001:#31074226
ODBA::Stub was unable to replace ODDB::Text::Chapter#31059256 from ODDB::FachinfoDocument2001:#31059249
ODBA::Stub was unable to replace ODDB::SimpleLanguage::Descriptions#30593148 from ODDB::Fachinfo:#30593146
ODBA::Stub was unable to replace ODDB::Text::Chapter#31095024 from ODDB::FachinfoDocument2001:#31095007

ch.oddb> ODBA.cache.fetch(31095007).amzv
-> 
ch.oddb> ODBA.cache.fetch(31095007).iksnrs
-> 
ch.oddb> ODBA.cache.fetch(31095007).date
-> 
ODBA.cache.fetch(30593148)
-> Unknown odba_id 30593148
ODBA.cache.fetch(31095013)
-> Unknown odba_id 31095013

Trying to find the fachinfos with problems in bin/admin

$fis = []; fachinfos.each{|key,value| $fis << value if value.iksnrs == nil }; $fis.size
-> 0

After restarting bin/oddbd import_daily continued with some warning like

       cp /var/www/oddb.org/data/xls/Packungen-2014.08.26.xlsx /var/www/oddb.org/data/xls/Packungen-latest.xlsx
Opened /var/www/oddb.org/log/oddb/debug/2014/08.log
*** WARNING: storage class not found for ../embeddings/oleObject1.bin (http://schemas.openxmlformats.org/officeDocument/2006/relationships/oleObject)
WARNING: RubyXL::Worksheet is not aware what to do with RubyXL::GenericStorageObject
*** WARNING: storage class not found for ../embeddings/oleObject1.bin (http://schemas.openxmlformats.org/officeDocument/2006/relationships/oleObject)
WARNING: RubyXL::Worksheet is not aware what to do with RubyXL::GenericStorageObject
*** WARNING: storage class not found for ../embeddings/oleObject1.bin (http://schemas.openxmlformats.org/officeDocument/2006/relationships/oleObject)
WARNING: RubyXL::Worksheet is not aware what to do with RubyXL::GenericStorageObject
Util.log_and_deliver_mail to=["ngiger@ywesee.com"] subject ch.ODDB.org Report - Swissmedic XLS - 07/2014 size ODDB::SwissmedicPlugin - Report 26.08.2014
<..>
NoMethodError: undefined method `values' for nil:NilClass when updating index 'sequence_vaccine' with a ODDB::Registration
["(eval):2:in `block in proc_instance_target'", "/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:249:in `call'", "/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:249:in `update_origin'", "/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:163:in `update'"]
[...]

oMethodError: undefined method `values' for nil:NilClass when updating index 'sequence_vaccine' with a ODDB::Registration

Pushed commits Initialize epha_interactions from CSV. Take 3 and Cleanup some test errors

Watir tests for prescriptions are now okay. Cleanup and running all watir specs before pushing. But it looks that my changes broke searching interactions using iksnr or atc-codes which is not acceptable. Found the problem.

Pushed Fix startup problem and Fix delete_all in prescription

Fix more errors for prescription

Zeno remarked that "Alle löschen" did not work when no ZSR_ID given. Added watir tests for this too. It looks like that delete_all worked correctly when there were comments for the drugs. Now trying to fix this issue.

Zeno also want the following cosmetic changes:

  • Interactions for a prescription should have the same visual appearance as in interactions (Currently they are inside a list)
  • When starting entering a prescription "Medikament hinzufügen" should have the same space above as below. (Currently it is in the same table row like "Rezept für").

Pushed commit Beautify layout of prescription

Make extended fachinfo search work again

The extended fachinfo search does not work. I already have a small watir test for it. Activating debugging output to which searches are done in src/util/oddbapp.rb in method retrieve_from_index.

After searching for a drug under http://oddb-ci2.dyndns.org/de/gcc/fachinfo_search/. We display the home page with the searchbar for the fachinfo below. Probably not what we intended. Also no drug is selected. The Javascript console gives me the following info

XHR finished loading: GET "http://oddb-ci2.dyndns.org/de/gcc/ajax_matches/index_name/oddb_package_name_with_size_company_name_ean13_fi/?search_query=As". dojo.js:15
XHR finished loading: GET "http://oddb-ci2.dyndns.org/de/gcc/ajax_matches/index_name/oddb_package_name_with_size_company_name_ean13_fi/?search_query=Asp". dojo.js:15
XHR finished loading: GET "http://oddb-ci2.dyndns.org/de/gcc/ajax_matches/index_name/oddb_package_name_with_size_company_name_ean13_fi/?search_query=Aspi". dojo.js:15
selectXhrRequest: window.location.href http://oddb-ci2.dyndns.org/de/gcc/fachinfo_search/ searchbar: 7680576730049 (index):97
xhrGet arg 7680576730049 ean13 7680576730049,7680576730049 for new_url http://oddb-ci2.dyndns.org/de/gcc/home_interactions (index):56
xhrGet call replace_element id drugs new_url http://oddb-ci2.dyndns.org/de/gcc/home_interactions,7680576730049 (index):66
XHR finished loading: GET "http://oddb-ci2.dyndns.org/de/gcc/home_interactions,7680576730049". dojo.js:15

Here we should probably go to http://oddb-ci2.dyndns.org/de/gcc/fachinfo_search/7680576730049 not http://oddb-ci2.dyndns.org/de/gcc/home_interactions,7680576730049. Trying this manually. This displays at least the again the fachinfo, but not the selected drug. But when adding ean the URL http://oddb-ci2.dyndns.org/de/gcc/fachinfo_search/ean/7680576730049 displays the search and the choosen drug. And with http://oddb-ci2.dyndns.org/de/gcc/fachinfo_search/7680576730049/7680193950301 it displays nicely two drugs.

Patching src/view/searchbar.rb to fix this problem. Now it is possible to add several drugs to the fachinfo_search view. But searching for a chapter does not work and resets URL to home. This will be fixed tomorrow.

view · edit · sidebar · attach · print · history
Page last modified on August 26, 2014, at 05:35 PM