Index>

20140616-sax-parser-for-fi-pi-import

Summary

• Use sax-parser to fix problem "Out of memory" for import_daily

Commits

• https://github.com/ngiger/oddb.org/commit/20c263348edccea66939f166edef5d10b4be85b1 Using sax-parser for text_info.
• https://github.com/ngiger/oddb.org/commit/c72b5862c4338cd1bb4506f1c786a9403a8e59e2 Updated to use sax-machine 0.2.1

Index

• Use sax-parser to fix problem "Out of memory" for import_daily
• jobs/import_swissmedic fails with undefined method `sequences' for "format_type":String

Keep in Mind

• Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
• I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
• The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
• One unit-test for searchbar fails and might be a clue why searching does not work correctly.
• Added two skip in test/test_plugin/rss.rb. Why does the mocking not work there anymore?

---

Use sax-parser to fix problem "Out of memory" for import_daily

We must now find

• a way to get the IKSNRs of all names found via "Neue Einträge" when running jobs/update_textinfo_swissmedicinfo
• a way to get the IKSNRs of all companies when running jobs/update_textinfo_swissmedicinfo company1 company2

Must also look that import work whether reparse is given or not

• jobs/update_textinfo_swissmedicinfo --no-download --target=both CBI (has20 entries)
• jobs/update_textinfo_swissmedicinfo --no-download --target=both --reparse CBI (has20 entries)
• jobs/update_textinfo_swissmedicinfo --no-download --target=pi 42542
• jobs/update_textinfo_swissmedicinfo --no-download --target=fi 42542
• jobs/update_textinfo_swissmedicinfo --no-download --target=both 42542
• jobs/update_textinfo_swissmedicinfo --no-download --target=both --reparse 42542 (has 5 entries)
• jobs/update_textinfo_swissmedicinfo # should call import_swissmedicinfo_by_index, we have no unit-test for it!!!
• jobs/update_swissmedicno_fi_pi
• jobs/import_daily

I must add unit-tests for these combination in test/test_plugin/text_info_swissmedicinfo.rb. Deleted the fachinfo for 42542.

After calling jobs/update_textinfo_swissmedicinfo --no-download --target=both 42542 the fachinfo was not updated. Why? Checking out master branch to test, whether it works there. Now the fachinfo gets updated, but the import hangs(?) while reparse. Manually aborting after letting it run 10 minutes with 100% CPU-usage.

Analysing the context I see that @format and @title are implicit parameters when calling parse_fachinfo. Setting them therefore. And bingo, the import works! Changing @title and @format to explicit parameter with default values. But the fachinfo is only show and recognised after restarting oddbd.

Anyways I think this is good enough for today. Running import_daily and will push commit tomorrow morning if everything is okay.

Pushed commit Using sax-parser for text_info.

Updated to use sax-machine 0.2.1. Running import-daily again.