Index>

20140519-export-oddb2dat

Summary

• jobs/export_oddbdat muss durchlaufen. Fix Message: undefined method `formats' for (image):ODDB::Text::ImageLink

Commits

• https://github.com/ngiger/oddb.org/commit/83eefb14e88c1c54c99463c59224b0ce022c8f8d Don't call export_oddb2tdat/export_oddb2tdat_with_migel from import_daily (via bsv_follower)

Index

• jobs/export_oddbdat muss durchlaufen. Fix Message: undefined method `formats' for (image):ODDB::Text::ImageLink
• update (import_daily) should not call oddb2dat and oddb2dat_with_migel
• Fix Error: Patents with could not connect to www.swissreg.ch: #<Net::HTTPInternalServerError:0x007f8a7d69bb58>

Keep in Mind

• Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
• We should probably replace all occurrences of ch.oddb.org by the value of SERVER_NAME from src/util/oddbconfig.rb.

---

jobs/export_oddbdat muss durchlaufen. Fix Message: undefined method `formats' for (image):ODDB::Text::ImageLink

We got the following proble

Error: NoMethodError
Message: undefined method `formats' for (image):ODDB::Text::ImageLink
Backtrace:
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:173:in `block (2 levels) in format_line'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:166:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:166:in `block in format_line'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:161:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:161:in `format_line'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:139:in `block (2 levels) in lines'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:206:in `call'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:206:in `block in each_chapter'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:204:in `each'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:204:in `each_chapter'
(druby://localhost:10005) /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:138:in `block in lines'
(druby://localhost:10005) /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `each'
(druby://localhost:10005) /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:136:in `lines'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:190:in `block (2 levels) in export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:189:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:189:in `block in export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:187:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:187:in `export_oddbdat'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1508:in `perform'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
/var/www/oddb.org/src/plugin/oddbdat_export.rb:144:in `export_fachinfos'
/var/www/oddb.org/src/util/exporter.rb:196:in `block in export_oddbdat'
/var/www/oddb.org/src/util/exporter.rb:425:in `call'
/var/www/oddb.org/src/util/exporter.rb:425:in `safe_export'
/var/www/oddb.org/src/util/exporter.rb:192:in `export_oddbdat'
/var/www/oddb.org/src/util/exporter.rb:67:in `block in run'
/var/www/oddb.org/src/util/schedule.rb:15:in `call'
/var/www/oddb.org/src/util/schedule.rb:15:in `run_on_weekday'
/var/www/oddb.org/src/util/exporter.rb:63:in `run'
jobs/export_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/export_daily:12:in `<module:Util>'
jobs/export_daily:11:in `<module:ODDB>'
jobs/export_daily:10:in `<main>'

Verifying that I get the same error when calling jobs/export_oddbdat. Yes. Added debug statement and a possible workaround.

Fix seems to work, but I got now another problem:

Error: NoMethodError
Message: undefined method `heading' for nil:NilClass
Backtrace:
(druby://localhost:10005) /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:160:in `format_line'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:139:in `block (2 levels) in lines'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:206:in `call'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:206:in `block in each_chapter'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:204:in `each'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:204:in `each_chapter'
(druby://localhost:10005) /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:138:in `block in lines'
(druby://localhost:10005) /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `each'
(druby://localhost:10005) /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:136:in `lines'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:190:in `block (2 levels) in export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:189:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:189:in `block in export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:187:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:187:in `export_oddbdat'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1508:in `perform'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
/var/www/oddb.org/src/plugin/oddbdat_export.rb:130:in `export_by_company_name'
/var/www/oddb.org/src/util/exporter.rb:209:in `block in export_oddbdat_by_company_name'
/var/www/oddb.org/src/util/exporter.rb:425:in `call'
/var/www/oddb.org/src/util/exporter.rb:425:in `safe_export'
/var/www/oddb.org/src/util/exporter.rb:207:in `export_oddbdat_by_company_name'
jobs/export_oddbdat:14:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/export_oddbdat:12:in `<module:Util>'
jobs/export_oddbdat:11:in `<module:ODDB>'
jobs/export_oddbdat:10:in `<main>'

Again. More debug info & trying a fix.

Got another problem

Error: NoMethodError
Message: undefined method `-' for nil:NilClass
Backtrace:
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:143:in `block (2 levels) in lines'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:209:in `call'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:209:in `block in each_chapter'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:206:in `each'
(druby://localhost:10005) /var/www/oddb.org/src/model/fachinfo.rb:206:in `each_chapter'
(druby://localhost:10005) /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:138:in `block in lines'
(druby://localhost:10005) /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `each'
(druby://localhost:10005) /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/oddbdat.rb:136:in `lines'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:190:in `block (2 levels) in export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:189:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:189:in `block in export_oddbdat'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:187:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:187:in `export_oddbdat'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1508:in `perform'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:10005) /usr/lib64/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
/var/www/oddb.org/src/plugin/oddbdat_export.rb:130:in `export_by_company_name'
/var/www/oddb.org/src/util/exporter.rb:209:in `block in export_oddbdat_by_company_name'
/var/www/oddb.org/src/util/exporter.rb:425:in `call'
/var/www/oddb.org/src/util/exporter.rb:425:in `safe_export'
/var/www/oddb.org/src/util/exporter.rb:207:in `export_oddbdat_by_company_name'
jobs/export_oddbdat:14:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/export_oddbdat:12:in `<module:Util>'
jobs/export_oddbdat:11:in `<module:ODDB>'
jobs/export_oddbdat:10:in `<main>'

Trying to fix it, too. Seems to be in an eternal loop (still consuming 100% after 16h). Stopping all services again and restarting.

update (import_daily) should not call oddb2dat and oddb2dat_with_migel

Fixed with commit Don't call export_oddb2tdat/export_oddb2tdat_with_migel from import_daily (via bsv_follower)

Fix did not work. Trying another one and relaunching export.

Fix Error: Patents with could not connect to www.swissreg.ch: #<Net::HTTPInternalServerError:0x007f8a7d69bb58>

Error: RuntimeError
Message: could not connect to www.swissreg.ch https://www.swissreg.ch/srclient/faces/jsp/spc/sr3.jsp: #<Net::HTTPInternalServerError:0x00000002de0240>
Backtrace:
(druby://localhost:10007) /var/www/oddb.org/src/util/http.rb:98:in `post'
(druby://localhost:10007) /var/www/oddb.org/ext/swissreg/src/session.rb:203:in `post'
(druby://localhost:10007) /var/www/oddb.org/ext/swissreg/src/session.rb:196:in `get_result_list'
(druby://localhost:10007) /var/www/oddb.org/ext/swissreg/src/swissreg.rb:12:in `search'
(druby://localhost:10007) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block'
(druby://localhost:10007) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1508:in `perform'
(druby://localhost:10007) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:10007) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:10007) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
/var/www/oddb.org/src/plugin/swissreg.rb:58:in `update_registrations'
/var/www/oddb.org/src/plugin/swissreg.rb:52:in `block in update_news'
/var/www/oddb.org/src/plugin/swissreg.rb:49:in `each_key'
/var/www/oddb.org/src/plugin/swissreg.rb:49:in `update_news'
/var/www/oddb.org/src/util/updater.rb:510:in `update_immediate'
/var/www/oddb.org/src/util/updater.rb:423:in `update_swissreg_news'
/var/www/oddb.org/src/util/updater.rb:411:in `update_swissmedic_followers'
jobs/import_swissmedic:15:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_swissmedic:12:in `<module:Util>'
jobs/import_swissmedic:11:in `<module:ODDB>'
jobs/import_swissmedic:10:in `<main>'

Analysing the code in ext/swissreg/src/swissreg.rb I see that we already try twice with a sleep of 1 second between. Trying up 5 times and sleeping 10 seconds after a failure.

Look at unit test and after a few modifications I got exactly the error I was looking at!

But the unit-tests do not work, because the code in ext/swissreg/src changed in 2013 with commits by Yasu. Therefore I must see, what was changed since commit 7ddafd11e09de73ee957ddf02050daed0f7aee44 by Hannes.

I think a fieldname has changed in between, but I have troubles debugging the stuff.