One of my commits broke running the Migel imports. E.g
NoMethodError undefined method `session' for ODDB::Swissindex:Module (druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/invokemethod.rb:14:in `perform_with_block' (druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1506:in `perform' (druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop' (druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop' (druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop' /var/www/migel/lib/migel/plugin/swissindex.rb:36:in `block in save_all_products' /usr/local/lib/ruby/1.9.1/csv.rb:1367:in `open' /var/www/migel/lib/migel/plugin/swissindex.rb:35:in `save_all_products' /var/www/migel/lib/migel/util/importer.rb:350:in `save_all_products' /var/www/migel/lib/migel/util/importer.rb:234:in `reported_save_all_products' /var/www/migel/lib/migel/util/importer.rb:224:in `block in save_all_products_all_languages' /var/www/migel/lib/migel/util/importer.rb:221:in `each' /var/www/migel/lib/migel/util/importer.rb:221:in `save_all_products_all_languages' jobs/update_migel_products_with_report:13:in `block in <module:Util>' /var/www/migel/lib/migel/util/job.rb:22:in `call' /var/www/migel/lib/migel/util/job.rb:22:in `run' jobs/update_migel_products_with_report:12:in `<module:Util>' jobs/update_migel_products_with_report:11:in `<module:Migel>' jobs/update_migel_products_with_report:10:in `<main>'
The migel import do not work anymore, because some classes were renamed and not tested. When removing ext/swissindex/bin/swissindex_(non)pharmad, I did not think of the migel import and update_migel_products_with_report is run only on 1 of January and June. The commit Changed URL for refdata changed the names SwissindexPharma and SwissindexNonpharma -> SwissindexMigel. But no corresponding change in the migel repository was done.
Made some changes and the update created several files, but crashed with
2016-01-12 10:23:04 +0100: update_group ["99", "50", "01", "00", "1"] groupcd 99 de /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:525: [BUG] Segmentation fault ruby 1.9.3p545 (2014-02-24 revision 45159) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0028 p:0020 s:0101 b:0099 l:000086 d:000098 BLOCK /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:525 c:0027 p:---- s:0096 b:0096 l:000095 d:000095 FINISH c:0026 p:---- s:0094 b:0094 l:000089 d:000093 IFUNC c:0025 p:---- s:0092 b:0092 l:000091 d:000091 CFUNC :each c:0024 p:---- s:0090 b:0090 l:000089 d:000089 CFUNC :any? c:0023 p:0040 s:0087 b:0087 l:000086 d:000086 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:524 c:0022 p:0051 s:0083 b:0083 l:000082 d:000082 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:37 c:0021 p:0024 s:0076 b:0076 l:000075 d:000075 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:72 c:0020 p:0028 s:0073 b:0073 l:000072 d:000072 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:309 c:0019 p:0018 s:0066 b:0066 l:0022b8 d:0022b8 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:225 c:0018 p:0056 s:0061 b:0061 l:000060 d:000060 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:45 c:0017 p:0014 s:0057 b:0057 l:000046 d:000056 BLOCK /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:97 c:0016 p:---- s:0052 b:0052 l:000051 d:000051 FINISH c:0015 p:---- s:0050 b:0050 l:000049 d:000049 CFUNC :delete_if c:0014 p:0073 s:0047 b:0047 l:000046 d:000046 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:96 c:0013 p:0049 s:0042 b:0042 l:000028 d:000041 BLOCK /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:105 c:0012 p:---- s:0039 b:0039 l:000038 d:000038 FINISH c:0011 p:---- s:0037 b:0037 l:000036 d:000036 CFUNC :each_value c:0010 p:0011 s:0034 b:0034 l:000028 d:000033 BLOCK /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:102 c:0009 p:0019 s:0032 b:0032 l:000031 d:000031 METHOD <internal:prelude>:10 c:0008 p:0050 s:0029 b:0029 l:000028 d:000028 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:101 c:0007 p:0110 s:0021 b:0021 l:000020 d:000020 METHOD /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:79 c:0006 p:0030 s:0014 b:0014 l:000968 d:000013 BLOCK /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:490 c:0005 p:---- s:0011 b:0011 l:000010 d:000010 FINISH c:0004 p:---- s:0009 b:0009 l:000008 d:000008 CFUNC :loop c:0003 p:0040 s:0006 b:0006 l:000968 d:000005 BLOCK /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:487 c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH c:0001 p:---- s:0002 b:0002 l:000001 d:000001 TOP -- Ruby level backtrace information ---------------------------------------- /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:487:in `block in start_cleaner' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:487:in `loop' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:490:in `block (2 levels) in start_cleaner' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:79:in `clean' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:101:in `_clean' <internal:prelude>:10:in `synchronize' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:102:in `block in _clean' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:102:in `each_value' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:105:in `block (2 levels) in _clean' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:96:in `odba_retire' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:96:in `delete_if' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:97:in `block in odba_retire' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:45:in `odba_id2ref' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:225:in `fetch' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:309:in `fetch_or_do' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:72:in `_odba_object' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache_entry.rb:37:in `object_id2ref' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:524:in `odba_unsaved?' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:524:in `any?' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:524:in `each' /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:525:in `block in odba_unsaved?' -- C level backtrace information ------------------------------------------- /usr/lib64/libruby19.so.1.9(+0x159cec) [0x7f32f374ccec] /usr/lib64/libruby19.so.1.9(+0x5a119) [0x7f32f364d119] /usr/lib64/libruby19.so.1.9(rb_bug+0xb7) [0x7f32f364da47] /usr/lib64/libruby19.so.1.9(+0xf939e) [0x7f32f36ec39e] /lib64/libc.so.6(+0x38500) [0x7f32f3281500] /usr/lib64/libruby19.so.1.9(+0x1444f8) [0x7f32f37374f8] /usr/lib64/libruby19.so.1.9(+0x146721) [0x7f32f3739721] /usr/lib64/libruby19.so.1.9(+0x14e9e7) [0x7f32f37419e7] /usr/lib64/libruby19.so.1.9(+0x14f3f3) [0x7f32f37423f3] /usr/lib64/libruby19.so.1.9(+0x5436e) [0x7f32f364736e] /usr/lib64/libruby19.so.1.9(+0x147856) [0x7f32f373a856] /usr/lib64/libruby19.so.1.9(rb_yield+0x4db) [0x7f32f374861b] /usr/lib64/libruby19.so.1.9(rb_ary_each+0x52) [0x7f32f3620862] /usr/lib64/libruby19.so.1.9(+0x1504e7) [0x7f32f37434e7] /usr/lib64/libruby19.so.1.9(rb_iterate+0xd5) [0x7f32f3739da5] /usr/lib64/libruby19.so.1.9(rb_block_call+0x2d) [0x7f32f3739f6d] /usr/lib64/libruby19.so.1.9(+0x53970) [0x7f32f3646970] /usr/lib64/libruby19.so.1.9(+0x152dc1) [0x7f32f3745dc1] /usr/lib64/libruby19.so.1.9(+0x149272) [0x7f32f373c272] /usr/lib64/libruby19.so.1.9(+0x14f3f3) [0x7f32f37423f3] /usr/lib64/libruby19.so.1.9(rb_yield_values+0x2fe) [0x7f32f374895e] /usr/lib64/libruby19.so.1.9(+0x734dc) [0x7f32f36664dc] /usr/lib64/libruby19.so.1.9(+0x72782) [0x7f32f3665782] /usr/lib64/libruby19.so.1.9(st_foreach+0xd1) [0x7f32f36f4871] /usr/lib64/libruby19.so.1.9(+0x73a0a) [0x7f32f3666a0a] /usr/lib64/libruby19.so.1.9(rb_ensure+0xbb) [0x7f32f3651d7b] /usr/lib64/libruby19.so.1.9(rb_hash_foreach+0x36) [0x7f32f3666fa6] /usr/lib64/libruby19.so.1.9(rb_hash_delete_if+0x27) [0x7f32f36680b7] /usr/lib64/libruby19.so.1.9(+0x152dc1) [0x7f32f3745dc1] /usr/lib64/libruby19.so.1.9(+0x149272) [0x7f32f373c272] /usr/lib64/libruby19.so.1.9(+0x14f3f3) [0x7f32f37423f3] /usr/lib64/libruby19.so.1.9(rb_yield+0x26e) [0x7f32f37483ae] /usr/lib64/libruby19.so.1.9(+0x72442) [0x7f32f3665442] /usr/lib64/libruby19.so.1.9(+0x72782) [0x7f32f3665782] /usr/lib64/libruby19.so.1.9(st_foreach+0xd1) [0x7f32f36f4871] /usr/lib64/libruby19.so.1.9(+0x73a0a) [0x7f32f3666a0a] /usr/lib64/libruby19.so.1.9(rb_ensure+0xbb) [0x7f32f3651d7b] /usr/lib64/libruby19.so.1.9(rb_hash_foreach+0x36) [0x7f32f3666fa6] /usr/lib64/libruby19.so.1.9(+0x741be) [0x7f32f36671be] /usr/lib64/libruby19.so.1.9(+0x152dc1) [0x7f32f3745dc1] /usr/lib64/libruby19.so.1.9(+0x149272) [0x7f32f373c272] /usr/lib64/libruby19.so.1.9(+0x14f3f3) [0x7f32f37423f3] /usr/lib64/libruby19.so.1.9(+0x1543d3) [0x7f32f37473d3] /usr/lib64/libruby19.so.1.9(rb_rescue2+0x16b) [0x7f32f3651a0b] /usr/lib64/libruby19.so.1.9(+0x145c7e) [0x7f32f3738c7e] /usr/lib64/libruby19.so.1.9(+0x152dc1) [0x7f32f3745dc1] /usr/lib64/libruby19.so.1.9(+0x149272) [0x7f32f373c272] /usr/lib64/libruby19.so.1.9(+0x14f3f3) [0x7f32f37423f3] /usr/lib64/libruby19.so.1.9(+0x150152) [0x7f32f3743152] /usr/lib64/libruby19.so.1.9(+0x1606ee) [0x7f32f37536ee] /usr/lib64/libruby19.so.1.9(+0x1607d7) [0x7f32f37537d7] /lib64/libpthread.so.0(+0x91a7) [0x7f32f30341a7] /lib64/libc.so.6(clone+0x6d) [0x7f32f333720d] -- Other runtime information ----------------------------------------------- <...>
The produced files migel_products_<lang>.csv contain only the header. MiGeL-2016.01.12.xls looks okay and contains the changes mentioned by the BAG in http://www.bag.admin.ch/themen/krankenversicherung/00263/00264/04184/index.html?lang=de&download=NHzLpZeg7t,lnp6I0NTU042l2Z6ln1acy4Zn4Z2qZpnO2Yuq2Z6gpJCMe4B4g2ym162epYbg2c_JjKbNoKSn6A--
One of the changes is that position 21.05.02.03.2 was superseded 21.05.02.03.3. Looking whether I can find the new item in the database.
Verifying with bin/admin of oddb.org
ch.oddb> search_migel_products('21.05.02.03.3','de')
-> [Miete/Monat]
ch.oddb> search_migel_products('21.05.02.03.2','de')
-> [Miete/Monat]
ch.oddb> search_migel_products('21.05.02.03.4','de')
-> []
Running jobs/update_migel_products_with_report a second time finished without sending any e-mail and aborting silently. What happened?
I don't understand why the update_migel_products_with_report needs to call a download_all of all regdata articles before iterating over all migel products. Also, why has this to be called several time. Doing it once for each language should suffice.
Modified it to skip this step and iterate over the konw migel codes. Waiting for update_migel_products_with_report to finish (Takes about 4 hours). Did not seen lines like last years import
2 / 568 Estimate total: 2.10 [h] It will be done in: 2.09 [h] 2015-02-02 18:10:40 +0100: save_all_products migel_code 01.01.02.01.2 count 2
Also remarked that update_products_by_migel_code checks the (outdated) pharmacode. This is not good. Do we need a rewrite part of the Migel code?
Found the following error in import-daily
Plugin: ODDB::TextInfoPlugin Error: NotImplementedError Message: method `method_missing' called on terminated object (0x0000000820b2e8 flags=0x0 klass=0x0) Backtrace: /var/www/oddb.org/src/model/fachinfo.rb:91:in `localized_name' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing' /var/www/oddb.org/src/view/rss/fachinfo.rb:46:in `item_to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:89:in `block (2 levels) in to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:83:in `each' /var/www/oddb.org/src/view/rss/fachinfo.rb:83:in `block in to_html' /usr/local/lib/ruby/1.9.1/rss/maker/base.rb:438:in `make' /usr/local/lib/ruby/1.9.1/rss/maker/base.rb:402:in `make' /usr/local/lib/ruby/1.9.1/rss/maker.rb:9:in `make' /var/www/oddb.org/src/view/rss/fachinfo.rb:74:in `to_html' /var/www/oddb.org/src/plugin/plugin.rb:91:in `block (2 levels) in update_rss_feeds' /var/www/oddb.org/src/plugin/plugin.rb:90:in `open' /var/www/oddb.org/src/plugin/plugin.rb:90:in `block in update_rss_feeds' /var/www/oddb.org/src/plugin/plugin.rb:49:in `call' /var/www/oddb.org/src/plugin/plugin.rb:49:in `block in l10n_sessions' /var/www/oddb.org/src/plugin/plugin.rb:46:in `each_key' /var/www/oddb.org/src/plugin/plugin.rb:46:in `l10n_sessions' /var/www/oddb.org/src/plugin/plugin.rb:82:in `update_rss_feeds' /var/www/oddb.org/src/plugin/plugin.rb:77:in `block in update_yearly_fachinfo_feeds' /var/www/oddb.org/src/plugin/plugin.rb:76:in `each' /var/www/oddb.org/src/plugin/plugin.rb:76:in `update_yearly_fachinfo_feeds' /var/www/oddb.org/src/plugin/text_info.rb:121:in `postprocess' /var/www/oddb.org/src/plugin/text_info.rb:1490:in `import_swissmedicinfo' /var/www/oddb.org/src/util/updater.rb:573:in `block in update_notify_simple' /var/www/oddb.org/src/util/updater.rb:549:in `call' /var/www/oddb.org/src/util/updater.rb:549:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:567:in `update_notify_simple' /var/www/oddb.org/src/util/updater.rb:344:in `update_textinfo_swissmedicinfo' /var/www/oddb.org/src/util/updater.rb:198:in `run' jobs/import_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_daily:12:in `<module:Util>' jobs/import_daily:11:in `<module:ODDB>'
Pushed commit Corrected rescue. Show backtrace. Restarted import_daily on thinpower in screen import_daily.
Found the following (expected) lines in the screen
rss/fachinfo 68: rescue #<NotImplementedError: method `method_missing' called on unexpected T_NODE object (0x0000000a668cd0 flags=0x21c klass=0x0)> for rss/fachinfo 68: #<ODBA::Stub:738632360#31115107 @odba_class= @odba_container=745733800#21>: backtrace is: /var/www/oddb.org/src/model/fachinfo.rb:91:in `localized_name' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing' /var/www/oddb.org/src/view/rss/fachinfo.rb:46:in `item_to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:89:in `block (2 levels) in to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:83:in `each' /var/www/oddb.org/src/view/rss/fachinfo.rb:83:in `block in to_html' Util.log_and_deliver_mail to=["ngiger@ywesee.com", "zdavatz@ywesee.com"] subject ch.ODDB.org Report - Fach- und Patienteninfo Updates (swissmedicinfo.ch) - 01/2016 size 1497 with 0 attachments.
Pat-Info completed successfully.
Herr Sgier reported that the value '12', aka 'E', Kontraindiziert is never used. When adding support for EPHA http://dev.ywesee.com/Niklaus/20131126-integrate-epha-interaction they used 'E'. This was changed later, see http://dev.ywesee.com/Niklaus/20140331-add-e-option-to-oddb2xml where it changed to 'X'. The oddb2xml.xsd is correct and does not list a value of 12. But I forgot to remove it completely.
Pushed commits: