view · edit · sidebar · attach · print · history

Index>

20140909-fix-another-unimplemented-method-in-import-daily

Summary

  • Fix another error NotImplementedError: method `method_missing' called on terminated object in import_daily

Commits

Index

Keep in Mind
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • One unit-test for searchbar fails and might be a clue why searching does not work correctly.
  • Added two skip in test/test_plugin/rss.rb. Why does the mocking not work there anymore?

---

Icon for rezept is missing in just-medical when search via trademark

Added just-medical.oddb.org to 127.0.0.1 in /etc/hosts to test just-medical locally. When calling http://just-medical.oddb.org/de/just-medical/search/zone/drugs/search_query/Aspirin/search_type/st_sequence#best_result it does not display this correctly.

Now the question is whether/how we could adapt the watir tests to be able to check this just-medical, too. Problem is that many tests would fail as just-medical offers only a subset of the feature of ch.oddb.org. Can I find a generic solution to keep e.g in the spec_helpers.ch a Hash which allows me to disable failing tests.

Setting env variable ODDB_CH to http://just-medical.oddb.org and running spec/rezept_and_instantsearch_spec.rb. Did see that searching used oddb-ci2 instead of just-medical. Added a check in add_one_drug_to_rezept to assert that the URL is correct. Now the 16 of the 17 tests fail early.$

When running under just-medial the PrivateTemplate is used.

Okay. I looked at the wrong place, because I thought this was an error induced by my changes during the last weeks. But it was just a matter of adapting the look_and_feel with commit Add prescription to result_list of just-medical. Now the search looks okay.

import_swissmedic does not deliver a mail

When looking at the output of the screen I see 

There is nil data in the patents.xls file. Most probably "Bezeichnung" is missing.
Util.send_mail rescue: error is #<RuntimeError: At least one recipient must be a list ["kgood@weleda.ch"]> recipients nil ["/var/www/oddb.org/src/util/mail.rb:91:in `send_mail'", "/var/www/oddb.org/src/util/log.rb:51:in `notify'", "/var/www/oddb.org/src/plugin/swissmedic.rb:365:in `block in mail_notifications'", "/var/www/oddb.org/src/plugin/swissmedic.rb:351:in `each'", "/var/www/oddb.org/src/plugin/swissmedic.rb:351:in `mail_notifications'", "/var/www/oddb.org/src/util/exporter.rb:427:in `mail_swissmedic_notifications'", "/var/www/oddb.org/src/util/updater.rb:413:in `update_swissmedic_followers'", "jobs/import_swissmedic:15:in `block in <module:Util>'", "/var/www/oddb.org/src/util/job.rb:40:in `call'", "/var/www/oddb.org/src/util/job.rb:40:in `run'", "jobs/import_swissmedic:12:in `<module:Util>'"]
/var/www/oddb.org/log/job.pid is deleted
/var/www/oddb.org/src/util/mail.rb:155:in `check_and_get_all_recipients': At least one recipient must be a list ["kgood@weleda.ch"] (RuntimeError)
        from /var/www/oddb.org/src/util/mail.rb:92:in `send_mail'
        from /var/www/oddb.org/src/util/log.rb:51:in `notify'
        from /var/www/oddb.org/src/plugin/swissmedic.rb:365:in `block in mail_notifications'
        from /var/www/oddb.org/src/plugin/swissmedic.rb:351:in `each'
        from /var/www/oddb.org/src/plugin/swissmedic.rb:351:in `mail_notifications'
        from /var/www/oddb.org/src/util/exporter.rb:427:in `mail_swissmedic_notifications'
        from /var/www/oddb.org/src/util/updater.rb:413:in `update_swissmedic_followers'
        from jobs/import_swissmedic:15:in `block in <module:Util>'
        from /var/www/oddb.org/src/util/job.rb:40:in `call'
        from /var/www/oddb.org/src/util/job.rb:40:in `run'
        from jobs/import_swissmedic:12:in `<module:Util>'
        from jobs/import_swissmedic:11:in `<module:ODDB>'
        from jobs/import_swissmedic:10:in `<main>'

Okay. Mus fix that too. Added a list swissmedic in etc/oddb.yml, added Zeno and myself as receivers and a new receiver 

 - :email: kgood@weleda.ch
    :lists:
      - swissmedic

Next error is 

Util.log_and_deliver_mail to=["ngiger@ywesee.com"] subject ch.ODDB.org Report - Error: SL-Update (XML) - 09/2014 size Plugin: ODDB::BsvXmlPlugin
Error: NoMethodError
Message: undefined method `index' for nil:NilClass
Backtrace:
/var/www/oddb.org/src/util/mail.rb:50:in `block (2 levels) in get_mailing_list_anrede'
/var/www/oddb.org/src/util/mail.rb:50:in `each'
/var/www/oddb.org/src/util/mail.rb:50:in `block in get_mailing_list_anrede'
/var/www/oddb.org/src/util/mail.rb:49:in `each'
/var/www/oddb.org/src/util/mail.rb:49:in `get_mailing_list_anrede'
/var/www/oddb.org/src/plugin/bsv_xml.rb:913:in `report_bsv'
/var/www/oddb.org/src/plugin/bsv_xml.rb:833:in `log_info_bsv'
/var/www/oddb.org/src/util/updater.rb:144:in `log_info'
/var/www/oddb.org/src/util/updater.rb:473:in `log_notify_bsv'
/var/www/oddb.org/src/util/updater.rb:251:in `block in update_bsv'
/var/www/oddb.org/src/util/updater.rb:496:in `call'
/var/www/oddb.org/src/util/updater.rb:496:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:244:in `update_bsv'
/var/www/oddb.org/src/util/updater.rb:198:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

Fixed this problem and I received the following email 

Sehr geehrter Herr Giger

Am 09.09.2014 haben wir Ihren aktuellen SL-Export (XML)
wieder überprüft. Dabei ist uns folgendes aufgefallen:

1. Bei 44 Produkten hat die SL einen anderen 5-Stelligen Swissmedic-Code als
Swissmedic Excel.

2. Bei 0 Produkten hat die SL keinen 8-Stelligen Swissmedic-Code. Ev.
befinden sich dort auch Produkte darunter, welche nicht bei der Swissmedic
registriert werden müssen. Es hat aber sicherlich auch Produkte darunter,
welche einen Swissmedic-Code haben sollten.

3. Bei 0 Produkten hat die SL keinen 8-Stelligen Swissmedic-Code, die
Produkte sind laut RefData ausser Handel. Der Swissmedic Code sollte in der SL
gemäss SR 830.1, Art. 24, Abs. 1 trotzdem korrekt vorhanden sein. Die
Krankenkasse muss bis 5 Jahre in der Vergangenheit abrechnen können.

4. Bei 32 Produkten gibt es im Swissmedic-Excel keine Zeile mit diesem
8-stelligen Swissmedic-Code, die Packung kommt aber in Medwin vor.

5. Bei 5 Produkten gibt es im Swissmedic-Excel keine Zeile mit diesem
5-stelligen Swissmedic-Code. Die Produkte sind aber sicherlich bei der
Swissmedic registriert.

6. Bei 148 Produkten gibt es im Swissmedic-Excel keine Zeile mit diesem
8-stelligen Swissmedic-Code. Die Produkte sind wohl ausser Handel aber sicher
noch bei der Swissmedic registriert. Der Swissmedic Code sollte in der SL
gemäss SR 830.1, Art. 24, Abs. 1 trotzdem korrekt vorhanden sein.

7. Bei 366 Produkten fehlt der Pharmacode. Hier stellen wir uns die Frage,
weshalb bei diesen Produkten der Pharmacode fehlt. Eigentlich dürften keine
Pharmacodes fehlen, denn 99% Prozent aller Apotheken, Spitäler, Heime etc.
rechnen alle mit dem Pharmacode ab.

8. 0 5-stellige Swissmedic-Nummern kommen im BAG-XML-Export doppelt vor.
Siehe auch Attachment: Duplicate_Registrations_in_SL_09.09.2014.txt

Um die obigen Beobachtungen kontrollieren zu können, speichern Sie bitte die
Attachments auf Ihrem Schreibtisch.

Sie können die Attachments mit dem Windows-Editor öffnen. Sie finden den
Windows-Editor unter: Startmenu > Programme > Editor

Starten Sie den Editor und gehen Sie dann auf: Datei > Öffnen

Wählen sie obige Attachments von Ihrem Schreibtisch aus und schon können Sie
die Attachments anschauen.

Danke für Ihr Feedback.

Mit freundlichen Grüssen
Zeno Davatz
+41 43 540 05 50


Attachments:


SMeX/SL-Differences (Registrations) 09.09.2014               44
Missing Swissmedic-Codes in SL 09.09.2014                     0
Missing Swissmedic-Codes in SL (out of trade) 09.09.2014      0
Unknown Packages in SL 09.09.2014                            32
Unknown Registrations in SL 09.09.2014                        5
Unknown Packages in SL (out of trade) 09.09.2014            148
Missing Pharmacodes in SL 09.09.2014                        366
Duplicate Registrations in SL 09.09.2014                      0

Pushed commits Avoid nil error when no anrede given and Added list swissmedic as mail receivers to swissmedic plugin

Fix error NotImplementedError: method `method_missing' called on terminated object in import_daily

Got another error today when running import_daily 

Plugin: ODDB::BsvXmlPlugin
Error: NotImplementedError
Message: method `method_missing' called on terminated object (0x00000101c0d210 flags=0x0 klass=0x0)
Backtrace:
/var/www/oddb.org/src/model/registration.rb:234:in `block in packages'
/var/www/oddb.org/src/model/registration.rb:233:in `each'
/var/www/oddb.org/src/model/registration.rb:233:in `inject'
/var/www/oddb.org/src/model/registration.rb:233:in `packages'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
/var/www/oddb.org/src/plugin/bsv_xml.rb:462:in `tag_end'
/usr/local/lib/ruby/1.9.1/rexml/parsers/streamparser.rb:26:in `parse'
/usr/local/lib/ruby/1.9.1/rexml/document.rb:205:in `parse_stream'
/var/www/oddb.org/src/plugin/bsv_xml.rb:1005:in `update_preparations'
/var/www/oddb.org/src/plugin/bsv_xml.rb:660:in `block (2 levels) in _update'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry.rb:500:in `get_input_stream'
/var/www/oddb.org/src/plugin/bsv_xml.rb:660:in `block in _update'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `call'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `block in each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/central_directory.rb:182:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:139:in `block in foreach'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:99:in `open'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:138:in `foreach'
/var/www/oddb.org/src/plugin/bsv_xml.rb:655:in `_update'
/var/www/oddb.org/src/plugin/bsv_xml.rb:650:in `update'
/var/www/oddb.org/src/util/updater.rb:246:in `block in update_bsv'
/var/www/oddb.org/src/util/updater.rb:496:in `call'
/var/www/oddb.org/src/util/updater.rb:496:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:244:in `update_bsv'
/var/www/oddb.org/src/util/updater.rb:198:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

It looks for me that Ruby 1.9.3-p0 has another problem handling some cases. We will try to fix it using GC.disable/enable like yesterday.

But first I want to consider the following questions:

  • Are nested calls go GC.disable/enable possible? See http://www.ruby-doc.org/core-1.9.3/GC.html. GC.disable returns whether it was already disabled. Therefore I think we should have a look at all occurrences of GC.enable and call it only where it was disabled before.
  • Should we add them in the model or in the plugin bsv_xml? When comparing the errors (today, yesterday and Yasu) we see that they occured always when accessing data from a sequence.

Added support for nested GC.disable/enable everywhere we used it. Fixed problem with email-receiver of swissmedic. Started import_daily on oddb_ci2. Import finished without problems after 11 minutes, but it did not generate a new XMLPublications-latest.zip as it was the same as yesterday's. Changing XMLPublications-latest.zip to a previous version. Also rm swissmedicinfo.zip AipsDownload_latest.xml. Now starting import_daily again.

Found the following error (4 times) in the output of import_daily 

0 is not a symbol
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:122:in `respond_to?'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:122:in `respond_to?'
/var/www/oddb.org/src/util/persistence.rb:284:in `block in resolve'
/var/www/oddb.org/src/util/persistence.rb:278:in `each'
/var/www/oddb.org/src/util/persistence.rb:278:in `resolve'
/var/www/oddb.org/src/util/persistence.rb:208:in `issue_create'
/var/www/oddb.org/src/util/oddbapp.rb:110:in `block in create'
/var/www/oddb.org/src/util/failsafe.rb:10:in `call'
/var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe'
/var/www/oddb.org/src/util/oddbapp.rb:109:in `create'
/var/www/oddb.org/src/util/persistence.rb:288:in `block in resolve'
/var/www/oddb.org/src/util/persistence.rb:278:in `each'
/var/www/oddb.org/src/util/persistence.rb:278:in `resolve'
/var/www/oddb.org/src/util/persistence.rb:256:in `issue_update'
/var/www/oddb.org/src/util/oddbapp.rb:131:in `block in update'
/var/www/oddb.org/src/util/failsafe.rb:10:in `call'
/var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe'
/var/www/oddb.org/src/util/oddbapp.rb:129:in `update'
/var/www/oddb.org/src/util/oddbapp.rb:1537:in `update'
/var/www/oddb.org/src/plugin/bsv_xml.rb:427:in `block in tag_end'
/var/www/oddb.org/src/plugin/bsv_xml.rb:400:in `each'
/var/www/oddb.org/src/plugin/bsv_xml.rb:400:in `tag_end'
/usr/local/lib/ruby/1.9.1/rexml/parsers/streamparser.rb:26:in `parse'
/usr/local/lib/ruby/1.9.1/rexml/document.rb:205:in `parse_stream'
/var/www/oddb.org/src/plugin/bsv_xml.rb:1008:in `update_preparations'
/var/www/oddb.org/src/plugin/bsv_xml.rb:663:in `block (2 levels) in _update'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry.rb:500:in `get_input_stream'
/var/www/oddb.org/src/plugin/bsv_xml.rb:663:in `block in _update'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `call'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:42:in `block in each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:41:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/central_directory.rb:182:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:139:in `block in foreach'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:99:in `open'
/usr/local/lib/ruby/gems/1.9.1/gems/rubyzip-1.1.6/lib/zip/file.rb:138:in `foreach'
/var/www/oddb.org/src/plugin/bsv_xml.rb:658:in `_update'
/var/www/oddb.org/src/plugin/bsv_xml.rb:653:in `update'
/var/www/oddb.org/src/util/updater.rb:246:in `block in update_bsv'
/var/www/oddb.org/src/util/updater.rb:496:in `call'
/var/www/oddb.org/src/util/updater.rb:496:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:244:in `update_bsv'
/var/www/oddb.org/src/util/updater.rb:198:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

The errors lies in the method tag_end where I added a GC.disable. Updating a limitation text failed. Added a debug output, getting db_dump from thinpower of yesterday, loading it, resetting latest for swissmedic and restarting import_daily.

My log-entry was not good enough and I still do not see which registrations created the problem. But as the GC problem (and also the problem with the mail receiver) are solved I will push the commits as soon as the import finishes and will try to fix the problem with updates in a separate commit. Pushed commit Allow nested calls to GC.disable

Pulled changes, removed latest and started import_swissmedic on thinpower.

Doing some cleanup work with commits

view · edit · sidebar · attach · print · history
Page last modified on September 09, 2014, at 05:11 PM