---
Since last thurday we see the following error when running import_daily
Plugin: ODDB::SwissmedicPlugin Error: NotImplementedError Message: method `method_missing' called on terminated object (0x00000089058af0 flags=0x0 klass=0x0) Backtrace: /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:456:in `block in odba_unsaved_neighbors' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:454:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:454:in `odba_unsaved_neighbors' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:394:in `block in odba_store_unsaved' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:392:in `each' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:392:in `odba_store_unsaved' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:380:in `odba_store' /var/www/oddb.org/src/util/persistence.rb:261:in `issue_update' /var/www/oddb.org/src/util/oddbapp.rb:131:in `block in update' /var/www/oddb.org/src/util/failsafe.rb:10:in `call' /var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe' /var/www/oddb.org/src/util/oddbapp.rb:129:in `update' /var/www/oddb.org/src/util/oddbapp.rb:1537:in `update' /var/www/oddb.org/src/plugin/swissmedic.rb:845:in `update_sequence' /var/www/oddb.org/src/plugin/swissmedic.rb:779:in `block in update_registrations' /var/www/oddb.org/src/plugin/swissmedic.rb:775:in `each' /var/www/oddb.org/src/plugin/swissmedic.rb:775:in `update_registrations' /var/www/oddb.org/src/plugin/swissmedic.rb:84:in `update' /var/www/oddb.org/src/util/updater.rb:394:in `block in update_swissmedic' /var/www/oddb.org/src/util/updater.rb:496:in `call' /var/www/oddb.org/src/util/updater.rb:496:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:392:in `update_swissmedic' /var/www/oddb.org/src/util/updater.rb:195:in `run' jobs/import_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_daily:12:in `<module:Util>' jobs/import_daily:11:in `<module:ODDB>' jobs/import_daily:10:in `<main>'
Found the following entries in log/oddb/debug/2014/09.log
2014-09-03 07:47:00 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 698: update_package iksnr '56091' cd 006 should correct seqnr 01 -> 03?
2014-09-03 07:47:02 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 846: res Risperdal Consta 25 mg, Suspension zur i.m. Applikation == Risperdal Consta 25 mg, Suspension zur i.m. Applikation? seqnr 01
2014-09-03 07:47:02 +0200: update_compositions: row[0] 56092 iksnr 56092 01 seq Risperdal Consta 25 mg, Suspension zur i.m. Applikation opts {:create_only=>false, :date=>#<Date: 2014-09-03 ((2456904j,0s,0n),+0s,2299161j)>, :composition=>2, :label=>"III"} cell_content Praeparatio sicca: risperidonum 25 mg, copoly(dl-lactidum-glycolidum).
Solvens: carmellosum natricum, polysorbatum 20, dinatrii phosphas dihydricus, acidum citricum anhydricum, natrii chloridum, aqua ad iniectabilia q.s. ad solutionem pro 2 ml, pro vase, in suspensione recenter reconstituta.
2014-09-03 07:47:02 CEST Util.log_and_deliver_mail to=["ngiger@ywesee.com", "zdavatz@ywesee.com"] subject ch.ODDB.org Report - Error: swissmedic - 09/2014 size Plugin: ODDB::SwissmedicPlugin
<.. followed by stacktrace as above..>
Looking at IKSNR 56092 via bin admin
package('56092').class
-> NilClass
On ch.oddb.org http://ch.oddb.org/de/gcc/search/zone/drugs/search_query/56092/search_type/st_registration#best_result now shows the registration for 56092 without any sequences. oddb-ci2 (where the import-daily did not yet run) we see 4 sequences 01, 02, 03, 04. Looking at Packungen-2014.09.07.xlsx we find 3 sequences (01, 02, 03). Same for Packungen-2014.09.01.xlsx. Same for Packungen-2013.08.07.xlsx. Datum des Datenimports : 07.09.2014
Why do we find therefore the sequence 04 in 50692 as "04 Risperdal Consta 12.5 mg"
http://ch.oddb.org/de/gcc/fachinfo/reg/56092/chapter/packages says
Packungen Risperdal Consta Injektionskit mit Durchstechflasche 25 mg und Fertigspritze mit Lösungsmittel. [B] Risperdal Consta Injektionskit mit Durchstechflasche 37.5 mg und Fertigspritze mit Lösungsmittel. [B] Risperdal Consta Injektionskit mit Durchstechflasche 50 mg und Fertigspritze mit Lösungsmittel. [B] Jede Packung enthält 1 SmartSite® Vorrichtung zur Rekonstitution und 2 Needle-Pro®-Nadel zur i.m. Injektion (eine 21G UTW 1-Inch Sicherheits-Injektionsnadel mit Schutzhülle zur deltoiden Injektion und eine 20G TW 2-Inch Nadel mit Schutzhülle zur glutealen Injektion).
which is the same info as for oddb-ci2. Which says also "Datum des Datenimports : 05.11.2013"
Tried the swissmedic-update to delete all four sequences instead of just one? Adding more debug-messaged to update_sequence/registration in src/plug/swissmedic.rb. Running import_daily. Got a different error
Plugin: ODDB::SwissmedicPlugin Error: NoMethodError Message: undefined method `export_flag' for ["30603842"]:Array Backtrace: /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing' /var/www/oddb.org/src/plugin/swissmedic.rb:144:in `block (2 levels) in set_all_export_flag_false' /var/www/oddb.org/src/plugin/swissmedic.rb:142:in `each' /var/www/oddb.org/src/plugin/swissmedic.rb:142:in `block in set_all_export_flag_false' /var/www/oddb.org/src/util/oddbapp.rb:816:in `block in each_registration' /var/www/oddb.org/src/util/oddbapp.rb:815:in `each' /var/www/oddb.org/src/util/oddbapp.rb:815:in `each_registration' /var/www/oddb.org/src/plugin/swissmedic.rb:137:in `set_all_export_flag_false' /var/www/oddb.org/src/plugin/swissmedic.rb:85:in `update' /var/www/oddb.org/src/util/updater.rb:394:in `block in update_swissmedic' /var/www/oddb.org/src/util/updater.rb:496:in `call' /var/www/oddb.org/src/util/updater.rb:496:in `wrap_update' /var/www/oddb.org/src/util/updater.rb:392:in `update_swissmedic' /var/www/oddb.org/src/util/updater.rb:195:in `run' jobs/import_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `call' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_daily:12:in `<module:Util>' jobs/import_daily:11:in `<module:ODDB>' jobs/import_daily:10:in `<main>'
Added some debug-messages and workaround for this error. It is getting worse and worse. After adding some debug-statements I get in the output of sudo -u apache /usr/local/bin/ruby jobs/import_daily
<..> Util.log_and_deliver_mail to=["ngiger@ywesee.com"] subject ch.ODDB.org Report - Fach- und Patienteninfo Updates (swissmedicinfo.ch) - 09/2014 size Searched for <..> 0 wrong metatags: ruby: string.c:680: rb_str_new_frozen: Assertion `(!!((!(((VALUE)((str)) & RUBY_IMMEDIATE_MASK) || !(((VALUE)((str)) & ~((VALUE)RUBY_Qnil)) != 0)) && (int)(((struct RBasic*)((str)))->flags & RUBY_T_MASK) != RUBY_T_NODE)?(((struct RBasic*)((str)))->flags&((((VALUE)1)<<11))):0))' failed.
without any stack-trace!
I will try upgrading from ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux] to the newest ruby 1.9.3 (download http://ftp.ruby-lang.org/pub/ruby/ruby-1.9.3-p547.tar.gz). Running
tar -zxf ruby-1.9.3-p547.tar.gz cd ruby-1.9.3-p547 ./configure --prefix=/usr/local/ruby-1.9.3-p547 make -j 8 sudo make install cd /usr/local/bin sudo rm erb gem irb rake rdoc ri ruby testrb sudo ln -s /usr/local/ruby-1.9.3-p547/bin/* . sudo /usr/local/bin/gem install bundler sudo /usr/local/bin/gem content bundler | tail -n 1 # /usr/local/ruby-1.9.3-p547/lib/ruby/gems/1.9.1/gems/bundler-1.7.2/spec/update/path_spec.rb
Changed install_gems.sh to use /usr/local/bin/gem.
Discussed with Zeno. We arrived at the conclusion that I should use ruby 1.9.3p0 again and that the changes are probably due to my fixed in the plugin/swissmedic due to the use of xlsx instead of xls. Also Zeno remarked that Hannes had the strategy to never delete stuff. When looking at http://dev.ywesee.com/Choddb/Swissmedic I found the following remarks Packages are deleted. Registrations and Sequences are deactivated. and Packages are only deleted if the Registration is still active.. When comparing src/plugin/swissmedic.rb to the commit b64488287dd158b62d5032b2ee1ca8db74b6a7f1 (before I enter the scene) I see that before the procedure delete called only @app.deletet ptr whereas now it deletes also explicitely all packages of the same registration/sequence. Adding debug message here to see whether my changes not working as expected.
I also see that the method update_sequence now calls update_sequence
It looks like Yasu did run into the same problem see http://dev.ywesee.com/Yasu/20130408-debug-odba-terminated-objects which he solved by adding GC.disable/enable around some critical code in the corresponding src/model/. See commit http://scm.ywesee.com/?p=oddb.org/.git;a=commitdiff;h=63995505938b38ebf8ff5b719624c56a30949dd8;hp=8945437f9dcee478dd09bcdbc1a9fad9c43ed4c9
Now the import_swissmedic finished without problems. Maybe my debug messages changed the behaviour of GC sufficiently to avoid the trigger. Will remove all newly introduced debug messages and add just a GC.disable/enable when updating one row of the XLS file. Will drop and reimport db and reset Packungen-latest.xls before rerunning a new import.
Okay. Getting mail for swissmedic and followers like lppv. Waiting for import to finish before pushing to thinpower. Now import finished with an error because I had no recipient defined for the medwin list. Pushed commit Disable GC when updating a registration to avoid accessing a terminated object. Pulling commit on thinpower and starting import_swissmedic.
Since some time I miss a real integration test. I would like to have one to answer the following questions
To produce repeatable result (in a timely matter) the following steps could be helpful
Estimated effort to construct such an integration test is between 2 and 5 days.
Creating a script test/integration/gen_aips_xml.rb which copies all IKSNRS defined via the constant IKSNRS_TO_EXTRACT in test/integration/common.rb to test/integration/data/AipsDownload_latest.xml.
Also I will need a script test/integration/prepare_integration_test.rb and test/integration/restore_after_integration_test.rb to move the production data to a safe place and restoring them. Will need to preserve the following directories under /var/www/oddb.org, e.g. mv etc etc.production, log -> log.production and data -> data.production.
Pushed preliminary work with commit Added some data/scripts for test/integration and Updated scripts for test-integration