<< | Index | >>
---
Looking at http://ch.oddb.org/de/gcc/search/search_query/A12AX I get the following error
SBSM::InvalidDataError e_invalid_ean_code
Found while looking around on ch.oddb.org. Why there is no entry in /var/log/apache2/access_log nor /var/log/apache2/error_log for this access?
Made some notes about my ideas on how to migrate to Ruby 2.0. See Ruby20
The import-daily of yesterday got an out of memory problem, while update_registration '57435' seqnr '01' ikscd '002'. What is remarkable, that treating this entry consumers gigabytes of memory, which I cannot explain.
Stack-Trace is
/var/www/oddb.org/src/model/text.rb:394:in `block in wrap': failed to allocate memory (NoMemoryError)
from /var/www/oddb.org/src/model/text.rb:376:in `each'
from /var/www/oddb.org/src/model/text.rb:376:in `wrap'
from /var/www/oddb.org/src/model/text.rb:351:in `block (2 levels) in to_s'
from /var/www/oddb.org/src/model/text.rb:350:in `collect'
from /var/www/oddb.org/src/model/text.rb:350:in `block in to_s'
from /var/www/oddb.org/src/model/text.rb:345:in `collect'
from /var/www/oddb.org/src/model/text.rb:345:in `to_s'
from /var/www/oddb.org/src/model/text.rb:430:in `block in to_s'
from /var/www/oddb.org/src/model/text.rb:430:in `collect'
from /var/www/oddb.org/src/model/text.rb:430:in `to_s'
from /var/www/oddb.org/src/model/text.rb:489:in `block in to_s'
from /var/www/oddb.org/src/model/text.rb:489:in `collect'
from /var/www/oddb.org/src/model/text.rb:489:in `to_s'
from (eval):2:in `to_s'
from /var/www/oddb.org/src/model/fachinfo.rb:116:in `unwanted_effect_text'
from (eval):3:in `block in proc_resolve_search_term'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:146:in `call'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:146:in `search_term'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:149:in `search_terms'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:206:in `block in update_target'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:204:in `each'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:204:in `update_target'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:161:in `update'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:590:in `block in update_indices'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:589:in `each'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:589:in `update_indices'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:515:in `store'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:274:in `odba_isolated_store'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:395:in `block in odba_store_unsaved'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:392:in `each'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:392:in `odba_store_unsaved'
from /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:380:in `odba_store'
from /var/www/oddb.org/src/util/persistence.rb:261:in `issue_update'
from /var/www/oddb.org/src/util/oddbapp.rb:127:in `block in update'
from /var/www/oddb.org/src/util/failsafe.rb:10:in `call'
from /var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe'
from /var/www/oddb.org/src/util/oddbapp.rb:126:in `update'
from /var/www/oddb.org/src/util/oddbapp.rb:1532:in `update'
from /var/www/oddb.org/src/plugin/swissmedic.rb:879:in `update_sequence'
from /var/www/oddb.org/src/plugin/swissmedic.rb:820:in `block in update_registrations'
from /var/www/oddb.org/src/plugin/swissmedic.rb:814:in `each'
from /var/www/oddb.org/src/plugin/swissmedic.rb:814:in `update_registrations'
from /var/www/oddb.org/src/plugin/swissmedic.rb:62:in `update'
from /var/www/oddb.org/src/util/updater.rb:387:in `block in update_swissmedic'
from /var/www/oddb.org/src/util/updater.rb:488:in `call'
from /var/www/oddb.org/src/util/updater.rb:488:in `wrap_update'
from /var/www/oddb.org/src/util/updater.rb:385:in `update_swissmedic'
from /var/www/oddb.org/src/util/updater.rb:195:in `run'
from /var/www/oddb.org/jobs/import_daily:13:in `block in <module:Util>'
from /var/www/oddb.org/src/util/job.rb:40:in `call'
from /var/www/oddb.org/src/util/job.rb:40:in `run'
from /var/www/oddb.org/jobs/import_daily:12:in `<module:Util>'
from /var/www/oddb.org/jobs/import_daily:11:in `<module:ODDB>'
from /var/www/oddb.org/jobs/import_daily:10:in `<main>'
Other important info from my logs are
139:/var/www/oddb.org/src/plugin/swissmedic.rb: 56 2013-10-21 23:26:27 +0200: @diff.news has 193 entries 140:/var/www/oddb.org/src/plugin/swissmedic.rb: 57 2013-10-21 23:26:27 +0200: @diff.updates has 17419 entries 141:/var/www/oddb.org/src/plugin/swissmedic.rb: 58 2013-10-21 23:26:27 +0200: @diff.replacements has 38 entries 1511:/var/www/oddb.org/src/plugin/swissmedic.rb: 61 2013-10-21 23:31:21 +0200: update_registrations news done 91581:/var/www/oddb.org/src/plugin/swissmedic.rb: 816 2013-10-22 03:47:51 +0200: update_registrations iksnr '57435'
Completed about 11000 of the 17419 updates. Why do we have so many updates? Will examine the output of the swissmedic-diff another time.
Compared using a csv-version of the xls I get for the old
old: 57435,1,"Baraclude 0.5 mg, Filmtabletten",Bristol-Myers Squibb SA,08.03.0.,J05AF10,Synthetika human,10.08.2006,10.08.2006,09.08.2016,002,30,Tablette(n),A,entecavirum,"entecavirum 0.5 mg, excipients per coated tablet.",Chronische Hepatitis B, new: 57435,1,"Baraclude 0.5 mg, Filmtabletten",Bristol-Myers Squibb SA,08.03.0.,J05AF10,Synthetika human,10.08.2006,10.08.2006,09.08.2016,002,30,Tablette(n),A,entecavirum,"entecavirum 0.5 mg, excipiens pro compresso obducto.",Chronische Hepatitis B
The only change comes in row P (Zusammensetzung/Compositions).
I cannot explain the problem. Running the patched import which udates only 57435, as we might have a inconsisten db state. At 9:23 the memory starts going up starting from 1,4 GB, now at 1,9 GB. Killed the job at 2,5 GB to avoid loosing more time.
Called delete_registration('57435') in bin/admin and restarted importing 57435 again. This does not fix the problem. I go into a loop again. Waiting for memory exhaust to see the stack dump.
Updated the unit-test for test/test_plugin/text_info_swissmedicinfo.rb to include also the fachinfo for 57435 Baraclude. Must fix the test to really parse the fachinfo.
Looking at the XML one sees that swissmedic says <substances>Entecavir</substances> whereas in Packungen.xls it is called entecavirum. It looks like swissmedic will soon invent a new branch of mathematics where 1 + 1 != 2.
text_info_swissmedicinfo.rb now correctly parses 57435. Therefore I believe that we have a problem with odba/persistence updating.
Uncommented line 124 of oddbapp.rb to add debugging info for differences. Starting jobs/import_swissmedicinfo. Got the same error but no debug-info. Added $stdout.flush and restarted again.
The Baraclude.yaml seems to have a correct list of Hilfststoffe. E.g.
formats:
- !ruby/object:ODDB::Text::Format
end: 10
start: 0
values:
- :italic
- *id002
preformatted: false
raw_txt:
text: Hilfsstoffe
- !ruby/object:ODDB::Text::Paragraph
format: &id003 !ruby/object:ODDB::Text::Format
end: -1
start: 0
values: []
formats:
- *id003
preformatted: false
raw_txt:
text: "Filmtabletten: Lactosum monohydricum, Cellulosum microcristallinum, Crospovidonum, Povidonum, Magnesii stearas, Hypromellosum, Macrogolum, Polysorbatum (nur 0,5 mg Tabl.), Color.: Titanii dioxidum (E171), F
- !ruby/object:ODDB::Text::Paragraph
format: &id004 !ruby/object:ODDB::Text::Format
end: -1
start: 0
values: []
formats:
- *id004
preformatted: false
raw_txt:
I will compare it with the yaml on thinpower. The file Baraclude__swissmedicinfo.yaml was also recreated today. The yaml looks okay too. Therefore I suppose that it that the current version in the database might have a very long string (Filmtabletten:Lactosummonohydricum,Cellulosummicrocristallinum,Crospovidonum,Povidonum,Magnesiistearas,Hypromellosum,Macrogolum,Polysorbatum(nur0,5mgTabl.),Color.:Titaniidioxidum(E171),Ferrioxidumrubr.(E172,nur1 mgTabl.) which triggers a bizarre behaviours.
Adding a debug output to see which index triggers the problem and restarting the import. Added a line 590 $stdout.puts "#{__FILE__}:#{__LINE__} update_indices name #{index_name}" in /usr/local/lib64/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb. Once I know the name I will rebuild the index via bin.admin.
Trying to delete FI of Baraclude 57436 via
ch.oddb> reg = @system.registration('57436'); delete reg.fachinfo.pointer; update reg.pointer, :fachinfo => nil
->
This gives me in the bin/oddbd console output
ODBA::Stub was unable to replace #27689434 from Array:#28248681
ODBA::Stub was unable to replace #27690642 from Array:#28248856
ODBA::Stub was unable to replace #23730618 from Array:#28249697
ODBA::Stub was unable to replace #21270294 from Array:#28250213
ODBA::Stub was unable to replace #25760331 from Array:#28250870
ODBA::Stub was unable to replace #1848087 from Array:#28250951
ODBA::Stub was unable to replace #25767464 from Array:#28251506
ODBA::Stub was unable to replace #27691917 from Array:#28263615
ODBA::Stub was unable to replace #27691960 from Array:#28263768
ODBA::Stub was unable to replace #27322557 from Array:#28263870
ODBA::Stub was unable to replace #26888944 from Array:#28263957
/var/www/oddb.org/src/util/oddbapp.rb:1534:update(:!registration,57436., {:fachinfo=>nil})
/var/www/oddb.org/src/util/oddbapp.rb:1535:orign:update(:!registration,57436., )
/var/www/oddb.org/src/util/oddbapp.rb:124:update(:!registration,57436., {:fachinfo=>nil})
Running jobs/import_swissmedic (only for 57435). Now the import continues. It passed all registrations, I will restart it importing all the differences and with less verbose output. This did not take long as I forgot to remove the Packungen-latest.xls and therefore nothing was done.
Re-Importing FI for 37435 did not complete because of out of memory error. Seen in the output of bin/oddbd
ODBA::Stub was unable to replace ODDB::SimpleLanguage::Descriptions#30484132 from ODDB::Fachinfo:#30484130 ODBA::Stub was unable to replace ODDB::SimpleLanguage::Descriptions#30484132 from ODDB::Fachinfo:#30484130
The following index could not be updated unwanted_effects_index_de. Calling jobs/rebuild_indices unwanted_effects_index_de and rerunning the import again.
Updated http://dev.ywesee.com/Choddb/Index with link to http://dev.ywesee.com/Main/CHoddbdeleteFI