< Niklaus.20220309-refdata-update | Index | Niklaus.20220121-ruby-31 >>
oddb.org:
sbsm:
Pushed a few commits to cleanup warnings, unit tests errors. Removed unused dependencies to watir and pry. Remarked that the ruby version 3.0 did not have a quote, which whould have meant it to equal to highest available 3.x version of ruby.
Lost an hour by checking that on thin-power and a personal debian machine, there are NO test errors when using ruby 3.0.3 and ruby 3.1.0 even when github/actions give failures. Do not know why this happens.
Getting a current dump from thinpower via scp ywesee@ywesee.com:/var/backup/thinpower/db/postgresql/January-2022/2022-01-22/22:00-postgresql_database-ch_oddb-backup.bz2 /var/backup/thinpower/db_dumps/pg-db-ch_oddb-backup.bz2 and /usr/local/bin/pg_create_db.sh after having stopped all services.
Using ruby-3.1 without the new
Restarting the services yus, migel and ch-oddb*. Patching src/plugin/text_info.rb to break on the error reported last friday. Restarting import_daily.
Parsing FI 36784 RHINOCAP® Kapseln: leads to errors. Looks like methods text and changelog throw errors, where they didn't before. But it is also a FI which has not been updated since 2007. Running sudo -u apache bundle-310 exec ruby-310 jobs/update_textinfo_swissmedicinfo --target=fi --reparse 36784 and placing a binding.irb to catch the error.
Used the new gem debug (replaces pry-byebug) to debug the problem. But for both FI and PI I got errors when calling odba_store. Tried various approaches without getting any success.
Storing the PI 67083 Vyndaqel failed. When I look at it on oddb-ci2 via the web I get an NoMethodError 'first' for PatinfoDocument
Deleting FI and PI of these two registrations to see, whether this are just some old hidden errors in the database. Calling bin/admin with
@h.oddb> reg = @system.registration('36784'); delete reg.fachinfo.pointer; update reg.pointer, :fachinfo => nil
-> #<ODDB::Registration:0x00007f2ecd26ea00>
ch.oddb> reg = @system.registration('67083'); delete reg.sequences.values.first.patinfo.pointer; update reg.sequences.values.first.patinfo = nil
-> wrong number of arguments (given 1, expected 2..3)
ch.oddb> reg = @system.registration('67083'); reg.sequences.values.first.patinfo
->
@
Restarting import sudo -u apache bundle-310 exec ruby-310 jobs/update_textinfo_swissmedicinfo --target=both --reparse 67083 36784
Now I am getting the following error ((recurses till the process gets killed)
@OdbaError /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:47:in `odba_receiver' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:27:in `is_a?' /var/www/oddb.org/src/util/language.rb:31:in `description' /var/www/oddb.org/src/util/language.rb:54:in `method_missing' /var/www/oddb.org/src/util/language.rb:31:in `description' /var/www/oddb.org/src/util/language.rb:54:in `method_missing' /var/www/oddb.org/src/model/fachinfo.rb:79:in `localized_name' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:116:in `method_missing' /var/www/oddb.org/src/view/rss/fachinfo.rb:85:in `block (2 levels) in to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `each' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `block in to_html' ODBA::Stub was unable to replace ODDB::SimpleLanguage::Descriptions#30593148 from ODDB::Fachinfo:#30593146. raise OdbaError Descriptions.description error /var/www/oddb.org/src/util/language.rb:29:in `description' /var/www/oddb.org/src/util/language.rb:54:in `method_missing' /var/www/oddb.org/src/util/language.rb:31:in `description' /var/www/oddb.org/src/util/language.rb:54:in `method_missing' /var/www/oddb.org/src/model/fachinfo.rb:79:in `localized_name' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:116:in `method_missing' /var/www/oddb.org/src/view/rss/fachinfo.rb:85:in `block (2 levels) in to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `each' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `block in to_html' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:439:in `make' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:403:in `make' /usr/local/bin/ruby-310: line 3: 16019 Killed /usr/local/ruby-3.1.0/bin/ruby-310 $@
Looking at when the FI was parsed on thinpower I get
ywesee@thinpower /var/www/oddb.org $ find . -name "*36784*" | xargs ls -lrt find: "./main": Keine Berechtigung find: "./svc/oddb_google_crawler/log/supervise": Keine Berechtigung -rw-r--r-- 1 apache users 75054 14. Dez 2011 ./doc/resources/patinfo/36784_01_1182520392.08234.pdf -rw-r--r-- 1 apache users 67697 14. Dez 2011 ./doc/resources/patinfo/36784_01.pdf -rw-r--r-- 1 apache users 75126 14. Dez 2011 ./doc/resources/patinfo/36784_01_1187354064.59001.pdf -rw-r--r-- 1 apache apache 14487 25. Feb 2016 ./data/details/36784_fi_de.xml -rw-r--r-- 1 apache apache 26280 25. Feb 2016 ./data/details/36784_fi_fr.xml -rw-r--r-- 1 apache apache 11049 25. Feb 2016 ./data/details/36784_pi_de.xml -rw-r--r-- 1 apache apache 11801 25. Feb 2016 ./data/details/36784_pi_fr.xml -rw-r--r-- 1 apache apache 14276 25. Feb 2016 ./data/details/36784_pi_it.xml -rw-r--r-- 1 apache apache 12894 20. Jan 07:11 ./data/details/36784_pi_de_Rhinocap___Kapseln_problem.xml -rw-r--r-- 1 apache apache 13180 20. Jan 07:11 ./data/details/36784_pi_de_RHINOCAP__Kapseln_problem.xml -rw-r--r-- 1 apache apache 12698 20. Jan 07:11 ./data/details/36784_pi_fr_RHINOCAP__Capsules_problem.xml -rw-r--r-- 1 apache apache 14058 20. Jan 07:11 ./data/details/36784_pi_fr_Rhinocap__G_lules_problem.xml -rw-r--r-- 1 apache apache 42423 20. Jan 07:11 ./data/details/36784_pi_it_RHINOCAP__capsule_problem.xml -rw-r--r-- 1 apache apache 12574 20. Jan 07:11 ./data/details/36784_pi_it_Rhinocap___Capsule_problem.xml -rw-r--r-- 1 apache apache 16576 25. Jan 07:09 ./data/details/36784_fi_de_RHINOCAP__Kapseln_problem.xml -rw-r--r-- 1 apache apache 18601 25. Jan 07:09 ./data/details/36784_fi_de_Rhinocap___Kapseln_problem.xml -rw-r--r-- 1 apache apache 20558 25. Jan 07:09 ./data/details/36784_fi_fr_Rhinocap___G_lules_problem.xml -rw-r--r-- 1 apache apache 19219 25. Jan 07:09 ./data/details/36784_fi_fr_RHINOCAP__capsules_problem.xml -rw-r--r-- 1 apache apache 18787 25. Jan 07:09 ./data/details/36784_fi_it.xml -rw-r--r-- 1 apache apache 18787 25. Jan 07:36 ./data/details/36784_fi_it_Rhinocap___Capsule_problem.xml
And the PI for 67083
/var/www/oddb.org $ find . -name "*67083*" | xargs ls -lrt find: "./main": Keine Berechtigung find: "./svc/oddb_google_crawler/log/supervise": Keine Berechtigung -rw-r--r-- 1 apache apache 184997 8. Apr 2020 ./data/details/67083_fi_de.xml -rw-r--r-- 1 apache apache 187655 8. Apr 2020 ./data/details/67083_fi_fr.xml -rw-r--r-- 1 apache apache 170143 8. Apr 2020 ./data/details/67083_fi_it.xml -rw-r--r-- 1 apache apache 19261 8. Apr 2020 ./data/details/67083_pi_de.xml -rw-r--r-- 1 apache apache 21107 8. Apr 2020 ./data/details/67083_pi_fr.xml -rw-r--r-- 1 apache apache 18519 8. Apr 2020 ./data/details/67083_pi_it.xml -rw-r--r-- 1 apache apache 184997 8. Apr 2020 ./data/details/67083_fi_de_Vyndaqel___Weichkapseln_problem.xml -rw-r--r-- 1 apache apache 187655 8. Apr 2020 ./data/details/67083_fi_fr_Vyndaqel___Capsule_molle_problem.xml -rw-r--r-- 1 apache apache 170143 8. Apr 2020 ./data/details/67083_fi_it_Vyndaqel___Capsule_molli_problem.xml -rw-r--r-- 1 apache apache 19261 10. Apr 2020 ./data/details/67083_pi_de_Vyndaqel___Weichkapseln_problem.xml -rw-r--r-- 1 apache apache 21107 10. Apr 2020 ./data/details/67083_pi_fr_Vyndaqel___Capsules_molles_problem.xml -rw-r--r-- 1 apache apache 78448 10. Apr 2020 ./data/details/67083_pi_it_Vyndaqel___Capsule_molli_problem.xml -rw-r--r-- 1 apache apache 185121 20. Dez 07:08 ./data/details/67083_fi_de_Vyndaqel_problem.xml -rw-r--r-- 1 apache apache 187757 20. Dez 07:08 ./data/details/67083_fi_fr_Vyndaqel_problem.xml -rw-r--r-- 1 apache apache 170288 20. Dez 07:08 ./data/details/67083_fi_it_Vyndaqel_problem.xml
After adding a few odba_store in @src/util/textinfo.rb@ with
diff --git a/src/plugin/text_info.rb b/src/plugin/text_info.rb
index cc4efa8f..9c3b6b33 100644
--- a/src/plugin/text_info.rb
+++ b/src/plugin/text_info.rb
@@ -320,18 +320,24 @@ module ODDB
msg += ' new patinfo'
else
package.patinfo = @app.create_patinfo
+ package.patinfo.odba_store
package.patinfo.descriptions[lang] = patinfo_lang
package.patinfo.odba_store
msg += ' created patinfo'
end
+ package.sequence.odba_store
package.odba_store
unless package.patinfo.descriptions.values.first.is_a?(ODDB::PatinfoDocument)
msg = "class #{package.patinfo.descriptions.values.first.class} is not a PatinfoDocument"
raise msg
end
# Update patinfo of sequence
+ package.patinfo.odba_store
+ package.sequence.odba_store
package.sequence.patinfo = package.patinfo unless package.sequence.patinfo.object_id == package.patinfo.object_id
package.sequence.odba_store
+ package.patinfo.odba_store
+ package.odba_store
LogFile.debug "called odba_store #{msg}"
package.patinfo
end
Now the reparsing continues, but at the end I get the following recursion which results in a out of memory error
2022-02-22 10:52:13 +0100: /var/www/oddb.org/src/plugin/text_info.rb:341:in `store_package_patinfo': called odba_store 67518/01/001: fr Vyndaqel®, Capsules molles change_diff 2022-02-22 10:52:13 +0100: job is done. now postprocess works ... OdbaError /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:47:in `odba_receiver' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:27:in `is_a?' /var/www/oddb.org/src/util/language.rb:31:in `description' /var/www/oddb.org/src/util/language.rb:54:in `method_missing' /var/www/oddb.org/src/model/fachinfo.rb:79:in `localized_name' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:116:in `method_missing' /var/www/oddb.org/src/view/rss/fachinfo.rb:85:in `block (2 levels) in to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `each' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `block in to_html' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:439:in `make' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:403:in `make' ODBA::Stub was unable to replace ODDB::SimpleLanguage::Descriptions#30593148 from ODDB::Fachinfo:#30593146. raise OdbaError Descriptions.description error /var/www/oddb.org/src/util/language.rb:29:in `description' /var/www/oddb.org/src/util/language.rb:54:in `method_missing' /var/www/oddb.org/src/model/fachinfo.rb:79:in `localized_name' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/odba-1.1.8/lib/odba/stub.rb:116:in `method_missing' /var/www/oddb.org/src/view/rss/fachinfo.rb:85:in `block (2 levels) in to_html' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `each' /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `block in to_html' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:439:in `make' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:403:in `make' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker.rb:29:in `make' /var/www/oddb.org/src/view/rss/fachinfo.rb:74:in `to_html' OdbaError /var/www/odd <...> /var/www/oddb.org/src/view/rss/fachinfo.rb:84:in `block in to_html' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:439:in `make' /var/www/oddb.org/vendor/bundle/ruby/3.1.0/gems/rss-0.2.9/lib/rss/maker/base.rb:403:in `make' [FATAL] failed to allocate memory
The patch (zdavatz#152) Fix reparsing textinfo for FI contains the following improvements: