Must fix the error that we only stored the last changelog. The unit tests showed, that it worked without odba, but somehow we did not store correctly the ODBA entries.
When trying to save the change_log I get the following error
Error: IOError Message: closed stream Backtrace: /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/marshal.rb:10:in `internal_encoding' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/marshal.rb:10:in `dump' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/marshal.rb:10:in `dump' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:269:in `odba_isolated_dump' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/cache.rb:511:in `store' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:274:in `odba_isolated_store' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:395:in `block in odba_store_unsaved' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:392:in `each' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:392:in `odba_store_unsaved' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:380:in `odba_store' /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/stub.rb:113:in `method_missing' /var/www/oddb.org/src/plugin/text_info.rb:152:in `store_fachinfo'
I can reproduce this problem in bin/admin like this
ch.oddb> $fi = registration('65886').fachinfo.fr
-> #<ODDB::FachinfoDocument2001:0x007fd3173459f8>
ch.oddb> $item = ODDB::ChangeLog.add_change_log_item($fi, 'old_text', "newer_text #{Time.now}")
-> #<ODDB::FachinfoDocument::ChangeLogItem:0x00558791b76d30>
ch.oddb> $fi.odba_store
-> closed stream
ch.oddb>
->
When I emit afterward a secon time $fi.odba_store, I have no problem and the item is stored correctly.
Reverting to the old fachinfo changelog. I just do not have the know how on how to migrate it safely to new separated class.
Deleted all old fi and found that this forced a reparsing with changes. E.g. http://oddb-ci2.dyndns.org/de/gcc/show/fachinfo/21109/diff/16.08.2017 has
6 Änderungen an Fachinfo Lecicarbon -AMZV +1 Zäpfchen für Erwachsene enthält: - -Lecicarbon-Suppositorien beseitigen die Obstipation durch die Zufuhr von CO, das sich aus den Zäpfchen nach dem Einführen in die Ampulla recti langsam entwickelt. Das COaktiviert reflektorisch die Peristaltik und löst den Entleerungsreflex im Enddarm meist innerhalb von 15–20 Min. aus. +Lecicarbon-Suppositorien beseitigen die Obstipation durch die Zufuhr von CO2, das sich aus den Zäpfchen nach dem Einführen in die Ampulla recti langsam entwickelt. Das CO2 aktiviert reflektorisch die Peristaltik und löst den Entleerungsreflex im Enddarm meist innerhalb von 15–20 Min. aus. +Packungen +Lecicarbon Supp Inf 10. (D) +Lecicarbon Supp Ad 10. (D) +Lecicarbon Supp Ad 100. (D)
I think we should reparse all PI/FI on thinpower.
But I want to get rid of the class ChangeLogItem inside Fachinfo, where I found the comment # we cannot remove this class or loading the ODBA cache will fail!. Is this still true?
bin/admin shows
ch.oddb> $with_cls = active_sequences.find_all{|seq| seq.fachinfo.respond_to?(:change_log) }; $with_cls.size
-> 0
ch.oddb> $with_cls = active_sequences.find_all{|seq| seq.fachinfo && seq.fachinfo.de.change_log.size > 0 }; $with_cls.size
-> 5233
ch.oddb> $with_cls = active_sequences.find_all{|seq| seq.fachinfo && seq.fachinfo.de.change_log.size > 0 && seq.fachinfo.de.change_log.first.respond_to?(:chapter) }; $with_cls.size
-> 0
Ideas for speeding up are:
start_time = Time.now; system("ssconvert Packungen-latest.xlsx Packungen-latest.csv"); array = CSV.read('Packungen-latest.csv'); end_time = Time.now; end_time - start_time
Unexpected element 'v:formulas' in state :
{catch all)} -> shapetype
Unexpected element 'v:imagedata' in state :
{catch all)} -> shape
Unhandled object of type Pict
Unexpected element 'x:CF' in state :
{catch all)} -> shape -> ClientData
Unexpected element 'tableParts' in state :
worksheet
=> 3.295281478
Dropping and reloading the database. http://oddb-ci2.dyndns.org/de/gcc/show/fachinfo/62131/diff/19.05.2017 has a nice change_log. Reparsing should add a second one.
Looks that I can get rid of the Fachinfo::ChangeLogItem class by removing ODBA_SERIALIZABLE = ['@change_log'], as Hashes and Arrays are automatically persisted as documented under lib/odba/persistable.rb
Also adding some changes in sbsm and oddb.org to be able to run use a different port, e.g. http://localhost:8712. This allows me to have separate workspace for running a long lasting import while at the same time testing local changes for a view.
Still having problems with updating. Now the number of change_log jumps from 1 to 3, when trying to add just one. And the two new of them are invalid as show by this bin/admin snippet
ch.oddb> registration('60384').fachinfo.de.change_log.size
-> 3
ch.oddb> registration('60384').fachinfo.de.change_log.first.time
-> 2017-05-05
ch.oddb> registration('60384').fachinfo.de.change_log[1].time
-> undefined method `time' for nil:NilClass
Did you mean? timeout
ch.oddb> registration('60384').fachinfo.de.change_log[2].time
-> undefined method `time' for nil:NilClass
Did you mean? timeout
Somehow the new change_log has a verify different method, bin/admin shows
ch.oddb> (registration('60384').fachinfo.de.change_log.last.methods-Object.methods).sort[0..10]
-> [:add_change_log_item, :amzv, :amzv=, :atc_code, :atc_code=, :change_log, :change_log=, :chapter_names, :chapters, :checkout, :composition]
ch.oddb> (registration('56305').fachinfo.de.change_log.last.methods-Object.methods).sort[0..10]
-> [:add_change_log_item, :amzv, :amzv=, :atc_code, :atc_code=, :change_log, :change_log=, :chapter_names, :chapters, :checkout, :composition]
ch.oddb> (registration('56305').fachinfo.de.change_log.first.methods-Object.methods).sort[0..10]
-> [:checkout, :data_origin, :data_origins, :diff, :diff=, :init, :nil_if_empty, :odba_add_observer, :odba_collection, :odba_cut_connection, :odba_delete]
ch.oddb> registration('56305').fachinfo.de.change_log.first.time
-> 2016-09-26
Is there a problem with the languages? Or do I need to disable garbage collection or protect odba_store with a Mutex?
When I look at the same change_log via the pry I have there
[59] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.last.odba_id
=> 36113281
[60] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.last.time
=> 2017-08-16
app.registration('62131').fachinfo.de.change_log.odba_store
<..>
@odba_id=36113281,
@odba_name=nil,
@odba_observers=[],
@odba_persistent=true,
@oid=36113281,
@revision=2017-08-16 16:53:07 +0200,
@time=2017-08-16>]
[63] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.first.oid
=> 35855444
[64] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.first.revision
=> 2017-05-19 07:19:32 +0200
[65] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.last.revision
=> 2017-08-16 16:53:07 +0200
[66] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.last.oid
=> 36113281
[70] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.odba_id
=> 36113260
90] pry(ODDB::TextInfoPlugin)> xy = ODBA.cache.fetch(36113260); xy.class
=> ODDB::FachinfoDocument2001
[91] pry(ODDB::TextInfoPlugin)> xy2 = ODBA.cache.fetch(36113281); xy2.class
ODBA::OdbaError: Unknown odba_id 36113281
from /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/cache.rb:639:in `restore_object'
[92] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.last.odba_store
IOError: closed stream
from /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/marshal.rb:10:in `internal_encoding'
[93] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.change_log.last.odba_isolated_store
IOError: closed stream
from /var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/marshal.rb:10:in `internal_encoding'
and via bin/admin I get
[70] pry(ODDB::TextInfoPlugin)> app.registration('62131').fachinfo.de.odba_id
=> 36113260
ch.oddb> $x3 = ODBA.cache.fetch(36113281)
-> Unknown odba_id 36113281
The big question is, why I cannot call odba_store for the last change_log item, which always throws IOError: closed stream.