view · edit · sidebar · attach · print · history

Index>

20170510-virbac-rack-importer

Summary

  • Fix remaining problems (importer) in new rack based virbac
  • Keep in Mind

Commits

Index

Fix remaining problems (importer) in new rack based virbac

We have to fix various problems:

  • Visiting Customer (ID 708427, name Ruesch-Perk) gives undefined method `total' for nil:NilClass
  • while running importer
    • Importer creates too many new customers (e.g. Created new customer_id 200077 which is a BBMB::Model::Customer with odba_id 54763)
    • Rescue undefined method `compact' for #<BBMB::Model::Order:0x0055dcc42b8198> in import
  • Following error seen on fastpower
nNoMethodError
undefined method `to_i' for false:FalseClass
Did you mean?  to_s
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/updater.rb:21:in `block in import'
/usr/local/lib/ruby/2.3.0/logger.rb:427:in `add'
/usr/local/lib/ruby/2.3.0/logger.rb:455:in `debug'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/updater.rb:21:in `import'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/updater.rb:14:in `block in run'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:31:in `block in poll_path'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:30:in `open'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:30:in `poll_path'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:26:in `block in poll'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:25:in `each'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:25:in `poll'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:139:in `block in poll_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:132:in `block in load_sources'
/usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `each'
/usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `load_documents'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:131:in `load_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/polling_manager.rb:138:in `poll_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/updater.rb:11:in `run'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/server.rb:161:in `update'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/server.rb:128:in `block (2 levels) in run_updater'
/usr/local/lib/ruby/2.3.0/delegate.rb:85:in `loop'
/usr/local/lib/ruby/2.3.0/delegate.rb:85:in `call'
/usr/local/lib/ruby/2.3.0/delegate.rb:85:in `method_missing'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.1.3/lib/bbmb/util/server.rb:116:in `block in run_updater'

The last error is clear. A Debug statement was wrong. Corrected here.

Made a small reorganisation specifiying a VIRBAC::App and VIRBAC::Validator in config.ru to make the naming clearer and to avoid confusion with names from the BBMB gem.

Simplified the require list of bin/update_abschluss_artikel_artikel_fr_kunden. Now the importer fails with RegexpError: target of repeat operator is not specified: /*farmaciaportone@bluewin.ch/>. This looks like an illegal address. Fixed this error to search whether the string is equal, not via a regular expression.

Importing the articles gets the error undefined method `compact' for #<BBMB::Model::Order:0x0055b1697a1758> in import Fixing it. The Customer 4010 Tierarztpraxis Waldmeier GmbH does not have an array of quotas but only a single BBMB::Model::Order element. Why? Adding a patch to fix this problem. Dropping databases virbac_yus and virbac_bbmb and loading the with yesterday dumps from fastpower. Running import again. No new customer is created. Now the import finished in less than 11 minutes. Fetching also yesterday CSV files and rerunning the import. This showed Created 3 customers with ID 4432,700399,700400.

Attacking now the problem with Rüsch-Persk undefined method `total' for nil:NilClass. Debugging with pry shows that we have problems with ODBA which cannot reconstruct the positions

From: /home/niklaus/git/bbmb/lib/bbmb/model/order.rb @ line 210 BBMB::Model::Order#total:

    206: def total
    207:   @positions.inject(@shipping) { |memo, pos| pos.total + memo }
    208: rescue => error
    209:   require 'pry'; binding.pry
 => 210: end

[1] pry(#<BBMB::Model::Order>)> @positions.size 
=> 4
[2] pry(#<BBMB::Model::Order>)> @positions
ODBA::Stub was unable to replace BBMB::Model::Order::Position#269595 from Array:#269468
ODBA::Stub was unable to replace BBMB::Model::Order::Position#269594 from Array:#269468
ODBA::Stub was unable to replace BBMB::Model::Order::Position#269596 from Array:#269468
ODBA::Stub was unable to replace BBMB::Model::Order::Position#269597 from Array:#269468
=> [nil, nil, nil, nil]

Fixed this problem, albeit at the cost of returning 0 as Umsatz, as we cannot calculate the price. Now I have other problen when clicking on "0.00 Umsatz", as we cannot correctly display the orders. Trying to debug the underlying ODBA

[4] pry(#<BBMB::Model::Order>)> ODBA.cache.fetch('26595').class
=> BBMB::Model::Order
[7] pry(#<BBMB::Model::Order>)> ODBA.cache.fetch('26595').positions.inject(0) { |memo, pos| memo + pos.quantity }
=> 2
[8] pry(#<BBMB::Model::Order>)> ODBA.cache.fetch('26595').positions.inject(@shipping) { |memo, pos| pos.total + memo }
=> #<BBMB::Util::Money:0x007f3d5968bb70 @credits=2060>
# 
[11] pry(#<BBMB::Model::Order>)> @positions =  ODBA.cache.fetch('26595').positions;
[12] pry(#<BBMB::Model::Order>)> self.odba_store;

I think this should be possible to do via similar commands on the virbac_admin console. Dropping and reloading the database here. Looking why I cannot start virbac_admin. Done by removing the no longer used require 'sbsm/drb'.

Looks like we still have a problem, as the console does not find the method _admin

sudo -u bbmb /usr/local/bin/bundle-240 exec bin/virbac_admin 
Force the use of /home/niklaus/git/virbac.bbmb.ch/etc/config.yml
ch.bbmb.virbac> $perk = Model::Customer.find_by_customer_id(708427) 
-> undefined method `_admin' for #<Yus::Server:0x005603f829f1d8>
ch.bbmb.virbac> 

This was fixed by adding a new class SBSM::AdminServer which implements the _admin method. This should enable an easy porting of other admin interfaces (e.g. oddb.org) when switching to the Rack based SBSM.

Added the old code from bbmbd to start the updater, invoicer and the virbac service.

In virbac_admin

$perk = Model::Customer.find_by_customer_id(708427) 
-> uninitialized constant #<Class:#<BBMB::Util::Server:0x0056234022cd60>>::Model
ch.bbmb.virbac> $perk = BBMB::Model::Customer.find_by_customer_id(708427) 
-> #<BBMB::Model::Customer:0x0056233e752198>
ODBA.cache.fetch('26595').class

When I start the Virbac again, I suddenly find two customers with the name Perk, eg.

  • 16759 Rüsch-Perk Ruth 8427 (This one has an Umsatz > 0 and various past orders)
  • 708427Ruesch-Perk 8427 Rorbas

Deactivating my changes in BBMB::Model::Order and BBMB::Model::Customer. Reimporting the database and visiting perk again. No problems displaying. Importing CSV files again with sudo -u bbmb bundle exec bin/update_abschluss_artikel_artikel_fr_kunden config=etc/config.yml.

Now I am getting the following error while running the importer

...ProductImporter postprocess
ODBA::Stub was unable to replace #269599 from BBMB::Model::Customer:#2617
Rescue undefined method `add' for nil:NilClass in import
/home/niklaus/git/virbac.bbmb.ch/vendor/bundle/ruby/2.4.0/gems/odba-1.1.2/lib/odba/stub.rb:112:in `method_missing'
/home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:303:in `block (2 levels) in postprocess'
/home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:302:in `each'
/home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:302:in `block in postprocess'
/home/niklaus/git/virbac.bbmb.ch/vendor/bundle/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:150:in `block in odba_extent'
/home/niklaus/git/virbac.bbmb.ch/vendor/bundle/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:150:in `each'
/home/niklaus/git/virbac.bbmb.ch/vendor/bundle/ruby/2.4.0/gems/odba-1.1.2/lib/odba/persistable.rb:150:in `odba_extent'
/home/niklaus/git/bbmb/lib/bbmb/persistence/odba.rb:18:in `all'
/home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:298:in `postprocess'
/home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:51:in `import'
<..>
[8] pry(#<BBMB::Util::ProductImporter>)> ODBA.cache.fetch(269599).class
ODBA::OdbaError: Unknown odba_id 269599
from /home/niklaus/git/virbac.bbmb.ch/vendor/bundle/ruby/2.4.0/gems/odba-1.1.2/lib/odba/cache.rb:639:in `restore_object'
[9] pry(#<BBMB::Util::ProductImporter>)> ODBA.cache.fetch(2617).class
=> BBMB::Model::Customer
[10] pry(#<BBMB::Util::ProductImporter>)> ODBA.cache.fetch(2617).customer_id
=> "708427"
12] pry(#<BBMB::Util::ProductImporter>)> ODBA.cache.fetch(2617).organisation
=> "Ruesch-Perk"
[13] pry(#<BBMB::Util::ProductImporter>)> ODBA.cache.fetch(2617).firstname
=> nil
[14] pry(#<BBMB::Util::ProductImporter>)> ODBA.cache.fetch(2617).lastname
=> nil
[15] pry(#<BBMB::Util::ProductImporter>)> ODBA.cache.fetch(2617).plz
=> "8427"

Running the importer a second time did not show any errors. But now I have problems logging in as claude meier and get undefined method `current_order' for nil:NilClass. But logging in as Ruesch-Perk with the newly set password works.

Setting the password to 1234 worked and afterwards I was able to log in. Tried creating a new password. Clicking on "Passwort anzeigen" show the popup. But we go into an eternal loop. Somehow we do not logout correctly. Postponing the problem as we have at least on possibility to change the password. Maybe we could remove the generate_password and when clicking on change password we could just offer a new cleartext password generate in the same way as done in the generate_password method.

Pushed commit SBSM: Changes needed for rack, viral modules and KnownUser

When visiting inexistent customers (given by customerid), admin users create new customer if it does not exist.

Dropped and reloaded the virbac_bbmb and virbac_yus database. Now I am getting the following backtrace when logging in with claude.meier

session.rb:242:in `rescue in process_rack' Error in process_rack /home/niklaus/git/bbmb/lib/bbmb/html/state/current_order.rb:20:in `init'
/home/niklaus/git/sbsm/lib/sbsm/session.rb:222:in `block in process_rack'
/home/niklaus/git/sbsm/lib/sbsm/session.rb:206:in `synchronize'
/home/niklaus/git/sbsm/lib/sbsm/session.rb:206:in `process_rack'
/home/niklaus/git/sbsm/lib/sbsm/app.rb:119:in `call'
/var/www/virbac_rack.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.2/lib/rack/lint.rb:49:in `_call'
NoMethodError: undefined method `current_order' for nil:NilClass
        /home/niklaus/git/bbmb/lib/bbmb/html/state/current_order.rb:20:in `init'
        /home/niklaus/git/sbsm/lib/sbsm/session.rb:222:in `block in process_rack'
        /home/niklaus/git/sbsm/lib/sbsm/session.rb:206:in `synchronize'
        /home/niklaus/git/sbsm/lib/sbsm/session.rb:206:in `process_rack'
        /home/niklaus/git/sbsm/lib/sbsm/app.rb:119:in `call

virbac_admin shows

ch.bbmb.virbac> BBMB::Model::Customer.find_by_email('claude.meier@gmx.net').class
-> NilClass
ch.bbmb.virbac> BBMB::Model::Customer.odba_extent.find{|x| x.email.eql?('claude.meier@gmx.net')}.class
-> NilClass

What has happened while dropping and reloading?

On fastpower I get in virbac_admin

ch.bbmb.virbac> BBMB::Model::Customer.odba_extent.find{|x| x.email.eql?('claude.meier@gmx.net')}.customer_id
-> 700010
ch.bbmb.virbac> BBMB::Model::Customer.odba_extent.find{|x| x.email.eql?('claude.meier@gmx.net')}.odba_id
-> 13758
ch.bbmb.virbac> ODBA.cache.fetch(13758).class
-> BBMB::Model::Customer
ch.bbmb.virbac> ODBA.cache.fetch(13758).customer_id
-> 700010

On my system I get

ch.bbmb.virbac> ODBA.cache.fetch(13758).class
-> BBMB::Model::Order
ch.bbmb.virbac> ODBA.cache.fetch(13758).customer
-> #<BBMB::Model::Customer:0x0055e48edd3c90>
ch.bbmb.virbac> ODBA.cache.fetch(13758).customer.customer_id
-> 17726

Which means that we somehow do not restore the database correctly, as ODBA ID 13758 was a Customer on fastpower and is now an Order. This is a total mess. And explains why I have problems running my tests. Checking SHA256 sums of databases

ngiger@fastpower /var/www/virbac.bbmb.ch $ sha256sum /var/backup/fast/db/postgresql/May-2017/2017-05-09/22:00-postgresql_database-virbac_bbmb-backup.gz
b871671a8d40d2e2b8a407fff54f3ba0d84bdb9430e4114d741c262302839e47  /var/backup/fast/db/postgresql/May-2017/2017-05-09/22:00-postgresql_database-virbac_bbmb-backup.gz
niklaus@oddb-ci2 ~/g/virbac.bbmb.ch> sha256sum /var/backup/fast/db_dumps/virbac_bbmb.20170509.sql.gz 
b871671a8d40d2e2b8a407fff54f3ba0d84bdb9430e4114d741c262302839e47  /var/backup/fast/db_dumps/virbac_bbmb.20170509.sql.gz

They are identical.

Adding
update?: false
invoice?: false

to etc/config.yml to disable running update/invoice.

Somehow my changes for the _admin interface broke the connections. Suspects are:

  • variable @persistenc in VIRBAC::App
  • Why we have a nil @model in bbmb/lib/bbmb/html/state/current_order.rb @ line 21 BBMB::Html::State::CurrentOrder#init?
view · edit · sidebar · attach · print · history
Page last modified on May 10, 2017, at 06:30 PM