bin/update_abschluss_artikel_artikel_fr_kunden show the following error
2016-07-06 16:57:24 +0200: Calling update
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/mail.rb:60 Suppress sending mail with subject: Bbmb(Virbac): (<unknown>): did not find expected alphabetic or numeric character while scanning an alias at line 4 column 15
from orders.virbac@bbmb.ch to: ["ngiger@ywesee.com"] cc: [] reply_to:
Psych::SyntaxError
(<unknown>): did not find expected alphabetic or numeric character while scanning an alias at line 4 column 15
/usr/local/lib/ruby/2.3.0/psych.rb:380:in `parse'
/usr/local/lib/ruby/2.3.0/psych.rb:380:in `parse_stream'
/usr/local/lib/ruby/2.3.0/psych.rb:463:in `load_stream'
/usr/local/lib/ruby/2.3.0/psych/deprecated.rb:30:in `load_documents'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:129:in `load_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:136:in `poll_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:12:in `run'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:158:in `update'
bin/update_abschluss_artikel_artikel_fr_kunden:39:in `<module:BBMB>'
bin/update_abschluss_artikel_artikel_fr_kunden:12:in `<main>'
2016-07-06 16:57:24 +0200: Finished update
Fixed this error by replacin in /var/www/virbac.bbmb.ch/etc/polling.yml glob_pattern: *.CSV by glob_pattern: "*.CSV" Also changed the the directory to be absolute, .eg. /var/www/virbac.bbmb.ch/var/ftp. But as the directory /var/www/virbac.bbmb.ch/var/ftp is empty. No import happens.
Copied /var/www/virbac.bbmb.ch/var/ftp/*CSV from fastpower. Now I have the following UTF-8 error
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/mail.rb:60 Suppress sending mail with subject: Bbmb(Virbac): invalid byte sequence in UTF-8
from orders.virbac@bbmb.ch to: ["ngiger@ywesee.com"] cc: [] reply_to:
ArgumentError
invalid byte sequence in UTF-8
/usr/local/lib/ruby/2.3.0/csv.rb:2016:in `=~'
/usr/local/lib/ruby/2.3.0/csv.rb:2016:in `init_separators'
/usr/local/lib/ruby/2.3.0/csv.rb:1538:in `initialize'
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:21:in `new'
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:21:in `import'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:21:in `import'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:15:in `block in run'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:31:in `block in poll_path'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:30:in `open'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:30:in `poll_path'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:26:in `block in poll'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:25:in `each'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:25:in `poll'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:139:in `block in poll_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:132:in `block in load_sources'
/usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `each'
/usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `load_documents'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:130:in `load_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:138:in `poll_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:12:in `run'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:158:in `update'
bin/update_abschluss_artikel_artikel_fr_kunden:39:in `<module:BBMB>'
bin/update_abschluss_artikel_artikel_fr_kunden:12:in `<main>'
Reason is probably that this file is ISO 8859, as shown by the output of file /var/www/virbac.bbmb.ch/var/ftp/ABSCHLUSS.CSV /var/www/virbac.bbmb.ch/var/ftp/ABSCHLUSS.CSV: ISO-8859 text, with CRLF line terminators
After loading the csv in csv_importer.rb like csv = CSV.new(File.read(io.to_path).encode('UTF-8', {invalid: :replace, undef: :replace, replace: ''})) I am able to import ARTIKEL.CSV and ABSCHLUSS.CSV.
But for KUNDEN.CSV I get
DRb::DRbConnError DRb::DRbServerNotFound /usr/local/lib/ruby/2.3.0/drb/drb.rb:1719:in `current_server' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1788:in `to_id' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1103:in `initialize' /usr/local/lib/ruby/2.3.0/drb/drb.rb:651:in `new' /usr/local/lib/ruby/2.3.0/drb/drb.rb:651:in `make_proxy' /usr/local/lib/ruby/2.3.0/drb/drb.rb:568:in `rescue in dump' /usr/local/lib/ruby/2.3.0/drb/drb.rb:565:in `dump' /usr/local/lib/ruby/2.3.0/drb/drb.rb:614:in `send_request' /usr/local/lib/ruby/2.3.0/drb/drb.rb:926:in `send_request' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1253:in `send_message' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1142:in `block (2 levels) in method_missing' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1229:in `open' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1141:in `block in method_missing' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1160:in `with_friend' /usr/local/lib/ruby/2.3.0/drb/drb.rb:1140:in `method_missing' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:83:in `rename_user' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/model/customer.rb:64:in `email=' /home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:85:in `block in import_record' /home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:76:in `each' /home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:76:in `import_record' /home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:29:in `block in import' /usr/local/lib/ruby/2.3.0/csv.rb:1748:in `each' /home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:24:in `each_with_index' /home/niklaus/git/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:24:in `import' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:21:in `import' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:15:in `block in run' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:31:in `block in poll_path' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:30:in `open' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:30:in `poll_path' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:26:in `block in poll' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:25:in `each' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:25:in `poll' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:138:in `block in poll_sources' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:131:in `block in load_sources' /usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `each' /usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `load_documents' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:130:in `load_sources' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:137:in `poll_sources' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:12:in `run' /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:158:in `update' bin/update_abschluss_artikel_artikel_fr_kunden:39:in `<module:BBMB>' bin/update_abschluss_artikel_artikel_fr_kunden:12:in `<main>'
Why is the BBMB.auth.autosession not correctly initialized? The connection to the bbmb server is okay. But the connection to the yus server running on port 12003 is never established, as the method login of bbmb-2.0.5/lib/bbmb/util/server.rb is never called.
Therefore trying now to use the built in updater. Now I do not get any error, but creating a new user hangs. And the login is not called, neither. Tried to debug the situation with pry. Did not help yet. Trying to add a pry debug statement in virbac.bbmb.ch-yus. Did not work. When I kill the virbac.bbmb.ch-yus while the updater is hanging I get the following stack trace:
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/mail.rb:60 Suppress sending mail with subject: Bbmb(Virbac): connection closed
from orders.virbac@bbmb.ch to: ["ngiger@ywesee.com"] cc: [] reply_to:
DRb::DRbConnError
connection closed
/usr/local/lib/ruby/2.3.0/drb/drb.rb:579:in `load'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:641:in `recv_reply'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:941:in `recv_reply'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:1254:in `send_message'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:1142:in `block (2 levels) in method_missing'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:1229:in `open'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:1141:in `block in method_missing'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:1160:in `with_friend'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:1140:in `method_missing'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:85:in `rename_user'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/model/customer.rb:64:in `email='
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:87:in `block in import_record'
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:78:in `each'
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:78:in `import_record'
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:30:in `block in import'
/usr/local/lib/ruby/2.3.0/csv.rb:1748:in `each'
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:25:in `each_with_index'
/var/www/virbac.bbmb.ch/lib/bbmb/util/csv_importer.rb:25:in `import'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:21:in `import'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:15:in `block in run'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:31:in `block in poll_path'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:30:in `open'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:30:in `poll_path'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:26:in `block in poll'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:25:in `each'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:25:in `poll'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:137:in `block in poll_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:130:in `block in load_sources'
/usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `each'
/usr/local/lib/ruby/2.3.0/psych/deprecated.rb:32:in `load_documents'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:129:in `load_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/polling_manager.rb:136:in `poll_sources'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/updater.rb:12:in `run'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:172:in `update'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:139:in `block (2 levels) in run_updater'
/usr/local/lib/ruby/2.3.0/delegate.rb:85:in `loop'
/usr/local/lib/ruby/2.3.0/delegate.rb:85:in `call'
/usr/local/lib/ruby/2.3.0/delegate.rb:85:in `method_missing'
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/util/server.rb:121:in `block in run_updater'
csv_importer.rb: importing /var/www/virbac.bbmb.ch/var/ftp/ARTIKEL.CSV
D, [2016-07-11T13:51:16.904332 #18197] DEBUG -- updater: ProductImporter imported 0 entities
csv_importer.rb: importing /var/www/virbac.bbmb.ch/var/ftp/ABSCHLUSS.CSV
D, [2016-07-11T13:51:16.904734 #18197] DEBUG -- updater: QuotaImporter imported 0 entities
csv_importer.rb: importing /var/www/virbac.bbmb.ch/var/ftp/ARTIKEL_FR.CSV
D, [2016-07-11T13:51:16.905621 #18197] DEBUG -- updater: ProductImporter imported 0 entities
D, [2016-07-11T13:51:16.905998 #18197] DEBUG -- : update finished
Added require 'odba/18_19_loading_compatibility' in bin/yusd to make it run under Ruby 2.3.1, as I want to be able to drop into pry. Now I see, that the autosession starts and the DRB call hangs while executing the line 17 block_value = block_yield(x) of /usr/local/lib/ruby/2.3.0/drb/invokemethod.rb line 17 DRb::DRbServer::InvokeMethod18Mixin#perform_with_block:
When I simply skip renaming I find about 50 lines (among 1201 customers) which contain a pattern like SKIPPING rename_user old_name nil to new_name user@dummy.com. Sometimes the newname contains ISO-8859-14 characters like \xC3\xB6
Changed bbmb/lib/bbmb/model/customer.rb to convert the email parameter of the email= method to email = email.encode('UTF-8'). Now I get many lines of Duplicate name: mamaleu@bluewin.ch. This stems from the fact that the virbac_yus is not yet converted to UTF-8. Must add a script to convert it.
Also creating a script to convert the virbac_bbmb to UTF-8. Discovered that I have a mess inside yus as the lib/yus/helpers.rb defines the module as "YUS" (all capital), whereas everywhere else we used "Yus" (only capitalized). Will fix this problem.
The migration works now. Pushed commits: