Two problems are remaining:
Started yesterday at 21:30 jobs/update_migel. Last line see is 2016-08-15 21:40:31 +0200: update_group ["99", "50", "01", "00", "1"] groupcd 99 de. Looking for jobs
niklaus 523 2853 0 08:19 pts/2 00:00:00 grep --color=auto migel root 1875 1835 0 Aug15 ? 00:00:00 supervise migeld niklaus 17023 2879 0 Aug15 pts/3 00:00:00 migel (Admin-Tool) root 17303 1875 0 Aug15 ? 00:00:00 sudo -u apache /usr/local/bin/ruby bin/migeld apache 17305 17303 0 Aug15 ? 00:01:43 /usr/local/bin/ruby bin/migeld postgres 17308 1774 0 Aug15 ? 00:00:02 postgres: migel migel [local] idle postgres 17309 1774 0 Aug15 ? 00:00:02 postgres: migel migel [local] idle postgres 17310 1774 0 Aug15 ? 00:00:02 postgres: migel migel [local] idle postgres 17311 1774 0 Aug15 ? 00:00:02 postgres: migel migel [local] idle postgres 17312 1774 0 Aug15 ? 00:00:02 postgres: migel migel [local] idle root 22277 2409 0 Aug15 pts/1 00:00:00 sudo -u apache jobs/update_migel niklaus 22278 2409 0 Aug15 pts/1 00:00:00 tee /home/niklaus/git/migel/update_migel.log apache 22281 22277 4 Aug15 pts/1 00:28:15 /usr/local/bin/ruby jobs/update_migel postgres 22284 1774 0 Aug15 ? 00:04:59 postgres: migel migel [local] idle postgres 22285 1774 0 Aug15 ? 00:05:00 postgres: migel migel [local] idle postgres 22286 1774 0 Aug15 ? 00:05:00 postgres: migel migel [local] DELETE postgres 22287 1774 0 Aug15 ? 00:04:59 postgres: migel migel [local] idle postgres 22288 1774 0 Aug15 ? 00:05:00 postgres: migel migel [local] idle
There is a postgres DELETE statement which hangs. I think a call to delete for a product does not work anymore, as yesterday Migel::Model::Product.all.find_all{|x| /Bauerfeind/i.match(x.company_name.send(:de)) }.each{|x| x.delete } did not finish, too. Trying to delete a singel product via bin/admin (after killing the import and restarting the migeld).
The following lines must be checked 11041403089601;4046445600227;GenuTrain P3 mit SR,titan,re,1;GenuTrain P3,silicone,titane,droit,1;6509897;05.04.05.00.1; Fr. 179.20 ;GenuTrain P3 mit SR,titan,re,1;GenuTrain P3,silicone,titane,droit,1 11041403080601;4046445600104;GenuTrain P3,titan,rechts,1;GenuTrain P3,titane,droit,1;2244775;05.04.05.00.1; Fr. 160.60 ;GenuTrain P3,titan,rechts,1;GenuTrain P3,titane,droit,1
bin/admin show that the product with EAN 4046445107818 has the wrong migel_code 05.04.03.00.1
migel> Migel::Model::Product.all.find{|x| '4046445107818'.eql?(x.ean_code)}.migel_code
-> 05.04.03.00.1
migel> Migel::Model::Product.all.find{|x| '4046445107818'.eql?(x.ean_code)}
-> GENUTRAIN P3 Kniebandage links titan Gr 5 40-43cm
migel> Migel::Model::Product.all.find{|x| '4046445107818'.eql?(x.ean_code)}.status
-> A
migel> Migel::Model::Product.all.find{|x| '4046445107818'.eql?(x.ean_code)}.company_name
-> Bauerfeind AG
migel> Migel::Model::Product.all.find{|x| '4046445107818'.eql?(x.ean_code)}.company_name.send(:de)
-> Bauerfeind AG
migel> Migel::Model::Product.all.find{|x| '4046445107818'.eql?(x.ean_code)}.company_name.send(:fr)
-> Bauerfeind AG
Looks, as I have forgotton to save the product, after changing its status. Calling Migel::Model::Product.all.find{|x| x.ean_code.eql?('4046445419065')}.delete hangs, too. Replacing the delete by status = 'I' to deactivate it.
Added debug output for deletes. They did not show up. Running jobs/import_migel_bauerfeind always takes over 13 minutes to get the interesting code parts.
After running http://oddb-ci2.dyndns.org/de/gcc/search/zone/migel/search_query/4046445419065 shows two products, one with status 'I' and '05.04.02.00.1', the other one with status 'A' and ) MiGel Code: '05.04.05.00.1'. This is okay. Only the one with status 'A' shows up after searching for genutrain via http://oddb-ci2.dyndns.org/de/gcc/search/zone/migel/search_query/GenuTrain?
Preparing commit for push. Eliminating superfluos puts statements. Running it again. Remarked that the field "Packungsgrösse" is empty after an import for 4046445419065, as it is not present inte CSV file from Bauerfeind.
After running the import, the old entry with pharmacode '4313351' is correctly marked as inactive, has the correct migel_code and article_name. But it does not show up unter http://oddb-ci2.dyndns.org/de/gcc/migel_search/migel_code/17.03.02.00.1. Why?
Found out I can delete a product from the migel_id
$id = Migel::Model::Migelid.find_by_migel_code('17.03.02.00.1')
-> Paar
migel> $id.products.find_all{|x| x.ean_code.eql?('4046445241024')}.size
-> 2
migel> $id.products.values.delete_if{|x| x.ean_code.eql?('4046445241024') && x.status.eql?('I')}
-> Array
migel> $id.products.find_all{|x| x.ean_code.eql?('4046445241024')}.size
-> 1
Remarked that after running import_bauerfeind, I have problems calling Migel::Model::Migelid.all in the migel/bin/admin
migel> Migel::Model::Migelid.all -> incompatible character encodings: UTF-8 and ASCII-8BIT
After reloading the database from thinpower I have
migel> Migel::Model::Migelid.all.first.name.to_s.encoding -> ASCII-8BIT migel> Migel::Model::Migelid.all.first.name.to_s -> Gripper für Port-A-Cath
Calling Migel::Model::Migelid.all on thinpower takes several minutes.
Running migrate_utf8 on oddb-ci2 gives
ODBA::Stub was unable to replace Hash#32292 from Migel::Model::Migelid:#58 ODBA::Stub was unable to replace Hash#32293 from Migel::Model::Migelid:#880
On thinpower the Migel::Model::Migelid.all.first.name.to_s.encoding returns UTF-8.
Checked that the migel-db on oddb-ci2 is really UTF-8 with
psql -U apache migel --host=localhost -c 'SHOW SERVER_ENCODING' server_encoding ----------------- UTF8 (1 Zeile)
Deleting all products from Bauerfeind works with this method
def clear_bauerfeind_products
@nr_products_before = get_nr_active_bauerfeind_products
items = 0
Migel::Model::Migelid.all.each do |migel_id|
migel_code = migel_id.migel_code
nr_invalids = migel_id.products.find_all{|x| /Bauerfeind/i.match(x.companyname) }.size
if nr_invalids > 0
items += nr_invalids
puts "#{Time.now}: Total #{items}: Deleting #{nr_invalids} items for #{migel_code} by #{company_re_de}"
migel_id.products.delete_if{|x| /Bauerfeind/i.match(x.companyname) }
migel_id.save
end
end
@nr_products_after = get_nr_active_bauerfeind_products
puts "#{Time.now}: Updated via DE #{items} items"
end
Reorganizing the import to first mark all Bauerfeind products as inactive and deleting at the end all inactive Bauerfeind products. This allows me to conserver the Packungsgrösse for existing products. Dropping and reloading the migel database before running the import.
Pushed commit Inactive Bauerfeind products before import via CSV
Pushed commit tested over night Add update_evidentia_fi_search_links to import_daily