view · edit · sidebar · attach · print · history

< Niklaus.20160815-delete-bauernfeind-before-import | Index | Niklaus.20160809-update-evidenta-productportrait >>

20160810-download-evidentia-link

Summary

  • download evidentia product link CSV
  • jobs/update_migel_products_with_report hangs
  • import Bauerfeind CSV must add a space after ',' in article_names
  • Fix BBMB unit tests
  • Keep in Mind

Commits

Index

download evidentia product link CSV

When running jobs/import_evidentia_fi_search_links we must download the file from http://www.evidentia.beatsteinegger.com/evidentia_fi_link.csv. If it matches the latest download file, the import should be skipped.

Improved src/util/latest.rb to remove yesterdays file if it has the same size as todays file.

Pushed commit Download csv-file for evidentia_search_links

Zeno wants the import in the import_daily. Therefore I must return an empty report.

Having problems, as http://www.evidentia.beatsteinegger.com/evidentia_fi_link.csv should be ISO-8859-1, but has \xa8 characters, which make no sense as link.

My test was running with a UTF-8 encoded file.

jobs/update_migel_products_with_report hangs

Zeno reported that the jobs/update_migel_products_with_report hangs. Trying to reproduce it on oddb-ci2.

Got (after over 5 hours) an e-mail with the following backtrace

Wed Aug 10 10:56:24 2016: migel Migel::Util::Importer#reported_save_all_products(de)
NoMethodError
undefined method `cleanup_items' for #<ODDB::Swissindex::SwissindexMigel:0x00000003f383f8>
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1508:in `perform'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
(druby://oddb-ci2:43623) /var/www/migel/lib/migel/plugin/swissindex.rb:84:in `block (2 levels) in save_all_products'
(druby://oddb-ci2:43623) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1548:in `call'
(druby://oddb-ci2:43623) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block'
(druby://oddb-ci2:43623) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1508:in `perform'
(druby://oddb-ci2:43623) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://oddb-ci2:43623) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://oddb-ci2:43623) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/invokemethod.rb:10:in `block_yield'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/invokemethod.rb:17:in `block in perform_with_block'
(druby://localhost:50002) /var/www/oddb.org/ext/swissindex/src/swissindex.rb:25:in `session'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/invokemethod.rb:14:in `perform_with_block'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1506:in `perform'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:50002) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
/var/www/migel/lib/migel/plugin/swissindex.rb:38:in `block in save_all_products'
/usr/local/lib/ruby/1.9.1/csv.rb:1354:in `open'
/var/www/migel/lib/migel/plugin/swissindex.rb:37:in `save_all_products'
/var/www/migel/lib/migel/util/importer.rb:354:in `save_all_products'
/var/www/migel/lib/migel/util/importer.rb:238:in `reported_save_all_products'
/var/www/migel/lib/migel/util/importer.rb:225:in `block in save_all_products_all_languages'
/var/www/migel/lib/migel/util/importer.rb:222:in `each'
/var/www/migel/lib/migel/util/importer.rb:222:in `save_all_products_all_languages'
jobs/update_migel_products_with_report:13:in `block in <module:Util>'
/var/www/migel/lib/migel/util/job.rb:22:in `call'
/var/www/migel/lib/migel/util/job.rb:22:in `run'
jobs/update_migel_products_with_report:12:in `<module:Util>'
jobs/update_migel_products_with_report:11:in `<module:Migel>'
jobs/update_migel_products_with_report:10:in `<main>'

Fixed with commit Removed call to obsolete method swissindex.cleanup_items

Fix BBMB unit tests

Problem locally solved yesterday evening by saving a copy of BBMB.config via test_helper and adding teardown methods in all tests which changed the BBMB.config.

Still having problems after commit Fixed running most unit tests

E.g. running with SEED=64021 bundle exec rake test hangs. Also there are still a few skipped tests.

import Bauerfeind CSV must add a space after ',' in article_names

Fixed with Replace ',' by ', ' for articlename from Bauerfeind

Also fixed a typo with Fixed typo in job name

view · edit · sidebar · attach · print · history
Page last modified on August 11, 2016, at 09:33 AM