20130213-debug-production-migel-data-rebuild-index

<< | Index | >>

Summary

• Debug migel database.
• Update MiGel Products and Importer.

Commits

migel

• Fix missing migel products
• Update path argment for jobs
• Update migel import job STDOUT for debug

ch.oddb.org

• Add id for explain_result

Index

• Debug-migel-search
• Debug-legends-toggle-switch
• update-job-output

Debug-migel-search

broken INDEX, data on production.

• migel_model_product_pharmacode
• migel_product_fulltext_index_de
• all_products

Encoding problem

File access error in Job.

migel> Importer.new.import_all_products_from_csv('data/csv/migel_product_de.csv', 'de', true)
-> No such file or directory - data/csv/migel_product_de.csv
migel> Importer.new.import_all_products_from_csv('/var/www/migel/data/csv/migel_product_de.csv', 'de', true)
-> invalid byte sequence in US-ASCII

@40000000511ad98a3b7f8b64 E, [2013-02-13T01:08:32.998105 #17684] ERROR -- admin: ArgumentError
@40000000511ad98a3b807d94 E, [2013-02-13T01:08:32.998244 #17684] ERROR -- admin: invalid byte sequence in US-ASCII
@40000000511ad98a3b863e3c E, [2013-02-13T01:08:32.998617 #17684] ERROR -- admin: ["/var/www/migel/lib/migel/util/server.rb:19:in `instance_eval'",
@40000000511ad98a3b864224  "/usr/local/lib/ruby/1.9.1/csv.rb:1868:in `block in shift'",
@40000000511ad98a3b864224  "/usr/local/lib/ruby/1.9.1/csv.rb:1862:in `loop'",
@40000000511ad98a3b86460c  "/usr/local/lib/ruby/1.9.1/csv.rb:1862:in `shift'",
@40000000511ad98a3b86460c  "/usr/local/lib/ruby/1.9.1/csv.rb:1804:in `each'",
@40000000511ad98a3b86460c  "/usr/local/lib/ruby/1.9.1/csv.rb:1221:in `block in foreach'",
@40000000511ad98a3b86de64  "/usr/local/lib/ruby/1.9.1/csv.rb:1367:in `open'",
@40000000511ad98a3b86de64  "/usr/local/lib/ruby/1.9.1/csv.rb:1220:in `foreach'",
@40000000511ad98a3b86e24c  "/var/www/migel/lib/migel/util/importer.rb:290:in `import_all_products_from_csv'",
@40000000511ad98a3b86e24c  "(eval):1:in `block in _admin'",
@40000000511ad98a3b86e24c  "/var/www/migel/lib/migel/util/server.rb:19:in `instance_eval'",
@40000000511ad98a3b86e634  "/var/www/migel/lib/migel/util/server.rb:19:in `block in _admin'"]

migel_code parse error

Importer Job does not parse migel_code, correctly.
(or, the format of migel_product_de.csv may have been changed.)

z.B.

migel job must parse migel_code like this:

• "151002001" --> "15.10.02.00.1"

can't add a new key into hash during iteration

ODBA brokes HASH itself at save.

commit

• Fix missing migel products
  • fix bugs of import job
  • fix encoding problem on production
  • update to prevent "can't add a new key into hash during iteration" error by ODBA
• Update path argment for jobs

restore step

1. . Importer.new.import_all_products_from_csv('/var/www/migel/data/csv/migel_product_de.csv', 'de', true)
2. . Importer.new.import_all_products_from_csv('/var/www/migel/data/csv/migel_product_fr.csv', 'fr', true)
3. . init_products
4. . rebuild_fulltext_index_tables

migel> products.length
-> 32187

• http://ch.oddb.org/de/gcc/migel_search/migel_code/341501031
• http://ch.oddb.org/de/gcc/migel_search/migel_code/17.05.01.00.1

Debug-legends-toggle-switch

console error.
There is no legend in migel search.

TypeError: legends is null
[Break On This Error] 	

isDebug:              false,

commit

• Add id for explain_result

Update job output

Add parsed migel_code and state to STDOUT of Job process for debugging.

• state (updating, ignoring, saving)
• migel_code

updating: 1469 / 1472   Estimate total: 0.15 [m] It will be done in: 0.00 [m] migel_code: 05.02.04.00.1
updating: 1470 / 1472   Estimate total: 0.15 [m] It will be done in: 0.00 [m] migel_code: 05.02.04.00.1
updating: 1471 / 1472   Estimate total: 0.15 [m] It will be done in: 0.00 [m] migel_code: 05.02.04.00.1
...
ignoring: 1469 / 1472   Estimate total: 0.15 [m] It will be done in: 0.00 [m] migel_code: 0000000000
ignoring: 1470 / 1472   Estimate total: 0.15 [m] It will be done in: 0.00 [m] migel_code: 0000000000
ignoring: 1471 / 1472   Estimate total: 0.15 [m] It will be done in: 0.00 [m] migel_code: 0000000000
...

This commit may be a little usefull for debugging of next time.

commit

• Update migel import job STDOUT for debug