view · edit · sidebar · attach · print · history

20120403-update-swissindex-updatetr-job-search-form-type-import-gkv-de-oddb-org

<< | Index | >>

summary

  • updated swissindex plugin
    • learned swissindex updater job.
    • to update out_of_trade and to add pharmacode without debug(logging) mode.
  • checked import_gkv job in de.oddb.org
    • setup database for de.oddb.org
    • fixed htmlgrid nomethod error for String of Ruby 1.8
    • installed mechanize, nokogiri for Ruby 1.8.7 to gems of Ruby 1.8.6 (same env Production)
    • ran import_gkv job
    • fixed ssl error problem (patch Attach:ssl-error-no-verify-solution-20120403.patch.txt)
    • to check syntax error in pg, dbi (It may be version problem)

commit

ch.oddb.org
htmlgrid
de.oddb.org

index

Debug Swissindex updater job

Swiss Registration (Ausser Handel:On) > Pharmacode in Packung > Such Resultat (Ausser Handel:Off)

RefData Artikel-Datenbank = Aktive / Inactive

  • Ausser Handel (MedRef)
  • trotzdem anzeigen
  • Pharmacode
Done mail(update_package_trade_status_by_swissindex)
Checked 24765 packages
Updated in trade     (out_of_trade:false): 0 packages
Updated out of trade (out_of_trade:true) : 66 packages
Updated pharmacode: 0 packages
Deleted pharmacode: 13 packages

Updated in trade     (out_of_trade:false): 0 packages
Check swissindex by eancode and then check if the package is out of trade (true) in ch.oddb,
if so the package becomes in trade (false)

Updated out of trade (out_of_trade:true) : 66 packages
If there is no eancode in swissindex and the package is in trade in ch.oddb,
then the package becomes out of trade (true) in ch.oddb
7680549300118: http://ch.oddb.org/de/gcc/reg/54930/seq/01/pack/011
...
Note

Refs

tried update_package_trade_status_by_swissindex (parameter logging=false, im Updater#update_bsv_followers)

Note
  • need /var/www/oddb.org/ext/swissindex/bin/swissindex_pharmad
  • call SwissmedicPharmaPlugin#update_package_trade_status

Log in Console. 

...
D, [2012-04-03T09:07:06.415950 #7983] DEBUG -- : HTTPI executes HTTP POST using the net_http adapter
D, [2012-04-03T09:07:06.818354 #7983] DEBUG -- : HTTPI executes HTTP GET using the net_http adapter
D, [2012-04-03T09:07:07.002889 #7983] DEBUG -- : HTTPI executes HTTP POST using the net_http adapter
D, [2012-04-03T09:07:07.492532 #7983] DEBUG -- : HTTPI executes HTTP GET using the net_http adapter

ean code (13) via bin/admin. 

ch.oddb> registration('62069').package('01').barcode
-> 7680620690015
ch.oddb> registration('61848').package('01').barcode
-> 7680618480017
ch.oddb>

ran with these 2 rows

Result
Checked 2 packages
Updated in trade     (out_of_trade:false): 1 packages
Updated out of trade (out_of_trade:true) : 0 packages
Updated pharmacode: 0 packages
Deleted pharmacode: 0 packages

Updated in trade     (out_of_trade:false): 1 packages
Check swissindex by eancode and then check if the package is out of trade (true) in ch.oddb,
if so the package becomes in trade (false)
7680618480017: http://ch.oddb.org/de/gcc/reg/61848/seq/01/pack/001

Updated out of trade (out_of_trade:true) : 0 packages
If there is no eancode in swissindex and the package is in trade in ch.oddb,
then the package becomes out of trade (true) in ch.oddb

Updated pharmacode: 0 packages
If the package does not have a pharmacode and there is a pharmacode found in swissindex,
then put the pharmacode into ch.oddb

Deleted pharmacode: 0 packages
If there is no eancode in swissindex then delete the according pharmacode in ch.oddb
before updater
ch.oddb> registration('61848').package('01').out_of_trade
-> true
after updater
ch.oddb> registration('61848').package('01').out_of_trade
-> false

It works in debug (logging) mode.
Then updated to be updated :out_of_trade by swissindex data whitout debug mode.

Note

to set :out_of_trade (Ja/Nein) via bin/admin 

ch.oddb> update(registration('61848').sequence.('01').package('011').pointer, {:out_of_trade => false, :refdata_override => false}, :refdata)

Attachments

Log files

Update search form type name

Updated "Normal" to new name "Plus"

Commit

Updated name of search form type Normal to Plus

GKV import in de.oddb.org

Check de.oddb.org setting

steps
settings
patches
Note

to generate with only diff
diff -Natur

Note

String of Ruby 1.8.6 doesn't have force_encoding method. 

error in SBSM::Session#to_html: /de/drugs/search/query/pfizer/dstype/company
NoMethodError
undefined method `force_encoding' for #<String:0xb5a3dae4>
/home/yasu/usr/local/lib/ruby/gems/1.8/gems/htmlgrid-1.0.5/lib/htmlgrid/composite.rb:264:in `to_html'
/home/yasu/usr/local/lib/ruby/gems/1.8/gems/htmlgrid-1.0.5/lib/htmlgrid/divcomposite.rb:42:in `to_html'
/var/www/de.oddb.org/lib/oddb/html/util/session.rb:33:in `inject'
...

Htmlgrid use force_encoding against String.

Commit

Production server use with Ruby 1.9 ?

Todo

to check ruby version, enviroment in production server.

Refs

via bin/admin 

de.oddb> Updater.import_gkv

via jobs/import_gkv 

$ ~/usr/local/bin/ruby18 -r rubygems jobs/import_gkv

does not work. 

/var/www/de.oddb.org/lib/oddb/util/updater.rb:56:in `import_gkv': undefined method `new' for Mechanize:Module (NoMethodError)
        from jobs/import_gkv:16
        from /var/www/de.oddb.org/lib/oddb/util/job.rb:16:in `call'
        from /var/www/de.oddb.org/lib/oddb/util/job.rb:16:in `run'
        from jobs/import_gkv:15

Checked gems on production server

$ gem list -d mechanize
*** LOCAL GEMS ***

mechanize (1.0.0, 0.6.11)
    Authors: Aaron Patterson, Mike Dalessio
    Rubyforge: http://rubyforge.org/projects/mechanize
    Homepage: http://mechanize.rubyforge.org
    Installed at (1.0.0): /usr/lib64/ruby/gems/1.8
                 (0.6.11): /usr/lib64/ruby/gems/1.8

    The Mechanize library is used for automating interaction with
    websites
Note
  • 1.0.0 reuire Ruby 1.8.7
  • de.oddb.org is running with Ruby 1.8.6

once, installed mechanize -v 1.0.0 with ruby-1.8.7,
copied to gems directory of Ruby 1.8.6 for de.oddb.org.

To make gem dependency with production server.

Condition
  • Ruby 1.8.6 (for de.oddb.org, with oniguruma) is $HOME/usr/local [compiledd from source]
  • Ruby 1.8.7 (for yus) is /usr [emerged]
Step
  $ ruby18 -v
  ruby 1.8.7 (2011-12-28 patchlevel 357) [i686-linux]
  $ sudo gem18 install nokogiri -v 1.4.0
  $ sudo gem18 install mechanize -v 1.0.0
  $ cd /usr/lib/ruby/gems/1.8/gems/
  nokogiri
  $ cp -r nokogiri-1.4.0 ~/usr/local/lib/ruby/gems/1.8/gems/
  $ cp ../cache/nokogiri-1.5.2.gem ~/usr/local/lib/ruby/gems/1.8/cache/
  $ cp ../specifications/nokogiri-1.4.0.gemspec ~/usr/local/lib/ruby/gems/1.8/specifications/
  mechanize
  $ cp -r mechanize-1.0.0 ~/usr/local/lib/gems/1.8/gems/
  $ cp ../specifications/mechanize-1.0.0.gemspec ~/usr/local/lib/ruby/gems/1.8/specifications/
  $ cp ../cache/mechanize-1.0.0.gem ~/usr/local/lib/ruby/gems/1.8/cache/

installed

  • nokogiri 1.4.0 (1.5.2 does not work...on Ruby 1.8.6)
  • mechanize 1.0.0
Result
$ ~/usr/local/bin/gem18 list -d mechanize

*** LOCAL GEMS ***

mechanize (1.0.0)
    Authors: Aaron Patterson, Mike Dalessio
    Rubyforge: http://rubyforge.org/projects/mechanize
    Homepage: http://mechanize.rubyforge.org
    Installed at: /home/yasu/usr/local/lib/ruby/gems/1.8

    The Mechanize library is used for automating interaction with
    websites
Warning
/home/yasu/usr/local/lib/ruby/gems/1.8/gems/nokogiri-1.4.0/lib/nokogiri/xml/builder.rb:272: warning: parenthesize argument(s) for future version

Refs

Import GKV

ran with 

$ ~/usr/local/bin/ruby18 -r rubygems jobs/import_gkv
Problem
E, [2012-04-03T16:02:14.906128 #32208] ERROR -- Gkv: certificate verify failed

folloiwng Web-site use ssl.
It seems that this error caused by ssl certificate.

This time, skipped verify check.
(There are not any other ssl protocol in de.oddb.org, this site have same contents with protocol non-ssl)

Patch

Attach:ssl-error-no-verify-solution-20120403.patch.txt

Refs

Check dbi error

Problem
ERROR:  syntax error at or near "$1"
LINE 1: ...E FROM oddb_drugs_package_code WHERE origin_id IS $1 AND cou...
                                                             ^

DBI::ProgrammingError: ERROR:  syntax error at or near "$1"
LINE 1: ...E FROM oddb_drugs_package_code WHERE origin_id IS $1 AND cou...
                                                             ^
 when updating index 'oddb_drugs_package_code' with a ODDB::Drugs::Package
/home/yasu/usr/local/lib/ruby/gems/1.8/gems/dbd-pg-0.3.9/lib/dbd/pg/statement.rb:62:in `execute'/home/yasu/usr/local/lib/ruby/gems/1.8/gems/dbi-0.4.5/lib/dbi/base_classes/database.rb:96:in `execute'/home/yasu/usr/local/lib/ruby/gems/1.8/gems/dbi-0.4.5/lib/dbi/base_classes/database.rb:114:in `do'/home/yasu/usr/local/lib/ruby/gems/1.8/gems/dbi-0.4.5/lib/dbi/handles/database.rb:106:in `do'
[...]

import_gkv jobs is running over 1 hour.

Todo

Check pg version.
It seems that above error comes from pg version.

And, Production server does not have pg in ruby 1.8.6 gems.

Related error in production server error-log.
sometimes, gems for ruby 1.9.1 are used. 

2012-04-03 15:12:21.324345500 error in SBSM::Session#process: /de/drugs/compare_remote/uid/0.764256/sortvalue/difference
2012-04-03 15:12:21.324347500 RangeError
2012-04-03 15:12:21.324347500 Unknown odba_id 764256
2012-04-03 15:12:21.324347500 (druby://localhost:10013) /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.0.8/lib/odba/drbwrapper.rb:73:in `rescue in to_obj'
2012-04-03 15:12:21.324348500 (druby://localhost:10013) /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.0.8/lib/odba/drbwrapper.rb:66:in `to_obj'
2012-04-03 15:12:21.324349500 (druby://localhost:10013) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1410:in `to_obj'
2012-04-03 15:12:21.324361500 (druby://localhost:10013) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1708:in `to_obj'
2012-04-03 15:12:21.324361500 (druby://localhost:10013) /usr/local/lib/ruby/1.9.1/drb/drb.rb:613:in `recv_request'
2012-04-03 15:14:40.971699500 can't iterate from ODDB::Dose
2012-04-03 15:15:55.851849500 undefined method `de' for #<DRb::DRbUnknown:0x0000001e7f6818>
2012-04-03 15:19:42.284386500 undefined method `de' for #<DRb::DRbUnknown:0x0000001e6de110>
2012-04-03 15:22:55.935646500 error in SBSM::Session#process: /de/drugs/search/query/voltaren/dstype/company
2012-04-03 15:22:55.935648500 NoMethodError
2012-04-03 15:22:55.935648500 undefined method `unit' for nil:NilClass
2012-04-03 15:22:55.935649500 /var/www/de.oddb.org/lib/oddb/drugs/dose.rb:93:in `<=>'
2012-04-03 15:22:55.935649500 /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:154:in `_odba_weight'
2012-04-03 15:22:55.935650500 /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:154:in `<=>'
2012-04-03 15:22:55.935650500 /var/www/de.oddb.org/lib/oddb/remote/drugs/active_agent.rb:21:in `<=>'
2012-04-03 15:22:55.935670500 /usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:154:in `_odba_weight'
2012-04-03 15:23:10.814687500 error in SBSM::Session#process: /de/drugs/search/query/voltaren/dstype/company
2012-04-03 15:23:10.814689500 NoMethodError
2012-04-03 15:23:10.814689500 undefined method `unit' for nil:NilClass
2012-04-03 15:23:10.814690500 /var/www/de.oddb.org/lib/oddb/drugs/dose.rb:93:in `<=>'
Note

druby://localhost:10013 YUS_URI (readonly)

Refs

same problem
view · edit · sidebar · attach · print · history
Page last modified on April 09, 2012, at 08:14 AM