view · edit · sidebar · attach · print · history

Index>

20140120-hpc-parser-does-not-find-recent-entry

Summary

  • HPC-Parser did not find recent entry Entumin
  • Adding entries under bin with bundle install and bundle exec
  • Error: swissmedic - 01/2014 when running import_daily on thinpower
  • installed etckeeper

Commits

Index

---

installed etckeeper

As a fallback when some changes under /etc have fatal consequences, I always install etckeeper on my machines. Zeno agreed and as root I did the following stuff

emerge etckeeper
cd /etc
git config --global user.email "zdavatz@ywesee.com"
git config --global user.name "Zeno Davatz"
etckeeper init
echo webalizer/dns_cache.db >> .gitignore
git add .gitignore
git commit -m "Initial import. .gitignore ignore webalizer/dns_cache.db" -a

HPC-Parser did not find recent entry Entumin

Zeno remarked that even after chaning the name of the link the new entry https://www.swissmedic.ch/marktueberwachung/00135/00136/00140/00142/00184/01835/index.html?lang=de was not found. Must fix this problem.

We should parse https://www.swissmedic.ch/marktueberwachung/00135/index.html?lang=de and find the link "15.01.2014 Entumin 40 mg, Tabletten Out-of-Stock – befristetes Inverkehrbringen in ausländischer Aufmachung." Adding some debug output and running only jobs/update_hpc_feed.

Adding entries under bin with bundle install and bundle exec

As we are slowly migrating to use bundle on thinpower it is important, that alls jobs scheduled via crontab entries

  1. execute first bundle install uptodate version of all gems
  2. then execute bundle exec use exactly these version of gems
  3. redirect their stderr/stdout to /dev/null (if not they produce way too much output and overflow the log)

These are:

  1. /var/www/oddb.org/jobs/import_daily >/dev/null 2>&1
  2. /var/www/oddb.org/jobs/export_fachinfo_yaml
  3. /var/www/oddb.org/jobs/export_patinfo_yaml
  4. /var/www/oddb.org/jobs/mail_index_therapeuticus_csv

Each jobs look like this

#!/bin/bash -v
cd `dirname $0`
bundle install
bundle exec ruby ../jobs/mail_index_therapeuticus_csv  >/dev/null 2>&1

I added the verbose flag to bash as personally I like to see what commands are executed if run the job manually.

See commit Added some jobs with bundle install/exec for crontab

Questions:

  • Should we add an @ulimit -v 10240000@ ? (NO. We have solved the problem inside Ruby)
  • Should we add a @set -e@ to terminate with an error if any command fails, eg. bundle import? Yes.

I have no a redundancy with vagrant-oddb where I create /etc/cron.daily/scripts.

Specifying bundle exec in the jobs/import_daily does not seem to be easy. With the sheban line @#!/usr/local/bin/bundle exec ruby@ I get the following error

pwd; sudo -u apache /usr/local/bin/bundle exec ruby jobs/import_swissmedic 
/var/www
Could not locate Gemfile

pwd; sudo -u apache /usr/local/bin/ruby /var/www/oddb.org/jobs/import_swissmedic 
/var/www
/usr/lib64/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:53:in `require': iconv will be deprecated in the future, use String#encode instead.
could not find htmlgrid.so, falling back to pure-ruby class
/usr/lib64/ruby/site_ruby/1.9.1/rubygems/specification.rb:2007:in `raise_if_conflicts': Unable to activate mechanize-2.7.3, because mime-types-1.25 conflicts with mime-types (~> 2.0) (Gem::LoadError)
        from /usr/lib64/ruby/site_ruby/1.9.1/rubygems/specification.rb:1176:in `activate'
        from /usr/lib64/ruby/site_ruby/1.9.1/rubygems.rb:186:in `rescue in try_activate'
        from /usr/lib64/ruby/site_ruby/1.9.1/rubygems.rb:183:in `try_activate'

Therefore Zeno and me decided to switch to use bundler for all services and crontab. As a consequence one should always start jobs under /var/www/oddb.org/jobs like this

cd /var/www/oddb.org
sudo -u apache bundle install --deployment 
exec sudo -u apache bundle exec /usr/local/bin/ruby bin/oddbd 

Calling bundle install does not add much to the startup, as it is a completly local operation which

Die Crontab-Jobs sehen nun wie folgt aus:

1 9 * * *       apache cd /var/www/oddb.org && bundle exec /usr/local/bin/ruby jobs/import_daily >/dev/null 2>&1
1 2 * * *       apache cd /var/www/oddb.org && bundle exec /usr/local/bin/ruby jobs/export_daily >/var/log/export_daily.log >&1
1 6 28 * *      apache cd /var/www/oddb.org && bundle exec /usr/local/bin/ruby jobs/export_fachinfo_yaml
1 6 27 * *      apache cd /var/www/oddb.org && bundle exec /usr/local/bin/ruby jobs/export_patinfo_yaml
1 7 25 8 *      apache cd /var/www/oddb.org && bundle exec /usr/local/bin/ruby jobs/mail_index_therapeuticus_csv

I am not 100% sure whether we must replace bundle by usr/local/bin/bundle or not. Trying without.

Reverted my commit to add some jobs to bin with Revert "Added some jobs with bundle install/exec for crontab"

Error: swissmedic - 01/2014 when running import_daily on thinpower

We get the following error on thinpower

Plugin: ODDB::SwissmedicPlugin
Error: Ole::Storage::FormatError
Message: OLE2 signature is invalid
Backtrace:
/usr/local/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.7/lib/ole/storage/base.rb:378:in `validate!'
/usr/local/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.7/lib/ole/storage/base.rb:370:in `initialize'
/usr/local/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.7/lib/ole/storage/base.rb:112:in `new'
/usr/local/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.7/lib/ole/storage/base.rb:112:in `load'
/usr/local/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.7/lib/ole/storage/base.rb:79:in `initialize'
/usr/local/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.7/lib/ole/storage/base.rb:85:in `new'
/usr/local/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.7/lib/ole/storage/base.rb:85:in `open'
/usr/local/lib/ruby/gems/1.9.1/gems/spreadsheet-0.9.6/lib/spreadsheet/excel/reader.rb:1276:in `setup'
/usr/local/lib/ruby/gems/1.9.1/gems/spreadsheet-0.9.6/lib/spreadsheet/excel/reader.rb:134:in `read'
/usr/local/lib/ruby/gems/1.9.1/gems/spreadsheet-0.9.6/lib/spreadsheet/excel/workbook.rb:32:in `open'
/usr/local/lib/ruby/gems/1.9.1/gems/spreadsheet-0.9.6/lib/spreadsheet.rb:63:in `open'
/usr/local/lib/ruby/gems/1.9.1/gems/spreadsheet-0.9.6/lib/spreadsheet.rb:66:in `block in open'
/usr/local/lib/ruby/gems/1.9.1/gems/spreadsheet-0.9.6/lib/spreadsheet.rb:65:in `open'
/usr/local/lib/ruby/gems/1.9.1/gems/spreadsheet-0.9.6/lib/spreadsheet.rb:65:in `open'
/var/www/oddb.org/src/plugin/swissmedic.rb:268:in `initialize_export_registrations'
/var/www/oddb.org/src/plugin/swissmedic.rb:57:in `update'
/var/www/oddb.org/src/util/updater.rb:393:in `block in update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:495:in `call'
/var/www/oddb.org/src/util/updater.rb:495:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:391:in `update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:196:in `run'
/var/www/oddb.org/jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
/var/www/oddb.org/jobs/import_daily:12:in `<module:Util>'
/var/www/oddb.org/jobs/import_daily:11:in `<module:ODDB>'
/var/www/oddb.org/jobs/import_daily:10:in `<main>'

Reason: Did run once an import_daily when not all of my changes were integrated. Must move /var/www/oddb.org/data/xlsPräparateliste-latest.xlsx out of the way. Done using

cd /var/www/oddb.org/data/xls 
mv Präparateliste-latest.xlsx Präparateliste-latest.xlsx.caused_errors

interaction link for epha with barcode and iksnr

see also task interaction link

It was easy to extend the search using not only atc, but also iksnr and ean13. Therefore locally the following links work:

  1. http://oddb-ci2.dyndns.org/de/gcc/interaction_search/atc/C07AA05,N02BA01
  2. http://oddb-ci2.dyndns.org/de/gcc/interaction_search/iksnr/39053,58392,51795
  3. http://oddb-ci2.dyndns.org/de/gcc/interaction_search/ean13/7680317061142,7680583920112

Now I must fix the problem, that manually adding more items does not update the link. As after selecting one sees the barcode in the value to add, I think it should be obvious for newcomers, when we use afterwards always links for the third form ean13/ean_1,ean_2,ean_3

The problem lies that the ajax script in src/view/searchbar.rb returns only a /ean13/id value and does not take into account the existing search_path. Therefore changing the displayed URL to reflect the state of all selected items, will be quite time consuming and means digging even deeper.

view · edit · sidebar · attach · print · history
Page last modified on January 20, 2014, at 10:15 PM