view · edit · sidebar · attach · print · history

< Niklaus.20160601-sandoz-xmlconv | Index | Niklaus.20160530-sandoz-xmlconv >>

20160531-sandoz-xmlconv

Summary

  • Setup sandoz.xmlconv.bbmb.ch on oddb-ci2

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Order of entering search type and value should not matter. Both should show long URL with search
  • Remove parser for minifi (but keep the minifi)
  • BSV-Plugin does not send mail. Is it sending two mails at once?
  • remove obsolete pdf_patinfo in model, admin/views
  • Fix sbsm/htmlgrid when DEFAULT_FLAVOUR is nil

Setup sandoz.xmlconv.bbmb.ch on oddb-ci2

First must verify that I can log in as admin user as in http://sandoz.bbmb.ch/.

Dropped and reloaded the sandoz_yus db with the dump taken from fastpower. Took yus code from fastpower lke this:

  • git clone git://scm.ywesee.com/yus/.git /usr/local/share/src/yus
  • cd /usr/local/share/src/yus && git checkout c5f57e33a8967e9d6a9417e8af608e15215b74a4 -b fastpower

Now I am getting the following error

niklaus@oddb-ci2 /u/l/s/s/yus> sudo -u apache /usr/local/1.8.6-p388/bin/ruby18  -rubygems /usr/local/share/src/yus/bin/yusd config=/var/www/sandoz.bbmb.ch/etc/yus.yml
/usr/local/share/src/yus/lib/yus/server.rb:9: warning: already initialized constant VERSION
FEHLER:  Relation »object« existiert bereits

FEHLER:  Relation »prefetchable_index« existiert bereits

FEHLER:  Relation »extent_index« existiert bereits

FEHLER:  Relation »object_connection« existiert bereits

FEHLER:  Relation »target_id_index« existiert bereits

FEHLER:  Relation »collection« existiert bereits

/usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/marshal.rb:15:in `load': undefined class/module Encoding:: (ArgumentError)
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/marshal.rb:15:in `load'
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:629:in `restore'
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:319:in `fetch_or_restore'
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:314:in `call'
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:314:in `fetch_or_do'
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:318:in `fetch_or_restore'
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:66:in `bulk_restore'
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:63:in `each'
         ... 11 levels...
        from /usr/local/1.8.6-p388/lib/ruby/site_ruby/1.8/odba/cache.rb:297:in `fetch_named'
        from /usr/local/share/src/yus/lib/yus/persistence/odba.rb:14:in `initialize'
        from /usr/local/share/src/yus/bin/yusd:53:in `new'

On fastpower I have

ngiger@fastpower /usr/local/share/src/yus $ find  /usr/local/bin/ruby186 -name marshal.rb 
/usr/local/bin/ruby186/lib/ruby/1.8/soap/marshal.rb
/usr/local/bin/ruby186/lib/ruby/1.8/xmlrpc/marshal.rb
/usr/local/bin/ruby186/lib/ruby/gems/1.8/gems/odba-1.1.0/lib/odba/marshal.rb
ngiger@fastpower /usr/local/share/src/yus $ sha256sum /usr/local/bin/ruby186/lib/ruby/gems/1.8/gems/odba-1.1.0/lib/odba/marshal.rb
7db1f320497ff4e288ea48e7f01a52fdc310619fe0af3f43b97dcf9d0e9715a5  /usr/local/bin/ruby186/lib/ruby/gems/1.8/gems/odba-1.1.0/lib/odba/marshal.rb

Copyied /usr/local/src/ruby-1.8.6-p369.tar.gz from thinpower and building it on oddb-ci2 (+ Rubygems 1.4.1) with this script: Attach:gen_ruby_186-p369.txt

Installed the gems with the following commands

sudo /usr/local/bin/ruby186/bin/gem install character-encodings --version=0.3.0 -- --with-cflags=\"-Wno-unused-but-set-variable\"
sudo /usr/local/bin/ruby186/bin/gem install needle --version=1.3.0
sudo /usr/local/bin/ruby186/bin/gem install pg --version=0.8.0
sudo /usr/local/bin/ruby186/bin/gem install dbd-pg --version=0.3.9
sudo /usr/local/bin/ruby186/bin/gem install rclconf --version=1.0.0
sudo /usr/local/bin/ruby186/bin/gem install odba --version=1.1.0
sudo /usr/local/bin/ruby186/bin/gem install pg  --version 0.8.0
sudo /usr/local/bin/ruby186/bin/gem install dbd-pg --version=0.3.9
sudo /usr/local/bin/ruby186/bin/gem install yus --version=1.0.1

Now yusd starts when colled like this sudo -u apache /usr/local/bin/ruby186/bin/ruby -rubygems /usr/local/bin/ruby186/bin/yusd config=/var/www/sandoz.bbmb.ch/etc/yus.yml

But on fastpower I have 4 different yusd instances running

ngiger@fastpower /var/www/sandoz.bbmb.ch $ ps -ef | grep yusd
apache    2989  2975  0  2015 ?        00:33:39 /usr/local/bin/ruby186/bin/ruby -rubygems /usr/local/bin/ruby186/bin/yusd
bbmb      3001  2997  0  2015 ?        1-01:19:19 /usr/local/bin/ruby186/bin/ruby -rubygems /usr/local/share/src/yus/bin/yusd config=/var/www/sandoz.bbmb.ch//etc/yus.yml
bbmb      3022  3019  0  2015 ?        22:53:41 /usr/local/bin/ruby186/bin/ruby -rubygems /usr/local/share/src/yus/bin/yusd config=/var/www/vetoquinol.bbmb.ch//etc/yus.yml
bbmb      3034  3028  0  2015 ?        1-12:14:13 /usr/local/bin/ruby186/bin/ruby -rubygems /usr/local/share/src/yus/bin/yusd config=/var/www/virbac.bbmb.ch//etc/yus.yml

The situation in /usr/local/share/src/yus is the following. Checkout is

ngiger@fastpower /var/www/sandoz.bbmb.ch $ git log -1
commit c5f57e33a8967e9d6a9417e8af608e15215b74a4
Author: Yasuhiro Asaka <grauwoelfchen@gmail.com>
Date:   Sat Jun 15 02:26:07 2013 +0900

    Rename pg driver name to Pg

    - pg(ruby-pg) gem expects 'Pg' (not 'pg')
ngiger@fastpower /usr/local/share/src/yus $ git status -uno
# On branch master
nothing to commit (use -u to show untracked files)
ngiger@fastpower /usr/local/share/src/yus $ cat .git/config 
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = git://scm.ywesee.com/yus/.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master

Checkout out the same commit as branch fastpower on /usr/local/src/yus and now /service/sandoz.bbmb.ch-yus/ starts up correctly.

A succesfull login on http://sandoz.bbmb.ch produces entries like ths

==> /service/sandoz.bbmb.ch-yus/log/main/current <==
@40000000574d43ce2739fc0c I, [2016-05-31T09:56:52.657976 #3001]  INFO -- Yus::Server: Login attempt for xxx@sandoz.ch from ch.bbmb.sandoz
@40000000574d43ce273b28d4 I, [2016-05-31T09:56:52.658155 #3001]  INFO -- Yus::Server: Authentication succeeded for xxx@sandoz.ch
@40000000574d43ce27a6620c D, [2016-05-31T09:56:52.665172 #3001] DEBUG -- Yus::EntitySession: allowed?(login, ch.bbmb.sandoz.Admin)

==> /service/sandoz.bbmb.ch/log/main/current <==
@40000000574d43ce27b398dc D, [2016-05-31T09:56:52.665946 #11429] DEBUG -- User: xxx@sandoz.ch: allowed?(login, ch.bbmb.sandoz.Admin) -> true

==> /service/sandoz.bbmb.ch-yus/log/main/current <==
@40000000574d43ce27bbe9c4 D, [2016-05-31T09:56:52.666560 #3001] DEBUG -- Yus::EntitySession: allowed?(login, ch.bbmb.sandoz.Customer)

==> /service/sandoz.bbmb.ch/log/main/current <==
@40000000574d43ce27e9376c D, [2016-05-31T09:56:52.669520 #11429] DEBUG -- User: admin@sandoz.ch: allowed?(login, ch.bbmb.sandoz.Customer) -> false
@40000000574d43cf1054afdc D, [2016-05-31T09:56:53.273843 #11429] DEBUG -- State: Customers#init: loaded 1661 customers in 0.60362s

Where as on oddb-ci2 it fails like this

==> /service/sandoz.bbmb.ch-yus/log/main/current <==
@40000000574d40cf39159814 I, [2016-05-31T09:44:05.957637 #6779]  INFO -- Yus::Server: Login attempt for xxx@sandoz.ch from ch.bbmb
@40000000574d40cf3917035c I, [2016-05-31T09:44:05.957775 #6779]  INFO -- Yus::Server: Authentication succeeded for xxx@sandoz.ch
@40000000574d40cf3a06d1fc D, [2016-05-31T09:44:05.973448 #6779] DEBUG -- Yus::EntitySession: allowed?(login, ch.bbmb.Admin)

==> /service/sandoz.bbmb.ch/log/main/current <==
@40000000574d40cf3a0e731c D, [2016-05-31T09:44:05.973949 #7956] DEBUG -- User: xxx@sandoz.ch: allowed?(login, ch.bbmb.Admin) -> false

It looks as if some msiconfiguratioin. On fastpower bbmb looks for ch.bbmb.sandoz.Admin whereas here it looks for ch.bbmb.Admin. Found it. I uncommented the corresponding line for the auth_domain in /var/www/sandoz.bbmb.ch/etc/config.yml. But now I get a [2016-05-31T10:07:30.792452 #6779] WARN -- Yus::Server: Authentication failed for admin@sandoz.ch Addes user bbmb to group postgres. And now I find Yus::Server: Authentication succeeded for xxx@sandoz.ch. But the Apache configuration is still wrong I see mod_ruby: failed to require sbsm/trans_handler. Made this problem going away by correcting the RubyAddPath. Now I can login via http://sandoz.oddb-ci2.dyndns.org/de/customers/, but after the login I get the error ERR_TOO_MANY_REDIRECTS. And in the /var/www/sandoz.bbmb.ch/log/access_log I see many entries like 172.25.1.75 - - [31/May/2016:11:18:49 +0200] "GET /de/customers/ HTTP/1.1" 302 - "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.36 Safari/537.36" Also in the /service/sandoz.bbmb.ch-yus/log/main/current I see several occurrences of Server: Authentication succeeded for xxx@sandoz.ch. What is happening? Looks that we have a problem handling the states in bbmb, as lib/bbmb/html/state/customers.rb is responsible for printing State: Customers#init: loaded 1661 customers in 0.60362s

Apache config looks this Attach:apache_sandoz_bbmb_ch.txt

Simplyfied the apache_sandoz_bbmb_ch.conf to retain only the relevant stuff of sandoz.bbmb.ch. Tried changing the servername to sandoz.localhost. This did not work. Changed it to sandoz.oddb-ci2.dyndns.org and it work again.

Changed to RubyAddPath /usr/local/lib/ruby/gems/2.3.0/gems/sbsm-1.2.7/lib/sbsm/trans_handler.rb. Now I am getting the error

[Tue May 31 14:37:18 2016] [error] mod_ruby: error in ruby                                     
[Tue May 31 14:37:18 2016] [error] mod_ruby: /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:62:in `file?': Insecure operation - file? (SecurityError)
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:62:in `block (2 levels) in contains_requirable_file?'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:62:in `each'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:62:in `any?'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:62:in `block in contains_requirable_file?'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:60:in `each'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:60:in `any?'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/basic_specification.rb:60:in `contains_requirable_file?'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:925:in `block in find_inactive_by_path'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:924:in `each'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:924:in `find'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:924:in `find_inactive_by_path'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems.rb:185:in `try_activate'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:132:in `rescue in require'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:144:in `require'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /var/www/sandoz.bbmb.ch/doc/index.rbx:3:in `<top (required)>'
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/1.9.1/apache/ruby-run.rb:53:in `load'         
[Tue May 31 14:37:18 2016] [error] mod_ruby:   from /usr/local/lib/ruby/1.9.1/apache/ruby-run.rb:53:in `handler'
[Tue May 31 14:37:18 2016] [error] [client 172.25.1.75] File does not exist: /var/www/sandoz.bbmb.ch/doc/favicon.ico, referer: http://sandoz.oddb-ci2.dyndns.org/

Attach:apache_sandoz_bbmb_ch2.txt

Trying to fix the error using untaint did not help in April when I ported ydim to Ruby 2.3.0. The RubyAddPath directive must only include the modules used for the mod_ruby (running under Ruby 1.9 in our case). Then a DRB session must be opened. Therefor I compare the ydim/doc/index.rbx with the sandoz.bbmb.ch/doc/index.rb. The Apache configuration differs because ydim uses https, and sandoz.bbmb.ch uses http.

Trying to replace "DRb.start_service('druby://localhost:0')" with "::DRb.start_service('druby://localhost:0')" in the doc/index.rbx. (I already had removed the require 'encoding/character/utf-8'

Reverting to use the apache_sandoz_bbmb_ch.txt. Slowly uncommenting lines to see what breaks loading the the doc/index.rbx. Only difference now is RubyAddPath /usr/local/lib64/ruby/gems/1.9.1/gems/sbsm-1.2.3/lib instead of RubyAddPath /usr/local/lib/ruby/gems/2.3.0/gems/sbsm-1.2.7/lib/sbsm/trans_handler.rb

Stil having the question why in /service/sandoz.bbmb.ch-yus/log/main/current I see after each login 8 times

@40000000574d98ad087a95ec D, [2016-05-31T15:58:59.142194 #18462] DEBUG -- User: xxx@sandoz.ch: allowed?(login, ch.bbmb.sandoz.Admin) -> true
@40000000574d98ad0886461c D, [2016-05-31T15:58:59.142981 #18462] DEBUG -- User: xx@sandoz.ch: allowed?(login, ch.bbmb.sandoz.Customer) -> false
@40000000574d98ad147cf114 D, [2016-05-31T15:58:59.343660 #18462] DEBUG -- State: Customers#init: loaded 1661 customers in 0.20024s

When looking at etc/ydim.yml I see the etc/ydim.yml file which specified a YDIM-URL. Adapting it to oddb-ci2 and trying to start the /service/ydim before starting sandoz.bbmb.ch. The services ydim and ydim-html are running and I can the the port 12375 using

niklaus@oddb-ci2 /v/w/sandoz.bbmb.ch> sudo netstat -lptu | grep 12375
tcp6       0      0 oddb-ci2.ngiger.d:12375 [::]:*                  LISTEN      29359/bin/ydimd     

Remarked that http://oddb-ci2.dyndns.org/ displays the same stuff as http://sandoz.oddb-ci2.dyndns.org/.

Found a /usr/local/share/src/yus/bin/yusd config=/var/www/sandoz.bbmb.ch/etc/yus.yml belonging to the apache user, which was not killed correctly this morning. Killed it with -9. Shutting down /service/yus to ensure that it does not answer any request from sandoz.

Renamed the Servername to sandoz.bbmb.ngiger.ch. Removed duplicated LoadModule ruby_module in /etc/apache/vhost.d/*.conf files.

Tasks for tomorrow are:

  • Do we need to grant yus entities as described by Mase in http://dev.ywesee.com/Masa/20101110-setup-eshop?
  • Copy sandoz.xmlconf files from fastpower, generate a git repository and try to convert it into a gem
  • Fix skipped unit tests in bbmb
  • Try to make the selenium test run under bbmb. If successfull also in sandoz.bbmb.ch and sandoz.xmlconf
view · edit · sidebar · attach · print · history
Page last modified on May 31, 2016, at 05:16 PM