view · edit · sidebar · attach · print · history

Index>

20160518-sandoz-bbmb

Summary

  • Setup sandoz.bbmb.ch on oddb-ci2

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Import via data/medreg_companies.yaml
  • Fix problem with radioactivatum 99m-technetio when parsing Wirkstoffe
  • Fix galenic_forms when parsing swissmedic.xlsx
  • Cleanup generic_type. Replace it everywhere by sl_generic_type and adapt code accordingly.
  • Get updated ATC-codes from EPha for oddb.org, too.
  • Use refdatabase for oddb.org, too.
  • Check whether we should revert the part which touche src/plugin/text_info.rb of commit 17af82ba4d76a5838683411b260de265531f9e74. We should improve test/stub/oddbapp.rb to work similar for update/pointer as the real oddbapp. In this case we would have a good Stub for plugins. May we need a different stub when working with plugins (which create/modify/destroy ODDB-Objects), when in most other cases a very simple stub is sufficient.
  • When a logged in admin user changes an atc_code of a product, the corresponding atc_class must update its sequences, too.
  • Order of entering search type and value should not matter. Both should show long URL with search
  • Remove parser for minifi (but keep the minifi)
  • BSV-Plugin does not send mail. Is it sending two mails at once?
  • remove obsolete pdf_patinfo in model, admin/views
  • Fix sbsm/htmlgrid when DEFAULT_FLAVOUR is nil

Setup sandoz.bbmb.ch on oddb-ci2

Creating a script to get the backup of all databases from thinpower. See Attach:get_fastpower_db_dumps.rb.txt

Copied /etc/apache2/vhosts.d/sandoz.bbmb.ch.conf from fastpowet to oddb-ci2 and adapting it slightly. Before starting apache, cloned git@github.com:ngiger/sandoz.bbmb.ch.git to /var/www/sandoz.bbmb.ch and ran 2@sudo -u apache mkdir /var/www/sandoz.bbmb.ch/webalizer@@ Now I am getting a Internal Server Error as my ruby path are not yet correct. Looking at /var/www/sandoz.bbmb.ch/log/error_log I see, that the /etc/trans_handler.yml is missing. After changing the RubyAddPath as for ydim I get /resources/errors/appdown.html, as the DOCUMENT_ROOT/../etc/trans_handler.yml is not loaded correctly.

Had to remove the line require 'encoding/character/utf-8' in doc/index.rbx.

Now I must bring the up the following services:

  • /service/sandoz.bbmb.ch-yus/run
  • /service/sandoz.bbmb.ch/run
  • /service/sandoz.xmlconv.bbmb.ch/run

Copied all files from /var/www/sandoz.bbmb.ch/etc. Adapted etc/ydim.yml to make client and server use my IP.

Called sudo useradd bbmb and /usr/local/bin/gem231 install yus.

Trying to start yus with /usr/local/bin/ruby231 via sudo -u bbmb /usr/local/bin/ruby231 /usr/local/bin/yusd config=/var/www/sandoz.bbmb.ch/etc/yus.yml. I am getting the following error 

sudo -u bbmb /usr/local/bin/yusd config=/var/www/sandoz.bbmb.ch/etc/yus.yml
Ignoring byebug-8.2.4 because its extensions are not built.  Try: gem pristine byebug --version 8.2.4
Ignoring ffi-1.9.10 because its extensions are not built.  Try: gem pristine ffi --version 1.9.10
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `initialize' may cause serious problems
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `object_id' may cause serious problems
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `__send__' may cause serious problems
/usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.2/lib/dbi.rb:317:in `rescue in load_driver': Could not load driver (uninitialized constant DBI::VERSION (DBI::InterfaceError)
Did you mean?  VERSION)
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.2/lib/dbi.rb:241:in `load_driver'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.2/lib/dbi.rb:159:in `_get_full_driver'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.2/lib/dbi.rb:144:in `connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:63:in `block in _connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:62:in `times'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:62:in `_connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:59:in `block in connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:59:in `synchronize'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:59:in `connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:22:in `initialize'
        from /usr/local/lib/ruby/gems/2.3.0/gems/yus-1.0.2/bin/yusd:51:in `new'
        from /usr/local/lib/ruby/gems/2.3.0/gems/yus-1.0.2/bin/yusd:51:in `<top (required)>'
        from /usr/local/bin/yusd:23:in `load'
        from /usr/local/bin/yusd:23:in `<main>'

Tried with another version of ruby. Called

  • /var/www/.rbenv/versions/1.8.7-p375/bin/gem install rclconf
  • /var/www/.rbenv/versions/1.8.7-p375/bin/gem install odba --version 1.1.0
  • /var/www/.rbenv/versions/1.8.7-p375/bin/gem install pg --version=0.8.0
  • /var/www/.rbenv/versions/1.8.7-p375/bin/gem install dbd-pg --version=0.3.9

Now I am able to start /service/yus/run containing 

#!/bin/sh
exec 2>&1
ulimit -v 1536000
cd /usr/local/src/yus18
exec sudo -u apache /var/www/.rbenv/versions/1.8.7-p375/bin/ruby bin/yusd

I can work around the missing DBI::VERSION by adding a line require 'dbi/version' in /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.2/lib/dbi.rb.

Now I am getting the error 

sudo -u apache /usr/local/bin/yusd config=/var/www/sandoz.bbmb.ch/etc/yus.yml
Ignoring byebug-8.2.4 because its extensions are not built.  Try: gem pristine byebug --version 8.2.4
Ignoring ffi-1.9.10 because its extensions are not built.  Try: gem pristine ffi --version 1.9.10
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `initialize' may cause serious problems
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `object_id' may cause serious problems
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `__send__' may cause serious problems
FEHLER:  Relation »object« existiert bereits
FEHLER:  Relation »prefetchable_index« existiert bereits
FEHLER:  Relation »extent_index« existiert bereits
FEHLER:  Relation »object_connection« existiert bereits
FEHLER:  Relation »target_id_index« existiert bereits
FEHLER:  Relation »collection« existiert bereits
/usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/marshal.rb:15:in `load': undefined class/module Encoding::Character:: (ArgumentError)
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/marshal.rb:15:in `load'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:629:in `restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:319:in `block in fetch_or_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:314:in `fetch_or_do'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:318:in `fetch_or_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:66:in `block in bulk_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:63:in `each'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:63:in `bulk_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:57:in `bulk_fetch'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:261:in `fetch_collection'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:633:in `restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:319:in `block in fetch_or_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:314:in `fetch_or_do'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:318:in `fetch_or_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:305:in `block in fetch_named'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:314:in `fetch_or_do'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:297:in `fetch_named'
        from /usr/local/lib/ruby/gems/2.3.0/gems/yus-1.0.2/lib/yus/persistence/odba.rb:14:in `initialize'
        from /usr/local/lib/ruby/gems/2.3.0/gems/yus-1.0.2/bin/yusd:54:in `new'
        from /usr/local/lib/ruby/gems/2.3.0/gems/yus-1.0.2/bin/yusd:54:in `<top (required)>'
        from /usr/local/bin/yusd:23:in `load'
        from /usr/local/bin/yusd:23:in `<main>'

Comes because the sandoz-yus database is not UTF-8. Must I create a migrate script for it? Starting with an empty database works fine.

Pushed commit [https://github.com/ngiger/ydbi/commit/e49a392b92d3ef67caa8ce9f7b9c24cfaa517bd4|Added missing require 'dbi/version' to fix errors when starting yus]]

Now trying to start sandoz using sudo -u bbmb /usr/local/bin/ruby231 /var/www/sandoz.bbmb.ch/bin/bbmbd config=/var/www/sandoz.bbmb.ch/etc/config.yml. Fails miserably, as the bin directory is not part of the git checkout. Copied from fastpower. Now installing sudo /usr/local/bin/gem231 install /home/niklaus/git/bbmb/pkg/bbmb-2.0.1.gem. After removing the require 'encoding' in bin/bbmbd, I face an error connecting to the PostgreSQL. 

 sudo -u bbmb /usr/local/bin/ruby231 /var/www/sandoz.bbmb.ch/bin/bbmbd config=/var/www/sandoz.bbmb.ch/etc/config.yml
Ignoring byebug-8.2.4 because its extensions are not built.  Try: gem pristine byebug --version 8.2.4
/var/www/sandoz.bbmb.ch/bin/bbmbd:3: warning: variable $KCODE is no longer effective; ignored
Ignoring ffi-1.9.10 because its extensions are not built.  Try: gem pristine ffi --version 1.9.10
could not find htmlgrid.so, falling back to pure-ruby class
/usr/local/lib/ruby/gems/2.3.0/gems/ydbd-pg-0.5.2/lib/dbd/pg/database.rb:82:in `rescue in initialize': could not connect to server: Permission denied (DBI::OperationalError)
        Is the server running locally and accepting
        connections on Unix domain socket "/run/postgresql/.s.PGSQL.5432"?
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbd-pg-0.5.2/lib/dbd/pg/database.rb:41:in `initialize'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbd-pg-0.5.2/lib/dbd/Pg.rb:157:in `new'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbd-pg-0.5.2/lib/dbd/Pg.rb:157:in `connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.2/lib/dbi/handles/driver.rb:33:in `connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.2/lib/dbi.rb:149:in `connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:63:in `block in _connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:62:in `times'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:62:in `_connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:59:in `block in connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:59:in `synchronize'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:59:in `connect'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:22:in `initialize'
        from /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.1/lib/bbmb/persistence/odba.rb:39:in `new'
        from /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.1/lib/bbmb/persistence/odba.rb:39:in `<module:BBMB>'
        from /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.1/lib/bbmb/persistence/odba.rb:14:in `<top (required)>'
        from /usr/local/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/local/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /var/www/sandoz.bbmb.ch/bin/bbmbd:17:in `<module:BBMB>'
        from /var/www/sandoz.bbmb.ch/bin/bbmbd:13:in `<main>'

When connecting as user apache (instead of bbmb) I get an error about the missing db sandoz_bbmb. Creating a script to load this database see Attach:load_sandoz_bb.sh.txt.

Now I am getting the following error when calling sudo -u apache /usr/local/bin/ruby231 /var/www/sandoz.bbmb.ch/bin/bbmbd config=/var/www/sandoz.bbmb.ch/etc/config.yml 

Ignoring byebug-8.2.4 because its extensions are not built.  Try: gem pristine byebug --version 8.2.4
/var/www/sandoz.bbmb.ch/bin/bbmbd:3: warning: variable $KCODE is no longer effective; ignored
Ignoring ffi-1.9.10 because its extensions are not built.  Try: gem pristine ffi --version 1.9.10
could not find htmlgrid.so, falling back to pure-ruby class
FEHLER:  Relation »object« existiert bereits
FEHLER:  Relation »prefetchable_index« existiert bereits
FEHLER:  Relation »extent_index« existiert bereits
FEHLER:  Relation »object_connection« existiert bereits
FEHLER:  Relation »target_id_index« existiert bereits
FEHLER:  Relation »collection« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_product_ean13« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_product_pcode« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_customer_ean13« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_customer_email« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_product_catalogue1« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_product_catalogue2« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_product_description« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_customer_customer_id« existiert bereits
FEHLER:  Relation »target_id_bbmb_model_product_article_number« existiert bereits
/usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/marshal.rb:15:in `load': undefined class/module Encoding::Character:: (ArgumentError)
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/marshal.rb:15:in `load'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:629:in `restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:319:in `block in fetch_or_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:314:in `fetch_or_do'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:318:in `fetch_or_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:66:in `block in bulk_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:63:in `each'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:63:in `bulk_restore'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:417:in `prefetch'
        from /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.1/lib/bbmb/persistence/odba.rb:42:in `<module:BBMB>'
        from /usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.1/lib/bbmb/persistence/odba.rb:14:in `<top (required)>'
        from /usr/local/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:68:in `require'
        from /usr/local/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:68:in `require'
        from /var/www/sandoz.bbmb.ch/bin/bbmbd:17:in `<module:BBMB>'
        from /var/www/sandoz.bbmb.ch/bin/bbmbd:13:in `<main>'

Tried to create a migration script. Getting stuck when lauching with the same config as above sudo -u apache /usr/local/bin/ruby231 /var/www/sandoz.bbmb.ch/bin/migrate_to_utf_8 config=/var/www/sandoz.bbmb.ch/etc/config.yml 

/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `initialize' may cause serious problems
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `object_id' may cause serious problems
/usr/local/lib/ruby/gems/2.3.0/gems/needle-1.3.0/lib/needle/definition-context.rb:36: warning: undefining `__send__' may cause serious problems
2016-05-18 15:48:29 +0200: Calling migrate_to_utf8
FEHLER:  Relation &#65533;object&#65533; existiert bereits
FEHLER:  Berechtigung nur f&#65533;r Eigent&#65533;mer der Relation object
FEHLER:  Berechtigung nur f&#65533;r Eigent&#65533;mer der Relation object
FEHLER:  Relation &#65533;object_connection&#65533; existiert bereits
FEHLER:  Berechtigung nur f&#65533;r Eigent&#65533;mer der Relation object_connection
FEHLER:  Relation &#65533;collection&#65533; existiert bereits
FEHLER:  keine Berechtigung f&#65533;r Relation object
/usr/local/lib/ruby/gems/2.3.0/gems/ydbd-pg-0.5.3/lib/dbd/pg/statement.rb:62:in `rescue in execute': FEHLER:  keine Berechtigung f&#65533;r Relation object (DBI::ProgrammingError)
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbd-pg-0.5.3/lib/dbd/pg/statement.rb:37:in `execute'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.3/lib/dbi/base_classes/database.rb:96:in `execute'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.3/lib/dbi/handles/database.rb:81:in `execute'
        from /usr/local/lib/ruby/gems/2.3.0/gems/ydbi-0.5.3/lib/dbi/handles/database.rb:115:in `select_one'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:39:in `block in method_missing'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:29:in `next_connection'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/connection_pool.rb:38:in `method_missing'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/storage.rb:513:in `restore_named'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:298:in `block in fetch_named'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:314:in `fetch_or_do'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:297:in `fetch_named'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:352:in `indices'
        from /usr/local/lib/ruby/gems/2.3.0/gems/odba-1.1.2/lib/odba/cache.rb:475:in `setup'
        from /var/www/sandoz.bbmb.ch/bin/migrate_to_utf_8:173:in `<main>'

After sudo -u postgres psql -c "alter user sandoz with superuser;" I got farther. But I have problems finding the right injection point for the migration. In ydim I could use functions like debitors in lib/ydim/root_session.rb which looked like this 

    def debitors
      @serv.logger.debug(whoami) { "debitors" } if @serv &&  @serv.respond_to?(:logger)
      Debitor.odba_extent
    end

In bbmb I do not have a RootSession class. Must I monkey patch it? Looks like. But I do not come much farther, as I see Start migrating 0 orders

Pushed commit in ydim Log number of items to migrate

Saved work by pushing:

view · edit · sidebar · attach · print · history
Page last modified on May 18, 2016, at 05:25 PM