view · edit · sidebar · attach · print · history

20110817-migel-drb-server

<< | Index | >>

Log update_migel_nonpharma last night suspend
Implement search function by artcle name suspend
Create a new database for migel
Create a new (simplest) drb server for migel
Set up bin/admin
Set up logger
Define migel data class (Migel::Model::Group)
How to save an instance
How to delete an instance
Define migel data class (Migel::Model::Subgroup)

Goal/Estimate/Evaluation

Migel new database and drb server / 80% / 70%

Milestones

~~Create a new database~~
~~Create a new drb server (simplest)~~
Decoration drb server
1. ~~bin/admin~~
2. Migel data class
Import and show some migel data
Search by article name suspend
Search by company name suspend
Comparison with cost suspend

Summary

http://masay.ch/2011/08/17/developed-migel-drb-server/

Commits
ToDo Tomorrow

Log update_migel_nonpharma last night

bin/admin

ch.oddb> ODDB::MiGeLPlugin.new(self).update_items_by_migel true
-> undefined method `update_values' for []:Array

Console

474 / 571       Estimate total: 7.38 [h] It will be done in: 1.25 [h]
475 / 571       Estimate total: 7.37 [h] It will be done in: 1.24 [h]
failsafe rescued NoMethodError < StandardError
undefined method `update_values' for []:Array
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1434:in `_admin'
/usr/lib64/ruby/site_ruby/1.8/odba/stub.rb:141:in `method_missing'
/home/masa/ywesee/oddb.org/src/util/language.rb:66:in `update_values'
/home/masa/ywesee/oddb.org/src/util/persistence.rb:278:in `issue_update'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:119:in `update'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:118:in `update'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1415:in `update'
/home/masa/ywesee/oddb.org/src/plugin/migel.rb:257:in `update_item'
/home/masa/ywesee/oddb.org/src/plugin/migel.rb:195:in `update_items_by_migel'
/home/masa/ywesee/oddb.org/src/plugin/migel.rb:193:in `each'
/home/masa/ywesee/oddb.org/src/plugin/migel.rb:193:in `update_items_by_migel'
/usr/lib64/ruby/1.8/drb/drb.rb:557:in `each_with_index'
/home/masa/ywesee/oddb.org/src/plugin/migel.rb:189:in `each'
/home/masa/ywesee/oddb.org/src/plugin/migel.rb:189:in `each_with_index'
/home/masa/ywesee/oddb.org/src/plugin/migel.rb:189:in `update_items_by_migel'
(eval):1:in `_admin'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call'
/home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1433:in `_admin'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1431:in `initialize'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1431:in `new'
/home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1431:in `_admin'
/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `__send__'
/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'
/usr/lib64/ruby/1.8/drb/drb.rb:1515:in `perform'
/usr/lib64/ruby/1.8/drb/drb.rb:1589:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1430:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1347:in `initialize'
/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `new'
/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `start_service'
bin/oddbd:38

Check all the item instances

ch.oddb> c=0; migel_products.each{|pr| if items=pr.items then c+=items.values.select{|it| it.is_a?(ODDB::Migel::Item)}.length end}; c
-> 29943
ch.oddb> c=0; migel_products.each{|pr| if items=pr.items then c+=items.values.select{|it| (it.is_a?(Array) or it.is_a?(Part) or it.is_a?(Package) or it.is_a?(ActiveAgent) or it.is_a?(Composition) or it.is_a?(Hash) or it.is_a?(Sequence)) and it.is_a?(ODDB::Migel::Item)}.length end}; c
-> 41

Note

There are still some wrong data
They are not overwritten by Migel::Item instance

suspend

Implement search function by artcle name

Experiment

src/state/global.rb#search
src/util/session.rb#search_migel_companies
src/util/oddbapp.rb#search_migel_companies

suspend

Create a new database for migel

masa@masa ~/ywesee $ sudo -u postgres createdb -E UTF8 -T template0 migel
Passwort: 
masa@masa ~/ywesee $ psql -U postgres
psql (8.4.2)
Geben Sie �help� f�r Hilfe ein.

postgres=# \l
                                      Liste der Datenbanken
        Name        | Eigent�mer | Kodierung | Sortierfolge | Zeichentyp |    Zugriffsrechte     
--------------------+-------------+-----------+--------------+------------+-----------------------
 ch_oddb_org        | postgres    | UTF8      | C            | C          | 
 migel              | postgres    | UTF8      | C            | C          | 
...

Note

I expect that the tables (and relations) are created automatically by ODBA
The owner of the ch_oddb_org on the server is not 'postgres', which is related to the db_connection.yml file (ODBA setting)

Create a new (simplest) drb server for migel

bin/migeld

#!/usr/bin/env ruby
# encoding: utf-8
# MigelDrbServer -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

$: << File.expand_path('../lib', File.dirname(__FILE__))

require 'rclconf'
require 'drb'
require 'migel/server'

default_dir = File.expand_path('../etc', File.dirname(__FILE__))
default_config_files = [
  File.join(default_dir, 'migel.yml'), '/etc/migel/migel.yml',
]
defaults = {
  'config'            => default_config_files,
  'db_name'           => 'migel',
  'db_user'           => 'migel',
  'db_auth'           => 'migel',
  'persistence'       => 'odba',
  'server_url'        => 'druby://localhost:11000',
  'migel_dir'         => default_dir,
}

config = RCLConf::RCLConf.new(ARGV, defaults)
config.load(config.config)

require File.join('migel', 'persistence', config.persistence)
persistence = nil
require 'odba/connection_pool'
require 'odba/drbwrapper'
DRb.install_id_conv ODBA::DRbIdConv.new
ODBA.storage.dbi = ODBA::ConnectionPool.new("DBI:pg:#{config.db_name}", config.db_user, config.db_auth)
ODBA.cache.setup
persistence = Migel::Persistence::Odba.new

begin
  #server = Yus::Server.new(persistence, config, logger)
  server = Migel::Server.new
  server.extend(DRbUndumped)

  url = config.server_url
  drbconf = {}
  url.untaint
  DRb.start_service(url, server, drbconf)
  $SAFE = 1
  DRb.thread.join
rescue Exception => error
  raise
end

lib/migel/server.rb

#!/usr/bin/env ruby
# Migel::Server -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'drb'
#require 'yus/entity'
#require 'yus/session'
require 'needle'

VERSION = '1.0.0'

module Migel
  class Server
  end
end

lib/migel/persistence/odba.rb

#!/usr/bin/env ruby
# Migel::Persistence::ODBA -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'odba'
require 'odba/drbwrapper'

module Migel
  module Persistence
    class Odba
      def initialize
        @system = ODBA.cache.fetch_named('system', self) { Hash.new }
      end
    end
  end
end

etc/migel.yml

---
db_user: 'postgres'
db_auth: ''

Run

masa@masa ~/ywesee/migel $ bin/migeld 
/home/masa/ywesee/migel/lib/migel/server.rb:9: warning: already initialized constant VERSION
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_pkey" for table "object"
NOTICE:  CREATE TABLE / UNIQUE will create implicit index "object_name_key" for table "object"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_connection_pkey" for table "object_connection"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "collection_pkey" for table "collection"

Note

Looks working
VERSION constant is also used in ODBA

Question

ODBA.cache.fetch_named
ODBA_SERIALIZABLE

Refactoring

These are coming from yus project
Next, refered to de.oddb.org
bin/migeld

#!/usr/bin/env ruby
# encoding: utf-8
# MigelDrbServer -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

$: << File.expand_path('../lib', File.dirname(__FILE__))
$KCODE = 'u'

require 'drb'
require 'migel/config'
require 'migel/persistence'
require 'migel/util'

module Migel

  begin
    server = Migel::Util::Server.new
    server.extend(DRbUndumped)
    @server = server

    url = @config.server_url
    url.untaint
    DRb.start_service(url, server)
    $SAFE = 1
#    logger.info('start') { sprintf("starting oddb-server on %s", url) }
    DRb.thread.join
  rescue Exception => error
#    logger.error('fatal') { error }
    raise
  end
end

lib/migel.rb

#!/usr/bin/env ruby
# Migel -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com


module Migel
   VERSION = '1.0.0'
#  Migel_VERSION =
#    File.read(File.expand_path('../.git/refs/heads/master',
#                               File.dirname(__FILE__)))
  class << self
    attr_accessor :config, :logger, :persistence, :server
  end
end

lib/migel/config.rb

#!/usr/bin/env ruby
# Migel@config -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'rclconf'
require 'migel'

module Migel
  default_dir = File.expand_path('../../etc', File.dirname(__FILE__))
  default_config_files = [
    File.join(default_dir, 'migel.yml'), '/etc/migel/migel.yml',
  ]
  defaults = {
    'config'            => default_config_files,
    'db_name'           => 'migel',
    'db_user'           => 'migel',
    'db_auth'           => 'migel',
    'persistence'       => 'odba',
    'server_url'        => 'druby://localhost:33000',
    'migel_dir'         => default_dir,
  }

  config = RCLConf::RCLConf.new(ARGV, defaults)
  config.load(config.config)
  @config = config
end

lib/migel/persistence.rb

require 'migel/config'

module Migel
  require File.join('migel', 'persistence', @config.persistence)
  persistence = nil
  DRb.install_id_conv ODBA::DRbIdConv.new
  @persistence = Migel::Persistence::ODBA
end

lib/migel/util.rb

#!/usr/bin/env ruby
# Util -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'migel/util/server'

lib/migel/persistence/odba.rb

#!/usr/bin/env ruby
# Migel::Persistence::ODBA -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'migel/config'
require 'odba'
require 'odba/connection_pool'
require 'odba/drbwrapper'

module Migel
  module Persistence
    module ODBA
    end
  end
  ODBA.storage.dbi = ODBA::ConnectionPool.new("DBI:pg:#{@config.db_name}",
                                              @config.db_user, @config.db_auth)
  ODBA.cache.setup
end

lib/migel/util/server.rb

#!/usr/bin/env ruby
# enconding: utf-8
# Migel::Util::Server -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'sbsm/drbserver'

module Migel
  module Util
    class Server < SBSM::DRbServer
    end
  end
end

Run

masa@masa ~/ywesee/migel $ bin/migeld
/home/masa/ywesee/migel/lib/migel.rb:4: warning: already initialized constant VERSION
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_pkey" for table "object"
NOTICE:  CREATE TABLE / UNIQUE will create implicit index "object_name_key" for table "object"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_connection_pkey" for table "object_connection"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "collection_pkey" for table "collection"

Setup bin/admin (a client to connect to the migel drb server)

Set up bin/admin

Attach:admin.rb.20110817.txt
lib/migel/util/server.rb

      def _admin(src, result, priority=0)
        raise "admin interface disabled" unless(self::class::ENABLE_ADMIN)
        t = Thread.new {
          Thread.current.abort_on_exception = false
          begin
            response = instance_eval(src)
            str = response.to_s
            result << if(str.length > 200)
              response.class
            else
              str
            end.to_s
          rescue StandardError => error
            result << error.message
            require 'pp'
#            ODDB.logger.error('admin') { error.class }
#            ODDB.logger.error('admin') { error.message }
#            ODDB.logger.error('admin') { error.backtrace.pretty_inspect }
            error
          end
        }
        t[:source] = src
        t.priority = priority
        @admin_threads.add(t)
        t
      end

Result

masa@masa ~/ywesee/migel $ bin/admin 
migel> Migel::VERSION
-> 1.0.0

Logger

Set up logger

lib/migel/util/Attach:logger.rb.20110817.txt Δ
lib/migel/util.rb

#!/usr/bin/env ruby
# Util -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'migel/util/logger'
require 'migel/util/server'

bin/migeld

module Migel
  begin
    server = Migel::Util::Server.new
    server.extend(DRbUndumped)
    @server = server

    url = @config.server_url
    url.untaint
    DRb.start_service(url, server)
    $SAFE = 1
    logger.info('start') { sprintf("starting oddb-server on %s", url) }
    DRb.thread.join
  rescue => error
    logger.error('fatal') { error }
    raise
  end
end

Run

 masa@masa ~/ywesee/migel $ bin/migeld 
 ERROR:  relation "object" already exists
 ERROR:  relation "prefetchable_index" already exists
 ERROR:  relation "extent_index" already exists
 ERROR:  relation "object_connection" already exists
 ERROR:  relation "target_id_index" already exists
 ERROR:  relation "collection" already exists
 I, [2011-08-17T11:19:43.678885 #28584]  INFO -- start: starting oddb-server on druby://localhost:33000

Define migel data classes

Migel::Model::Group

Base classes

lib/migel/model/group.rb
lib/migel/model/subgroup.rb
lib/migel/model/product.rb
lib/migel/model/item.rb
lib/migel/model/items.rb
lib/migel/model/group.rb

#!/usr/bin/env ruby
# Migel::Model::Group -- oddb -- 13.09.2005 -- ffricker@ywesee.com

#$: << File.expand_path("../../../src", File.dirname(__FILE__))

module Migel
  module Model
    class Group < Migel::SuperModel
      attr_reader :subgroups, :code
      attr_accessor :limitation_text, :group
      alias :pointer_descr :code
    end
  end
end

lib/migel/Attach:model_super.rb.20110817.txt
lib/migel/model.rb

require 'migel/model_super'
require 'migel/model/group'

bin/migeld

require 'migel/model'

Run

masa@masa ~/ywesee/migel $ bin/admin
migel> Migel::Model::Group.class
-> Class
migel> a = Migel::Model::Group.new; 
-> #<Migel::Model::Group:0x7fdd57930238>

suspend

How to save an instance

Probably

It is related to Migel::Persistence::ODBA module
lib/migel/persistence/odba/Attach:model_super.rb.20110817.txt
lib/migel/persistence/odba/model/group.rb

#!/usr/bin/env ruby
# Migel::Model::Group -- migel -- 17.98.2011 -- mhatakeyama@ywesee.com

require 'migel/model/group'
require 'migel/persistence/odba/model_super'

module Migel
  module Model
    class Group < ModelSuper
      odba_index :code
    end
  end
end

lib/migel/persistence/odba.rb

require 'migel/persistence/odba/model/group'

Run

masa@masa ~/ywesee/migel $ bin/admin
migel> a=Migel::Model::Group.new; a.code='999'; a.save; a.uid
-> 13
migel> a=ODBA.cache.fetch(13); a.code
-> 999
migel> exit
-> Goodbye

reboot bin/migeld

masa@masa ~/ywesee/migel $ bin/admin
migel> a=ODBA.cache.fetch(13); a.code
-> 999

Question

What are 'odba_index', 'selialize' class methods?
How to use them

suspend

How to delete an instance

migel> a=ODBA.cache.fetch(13); a.code
-> 999
migel> a=ODBA.cache.fetch(13); a.delete
-> #<Migel::Model::Group:0x7f03d5e24c00>
migel> a=ODBA.cache.fetch(13); a.code
-> /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:626:in `restore_object': Unknown odba_id 13
migel> a=Migel::Model::Group.new; a.code='888'; a.save; a.uid
-> 13
migel> a=ODBA.cache.fetch(13); a.code
-> 888

Define migel data class (Migel::Model::Subgroup, Migelid, Product)

Subgroup

lib/migel/model/group.rb

#!/usr/bin/env ruby
# Migel::Model::Group -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com
module Migel
  module Model
    class Group < Migel::ModelSuper
      attr_accessor :code, :limitation_text
      has_many :subgroups, on_delete(:cascade), on_save(:cascade)
      multilingual :limitation_text
      alias :pointer_descr :code
    end
  end
end

lib/migel/model/subgroup.rb

#!/usr/bin/env ruby
# Migel::Model::Subgroup -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

module Migel
  module Model
    class Subgroup < Migel::ModelSuper
      belongs_to :group
      attr_accessor :code
    end
  end
end

lib/migel/model.rb

#!/usr/bin/env ruby
# enconding: utf-8
# Model -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

require 'migel/model_super'
require 'migel/model/group'
require 'migel/model/subgroup'

Run

migel> a=Migel::Model::Group.new; a.code = '111'; a.save; a.uid
-> 21
migel> a=ODBA.cache.fetch(21); a.code
-> 111
migel> a=ODBA.cache.fetch(21); a.subgroups.push Migel::Model::Subgroup.new
-> #<Migel::Model::Subgroup:0x7f0494cfdad8>
migel> a=ODBA.cache.fetch(21); a.subgroups[0].code = '222'
-> 222
migel> a=ODBA.cache.fetch(21); a.subgroups[0].code
-> 222
migel> a=ODBA.cache.fetch(21); a.save
->

reboot bin/migeld

migel> a=ODBA.cache.fetch(21); a.subgroups[0].code
-> 222

Migelid

lib/migel/model/migelid.rb

#!/usr/bin/env ruby
# encoding: utf-8
# Migel::Model::Migelid -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com

module Migel
  module Model
    class Migelid < Migel::ModelSuper
      belongs_to :subgroup
      has_many :products, on_delete(:cascade), on_save(:cascade)
      has_many :accessories
      attr_accessor :limitation, :price, :type, :date, :qty, :unit, :code, :product_text
      alias :pointer_descr :code
      multilingual :limitation_text
      multilingual :product_text
   end
  end
end

Product

lib/migel/model/product.rb

#!/usr/bin/env ruby
# encondig: utf-8
# Migel::Model::Product -- migel -- 17.08.2011 -- mhatakeyama@ywesee.com


module Migel
  module Model
    class Product < Migel::ModelSuper
      belongs_to :migelid, delegates(:price, :qty, :unit, :migel_code)
      attr_accessor :ean_code, :pharmacode, :article_name, :companyname, :companyean, :ppha, :ppub, :factor, :pzr, :size, :status, :datetime, :stdate, :language
      alias :pointer_descr :migel_code
    end
  end
end

migel> a=ODBA.cache.fetch(26); a.products[0].migelid.class
-> Migel::Model::Migelid
migel> a=ODBA.cache.fetch(26); a.products[0].qty
-> 
migel> a=ODBA.cache.fetch(26); a.products[0].qty.class
-> NilClass
migel> a=ODBA.cache.fetch(26); a.qty = 'qty'
-> qty
migel> a=ODBA.cache.fetch(26); a.products[0].qty
-> qty

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

20110817-migel-drb-server

Log update_migel_nonpharma last night

Implement search function by artcle name

Create a new database for migel

Create a new (simplest) drb server for migel

Set up bin/admin

Set up logger

Define migel data classes

Migel::Model::Group

How to save an instance

How to delete an instance

Define migel data class (Migel::Model::Subgroup, Migelid, Product)

Subgroup

Migelid

Product

ywesee Developer-Wiki Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

20110817-migel-drb-server

Log update_migel_nonpharma last night

Implement search function by artcle name

Create a new database for migel

Create a new (simplest) drb server for migel

Set up bin/admin

Set up logger

Define migel data classes

Migel::Model::Group

How to save an instance

How to delete an instance

Define migel data class (Migel::Model::Subgroup, Migelid, Product)

Subgroup

Migelid

Product

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler