view · edit · sidebar · attach · print · history

20110822-search_by_name-migel-drb-client-oddb_org

<< | Index | >>

Log update_all_products migel suspend
Refactor delegate change last week
Update name search function of migel
Replace A-Z search function of migel

Goal/Estimate/Evaluation

Migel drb client in oddb.org / 90% / 70%

Milestones

Refactoring
Update the functions, migel name search, FB, Email in the result view of migel A-Z
Replace all the migel methods in oddbapp.rb suspend
Wrap migel classes in oddb.org suspend

Summary

http://masaywesee.wordpress.com/2011/08/22/update-search-function-of-migel-client-in-oddb-org/

Commits

Fix the number of argument error of DRbObject respond_to method (sbsm)

ToDo

All products have 'Limitation'
what is related to require 'oddb/util/m10l_document' in lib/migel/model_super.rb?
prune_old_revision
The other language (fr)
Migel::Util::Swissindex.search_migel_table(pharmacode, language)
Refactoring (in particluar, attr_accessor)
PPUB sort is ordered by dictionary, not by Fixnum
Testcases oddb.org

Log update_all_products migel

Console of swissindex_nonpharmad

D, [2011-08-22T05:44:53.466123 #4461] DEBUG -- : HTTPI executes HTTP POST using the net_http adapter
warning: peer certificate won't be verified in this SSL session
D, [2011-08-22T05:44:53.775162 #4461] DEBUG -- : HTTPI executes HTTP GET using the net_http adapter
warning: peer certificate won't be verified in this SSL session
D, [2011-08-22T05:44:53.931043 #4461] DEBUG -- : HTTPI executes HTTP POST using the net_http adapter
warning: peer certificate won't be verified in this SSL session
(stop)

Console of migeld

...
387 / 571       Estimate total: 7.39 [h] It will be done in: 2.38 [h]
388 / 571       Estimate total: 7.38 [h] It will be done in: 2.37 [h]
389 / 571       Estimate total: 7.38 [h] It will be done in: 2.35 [h]
390 / 571       Estimate total: 7.37 [h] It will be done in: 2.34 [h]
391 / 571       Estimate total: 7.36 [h] It will be done in: 2.32 [h]
392 / 571       Estimate total: 7.34 [h] It will be done in: 2.30 [h]
393 / 571       Estimate total: 62.83 [h] It will be done in: 19.59 [h]
394 / 571       Estimate total: 62.77 [h] It will be done in: 19.46 [h]
395 / 571       Estimate total: 62.61 [h] It will be done in: 19.30 [h]
396 / 571       Estimate total: 62.46 [h] It will be done in: 19.14 [h]
397 / 571       Estimate total: 73.63 [h] It will be done in: 22.44 [h]
...
503 / 571       Estimate total: 68.16 [h] It will be done in: 8.12 [h]
504 / 571       Estimate total: 68.02 [h] It will be done in: 7.98 [h]
505 / 571       Estimate total: 67.89 [h] It will be done in: 7.85 [h]
506 / 571       Estimate total: 67.75 [h] It will be done in: 7.71 [h]
507 / 571       Estimate total: 67.62 [h] It will be done in: 7.58 [h]
508 / 571       Estimate total: 67.51 [h] It will be done in: 7.45 [h]
509 / 571       Estimate total: 67.38 [h] It will be done in: 7.32 [h]
510 / 571       Estimate total: 67.44 [h] It will be done in: 7.20 [h]
(stop)

Note

I do not know why it stops. (it stopped also last time in the early morning)
I should make another update method to update only the products that is not updated
Or a update method to point the start number of migelid

suspend

Refactor delegate change last week

/usr/lib64/ruby/1.8/delegate.rb

  # 
  # Checks for a method provided by this the delegate object by fowarding the 
  # call through \_\_getobj\_\_.
  # 
  def respond_to?(m, include_private = false)
    return true if super
    return self.__getobj__.respond_to?(m, include_private)
  end

oddb.org/src/util/oddbapp.rb

    def search_migel_products(query, lang)
      migel_code = query.split(/(\d\d)/).select{|s| !s.empty?}.join('.')
      MIGEL_SERVER.migelid.search_by_migel_code(migel_code)
    end

Run

bin/oddbd
bin/migeld

Search migel code

http://oddb.masa.org/de/gcc/search/zone/migel/search_query/151003011#best_result

Result

error in SBSM::Session#to_html: /de/gcc/search/zone/migel/search_query/151003011
ArgumentError
wrong number of arguments (2 for 1)
/usr/lib64/ruby/1.8/delegate.rb:172:in `respond_to?'
/usr/lib64/ruby/1.8/delegate.rb:172:in `respond_to?'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:83:in `migel_code'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:112:in `compose_subheader'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:103:in `compose_list'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:97:in `each'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:97:in `compose_list'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/list.rb:54:in `compose'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:55:in `init'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/list.rb:129:in `init'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:59:in `init'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/component.rb:138:in `initialize'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:59:in `new'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:59:in `create'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:280:in `compose_component'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:209:in `compose'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:203:in `each'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:203:in `compose'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:55:in `init'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/component.rb:138:in `initialize'
/home/masa/ywesee/oddb.org/src/view/publictemplate.rb:53:in `new'
/home/masa/ywesee/oddb.org/src/view/publictemplate.rb:53:in `content'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:66:in `send'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:66:in `create'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:280:in `compose_component'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:209:in `compose'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:203:in `each'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:203:in `compose'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/composite.rb:55:in `init'
/home/masa/ywesee/oddb.org/src/view/publictemplate.rb:50:in `init'
/home/masa/ywesee/oddb.org/src/view/privatetemplate.rb:16:in `init'
/usr/lib64/ruby/site_ruby/1.8/htmlgrid/component.rb:138:in `initialize'
/usr/lib64/ruby/site_ruby/1.8/sbsm/state.rb:285:in `new'
/usr/lib64/ruby/site_ruby/1.8/sbsm/state.rb:285:in `view'
/usr/lib64/ruby/site_ruby/1.8/sbsm/state.rb:197:in `to_html'
/usr/lib64/ruby/site_ruby/1.8/sbsm/session.rb:529:in `to_html'
/usr/lib64/ruby/site_ruby/1.8/sbsm/session.rb:177:in `drb_process'
/usr/lib64/ruby/site_ruby/1.8/sbsm/session.rb:174:in `synchronize'
/usr/lib64/ruby/site_ruby/1.8/sbsm/session.rb:174:in `drb_process'
/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `__send__'
/usr/lib64/ruby/1.8/drb/drb.rb:1555:in `perform_without_block'
/usr/lib64/ruby/1.8/drb/drb.rb:1515:in `perform'
/usr/lib64/ruby/1.8/drb/drb.rb:1589:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1585:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1581:in `main_loop'
/usr/lib64/ruby/1.8/drb/drb.rb:1430:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `start'
/usr/lib64/ruby/1.8/drb/drb.rb:1427:in `run'
/usr/lib64/ruby/1.8/drb/drb.rb:1347:in `initialize'
/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `new'
/usr/lib64/ruby/1.8/drb/drb.rb:1627:in `start_service'
bin/oddbd:38

ODDB::View::Migel::ResultComposite::COMPONENTS[[0, 0]] in create(ODDB::View::Migel::ResultList)
ODDB::View::Migel::Result::COMPONENTS[[0, 3]] in create(content)
error in SBSM::Session#http_headers: /de/gcc/search/zone/migel/search_query/151003011
ArgumentError
wrong number of arguments (2 for 1)
/usr/lib64/ruby/1.8/delegate.rb:172:in `respond_to?'
/usr/lib64/ruby/1.8/delegate.rb:172:in `respond_to?'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:83:in `migel_code'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:112:in `compose_subheader'
/home/masa/ywesee/oddb.org/src/view/migel/result.rb:103:in `compose_list'

Experiment

/usr/lib64/ruby/1.8/delegate.rb

  def respond_to?(m, include_private = false)
    return true if super
print "self.class = "
p self.class
print "self.__getobj__ = "
p self.__getobj__.class
print "m = "
p m
print "include_private = "
p include_private
    return self.__getobj__.respond_to?(m, include_private)
  end

Result

 self.class = ODDB::State::Migel::Result::SubgroupFacade
 self.__getobj__ = DRb::DRbObject
 m = :items
 include_private = false

Note

Usually this values are as follows:

 self.class = ODDB::State::Migel::Result::SubgroupFacade
 self.__getobj__ = ODDB::Migel::Subgroup
 m = :items
 include_private = false

Consideration

The problem is that the return value of getobj becomes DRbObject
respond_to? method is NOT applied to the server side object, but DRbObject itself

Question

For DRbObject, in other words a server side object called by reference, Does the respond_to? method work?

Experiment

foo.rb

require 'drb'

class Foo
  include DRb::DRbUndumped
  def initialize(name)
    @name = name
  end
  attr_accessor :name
end

server.rb

require 'drb'
require 'foo'

front = {
  :test => 'test1',
  :foo  => Foo.new('foo')
}
DRb.start_service('druby://localhost:22345', front)
DRb.thread.join

client.rb

require 'drb'
require 'foo'
server = DRbObject.new_with_uri('druby://localhost:22345')
p server[:test]
p server[:test].class

p server[:foo]
p server[:foo].class
p server[:foo].respond_to?(:name, false)
p server[:foo ].name

Run

$ ruby server.rb

Result

masa@masa ~/work $ ruby client.rb 
"test1"
String
#<DRb::DRbObject:0x7f2827cc2238 @ref=70056854980340, @uri="druby://localhost:22345">
DRb::DRbObject
true
"foo"

Note

It works correctly

Consideration

The problem is Delegate#respond_to? ?
The Object#renpond_to? has TWO arguments.

Experiment

server.rb

require 'drb'
require 'foo'
require 'delegate'

front = {
  :test => 'test1',
  :foo  => Foo.new('foo'),
  :delegator => SimpleDelegator.new(Foo.new('foodelegator')),
}
DRb.start_service('druby://localhost:22345', front)
DRb.thread.join

client.rb

require 'drb'
require 'foo'
server = DRbObject.new_with_uri('druby://localhost:22345')
p server[:test]
p server[:test].class

p server[:foo]
p server[:foo].class
p server[:foo].respond_to?(:name, false)
p server[:foo].name
p server[:delegator].class
p server[:delegator].respond_to?(:name, false)
p server[:delegator].name

Result

masa@masa ~/work $ ruby client.rb 
"test1"
String
#<DRb::DRbObject:0x7f3bc7d5c708 @uri="druby://localhost:22345", @ref=70021258046080>
DRb::DRbObject
true
"foo"
DRb::DRbObject
true
"foodelegator"

Note

It also works
Delegator#respond_to? can take TWO arguments too.

Experiment

migel> Migel::Model::Subgroup.all[0].respond_to?(:items, false)
-> true
migel> Migel::Model::Subgroup.all[0].respond_to?(:product_text, false)
-> true

Note

respond_to? method of server side object can also TWO arguments

Experiment

test_migel_client.rb

require 'drb'
require 'delegate'
server = DRbObject.new_with_uri('druby://localhost:33000')

p server.subgroup.all.length
p server.subgroup.all[0].respond_to?(:items, false)
p server.subgroup.all[0].respond_to?(:product_text, false)

delegator = SimpleDelegator.new(server.subgroup.all[0])
p '----'
p delegator.respond_to?(:items)
p delegator.respond_to?(:items, false)
p delegator.respond_to?(:product_text)
p delegator.respond_to?(:product_text, false)
p '----'

Result

masa@masa ~/work $ ruby migel_client.rb 
118
true
true
"----"
self.class = SimpleDelegator
self.__getobj__ = DRb::DRbObject
m = :items
include_private = false
true
self.class = SimpleDelegator
self.__getobj__ = DRb::DRbObject
m = :items
include_private = false
true
self.class = SimpleDelegator
self.__getobj__ = DRb::DRbObject
m = :product_text
include_private = false
true
self.class = SimpleDelegator
self.__getobj__ = DRb::DRbObject
m = :product_text
include_private = false
true
"----"

Note

The problem is NOT server side
There must be a problem in the client side (oddb.org)

Experiment

sbsm/drb.rb

 module DRb
   class DRbObject
     #def respond_to?(msg_id)
     def respond_to?(msg_id, *args)

Search migel code

http://oddb.masa.org/de/gcc/search/zone/migel/search_query/153050001#best_result

Result

Success

Commit

Fix the number of argument error of DRbObject respond_to method (sbsm)

Update name search function of migel

Problem

Migel search works only by migel code at the moment

Reason

oddb.org/src/util/oddbapp.rb

    def search_migel_products(query, lang)
      migel_code = query.split(/(\d\d)/).select{|s| !s.empty?}.join('.')
      MIGEL_SERVER.migelid.search_by_migel_code(migel_code)
    end

Note

search_migel_products calls 'search_by_migel_code' method of server side object

The original search_migel_products method

  def search_migel_products(query, lang)
    if(lang.to_s != "fr")
      lang = "de"
    end
    index_name = "migel_fulltext_index_#{lang}"
    ODBA.cache.retrieve_from_index(index_name, query)
  end

Note

It seems that ODBA.cache.retrive_from_index method can search both migel code and migel name too

Experiment

/usr/lib64/ruby/site_ruby/1.8/odba/cache.rb

    def retrieve_from_index(index_name, search_term, meta=nil)
print indices.keys.pretty_inspect

Run

migel> ODBA.cache.retrieve_from_index('migel_model_migelid_migel_code', '29.01.01.00.1').length
-> 1

Result

["migel_model_subgroup_code",
 "migel_model_migelid_migel_code",
 "migel_model_group_code",
 "migel_model_migelid_code",
 "migel_model_product_pharmacode"]
["migel_model_subgroup_code",
 "migel_model_migelid_migel_code",
 "migel_model_group_code",
 "migel_model_migelid_code",
 "migel_model_product_pharmacode"]

find_by_name method and refer to de.oddb.org code

Note

odba/persistable.rb
- Both methods call ODBA.cache. retrieve_from_index method
- find_by_(name) and search_by_(name) methods uses the same algorithm, but the last argument of ODBA.cache.retrieve_from_index is different
- the return value of find_by_(name) is the first value of the return value (Array) of search_by_(name) method
- if nothing is found by searching find_by_(name) method returns nil, and search_by_(name) returns empty array

Experiment

lib/migel/persistence/odba/model/migelid.rb

 module Migel
    module Model
      class Migelid < ModelSuper
        odba_index :code
        odba_index :migel_code
        odba_index :name
      end
    end
 end

Restore database

masa@masa ~/ywesee/migel $ sudo -u postgres dropdb migel
masa@masa ~/ywesee/migel $ sudo -u postgres createdb -E UTF8 -T template0 migel
masa@masa ~/ywesee/migel $ bin/migeld

migel> Migel::Importer.new.update('data/csv/migel_de_test.csv', 'de')
-> Array

Conclusion

It is the data structure mistake
I have to import data again, when the class data structure is updated

How to search migel code and migel name at the same time with one function

Experiment

oddb.org/src/util/oddbapp.rb

    def search_migel_products(query, lang)
      migel_code = if query =~ /(\d){9}/
                     query.split(/(\d\d)/).select{|x| !x.empty?}.join('.')
                   elsif query =~ /(\d\d\.){4}\d/
                     query
                   end
      if migel_code
         MIGEL_SERVER.migelid.search_by_migel_code(migel_code)
      else
         MIGEL_SERVER.migelid.search_by_name(query)
      end
    end

Search & Result

search '15.30.50.00.1' : http://oddb.masa.org/de/gcc/search/zone/migel/search_query/15.30.50.00.1#best_result => success
search '153050001' : http://oddb.masa.org/de/gcc/search/zone/migel/search_query/153050001#best_result => success
search 'Wegwerf-Scheidenpessar' : http://oddb.masa.org/de/gcc/search/zone/migel/search_query/Wegwerf-Scheidenpessar#best_result => failure
search 'Wegwerf' : http://oddb.masa.org/de/gcc/search/zone/migel/search_query/Wegwerf#best_result => success

Note

The failure depends on ODBA#search_term in src/state/global.rb and Migel::Model::Migelid.search_by_name
The 'Wegwerf-Scheidenpessar' becomes 'WegwerfScheidenpessar' by ODDB.search_term method
Migel.search_by_name method fails if it is 'WegwerfScheidenpessar'

Question

What is the index_name, 'migel_fulltext_index_', of ODBA.cache.retrieve_from_index method? (src/util/oddbapp.rb#search_migel_products)
How do I use the index_name?

suspend

Replace A-Z search function of migel

Problem

The result view of A-Z search function calls still the old data and code

For example,

http://oddb.masa.org/de/gcc/migel_alphabetical/range/e

Note

There is no method, 'migel_alphabetical', as an event in src/state/global.rb
Probably it is related to 'DIRECT_EVENT = :migel_alphabetical' in src/state/migel/alphabetical.rb

Question

What is the 'direct_event'?
How to use it
Where is the model data for State and View coming from?

Note

'direct_event' is defined in SBSM, not HtmlGrid

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

20110822-search_by_name-migel-drb-client-oddb_org

Log update_all_products migel

Refactor delegate change last week

Update name search function of migel

Replace A-Z search function of migel

ywesee Developer-Wiki Dieses Wiki richtet sich an alle ywesee-Entwickler

About

EBPS

Bbmb

ODBA

Oddb

Rpdf2txt

YDPM

YDIM

XmlConv

20110822-search_by_name-migel-drb-client-oddb_org

Log update_all_products migel

Refactor delegate change last week

Update name search function of migel

Replace A-Z search function of migel

ywesee Developer-Wiki
Dieses Wiki richtet sich an alle ywesee-Entwickler