view · edit · sidebar · attach · print · history

20101116-create-grant_download-command-de_oddb_org

<< Masa.20101117-create-grant_download-command-de_oddb_org | 2010 | Masa.20101115-create-grant_download-command-de_oddb_org >>


  1. Test a new method to download a file in de.oddb.org
  2. Test ODBA for the expiration of a user
  3. Read Persistence module in oddb.org and de.oddb.org
  4. Test ODBA again

Goal
  • Create grant_download / 70%
Milestones
  1. Check a bug of import_gkv Later
  2. Test a new command 9:50
  3. Test ODBA suspend
    • BraSt
  4. Read Persistence module 15:00
  5. Test ODBA again
Summary
Commits
ToDo Tomorrow
  • Check the procedure of restore ODBA data on the cache
  • Implement grant_download in de.oddb.org
Keep in Mind
  1. A bug import_gkv Tue Nov 16 02:00:10 2010: de.oddb.org Zubef (PDF)
  2. On Ice
  3. emerge --sync

Test a new method to download a file in de.oddb.org

Experiment (oddb.org)

/usr/lib64/ruby/site_ruby/1.8/sbsm/state.rb#trigger

        def trigger(event)
      if(@redirected)
        @redirected = false
      else
        @errors = {}
        @infos = []
        @warnings = []
      end
            state = if(event && !event.to_s.empty? && self.respond_to?(event))
open("/home/masa/work/test.dat","a"){|f| f.print "self.class=", self.class, "\n"}
                _trigger(event)
            elsif(klass = @events[event])
...
        end
...
    def _trigger(event)
      self.send(event)
    end

Note

  • This method is called from SBSM::Session class in order to decide the current State class depending on the 'event'

bin/admin (create the donwload link)

masa@masa ~/ywesee/oddb.org $ bin/admin 
ch.oddb> grant_download 'aaa@bbb.ccc', 'test.dat', 1, Time.local(2011,1,1)
-> http://oddb.masa.org/de/gcc/download/invoice/25518091/email/aaa@bbb.ccc/filename/test.dat

Reboot oddb.org

Access to http://oddb.masa.org/de/gcc/download/invoice/25518091/email/aaa@bbb.ccc/filename/test.dat

Result

masa@masa ~/work $ cat test.dat
self.class=ODDB::State::Drugs::Init

Notes

  • The class is NOT ODDB::State::Global but actually the ODDB::State::Global#download method is called,
  • because ODDB::State::Drugs::Init (every State class) eventually inherits ODDB::State::Global class
  • So, 'self.respond_to?(event)' becomes 'true' even if 'self' is any State class
  • So, I should add a new method for the downloading in ODDB::State::Global class of de.oddb.org

Experiment (de.oddb.org)

1. Define a new method lib/oddb/html/state/global.rb

  def masa
    open("/home/masa/work/test.dat","a"){|f| f.print "MASA\n"}
  end

2. Add a new event lib/oddb/html/util/validator.rb

  EVENTS = [ :ajax_autofill, :ajax_create_active_agent,
...
#             :sequence, :sort, :update ]
             :sequence, :sort, :update, :masa]

3. Write the event in the 3rd part of URL

Result

masa@masa ~/work $ cat test.dat
getin process
key=zone        value=bbb
key=language    value=de
key=event       value=masa
key=default_flavor      value=oddb
event()=masa
getin trigger
self.class=ODDB::Html::State::Drugs::Init
event=masa
MASA

getin process
key=zone        value=drugs
key=language    value=de
key=event       value=home
key=default_flavor      value=oddb
event()=home
getin trigger
self.class=ODDB::Html::State::Drugs::Init
event=home

Notes

  • The method 'masa' in oddb/html/state/global.rb is called but quickly after that a new process runs
  • and the 'home' event happens

Experiment (de.oddb.org)

Modify the new method in lib/oddb/html/state/global.rb

  def masa
    open("/home/masa/work/test.dat","a"){|f| f.print "getin masa method\n"}
    Download.new(@session, '/home/masa/ywesee/oddb.org/data/downloads/test.dat')
  end

Reboot de.oddb.org/bin/oddbd

Access to http://de.oddb.masa.org/zz/xxx/masa/

Result

  • Succeed in downloading a file

Notes

  • We do not have to be care about the 2nd string of the URL, 'xxx'
  • The 1st string of the URL must be 2 characters
  • Now, I have done the preparation for the creating of a new method in de.oddb.org
  • But I should also check (understand) how to use ODBA to check the expiration for the downloading

Test ODBA for the expiration of a user

Check again the save and load parts of data in oddb.org

Save part src/util/oddbapp.rb#grant_download

    def grant_download(email, filename, price, expires=Time.now+2592000)
      ip = Persistence::Pointer.new(:invoice)
      inv = update ip.creator, :yus_name => email, :currency => 'EUR'
      itp = inv.pointer + :item
      update itp.creator, :text => filename, :price => price, :time => Time.now,
                          :type => :download, :expiry_time => expires,
                          :duration => (Time.now - expires) / 86400,
                          :vat_rate => 7.6
      inv.payment_received!
      inv.odba_store
      "http://#{SERVER_NAME}/de/gcc/download/invoice/#{inv.oid}/email/#{email}/filename/#{filename}"
    end

Load part

src/state/global.rb#download

            def download
                if(@session.is_crawler?)
                    return State::Drugs::Init.new(@session, nil)
                end
                email = @session.user_input(:email)
                email ||= @session.get_cookie_input(:email)
                oid = @session.user_input(:invoice)
                file = @session.user_input(:filename)
                if((invoice = @session.invoice(oid)) \
          && invoice.yus_name == email \
                    && invoice.payment_received? \
                    && (item = invoice.item_by_text(file)) \
                    && !item.expired?)
                    State::User::Download.new(@session, item)
                else
                    State::PayPal::Return.new(@session, invoice)
                end
            end

BraSt

  • There is not Pointer class defined in de.oddb.org
  • Without understanding ODBA, I cannot create a new system
  • It would be good to read ODBA source code
  • It would be good to compare the Persitence module between oddb.org and de.oddb.org
  • Updater may tell me about registering data through ODBA
  • Exporter may tell me about loading data through ODBA
  • There are many classes which include ODBA module in both oddb.org and de.oddb.org
  • There look many differences about ODBA between in oddb.org and in de.oddb.org
  • At least, I need to save 'user (email)' and 'expire time' and load them when the user downloads the file
  • Additionally, the command must have a function to show all old and active dowloads by email address of the user.

What to do

  • Read and compare Persistence module in both oddb.org and de.oddb.org
  • in particular, focus on saving and loading parts

Read Persistence module in oddb.org and de.oddb.org

grep check

masa@masa ~/ywesee/de.oddb.org $ grep -r odba_isolated_store lib
lib/oddb/persistence/odba/model.rb:      odba_isolated_store

masa@masa ~/ywesee/oddb.org $ grep -r odba_store src
src/command/accept_orphan.rb:        app.accepted_orphans.odba_store
src/command/accept_orphan.rb:        parent.odba_store
src/command/merge.rb:      target.odba_store
...

Notes

  • in de.oddb.org, saving point (class) is simplified more than oddb.org
masa@masa ~/ywesee/de.oddb.org $ grep -r Persistable lib
lib/oddb/persistence/odba/export.rb:  module Persistable
lib/oddb/persistence/odba/model.rb:    include ODBA::Persistable

masa@masa ~/ywesee/oddb.org $ grep -r Persistable src
src/model/commercial_form.rb:    include ODBA::Persistable ## include directly to get odba_index
src/model/galenicform.rb:    include ODBA::Persistable ## include directly to get odba_index
src/model/index_therapeuticus.rb:    include ODBA::Persistable ## include directly to get odba_index
src/model/minifi.rb:            include ODBA::Persistable
src/model/package.rb:    include ODBA::Persistable ## include directly to get odba_index
src/util/notification_logger.rb:                include ODBA::Persistable
src/util/oddbapp.rb:    include ODBA::Persistable
src/util/oddbapp.rb:      when ODBA::Persistable, ODBA::Stub
src/util/persistence.rb:                include ODBA::Persistable
src/util/persistence.rb:                                # see ODBA::Persistable

Notes

  • In de.oddb.org, the class which includes 'ODBA::Persistable module' is only ODBA::Model class
  • The other model classes inherit this ODBA::Model class, which means the class can save itself automatically by calling 'save' method of ODBA::Model class

Test ODBA again

Sample test

require 'odba'
require 'odba/connection_pool'

# connect default storage manager to a relational database

###ODBA.storage.dbi = ODBA::ConnectionPool.new('DBI::pg::database', 'user', 'pw') 
ODBA.storage.dbi = ODBA::ConnectionPool.new('DBI:pg:testdb', 'masa', '')

class Counter
    include ODBA::Persistable
    def initialize
      @pos = 0
    end
    def up
      @pos += 1
      self.odba_store
      @pos
    end
    def down
      @pos -= 1
      self.odba_store
      @pos
    end
    def show
        p @pos
    end
end

test=Counter.new
test.show
test.up

Result

  • Failed
masa@masa ~/work $ ruby odba.rb 
0
ERROR:  relation "object" does not exist
LINE 1: SELECT odba_id FROM object ORDER BY odba_id DESC LIMIT 1
                            ^

/usr/lib64/ruby/site_ruby/1.8/DBD/Pg/Pg.rb:671:in `execute': ERROR:  relation "object" does not exist (DBI::ProgrammingError)
LINE 1: SELECT odba_id FROM object ORDER BY odba_id DESC LIMIT 1
                            ^
        from /usr/lib64/ruby/site_ruby/1.8/dbi.rb:912:in `execute'
        from /usr/lib64/ruby/site_ruby/1.8/dbi.rb:504:in `execute'
        from /usr/lib64/ruby/site_ruby/1.8/dbi.rb:526:in `select_one'
        from /usr/lib64/ruby/site_ruby/1.8/odba/connection_pool.rb:36:in `send'
        from /usr/lib64/ruby/site_ruby/1.8/odba/connection_pool.rb:36:in `method_missing'
        from /usr/lib64/ruby/site_ruby/1.8/odba/connection_pool.rb:26:in `next_connection'
        from /usr/lib64/ruby/site_ruby/1.8/odba/connection_pool.rb:35:in `method_missing'
        from /usr/lib64/ruby/site_ruby/1.8/odba/storage.rb:605:in `restore_max_id'
         ... 10 levels...
        from /usr/lib64/ruby/site_ruby/1.8/odba/persistable.rb:381:in `odba_store_unsaved'
        from /usr/lib64/ruby/site_ruby/1.8/odba/persistable.rb:369:in `odba_store'
        from odba.rb:17:in `up'
        from odba.rb:32

Create user for a database

masa@masa ~/ywesee/de.oddb.org/data/sql $ su postgres
Passwort: 
postgres@masa /home/masa/ywesee/de.oddb.org/data/sql $ createuser
Geben Sie den Namen der neuen Rolle ein: name
Soll die neue Rolle ein Superuser sein? (j/n) n
Soll die neue Rolle Datenbanken erzeugen dürfen? (j/n) j
Soll die neue Rolle weitere neue Rollen erzeugen dürfen? (j/n) j

Create a database

masa@masa ~/ywesee/de.oddb.org/data/sql $ createdb -E UTF8 -T template0 testdb

Check the database

masa@masa ~/ywesee/de.oddb.org/data/sql $ psql -U postgres
psql (8.4.2)
Geben Sie »help« für Hilfe ein.

postgres=# \l
                                   Liste der Datenbanken
    Name     | Eigentümer | Kodierung | Sortierfolge | Zeichentyp |    Zugriffsrechte     
-------------+-------------+-----------+--------------+------------+-----------------------
 template0   | postgres    | SQL_ASCII | C            | C          | =c/postgres
                                                                   : postgres=CTc/postgres
 template1   | postgres    | SQL_ASCII | C            | C          | =c/postgres
                                                                   : postgres=CTc/postgres
 testdb      | masa        | UTF8      | C            | C          | 

postgres=# \q

Modified sample

#!/usr/bin/env ruby

require 'odba'
require 'odba/connection_pool'


ODBA.storage.dbi = ODBA::ConnectionPool.new("DBI:pg:testdb", "masa", "")
ODBA.cache.setup

class Counter
include ODBA::Persistable
  def initialize
    @pos = 0
  end
  def up
    @pos += 1
    self.odba_store
  end
  def show
    p @pos
 end
end

begin
  test=ODBA.cache.fetch(2)
rescue
  test=Counter.new
end
p test.odba_id
test.show
test.up
#test.odba_delete

Result

  • Succeeded
masa@masa ~/work $ ruby odba.rb 
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_pkey" for table "object"
NOTICE:  CREATE TABLE / UNIQUE will create implicit index "object_name_key" for table "object"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "object_connection_pkey" for table "object_connection"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "collection_pkey" for table "collection"
odba_id=2
0
masa@masa ~/work $ vim odba.rb 
masa@masa ~/work $ ruby odba.rb 
ERROR:  relation "object" already exists

ERROR:  relation "prefetchable_index" already exists

ERROR:  relation "extent_index" already exists

ERROR:  relation "object_connection" already exists

ERROR:  relation "target_id_index" already exists

ERROR:  relation "collection" already exists

odba_id=2
1

Check database

masa@masa ~/work $ psql -U postgres
psql (8.4.2)
Geben Sie »help« für Hilfe ein.

postgres=# \l
                                   Liste der Datenbanken
    Name     | Eigentümer | Kodierung | Sortierfolge | Zeichentyp |    Zugriffsrechte     
-------------+-------------+-----------+--------------+------------+-----------------------
 oddb        | postgres    | UTF8      | C            | C          | 
 oddb.org    | postgres    | UTF8      | C            | C          | 
 postgres    | postgres    | SQL_ASCII | C            | C          | 
 sandoz_bbmb | postgres    | UTF8      | C            | C          | 
 template0   | postgres    | SQL_ASCII | C            | C          | =c/postgres
                                                                   : postgres=CTc/postgres
 template1   | postgres    | SQL_ASCII | C            | C          | =c/postgres
                                                                   : postgres=CTc/postgres
 testdb      | masa        | UTF8      | C            | C          | 
 ydpm        | postgres    | SQL_ASCII | C            | C          | 
 yus         | postgres    | SQL_ASCII | C            | C          | 
(9 Zeilen)

postgres=# \c testdb
psql (8.4.2)
Sie sind jetzt verbunden mit der Datenbank »testdb«.
testdb=# \d
              Liste der Relationen
 Schema |       Name        |  Typ  | Eigentümer 
--------+-------------------+-------+------------
 public | collection        | table | masa
 public | object            | table | masa
 public | object_connection | table | masa
(3 Zeilen)

testdb=# select * from collection;
 odba_id | key | value 
---------+-----+-------
(0 Zeilen)

testdb=# select * from object;
 odba_id |                                                                                          content                                                                                           |           name           | prefetchable | extent  
---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+--------------+---------
       1 | 0408497b00093a0f406f6462615f6e616d65221d5f5f63616368655f7365727665725f696e64696365735f5f3a15406f6462615f70657273697374656e74543a0d406f6462615f696469063a14406f6462615f6f627365727665727330 | __cache_server_indices__ | f            | Hash
       2 | 04086f3a0c436f756e746572093a0940706f7369073a0d406f6462615f696469073a15406f6462615f70657273697374656e74543a14406f6462615f6f627365727665727330                                               |                          | f            | Counter
(2 Zeilen)

testdb=# select * from object_connection;
 origin_id | target_id 
-----------+-----------
(0 Zeilen)

testdb=# \q

Notes

  • If ODBA.cache.fetch failed, there is no data corresponding to the odba_id, ODBA::OdbaError happens
  • ODBA.cache.setup is necessary at only the first run, otherwise the error like the first sample happens
  • After the second run, Error comes but no problem
  • As a default, Persistable object will be saved in the table 'object'.
  • The important methods are 1. odba_id, 2. odba_store, 3. odba_delete, 4. ODBA.cache.fetch
  • in de.oddb.org, save and delete methods for ODBA are arranged in lib/oddb/model.rb and lib/oddb/persistence/odba/model.rb.

Question

  • How does oddb.org and de.oddb.org restore all the data on the cache?

Expiry check sample

#!/usr/bin/env ruby
# Persistence::ODBA -- de.oddb.org -- 01.09.2006 -- hwyss@ywesee.com

require 'odba'
require 'odba/connection_pool'
#require 'odba/drbwrapper'


ODBA.storage.dbi = ODBA::ConnectionPool.new("DBI:pg:testdb", "masa", "")
ODBA.cache.setup

class Masa
  include ODBA::Persistable
  attr_reader :expiry_time
  def expired?
    (@expiry_time - Time.now) < 0
  end
  def save(expires=Time.now+2592000)  # in 30 days
    @expiry_time = expires
    self.odba_store
  end
end

if ARGV[0]
  masa = ODBA.cache.fetch(ARGV[0].to_i)
  print "expired?=", masa.expired?, "\n"
  print "this id is valid in ", (masa.expiry_time - Time.now).to_i/60/60/24, " days\n"
else
  masa = Masa.new
  print "odba_id=", masa.odba_id, "\n"
  masa.save Time.local(2011,1,1)
end

Result

masa@masa ~/work $ ruby odba3.rb 
odba_id=5

masa@masa ~/work $ ruby odba3.rb 5
expired?=false
this id is valid in 45 days

Notes

  • The data will remain forever in this example
  • Let's think if the deleting process is necessary in the actual system (de.oddb.org).

Question

  • How can I search the cache data by the other data than odba_id?
view · edit · sidebar · attach · print · history
Page last modified on July 13, 2011, at 11:54 AM