20111109-testcase-link-format-benchmark-oddb_org

<< | Index | >>

1. Check new parser again
2. Testcases (mainly) for link format updates
3. Check Threads oddb.org
4. Remove pointer link from just medical

Goal/Estimate/Evaluation

• Testcases for link format / 100% / 100%

Commit

• Updated testcases mainly for url format changes (oddb.org)
• Replaced Wrikstoff link without pointer in just-medical and santesuisse flavor (oddb.org)

Check new parser again

Compare comparable_size and size

• Attach:rockit1.rb.20111109.txt
• Attach:parser1.rb.20111109.txt

Run

$ ruby parser1.rb > dat.parser
$ ruby rockit1.rb > dat.rockit
$ diff -y --suppress-common-lines -W 200 dat.rockit dat.parser > diff.txt

Result

• Attach:size_example.20111109.txt
• Attach:rockit_result.dat.20111109.txt
• Attach:parser_result.dat.20111109.txt
• Attach:diff1.20111109.txt

Experiment

• src/model/part.rb

    def set_comparable_size!
      measure = (@measure.nil? || @measure == UNIT)               ? _composition_scale               : @measure
      measure ||= UNIT
      scale = @scale || UNIT
      @comparable_size = multiplier * measure / scale
 p "@comparable_size = "
 p @comparable_size
 @comparable_size
    end
    def size=(size)
 p "getin size="
 p size
      unless size.to_s.strip.empty?
        @addition, @multi, @count, @measure, @scale, @comform = parse_size(size)
        if @count == 0
          @count, @multi = @multi, nil
        end
        set_comparable_size!
      end
    end

Run

ch.oddb> Updater.new(self).update_bsv
ch.oddb> Updater.new(self).update_swissmedic

Result

• ch.ODDB.org Report - SL-Update (XML) - 09/11/2011
• ch.ODDB.org Report - Swissmedic XLS - 09/11/2011
• No standard output -> Part#size= method is never used in these cases

Testcases (mainly) for link format updates

Commit

• Updated testcases mainly for url format changes (oddb.org)

Result

• 4184 tests, 6665 assertions, 0 failures, 0 errors
• 89.97%

Check Threads oddb.org

Task

• Check how threads are created (and deleted) in oddb.org with respect to the number of sessions

Email

• optimizing thread cleaning (08.11.2011)

Experiment

ch.oddb> Thread.list.map{|t| t.status}.select{|i| i=='sleep'}.length
-> 44
ch.oddb> Thread.list.map{|t| t.status}.select{|i| i=='run'}.length
-> 2

Bechmark

 $ /usr/sbin/ab -n 500 -c 100 http://oddb.masa.org/
 This is ApacheBench, Version 2.3 <$Revision: 655654 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/

 Benchmarking oddb.masa.org (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Finished 500 requests


 Server Software:        Apache
 Server Hostname:        oddb.masa.org
 Server Port:            80

 Document Path:          /
 Document Length:        16142 bytes

 Concurrency Level:      100
 Time taken for tests:   120.696 seconds
 Complete requests:      500
 Failed requests:        8
   (Connect: 0, Receive: 0, Length: 8, Exceptions: 0)
 Write errors:           0
 Non-2xx responses:      8
 Total transferred:      8144532 bytes
 HTML transferred:       7941864 bytes
 Requests per second:    4.14 [#/sec] (mean)
 Time per request:       24139.177 [ms] (mean)
 Time per request:       241.392 [ms] (mean, across all concurrent requests)
 Transfer rate:          65.90 [Kbytes/sec] received 

 Connection Times (ms)
              min  mean[+/-sd] median   max
 Connect:        0    0   0.9      0       2
 Processing: 12319 22944 4010.8  22650   34084
 Waiting:    12319 22943 4010.8  22650   34083
 Total:      12322 22944 4010.6  22650   34086

 Percentage of the requests served within a certain time (ms)
  50%  22650
  66%  25637
  75%  26324
  80%  26703
  90%  27216
  95%  27610
  98%  30651
  99%  32981
 100%  34086 (longest request)

...(after booting)

ch.oddb> "total:" + Thread.list.length.to_s + " sleep:" + Thread.list.map{|t| t.status}.select{|i| i=="sleep"}.length.to_s + " run:" + Thread.list.map{|t| t.status}.select{|i| i=="run"}.length.to_s
-> total:8 sleep:6 run:2

...(start benchmarking)

ch.oddb> "total:" + Thread.list.length.to_s + " sleep:" + Thread.list.map{|t| t.status}.select{|i| i=="sleep"}.length.to_s + " run:" + Thread.list.map{|t| t.status}.select{|i| i=="run"}.length.to_s
-> total:67 sleep:60 run:7
-> total:80 sleep:64 run:16
-> total:86 sleep:82 run:4
-> total:88 sleep:80 run:8
-> total:105 sleep:94 run:11
-> total:105 sleep:96 run:9
-> total:105 sleep:97 run:8
-> total:111 sleep:102 run:9
-> total:113 sleep:98 run:15
-> total:113 sleep:106 run:7
-> total:113 sleep:100 run:13
-> total:113 sleep:101 run:12
-> total:113 sleep:104 run:9
-> total:113 sleep:102 run:11
-> total:113 sleep:107 run:6
-> total:113 sleep:104 run:9
-> total:113 sleep:104 run:9
-> total:113 sleep:102 run:11
-> total:113 sleep:102 run:11
-> total:102 sleep:96 run:6
-> total:100 sleep:96 run:4

...(Benchmark done)

-> total:60 sleep:58 run:2
-> total:58 sleep:56 run:2
-> total:56 sleep:54 run:2
-> total:55 sleep:53 run:2
-> total:54 sleep:52 run:2
...
-> total:18 sleep:16 run:2

ch.oddb> self.instance_variable_get('@sessions').size
-> 492

... (30 minutes no access)

ch.oddb> self.instance_variable_get('@sessions').size
-> 1
ch.oddb> "total:" + Thread.list.length.to_s + " sleep:" + Thread.list.map{|t| t.status}.select{|i| i=="sleep"}.length.to_s + " run:" + Thread.list.map{|t| t.status}.select{|i| i=="run"}.length.to_s
-> total:9 sleep:7 run:2

Experiment2

masa@masa ~/work $ /usr/sbin/ab -n 1000 -c 200 http://oddb.masa.org/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking oddb.masa.org (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests

Server Software:        Apache
Server Hostname:        oddb.masa.org
Server Port:            80

Document Path:          /
Document Length:        16142 bytes

Concurrency Level:      200
Time taken for tests:   283.313 seconds
Complete requests:      1000
Failed requests:        22
   (Connect: 0, Receive: 0, Length: 22, Exceptions: 0)
Write errors:           0
Non-2xx responses:      22
Total transferred:      16190838 bytes
HTML transferred:       15786876 bytes
Requests per second:    3.53 [#/sec] (mean)
Time per request:       56662.511 [ms] (mean)
Time per request:       283.313 [ms] (mean, across all concurrent requests)
Transfer rate:          55.81 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   2.0      0       5
Processing: 12809 52408 17265.0  54828   74966
Waiting:    12808 52407 17265.0  54827   74966
Total:      12813 52409 17263.7  54828   74966

Percentage of the requests served within a certain time (ms)
  50%  54828
  66%  66605
  75%  69790
  80%  70658
  90%  72668
  95%  73558
  98%  74113
  99%  74500
 100%  74966 (longest request)

ch.oddb> "total:" + Thread.list.length.to_s + " sleep:" + Thread.list.map{|t| t.status}.select{|i| i=="sleep"}.length.to_s + " run:" + Thread.list.map{|t| t.status}.select{|i| i=="run"}.length.to_s + " session:" + self.instance_variable_get('@sessions').size.to_s
-> total:8 sleep:6 run:2 session:0

...(start benchmark)

ch.oddb> "total:" + Thread.list.length.to_s + " sleep:" + Thread.list.map{|t| t.status}.select{|i| i=="sleep"}.length.to_s + " run:" + Thread.list.map{|t| t.status}.select{|i| i=="run"}.length.to_s + " session:" + self.instance_variable_get('@sessions').size.to_s
-> total:150 sleep:135 run:15 session:958

...(after the benchmark)

lect{|i| i=="run"}.length.to_s + " session:" + self.instance_variable_get('@sessions').size.to_s
-> total:18 sleep:16 run:2 session:978

Note

• Most of the threads are sleeping
• It seems that the number of threads increases with the number of accesses at the same time (-c option)
• It seems that the number of sessions increases with the number of requests (-n option)
• The number of threads decreases quickly but the number of sessions decreases slowly after the benchmark
  • ODDB::Session::EXPIRES = 30 * 60 (probably 30 min later a session instance will be released)

Reference

• ab benchmark (Japanese) http://www.atmarkit.co.jp/flinux/rensai/apache15/apache15b.html

Bechmark on Ruby193rc1

 $ /usr/sbin/ab -n 500 -c 100 http://oddb.masa.org/
 This is ApacheBench, Version 2.3 <$Revision: 655654 $>
 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
 Licensed to The Apache Software Foundation, http://www.apache.org/

 Benchmarking oddb.masa.org (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Finished 500 requests


 Server Software:        Apache
 Server Hostname:        oddb.masa.org
 Server Port:            80

 Document Path:          /
 Document Length:        16334 bytes

 Concurrency Level:      100
 Time taken for tests:   23.782 seconds
 Complete requests:      500
 Failed requests:        0
 Write errors:           0
 Total transferred:      8371500 bytes
 HTML transferred:       8167000 bytes
 Requests per second:    21.02 [#/sec] (mean)
 Time per request:       4756.406 [ms] (mean)
 Time per request:       47.564 [ms] (mean, across all concurrent requests)
 Transfer rate:          343.76 [Kbytes/sec] received

 Connection Times (ms)
              min  mean[+/-sd] median   max
 Connect:        0    1   1.0      0       3
 Processing:   186 4454 2198.6   4364   17308
 Waiting:      186 4454 2198.6   4364   17308
 Total:        188 4455 2198.2   4364   17308
 WARNING: The median and mean for the initial connection time are not within a normal deviation
        These results are probably not that reliable.

 Percentage of the requests served within a certain time (ms)
 50%   4364
 66%   4663
 75%   4859
 80%   5019
 90%   5610
 95%   7043
 98%  10922
 99%  15974
 100%  17308 (longest request)

Note

• About 5 times more requests are passed than Ruby 1.8

Remove pointer link from just medical

Commit

• Replaced Wrikstoff link without pointer in just-medical and santesuisse flavor (oddb.org)