view · edit · sidebar · attach · print · history

< Niklaus.20161220-davaz-com-multithreaded | Index | Niklaus.20161214-sbsm-without-drb >>



  • Fix error in import swissmedic TextInfoPlugin
  • should support heavy workload
  • Keep in Mind



Fix error in import swissmedic TextInfoPlugin

Found the following error this morning from import_daily

Plugin: ODDB::TextInfoPlugin
Error: NoMethodError
Message: undefined method `packages' for ["rel", "stylesheet"]:Array
/var/www/ `block in packages'
/var/www/ `each'
/var/www/ `inject'
/var/www/ `packages'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
/var/www/ `block (2 levels) in report_problematic_names'
/var/www/ `each'
/var/www/ `block in report_problematic_names'
/var/www/ `open'
/var/www/ `report_problematic_names'
/var/www/ `parse_aips_download'
/var/www/ `import_swissmedicinfo'
/var/www/ `block in update_notify_simple'
/var/www/ `call'
/var/www/ `wrap_update'
/var/www/ `update_notify_simple'
/var/www/ `update_textinfo_swissmedicinfo'
/var/www/ `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/ `call'
/var/www/ `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

This was again the result of an out-of-memory error as shown by the following lines from log/oddb/debug/2016/12.log

2016-12-08 08:29:06 +0100: /var/www/ `update_active_agent': update_inactive_agent 00701/02 Polysorbatum 80  false dose 0  agent.pointer :!registration,00701!sequence,02!composition,29041064!inactive_agent,Polysorbatum 80.
2016-12-08 08:29:07 +0100: /var/www/ `trace_memory_useage': Using 16744 MB of memory. Limit is 16384. Swissmedic_do_tracing true
2016-12-08 08:29:07 +0100: /var/www/ `block in trace_memory_useage': Aborting as using 16744 MB of memory > than limit of 16384
2016-12-08 08:29:07 CETlog notify Error: swissmedic: start outgoing process ["log"]. Must attach 0 files and 0 parts.
2016-12-08 08:29:07 CETUtil.send_mail list_and_recipients ["log"]
2016-12-08 08:29:07 CET Util.log_and_deliver_mail to=["", ""] subject Report - Error: swissmedic - 12/2016 size 2884 with 0 attachments. Plugin: ODDB::SwissmedicPlugin
Error: SystemExit
Message: SystemExit

We must correct the e-mail text sent in this case.

Pushed commits Meaningfull error message for out of memory and Memory error when > 16 GB, not 2GB should support heavy workload

Updating to work with SBSM 1.3.5.

Asking myself why in src/state/works/movies.rb the init method cannot call @session.load_movies even when the @session is a DaVaz::Util::Session.

Before I started migrating to Rack there were many actions, loading, counting defined in src/util/app.rb which was a class not derived from a SBSM class. Now I made it derive from SBSM::App. Created and pushed a tag before_rack to see more easily the difference. Examining the diff generated via git diff -w before_rack > rack.diff

The load_movies method is currently accessible via instead of the earlie @session. Shall we make it accessible by implmenting a method_missing to SBSM::Session? Looks like a good idea. I even got the first time into the extend methods of lib/sbsm/state.rb where SBSM looks for @viral_modules. And indeed it detects the mod::EVENT_MAP.

Next question is now: The login rejects wrong username/password combinations, but why do we still see the login button after the login?

Using pry I see that I messed up the cookie handling. When stopping after the login I have in lib/sbsm/app.rb @ line 112

request.cookies  {"" =>"{:language=>\"en\"}", "_session_id"=>"10e524151d7f0da819f4222ecc1"}
request.env['rack.request.cookie_hash']   => {""=>"{:language=>\"en\"}", "_session_id"=>"10e524151d7f0da819f4222ecc1"}
request.env['rack.request.cookie_string'] => "; _session_id=10e524151d7f0da819f4222ecc1;; _session_id=98003e3199d3e50976a9c54352"

this results in lib/sbsm/session.rb @ line 217 to

=> {"language"=>"en", "zone"=>"works", "event"=>"movies"}
[16] pry(#<DaVaz::Util::Session>)> @cookie_input
=> {:language=>"en"}

Adding a unit test for this failure in sbsm. But also testing Looking at how Rack checks for cookies by looking at /home/niklaus/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/request.rb

204       def cookies
205         hash = fetch_header(RACK_REQUEST_COOKIE_HASH) do |k|
206           set_header(k, {})
207         end
208         string = get_header HTTP_COOKIE
210         return hash if string == get_header(RACK_REQUEST_COOKIE_STRING)
211         hash.replace Utils.parse_cookies_header get_header HTTP_COOKIE
212         set_header(RACK_REQUEST_COOKIE_STRING, string)
213         hash
214       end

Searching for a solution where I can use Rack methods to set and retrieve the indivual cookie items. Wondering why I don't see state_id in the second line when looking at the log file from test/feature/login_out_test.rb

::1 - - [19/Dec/2016:16:03:13 +0100] "GET /en/admin? &remember_me=1&flavor=&language=en&event=login&state_id=47327897449080&fragment= HTTP/1.1" 200 16 0.0635
D, [2016-12-19T16:03:13.393949 #15679] DEBUG -- : app.rb:108:in `call' starting session_id 102f4e878f9b29c39d75a5f2077  session DaVaz::Util::Session /en/works/movies/: cookies  are {"language"=>"en", "_session_id"=>"102f4e878f9b29c39d75a5f2077", ""=>"102f4e878f9b29c39d75a5f2077", "remember"=>"5c044e2134236fd263cca584634d2fbf14b575b1de0ca4c77f1b03dc28c0cf28", "name"=>""} @cgi CGI

Okay. Rack displays in its GET log also the content of request.env['QUERY_STRING']. This is a rack parameter I ignored till now. In my debug session its content was

The test/feature/login_out_test.rb continues to fail but allows now a login and logout. Looks like handling fragments in rack/test work differently than I expected.

I solved the original problem, that we got exception when running wrk -t12 -c40000 -d30s by using a module global mutex around session.process_rack as the application is apparently not thread safe in the current state. Number of request per second is around 30, which is much less than the 3280 seen when calling the same test on

Running all watir tests shows 6 Failures

  1) Error:
Selenium::WebDriver::Error::UnknownError: unknown error: Element <a name="serie_link" href="javascript:void(0);" class="serie-link even" id="ABD" onclick="return toggleShow('show', 'http://localhost:11090/en/gallery/ajax_rack/serie_id/ABD', null,
  'show_container', 'ABD');
">...</a> is not clickable at point (570, 146). Other element would receive the click: <div data-dojo-attach-point="thumbContainer" class="thumb-container"></div>
  1) Failure:
TestShop#test_shopping_cart_calculation_with_publications [test/feature/shop_test.rb:60]:
Expected: "Title of ArtObject 112"
  2) Failure:
TestShop#test_checkout [test/feature/shop_test.rb:204]:
browser text should match Your order has been succesfully sent. but is 
  3) Failure:
TestShop#test_checkout_fails_with_validation_error [test/feature/shop_test.rb:132]:
browser text should match Please fill out the fields that are marked with red. but is 
  4) Failure:
TestShop#test_checkout_fails_without_user_info [test/feature/shop_test.rb:118]:
browser text should match Please fill out the fields that are marked with red. but is 
  5) Failure:
TestShop#test_checkout_with_error [test/feature/shop_test.rb:49]:
Should show correct total of 568 CHF, but is Total       CHF 342.- / $ 273.- /  204.- Remove all items
  1) Error:
Net::ReadTimeout: Net::ReadTimeout
  1) Error:
Net::ReadTimeout: Net::ReadTimeout
view · edit · sidebar · attach · print · history
Page last modified on December 30, 2016, at 05:37 PM