We want to use fiddler to submit orders to sandoz.xmlconv. Must use a Windows Virtualbox machine for it. Installing from http://www.fiddler2.com/fiddler2/.
Modified C:\windows\sytem32\drivers\etc\hosts to add the IP of my used host sandoz.xmlconv.bbmb.ngiger.ch. Added confuse.xml. The order was placed correctly. But adding a breakpoint in lib/sbsm/session.rb when processing a POST request to see, how the POST looked liked.
From: /home/niklaus/git/sbsm/lib/sbsm/session.rb @ line 215 SBSM::Session#process_rack:
210: @request_path ||= rack_request.path
211: @rack_request = rack_request
212: @post_content = nil
213: if rack_request.request_method.eql?('POST')
214: binding.pry
=> 215: rack_request.params.each do |k, v|
216: # needed to test POST requests generated by curl (first parameter) or ARC (second parameter)
217: if /xml/i.match(k)
218: @post_content = "#{k} #{v}"
219: break
220: end
[1] pry(#<XmlConv::Util::Session>)> rack_request.params
=> {"<?xml version"=>
"\"1.0\" encoding=\"utf-8\"?>\n" +
"<customerOrder xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" version=\"1.0\" language=\"fr\" productDescriptionDesired=\"true\" communicationSoftwareId=\"ywesee Test\" xmlns=\"http://www.e-galexis.com/schemas/\">\n" +
" <client number=\"99\" password=\"\" />\n" +
" <orderHeader deliveryDate=\"2016-07-15\" referenceNumber=\"99\"/>\n" +
" <orderLines>\n" +
" <productOrderLine orderQuantity=\"2\">\n" +
" <pharmaCode id=\"2508375\" />\n" +
" </productOrderLine>\n" +
" <productOrderLine orderQuantity=\"2\">\n" +
" <pharmaCode id=\"5195126\" />\n" +
" </productOrderLine>\n" +
" </orderLines>\n" +
"</customerOrder>\n"}
Will test with the other xml files, too. Using
Pushed commit Use SBSM 1.4.5
Updating on thinpower as documented under http://dev.ywesee.com/Niklaus/20170613-oddb-org-rack#update-sandoz. This seems to work. Placed an order via confuse.xml which appears as 33855 http://80.218.53.88 19.06.2017 12:08:51 file:/.../var/output/99_20170619120851065.txt Bestellung via BBMB erfolgreich
Uncommenting the smtp_fields in //var/www/sandoz.xmlconv.bbmb.ch/etc/xmlconv.xml to avoid XmlConv2: Net::SMTPAuthenticationError in /var/www/sandoz.xmlconv.bbmb.ch/log/2017/06/19/app.log when polling sources.
Got another empty order 33856 http://194.158.242.196 19.06.2017 13:36:39 file:/.../sandoz.xmlconv.bbmb.ch/var/output
Looking at the debug in /var/www/sandoz.xmlconv.bbmb.ch/log/2017/06/19/app.log I see
D, [2017-06-19T13:36:38.802475 #20954] DEBUG -- : app.rb:118:in `call' starting session_id 1d9cd1a2446528e624caa4b789a session XmlConv::Util::Session /pharmatic: cookies are {} @cgi NilClass
D, [2017-06-19T13:36:38.802616 #20954] DEBUG -- : session.rb:227:in `process_rack' @post_content is <?xml version="1.0" encoding="utf-8"?><customerOrder xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" version="1.0" language="de" productDescriptionDesired="true" communicationSoftwareId="Pharmatic NETREQ.EXE V1.20.0.4 (Requestdate: 19.06.2017 13:38)" xmlns="http://www.e-galexis.com/schemas/"><client number="4100603668" password="7601001007307" /><orderHeader deliveryDate="2017-06-19" referenceNumber="173551" /><orderLines><productOrderLine orderQuantity="10"><pharmaCode id="2347937" /></productOrderLine><productOrderLine orderQuantity="2"><pharmaCode id="2324072" /></productOrderLine><productOrderLine orderQuantity="10"><pharmaCode id="5797463" /></productOrderLine></orderLines></customerOrder>
D, [2017-06-19T13:36:41.611455 #20954] DEBUG -- : session.rb:412:in `logout' logout "/pharmatic" setting @state 70116712063980 XmlConv::State::Login remember nil XmlConv::Util::KnownUser
D, [2017-06-19T13:36:41.612582 #20954] DEBUG -- : session.rb:256:in `block in process_rack' active_state.trigger state 70116712063980 XmlConv::State::Login remember nil
D, [2017-06-19T13:36:41.612651 #20954] DEBUG -- : session.rb:265:in `block in process_rack' Changing from 70116712063980 to state XmlConv::State::Login 70116712063980 remember nil XmlConv::Util::KnownUser
D, [2017-06-19T13:36:41.614907 #20954] DEBUG -- : app.rb:150:in `call' finish session_id.1 1d9cd1a2446528e624caa4b789a: matches response.headers['Set-Cookie']
194.158.242.196 - - [19/Jun/2017:13:36:41 +0200] "POST /pharmatic HTTP/1.1" 200 404 2.8132
I, [2017-06-19T13:36:43.522770 #20954] INFO -- : session.rb:114:in `initialize' initialize th #<SBSM::TransHandler:0x007f8aa319f930> validator XmlConv::Util::Validator app XmlConvApp multi_threaded false
D, [2017-06-19T13:36:43.522934 #20954] DEBUG -- : session.rb:135:in `initialize' session initialized #<XmlConv::Util::Session:0x007f8aab189768> with @cgi #<CGI:0x007f8aab183d18> multi_threaded false app 70116709207600 and user SBSM::UnknownUser @unknown_user XmlConv::Util::KnownUser
D, [2017-06-19T13:36:43.523002 #20954] DEBUG -- : app.rb:118:in `call' starting session_id 1fbfb6258628fb96a08c0159ead session XmlConv::Util::Session /: cookies are {} @cgi NilClass
D, [2017-06-19T13:36:43.523350 #20954] DEBUG -- : session.rb:412:in `logout' logout "/" setting @state 70116776320380 XmlConv::State::Login remember nil XmlConv::Util::KnownUser
D, [2017-06-19T13:36:43.524581 #20954] DEBUG -- : session.rb:256:in `block in process_rack' active_state.trigger state 70116776320380 XmlConv::State::Login remember nil
D, [2017-06-19T13:36:43.524651 #20954] DEBUG -- : session.rb:265:in `block in process_rack' Changing from 70116776320380 to state XmlConv::State::Login 70116776320380 remember nil XmlConv::Util::KnownUser
D, [2017-06-19T13:36:43.524739 #20954] DEBUG -- : session.rb:504:in `set_cookie_input' cookie_set_or_get language de
Uncommented all smtp_fields in /var/www/sandoz.xmlconv.bbmb.ch/etc/xmlconf.yml and restored as documented yesteray. Submitting adler.xml locally and on fastpower was okay.
My local setup does not have a polling_file declaration in etc/xmlconv.etc. I think that I have therefore never produced the error
E, [2017-06-19T12:45:10.887812 #16745] ERROR -- XmlConv2: Net::SMTPAuthenticationError 535-5.7.8 Username and Password not accepted. Learn more at /usr/local/ruby-2.4.0/lib/ruby/2.4.0/net/smtp.rb:981:in `check_auth_response' /usr/local/ruby-2.4.0/lib/ruby/2.4.0/net/smtp.rb:736:in `auth_plain' /usr/local/ruby-2.4.0/lib/ruby/2.4.0/net/smtp.rb:728:in `authenticate' /usr/local/ruby-2.4.0/lib/ruby/2.4.0/net/smtp.rb:565:in `do_start' /usr/local/ruby-2.4.0/lib/ruby/2.4.0/net/smtp.rb:518:in `start' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/mail-2.6.6/lib/mail/network/delivery_methods/smtp.rb:111:in `deliver!' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/mail-2.6.6/lib/mail/message.rb:2149:in `do_delivery' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/mail-2.6.6/lib/mail/message.rb:239:in `deliver' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/mail-2.6.6/lib/mail/mail.rb:141:in `deliver' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/mail.rb:27:in `notify' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/polling_manager.rb:197:in `rescue in block in poll_sources' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/polling_manager.rb:186:in `block in poll_sources' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/polling_manager.rb:179:in `block in load_sources' /usr/local/ruby-2.4.0/lib/ruby/2.4.0/psych/deprecated.rb:32:in `each' /usr/local/ruby-2.4.0/lib/ruby/2.4.0/psych/deprecated.rb:32:in `load_documents' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/polling_manager.rb:178:in `load_sources' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/polling_manager.rb:185:in `poll_sources' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/application.rb:172:in `block (2 levels) in start_polling' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/application.rb:170:in `loop' /var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.5/lib/xmlconv/util/application.rb:170:in `block in start_polling'
on oddb-ci2. Readding the polling_file on oddb_ci2.
On fastpower sandoz.xmlconf still uses via /usr/local/bin/xmlconv the version 1.0.1 of xmlconv. Therefore we see in /usr/local/bin/xmlconvd lines like
Skipping as RUBY_VERSON is 2.3.1 XmlConv::Util::Mail.notify xmlconv@ywesee.com -> ["ngiger@ywesee.com"] subject: XmlConv2 - Polling-Error
Patching /usr/local/lib/ruby/gems/2.3.0/gems/xmlconv-1.1.0/lib/xmlconv/util/polling_manager.rb to get some more info about the config.
Created on fastpower two variants of etc/xmlconv.yml, namely etc/xmlconv.yml.pre_rack and etc/xmlconv.yml.with_rack. etc/xmlconv.yml.with_rack has commented out the polling_file and does define the smtp-values for sending e-mails.
Re-adding the commands. If the errors persists, added the commit Debug output for polling and transactions
Using the sbsm 1.4.5 gem (locally built). The main page works. Searches for pharmacies like http://oddb-ci2.dyndns.org/de/gcc/search/zone/pharmacies/search_query/Glarus? work fine. Displaying a pharmacie works, too. Search by practitioner in Glarus and displaying works. Interactions work too. Migel search by Krücke worked, too. Search and display for hospital Glarus worked, too. Search and display for trademark owner Sandoz.
Debugging the trademark search for Nolvadex via breakpoint in to_html of lib/sbsm/state.rb
[1] pry(#<ODDB::Session>)> @state.class
=> ODDB::State::Drugs::Result
[2] pry(#<ODDB::Session>)> @state.model.class
=> ODDB::State::Drugs::Result
[5] pry(#<ODDB::Session>)> @state.model.package_count
=> 8
[6] pry(#<ODDB::Session>)> @state.model.atc_classes.class
=> Array
[7] pry(#<ODDB::Session>)> @state.model.atc_classes.size
=> 1
[9] pry(#<ODDB::Session>)> @state.model.atc_classes.first.code
=> "L02BA01"
[11] pry(#<ODDB::Session>)> @state.model.atc_classes.first.packages.size
=> 14
[12] pry(#<ODDB::Session>)> @state.model.atc_classes.first.active_packages.size
=> 8
[13] pry(#<ODDB::Session>)> @state.model.atc_classes.first.active_packages.first.name
=> "Tamec 20 mg, Filmtabletten"
[14] pry(#<ODDB::Session>)> @state.model.atc_classes.first.active_packages.first.iksnr
=> "54989"
[15] pry(#<ODDB::Session>)> @state.model.atc_classes.first.active_packages.collect{|x| x.iksnr}
=> ["54989", "54989", "49059", "49059", "49059", "49059", "39053", "39053"]
# it hangs while evaluating
# lib/sbsm/state.rb @ line 173 SBSM::State#to_html:
# 172: def to_html(context)
# => 173: name = view
Tracking down to super call of init in src/view/resulttemplate.rb -> src/view/publictemplate.rb -> lib/htmlgrid/composite.rb -> lib/htmlgrid/composite.rb (compose)
The warning NoMethodErrror undefined method `size' for nil:NilClass comes from lib/htmlgrid/composite.rb(around line 209)
Tracked the problem to the fact, that ajax script was not get loaded, as we had set multithreaded to false for SBSM. Now we must fix this problem. After changing multi_threaded: false to multi_threaded: true in oddb.org/src/util/session.rb searching drugs worked. Adding some additional log entries to track down the use of the session_id (cookie).
Here is the grep for session_id of a simple search for Nolvadex
D, [2017-06-19T11:12:22.261897 #11424] DEBUG -- : app.rb:118:in `call' starting session_id 1e764a8a48dd6a7d02bbcf0753e session ODDB::Session /de/gcc/home/: cookies are {"oddb-preferences"=>"language=de", "_session_id"=>"1e764a8a48dd6a7d02bbcf0753e", "resultview"=>"", "language"=>"de", "search_form"=>"plus", "__utma"=>"120231382.876330393.1496818562.1497595629.1497855722.8", "__utmb"=>"120231382.55.10.1497855722", "__utmc"=>"120231382", "__utmz"=>"120231382.1496818562.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)"} @cgi NilClass
D, [2017-06-19T11:12:26.806669 #11424] DEBUG -- : app.rb:152:in `call' finish session_id.2 1e764a8a48dd6a7d02bbcf0753e: headers {"Content-Length"=>"20931", "Content-Type"=>"text/html; charset=UTF-8", "Cache-Control"=>"private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0", "Pragma"=>"no-cache", "Expires"=>"Mon, 19 Jun 2017 09:12:14 UTC", "P3P"=>"CP='OTI NID CUR OUR STP ONL UNI PRE'", "Set-Cookie"=>"language=de; path=/\nsearch_form=plus; path=/"}
D, [2017-06-19T11:12:33.156202 #11424] DEBUG -- : app.rb:118:in `call' starting session_id 1e764a8a48dd6a7d02bbcf0753e session ODDB::Session /de/gcc/search/zone/drugs/search_query/Nolvadex/search_type/st_sequence: cookies are {"oddb-preferences"=>"language=de", "_session_id"=>"1e764a8a48dd6a7d02bbcf0753e", "resultview"=>"", "language"=>"de", "search_form"=>"plus", "__utmt"=>"1", "__utma"=>"120231382.876330393.1496818562.1497595629.1497855722.8", "__utmb"=>"120231382.56.10.1497855722", "__utmc"=>"120231382", "__utmz"=>"120231382.1496818562.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)"} @cgi NilClass
D, [2017-06-19T11:12:33.214132 #11424] DEBUG -- : app.rb:118:in `call' starting session_id 4179925c644b5446a409fabd21 session ODDB::Session /de/gcc/ajax_swissmedic_cat/reg/39053/seq/02/pack/039: cookies are {} @cgi NilClass
D, [2017-06-19T11:12:40.675502 #11424] DEBUG -- : app.rb:150:in `call' finish session_id.1 4179925c644b5446a409fabd21: matches response.headers['Set-Cookie']
D, [2017-06-19T11:12:40.690417 #11424] DEBUG -- : app.rb:118:in `call' starting session_id 18922391a433221ad4f4f6f0a90 session ODDB::Session /de/gcc/ajax_swissmedic_cat/reg/39053/seq/02/pack/047: cookies are {} @cgi NilClass
D, [2017-06-19T11:12:40.703536 #11424] DEBUG -- : app.rb:150:in `call' finish session_id.1 18922391a433221ad4f4f6f0a90: matches response.headers['Set-Cookie']
D, [2017-06-19T11:12:40.711646 #11424] DEBUG -- : app.rb:152:in `call' finish session_id.2 1e764a8a48dd6a7d02bbcf0753e: headers {"Content-Length"=>"33300", "Content-Type"=>"text/html; charset=UTF-8", "Cache-Control"=>"private, no-store, no-cache, must-
The main session 1e764a8a48dd6a7d02bbcf0753e started to sub-session 4179925c644b5446a409fabd21 and 18922391a433221ad4f4f6f0a90. Using http_fox to see whether in the pre-rack we used also three sessions. Here some snippets from the httpfox
(Request-Zeile) GET / HTTP/1.1 Host ch.oddb.org User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0 Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language de-CH,en-US;q=0.7,en;q=0.3 Accept-Encoding gzip, deflate Cookie __utma=69243458.1492321151.1496340195.1496340195.1496340195.1; __utmz=69243458.1496340195.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); ["oddb-preferences=language%3Dde (Request-Zeile) GET /de/gcc/search/zone/drugs/search_query/Nolvadex/search_type/st_sequence HTTP/1.1 Host ch.oddb.org User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0 Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language de-CH,en-US;q=0.7,en;q=0.3 Accept-Encoding gzip, deflate Referer http://ch.oddb.org/ Cookie __utma=69243458.1492321151.1496340195.1496340195.1497864556.2; __utmz=69243458.1496340195.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); ["oddb-preferences=language%3Dde; _session_id=bf7963d64eec7cef454262a7ebfdfe1b; __utmb=69243458.1.10.1497864556; __utmc=69243458; __utmt=1
But I do not see any request for de/gcc/ajax_swissmedic_cat/reg/39053/seq/02/pack/039. mod_ruby based SBSM fetched this content in a different way. But on thinpower I using grep -w GET /var/www/oddb.org/log/oddb/access_log showed also many entries like 62.12.131.38 - - [19/Jun/2017:00:00:10 +0200] "GET /en/gcc/ajax_swissmedic_cat/reg/58327/seq/02/pack/020 HTTP/1.1" 200 1287 "-" "Ruby"
To search for long lasting requests it would suffice to analyse the last value when grepping for the word GET in the app_log, as there we have the duration (in seconds) of the query, e.g. the (first search) vor Nolvadex took 7.557 seconds.
172.25.1.75 - - [19/Jun/2017:11:12:40 +0200] "GET /de/gcc/search/zone/drugs/search_query/Nolvadex/search_type/st_sequence HTTP/1.1" 200 33300 7.5570
whereas the second one took less and 0.1 seconds (172.25.1.75 - - [19/Jun/2017:11:27:04 +0200] "GET /de/gcc/search/zone/drugs/search_query/Nolvadex/search_type/st_sequence HTTP/1.1" 200 33300 0.0708)
Checking exports. Here the lists of the exports run and the generateded files
does not found any chapter/description
Watir tests cannot login, whereas testing by myself I had no problem logging in. Why? Firefox pops up now "Insecure password warning in Firefox". Can we suppress this?