Got yesterday the following error
NoMethodError at /de/sbsm undefined method 'deliveries' for nil:NilClass transction.model.deliveries:each_with_index do | delivery, idx|
and while handling a transaction
I, [2017-06-20T15:32:45.197687 #27720] INFO -- : application.rb:44:in `_execute' Starting_transaction 33901 I, [2017-06-20T15:32:45.197898 #27720] INFO -- : application.rb:51:in `rescue in _execute' error_transaction 33901 error "\xC3" from ASCII-8BIT to UTF-8 ["/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.6/lib/xmlconv/util/transaction.rb:32:in `encode'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.6/lib/xmlconv/util/transaction.rb:32:in `execute'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.6/lib/xmlconv/util/application.rb:45:in `_execute'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.6/lib/xmlconv/util/application.rb:36:in `execute'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.6/lib/xmlconv/util/application.rb:148:in `execute_with_response'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/xmlconv-1.1.6/lib/xmlconv/state/login.rb:30:in `initialize'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/sbsm-1.4.5/lib/sbsm/session.rb:411:in `new'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/sbsm-1.4.5/lib/sbsm/session.rb:411:in `logout'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/sbsm-1.4.5/lib/sbsm/session.rb:251:in `block in process_rack'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/sbsm-1.4.5/lib/sbsm/session.rb:235:in `synchronize'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/sbsm-1.4.5/lib/sbsm/session.rb:235:in `process_rack'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/sbsm-1.4.5/lib/sbsm/app.rb:119:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/lint.rb:49:in `_call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/lint.rb:37:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/show_exceptions.rb:23:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/content_length.rb:15:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/static.rb:149:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/common_logger.rb:33:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/tempfile_reaper.rb:15:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/lint.rb:49:in `_call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/lint.rb:37:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/show_exceptions.rb:23:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/common_logger.rb:33:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/chunked.rb:54:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/content_length.rb:15:in `call'", "/var/www/sandoz.xmlconv.bbmb.ch/vendor/ruby/2.4.0/gems/rack-2.0.3/lib/rack/handler/webrick.rb:86:in `service'", "/usr/local/ruby-2.4.0/lib/ruby/2.4.0/webrick/httpserver.rb:140:in `service'", "/usr/local/ruby-2.4.0/lib/ruby/2.4.0/webrick/httpserver.rb:96:in `run'", "/usr/local/ruby-2.4.0/lib/ruby/2.4.0/webrick/server.rb:290:in `block in start_thread'"]
Submitting the same XML to oddb-ci2 worked without a hitch. Why?
On fastpower we as language enviornments LC_COLLATE=POSIX and LANG=en_US.UTF-8. On oddb-ci2 we only have LANG=en_US.UTF-8.
Ran sandox.xmlconv with LC_COLLATE=POSIX and still got not error. Using pry
[2] pry(#<XmlConv::Util::Transaction>)> ENV['LC_COLLATE'] => "POSIX" [3] pry(#<XmlConv::Util::Transaction>)> @input.valid_encoding? => true [5] pry(#<XmlConv::Util::Transaction>)> @input.encoding => #<Encoding:UTF-8> [6] pry(#<XmlConv::Util::Transaction>)> puts @input <?xml version "1.0" encoding="utf-8"?><customerOrder xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" compressionDesired="false" productDescriptionDesired="true" backLogDesired="true" language="fr" roundUpForCondition="true" version="1.0"> <client number="4100602840" password="MDPOK"/> <orderHeader deliveryDate="2017-06-20" referenceNumber="200620171" mannerOfTransport="tour" phoneCallDesired="false" urgent="false"> <deliveryAddress line1="16, bd des Philosophes" line4="Pharmacie des PHILOSOPHES" line5PostalCode="1205" line5City="Genève"> <addressLine2And3Text line2="Genève" line3="des PHILOSOPHES"/> </deliveryAddress> </orderHeader> <orderLines> <productOrderLine orderQuantity="2" roundUpForCondition="true" backLogDesired="true"> <pharmaCode id="3023007"/> </productOrderLine> <productOrderLine orderQuantity="2" roundUpForCondition="true" backLogDesired="true"> <pharmaCode id="1586108"/> </productOrderLine> <productOrderLine orderQuantity="1" roundUpForCondition="true" backLogDesired="true"> <pharmaCode id="3463410"/> </productOrderLine> <productOrderLine orderQuantity="1" roundUpForCondition="true" backLogDesired="true"> <pharmaCode id="5621416"/> </productOrderLine> <productOrderLine orderQuantity="1" roundUpForCondition="true" backLogDesired="true"> <pharmaCode id="4714216"/> </productOrderLine> </orderLines></customerOrder>
Trying with test/data/umlaut_utf_8_problem.xml and LC_COLLATE=posix did not produce any error. Looking at our unit tests. We have one unit test named test_execute. Creating a new one test_execute_utf_8_problem. Was able to create a test case important steps were
File.open("#{test_data_dir}/umlaut_iso8859.xml", 'rb').read
iconv -f utf8 -t iso8859-14 test/data/umlaut_utf_8_problem.xml -o test/data/umlaut_iso8859.xml
Now the test fails like follows
1) Error:
XmlConv::Util::TestTransaction#test_execute_utf_8_problem:
Encoding::UndefinedConversionError: "\xEA" from ASCII-8BIT to UTF-8
/home/niklaus/git/xmlconv/lib/xmlconv/util/transaction.rb:36:in `encode'
/home/niklaus/git/xmlconv/lib/xmlconv/util/transaction.rb:36:in `execute'
test/test_util/transaction.rb:126:in `test_execute_utf_8_problem'
And Zeno was right using @input.force_encoding("UTF-8") fixed the problem, but only if we encode it afterwards a second time.
Pushed the following commits
Release xmlconv 1.1.7 and pushed commit Use xmlconv 1.1.7
Installed and started it on fastpower. Tested it with confuse.xml and umlaut_ . Okay. Also already got another command where we have in the log/2017/06/21/app.logfile an entry
D, [2017-06-21T12:04:53.388517 #31495] DEBUG -- : login.rb:17:in `initialize' xml_src ASCII-8BIT now <?xml version="1.0" encoding="utf-8"?><customerOrder xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" compressionDesired="false" productDescriptionDesired="true" backLogDesired="true" language="fr" roundUpForCondition="true" version="1.0">
Remarked that my curl test pushed it as UTF-8 as seen in the log
D, [2017-06-21T12:04:50.042920 #31495] DEBUG -- : login.rb:17:in `initialize' xml_src UTF-8 now <?xml version "1.0" encoding "utf-8"?><customerOrder xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" compressionDesired="false" productDescriptionDesired="true" backLogDesired="true" language="fr" roundUpForCondition="true" version="1.0"> <client number="99" password="" /> <orderHeader deliveryDate="2017-06-06" referenceNumber="060620172" mannerOfTransport="tour" phoneCallDesired="false" urgent="false"> <deliveryAddress line1="136, Ch. De-La-Montagne" line4="Pharmacie GOUDA" line5PostalCode="1224" line5City="Ch�ne-Bougeries"> <addressLine2And3Text line2="Ch�ne-Bougeries" line3="GOUDA" /> </deliveryAddress> </orderHeader> <orderLines> <productOrderLine orderQuantity="10" roundUpForCondition="true" backLogDesired="true"> <pharmaCode id="3463433" /> </productOrderLine> <productOrderLine orderQuantity="2" roundUpForCondition="true" backLogDesired="true"> <pharmaCode id="3029702" /> </productOrderLine> </orderLines></customerOrder>
Trying to add --raw option and the --data-binary, eg. curl -v http://sandoz.xmlconv.bbmb.ch/propharma --data-binary @test/data/umlaut_iso8859.xml. But this failed, too. The uploaded content as seen by sandoz.xmlconv.bbmb.ch was always UTF-8.
Export jobs to check are:
Running all watirs tests produced the following resumen
Finished in 53 minutes 17 seconds (files took 24.29 seconds to load) 179 examples, 59 failures, 34 pending Failed examples: rspec ./spec/rezept_and_instantsearch_spec.rb:571 # ch.oddb.org should be possible to print a presciption with 10 drugs rspec ./spec/rezept_and_instantsearch_spec.rb:250 # ch.oddb.org should be possible to add drugs after delete_all when neither ZSR nor comment given rspec ./spec/rezept_and_instantsearch_spec.rb:452 # ch.oddb.org should not loose existing comment after adding a new prescription rspec ./spec/rezept_and_instantsearch_spec.rb:288 # ch.oddb.org should be possible to add drugs after delete_all when no ZSR is given but a comment rspec ./spec/searchbar_spec.rb[1:2] # ch.oddb.org should be possible to find 1,25-Dihydroxycholecalciferol when searching via 125 in analysen rspec ./spec/searchbar_spec.rb:116 # ch.oddb.org should should be possible to add and delete several drugs: rspec ./spec/searchbar_spec.rb:339 # ch.oddb.org should show no drugs for Fortex via unwanted effects search rspec ./spec/searchbar_spec.rb:135 # ch.oddb.org should should be possible to delete all drugs: rspec ./spec/searchbar_spec.rb[1:5] # ch.oddb.org should should work (58868 Actemra) with Dos./Anw. and Kinder rspec ./spec/searchbar_spec.rb:391 # ch.oddb.org should set best_result when searching Rivoleve via search_type rspec ./spec/searchbar_spec.rb[1:6] # ch.oddb.org should should work (58868 Actemra) with Interakt. and Tocilizumab rspec ./spec/searchbar_spec.rb[1:4] # ch.oddb.org should should work (58868 Actemra) with Unerw.Wirkungen and Kopfschmerzen rspec ./spec/admin_spec.rb[1:1] # ch.oddb.org should be possible to upload dummy_patinfo.pdf to a given package rspec ./spec/admin_spec.rb[1:2] # ch.oddb.org should be possible to upload dummy_patinfo_2.pdf to a given package rspec ./spec/admin_spec.rb:129 # ch.oddb.org should be possible to create a CompanyUser rspec ./spec/evidentia_spec.rb:49 # ch.oddb.org should list C09DB02 before C09DX03 when looking for Sevikar rspec ./spec/evidentia_spec.rb:172 # ch.oddb.org should list trademark first e.g. Duodopa rspec ./spec/evidentia_spec.rb:165 # ch.oddb.org should display lamivudin with SO and SG in category (price comparision) rspec ./spec/evidentia_spec.rb:60 # ch.oddb.org should list Keppra at the top when searching for Levetiracetam rspec ./spec/evidentia_spec.rb:143 # ch.oddb.org should contain a link to the fachinfo for Lamivudin-Zidovudin rspec ./spec/evidentia_spec.rb:72 # ch.oddb.org should list all SL products before the Non-SL rspec ./spec/evidentia_spec.rb:127 # ch.oddb.org should contain a link to the FI for the drug when in price comparison rspec ./spec/evidentia_spec.rb:117 # ch.oddb.org should contain a link to the price comparision in price public rspec ./spec/evidentia_spec.rb:218 # ch.oddb.org should list Levetiracetam Desitin with a link to the product overview rspec ./spec/evidentia_spec.rb:66 # ch.oddb.org should list Levetiracetam Desitin at the top when searching for Levetiracetam Desitin rspec ./spec/evidentia_spec.rb:93 # ch.oddb.org should not contain a column Fachinfo rspec ./spec/evidentia_spec.rb:106 # ch.oddb.org should contain a link to the limiation in Sevikar HCT preparation rspec ./spec/evidentia_spec.rb:210 # ch.oddb.org should display Cellcept before other rspec ./spec/evidentia_spec.rb:151 # ch.oddb.org should display a limitation link for Sevikar HCT rspec ./spec/download_spec.rb:73 # ch.oddb.org should be possible to run grant_download oddb2.csv rspec ./spec/download_spec.rb:27 # ch.oddb.org should download the results of a search to Marcoumar rspec ./spec/download_spec.rb:45 # ch.oddb.org should be possible to run a bin/admin command rspec ./spec/paypal_spec.rb:86 # ch.oddb.org should be possible to checkout oddb.csv via paypal rspec ./spec/paypal_spec.rb:177 # ch.oddb.org should be possible to cancel a paypal before login rspec ./spec/paypal_spec.rb:113 # ch.oddb.org should be checkout via paypal a poweruser rspec ./spec/paypal_spec.rb:188 # ch.oddb.org should be possible to cancel a paypal after login but before paying rspec ./spec/paypal_spec.rb:163 # ch.oddb.org should not download a CSV file if the payment was not accepted rspec ./spec/paypal_spec.rb:140 # ch.oddb.org should return a correct link to a CSV file if the payment is okay rspec ./spec/interactions_spec.rb:276 # ch.oddb.org after delete all drugs a new search must be possible rspec ./spec/snapback_spec.rb[1:6] # ch.oddb.org snapback should a working Home button for step 4 rspec ./spec/snapback_spec.rb[1:5] # ch.oddb.org snapback should a working Home button for step 3 rspec ./spec/snapback_spec.rb:107 # ch.oddb.org snapback should allow going back, then forward rspec ./spec/snapback_spec.rb[1:4] # ch.oddb.org snapback should a working Home button for step 2 rspec ./spec/snapback_spec.rb:174 # ch.oddb.org snapback should work follow correctly the expected paths rspec ./spec/address_correction_spec.rb:84 # ch.oddb.org should be possible to correct an address for a company rspec ./spec/smoketest_spec.rb:279 # ch.oddb.org should show correct Tramal Tropfen Lösung zum Einnehmen mit Dosierpumpe (4788/01/035) rspec ./spec/smoketest_spec.rb:354 # ch.oddb.org should download the results of a search rspec ./spec/smoketest_spec.rb[1:31] # ch.oddb.org should have a working instant search for Augmentin and takeskosten rspec ./spec/smoketest_spec.rb:230 # ch.oddb.org should have a link to the english language versions rspec ./spec/smoketest_spec.rb:127 # ch.oddb.org should display lamivudin with SO and SG in category (price comparision) rspec ./spec/smoketest_spec.rb:107 # ch.oddb.org should display a limitation link for Sevikar rspec ./spec/smoketest_spec.rb:397 # ch.oddb.org should display the correct calculation for Bicalutamid Actavis rspec ./spec/rss_spec.rb[1:1] # ch.oddb.org should have a working RSS-feed hpc rspec ./spec/rss_spec.rb[1:3] # ch.oddb.org should have a working RSS-feed price_rise rspec ./spec/rss_spec.rb[1:5] # ch.oddb.org should have a working RSS-feed sl_introduction rspec ./spec/rss_spec.rb[1:4] # ch.oddb.org should have a working RSS-feed recall rspec ./spec/rss_spec.rb[1:2] # ch.oddb.org should have a working RSS-feed price_cut rspec ./spec/rss_spec.rb[1:6] # ch.oddb.org should have a working RSS-feed fachinfo rspec ./spec/rss_spec.rb:67 # ch.oddb.org should have a working fachinfo-2008
Made some changes to handle the different flavor as I cannot use the approach from mod_ruby where we had an environment flag SetEnv DEFAULT_FLAVOR gcc in the apache conf. But it does not work correctly for oddb-ci2.dyndns.org. Fixed by not converting nil values to string.
Now the flavors work where they a key in CssTemplate and this key is the start of the servername, eg. santesuisse, evidentia. But must fix it for other cases like homeopathy, phyto-pharma, oekk.
Pushed the commits
Remarked that when running the test oddbapp*.rb the log_size thread is started many times. Added a guard to ensure, that it is started only once. Now the unit test pass a lot faster.
Fixing some other failing tests in oddbapp_2.rb. Changed sbsm/session_store.rb to handle clear method.
Pushed commit