20130422-debug-medwin-updater

<< | Index | >>

Summary

• Update MedWin Updater

Commits

• Updater MedWin Updater
  • Add gzip content support at POST in net/http (also actually not gzip content)
  • Add some parameters for new form on medwin.ch
• Create Job for update_medwin_companies

Index

• Update MedWin Updater

Update MedWin Updater

[46] 1.9.3-p194(#<ODDB::MedData::Session>)> resp.response.to_hash
=> {"connection"=>["Keep-Alive"],
 "transfer-encoding"=>["chunked"],
 "date"=>["Mon, 22 Apr 2013 05:31:22 GMT"],
 "content-type"=>["text/html; charset=utf-8"], #=> UTF-8
 "content-encoding"=>["gzip"],
 "server"=>["Microsoft-IIS/7.0"],
 "cache-control"=>["private"],
 "x-aspnet-version"=>["2.0.50727"],
 "x-powered-by"=>["ASP.NET"],
 "vary"=>["Accept-Encoding"]}
[47] 1.9.3-p194(#<ODDB::MedData::Session>)> resp.response.body
=> "\x1F\x8B\b\x00\x00\x00\x00\x00\x00\v\xC5\\\xDBr\xE2H\x9A\xBE\x9E\x8E\xE8wP3\xBB\xD3UQ\xDD\xE6d\xBB\xEC*\xDB\e\x12H\x02\f\xD
8\x02qRGG\x85NH
...

I tried some encodings for invalid response from medwin.ch

[2] 1.9.3-p194(#<ODDB::MedData::Session>)> body.encoding
=> #<Encoding:ASCII-8BIT>
[3] 1.9.3-p194(#<ODDB::MedData::Session>)> body.encode('windows-1252')
Encoding::UndefinedConversionError: "\x8B" to UTF-8 in conversion from ASCII-8BIT to UTF-8 to WINDOWS-1252
from (pry):6:in `encode'
[4] 1.9.3-p194(#<ODDB::MedData::Session>)> body.encode('us-ascii')
Encoding::UndefinedConversionError: "\x8B" to UTF-8 in conversion from ASCII-8BIT to UTF-8 to US-ASCII
from (pry):7:in `encode'
[5] 1.9.3-p194(#<ODDB::MedData::Session>)> body.encode('iso-8859-1')
Encoding::UndefinedConversionError: "\x8B" to UTF-8 in conversion from ASCII-8BIT to UTF-8 to ISO-8859-1
from (pry):8:in `encode'
[6] 1.9.3-p194(#<ODDB::MedData::Session>)> body.encode('windows-1258')
Encoding::ConverterNotFoundError: code converter not found (ASCII-8BIT to Windows-1258)
from (pry):9:in `encode'
[10] 1.9.3-p194(#<ODDB::MedData::Session>)> body.encode('iso-8859-2')
Encoding::UndefinedConversionError: "\x8B" to UTF-8 in conversion from ASCII-8BIT to UTF-8 to ISO-8859-2

I could not find encoding of these broken response.
Content-Type is UTF-8. but Response is not correct UTF-8.

To avoid error (But cannot get correkt response)

body.encode('UTF-8', :undef => :replace, :invalid => :replace, :replace => ' ')

Link

• http://stackoverflow.com/questions/2982677/ruby-1-9-invalid-byte-sequence-in-utf-8

Then I got a solution for this response.
This response needs to be handled as gzip content.

• https://github.com/ruby/ruby/blob/v1_9_3_327/lib/net/http.rb#L1031

like this:

Zlib::GzipReader.new(StringIO.new(body), encoding: "ASCII-8BIT").read

If response is not gzip (But server returns content as gzip), try to rescue.

        # content
        body = if @response['content-encoding'] == 'gzip'
          begin
            Zlib::GzipReader.new(StringIO.new(body), :encoding => "ASCII-8BIT").read
          rescue Zlib::GzipFile::Error, Zlib::Error # actually this is not gzip :(
            body
          end
        end

And I updated some post parameters for new Form on medwin.ch.
Finaly, MedWin-Updater works fine, again.

[1] 1.9.3-p194(#<ODDB::MedwinCompanyPlugin>)> data[:phone]
=> "041 8206651 "

commit

• Updater MedWin Updater
  • Add gzip content support at POST in net/http (also actually not gzip content)
  • Add some parameters for new form on medwin.ch
• Create Job for update_medwin_companies

NOTE

ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.length
-> 1
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.class
-> ODDB::Address2
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.name
-> 
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.title
-> 
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.fon
-> []
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.fax
-> []
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.location
-> 8045 Zürich
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.street
-> Grubenstrasse
ch.oddb> company_by_name("Bayer (Schweiz) AG").addresses.first.address
-> Grubenstrasse  6