---
Zeno specified the following criterias:
Feld CMUT 1 = Datensatz (Artikel) neu (im Handel) (nicht verwendet) 2 = Datensatz (Artikel) Update 3 = Datensatz (Artikel) ausser Handel (nicht verwendet)
Looking at SALECD-fields in data from Juni 2014 I find
<PHAR>0598003</PHAR>
<SALECD>A</SALECD>
<DSCRD>SOFRADEX Gtt Auric</DSCRD>
and the corresponding items in zurrose_transfer.dat
1130598003SOFRADEX Gtt Auric 8 ml 000718001545300B120130076803169501572
And I find only the values I or A in the oddb_article.xml.
First added the failing tests to spec/extractor_spec.rb which reports now
ailures:
1) Oddb2xml::ZurroseExtractor when SOFRADEX is given should set the correct SALECD code
Failure/Error: it "should set the correct SALECD code" do expect(subject.to_hash.values.first[:SALECD]).to eq("A") end
expected: "A"
got: nil
(compared using ==)
# ./spec/extractor_spec.rb:173:in `block (3 levels) in <top (required)>'
2) Oddb2xml::ZurroseExtractor when SELSUN Shampoo is given should set the correct SALECD code
Failure/Error: it 'should set the correct SALECD code' do expect(subject.to_hash.values.first[:SALECD]).to eq("I") end
expected: "I"
got: nil
(compared using ==)
# ./spec/extractor_spec.rb:188:in `block (3 levels) in <top (required)>'
Finished in 0.56301 seconds
36 examples, 2 failures, 8 pending
Failed examples:
rspec ./spec/extractor_spec.rb:173 # Oddb2xml::ZurroseExtractor when SOFRADEX is given should set the correct SALECD code
rspec ./spec/extractor_spec.rb:188 # Oddb2xml::ZurroseExtractor when SELSUN Shampoo is given should set the correct SALECD code
Fixed extractor. Needed more time to fix the builder. Now looking that option -f dat should also containt the correct data from transfer.dat.
Refactored @prices -> @infos_zur_rose. Added field :cmut to ZurRoseExtractor and used it to create the SALECD field. Added test for builder.
Running the tests gives me the following error
bundle exec bin/oddb2xml --log -x address
2014-08-04 15:17:28: Downloader from https://www.medregbm.admin.ch/Publikation/CreateExcelListBetriebs for Oddb2xml::MedregbmDownloader
2014-08-04 15:17:28: download_as file /opt/src/oddb2xml/downloads/medregbm_company.txt via /opt/src/oddb2xml/medregbm_company.txt from https://www.medregbm.admin.ch/Publikation/CreateExcelListBetriebs
2014-08-04 15:17:28: Downloader from https://www.medregbm.admin.ch/Publikation/CreateExcelListMedizinalPersons for Oddb2xml::MedregbmDownloader
2014-08-04 15:17:28: download_as file /opt/src/oddb2xml/downloads/medregbm_person.txt via /opt/src/oddb2xml/medregbm_person.txt from https://www.medregbm.admin.ch/Publikation/CreateExcelListMedizinalPersons
/home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent/ssl_reuse.rb:70:in `connect': SSL_connect SYSCALL returned=5 errno=0 state=unknown state (OpenSSL::SSL::SSLError)
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent/ssl_reuse.rb:70:in `block in connect'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/1.9.1/timeout.rb:55:in `timeout'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/1.9.1/timeout.rb:100:in `timeout'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent/ssl_reuse.rb:70:in `connect'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/1.9.1/net/http.rb:756:in `do_start'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/1.9.1/net/http.rb:751:in `start'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:700:in `start'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:631:in `connection_for'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:258:in `fetch'
from /home/niklaus/.rbenv/versions/1.9.3-p547/lib/ruby/gems/1.9.1/gems/mechanize-2.5.1/lib/mechanize.rb:407:in `get'
from /opt/src/oddb2xml/lib/oddb2xml/downloader.rb:21:in `download_as'
from /opt/src/oddb2xml/lib/oddb2xml/downloader.rb:178:in `download'
from /opt/src/oddb2xml/lib/oddb2xml/cli.rb:161:in `block in download'
Using wget on the command line works fine with the URL given. When reverting the gem to 1.8.5 I get also the same error.
Using the version 1.9.3p0 install in /usr/local/bin did not show this problem.
Pushed commits
Zeno reported the following error when running oddb2xml -f dat -a nonpharma
/usr/lib64/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:55:in
`require': iconv will be deprecated in the future, use String#encode
instead.
/usr/local/lib64/ruby/gems/1.9.1/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:304:in
`fetch': 404 => Net::HTTPNotFound for
https://www.swissmedic.ch/daten/00080/00251/index.html?lang=de --
unhandled response (Mechanize::ResponseCodeError)
from /usr/local/lib64/ruby/gems/1.9.1/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:949:in
`response_redirect'
from /usr/local/lib64/ruby/gems/1.9.1/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:299:in
`fetch'
from /usr/local/lib64/ruby/gems/1.9.1/gems/mechanize-2.5.1/lib/mechanize.rb:407:in
`get'
from /usr/local/lib64/ruby/gems/1.9.1/gems/oddb2xml-1.5.8/lib/oddb2xml/downloader.rb:239:in
`download'
from /usr/local/lib64/ruby/gems/1.9.1/gems/oddb2xml-1.5.8/lib/oddb2xml/cli.rb:202:in
`block in download'
Could not reproduce it here with version 1.8.5 nor 1.8.6.
ruby -v
ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]
oddb2xml -f dat -a nonpharma 2>&1 | tee zeno.log
DE
Pharma products: 15747
NonPharma products: 28723
FR
Pharma products: 15747
NonPharma products: 28723
2014-08-04 14:32:14 +0200: 103 done. Took 57 seconds
May it was just an intermittent failure, that swissmedic was not correctly online? Runnig with the log to see, whether the page mentioned appeared
2014-08-04 14:35:51: Downloader from https://index.ws.e-mediat.net/Swissindex/Pharma/ws_Pharma_V101.asmx?WSDL for Oddb2xml::SwissIndexDownloader
2014-08-04 14:35:51: Downloader from for Oddb2xml::MigelDownloader
2014-08-04 14:35:51: Downloader from http://www.swissmedic.ch/arzneimittel/00156/00221/00222/00230/index.html?lang=de for Oddb2xml::SwissmedicDownloader
2014-08-04 14:35:51: Downloader from for Oddb2xml::LppvDownloader
2014-08-04 14:35:51: Downloader from https://index.ws.e-mediat.net/Swissindex/Pharma/ws_Pharma_V101.asmx?WSDL for Oddb2xml::SwissIndexDownloader
2014-08-04 14:35:51: Downloader from https://index.ws.e-mediat.net/Swissindex/NonPharma/ws_NonPharma_V101.asmx?WSDL for Oddb2xml::SwissIndexDownloader
2014-08-04 14:35:51: download_as file /home/niklaus/oddb2xml/downloads/oddb2xml_files_nonpharma.xls via /home/niklaus/oddb2xml/oddb2xml_files_nonpharma.xls from https://github.com/zdavatz/oddb2xml_files/raw/master/NON-Pharma.xls
2014-08-04 14:35:51: Downloader from https://index.ws.e-mediat.net/Swissindex/NonPharma/ws_NonPharma_V101.asmx?WSDL for Oddb2xml::SwissIndexDownloader
2014-08-04 14:35:51: Downloader from for Oddb2xml::ZurroseDownloader
2014-08-04 14:35:51: Downloader from for Oddb2xml::BMUpdateDownloader
2014-08-04 14:35:51: Downloader from for Oddb2xml::BagXmlDownloader
2014-08-04 14:35:51: download_as file /home/niklaus/oddb2xml/downloads/oddb2xml_files_lppv.txt via /home/niklaus/oddb2xml/oddb2xml_files_lppv.txt from https://raw.github.com/zdavatz/oddb2xml_files/master/LPPV.txt
2014-08-04 14:35:51: ZurroseDownloader 157 download http://zurrose.com/fileadmin/main/lib/download.php?file=/fileadmin/user_upload/downloads/ProduktUpdate/IGM11_mit_MwSt/Vollstamm/transfer.dat @url
2014-08-04 14:35:51: download_as file /home/niklaus/oddb2xml/downloads/oddb2xml_files_bm_update.txt via /home/niklaus/oddb2xml/oddb2xml_files_bm_update.txt from https://raw.github.com/zdavatz/oddb2xml_files/master/BM_Update.txt
2014-08-04 14:35:51: BagXmlDownloader 188: /home/niklaus/oddb2xml/XMLPublications.zip
2014-08-04 14:35:51: MigelExtractor added 57 migel items
2014-08-04 14:35:51: download_as file /home/niklaus/oddb2xml/downloads/zurrose_transfer.dat via /home/niklaus/oddb2xml/zurrose_transfer.dat from http://zurrose.com/fileadmin/main/lib/download.php?file=/fileadmin/user_upload/downloads/ProduktUpdate/IGM11_mit_MwSt/Vollstamm/transfer.dat
2014-08-04 14:35:51: LppvExtractor added 575 lppvs
2014-08-04 14:35:51: BMUpdateExtractor added 717 flags
2014-08-04 14:35:51: read_xml_from_zip target is (?-mix:Preparations.xml) zip: /home/niklaus/oddb2xml/downloads/XMLPublications.zip true
2014-08-04 14:35:51: read_xml_from_zip 108: reading Preparations.xml
2014-08-04 14:35:51: zurrose xml 13750407 bytes
2014-08-04 14:35:52: zurrose added 22211 items from xml with 13750407 bytes
2014-08-04 14:35:53: read_xml_from_zip saved as /home/niklaus/oddb2xml/downloads/Preparations.xml
2014-08-04 14:36:07: BagXmlDownloader added 9771 items
2014-08-04 14:36:24: download_finished saved as /home/niklaus/oddb2xml/downloads/swissmedic_package.xlsx 2678293 bytes.
2014-08-04 14:36:47: Start build
2014-08-04 14:36:47: to_dat subject dat for Oddb2xml::Builder
2014-08-04 14:36:47: prepare_articles starting with no articles
2014-08-04 14:36:47: prepare_articles done. Added 0 prices. Total 15785
2014-08-04 14:36:48: to_dat subject with_migel_dat for Oddb2xml::Builder
2014-08-04 14:36:48: prepare_articles starting with no articles
2014-08-04 14:36:48: prepare_articles done. Added 0 prices. Total 28032
DE
Pharma products: 15747
NonPharma products: 28723
FR
Pharma products: 15747
NonPharma products: 28723
2014-08-04 14:36:49 +0200: 103 done. Took 58 seconds
This was eays. I had forgotten that the a plugin can receive options when getting initialized. And as we have no good unit test for these kind of tests, they slip in too easily.
Fixed with commit Add optionnal second argument when creating EphaInteractionPlugin