Must fix the error
Plugin: ODDB::RssPlugin Error: Mechanize::ResponseCodeError Message: 404 => Net::HTTPNotFound for https://www.swissmedic.ch/marktueberwachung/00135/00166/index.html?lang=de -- unhandled response Backtrace: /var/www/oddb.org/vendor/ruby/2.4.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:308:in `fetch' /var/www/oddb.org/vendor/ruby/2.4.0/gems/mechanize-2.7.3/lib/mechanize.rb:440:in `get' /var/www/oddb.org/src/plugin/rss.rb:47:in `download' /var/www/oddb.org/src/plugin/rss.rb:119:in `block in swissmedic_entries_of' /var/www/oddb.org/src/plugin/rss.rb:115:in `each_key' /var/www/oddb.org/src/plugin/rss.rb:115:in `swissmedic_entries_of' /var/www/oddb.org/src/plugin/rss.rb:169:in `update_swissmedic_feed' /var/www/oddb.org/src/plugin/rss.rb:192:in `update_recall_feed' /var/www/oddb.org/src/util/updater.rb:566:in `update_immediate_with_error_report' /var/www/oddb.org/src/util/updater.rb:417:in `update_recall_feed' /var/www/oddb.org/src/util/updater.rb:413:in `update_swissmedic_feeds' /var/www/oddb.org/src/util/updater.rb:211:in `run' jobs/import_daily:13:in `block in <module:Util>' /var/www/oddb.org/src/util/job.rb:40:in `run' jobs/import_daily:12:in `<module:Util>' jobs/import_daily:11:in `<module:ODDB>' jobs/import_daily:10:in `<main>'
This looks like a bit more work, we here we really have to parse the pages.
The links to the de/fr/it versions are in cleartext and vary therefore greatly. Putting them into a hash for easier manipulation.
Trying to download the file via open-uir and bypass mechanize. Getting the index and one example for HPC and Recall. Working on the unit tests. Now most unit tests pass, but the generated description is different from the old one. I must dig a little bit deeper. Also the old links for the HPC do not work anymore.
I have problem going to the second page of the HPC. The following code does not work
agent.page.link_with(:text => '2').click # whereas the next works fine agent.page.link_with(:text => /Homepage/).click
The first links is a javascript. How can I handle it?
But it is possible to use the following links
Another alternative would be using watir. I will try it. This snippet works
require 'watir';
require 'pp'
require 'pry'
startTime = Time.now
homeUrl = 'https://www.swissmedic.ch/swissmedic/de/home/humanarzneimittel/marktueberwachung/qualitaetsmaengel-und-chargenrueckrufe/chargenrueckrufe.html'
@browser = Watir::Browser.new(:chrome)
@browser.goto homeUrl
@browser.link(:text=>/2$/).click
endTime = Time.now
puts "Took #{endTime - startTime} seconds"
@browser.close
and outputs Took 5.773831599 seconds
For the recall we have
The full URL, which might even old entries is https://www.swissmedic.ch/swissmedic/de/home/humanarzneimittel/marktueberwachung/health-professional-communication--hpc-/_jcr_content/par/teaserlist.content.paging-2.html?pageIndex=2&_charset_=UTF-8&teaserlistid=teaserList_3db5dc155ec8b8e033bc22fc181ed10c&fulltext=&datefrom=01.01.2017&dateto=&tags= HTTP/1.1
Pushed commit Use new URL for swissmedic RSS feeds and installed the changes on thinpower.
Must add a link to the ODDB in the generated RSS. For this I must
French navigation bar should have non breaking space to prevent new lines. Fixed with commit Non breaking spaces for french navigation