---
Remarked that after calling in bin/admin delete_registration('17233') and reimporting using jobs/update_textinfo_swissmedicinfo --no-download --target=both 17233 --reparse that the fachinfo got not updated correctly. Reworking src/plugin/textinfo.rb to ensure that after each app.update we have also a corresponding odba_store call.
But when trying to see info for 17233 via the web I get in bin/oddbd the following error
NoMethodError undefined method `gsub' for nil:NilClass /var/www/oddb.org/src/view/drugs/package.rb:234:in `name' <...>
To be able to undestand what is really going on, I would like to create a oddb.org database from scratch, which is an idea which Zeno and I had discussed before. Will invest two hours to see whether this is feasible or not.
Steps were:
ODBA.storage.dbi = ODBA::ConnectionPool.new('DBI:Pg:oddb.org.ruby21x', 'postgres', '')
if model.fachinfo_news to line 412 in src/view/drugs/centeredsearchform.rb
sudo -u postgres pg_dump oddb.org.ruby21x > ~/dump.sql
Stared bin/oddbd. Now I get the following error when searching for the
error in SBSM::Session#process: /de/gcc/search/zone/drugs/search_query/Aspirin/search_type/st_oddb KeyError key not found: "atc_index" /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:457:in `fetch' /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:457:in `retrieve_from_index' /var/www/oddb.org/src/util/oddbapp.rb:106:in `retrieve_from_index' /var/www/oddb.org/src/util/oddbapp.rb:1006:in `search_by_atc' /var/www/oddb.org/src/util/oddbapp.rb:944:in `search_oddb'
This error vanishes after rebuilding the indices and restarting bin/oddbd.
Then I have this error in the output of bin/oddbd
FEHLER: Textsuchekonfiguration »default_german« existiert nicht
ODBA::Storage.retrieve_from_fulltext_index rescued a DBI::ProgrammingError(FEHLER: Textsuchekonfiguration »default_german« existiert nicht
). Query:
self.dbi.select_all( SELECT target_id,
max(ts_rank(search_term, to_tsquery(?, ?))) AS relevance
FROM interactions_index_de
WHERE search_term @@ to_tsquery(?, ?)
GROUP BY target_id
ORDER BY relevance DESC
, default_german, merfen, default_german, merfen)
returning empty result
Correct sequences to start with an empty database is therefore (after applying the commit xxx)
sudo -u postgres dropb oddb.org.ruby21x
sudo -u postgres createdb -E UTF8 -T template0 oddb.org.ruby21x
ODBA.storage.dbi = ODBA::ConnectionPool.new('DBI:Pg:oddb.org.ruby21x', 'postgres', '')
sudo -u apache jobs/rebuild_indices
sudo svc -h /service/ch.oddb* (or at least sudo -u apache bin/oddbd)
Ignoring this error for the moment being and trying to import iksnr 17233. This starts be creating all registrations found in AipsDownload_latest.xml. Stopped it manually after a few imports and looked at the dump, where I could see in cleartext the created information like company_name, etc
As I achieved running with an empty database in less than an hour I will invest some more time to create a real integration test.
The integration test should to the following.
The set of used iksnr should be easily changeable and contain
Now I will create a small helper (test/data/integration/gen_aips_xml.rb) to extract all FI/PIs from AipsDownload_latest.xml for a some iksnr and saving it to a new file under test/data/integration.
Create the helper scripts
To fix the dictionary took a look at http://dev.ywesee.com/Yasu/20130314-repair-fulltext-index-dictonary-of-postgresql and added a script to add the dictionaries. Not bin/oddbd does not complain anymore about missing _french|_german. On the other hand I get the error
error in SBSM::Session#process: /de/gcc/home/ NoMethodError undefined method `[]' for #<DRb::DRbUnknown:0x00000002a8dbd8> /var/www/.gem/ruby/1.9.1/gems/sbsm-1.2.5/lib/sbsm/session.rb:208:in `import_cookies' /var/www/.gem/ruby/1.9.1/gems/sbsm-1.2.5/lib/sbsm/session.rb:360:in `process' /var/www/oddb.org/src/util/session.rb:125:in `process' /var/www/.gem/ruby/1.9.1/gems/sbsm-1.2.5/lib/sbsm/session.rb:172:in `block in drb_process' <internal:prelude>:10:in `synchronize'
when accessing http://oddb-ci2.dyndns.org/de/gcc/home/
Trying to run jobs/import_daily --xml_file=test/data/integration/AipsDownload_latest.xml. But we have the problem, that we should be able to skip download and use different filenames, e.g XMLPublications-current.zip, XMLPublications-previous.zip instead of XMLPublications-latest.zip. Therefore I will honor an environment variable ODDB_ORG_LATEST_NAME (if present). Must have a look at these files
Now import_daily starts to complete with an email. But it fails as seen in log/oddb/debug/2014/07.log
import_swissmedicinfo_by_iksnrs iksnr "57435" {}
2014-07-02 16:31:09 +0200: /var/www/oddb.org/src/plugin/text_info.rb:565:in `create_sequence_and_package_if_necessary': Could not find a registration for 57435 nil
Pushed commits to sbsm Fix sbsm when being called from oddbd and Revert "Tried migrating to ruby 2.1.2" to fix problem with DRbUnknown.
When running after removing all files under data/xml and data/xls I get the following errors on bin/oddbd output
Could not create: :!1!galenic_form., reason: :!1!galenic_form. -> OddbPrevalence::galenic_group(1) returned nil
and in log/oddb/debug/2014/07.log
2014-07-02 17:36:49 +0200: /var/www/oddb.org/src/plugin/text_info.rb:1345:in `import_swissmedicinfo_by_index': import_swissmedicinfo_by_index @new_iksnrs {"57435"=>"Baraclude®", "57436"=>"Baraclude®", "17233"=>"Selsun®", "32917"=>"Zyloric®"}
2014-07-02 17:36:49 +0200: /var/www/oddb.org/src/plugin/text_info.rb:1351:in `import_swissmedicinfo_by_iksnrs': import_swissmedicinfo_by_iksnrs ["57435", "57436", "17233", "32917"] target fi
2014-07-02 17:36:49 +0200: /var/www/oddb.org/src/plugin/text_info.rb:1357:in `block in import_swissmedicinfo_by_iksnrs': import_swissmedicinfo_by_iksnrs iksnr "57435" {}
2014-07-02 17:36:49 +0200: /var/www/oddb.org/src/plugin/text_info.rb:565:in `create_sequence_and_package_if_necessary': Could not find a registration for 57435 nil
<..>
2014-07-02 17:37:00 CESTlog notify Fach- und Patienteninfo Updates (swissmedicinfo.ch): sent mail
2014-07-02 17:37:02 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 276 updated download.size is 2672682.
2014-07-02 17:37:02 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 53 update target "/var/www/oddb.org/data/xls/Packungen-2014.07.02.xlsx" 2672682 bytes.
2014-07-02 17:37:03 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 276 updated download.size is 2215823.
2014-07-02 17:37:03 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 288 cp /var/www/oddb.org/data/xls/Präparateliste-2014.07.02.xlsx /var/www/oddb.org/data/xls/Präparateliste-latest.xlsx
2014-07-02 17:37:37 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 67 Found 17360 news, 0 updates, 0 replacements and 0 package_deletions
2014-07-02 17:37:37 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 68 changes:
<..>
2014-07-02 17:37:38 +0200: /var/www/oddb.org/src/plugin/swissmedic.rb: 831: res Cardio-Pulmo-Rénal Sérocytol, suppositoire == ? seqnr nil
2014-07-02 17:37:39 +0200: update_compositions: row[0] 00274 iksnr 274 01 seq Cardio-Pulmo-Rénal Sérocytol, suppositoire opts {:create_only=>true, :date=>#<Date: 2014-07-02 ((2456841j,0s,0n),+0s,2299161j)>} cell_content globulina equina (immunisé avec coeur, tissu pulmonaire, reins de porcins) 8 mg, propylenglycolum, conserv.: E 216, E 218, excipiens pro suppositorio.
<..>
Therefore Packungen-2014.07.02.xls and Präparateliste-2014.07.02.xlsx are not yet correct. Must patch src/plugin/swissmedic.rb, too. But this will be next monday. Not yet pushing patch as it contains a 2.3 MB file. See Attach:Added-support-for-integration_test.txt