view · edit · sidebar · attach · print · history

Index>

20140305-error-indexTherapeuticus

Summary

  • Fix error in export index_therapeuticus
  • Cleanup medical products

Commits

Index

Keep in Mind
  • Search nach Anwendung (indications), z.B. Konjunktivitis, does not report all occurrences in the section "Anwendung" of Fachinfo. Probably index is corrupted or not set up correctly.
  • Error: Patents with could not connect to www.swissreg.ch: #<Net::HTTPInternalServerError:0x007f8a7d69bb58>
  • vagrant-oddb.org: cleanup installation for yus with ruby 1.8, logrotate.conf and local vhosts for tests
  • Fix index with patch in oddbapp.rb? 
-  def search_by_indication(key)
+  def search_by_indication(key, lang='de')
+    # indication_index_atc_de
+    pattern = key.gsub(/[^A-z0-9]/, '.')
+    return ODBA.cache.retrieve_from_index("indication_index_atc_#{lang}", pattern)
     pattern = key.gsub(/[^A-z0-9]/, '.')
     atcs = []
     indications.map do |indication|

Fixed this problem today with added default parameter

---

Cleanup medical products

We have to cleanup the following errors:

  1. ean13 should be validated before accepting them Check validity of ean13 code before adding pseudo_fachinfo
  2. search by vendor name does not work fixed with
    1. Added package_by_ean13 and debug possiblity for searches to oddbapp
    2. Fix searching via ean13 for 7 digits pseudo-fachinfo
  3. search by comparing prices does not work Fix searching via ean13 for 7 digits pseudo-fachinfo
  4. atc-code should show medical-product and only unavailable Adding default ATC-code to pseudo-fachinfo
  5. cannot follow link to medication Fix searching via ean13 for 7 digits pseudo-fachinfo
  6. And last but not least. Document these decision somewhere under http://dev.ywesee.com/Choddb/Index

Reworked oddbapp.rb to enable an easy debugging, which indexes are used. This shows for the searches by "Preisvergleich" for Xeljanz and Sinovial the following output of bin/oddbd 

/var/www/oddb.org/src/util/oddbapp.rb:1008:in `search_by_atc': atc_index xeljanz
/var/www/oddb.org/src/util/oddbapp.rb:1030:in `search_by_sequence': sequence_index_atc xeljanz

/var/www/oddb.org/src/util/oddbapp.rb:1008:in `search_by_atc': atc_index sinovial
/var/www/oddb.org/src/util/oddbapp.rb:1030:in `search_by_sequence': sequence_index_atc sinovial
/var/www/oddb.org/src/util/oddbapp.rb:1013:in `search_by_company': atc_index_company sinovial
/var/www/oddb.org/src/util/oddbapp.rb:1049:in `search_by_substance': substance_index_atc sinovial
/var/www/oddb.org/src/util/oddbapp.rb:1030:in `search_by_sequence': sequence_index_atc sinovial
/var/www/oddb.org/src/util/oddbapp.rb:1038:in `search_by_interaction': interactions_index_de sinovial
HINWEIS:  Textsucheanfrage enthält nur Stoppwörter oder enthält keine Lexeme, ignoriert
/var/www/oddb.org/src/util/oddbapp.rb:1056:in `search_by_unwanted_effect': unwanted_effects_index_de sinovial
HINWEIS:  Textsucheanfrage enthält nur Stoppwörter oder enthält keine Lexeme, ignoriert

Therefore I must investigate why search_by_sequence does not find sinovial. Using bin/admin 

registration('62630').name_base
-> Xeljanz 5 mg
ch.oddb> registration('62630').sequences.first[1].atc_class
-> Selektive Immunsuppressiva
registration('1229105').name_base
-> Sinovial® 0,8%
ch.oddb> registration('1229105').sequences.first[1].atc_class.class
-> NilClass

Therefore addint a correct atc_class to the sequence should fix this problem, too.

Trying to add atc_codo for Mediziprodukt via bin/admin. Got inspiration from src/plugin/who.rb 

atc_class('Medizinalprodukt');@pointer = if @atc = atc_class('Medizinalprodukt');    @atc.pointer;  else;    Persistence::Pointer.new([:atc_class, 'Medizinalprodukt']).creator;  end;update @pointer.creator, :en => 'Name des Medizinalprodukt';atc_class('Medizinalprodukt')

atc_class('Medizinalprodukt')
-> Name des Medizinalprodukt

atc_class('Medizinalprodukt').code
-> Medizinalprodukt

After adding this dummy ATC to the sequence('00') of a newly imported pseudo-FI I can search via Verkaufspreis

Search via Zulassungsinhaber works too:

Rewording 'Gebraucht, damit Medizinprodukte ohne echte ATC-Klassierung gefunden werden' => 'Medizinprodukte ohne ATC-Klassierung' before pushing commit.

Fix error in export index_therapeuticus

Must fix the error reported as: 

Error: SBSM::InvalidDataError
Message: e_invalid_ean_code
Backtrace:
(druby://localhost:10005) /var/www/oddb.org/src/model/ean13.rb:12:in `initialize'
(druby://localhost:10005) /var/www/oddb.org/src/model/ean13.rb:26:in `new'
(druby://localhost:10005) /var/www/oddb.org/src/model/ean13.rb:26:in `new_unchecked'
(druby://localhost:10005) /var/www/oddb.org/src/model/package.rb:163:in `barcode'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/csv_exporter.rb:71:in `block in collect_data'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/csv_exporter.rb:67:in `collect'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/csv_exporter.rb:67:in `collect_data'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/csv_exporter.rb:99:in `block in dump'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/csv.rb:1367:in `open'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/csv_exporter.rb:98:in `dump'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:137:in `block (2 levels) in export_ean13_idx_th_csv'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:135:in `each'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:135:in `block in export_ean13_idx_th_csv'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:286:in `call'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:286:in `block in safe_export'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/tempfile.rb:316:in `open'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:285:in `safe_export'
(druby://localhost:10005) /var/www/oddb.org/ext/export/src/odba_exporter.rb:131:in `export_ean13_idx_th_csv'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1548:in `perform_without_block'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1508:in `perform'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1586:in `block (2 levels) in main_loop'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `loop'
(druby://localhost:10005) /usr/local/lib/ruby/1.9.1/drb/drb.rb:1582:in `block in main_loop'
/var/www/oddb.org/src/plugin/csv_export.rb:163:in `export_index_therapeuticus'
/var/www/oddb.org/src/util/exporter.rb:169:in `block in export_index_therapeuticus_csv'
/var/www/oddb.org/src/util/exporter.rb:425:in `call'
/var/www/oddb.org/src/util/exporter.rb:425:in `safe_export'
/var/www/oddb.org/src/util/exporter.rb:167:in `export_index_therapeuticus_csv'
/var/www/oddb.org/src/util/exporter.rb:74:in `run'
jobs/export_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/export_daily:12:in `<module:Util>'
jobs/export_daily:11:in `<module:ODDB>'
jobs/export_daily:10:in `<main>'

The export raises an error, because the checksum for the EAN13 is invalid. I will correct the error to show the invalid ean13 and search via bin/admin for invalid ean13.

The cleanest thing would be to create a new version of the sbsm gem with the following modification:

In replace in lib/sbsm/validator.rb in method InvalidDataError.initialize super(msg) by super("#{msg.to_s} #{key} #{value}"). Plus changing the test for it.

Pushed Better error msg for InvalidDataError

Also the import of medical products should validated first, whether the ean13 is valid or not. Therefore I added this check to the import_medical_products.rb. See commit Check validity of ean13 code before adding pseudo_fachinfo

Now I tried to search for invalid eans in the db via bin/admin (on oddb-ci2) 

 @ean=[]; packages.each{|pack| ean = Ean13.new(pack.barcode); @ean << ean if ean.to_s.size != 13 }
-> Array
ch.oddb> @ean
-> []

Running the same command on thinpower. Same result. Now using (as in export_therapeuticus the same method barcode) 

@ean=[]; packages.each{|pack| ean = pack.barcode; @ean << ean if ean.to_s.size != 13 }

Same result. Running therefore jobs/export_csv on oddb-ci2 to see whether I can reproduce the error here. Did not reproduce. Running a full export_daily and working on other issues for the time being.

On oddb-ci2 the export_daily passed now export_index_therapeuticus without problems 

2014-03-05 10:40:44 +0100: safe_export index_therapeuticus starting
2014-03-05 10:44:33 +0100: safe_export index_therapeuticus completed

Therefore I am quite sure, that restart exportd and oddbd on thinpower will fix the problem.

After thinking about my first patch pushed a second one, to avoid overflowing error-logs: Limit msg-siz for InvalidDataError

view · edit · sidebar · attach · print · history
Page last modified on March 05, 2014, at 02:37 PM