view · edit · sidebar · attach · print · history

Index>

20171213-postgres10-search

Summary

  • Update to postgres 10. Search for hospital/doctors do not work anymore
  • Fix evidentia search
  • Fix jobs/import_refdata_nat
  • Keep in Mind

Commits

Index

Update to postgres 10. Search for hospital/doctors do not work anymore

Ran update_all_indexs overnight. Search for drugs, interaction, Zulassungsinhaber work, but not for hospitals and doctors. Why?

Removed via /usr/local/bin/gem-240 all old installations of yus, ydbi, ydbd-pg and reinstalled yus. Now the yus daemon starts up without problem again. But the search problem for doctors/hospital persists.

Trying to pin down the problem with pry inside search_doctor of src/util/oddbapp.rb.

Looking via psql I find

oddb.org.ruby21x=# select count(*) from doctor_index;
 count 
-------
     0
(1 row)
oddb.org.ruby21x=# select count(*) from hospital_index;
 count 
-------
     0
(1 row)

Why did update_indices doctor_index not insert any values? Okay: PLSQL used the wrong database, when using ruby193 it is okay. and show (after running rebuild_hospitals

oddb.org.ruby193=# select count(*) from doctor_index;
 count 
-------
 84082
(1 row)

oddb.org.ruby193=# select count(*) from hospital_index;
 count 
-------
  1112
(1 row)

oddb.org.ruby193=# select * from hospital_index limit 1;
 origin_id |                                                                                                                                    search_term                                                                    
                                                                 | target_id 
-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------+-----------
    689846 | '022':25,27 '1211':21,31 '14':23,30 '24':15,20 '3829932':26 '3829940':28 '7601001049048':6 'cantonal':2 'cliniqu':10 'crest':19 'de':5 'et':8 'ge':24 'genev':22,29 'hopital':1 'hug':3 'micheli':12,17 'michelidu
cr':14 'pharmacologi':7 'rue':11,13,16 'servic':4 'toxicologi':9 |    689846
(1 row)

The hospital_index is a full text search index, but as we have 3 different Index classes in odba/lib/index.rb I suspect, that we do not use the correct class. I am getting closer looks lik select_all in /lib/odba/storage.rb:495 returns an error. bin/admin reports now

ch.oddb> search_hospitals('Luzern')
-> 2017-12-13 09:57:02.050 CET [20876] ERROR:  invalid input syntax for type oid: "german"
2017-12-13 09:57:02.050 CET [20876] STATEMENT:                                  SELECT target_id, 
                                                max(ts_rank(search_term, to_tsquery($1, $2))) AS relevance
                                        FROM hospital_index 
                                        WHERE search_term @@ to_tsquery($3, $4) 
                                        GROUP BY target_id
                                        ORDER BY relevance DESC

[]
ch.oddb> search_doctors('Luzern')
-> []
ch.oddb> 

In my case hospital_index used dictionary: 'german' and doctor_index had not dictionary element in the etc/index_definitions.yml.

The fulltext search in odba/storage.rb generates a statement like

oddb.org.ruby193=# select * from hospital_index where search_term @@ 'Luzern';
 origin_id | search_term | target_id 
-----------+-------------+-----------
(0 rows)

Not using the dict parameter in odba/db/storage.rb retrieve_from_fulltext_index solved the problem!

Pushed commits

Did not yet push the commit for Postgresql. Patch is Attach:oddb_postgresql_10.txt

Fix evidentia search

As evidentia does not want to adapt its CSS, we must use the default oddb.css. Done with commits

Update migel to use odba 1.1.4 for postgresql 10.

Fix jobs/import_refdata_nat

The job failed with

Plugin: ODDB::Doctors::RefdataNatPlugin

Error: ArgumentError
Message: invalid byte sequence in US-ASCII
Backtrace:
/var/www/oddb.org/src/model/address.rb:121:in `match'
/var/www/oddb.org/src/model/address.rb:121:in `plz'
/var/www/oddb.org/src/plugin/refdata_nat.rb:260:in `block in store_doctor'
/var/www/oddb.org/src/plugin/refdata_nat.rb:258:in `each'
/var/www/oddb.org/src/plugin/refdata_nat.rb:258:in `store_doctor'
/var/www/oddb.org/src/plugin/refdata_nat.rb:341:in `block in parse_xml'
/var/www/oddb.org/src/plugin/refdata_nat.rb:295:in `each'
/var/www/oddb.org/src/plugin/refdata_nat.rb:295:in `each_with_index'
/var/www/oddb.org/src/plugin/refdata_nat.rb:295:in `parse_xml'
/var/www/oddb.org/src/plugin/refdata_nat.rb:112:in `update'
/var/www/oddb.org/src/util/updater.rb:271:in `block in update_refdata_nat'
/var/www/oddb.org/src/util/updater.rb:532:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:269:in `update_refdata_nat'
jobs/import_refdata_nat:14:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_refdata_nat:12:in `<module:Util>'
jobs/import_refdata_nat:11:in `<module:ODDB>'
jobs/import_refdata_nat:10:in `<main>'

This should be fixed with commit Avoid problems with non UTF-8 addresses

view · edit · sidebar · attach · print · history
Page last modified on December 13, 2017, at 12:34 PM