view · edit · sidebar · attach · print · history

Index>

20141209-import-atc-code

Summary

  • import atc_code DDD
  • create separate gem for medical persons and companies

Commits

Index

Keep in Mind
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!

---

create separate gem for medical persons and companies

As the import of medreg takes several days and is interfering with our daily import we decided to split this task in two parts. First create yaml/csv files for all doctors/companies, then importing all data, Data comes http://www.medregom.admin.ch/ and https://www.medregbm.admin.ch/Betrieb/Search.

Create initial setup (including rake gem/install, travis-ci,etc) with commit Initial commit

Ported most stuff from oddb to medregom. Now unit tests begins to fail. Tomorrow is time to make them pass.

recalls do not correctly update the rss feed on ch.oddb.org

We find all recalls in the log/debug/2014/12.log when runing import_daily, e.g

2014-12-08 06:01:40 UTC rss adding new entry: 2014-12-03 Chargenrückruf / Meropenem-Teva 500 mg und 1 g, Pulver zur Herstellung einer Injektions- oder Infusionslösung
2014-12-08 06:01:40 UTC rss adding new entry: 2014-11-28 Chargenrückruf / Feniallerg, Injektionslösung
2014-12-08 06:01:41 UTC rss adding new entry: 2014-11-27 Chargenrückruf / Sanalepsi N, Tropfen
2014-12-08 06:01:41 UTC rss adding new entry: 2014-11-21 Chargenrückruf / Prolutex, Injektionslösung
2014-12-08 06:01:41 UTC rss adding new entry: 2014-11-19 Chargenrückruf / Omeprazol Sandoz eco 20mg, Filmtabletten
2014-12-08 06:01:41 UTC rss adding new entry: 2014-10-24 Chargenrückruf / Zovirax Augensalbe
2014-12-08 06:01:42 UTC rss adding new entry: 2014-10-15 Chargenrückruf / Epaxal, Injektionssuspension
2014-12-08 06:01:42 UTC rss adding new entry: 2014-10-13 Chargenrückruf / Torisel, Konzentrat zur Herstellung einer Infusionslösung
2014-12-08 06:01:42 UTC rss adding new entry: 2014-10-09 Chargenrückruf / Donepezil-Mepha 5 mg / 10 mg, Lactab
2014-12-08 06:01:42 UTC rss adding new entry: 2014-10-06 Chargenrückruf / Kiovig, Infusionslösung

But http://ch.oddb.org/de/gcc/rss/channel/recall.rss only shows recalls from 2013 and later. Why did that now work when e.g. http://ch.oddb.org/de/gcc/rss/channel/price_rise.rss is okay?

http://ch.oddb.org/de/gcc/rss/channel/hpc.rss is also okay.

Must fix error in import_daily

Plugin: ODDB::SwissmedicPlugin
Error: TypeError
Message: no _dump_data is defined for class PGresult
Backtrace:
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/marshal.rb:10:in `dump'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/marshal.rb:10:in `dump'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:269:in
`odba_isolated_dump'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:510:in `store'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:274:in
`odba_isolated_store'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:395:in
`block in odba_store_unsaved'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:392:in
`each'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:392:in
`odba_store_unsaved'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/persistable.rb:380:in
`odba_store'
/var/www/oddb.org/src/util/persistence.rb:279:in `issue_update'
/var/www/oddb.org/src/util/oddbapp.rb:132:in `block in update'
/var/www/oddb.org/src/util/failsafe.rb:10:in `call'
/var/www/oddb.org/src/util/failsafe.rb:10:in `failsafe'
/var/www/oddb.org/src/util/oddbapp.rb:130:in `update'
/var/www/oddb.org/src/util/oddbapp.rb:1577:in `update'
/var/www/oddb.org/src/plugin/swissmedic.rb:212:in `block in deactivate'
/var/www/oddb.org/src/plugin/swissmedic.rb:208:in `each'
/var/www/oddb.org/src/plugin/swissmedic.rb:208:in `deactivate'
/var/www/oddb.org/src/plugin/swissmedic.rb:127:in `update'
/var/www/oddb.org/src/util/updater.rb:424:in `block in update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:526:in `call'
/var/www/oddb.org/src/util/updater.rb:526:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:422:in `update_swissmedic'
/var/www/oddb.org/src/util/updater.rb:197:in `run'
jobs/import_daily:13:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_daily:12:in `<module:Util>'
jobs/import_daily:11:in `<module:ODDB>'
jobs/import_daily:10:in `<main>'

Error occured after 2014-12-09 11:00:15 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb: 210: deactivate iksnr '59276' seqnr 00 pack. But I found over 1200 entries deactivate iksnr. 51 of them concerned seqnr 00. But found also curios iksnr in entry likes 2014-12-09 10:59:51 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb: 210: deactivate iksnr '3363895078' seqnr 00 pack or 2014-12-09 10:59:50 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb: 210: deactivate iksnr '1229105863' seqnr 05 pack

Searching for 59276 in ch.oddb.org we find a sequence 01 Fluvimil SR 80 mg Retardtabletten. I cannot find Fluvimil via the text search on http://swissmedicinfo.ch/. It does not show in Packungen.xlsx, but can be found in Präparateliste.xls as 59276 1 Fluvimil SR 80 mg Retardtabletten Actavis Switzerland AG Exportzulassung 07.12.0. C10AA04 Synthetika human B B 22.04.10 22.04.10 21.04.15 fluvastatinum fluvastatinum 80 mg ut fluvastatinum natricum, excipiens pro compresso obducto. Reduktion der Serumcholesterinkonzentration, primäre gemischte Dyslipidämie

Fix name of medreg imports with commit Better names for medreg imports

Running import_daily. Now passed updating packages, but I don't find any entry for 59276 in my log. Bizarr.

Committed https://github.com/ngiger/oddb.org/commit/684a106975fae981330b54deba2b4ab1ae7056a71Bump rubyXL to 3.3.1

Committed Don't odba_store a PGresult. Restarting import_daily on thinpower.

Update ATC-Codes when importing

My code of yesterday produced about 140 entries like this

2014-12-08 18:57:30 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb: des 48734 atc_code_swissmedic G02CX04                                                                                                        

as it did not find an atc_code. Therefore the need to import an actual version of the atc_codes is clearly proven.

Pushed commit Check atc_codes for all sequences if swissmedic-atc_code length okay...

import atc_code and DDD

For the import we have two possibilities. Under src/plugin/who.rb we find an importer which traverses http://www.whocc.no/atc_ddd_index/. Or we can use the XLS file bought by Zeno.

First I try whether the existing plugin. Starting /usr/local/bin/ruby jobs/import_whocc.

After about 30 minutes the report completed with

Imported 1257 ATC-Codes
Updated  121 English descriptions
Updated  232 Guidelines
Updated    2 DDD-Guidelines

This looks okay. Verifiying the missing atc_code G02CX04.

ch.oddb> atc_class('G02CX04')
-> Cimicifugae Rhizoma
ch.oddb> atc_class('G02CX04').code
-> G02CX04

Under http://oddb-ci2.dyndns.org/de/gcc/atc_chooser/code/G02CX it is not yet listed. Restarting oddbd. Still not there. Updating indices via jobs/rebuild_indices indication_index_atc_de indication_index_atc_fr atc_index sequence_index_atc

Rebuilt failed because of the following error

dropping: indication_index_atc_de
creating: indication_index_atc_de
filling: indication_index_atc_de
@atc_classes.values
source.size: 7565
NoMethodError
undefined method `indication' for "RangeError":String
/var/www/oddb.org/src/model/sequence.rb:219:in `indication'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(eval):2:in `block (2 levels) in proc_instance_origin'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `collect'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/stub.rb:112:in `method_missing'
(eval):2:in `block in proc_instance_origin'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:68:in `call'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:68:in `block in fill'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:66:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/index.rb:66:in `fill'
/usr/local/lib/ruby/gems/1.9.1/gems/odba-1.1.0/lib/odba/cache.rb:335:in `fill_index'
/var/www/oddb.org/src/util/oddbapp.rb:1372:in `block in rebuild_indices'
/usr/local/lib/ruby/1.9.1/syck.rb:235:in `call'
/usr/local/lib/ruby/1.9.1/syck.rb:235:in `load_documents'
/usr/local/lib/ruby/1.9.1/syck.rb:235:in `load_documents'
/var/www/oddb.org/src/util/oddbapp.rb:1349:in `rebuild_indices'
/usr/local/lib/ruby/1.9.1/delegate.rb:72:in `method_missing'
jobs/rebuild_indices:15:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `call'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/rebuild_indices:12:in `<module:Util>'
jobs/rebuild_indices:11:in `<module:ODDB>'
jobs/rebuild_indices:10:in `<main>'
finished in 2.2439281841166667 min
all Indices Created in total: 0.0546138244925 h

Modified src/model/sequence.rb to log the error and avoid aborting rebuild. Found 2 errors, which I cannot yet identify. Maybe G02CX does not listen G02CX04 as at the moment no sequence is attached to. Rebuilding all indices which contain atc in their names. (atc_index sequence_index_atc atc_index_company substance_index_atc indication_index_atc_de indication_index_atc_fr atcless)

When logged in as admin use you can see the new atc-code via http://oddb-ci2.dyndns.org/de/gcc/atc_class/atc_code/G02CX04

view · edit · sidebar · attach · print · history
Page last modified on December 09, 2014, at 05:16 PM