view · edit · sidebar · attach · print · history

Index>

20150127-oddb2xml-no-veterinary

Summary

  • oddb2xml should not produce article/products for veterinary use
  • fix registrations without atc_codes
  • allow pharmacode as input for gtin2atc

Commits

Index

Keep in Mind for work to do
  • Fix dojo error http://www.sitepen.com/blog/2012/10/31/debugging-dojo-common-error-messages/#forgot-dom-ready
  • I removed on May-27 tests for ix_registrationss, fix_sequences, fix_compositions, fix_packages from test/test_plugin/swissmedic.rb,as he could not find any references for them in the src code. Did I erroneously remove stuff when cleaning up the swissmedic import earlier?
  • The whole test for older/newer Packages must be adapted to xlsx. One must compare the rows (e.g. by creating csv files) and do the same stuff in xlsx!
  • creat gem: task: input=file with ean-codes, standard output show ean-codes + atc-code. Source is Swissmedic Packungen.xlsx or XML.
  • Display 10 recalls not only those from this month
  • Import via data/medreg_companies.yaml

---

fix registrations without atc_codes

We currently have 107 products without an atc_code when searching via http://ch.oddb.org/de/gcc/search/zone/drugs/search_query/atcless/search_type/st_oddb#best_result.

First product is http://ch.oddb.org/de/gcc/drug/reg/62903, Adempas with

Sequenz : 01
Präparatname : Adempas 0.5 mg, Filmtabletten
Wirkstoffe : riociguatum
Zusammensetzung : riociguatum 0.5 mg, excipiens pro compresso obducto.

http://ch.oddb.org/de/gcc/drug/reg/62903/seq/01 does not have an atc_class.

Found the following log entries when running grep -r atc log/oddb/debug/2015/01.log | grep 62903

2015-01-05 14:25:09 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb: 232: delete object #<ODDB::Package:0x0000002947f198 @swissmedic_source={:import_date=>#<Date: 2014-12-04 ((2456996j,0s,0n),+0s,2299161j)>, :iksnr=>"50084", :seqnr=>"02", :name_base=>"Magnesium-Diasporal 300, Granulat", :company=>"Doetsch Grether AG", :index_therapeuticus=>"07.02.1.", :atc_class=>"A12CC04", :production_science=>"Synthetika human", :registration_date=>#<DateTime: 1989-08-21T00:00:00+00:00 ((2447760j,0s,0n),+0s,2299161j)>, :sequence_date=>#<DateTime: 1998-10-29T00:00:00+00:00 ((2451116j,0s,0n),+0s,2299161j)>, :expiry_date=>#<DateTime: 2018-10-27T00:00:00+00:00 ((2458419j,0s,0n),+0s,2299161j)>, :ikscd=>"054", :size=>"2 x 50", :unit=>"Sachet(s)", :ikscat=>"B", :substances=>"magnesium", :composition=>"magnesium 301 mg ut magnesii citras anhydricus, saccharum, aromatica, antiox.: E 320, excipiens ad granulatum pro charta 5.09 g.", :indication_registration=>"Magnesium-Präparat", :indication_sequence=>nil}, @oid=26019259, @data_origins={"ikscat"=>:swissmedic, "out_of_trade"=>:refdata, "refdata_override"=>:swissmedic, "swissmedic_source"=>:swissmedic}, @odba_persistent=true, @ikscat="B", @revision=2014-12-04 14:35:06 +0100, @sequence=#<ODBA::Stub:346303900#72815 @odba_class= @odba_container=346290380#26019259>, @odba_id=26019259, @parts=#<ODBA::Stub:346303600#26019260 @odba_class= @odba_container=346290380#26019259>, @odba_observers=[], @pointer=:!registration,50084!sequence,02!package,054., @ikscd="054", @feedbacks=#<ODBA::Stub:346302560#26019261 @odba_class= @odba_container=346290380#26019259>, @refdata_override=true, @out_of_trade=true, @narcotics=#<ODBA::Stub:346302300#26019262 @odba_class= @odba_container=346290380#26019259>>
2015-01-05 14:28:40 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb: 87: atc_code update nr 1 for iksnr 62903/1 atc_code_swissmedic C02KX05 atc_code_sequence C01DX odba_isolated_store
2015-01-05 14:28:40 +0100: /var/www/oddb.org/src/plugin/swissmedic.rb: 82: skipping update_atc_codes 62903 atc_code_swissmedic C02KX05

The problem seems to be that, we never set the atc_class if it is nil. Testing my local fix on oddb-ci2 by running jobs/import_swissmedic.

Got a nil pointer when running import, but http://oddb-ci2.dyndns.org/de/gcc/show/reg/62903/seq/01 now shows the desired atc_code C01DX. Trying to fix the NPE and restarting the import afterwards.

Got another NPE when the sequence did not yet exists. Calling update_sequence(registration, row) in this case. Restarted the import again.

Now the import completed, but I still have 45 sequences without an ATC-class. But only 27 of them have a sequence-number differen of '00' as shown by this bin/admin snippet

ch.oddb> atcless_sequences.size
-> 45
ch.oddb> atcless_sequences.first.seqnr
-> 00
ch.oddb> $x=[]; atcless_sequences.each{|x| $x << x if x.seqnr !='00'}; $x.size
-> 27
ch.oddb> $x.first
-> Gentiana/Zingiber comp., Extractum aquosum
ch.oddb> $x.first.iksnr
-> 63056
ch.oddb> $x.first.seqnr
-> 01

allow pharmacode as input for gtin2atc

We must finish the work.

Done with commit Allow pharmacode as input

Zeno wants the following refinement:

  • Lowercase name for BAG, SwissIndex and Swissmedic
  • Add an optional --full to --compare with
    • For each of the lines of the 3 reports a separate file ending with '.txt'
    • Name of the file is full text of the resumen (spaces replaced by '_'), e.g. generated file is called 830_are_the_same_in_SwissMedic_and_BAG.txt
    • Each files contains the detail output of the diff

Pushed commit Lowercase name for BAG, SwissIndex and Swissmedic

Now running bin/gtin2atc --compare --full produces the following output

Result of verifing data from bag (SL):
  bag-data fetched from http://bag.e-mediat.net/SL2007.Web.External/File.axd?file=XMLPublications.zip.
  bag had 9212 entries
  8 entries had no GTIN field
  Not in swissmedic 486
  Not in swissindex 248
Comparing ATC-Codes between bag and swissmedic
   compare_bag_to_swissmedic: 8123 items had the same ATC code in bag, swissindex and swissmedic
   compare_bag_to_swissmedic: 119 are the same in swissmedic and bag
   compare_bag_to_swissmedic: 204 are different in swissmedic and bag
   compare_bag_to_swissmedic: 265 are shorter in swissmedic than in bag
   compare_bag_to_swissmedic: 15 are longer in swissmedic than in bag
Comparing ATC-Codes between bag and swissindex
   compare_bag_to_swissindex: 8123 items had the same ATC code in bag, swissindex and swissmedic
   compare_bag_to_swissindex: 830 are the same in swissindex and bag
   compare_bag_to_swissindex: 0 are different in swissmedic and bag
   compare_bag_to_swissindex: 0 are shorter in swissindex than in bag
   compare_bag_to_swissindex: 11 are longer in swissindex than in bag
Result of verifing data from swissmedic:
  swissmedic had 18870 entries. Fetched from http://www.swissmedic.ch/arzneimittel/00156/00221/00222/00230/index.html?lang=de
  swissindex 15775 entries. Fetched from https://index.ws.e-mediat.net/Swissindex/Pharma/ws_Pharma_V101.asmx?WSDL
  bag 9212 entries. 8 entries had no GTIN field. Fetched from http://bag.e-mediat.net/SL2007.Web.External/File.axd?file=XMLPublications.zip
  Matching 0 items.
  Not in bag 4590
  Not in swissindex 5677
  Comparing ATC-Codes between swissmedic and swissindex
   swissmedic: 11465 match in swissindex and swissmedic
   swissmedic: 361 are different in swissindex and swissmedic
   swissmedic: 11465 are the same in swissindex and swissmedic
   swissmedic: 73 are shorter in swissindex
   swissmedic: 1294 are longer in swissindex
Comparing all GTIN-codes:
  Found infos about 21577 entries
  bag 9212 entries. 8 entries had no GTIN field. Fetched from http://bag.e-mediat.net/SL2007.Web.External/File.axd?file=XMLPublications.zip
  swissindex 15775 entries. Fetched from https://index.ws.e-mediat.net/Swissindex/Pharma/ws_Pharma_V101.asmx?WSDL
  swissmedic 18870 entries. Fetched from http://www.swissmedic.ch/arzneimittel/00156/00221/00222/00230/index.html?lang=de
   compare_all_gtins: 8592 items had the same ATC code in bag, swissindex and swissmedic
   compare_all_gtins: 4590 not in bag
   compare_all_gtins: 5677 not in swissindex
   compare_all_gtins: 2707 not in swissmedic
   compare_all_gtins: 11 ATC-Codes differed

and the following files

compare_all_gtins_11_ATC-Codes_differed.txt                                                      compare_bag_to_swissmedic_119_are_the_same_in_swissmedic_and_bag.txt
compare_all_gtins_2707_not_in_swissmedic.txt                                                     compare_bag_to_swissmedic_15_are_longer_in_swissmedic_than_in_bag.txt
compare_all_gtins_4590_not_in_bag.txt                                                            compare_bag_to_swissmedic_204_are_different_in_swissmedic_and_bag.txt
compare_all_gtins_5677_not_in_swissindex.txt                                                     compare_bag_to_swissmedic_265_are_shorter_in_swissmedic_than_in_bag.txt
compare_all_gtins_8592_items_had_the_same_ATC_code_in_bag_swissindex_and_swissmedic.txt          compare_bag_to_swissmedic_8123_items_had_the_same_ATC_code_in_bag_swissindex_and_swissmedic.txt
compare_bag_to_swissindex_0_are_different_in_swissmedic_and_bag.txt                              swissmedic_11465_are_the_same_in_swissindex_and_swissmedic.txt
compare_bag_to_swissindex_0_are_shorter_in_swissindex_than_in_bag.txt                            swissmedic_11465_match_in_swissindex_and_swissmedic.txt
compare_bag_to_swissindex_11_are_longer_in_swissindex_than_in_bag.txt                            swissmedic_1294_are_longer_in_swissindex.txt
compare_bag_to_swissindex_8123_items_had_the_same_ATC_code_in_bag_swissindex_and_swissmedic.txt  swissmedic_361_are_different_in_swissindex_and_swissmedic.txt
compare_bag_to_swissindex_830_are_the_same_in_swissindex_and_bag.txt                             swissmedic_73_are_shorter_in_swissindex.txt

Pushed commmits:

Discussed with Zeno on how to change the file names to make it clearer. He also wants to group according the atc code to grasp faster where ATC where absent in BAG or swissmedic.

Now the put looks like this

head compare_swissmedic_to_swisssindex_1294_atc_are_longer_in_swissindex.txt
compare swissmedic to swisssindex: 1294 atc are longer in swissindex
ATC code  for 7680092320366 longer in swissindex  R01AX10
ATC code  for 7680109990223 longer in swissindex  A01AZ

Pushed commit Bumped version to 0.1.3. Clearer filenames. Ordered by ATC-Code

oddb2xml should not produce article/products for veterinary use

We should

  • not create entries for oddb_article/products when the ATC-Code begins with 'Q'
  • create correct entries for quantity for expressions like 2 x 6 or 5 x 50 mg

Trying to see where we got the information GTIN 7680632780018 VOLINA 20 Filmtabl in oddb_article.xml

  <PRD DT="">
    <GTIN>7680632780018</GTIN>
    <PRODNO>632781</PRODNO>
    <ATC>G03AA12</ATC>
    <IT>09.02.1.</IT>
    <CPT/>
    <PackGrSwissmedic>1 x 21</PackGrSwissmedic>
    <EinheitSwissmedic>Tablette(n)</EinheitSwissmedic>
    <SubstanceSwissmedic>drospirenonum, ethinylestradiolum</SubstanceSwissmedic>
  </PRD>

and oddb_product.xml

  <ART DT="">
    <REF_DATA>1</REF_DATA>
    <PHAR>6133624</PHAR>
    <SMCAT>B</SMCAT>
    <SMNO>63278001</SMNO>
    <PRODNO>632781</PRODNO>
    <SALECD>I</SALECD>
    <CDBG>N</CDBG>
    <BG>N</BG>
    <QTY>21 Stk</QTY>
    <DSCRD>VOLINA 20 Filmtabl</DSCRD>
    <DSCRF>VOLINA 20 cpr pell</DSCRF>
    <SORTD>VOLINA 20 FILMTABL</SORTD>
    <SORTF>VOLINA 20 CPR PELL</SORTF>
    <ARTCOMP>
      <COMPNO>7601001397385</COMPNO>
    </ARTCOMP>
    <ARTBAR>
      <CDTYP>E13</CDTYP>
      <BC>7680632780018</BC>
      <BCSTAT>A</BCSTAT>
    </ARTBAR>
  </ART>

Exists in Packungen.xls

view · edit · sidebar · attach · print · history
Page last modified on January 27, 2015, at 06:59 PM