view · edit · sidebar · attach · print · history

< Niklaus.20161130-cleanup_active_agents | Index | Niklaus.20161128-fix-oddb2xml-perindoprilum >>

20161129-cleanupActiveAgents

Summary

  • Get rid of inactive_agents which have is_active_agent == nil
  • Keep in Mind

Commits

Index

Get rid of inactive_agents which have is_active_agent == nil

Remarked yesterday that we have about 1500 inactive_agents which have their boolean-field is_active_agent as nil. This leads to wrong number of active agents display in entries like http://i.ch.oddb.org/de/mobile/search/zone/drugs/search_query/Perindopril-Amlodipin-Mepha+5mg%252F5mg/search_type/st_oddb#best_result.

Started yesterday the job jobs/cleanup_active_agents at 21.58. It seems stuck. The last lines displayed are

cleanup_active_agents_in_compositions 2016-11-28 22:02:13 +0100: Skip inactive 41173/03 Array Array active? false
cleanup_active_agents_in_compositions 2016-11-28 22:02:13 +0100: Start 41174/04. Skip? false
cleanup_one_composition 2016-11-28 22:02:13 +0100: 41174/04 1951 nr_updates
cleanup_one_composition 2016-11-28 22:02:13 +0100: 41174/04. Done
cleanup_one_composition 2016-11-28 22:02:13 +0100: 41174/04  before [1, Array] after  1/4
2016-11-28 22:02:13 +0100: 9436853 timeout status  for (41174/04)
cleanup_active_agents_in_compositions 2016-11-28 22:02:13 +0100: Start 41174/01. Skip? false
cleanup_one_composition 2016-11-28 22:02:13 +0100: 41174/01 1952 nr_updates

Killing the job at 08:05.

I will try to find a solution to get rid of the explicit field @is_active_agent. Looking at the code I could see why the field can be created with a nil value (If one uses ActiveAgentCommon.new). Added the following unit test to avoid this situation

  def test_init_is_comon_agent
    name = 'Perindopril'
    active_agent = ODDB::ActiveAgentCommon.new(name)
    assert_equal(false, active_agent.is_active_agent)
  end

ActiveAgentCommon is also used in src/model/user.rb (Which I had not expected).

Added a cleanup method in oddbapp.rb for the bin/admin. It finished now cleanly on oddb-ci2. And running it second time returns an empty array in less than a minute (if the packages are loaded).

Added some debug puts to see, whether some methods are called with is_active_agent when displaying the packages.

But my fix is terrible. It deleted the active_agent, instead of the reference to it. Why did the (unchanged) composition.delete_active_agent not work correctly? Or did the jobs/update_active_agents create havoc? Reimporting the database then calling in bin/admin again $res=cleanup_active_agents_all_nil. The search for metoprolol spirig returned many articles without an active agent (e.g. 52110 04 165 Beloc Zok 25 mg).

I must rework the delete as againg bin/admin tells me

registration('52110').sequence('04').package('165').compositions.first.active_agents.size
ch.oddb> registration('52110').sequence('04').package('165').compositions.first.active_agents.size
-> 0

Writing a unit test in test_util/oddbapp_3.rb for cleanup_active_agents_nil made me discover my flaw in the intialization of the active agent. Changed iterator to delete active_sequence starting with the sequence.compositions and not package.compositions.

As I iterate over all sequences (not only the active_sequences) I delete a lot of unused active_agents, e.g. Deleting wrong_agent odba_isolated_store :!registration,52882!sequence,01!composition,9431651!active_agent,219014. is 219014 Biotinum 2.5 mg On thinpower bin/admin tells me that

ch.oddb> registration('52882').packages.last.active_agents.first.is_active_agent.class
-> NilClass

Should I force all of these to false or true? I think I better leave them and iterate only over the active_sequences. Somehow they get deleted because the ative_agent.chemical_substance has the same oid as the active_agent.oid whose is_active_agent is nil.

What is the desired output for IKSNR 00288 where I have a composition of

globulina equina (immunisé avec tissu intestinal et pulmonaire, foie, reins, pancréas, vésicule biliaire, peau, ganglions lymphatiques de porcin) 8 mg, propylenglycolum, conserv.:
# from thinpower
ch.oddb> registration('00288').sequences.values.first.compositions.first.active_agents.collect{|a| a.to_s}
-> ["Globulina Equina (Immunisé avec Tissu Intestinal et Pulmonaire", "Foie", "Reins", "Pancreas", "Vésicule Biliaire"]
ch.oddb> registration('00288').sequences.values.first.compositions.first.active_agents.collect{|a| a.is_active_agent}
-> [nil, nil, nil, nil, nil]
# from oddb2xml/calc
           <SUBSTANCE_NAME>Globulina Equina (immunisé Avec Tissu Intestinal Et Pulmonaire, Foie, Reins, Pancréas, Vésicule Biliaire, Peau, Ganglions Lymphatiques De Porcin)</SUBSTANCE_NAME>
            <IS_ACTIVE_AGENT>false</IS_ACTIVE_AGENT>
            <QTY>8</QTY>
            <UNIT>mg</UNIT>
          </SUBSTANCE>
          <SUBSTANCE>
            <SUBSTANCE_NAME>Propylenglycolum</SUBSTANCE_NAME>
            <IS_ACTIVE_AGENT>false</IS_ACTIVE_AGENT>
          </SUBSTANCE>
          <SUBSTANCE>
            <MORE_INFO>conserv.</MORE_INFO>
            <SUBSTANCE_NAME>E 216</SUBSTANCE_NAME>
            <IS_ACTIVE_AGENT>false</IS_ACTIVE_AGENT>
          </SUBSTANCE>
          <SUBSTANCE>
            <MORE_INFO>conserv.</MORE_INFO>
            <SUBSTANCE_NAME>E 218</SUBSTANCE_NAME>
            <IS_ACTIVE_AGENT>false</IS_ACTIVE_AGENT>

currently I have 5 active_agents which have is_active_agent == nil.

After a discussion with Zeno we decided that I should just remove all active_agents which have the is_active_agent == nil and use the Swissmedic importer (or a variant) of it to reparse all composition texts. Also I will replace the is_active_agent accessor by the method active_agent? which will respond using the derived class name (eg. self.is_a?(ActiveAgent)). But I think I will later get rid of the method, as I think it is no longer used anywhere (exept in test_model/ative_agent)

Made some fixes in the swissmedic plugin to delete active_agents with is_active_agent == nil before starting the import. Running. Will analyze the output tomorrow.

view · edit · sidebar · attach · print · history
Page last modified on November 29, 2016, at 05:34 PM