<< Masa.20110511-swissindex_plugin-ch_oddb-debug-atc-zuweisen-de_oddb | 2011 | Masa.20110509-swissindex_plugin >>
Plugin: ODDB::SwissindexPharmaPlugin Error: NameError Message: uninitialized class variable @@start_time in ODDB::SwissindexPlugin::Logging Backtrace: /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:39:in `append_estimate_time' /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:122:in `update_package_trade_status' /home/masa/ywesee/oddb.org/src/model/sequence.rb:139:in `each' /home/masa/ywesee/oddb.org/src/model/sequence.rb:139:in `each_package' ....
Fixed (src/plugin/swissindex.rb)
def Logging.start(file) if @@flag @@start_time = Time.now FileUtils.mkdir_p(File.dirname(file)) log_file = File.open(file, 'w') log_file.print "# ", Time.now, "\n" yield(log_file) log_file.close end end
Run again
ch.oddb> Updater.new(self).update_package_trade_status_by_swissindex true
Run
ch.oddb> Updater.new(self).migel_nonpharma "/home/masa/work/pharmacode_list.dat"
Error
Plugin: ODDB::SwissindexNonpharmaPlugin Error: NoMethodError Message: undefined method `path' for "/home/masa/ywesee/oddb.org/data/csv/swissindex_migel.csv":String Backtrace: /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:326:in `report' /home/masa/ywesee/oddb.org/src/plugin/plugin.rb:51:in `send' /home/masa/ywesee/oddb.org/src/plugin/plugin.rb:51:in `log_info' ...
Fixed (src/plugin/swissindex.rb)
def log_info hash = super if @output_file type = "text/csv" hash.store(:files, { @output_file => type }) end hash end def report File.expand_path(@output_file) end
Note
masa@masa ~/ywesee/oddb.org/data/csv $ cat swissindex_migel.csv position number, pharmacode, GTIN, datetime, status, stdate, lang, description, additional description, company name, company GLN, pharmpreis, ppub, faktor, pzr 341501011,1008386,8711428065510,2010-11-27T00:00:00+00:00,A,1999-12-28T00:00:00+00:00,DE,3M TEGADERM Transparentverband 6x7cm,100 Stk,3M Professional Health Care,7610182007044,,,100, 30405001,1009144,7610472121078,2010-08-16T00:00:00+00:00,A,2002-03-20T00:00:00+00:00,DE,SANOR Nadeln Luer 0.80x40mm grün,25 Stk,Lamprecht AG,7601001003859,3.50,6.25,25,0.25 ...
Run again
ch.oddb> Updater.new(self).migel_nonpharma "/home/masa/work/pharmacode_list.dat"
Result
/home/masa/ywesee/oddb.org/data/csv/swissindex_migel.csv
Note
Next
Error
Plugin: ODDB::SwissindexNonpharmaPlugin Error: Timeout::Error Message: execution expired Backtrace: (druby://localhost:50002) /usr/lib64/ruby/1.8/timeout.rb:60:in `rbuf_fill' (druby://localhost:50002) /usr/lib64/ruby/1.8/net/protocol.rb:132:in `rbuf_fill' (druby://localhost:50002) /usr/lib64/ruby/1.8/net/protocol.rb:86:in `read' (druby://localhost:50002) /usr/lib64/ruby/1.8/net/http.rb:2212:in `read_body_0' (druby://localhost:50002) /usr/lib64/ruby/1.8/net/http.rb:2173:in `read_body' (druby://localhost:50002) /usr/lib64/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize/chain/response_reader.rb:14:in `handle' (druby://localhost:50002) /usr/lib64/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize/chain.rb:24:in `handle' (druby://localhost:50002) /usr/lib64/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize.rb:543:in `fetch_page' (druby://localhost:50002) /usr/lib64/ruby/gems/1.8/gems/ntlm-http-0.1.1/lib/net/ntlm_http.rb:826:in `request' (druby://localhost:50002) /usr/lib64/ruby/1.8/net/http.rb:1053:in `old_request' (druby://localhost:50002) /usr/lib64/ruby/1.8/net/http.rb:2136:in `reading_body' (druby://localhost:50002) /usr/lib64/ruby/1.8/net/http.rb:1052:in `old_request' (druby://localhost:50002) /usr/lib64/ruby/gems/1.8/gems/ntlm-http-0.1.1/lib/net/ntlm_http.rb:821:in `request' (druby://localhost:50002) /usr/lib64/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize.rb:538:in `fetch_page' (druby://localhost:50002) /usr/lib64/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize.rb:259:in `get' (druby://localhost:50002) /home/masa/ywesee/oddb.org/ext/swissindex/src/swissindex.rb:70:in `search_migel' (druby://localhost:10000) /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:266:in `migel_nonpharma' (druby://localhost:10000) /usr/lib64/ruby/1.8/drb/drb.rb:1555:in `call' (druby://localhost:50002) /usr/lib64/ruby/1.8/drb/invokemethod.rb:10:in `block_yield' (druby://localhost:50002) /usr/lib64/ruby/1.8/drb/invokemethod.rb:17:in `perform_with_block' (druby://localhost:50002) /home/masa/ywesee/oddb.org/ext/swissindex/src/swissindex.rb:14:in `session' /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:263:in `migel_nonpharma' /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:260:in `each' /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:260:in `migel_nonpharma' /usr/lib64/ruby/1.8/open-uri.rb:32:in `open_uri_original_open' /usr/lib64/ruby/1.8/open-uri.rb:32:in `open' /home/masa/ywesee/oddb.org/src/plugin/swissindex.rb:257:in `migel_nonpharma' /home/masa/ywesee/oddb.org/src/util/updater.rb:483:in `send' /home/masa/ywesee/oddb.org/src/util/updater.rb:483:in `update_notify_simple' /home/masa/ywesee/oddb.org/src/util/updater.rb:463:in `call' /home/masa/ywesee/oddb.org/src/util/updater.rb:463:in `wrap_update' /home/masa/ywesee/oddb.org/src/util/updater.rb:481:in `update_notify_simple' /home/masa/ywesee/oddb.org/src/util/updater.rb:510:in `migel_nonpharma' (eval):1:in `_admin' /home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `call' /home/masa/ywesee/oddb.org/src/util/failsafe.rb:9:in `failsafe' /home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1417:in `_admin' /home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1415:in `initialize' /home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1415:in `new' /home/masa/ywesee/oddb.org/src/util/oddbapp.rb:1415:in `_admin' /usr/lib64/ruby/1.8/drb/drb.rb:1555:in `__send__' /usr/lib64/ruby/1.8/drb/drb.rb:1555:in `perform_without_block' /usr/lib64/ruby/1.8/drb/drb.rb:1515:in `perform' /usr/lib64/ruby/1.8/drb/drb.rb:1589:in `main_loop' /usr/lib64/ruby/1.8/drb/drb.rb:1585:in `loop' /usr/lib64/ruby/1.8/drb/drb.rb:1585:in `main_loop' /usr/lib64/ruby/1.8/drb/drb.rb:1581:in `start' /usr/lib64/ruby/1.8/drb/drb.rb:1581:in `main_loop' /usr/lib64/ruby/1.8/drb/drb.rb:1430:in `run' /usr/lib64/ruby/1.8/drb/drb.rb:1427:in `start' /usr/lib64/ruby/1.8/drb/drb.rb:1427:in `run' /usr/lib64/ruby/1.8/drb/drb.rb:1347:in `initialize' /usr/lib64/ruby/1.8/drb/drb.rb:1627:in `new' /usr/lib64/ruby/1.8/drb/drb.rb:1627:in `start_service' bin/oddbd:38
Check output data
masa@masa ~/ywesee/oddb.org/data/csv $ wc swissindex_migel.csv 1479 11669 229259 swissindex_migel.csv
Note
Timeout::Error
is not a subclass of StandardError
Updated (ext/swissindex/src/swissindex.rb)
def search_migel(pharmacode) agent = Mechanize.new try_time = 3 begin agent.get(@base_url + pharmacode) count = 100 line = [] agent.page.search('td').each_with_index do |td, i| text = td.inner_text.chomp.strip if text.is_a?(String) && text.length == 7 && text == pharmacode count = 0 end if count < 7 text = text.split(/\n/)[1] || text.split(/\n/)[0] text = text.gsub(/\302\240/, '').strip if text line << text count += 1 end end line rescue StandardError, Timeout::Error => err if try_time > 0 puts err puts err.backtrace puts puts "retry" sleep 10 agent = Mechanize.new try_time -= 1 retry else return [] end end end
Run again
ch.oddb> Updater.new(self).migel_nonpharma "/home/masa/work/pharmacode_list.dat"
Problem
Error
masa@masa ~/ywesee/de.oddb.org $ bin/oddbd ERROR: relation "object" already exists ERROR: relation "prefetchable_index" already exists ERROR: relation "extent_index" already exists ERROR: relation "object_connection" already exists ERROR: relation "target_id_index" already exists ERROR: relation "collection" already exists /usr/lib64/ruby/site_ruby/1.8/odba/marshal.rb:15:in `load': undefined class/module ODDB::Business::GrantDownload (ArgumentError) from /usr/lib64/ruby/site_ruby/1.8/odba/marshal.rb:15:in `load' from /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:616:in `restore' from /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:336:in `fetch_or_restore' from /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:330:in `call' from /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:330:in `fetch_or_do' from /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:335:in `fetch_or_restore' from /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:64:in `bulk_restore' from /usr/lib64/ruby/site_ruby/1.8/odba/cache.rb:61:in `each' ... 17 levels... from /usr/lib64/ruby/site_ruby/1.8/oddb/persistence.rb:4 from /usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from /usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from bin/oddbd:10
Run
masa@masa ~/ywesee/de.oddb.org $ ruby -I lib bin/oddbd
Result
./lib/oddb.rb:4: warning: already initialized constant VERSION ERROR: relation "object" already exists ERROR: relation "prefetchable_index" already exists ERROR: relation "extent_index" already exists ERROR: relation "object_connection" already exists ERROR: relation "target_id_index" already exists ERROR: relation "collection" already exists ERROR: relation "target_id_oddb_drugs_galenicform_code" already exists ERROR: relation "target_id_oddb_business_company_name" already exists ERROR: relation "target_id_oddb_drugs_substance_name" already exists ERROR: relation "target_id_oddb_drugs_galenicgroup_name" already exists ERROR: relation "target_id_oddb_drugs_package_product" already exists ERROR: relation "target_id_oddb_drugs_package_company" already exists ERROR: relation "target_id_oddb_drugs_sequence_fachinfo_indications_de" already exists ERROR: relation "target_id_oddb_drugs_sequence_product" already exists ERROR: relation "target_id_oddb_drugs_substance_code" already exists ERROR: relation "target_id_oddb_drugs_package_atc" already exists ERROR: relation "target_id_oddb_business_grantdownload_email" already exists ERROR: relation "target_id_oddb_drugs_substancegroup_name" already exists ERROR: relation "target_id_oddb_drugs_unit_name" already exists ERROR: relation "target_id_oddb_business_invoice_id" already exists ERROR: relation "target_id_oddb_drugs_atc_level_and_code" already exists ERROR: relation "target_id_oddb_business_invoice_yus_name" already exists ERROR: relation "target_id_oddb_drugs_package_name" already exists ERROR: relation "target_id_oddb_text_document_source" already exists ERROR: relation "target_id_oddb_drugs_atc_name" already exists ERROR: relation "target_id_oddb_drugs_galenicform_description" already exists ERROR: relation "target_id_oddb_drugs_sequence_code" already exists ERROR: relation "target_id_oddb_drugs_product_name" already exists ERROR: relation "target_id_oddb_drugs_package_code" already exists ERROR: relation "target_id_oddb_drugs_package_substance" already exists ERROR: relation "target_id_oddb_drugs_atc_code" already exists ERROR: relation "target_id_oddb_drugs_sequence_atc" already exists ./lib/oddb/html/view/drugs/package.rb:373: warning: parenthesize argument(s) for future version I, [2011-05-10T08:54:41.236173 #20038] INFO -- start: starting oddb-server on druby://localhost:11000
Note
Point
ruby -I lib bin/oddbd
Problem
Resolved
Then it works
Note
Log (on the console of de.oddb.org/bin/oddbd)
ODBA::Stub was unable to replace Hash#4257981 from ODDB::Drugs::Package:#2348571 ODBA::Stub was unable to replace Array#4257982 from ODDB::Drugs::Package:#2348571 ODBA::Stub was unable to replace Array#4257980 from ODDB::Drugs::Package:#2348571 ODBA::Stub was unable to replace Hash#4257983 from ODDB::Drugs::Sequence:#2348564 ODBA::Stub was unable to replace Array#4257984 from ODDB::Drugs::Sequence:#2348564 ODBA::Stub was unable to replace Hash#4257985 from ODDB::Util::M10lDocument:#2348567 ODBA::Stub was unable to replace Hash#4257987 from ODDB::Util::M10lDocument:#3525037
grep
masa@masa ~/ywesee/de.oddb.org $ grep -r atc_assign lib/ lib/oddb/html/state/viral/admin.rb: def atc_assign lib/oddb/html/state/viral/admin.rb: @errors.store :atc_assign, create_error(:e_unknown_atc, :atc_assign, code) lib/oddb/html/util/lookandfeel.rb: :atc_assign => 'ATC zuweisen', lib/oddb/html/util/validator.rb: :ajax_delete_part, :atc_assign, :atc_browser, :checkout, :collect, lib/oddb/html/view/drugs/result.rb: COMPONENTS = { [0,0,0] => :toggle, [0,0,1] => :atc_assign } lib/oddb/html/view/drugs/result.rb: EVENT = :atc_assign lib/oddb/html/view/drugs/result.rb: def atc_assign(model) lib/oddb/html/view/drugs/result.rb: style << ' display: none;' unless @session.error(:atc_assign) lib/oddb/html/view/drugs/result.rb: link = HtmlGrid::Link.new(:atc_assign, model, @session, self) lib/oddb/html/view/drugs/result.rb: unless @session.error(:atc_assign) lib/oddb/html/view/drugs/result.rb: def atc_assign lib/oddb/html/view/drugs/result.rb: @lookandfeel.lookup(:atc_assign), lib/oddb/html/view/drugs/result.rb: HtmlGrid::Input.new(:atc_assign, nil, @session, self),
Question
Html source code
... <FORM ACCEPT-CHARSET="UTF-8" NAME="stdform" METHOD="POST" ENCTYPE="application/x-www-form-urlencoded" ACTION="http://de.oddb.masa.org/de/drugs" ID="atc-assign-form"> <DIV> <A name="atc_assign" id="atc-assign-toggle" onclick="dojo.lfx.toggle.fade.show('atc-assign', 500); this.onclick=null;">ATC zuweisen</A> <INPUT name="code" type="text" id="atc-assign" value="" style="margin-left: 4px; display: none;"> </DIV> <DIV style="display:none"> <INPUT NAME="flavor" TYPE="hidden" VALUE="oddb"> <INPUT NAME="language" TYPE="hidden" VALUE="de"> <INPUT NAME="event" TYPE="hidden" ID="event" VALUE="atc_assign"> <INPUT NAME="state_id" TYPE="hidden" VALUE="69977664596440"> <INPUT NAME="query" TYPE="hidden" VALUE="Herz Ass A Pharma"> <INPUT NAME="dstype" TYPE="hidden" VALUE="tradename"> </DIV> </FORM> ...
Note
Hint (lib/oddb/html/view/drugs/result.rb#AtcAssin)
class AtcAssign < HtmlGrid::DivForm COMPONENTS = { [0,0,0] => :toggle, [0,0,1] => :atc_assign } EVENT = :atc_assign FORM_ID = 'atc-assign-form' def atc_assign(model) input = HtmlGrid::InputText.new(:code, model, @session, self) style = 'margin-left: 4px;' style << ' display: none;' unless @session.error(:atc_assign) input.set_attribute('style', style) input.css_id = 'atc-assign' input end def hidden_fields(context) %w{query dstype}.inject(super) { |memo, key| memo << context.hidden(key, @session.persistent_user_input(key)) } end def toggle(model) link = HtmlGrid::Link.new(:atc_assign, model, @session, self) unless @session.error(:atc_assign) link.onclick = "dojo.lfx.toggle.fade.show('atc-assign', 500); this.onclick=null;" end link.css_id = 'atc-assign-toggle' link end end
lib/oddb/html/state/viral/admin.rb#atc_assign
module ODDB module Html module State module Viral module Admin include SBSM::ViralState def atc_assign email = @session.user.email newstate = search code = user_input(:code) model = newstate.model if(atc = ODDB::Drugs::Atc.find_by_code(code)) packages = model.find { |list| list.atc.nil? } packages.each { |package| if((seq = package.sequence) && seq.atc.nil?) seq.atc = atc seq.data_origins[:atc] = email seq.save end } ## do a completely new search to get everything sorted properly model.query = nil newstate = newstate.search else @errors.store :atc_assign, create_error(:e_unknown_atc, :atc_assign, code) end newstate end
Note
Experiment (lib/oddb/html/state/viral/admin.rb#atc_assign)
def atc_assignp "getin_atc_assign"
email = @session.user.email newstate = search code = user_input(:code)print "code="
p code
model = newstate.model if(atc = ODDB::Drugs::Atc.find_by_code(code)) packages = model.find { |list| list.atc.nil? } packages.each { |package| if((seq = package.sequence) && seq.atc.nil?)print "atc="
p atc
seq.atc = atc seq.data_origins[:atc] = email seq.save end } ## do a completely new search to get everything sorted properly model.query = nil newstate = newstate.search else @errors.store :atc_assign, create_error(:e_unknown_atc, :atc_assign, code) end newstate end
Execute 'ATC zuweisen'
Result (on the console of de.oddb/bin/oddbd)
"getin_atc_assign" code="C09BA13" atc=#<ODDB::Drugs::Atc:0x7f9cfe9cb9e8 @odba_id=4662, @odba_persistent=true, @name=#<ODDB::Util::Multilingual:0x7f9cfe9cb920 @synonyms=[], @canonical={:de=>"Moexipril und Diuretika", :en=>"Moexipril and Diuretics"}>, @odba_observers=[], @code="C09BA13">
Note
Experiment (lib/oddb/html/view/drugs/result.rb#AtcAssign)
class AtcAssign < HtmlGrid::DivForm COMPONENTS = { [0,0,0] => :toggle, [0,0,1] => :atc_assign } EVENT = :atc_assign FORM_ID = 'atc-assign-form' def atc_assign(model)p "getin_atc_assing (view)"
input = HtmlGrid::InputText.new(:code, model, @session, self) style = 'margin-left: 4px;' style << ' display: none;' unless @session.error(:atc_assign) input.set_attribute('style', style) input.css_id = 'atc-assign' input end def hidden_fields(context)p "getin_hidden_fields (view)"
%w{query dstype}.inject(super) { |memo, key| memo << context.hidden(key, @session.persistent_user_input(key)) } end def toggle(model)p "getin_toggle (veiw)"
link = HtmlGrid::Link.new(:atc_assign, model, @session, self) unless @session.error(:atc_assign) link.onclick = "dojo.lfx.toggle.fade.show('atc-assign', 500); this.onclick=null;" end link.css_id = 'atc-assign-toggle' link end end
Execute 'ATC zuweisen'
Result
... ./lib/oddb/html/view/drugs/package.rb:373: warning: parenthesize argument(s) for future version I, [2011-05-10T15:27:01.683585 #11817] INFO -- start: starting oddb-server on druby://localhost:11000 D, [2011-05-10T15:27:15.440624 #11817] DEBUG -- User: mhatakeyama@ywesee.com: allowed?(login, org.oddb.de.Admin) -> true D, [2011-05-10T15:27:15.441883 #11817] DEBUG -- User: mhatakeyama@ywesee.com: allowed?(login, org.oddb.de.PowerUser) -> false D, [2011-05-10T15:27:29.998117 #11817] DEBUG -- User: mhatakeyama@ywesee.com: allowed?(login, org.oddb.de.Admin) -> true "getin_toggle (veiw)" "getin_atc_assing (view)" "getin_hidden_fields (view)" "getin_atc_assign" code="B01AC" atc=#<ODDB::Drugs::Atc:0x7f313e246438 @odba_id=2142, @sequences=#<ODBA::Stub:69924736347920#218448 @odba_class= @odba_container=69924736348700#2142>, @guidelines=#<ODDB::Util::Multilingual:0x7f313e246370 @synonyms=[], @canonical={:en=>"Acetylsalicylic acid preparations specifically intended for use as antithrombotic agents are classified in this group. This exception from the basic principle of only one code for each route of administration is made because of the extensive use of acetylsalicylic acid both as an antithrombotic agent and as an analgesic. Whether an acetylsalicylic acid product should be classified in this group or in N02BA, should be decided at the national level based on the main indication of the product.\n\n\nLysine acetylsalicylate is classified at the same 5th level as acetylsalicylic acid.\n\nSulfinpyrazone is classified in M04AB. Alprostadil is classified in C01EA and G04BE.\n\nCombinations of acetylsalicylic acid and statins are classified in C10B.\n\nCombinations of ramipril, simvastatin, and acetylsalicylic acid are classified in C10BX."}>, @odba_persistent=true, @odba_observers=[], @ddds=#<ODBA::Stub:69924736348180#218447 @odba_class= @odba_container=69924736348700#2142>, @name=#<ODDB::Util::Multilingual:0x7f313e246208 @synonyms=[], @canonical={:en=>"Platelet Aggregation Inhibitors Excl. Heparin", :de=>"Thrombozytenaggregationshemmer, exkl. Heparin"}>, @ddd_guidelines=#<ODDB::Util::Multilingual:0x7f313e245f38 @synonyms=[], @canonical={:en=>"The DDDs are based on prophylaxis of thrombosis. The DDDs of acetylsalicylic acid and carbasalate calcium are given as 1 tablet independent of tablet strength. This is due to the great variations between different countries in the dosages/strengths recommended for prophylaxis of thrombosis.\nThe DDD of iloprost is based on treatment of peripheral vascular disease.\n\nFor combinations products, see list of DDDs for combinations, www.whocc.no."}>, @code="B01AC"> ODBA::Stub was unable to replace Hash#4257061 from ODDB::Drugs::Package:#3479022 ODBA::Stub was unable to replace Array#4257062 from ODDB::Drugs::Package:#3479022 ODBA::Stub was unable to replace Array#4257060 from ODDB::Drugs::Package:#3479022 ODBA::Stub was unable to replace Hash#4257064 from ODDB::Drugs::Package:#3479025 ODBA::Stub was unable to replace Array#4257065 from ODDB::Drugs::Package:#3479025 ODBA::Stub was unable to replace Array#4257063 from ODDB::Drugs::Package:#3479025 ODBA::Stub was unable to replace Hash#4257066 from ODDB::Drugs::Sequence:#3479015 ODBA::Stub was unable to replace Array#4257067 from ODDB::Drugs::Sequence:#3479015 ODBA::Stub was unable to replace Hash#4257068 from ODDB::Util::M10lDocument:#3479018 ODBA::Stub was unable to replace Hash#4257069 from ODDB::Util::M10lDocument:#3716504
Note
Consideration