view · edit · sidebar · attach · print · history

< Niklaus.20180117-oddb-ruby-250 | Index | Niklaus.20180115-ydim-html-rack >>

20180116-oddb-doctors

Summary

  • Fix problem in import_daily with doctors
  • Fix problem with refData_nat/jur
  • Port oddb.org to Ruby 2.5.0
  • Keep in Mind

Commits

Index

cannot load FI 52904 Cosaar when logged in as admin

I get the following backtrace


NoMethodError at /de/gcc/fachinfo/reg/52904
undefined method `encode' for nil:NilClass
Ruby	/var/www/oddb.org/src/model/address.rb: in plz, line 121
Web	GET ch.oddb.org/de/gcc/fachinfo/reg/52904
Jump to:
GETPOSTCookiesENV
Traceback (innermost first)
/var/www/oddb.org/src/model/address.rb: in plz
            location_utf8 = @location.encode('UTF-8', invalid: :replace, undef: :replace, replace: "" )...
/var/www/oddb.org/src/model/company.rb: in invoiceable?
			![ @name, @contact, addr.address, addr.plz,...
/var/www/oddb.org/src/view/drugs/fachinfo.rb: in init
		unless(@model.company.invoiceable?)...
/var/www/oddb.org/vendor/ruby/2.4.0/gems/htmlgrid-1.1.4/lib/htmlgrid/component.rb: in initialize
			init()...
/var/www/oddb.org/src/view/publictemplate.rb: in new

I cannot reproduce it here on oddb-ci2. But I think it should be easy to create a unit test for it. Problem fixed with Fix city/plz if address.location is nil

Fix problem in import_daily with doctors

Must fix the following error from import_daily

Plugin: ODDB::Doctors::RefdataNatPlugin

Error: Encoding::CompatibilityError
Message: incompatible character encodings: UTF-8 and US-ASCII
Backtrace:
/var/www/oddb.org/src/plugin/refdata_nat.rb:247:in `block in store_doctor'
/var/www/oddb.org/src/plugin/refdata_nat.rb:242:in `each'
/var/www/oddb.org/src/plugin/refdata_nat.rb:242:in `store_doctor'
/var/www/oddb.org/src/plugin/refdata_nat.rb:341:in `block in parse_xml'
/var/www/oddb.org/src/plugin/refdata_nat.rb:295:in `each'
/var/www/oddb.org/src/plugin/refdata_nat.rb:295:in `each_with_index'
/var/www/oddb.org/src/plugin/refdata_nat.rb:295:in `parse_xml'
/var/www/oddb.org/src/plugin/refdata_nat.rb:112:in `update'
/var/www/oddb.org/src/util/updater.rb:271:in `block in update_refdata_nat'
/var/www/oddb.org/src/util/updater.rb:532:in `wrap_update'
/var/www/oddb.org/src/util/updater.rb:269:in `update_refdata_nat'
jobs/import_refdata_nat:14:in `block in <module:Util>'
/var/www/oddb.org/src/util/job.rb:40:in `run'
jobs/import_refdata_nat:12:in `<module:Util>'
jobs/import_refdata_nat:11:in `<module:ODDB>'
jobs/import_refdata_nat:10:in `<main>'

While waiting for the jobs/import_refdata_nat to go into my pry statement I try to add a unit test for it.

Creating a unit test for it.Should be fixed with Fix passing nil to RefdataJurPlugin.new as second parameter

The job comleted without error on oddb-ci2 and the email contains

Update of doctors

Number of doctors: 84400
Number of new doctors: 318
Number of updated doctors: 164
Number of inactive doctors: 2433


Details of new doctors are: 318
7601000455048: Béatrice Gay
7601000534613: Fatmir Demiri

Dropping database and reloading it with content from thinpower.

After installing the fix, I restarted jobs/import_swissmedic_followers in a separate screen on thinpower.

Displaying a newly imported doctor does not work

Visiting https://ch.oddb.org/de/gcc/doctor/ean/7601000536273 throws the following error (when not logged in as admin)

NoMethodError at /de/gcc/doctor/ean/7601000536273
undefined method `+' for nil:NilClass
Ruby	/var/www/oddb.org/src/view/doctors/doctor.rb: in addresses, line 159
Web	GET ch.oddb.org/de/gcc/doctor/ean/7601000536273
Jump to:
GETPOSTCookiesENV
Traceback (innermost first)
/var/www/oddb.org/src/view/doctors/doctor.rb: in addresses
      addr.pointer = model.pointer + [:address, 0]...
/var/www/oddb.org/vendor/ruby/2.4.0/gems/htmlgrid-1.1.4/lib/htmlgrid/composite.rb: in create
					self.send(component, *args)...
/var/www/oddb.org/vendor/ruby/2.4.0/gems/htmlgrid-1.1.4/lib/htmlgrid/composite.rb: in compose_component
				comp = create(component, model)...

This looks like a problem in the importer, as the newly imported doctor does not have a pointer. Verified via bin/admin

ch.oddb> doctor_by_gln(7601000536273).pointer.class
-> NilClass
ch.oddb> 
ch.oddb> doctor_by_gln(7601000536273).name
-> Zur-Pasierbek
ch.oddb> doctor_by_gln(7601000536273).firstname
-> Tamara

And it looks like we have many of them

doctors.values.find_all{|x| x.pointer == nil }.size
-> 34445
ch.oddb> doctors.values.find{|x| x.pointer }.pointer
-> :!doctor,8227.

It looks to me that the method create_doctor in src/util/oddbapp.rb is missing the line doctor.pointer = ODDB::Persistence::Pointer.new(:doctor, doctor.oid). Using git blame, I see that this method was added by xomex (as seen by 39bf451f2 (xomex 2004-09-30 15:34:02 +0200 394) def create_doctor.

As it is the first time I analyse a commit by xomex, I suppose, he was not an experienced contributor to oddb.org. I think we must fix this problem (besides correcting the code in oddb_app) via bin/admin.

doctors.values.find_all{|x| !x.pointer }.each { |doc|  doc.pointer = Persistence::Pointer.new([:doctor, doc.oid]); doc.odba_store }

But this takes a long time. I am trying it with an exampe

ch.oddb> $example = doctors.values.find{|x| !x.pointer }
-> #<ODDB::Doctor:0x007ff4a9211268>
ch.oddb> $example = doctors.values.find{|x| !x.pointer }
-> #<ODDB::Doctor:0x007ff4b6082b60>
ch.oddb> $example.pointer
-> :!doctor!37524548.
ch.oddb> $example.pointer = Persistence::Pointer.new(:doctor, $example.oid);
-> :!doctor!37524548.
ch.oddb> $example.odba_store
-> #<ODDB::Doctor:0x007ff4b6082b60>
ch.oddb> $example.pointer
-> :!doctor!37524548.

But this does not seem to be a good idea as I get now

ODDB::Persistence::InvalidPathError at /de/gcc/doctor/ean/7601000654489
:!. -> undefined Method ODDB::App::doctor()
Ruby	/var/www/oddb.org/src/util/persistence.rb: in block in resolve, line 314
Web	GET oddb-ci2.dyndns.org/de/gcc/doctor/ean/7601000654489
Jump to:
GETPOSTCookiesENV
Traceback (innermost first)
/var/www/oddb.org/src/util/persistence.rb: in block in resolve
              raise(InvalidPathError.new(msg, self))...
/var/www/oddb.org/src/util/persistence.rb: in each
          @directions.each { |step|...
/var/www/oddb.org/src/util/persistence.rb: in resolve
          @directions.each { |step|...
/var/www/oddb.org/src/util/persistence.rb: in block in structural_ancestors
				@pointer.ancestors.collect { |pointer| pointer.resolve(app) }...
/var/www/oddb.org/src/util/persistence.rb: in collect
				@pointer.ancestors.collect { |pointer| pointer.resolve(app) }...
/var/www/oddb.org/src/util/persistence.rb: in structural_ancestors
				@pointer.ancestors.collect { |pointer| pointer.resolve(app) }...
/var/www/oddb.org/vendor/ruby/2.4.0/gems/odba-1.1.2/lib/odba/stub.rb: in method_missing

After changing the code to use an array (e.g Persistence::Pointer.new([:doctor, $example.oid]); it worked.

Tested the code on thinpower, too. Starting updating all doctors.pointer now. It will probably take a few hours.

Pushed the commits

Pushed some commits to fix unit tests

Error running refData_nat

This error stems from the fact that we have illegal lines in the address lines. Using pry I got

[13] pry(#<ODDB::Address2>)> element.lines.last
=> "1941 Cries (Voll\xC3\xA8ges)"

I must escape this using

          items = (element.lines  +
                    [:fon] + (element.fon.is_a?(Array) ? element.fon : [element.fon] ) +
                    [:fax] +  (element.fax.is_a?(Array) ? element.fax : [element.fax] ))
          items.collect { |x| x.to_s.encode('utf-8', :invalid => :replace, :undef => :replace, :replace => '') }.join(",")+"\n"

Pushed the commit Fix addresses with incorrect charset

Port oddb.org to Ruby 2.5.0

Run gen_ruby_250 script. See Attach:gen_ruby_250.txt.

Must call bundle-250 install for each Gemfile I want to use. Installing some new gems

sudo /usr/local/bin/gem-250 install yus
sudo /usr/local/bin/gem-250 install ycurrency

Changing /service/yus/run to sudo -u apache /usr/local/bin/ruby-250 /usr/local/ruby-2.5.0/bin/yusd. Same for currency sudo -u apache /usr/local/bin/ruby-250 /usr/local/ruby-2.5.0/bin/yusd Same for migeld sudo -u apache bundle-250 exec ruby-250 bin/migel Restarting yus, migeld, currency without a problem.

Running the oddb.org test-suite. Adding ruby 2.5.0 to rvms to be tested on travis-ci for yus, migel, currency.

We need in oddb.org to add mathn to the required gems. Also had to add a require 'date' in spec/spec_helper.rb. Running watir spec tests now. We have at least 4 errors in the smoketest. Waiting for it to finish.

Examining the following errors

rspec ./spec/smoketest_spec.rb:166 # ch.oddb.org desitin should display Für den Preisvergleich in the price comparision
rspec ./spec/smoketest_spec.rb:375 # ch.oddb.org desitin should be possible to request a new password
rspec ./spec/smoketest_spec.rb[1:2:31] # ch.oddb.org desitin should have a working instant search for Inderal and going back
rspec ./spec/smoketest_spec.rb[1:2:33] # ch.oddb.org desitin should have a working status page status_oddb

We must fix the following problem

ArgumentError at /de/gcc/compare/ean13/7680605160014
incomplete format specifier; use %% (double %) instead
Ruby	/home/niklaus/git/oddb.org/src/view/drugs/compare.rb: in sprintf, line 101
Web	GET oddb-ci2.dyndns.org/de/gcc/compare/ean13/7680605160014
Jump to:
GETPOSTCookiesENV
Traceback (innermost first)
/home/niklaus/git/oddb.org/src/view/drugs/compare.rb: in sprintf
			sprintf('%+d%', diff*100.0)...
/home/niklaus/git/oddb.org/src/view/drugs/compare.rb: in price_difference
			sprintf('%+d%', diff*100.0)...
/home/niklaus/git/oddb.org/vendor/ruby/2.5.0/gems/htmlgrid-1.1.4/lib/htmlgrid/composite.rb: in create

This problem was easily fixed by changing sprintf('%+d%', diff*100.0) to sprintf('%+d', diff*100.0)

The problem with login was rooted in starting oddb.org in my git checkout not under /var/www/oddb.org.

Running now the full spec test suite (which will take over an hour). Also running unit test suite.

view · edit · sidebar · attach · print · history
Page last modified on January 16, 2018, at 05:04 PM