20101026-update-import_gkv debug-import_dimdi
<< Masa.20101027-debug-bsv_follower-autorun | 2010 | Masa.20101025-debug-import_gkv-autorun >>
- Goal
- update-import_gkv / 100 %
done - debug import_dimdi / 70 %
- Milestones
Test import_gkv option with ODDB.config9:00Test import_gkv in local both manually and automatically10:50- Debug import_dimdi NoMethodError
Tue Oct 26 09:45:24 2010: de.oddb.org ODDB::Import::Dimdi::GalenicForm (XLS)15:00NoMthodErrorTue Oct 26 10:05:04 2010: de.oddb.org Update Festbeträge (XLS)17:00NoMethodError
- Summary
- Commits
- Updated import_gkv pdf link option (de.oddb.org jobs/import_gkv)
- Debug import_dimdi_galenic_forms nil data error in a xls file (NoMethodError) (de.oddb.org jobs/import_dimdi)
- Debug import_product nil data error in a xls file (NoMethodError) (de.oddb.org jobs/import_dimdi)
- ToDo Tomorrow
- bsv_followers (oddb.org) autorun bug
- Keep in Mind
- emerge --sync
import_gkv (de.oddb.org) autorun bugimport_dimdi (de.odbb.org) NoMethodError- bsv_followers (oddb.org) autorun bug
- RCLConf README and test cases
- xls2odat argument files
- Debug importer de.oddb.org
- Search a fundamental cause of imoprt_chde_xls bug
- Tokyo Tyrant + ODBA
- adding a prefix for the pharmacode field in xls2odat
- look for the way to delete the product de.oddb.org import_gkv
- Organize the information of yus README
- Organize the information of de.oddb.org README
- Delete all SL-Entries before importing the new flags.
- Dimdi-Import-Error.
- 20100922-trace-limitation-data-flow-in-bsvXml?action=edit
- 20100908 rpdf2txt GPLv3
- block sample http://q.hatena.ne.jp/1217573703 (japanese)
debug ydpm (I cannot create and modify a task detail from my local)- check ydpm priorities
- admin tool hack
- Attached Files
Update import_gkv option
Experiment
def Updater.import_gkv(opts = {})
p "getin Updater.import_gkv"
p opts
p ODDB.config
p ODDB.config.test
exit
Run import_gkv
$ jobs/import_gkv test=123
Result
"getin Updater.import_gkv"
{}
#<RCLConf::RCLConf:0x7f178459e000 @defaults={"server_url_export"=>"druby://localhost:11001", "server_name"=>"de.oddb.org", "mail_from"=>"update@oddb.org", "server_url"=>"druby://localhost:11000", "oddb_dir"=>"/home/masa/ywesee/de.oddb.org", "db_backend"=>:psql, "smtp_user"=>"update@oddb.org", "smtp_domain"=>"oddb.org", "session_timeout"=>3600, "export_dir"=>"/home/masa/ywesee/de.oddb.org/var/downloads", "data_dir"=>"/home/masa/ywesee/de.oddb.org/data", "config"=>["/home/masa/ywesee/de.oddb.org/etc/oddb.yml", "/etc/oddb/oddb.yml"], "smtp_port"=>587, "run_updater"=>true, "smtp_authtype"=>:plain, "paypal_server"=>"www.paypal.com", "query_limit_phase"=>86400, "mail_charset"=>"utf8", "http_server"=>"http://localhost", "pharmacy_premium"=>8.1, "export_hour"=>4, "dojo_debug"=>false, "country"=>"DE", "auth_server"=>"drbssl://localhost:9997", "smtp_server"=>"localhost", "mail_invoice_smtp"=>"ipn@oddb.org", "log_file"=>#<IO:0x7f17866a1e78>, "db_user"=>"oddb", "db_name"=>"oddb", "auth_domain"=>"org.oddb.de", "pharmacy_percentage"=>3, "debug_recipients"=>[], "var"=>"/home/masa/ywesee/de.oddb.org/var", "default_dstype"=>"compare", "update_hour"=>2, "smtp_pass"=>nil, "payment_period"=>30, "query_limit"=>5, "mail_invoice_from"=>"\"de.ODDB.org\" ipn@oddb.org", "admins"=>[], "vat"=>19, "remote_databases"=>[], "persistence"=>"odba", "paypal_receiver"=>nil, "log_level"=>"INFO", "db_auth"=>"oddb", "currency_rates"=>"druby://localhost:10999", "run_exporter"=>true, "prices"=>{"org.oddb.de.download.1"=>{"fachinfos.de.oddb.yaml"=>800, "chde.xls"=>600, "compendium_de.oddb.org.htc.prc"=>17, "patinfos_de.oddb.org.firefox.epub"=>17, "patinfos_de.oddb.org.kindle.mobi"=>17, "patinfos.de.oddb.yaml"=>800, "patinfos_de.oddb.org.stanza.epub"=>17, "de.oddb.csv"=>500, "compendium_de.oddb.org.stanza.epub"=>17, "compendium_de.oddb.org.kindle.mobi"=>17, "compendium_de.oddb.org.firefox.epub"=>17, "patinfos_de.oddb.org.htc.prc"=>17, "de.oddb.yaml"=>600}, "org.oddb.de.export.csv"=>0.16, "org.oddb.de.view.30"=>50, "org.oddb.de.view.1"=>5, "org.oddb.de.download.12"=>{"fachinfos.de.oddb.yaml"=>1400, "chde.xls"=>2000, "patinfos.de.oddb.yaml"=>1400, "de.oddb.csv"=>2000, "de.oddb.yaml"=>2500}, "org.oddb.de.view.365"=>400}, "log_level_export"=>"INFO", "log_file_export"=>#<IO:0x7f17866a1e78>, "http_proxies"=>[], "invoice_server"=>"druby://localhost:12375", "download_uncompressed"=>["compendium_de.oddb.org.firefox.epub", "compendium_de.oddb.org.htc.prc", "compendium_de.oddb.org.kindle.mobi", "compendium_de.oddb.org.stanza.epub", "patinfos_de.oddb.org.firefox.epub", "patinfos_de.oddb.org.htc.prc", "patinfos_de.oddb.org.kindle.mobi", "patinfos_de.oddb.org.stanza.epub"], "credentials"=>{}}, @argv={"test"=>"123"}, @runtime={}, @loaded={"smtp_user"=>"mhatakeyama@ywesee.com", "smtp_domain"=>"ywesee.com", "run_updater"=>true, "paypal_server"=>"www.paypal.com", "http_server"=>"http://de.oddb.masa.org", "remote_export_dir"=>"/var/www/oddb.org/data/downloads", "mail_invoice_smtp"=>"zdavatz@ywesee.com", "smtp_server"=>"smtp.gmail.com", "debug_recipients"=>["mhatakeyama@ywesee.com"], "smtp_pass"=>"", "mail_invoice_from"=>"\"Zeno Davatz\" <zdavatz@ywesee.com>", "admins"=>["mhatakeyama@ywesee.com"], "paypal_receiver"=>"zdavatz@ywesee.com", "remote_databases"=>["druby://localhost:10013"], "log_level"=>"DEBUG", "run_exporter"=>true, "credentials"=>{"whocc"=>{"username"=>"hwyss@ywesee.com", "password"=>"a23w33"}, "product_infos"=>{"pop_pass"=>"pharma243298", "pop_user"=>"pharma24@oddb.org", "pop_server"=>"mail.ywesee.com"}}}>
"123"
Run
$ jobs/import_gkv
Result
/usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:42:in `method_missing': private method `test' called for #<RCLConf::RCLConf:0x7f510adc0070> (NoMethodError)
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:42:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:42:in `method_missing'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:41:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:41:in `method_missing'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:40:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:40:in `method_missing'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:39:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:39:in `method_missing'
from /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:58:in `import_gkv'
from jobs/import_gkv:12
from /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `call'
from /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `run'
from jobs/import_gkv:11
Notes
- If we puts a command line argument, z.B. 'test=123', the value '123' is saved in 'ODDB.config.test' method.
- This is implemented in more detail in http://scm.ywesee.com/?p=de.oddb.org/.git;a=blob;f=lib/oddb/config.rb;h=47295fe28ff02ab74f3f12ba69b79bf26c1b1bac;hb=HEAD#l108»lib/oddb/config.rb
- But if we do not put the command line argument, 'NoMethodError' comes
Solutions
- Modifying the jobs/import_gkv directly
- Using ODDB.config (lib/oddb/config)
Which is better...
My opinion
- The pdf link (url) of import_gkv is not regarding ODDB system itself, so
- I should not use ODDB.config variable in this case
- Updating jobs/import_gkv is a straightforward solution and it is easy to understand
- I guess it is not smart solution, though.
Run import_gkv manually
module ODDB
module Util
opt = {}
if ARGV[0].split("=")[0] == "pdf"
opt = {:pdf => ARGV[0].split("=")[1]}
end
Job.run do
Updater.import_gkv opt
end
end
end
Run import_gkv manually
$ jobs/import_gkv pdf=https://www.gkv-spitzenverband.de/upload/Zuzahlungsbefreit_sort_Name_101015_14671.pdf
Result
success- Email Tue Oct 26 09:25:38 2010: de.oddb.org Zubef (PDF)
Tue Oct 26 08:50:56 2010: de.oddb.org ODDB::Import::Gkv#import Imported 6557 Zubef-Entries on 26.10.2010: Visited 6557 existing Zubef-Entries Visited 6557 existing Companies Visited 1029 existing Substances Created 0 new Zubef-Entries Created 0 new Products Created 0 new Sequences Created 0 new Companies Created 0 new Substances Assigned 27 Chemical Equivalences Assigned 3 Companies Created 0 Incomplete Packages: Created 1 Product(s) without a name (missing product name): http://de.oddb.org/de/drugs/product/uid/3480899
Download files
masa@masa ~/ywesee/de.oddb.org $ ls var/pdf/gkv/ 2010.10.26-Zuzahlungsbefreit_sort_Name_101015_14671.pdf Zuzahlungsbefreit_sort_Name_101015_14671.pdf
Run import_gkv automatically
def initialize(*args)
super
@rss_mutex = Mutex.new
# run_exporter if(ODDB.config.run_exporter)
run_updater if(ODDB.config.run_updater)
end
def run_at(hour, &block)
Thread.new {
loop {
now = Time.now
#run_at = Time.local(now.year, now.month, now.day, hour)
run_at = Time.local(now.year, now.month, now.day, now.hour, 45)
while(now > run_at)
run_at += 24*60*60
end
print "now=", now, "\n"
print "run_at=", run_at, "\n"
sleep(run_at - now)
block.call
}
}
end
Run oddbd (Run import_gkv automatically)
masa@masa ~/ywesee/de.oddb.org $ bin/oddbd now=Tue Oct 26 09:40:33 +0200 2010 run_at=Tue Oct 26 09:45:00 +0200 2010 I, [2010-10-26T09:40:33.598382 #25828] INFO -- start: starting oddb-server on druby://localhost:11000
Result
- Emails
- Tue Oct 26 09:45:20 2010: de.oddb.org ODDB::Import::Dimdi::Substance (XLS) success
- Tue Oct 26 09:45:24 2010: de.oddb.org ODDB::Import::Dimdi::GalenicForm (XLS)
NoMthodError - Tue Oct 26 10:05:04 2010: de.oddb.org Update Festbeträge (XLS)
NoMethodError - Tue Oct 26 10:38:59 2010: de.oddb.org Zubef (PDF) success
Check new files
masa@masa ~/ywesee/de.oddb.org $ git status # On branch work # Changed but not updated: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: jobs/import_gkv # modified: lib/oddb/util/server.rb # # Untracked files: # (use "git add <file>..." to include in what will be committed) # # data/sql/ # etc/ # gitignore # log/ # var/ no changes added to commit (use "git add" and/or "git commit -a") masa@masa ~/ywesee/de.oddb.org $ ls log -al insgesamt 8 drwxr-xr-x 2 masa masa 16 26. Okt 10:05 . drwxr-xr-x 12 masa masa 4096 26. Okt 09:45 .. -rw-r--r-- 1 masa masa 0 26. Okt 09:45 import_dimdi -rw-r--r-- 1 masa masa 0 26. Okt 10:05 import_gkv masa@masa ~/ywesee/de.oddb.org $ ls var/ pdf xls masa@masa ~/ywesee/de.oddb.org $ ls var/pdf/ gkv masa@masa ~/ywesee/de.oddb.org $ ls var/pdf/gkv/ 2010.10.26-Zuzahlungsbefreit_sort_Name_101015_14671.pdf Zuzahlungsbefreit_sort_Name_101015_14671.pdf masa@masa ~/ywesee/de.oddb.org $ ls var/xls/ darreichungsformen-201007.xls festbetraege-201007.xls wirkstoffkuerzel-201007.xls
Commit Updated import_gkv pdf link option
For README of RCLConf
This is used at de.oddb.org/lib/oddb/config.rb
Summary
- This library is available and useful for a configuration of an application.
- A command line option is also managed in this library.
For example
config.rb
require 'rclconf'
module Application
default_config_files = [
'/home/masa/work/config.yml'
]
defaults = {
'admins' => [],
'passwd' => nil,
}
@config = RCLConf::RCLConf.new(ARGV, defaults)
@config.load(default_config_files)
end
config.yml
--- admins: - masa@ywesee.com users: - abc@ywesee.com - xyw@ywesee.com passwd: 'hata'
application.rb
require 'config' module Application print "@config=" p @config print "@config.admins=" p @config.admins print "@config.users=" p @config.users print "@config.passwd=" p @config.passwd print "@config.parse_argv(ARGV)=" p @config.parse_argv(ARGV) print "@config.test=" p @config.test end
Run application (with a new command line argument)
$ ruby application.rb test=123
@config=#<RCLConf::RCLConf:0x7f9e3b0696c8 @defaults={"passwd"=>nil, "admins"=>[]}, @argv={"test"=>"123"}, @runtime={}, @loaded={"passwd"=>"hata", "users"=>["abc@ywesee.com", "xyw@ywesee.com"], "admins"=>["masa@ywesee.com"]}>
@config.admins=["masa@ywesee.com"]
@config.users=["abc@ywesee.com", "xyw@ywesee.com"]
@config.passwd="hata"
@config.parse_argv(ARGV)={"test"=>"123"}
@config.test="123"
Run application (without a command line arugemnt)
$ ruby application.rb
@config=#<RCLConf::RCLConf:0x7f6ddae6f448 @defaults={"passwd"=>nil, "admins"=>[]}, @argv={}, @runtime={}, @loaded={"passwd"=>"hata", "users"=>["abc@ywesee.com", "xyw@ywesee.com"], "admins"=>["masa@ywesee.com"]}>
@config.admins=["masa@ywesee.com"]
@config.users=["abc@ywesee.com", "xyw@ywesee.com"]
@config.passwd="hata"
@config.parse_argv(ARGV)={}
/usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:42:in `method_missing': private method `test' called for #<RCLConf::RCLConf:0x7f6ddae6f448> (NoMethodError)
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:42:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:42:in `method_missing'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:41:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:41:in `method_missing'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:40:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:40:in `method_missing'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:39:in `fetch'
from /usr/lib64/ruby/site_ruby/1.8/rclconf/rclconf.rb:39:in `method_missing'
from application.rb:16
@config.test=
Notes
- @config variable can change dynamically
- but NoMethodError comes when an undefined configuration option is implemented (this should be updated)
Check tests
masa@masa ~/ywesee/rclconf $ ruby test/test_rclconf.rb
Loaded suite test/test_rclconf
Started
...E.EE.E......
Finished in 0.003053 seconds.
1) Error:
test_argv_overrides_file(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:48:in `test_argv_overrides_file'
test/test_rclconf.rb:47:in `open'
test/test_rclconf.rb:47:in `test_argv_overrides_file'
2) Error:
test_file_overrides_defaults(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:60:in `test_file_overrides_defaults'
test/test_rclconf.rb:59:in `open'
test/test_rclconf.rb:59:in `test_file_overrides_defaults'
3) Error:
test_load_mandatory__file_found(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:80:in `test_load_mandatory__file_found'
test/test_rclconf.rb:79:in `open'
test/test_rclconf.rb:79:in `test_load_mandatory__file_found'
4) Error:
test_load_optional__file_precedence(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:107:in `test_load_optional__file_precedence'
test/test_rclconf.rb:106:in `open'
test/test_rclconf.rb:106:in `test_load_optional__file_precedence'
15 tests, 18 assertions, 0 failures, 4 errors
Check a test case which succeeds
- No such a test case, all tests failed
masa@masa ~/work/rclconf $ git checkout 51b47ccbb9aa994e50289434ff87c9889de1c16a
masa@masa ~/work/rclconf $ ruby test/test_rclconf.rb
Loaded suite test/test_rclconf
Started
..E.EFE.FE....
Finished in 0.004718 seconds.
1) Error:
test_argv_overrides_file(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:39:in `test_argv_overrides_file'
test/test_rclconf.rb:38:in `open'
test/test_rclconf.rb:38:in `test_argv_overrides_file'
2) Error:
test_file_overrides_defaults(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:52:in `test_file_overrides_defaults'
test/test_rclconf.rb:51:in `open'
test/test_rclconf.rb:51:in `test_file_overrides_defaults'
3) Failure:
test_load_mandatory__corrupt_file__raises_error(RCLConf::RCLConfTest) [test/test_rclconf.rb:72]:
<ArgumentError> exception expected but none was thrown.
4) Error:
test_load_mandatory__file_found(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:84:in `test_load_mandatory__file_found'
test/test_rclconf.rb:83:in `open'
test/test_rclconf.rb:83:in `test_load_mandatory__file_found'
5) Failure:
test_load_optional__corrupt_file__raises_error(RCLConf::RCLConfTest) [test/test_rclconf.rb:106]:
<ArgumentError> exception expected but none was thrown.
6) Error:
test_load_optional__file_precedence(RCLConf::RCLConfTest):
NoMethodError: undefined method `>' for nil:NilClass
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `node_export'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `add'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:41:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `each'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:40:in `to_yaml'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `map'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:39:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:391:in `call'
/usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
/usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:38:in `to_yaml'
/usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
test/test_rclconf.rb:123:in `test_load_optional__file_precedence'
test/test_rclconf.rb:122:in `open'
test/test_rclconf.rb:122:in `test_load_optional__file_precedence'
14 tests, 13 assertions, 2 failures, 4 errors
Notes
- I guess this is not from rclconf but from yaml library
- This might be related to YDPM error in my local.
Experiment
require 'yaml'
YAML.dump( ['badger', 'elephant', 'tiger'], File.open('sample.yml', 'w'))
Result
$ ruby test.rb
/usr/lib64/ruby/1.8/yaml/rubytypes.rb:147:in `is_binary_data?': undefined method `>' for nil:NilClass (NoMethodError)
from /usr/lib64/ruby/1.8/yaml/rubytypes.rb:166:in `to_yaml'
from /usr/lib64/ruby/1.8/yaml.rb:391:in `call'
from /usr/lib64/ruby/1.8/yaml.rb:391:in `emit'
from /usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
from /usr/lib64/ruby/1.8/yaml/rubytypes.rb:165:in `to_yaml'
from /usr/lib64/ruby/1.8/yaml/rubytypes.rb:110:in `node_export'
from /usr/lib64/ruby/1.8/yaml/rubytypes.rb:110:in `add'
from /usr/lib64/ruby/1.8/yaml/rubytypes.rb:110:in `to_yaml'
... 6 levels...
from /usr/lib64/ruby/1.8/yaml.rb:391:in `quick_emit'
from /usr/lib64/ruby/1.8/yaml/rubytypes.rb:107:in `to_yaml'
from /usr/lib64/ruby/1.8/yaml.rb:117:in `dump'
from test2.rb:3
masa@masa ~/work $ ruby1.9 test2.rb
test2.rb:in `require': no such file to load -- auto_gem (LoadError)
masa@masa ~/work $ unset RUBYOPT
masa@masa ~/work $ ruby1.9 test2.rb
masa@masa ~/work $ cat sample.yml
---
- badger
- elephant
- tiger
Test
masa@masa ~/work/rclconf $ unset RUBYOPT masa@masa ~/work/rclconf $ ruby1.9 test/test_rclconf.rb Loaded suite test/test_rclconf Started ............... Finished in 0.002902 seconds. 15 tests, 25 assertions, 0 failures, 0 errors, 0 skips Test run options: --seed 53894
Conclusion
- Ruby 1.8.6 causes this error
Memo
- I can run correctly YDPM in my local as follows: 'ruby1.9 -I /home/masa/ywesee/rclconf/lib /usr/bin/ydpm'
Debug imoprt_dimdi NoMethodError
There comes NoMethodError mails when import_dimdi runs
- Tue Oct 26 09:45:24 2010: de.oddb.org ODDB::Import::Dimdi::GalenicForm (XLS)
NoMthodError - Tue Oct 26 10:05:04 2010: de.oddb.org Update Festbeträge (XLS)
NoMethodError - Tue Oct 26 09:45:20 2010: de.oddb.org ODDB::Import::Dimdi::Substance (XLS) success
Details
1. Tue Oct 26 09:45:24 2010: de.oddb.org ODDB::Import::Dimdi::GalenicForm (XLS) NoMthodError
Tue Oct 26 09:45:23 2010: de.oddb.org ODDB::Import::Dimdi::GalenicForm#import NoMethodError private method `split' called for nil:NilClass /home/masa/ywesee/de.oddb.org/lib/oddb/import/importer.rb:16:in `capitalize_all' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:80:in `import_row' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:31:in `import_worksheet' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:112:in `call' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:112:in `each' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:111:in `upto' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:111:in `each' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/excel/worksheet.rb:34:in `each' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:30:in `import_worksheet' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:25:in `import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:110:in `reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:117:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:117:in `_reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:110:in `reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:27:in `import_dimdi_galenic_forms' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:40:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:40:in `download' /usr/lib64/ruby/1.8/open-uri.rb:135:in `open_uri' /usr/lib64/ruby/1.8/open-uri.rb:518:in `open' /usr/lib64/ruby/1.8/open-uri.rb:30:in `open' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:39:in `download' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:26:in `import_dimdi_galenic_forms' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:20:in `import_dimdi' /home/masa/ywesee/de.oddb.org/jobs/import_dimdi:12 /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `run' /home/masa/ywesee/de.oddb.org/jobs/import_dimdi:11 Imported 216 Galenic Forms per 01.07.2010: Visited 215 existing Created 0 new
2. Tue Oct 26 10:05:04 2010: de.oddb.org Update Festbeträge (XLS) NoMethodError
Tue Oct 26 09:45:29 2010: de.oddb.org ODDB::Import::Dimdi::Product#import NoMethodError private method `split' called for nil:NilClass /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:348:in `import_substances' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:281:in `import_sequence' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:258:in `import_product' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:211:in `import_row' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:31:in `import_worksheet' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:112:in `call' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:112:in `each' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:111:in `upto' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:111:in `each' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/excel/worksheet.rb:34:in `each' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:30:in `import_worksheet' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:25:in `import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:110:in `reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:117:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:117:in `_reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:110:in `reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:34:in `import_dimdi_products' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:40:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:40:in `download' /usr/lib64/ruby/1.8/open-uri.rb:135:in `open_uri' /usr/lib64/ruby/1.8/open-uri.rb:518:in `open' /usr/lib64/ruby/1.8/open-uri.rb:30:in `open' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:39:in `download' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:33:in `import_dimdi_products' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:21:in `import_dimdi' /home/masa/ywesee/de.oddb.org/jobs/import_dimdi:12 /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `run' /home/masa/ywesee/de.oddb.org/jobs/import_dimdi:11 Imported 30494 Products per 01.07.2010: Visited 0 existing Products Visited 0 existing Sequences Ignored 1 unknown Products Created 0 new Sequences Created 0 new Substances from Combinations Renamed 0 Products Reassigned 0 PZNs Deleted 0 Products Deleted 0 Sequences
Notes
- Both are NoMethodErrors about split method for NilClass
- This looks the same error to export_chde_xls before
First focusing on 1. Tue Oct 26 09:45:24 2010: de.oddb.org ODDB::Import::Dimdi::GalenicForm (XLS)
def Updater.import_dimdi
if(date = Import::Dimdi.current_date(DIMDI_INDEX))
import_dimdi_substances(date)
import_dimdi_galenic_forms(date) #<= here
import_dimdi_products(date)
end
end
def Updater.import_dimdi_galenic_forms(date)
file = date.strftime("darreichungsformen-%Y%m.xls")
Import::Dimdi.download(file) { |io| #<= here
reported_import(Import::Dimdi::GalenicForm.new(date), io,
:filetype => 'XLS')
}
end
def Dimdi.download(file, &block)
url = File.join(DIMDI_PATH, file)
xls_dir = File.join(ODDB.config.var, 'xls')
FileUtils.mkdir_p(xls_dir)
dest = File.join(xls_dir, file)
unless(File.exist?(dest))
open(url) { |remote| #<= here
block.call(remote)
remote.rewind
open(dest, 'w') { |local|
local << remote.read
}
}
end
rescue StandardError => e
ODDB.logger.error('Dimdi') { e.message }
end
skip
def import_row(row)
@count += 1
abbr = cell(row, 0)
description = capitalize_all(cell(row, 1)) #<= here
galenic_form = Drugs::GalenicForm.find_by_code(:value => abbr,
:type => "galenic_form",
:country => 'DE')
galenic_form ||= Drugs::GalenicForm.find_by_description(description)
if(galenic_form)
@existing += 1
else
@created += 1
galenic_form = Drugs::GalenicForm.new
galenic_form.description.de = description
end
galenic_form.add_code(Util::Code.new("galenic_form",
abbr, 'DE', @date))
galenic_form.save
galenic_form
end
lib/oddb/import/importer.rb:16
def capitalize_all(str)
## benchmarked fastest against an append (<<) solution
str.split(/\b/).collect { |part| #<= here
@@lower.match(part) ? part.downcase : part.capitalize }.join
end
Experiment
lib/oddb/import/importer.rb:16
def capitalize_all(str)
p str
## benchmarked fastest against an append (<<) solution
str.split(/\b/).collect { |part| #<= here
@@lower.match(part) ? part.downcase : part.capitalize }.join
end
Result
... u"ÜBERZOGENE TABLETTEN" u"ÜBERZOGENE TABLETTEN" u"VAGINALCREME" u"VAGINALDEPOTOVULUM" u"VAGINALOVULA" u"VAGINALTABLETTEN" u"VAGINALZAEPFCHEN (SUPPOS.)" u"VAGINALSALBE" u"WEICHGELATINEKAPSELN" u"ZERBEISSKAPSELN" nil
Notes
- I have confirmed that it certainly became nil
Backtrace again
def import_worksheet(worksheet)
worksheet.each(@skip_rows) { |row|
import_row(row) #<= here
}
end
Experiment
def import_worksheet(worksheet)
print "worksheet="
p worksheet
worksheet.each(@skip_rows) { |row|
import_row(row)
}
end
def parse(io)
Spreadsheet.open(io)
end
end
Result
worksheet=#<Spreadsheet::Excel::Worksheet:0x003f85ad39a644 @changes= @columns= @reader=#<Spreadsheet::Excel::Reader:0x7f0b5a7bc4f8> @row_addresses= @selected= @ole=#<Ole::Storage::RangesIOMigrateable:0x7f0b5a7ae4e8> @name=Darreichungsform 01.07.2010 @links= @default_format= @offsets= @offset=8863 @workbook=#<Spreadsheet::Excel::Workbook:0x7f0b5a7b0450> @dimensions= @rows[397]>
Check xls file
- var/xls/darreichungsformen-201007.xls
Notes
- The data certainly ends at 'ZERBEISSKAPSELN' but the script tried to read the next data row
- Then the error happened
Question
- Why does not the script end at the end of data in the xls file?
Experiment
abbr = cell(row, 0)
value = cell(row, 1)
print "abbr="
p abbr
print "value="
p value
print "abbr && value="
p (abbr && value)
if(abbr != nil && value != nil)
@count += 1
description = capitalize_all(cell(row, 1))
galenic_form = Drugs::GalenicForm.find_by_code(:value => abbr,
:type => "galenic_form",
:country => 'DE')
galenic_form ||= Drugs::GalenicForm.find_by_description(description)
if(galenic_form)
@existing += 1
else
@created += 1
galenic_form = Drugs::GalenicForm.new
galenic_form.description.de = description
end
galenic_form.add_code(Util::Code.new("galenic_form",
abbr, 'DE', @date))
galenic_form.save
galenic_form
end
Result
Tue Oct 26 14:41:11 2010: de.oddb.org ODDB::Import::Dimdi::GalenicForm#import Imported 215 Galenic Forms per 01.07.2010: Visited 215 existing Created 0 new
Commit Debug import_dimdi_galenic_forms nil data error in a xls file (NoMethodError)
Next focusing on 2. Tue Oct 26 10:05:04 2010: de.oddb.org Update Festbeträge (XLS) NoMethodError
Confirm the error
def Updater.import_dimdi
if(date = Import::Dimdi.current_date(DIMDI_INDEX))
# import_dimdi_substances(date)
# import_dimdi_galenic_forms(date)
import_dimdi_products(date)
end
end
Run jobs/import_dimdi
Result
- Email Tue Oct 26 15:31:37 2010: de.oddb.org Update Festbeträge (XLS)
NoMethodError
Tue Oct 26 15:11:42 2010: de.oddb.org ODDB::Import::Dimdi::Product#import NoMethodError private method `split' called for nil:NilClass /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:351:in `import_substances' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:284:in `import_sequence' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:261:in `import_product' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:214:in `import_row' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:31:in `import_worksheet' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:112:in `call' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:112:in `each' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:111:in `upto' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/worksheet.rb:111:in `each' /usr/lib64/ruby/gems/1.8/gems/spreadsheet-0.6.4.1/lib/spreadsheet/excel/worksheet.rb:34:in `each' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:30:in `import_worksheet' /home/masa/ywesee/de.oddb.org/lib/oddb/import/excel.rb:25:in `import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:110:in `reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:117:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:117:in `_reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:110:in `reported_import' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:34:in `import_dimdi_products' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:40:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:40:in `download' /usr/lib64/ruby/1.8/open-uri.rb:135:in `open_uri' /usr/lib64/ruby/1.8/open-uri.rb:518:in `open' /usr/lib64/ruby/1.8/open-uri.rb:30:in `open' /home/masa/ywesee/de.oddb.org/lib/oddb/import/dimdi.rb:39:in `download' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:33:in `import_dimdi_products' /home/masa/ywesee/de.oddb.org/lib/oddb/util/updater.rb:21:in `import_dimdi' jobs/import_dimdi:12 /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `call' /home/masa/ywesee/de.oddb.org/lib/oddb/util/job.rb:16:in `run' jobs/import_dimdi:11 Imported 30494 Products per 01.07.2010: Visited 0 existing Products Visited 0 existing Sequences Ignored 1 unknown Products Created 0 new Sequences Created 0 new Substances from Combinations Renamed 0 Products Reassigned 0 PZNs Deleted 0 Products Deleted 0 Sequences
New files
masa@masa ~/ywesee/de.oddb.org $ ls var xls masa@masa ~/ywesee/de.oddb.org $ ls var/xls/ festbetraege-201007.xls
Backtrace the error message
if(subs.empty?)
names = groupname.split('+') #<= here
subs = names.collect { |name|
assumed_name = name.strip[/^\S+/]
unless(assumed_name.empty?)
sub = Drugs::Substance.find_by_name(assumed_name)
if(sub.nil?)
@created_substances += 1
sub = Drugs::Substance.new
sub.name.de = assumed_name
sub.save
end
if(names.size == 1)
assign_substance_group(sub, groupname)
end
sub
end
}
end
subs.compact
end
Experiment
def import_row(row)
@package_date = cell(row, 13) || @date
@count += 1
pzn = u(cell(row, 0).to_i.to_s)
p cell(row, 0)
p pzn
Run jobs/import_dimdi
Result
u"9194307" 9194313.0 u"9194313" 9194336.0 u"9194336" nil u"0"
Look at the xls file
Notes
- This is also caused by nil data in a xls file.
Experiment
def import_row(row)
if(value = cell(row, 0))
@package_date = cell(row, 13) || @date
@count += 1
pzn = u(value.to_i.to_s)
package = Drugs::Package.find_by_code(:type => 'cid',
:value => pzn,
:country => 'DE')
name = product_name(row)
product = Drugs::Product.find_by_name(name)
if !package
## new package and possibly new product
import_product row, product, name
elsif !package.product
product ||= create_product name
if package.sequence.nil?
import_sequence row, product, package
else
seq = package.sequence
seq.product = product
seq.save
update_package row, package
end
else
## update package-data
update_package row, package
end
end
end
Run jobs/import_dimdi
Result
Tue Oct 26 16:21:46 2010: de.oddb.org ODDB::Import::Dimdi::Product#import Imported 30493 Products per 01.07.2010: Visited 0 existing Products Visited 0 existing Sequences Ignored 0 unknown Products Created 0 new Sequences Created 0 new Substances from Combinations Renamed 0 Products Reassigned 0 PZNs Deleted 0 Products Deleted 0 Sequences
Commit Debug import_product nil data error in a xls file (NoMethodError)