20110128-check-migrate-ramaze_oddb
<< | Index | >>
- Make class diagram (data structure) in each project
suspend - Read ticket#16 Import existing ch.oddb.org-data from the old database into 1.9
- Goal
- Understand data structure / 70%
- Milestones
Install ascii-diagramRead ticket#16- Read the code 'migrate'
- Summary
- Commits
- ToDo Tomorrow
- Keep in Mind
- Check drug data structure (class diagram) in ch.oddb.org, de.oddb.org, and ramaze oddb
- Check rdbi instead of dbi for ODBA
- Encoding woes (from Davatz-san)
- Feedback: This option indicates that the regular expression is parsed as 'UTF8' (from Davatz-san)
- pg on Ubuntu - see http://dev.ywesee.com/wiki.php/Gem/Pg (from Davatz-san)
- On Ice
- emerge --sync
Make class diagram (data structure) in each project
Dia install message
Package libpng12 was not found in the pkg-config search path. Perhaps you should add the directory containing `libpng12.pc' to the PKG_CONFIG_PATH environment variable Package 'libpng12', required by 'cairo', not found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables GTK_CFLAGS and GTK_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details.
Re-install cairo
sudo emerge cairo
Try install dia again
sudo emerge dia
Read ticket#16 Import existing ch.oddb.org-data from the old database into 1.9
http://trac.ywesee.com/ticket/16
Notes
- The class structure (class-hierarchy) is totally the same in de.oddb.org and ramaze oddb, which is an extended version from ch.oddb.org
- So, the data structure is available for both de.oddb.org and ch.oddb.org even if actually the source data is different
- But there are subtle differences in the old (oddb.org) and the new (de.oddb.org, ramaze)
- The 'migrate' script absorbs the difference
- migration diagram http://trac.ywesee.com/attachment/ticket/16/data_migration_diagram_ch.oddb.org_ruby-1.8_ruby-1.9.txt
- key word: ODBA::DRbWrapper
- key point: it is impossible to directly transfer the marshal data, then it converts the data through ODBA::DRbWrapper
- The migration does loading objects from the old database and re-creating new objects for the new database
Try to run migrate with -E:UTF-8 option
ruby1.9 -E:UTF-8 bin/migrate
Result
- the same error
Note (This is the first I should do)
The good news is, that I've managed to overcome the irregularities in the DRb connection
(by introducing a new method
OddbApp#each_remote_company?, which iterates over all companies wrapped in a ODBA::DRbWrapper
). Using this, I am now able to cleanly access Company data.
Note
- But there is not such a method defined in 'oddb.org', so I should define it
- I should understand how to use ODBA::DRbWrapper class
- I have so far created 'remote_each_company' method in OddbApp class with 'extended(DRb::DRbUndumped)' for 'call by reference'
- This may be wrong
Experiment
def remote_each_atc_class(&block)
ODBA::DRbWrapper.new(@atc_classes).each_value(&block)
#@atc_classes.extend(DRb::DRbUndumped).each_value(&block)
end
Run oddb.org/bin/oddbd
Run ch.oddb.org/bin/migrate
Result
- The error is same
- ODBA::DRbWrapper is basically the same to include DRb::DRbUndumped in the object to be called by reference through drb
Memo
- Important concept of the migration is changing the data structure (class structure)
From
+---------+ +--------------+ +----------+ +---------+ | Company |1---n| Registration |1---n| Sequence |1---n| Package | +---------+ +--------------+ +----------+ +---------+
To
+--------------+ +------+ +------+
| Registration | | FI | | PI |
+--------------+ +------+ +------+
n n 1 1
| | | |
1 1 n n
+---------+ +---------+ +----------+ +---------+
| Company |1---n| Product |1---n| Sequence |1---n| Package |
+---------+ +---------+ +----------+ +---------+
n
|
1
+-------+ +-------+
| ATC |1---n| DDD |
+-------+ +-------+
About Price
+-----------+ +-----------+ +----------------+ | | | Price | | Money | | Package |1---n| - type |1---n| - valid_from | | | | | | - amount | +-----------+ +-----------+ +----------------+
Note
- exactly: exfactory/pp
- valid_from: from when is this price valid (independent of the source of the price).
- amount: yes, the amount, e.g. 12.30.
Check flow of migrate script
- migrate_atc: ODDB::Drugs::Atc, ODDB::Drugs::Ddd
- migrate_multilingual
- migrate_company: ODDB::Business::Company
- migrate_product: ODDB::Drugs::Product, ODDB::Regulatory::Registration, ODDB::Regulatory::Authority
- migrate_sequence: ODDB::Drugs::Sequence,
- migrate_atc: ODDB::Drugs::Atc, ODDB::Drugs::Ddd
- migrate_composition: ODDB::Drugs::Composition
- migrate_active_agent: ODDB::Drugs::ActiveAgent
- migrate_package: ODDB::Drugs::Package, ODDB::Util::Code, ODDB::Business::Price
- migrate_part: ODDB::Drugs::Part, ODDB::Drugs::Unit
- migrate_migrate_feedback:
- migrate_m10l_document: ODDB::Text::Document, ODDB::Text::Chapter, ODDB::Text::Paragraph,
- migrate_sequence: ODDB::Drugs::Sequence,
- migrate_product: ODDB::Drugs::Product, ODDB::Regulatory::Registration, ODDB::Regulatory::Authority
Experiment
def remote_each_atc_class(&block)
ODBA::DRbWrapper.new(@atc_classes).each_value(&block)
end
def remote_each_company(&block)
ODBA::DRbWrapper.new(@companies).each_value(&block)
end
Run oddb.org/bin/oddbd
Run ch.oddb.org/bin/migrate (by Ruby 1.9.1)
Result
masa@masa ~/ywesee/ch.oddb.org $ /home/masa/bin/ruby191/bin/ruby bin/migrate ... (many errors) ...
Note
- It took more than five hours
- I know the grammatical reason of the errors but I have not completely understood the fundamental cause of the errors
- There is no description about the errors in the ticket#16
Note
- Looks good
- There are a little bit differences from online data but they are same to the data in ch.oddb.org online