view · edit · sidebar · attach · print · history

20110224-output-order-bbmb

<< Masa.20110225-testcases-oddb_org | 2011 | Masa.20110223-output-order-bbmb >>

  1. Merge two script to one
  2. Add option to choose the year
  3. Export xls file
  4. Summary to export xmlconv data
  5. Test-case for export_orders
  6. Check the command online server
Goal/Estimate
  • output order bbmb / 80%
Milestones
  1. Merge two script to one
  2. Add price column
  3. Add option to choose the year
  4. Export xls
  5. Resolve encoding problem suspend
  6. (Merge script and bin/admin) suspend
  7. Test-case export_orders
  8. Test-case ch.oddb.org
Summary
Commits

Merge two script to one

Review (The current procedure to export order)

lib/xmlconv/util/application.rb 

  def export_order
    open("/home/masa/work/test.dat", "w") do |f|
      self.transactions.reverse.each do |t|
        if t.commit_time >= Time.local(2010,1,1)
          f.print t.output
          #f.print "commit_time: ", t.commit_time, "\n"
          #f.print "input:", t.input, "\n"
          #f.print "output:", t.output, "\n\n"
        end
      end
    end
  end
  alias :ex :export_order

xmlconv.sort.rb 

File.readlines('test.dat').collect{|x|
  ret = ""
  if x =~ /(,\d+,)(\d\d)(\d\d)(2010)(,.+,)$/ || x =~ /(,\d+,)(\d\d)(\d\d)(2011)(,.+,)$/
    ret = $1 + $4 + $3 + $2 + $5
  end
  ret + "\n"
}.sort.each do |line|
  print line unless line.chomp == ""
end

add_kunden_artikel.rb 

kunden1 = {}
kunden2 = {}
File.readlines('txt2xls/backup/Kunden.txt').each do |line|
  x = line.split(/\t/)
#  print x[0], ",", x[1], ",", x[6], "\n"
  kunden1[x[0].to_i] = x[6]
  kunden2[x[0].to_i] = x[6]
end

artikel1 = {}
artikel2 = {}
File.readlines('txt2xls/backup/Artikel.txt').each do |line|
  x = line.split(/\t/)
  #print x[0], ",", x[3], ",", x[2], "\n"
  artikel1[x[4].to_i] = x[2]
  artikel2[x[3].to_i] = x[2]
end

File.readlines('xmlconv.sorted.dat').each do |line|
  x = line.split(/,/)
  kunden = ""
  if kunden1.keys.include?(x[1].to_i)
    kunden = kunden1[x[1].to_i]
  else kunden2.keys.include?(x[1].to_i)
    kunden = kunden2[x[1].to_i]
  end
  artikel = ""
  if artikel1.keys.include?(x[4].to_i)
    artikel = artikel1[x[4].to_i]
  else artikel2.keys.include?(x[5].to_i)
    artikel = artikel2[x[5].to_i]
  end
  x.map!{|z| z || ""}
  artikel ||=""
  kunden ||=""
  print "," + x[1] + "," + kunden + "," +\
      x[2] + "," + x[3] + "," + x[4] + "," + artikel + "," + \
      "," + x[5] + "," + x[6] + "," + x[7] + "," + x[8] +\
      "," + x[9] + "," + x[10]
end

Procedure

  • Run xmlconvd 
 masa@masa ~/ywesee/xmlconv $ bin/xmlconvd config="etc/xmlconv.yml"
  • Export order via bin/admin 
 masa@masa ~/ywesee/xmlconv $ bin/admin server_url="druby://localhost:12006"
 ch.xmlconv> ex
  • Sort by Kunden Nr. and Date 
 masa@masa ~/work $ ruby xmlconv.sort.rb > xmlconv.sorted.dat
  • Add Kunden Name and Artikel Name 
 masa@masa ~/work $ ruby add_kunden_artikel.rb > xmlconv.sorted.kund.art.dat

Next

  • Merge the two script into one

Attach:xmlconv_convert.20110224.rb.txt Δ

Result 

masa@masa ~/work $ ruby xmlconv_convert.rb -i test.dat 
.........done.

The following Kunden IDs were not found:
"",410060252120,4100602525,99

The following Artikel IDs were not found:
0002152,0002499,0002601,0002682,0002731,0002850,0002924,0003084,0003123,0003124,0003125,0003126,0003144,0003194,0004437,0009667,0013985,0013987,0014070,0058985,0097270,0097293,0125049,0384673,0384816,0384822,0384839,0384845,0385566,0607682,0607699,0607713,0607736,0629459,0629465,0674454,0683134,0700447,0723922,0757921,0767629,0879937,0879943,0887635,0917810,0933677,0935096,0968569,1014748,1195767,1241444,1258605,1276744,1276767,1295753,1301927,1301933,1301956,1326850,1336630,1350251,1350564,1362277,1363905,1363957,1363986,1369078,1399978,1407577,1407985,1408789,1408795,1408803,1408826,1425983,1472965,1484483,1525437,1525443,1556283,1571408,1586261,1625771,1667120,1793876,1808193,1852550,1860791,1901716,1901722,1901797,1906062,1948468,2018702,2059581,2068930,2122049,2126604,2164496,2174566,2174589,2174603,2195628,2215845,2215851,2233381,2265139,2338832,2359076,2476037,2504957,2508429,2509417,2613818,2631101,2692574,2759864,2759924,2808516,2810387,2811665,2820227,2820380,2919687,2928203,2928226,2937320,2942255,2965196,2965629,2965724,2970062,2970085,2993169,2993175,2993198,2993206,2993212,2993229,2993382,3011961,3028660,3028855,3033425,3050872,3053178,3083713,3083788,3105917,3119204,3275950,3320389,3341397,3406151,3415440,3463404,3515911,3515928,3515940,3581084,3592099,3593503,3621300,3625829,3625841,3698343,3698403,3698449,3716704,3716710,3754202,3805750,3805856,3894133,3894216,4006867,4006873,4014269,4085430,4139427,4146953,4152735,4161680,4190983,4240445,4290578,4291980,4291997,4292005,4421483,4448967,4484816,4552765,4552794,4552819,7012115

Usage 

masa@masa ~/work $ ruby xmlconv_convert.rb -h
  usage:
    ruby xmlconv_convert.rb -i [input_file]

  options:
    -i [input_file]  (default: input.dat)
    -o [output_file] (default: output.dat)
    -k [file]: Kunden.txt
    -a [file]: Artikel.txt
    -h help

Next

  • Add price column

Attach:xmlconv_convert2.20110224.rb.txt

Note

  • I have confirmed the calclation is correct

Add option to choose the year

Next

  • Add option to choose the year

util/xmlconv/util/application.rb 

  def export_orders(first=Time.local(1990,1,1), last=Time.local(2100,1,1), output_file=nil)
    range=Range.new(first, last)
    output_file ||= "#{ENV['HOME']}/xmlconv_export.dat"
    open(output_file, "w") do |f|
      self.transactions.reverse.each do |t|
        if range.include?(t.commit_time)
          f.print t.output
        end
      end
    end
  end

Restart xmlconvd 

masa@masa ~/ywesee/xmlconv $ bin/xmlconvd config="etc/xmlconv.yml"

Run (default: all data) 

masa@masa ~/ywesee/xmlconv $ bin/admin server_url="druby://localhost:12006"
ch.xmlconv> export_orders
-> Array

Note

  • The output file will be '$HOME/xmlconv_export.dat' as a default

Run (from 2010) 

masa@masa ~/ywesee/xmlconv $ bin/admin server_url="druby://localhost:12006"
ch.xmlconv> export_orders Time.local(2010,1,1)
-> Array

Run (only in 2010) 

masa@masa ~/ywesee/xmlconv $ bin/admin server_url="druby://localhost:12006"
ch.xmlconv> export_orders Time.local(2010,1,1), Time.local(2010,12,31)
-> Array

Run (only in 2010 and output file is '/home/masa/test.dat') 

masa@masa ~/ywesee/xmlconv $ bin/admin server_url="druby://localhost:12006"
ch.xmlconv> export_orders Time.local(2010,1,1), Time.local(2010,12,31), output_file='/home/masa/work/test.dat'
-> Array

Export xls file

Next

  • Convert output into xls data

Arrange output format (but text data)

Note

  • There remains still encoding problem

Summary to export xmlconv data

Commit

Procedure

  • Export raw data through bin/admin command 
 export_order Time.local(2010,1,1), Time.local(2010,12,31), output_file='/home/masa/export.csv'
  • Convert the data with Kunden.txt and Artikel.txt 
 ruby xmlconv_order.rb -i export.csv -o convert.csv -k Kunden.txt -a Artikel.txt

Note

  • Choose the encoding, 'ISO-8859-1', when you open it with OpenOffice

Test-case for export_orders

Commit

Total Coverage: 55%

Check the command online server

Commit

Note

  • The output file default path changed '/home/ywesee/xmlconv_export/xmlconv_export.csv'
  • 'xmlconvd' runs with apache
  • 'bin' directory is linked to '~/cogito/xmlconv/bin'

Export procedure

  • login online server
  • goto xmlconv directory under /var directory
  • call bin/admin with server_url option, server_url="druby://localhost:xxxxxx"
    • Check the 'etc/xmlconv.yml' file
  • Then type 'export_orders' command
  • It is necessary to reboot each server when 'xmlconv' is updated

Reference

view · edit · sidebar · attach · print · history
Page last modified on May 08, 2012, at 06:55 PM