<< Masa.20101028-debug-bsv_follower-autorun | 2010 | Masa.20101026-update-import_gkv debug-import_dimdi >>
suspend
Commit Added README and updated install script
# run ch.oddb.org updates 1 3 * * * xxx /xxx/oddb.org/jobs/import_daily 1 5 * * * xxx /xxx/oddb.org/jobs/export_daily
#!/usr/bin/env ruby
# must be scheduled in crontab to run as the same user as oddb
$: << File.expand_path('../src', File.dirname(__FILE__))
$: << File.expand_path('..', File.dirname(__FILE__))
require 'util/job'
require 'util/updater'
module ODDB
module Util
Job.run do |system|
Updater.new(system).run
end
end
end
Notes
def run
logfile_stats
...
if(update_bsv)
update_bsv_followers
end
...
end
def update_bsv
logs_pointer = Persistence::Pointer.new([:log_group, :bsv_sl])
logs = @app.create(logs_pointer)
this_month = Date.new(@@today.year, @@today.month)
if (latest = logs.newest_date) && latest > this_month
this_month = latest
end
klass = BsvXmlPlugin
plug = klass.new(@app)
subj = 'SL-Update (XML)'
wrap_update(klass, subj) {
if plug.update
log_notify_bsv(plug, this_month, subj)
end
}
end
Notes
def wrap_update(klass, subj, &block)
begin
block.call
rescue Exception => e #RuntimeError, StandardError => e
notify_error(klass, subj, e)
raise
end
rescue StandardError
nil
end
Test a asample to make sure
def wrap_method(&block)
begin
block.call
rescue
nil
end
end
def test_case1
wrap_method do
p "Fine case"
true
end
end
def test_case2
wrap_method do
raise "Error case"
end
end
if(test_case1)
p "test_case1"
end
if(test_case2)
p "test_case2"
end
Result
$ ruby test.rb "Fine case" "test_case1"
Notes
Experiment
def run
logfile_stats
check_return_value = update_bsv
print "check_return_value="
p check_return_value
if(check_return_value)
update_bsv_followers
end
end
Run jobs/update_daily manually
Result
check_return_value=["mhatakeyama@ywesee.com"] "getin update_bsv_followers
New files
masa@masa ~/ywesee/oddb.org $ git status # On branch work # Changed but not updated: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: src/util/updater.rb # # Untracked files: # (use "git add <file>..." to include in what will be committed) # # data/xml/ no changes added to commit (use "git add" and/or "git commit -a") masa@masa ~/ywesee/oddb.org $ ls data/xml/ XMLPublications-2010.10.27.zip XMLPublications-latest.zip
Run jobs/import_daily again
Result
check_return_value=nil
Notes
Davatz-san's comment
Check the source code that sends the second mail.
The second mail is sent from
def log_notify_bsv(plug, date, subj='SL-Update')
pointer = Persistence::Pointer.new([:log_group, :bsv_sl], [:log, date])
values = log_info(plug)
if log = pointer.resolve(@app)
change_flags = values[:change_flags]
if previous = log.change_flags
previous.each do |ptr, flgs|
if flags = change_flags[ptr]
flags.concat flgs
flags.uniq!
else
change_flags[ptr] = flgs
end
end
end
end
log = @app.update(pointer.creator, values)
log.notify(subj)
log2 = Log.new(date)
log2.update_values log_info(plug, :log_info_bsv) #<= here
log2.notify(subj)
end
And this log_notify_bsv is called from
def update_bsv
logs_pointer = Persistence::Pointer.new([:log_group, :bsv_sl])
logs = @app.create(logs_pointer)
this_month = Date.new(@@today.year, @@today.month)
if (latest = logs.newest_date) && latest > this_month
this_month = latest
end
klass = BsvXmlPlugin
plug = klass.new(@app)
subj = 'SL-Update (XML)'
wrap_update(klass, subj) {
if plug.update
log_notify_bsv(plug, this_month, subj) #<= here
end
}
end
Notes
Confirm stopping the second mail
def log_notify_bsv(plug, date, subj='SL-Update')
pointer = Persistence::Pointer.new([:log_group, :bsv_sl], [:log, date])
values = log_info(plug)
if log = pointer.resolve(@app)
change_flags = values[:change_flags]
if previous = log.change_flags
previous.each do |ptr, flgs|
if flags = change_flags[ptr]
flags.concat flgs
flags.uniq!
else
change_flags[ptr] = flgs
end
end
end
end
log = @app.update(pointer.creator, values)
log.notify(subj)
=begin
log2 = Log.new(date)
log2.update_values log_info(plug, :log_info_bsv)
log2.notify(subj)
=end
Result
Consideration
BraSt
Check cron log on production server
/var/log/crond/current
Oct 26 03:01:01 xxx@xxx cron[25294]: (xxx) CMD (/xxx/oddb.org/jobs/import_daily) Oct 27 03:01:01 xxx@xxx cron[11463]: (xxx) CMD (/xxx/oddb.org/jobs/import_daily)
Notes
Check installed cron on production server
$ emerge -s cron
Searching...
[ Results for search key : cron ]
[ Applications found : 16 ]
* app-admin/cronolog
Latest version available: 1.6.2-r3
Latest version installed: 1.6.2-r3
Size of files: 130 kB
Homepage: http://cronolog.org/
Description: Cronolog apache logfile rotator
License: GPL-2
* app-admin/fifo-cronolog [ Masked ]
Latest version available: 1.1.1
Latest version installed: [ Not Installed ]
Size of files: 3 kB
Homepage: http://git.overlays.gentoo.org/gitweb/?p=proj/fifo-cronolog.git
Description: cronolog wrapper for use with dumb daemons like squid, varnish and so on
License: as-is
* app-emacs/crontab-mode
Latest version available: 1.20
Latest version installed: [ Not Installed ]
Size of files: 2 kB
Homepage: http://www.mahalito.net/~harley/elisp/
Description: Mode for editing crontab files
License: GPL-2
* app-portage/porticron
Latest version available: 0.5.2
Latest version installed: [ Not Installed ]
Size of files: 4 kB
Homepage: http://bb.xnull.de/projects/porticron/
Description: porticron is a cron script to sync portage and send update mails to root
License: BSD
* dev-perl/Config-Crontab
Latest version available: 1.20
Latest version installed: [ Not Installed ]
Size of files: 37 kB
Homepage: http://search.cpan.org/search?query=Config-Crontab
Description: Read/Write Vixie compatible crontab(5) files
License: Artistic
* kde-base/kcron
Latest version available: 4.4.5
Latest version installed: [ Not Installed ]
Size of files: 1,397 kB
Homepage: http://www.kde.org/
Description: KDE Task Scheduler
License: GPL-2
* media-fonts/cronyx-fonts
Latest version available: 2.3.1-r2
Latest version installed: [ Not Installed ]
Size of files: 588 kB
Homepage: http://koi8.pp.ru/frame.html?xwin.html#xwin_fonts
Description: Cronyx Cyrillic bitmap fonts for X
License: freedist
* media-fonts/font-cronyx-cyrillic
Latest version available: 1.0.1
Latest version installed: [ Not Installed ]
Size of files: 251 kB
Homepage: http://xorg.freedesktop.org/
Description: X.Org Cronyx cyrillic fonts
License: MIT
* sys-process/anacron
Latest version available: 2.3-r2
Latest version installed: [ Not Installed ]
Size of files: 23 kB
Homepage: http://anacron.sourceforge.net/
Description: a periodic command scheduler
License: as-is
* sys-process/bcron
Latest version available: 0.09
Latest version installed: [ Not Installed ]
Size of files: 56 kB
Homepage: http://untroubled.org/bcron/
Description: A new cron system designed with secure operations in mind by Bruce Guenter
License: GPL-2
* sys-process/cronbase
Latest version available: 0.3.2-r1
Latest version installed: 0.3.2-r1
Size of files: 0 kB
Homepage: http://www.gentoo.org/
Description: base for all cron ebuilds
License: GPL-2
* sys-process/cronie
Latest version available: 1.4.4
Latest version installed: [ Not Installed ]
Size of files: 196 kB
Homepage: https://fedorahosted.org/cronie/wiki
Description: Cronie is a standard UNIX daemon cron based on the original vixie-cron.
License: ISC BSD BSD-2
* sys-process/dcron
Latest version available: 3.2-r1
Latest version installed: [ Not Installed ]
Size of files: 21 kB
Homepage: http://apollo.backplane.com/FreeSrc/
Description: A cute little cron from Matt Dillon
License: GPL-2
* sys-process/fcron
Latest version available: 3.0.6-r1
Latest version installed: [ Not Installed ]
Size of files: 539 kB
Homepage: http://fcron.free.fr/
Description: A command scheduler with extended capabilities over cron and anacron
License: GPL-2
* sys-process/incron
Latest version available: 0.5.8
Latest version installed: [ Not Installed ]
Size of files: 168 kB
Homepage: http://incron.aiken.cz/
Description: inotify based cron daemon
License: GPL-2
* sys-process/vixie-cron
Latest version available: 4.1-r10
Latest version installed: 4.1-r10
Size of files: 57 kB
Homepage: ftp://ftp.isc.org/isc/cron/
Description: Paul Vixie's cron daemon, a fully featured crond implementation
License: ISC BSD-2 BSD
Notes
Install vixie-cron in my local
$ sudo emerge vixie-cron $ sudo emerge cronolog
New files
masa@masa ~/ywesee/oddb.org $ vim /etc/cron cron.d/ cron.daily/ cron.deny cron.hourly/ cron.monthly/ crontab cron.weekly/
Start crond
masa@masa ~/ywesee/oddb.org $ sudo /etc/init.d/vixie-cron start Passwort: * Caching service dependencies... [ok] * Starting vixie-cron... [ok] masa@masa ~/ywesee/oddb.org $ sudo rc-update add vixie-cron default * service vixie-cron added to runlevel default
Check process
masa@masa ~/ywesee/oddb.org $ ps aux|grep cron root 17613 0.0 0.0 21536 764 ? Ss 15:13 0:00 /usr/sbin/cron
Test cron in local
masa@masa ~/work $ cat crons.cron 10 3 1 1 * /bin/echo "I don't really like cron" 30 16 * 1,2 * /bin/echo "I like cron a little" * * * * * /bin/echo "I really like cron" >> /home/masa/work/test.dat masa@masa ~/work $ sudo crontab crons.cron masa@masa ~/work $ sudo crontab -l # DO NOT EDIT THIS FILE - edit the master and reinstall. # (crons.cron installed on Wed Oct 27 15:58:26 2010) # (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $) 10 3 1 1 * /bin/echo "I don't really like cron" 30 16 * 1,2 * /bin/echo "I like cron a little" * * * * * /bin/echo "I really like cron" >> /home/masa/work/test.dat ... masa@masa ~/work $ cat /home/masa/work/test.dat I really like cron I really like cron I really like cron I really like cron I really like cron masa@masa ~/work $ ls -al insgesamt 3436 -rw-r--r-- 1 root root 209 27. Okt 15:59 test.dat
Check crontab on production server
# crontab -l no crontab for root
Notes
Test run import_daily by cron in local with outputting log
masa@masa ~/work $ sudo crontab -l # DO NOT EDIT THIS FILE - edit the master and reinstall. # (crons.cron installed on Wed Oct 27 16:13:17 2010) # (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $) 15 * * * * /home/masa/ywesee/oddb.org/jobs/import_daily >> /home/masa/work/log.dat
Notes
point: if you use crontab command, you do not have to set a user for the job in a cron setting file
but if you write /etc/crontab directly, you can set the command user before the command in the job line
For example
Write /etc/crontab directly
masa@masa ~/work $ cat /etc/crontab # for vixie cron # $Header: /var/cvsroot/gentoo-x86/sys-process/vixie-cron/files/crontab-3.0.1-r4,v 1.2 2009/05/12 09:13:46 bangert Exp $ # Global variables SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/ # check scripts in cron.hourly, cron.daily, cron.weekly and cron.monthly 59 * * * * root rm -f /var/spool/cron/lastrun/cron.hourly 9 3 * * * root rm -f /var/spool/cron/lastrun/cron.daily 19 4 * * 6 root rm -f /var/spool/cron/lastrun/cron.weekly 29 5 1 * * root rm -f /var/spool/cron/lastrun/cron.monthly */10 * * * * root test -x /usr/sbin/run-crons && /usr/sbin/run-crons * * * * * masa /bin/echo "hello" >> /home/masa/work/test.dat 2>&1 masa@masa ~/work $ ls -al /home/masa/work insgesamt 3448 -rw-r--r-- 1 masa masa 12 27. Okt 16:36 test.dat masa@masa ~/work $ cat /home/masa/work/test.dat hello hello hello
Use crontab command
masa@masa ~/work $ sudo crontab -u masa crons.cron.masa Passwort: masa@masa ~/work $ sudo crontab -u masa -l # DO NOT EDIT THIS FILE - edit the master and reinstall. # (crons.cron.masa installed on Wed Oct 27 16:45:58 2010) # (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $) * * * * * /bin/echo "hello" >> /home/masa/work/test.dat 2>&1 masa@masa ~/work $ ls -al insgesamt 3448 -rw-r--r-- 1 masa masa 327 27. Okt 10:48 test.rb masa@masa ~/work $ cat test.dat hello
References