So, your cronjob did not run?

Posted by Steve on Thu 21 Feb 2013 at 09:26

Tags: ,

Recently I was hit be a problem which was ultimately caused by a failure to run a cron-job. Here I'm going to document the three most common means for failure that I know about, in the hopes of avoiding them in the future.

We've briefly discussed cronjobs in the past.

In brief there are two different ways to schedule jobs on Debian systems:

  • In your personal crontab file, viable with "crontab -l" and editable with "crontab -e".
  • Via a system file.

The system files have the most failure cases. Generally speaking you will find directories such as the following, are each invoked from /etc/crontab:

/etc/cron.daily
/etc/cron.weekly
/etc/cron.monthly

The contents of these directories are executed via the run-parts tool, which is designed to run every executable in the named directory in turn.

Invalid Filenames

Because the contents of, for example, /etc/cron.daily are executed via run-parts your scripts must have suitable filenames.

As this demo shows run-parts will skip some files:

$ mkdir /tmp/x
$ ln -s /bin/ls /tmp/x/foo.bar
$ run-parts /tmp/x
$

Here the (horrid) name "foo.bar" has been skipped by run-parts. You can see by reading "man run-parts" which filenames are going to be executed.

This problem only affects the system cron entries, because they are the only ones executed by run-parts.

Missing newlines

A common cause with broken personal cron entries is that the crontab file doesn't contain a newline at the end of lines.

Bad permissions

If you were to add an entry to /etc/cron.d to define a system cronjob which executes with an arbitrary frequency you must be careful with permissions.

You must ensure that the file has permissions 0644, otherwise the script will not run and you'll be puzzled by this mysterious error in syslog:

(*system*foo) WRONG INODE INFO (/etc/cron.d/foo)

Fixing this should be as simple as running "chmod 644 /path/to/file", but some older versions of cron will ignore such updates. Instead you must rename your cron file.

Note: If you wish to remove group/other read-permissions that is fine. The important thing here is that the file must not be group/other writable. (For example a snippet owned by root:root, with permissions 664 would be ignored, even though only root may write to it.)

Hopefully this quick reference was useful, but if there are other common "gotchas" I'd love to hear about them!

 

 


Posted by Anonymous (194.71.xx.xx) on Thu 21 Feb 2013 at 10:51
Another thing to note that has bitten me a few times, is that the percent-sign '%' is a reserved character in the command portion of a crontab entry, and will be replaced by a newline character unless escaped by a back-slash.

So this is no good if you for example want to use "date" to rotate your cronjob logs:

This won't work as expected unless you replace % with \% below:
15 0 * * * ~/bin/nightly-backup.sh > ~/log/nightly-backup.log.`date +%w` 2>&1

[ Parent | Reply to this comment ]

Posted by Steve (46.43.xx.xx) on Thu 21 Feb 2013 at 10:54
[ Send Message | View Steve's Scratchpad | View Weblogs ]

That's a good thing to know, thanks. I had no idea!

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (79.215.xx.xx) on Thu 21 Feb 2013 at 15:02
Two more things to consider:

1) Your 'crond' probably has an environment different from the one used by your login shell. This can cause some strange effects:

- Your cronjob might not start at all because 'crond' is missing some directories in the 'PATH' or 'LD_LIBRARY_PATH' variable.

- Your cronjob will run some hours earlier/later than you expected. This happens if 'crond' uses a different timezone than your login shell. You need to specify the times in the crontab file with regard to the timezone of 'crond'.

- Sorting with 'sort' depends on one (or some) of the 'LC_...' variables.

There're much, much more possible effects. Remember: Just because your script works fine from the commandline doesn't mean it'll work as a cronjob. If you're getting strange effects, get the 'crond' environment with an entry like this:

0 * * * * /usr/bin/env > /tmp/env.log

2) The first start of your job should be at least some minutes in the future. If you schedule it too close to 'now', the change might register with crond only after the first start time has passed.

[ Parent | Reply to this comment ]

Posted by Anonymous (50.83.xx.xx) on Thu 14 Mar 2013 at 14:53
A good way to test a new job in the same environment is to use 'at' for one-off runs:
echo ~/bin/blah.test | at 09:00

I also always recommend using a script instead of commands in crontab, just a good habit.

Finally, make sure email from cron to yourself works for good feedback about errors.

[ Parent | Reply to this comment ]

Posted by Anonymous (83.170.xx.xx) on Wed 20 Mar 2013 at 02:35
good !

[ Parent | Reply to this comment ]

Posted by mcortese (193.78.xx.xx) on Thu 21 Mar 2013 at 13:42
[ Send Message | View Weblogs ]
Are you sure? According to the man page, most of the environment is copied from the one in effect at invocation time.
The at-job shall be executed in a separate invocation of the shell, running in a separate process group with no controlling terminal, except that the environment variables, current working directory, file creation mask, and other implementation-defined execution-time attributes in effect when the at utility is executed shall be retained and used when the at-job is executed.

It won't then accurately represent what happens with cron.

[ Parent | Reply to this comment ]

Posted by Anonymous (205.172.xx.xx) on Thu 21 Feb 2013 at 16:12
644??? Those permissions give read-write to the user, read to the group and to the world. The script is not executable, unless you call it specifically using something on the order of '/bin/sh scriptname'. Wouldn't 744 be a better permission set?

[ Parent | Reply to this comment ]

Posted by Steve (90.193.xx.xx) on Thu 21 Feb 2013 at 16:19
[ Send Message | View Steve's Scratchpad | View Weblogs ]

This permissions are for files beneath /etc/cron.d/ - that directory is for cron-snippest and not executable scripts, unlike /etc/cron.daily/,etc.

If any of the snippets are group/world writeable they will not execute, even if mode root:root and permissions 664.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (205.172.xx.xx) on Mon 25 Feb 2013 at 15:35
Interesting! I was unaware of that difference between cron.d and cron.daily. Thank you for bringing me up to speed!

[ Parent | Reply to this comment ]

Posted by Anonymous (62.178.xx.xx) on Tue 26 Feb 2013 at 00:05
Regarding bad permissions, newer versions of cron (Wheezy and above) provide error messages that are more helpful than the ancient "WRONG INODE INFO". If there's something wrong, the daemon will tell you exactly what (eg: wrong user, other-writable, etc.)

[ Parent | Reply to this comment ]

Posted by Steve (90.193.xx.xx) on Wed 27 Feb 2013 at 07:18
[ Send Message | View Steve's Scratchpad | View Weblogs ]

That's good to know, thanks!

Steve

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

Which init system are you using in Debian?






( 1068 votes ~ 7 comments )