Want to check if a bunch of servers on your network are up every few minutes? Want to use things that you already have? Do it with a shell script and cron!
Here we go over some stuff, in a rather disorganised way, and in way to much detail on particular things, the contents of this script:
#!/bin/sh
# Ping a server list and then send an email reporting all
# unreachable servers. Keep the file around for /etc/motd
# Setup your date format and email client here
alias d='date'
alias email='ssmtp -t'
# Setup your paths here
SVR_LIST=/etc/hosts
WD=/opt/pingcheck
STATUS=$WD/server-status
EMAIL=$WD/mail-header
works='\\tresolving {}:\\tokay'
fail='\\tresolving {}:\\tfail'
awk '/^[0-9]/ {print $2}' $SVR_LIST |\
xargs -n1 -P5 -I {} \
sh -c "ping -c 1 {} > /dev/null &&\
echo -e `d` $works ||\
echo -e `d` $fail" > $STATUS
grep fail $STATUS > /dev/null &&\
(cat $EMAIL <(grep fail $STATUS) | email)
hosts is a list of IPs and names. If your hosts file represents all
the servers that you want to report in, it works to use it. Otherwise,
it is a good, simple format.
cat << EOF
# comment
127.0.0.1 domain.com
127.0.0.1 mydomain.com
127.0.0.1 sub.mydomain.com
#127.0.0.1 commented.domain.com
127.0.0.1 gov.site.gov.au
127.0.0.1 host.localdomain host
EOFUsing awk cause it alows us to print fields from lines if they match
regex. The alternative would be grep regex file | tr | tr | cut -f
2. Not sure if this would be better.
echo "*** doesn't start with #"
awk '!/^#/' org-hosts
echo -e "\n*** starts with a hex number, or ::1"
awk '/^[0-9a-f:]/' org-hosts*** doesn't start with # 127.0.0.1 domain.com 127.0.0.1 mydomain.com 127.0.0.1 sub.mydomain.com 127.0.0.1 gov.site.gov.au 127.0.0.1 host.localdomain host *** starts with a hex number 127.0.0.1 domain.com 127.0.0.1 mydomain.com 127.0.0.1 sub.mydomain.com 127.0.0.1 gov.site.gov.au 127.0.0.1 host.localdomain host
- Ads redirected to 127.0.0.1
- http://someonewhocares.org/hosts/
- Ads redirected to 0.0.0.0
- http://someonewhocares.org/hosts/zero/
Assuming that your hosts file is the correct list, you can awk it
to get a list of hosts, and then ping each one using xargs. xargs
runs a command on input, and can be used to do things in parrallel.
I’m launching a new shell with xargs so that I can use and (and)
and or (||). In order to run a command when one is successful, we
use &&, so in a && b && c, c only runs if b is successfull, which
only runs if a is successful. On the other hand, || causes the
following command to be run only when the preceeding command fails.
a && b || c Is basically if a; then b; else c; fi.
The xargs option -I {} says that the input into xargs will be
placed in the command where {} is, -P1 specifies 1 process. Change
the number to launch more than one at once.
The \ character specified that the command continues on the next
line, and must be the last character before the newline.
We then put the output of xargs into results-out. I’ve then read
it with cat so that you can see the output.
works='\\tresolving {}:\\tokay'
fail='\\tresolving {}:\\tfail'
awk '/^[0-9]/ {print $2}' org-hosts |\
xargs -n1 -P1 -I {} \
sh -c "ping -c 1 {} > /dev/null &&\
echo -e `date` $works ||\
echo -e `date` $fail" > results-out
cat results-outSat Feb 8 21:37:58 AEDT 2020 resolving domain.com: okay Sat Feb 8 21:37:58 AEDT 2020 resolving mydomain.com: okay Sat Feb 8 21:37:58 AEDT 2020 resolving sub.mydomain.com: fail Sat Feb 8 21:37:58 AEDT 2020 resolving gov.site.gov.au: fail Sat Feb 8 21:37:58 AEDT 2020 resolving host.localdomain: fail
You probably want it to be reported, if something has
failed. ssmtp is one option for sending mail, if you can get it
up in the environment.
cat > mail-header <<EOF
To: [email protected]
Cc: [email protected]
From: $HOSTNAME
Subject: It's borked.
Hey buddy, it's borked.
Here is the borked stuff:
EOF
grep fail results-out > /dev/null &&\
(cat mail-header <(grep fail results-out) | ssmtp -t)To: [email protected] Cc: [email protected] From: HOSTNAME Subject: It's borked. Hey buddy, it's borked. Here is the borked stuff: Sat Feb 8 20:45:17 AEDT 2020 resolving gov.site.gov.au: fail Sat Feb 8 20:45:17 AEDT 2020 resolving sub.mydomain.com: fail Sat Feb 8 20:45:17 AEDT 2020 resolving host.localdomain: fail
I don’t recommend actually creating that email header file every time
you run the script. The other thing this does is concatinate the
mail-header file with the output of grep fail results-out, so it is
only sending an email containing anything that ping failed on.
The advantage of putting the results in a file is that you can then
report it in multiple ways. For instance, you can have your
/etc/motd file look something like this:
figlet -ck $HOSTNAME
echo `tput bold` These servers are borked: `tput sgr0`
grep fail results-out _ _ ___ ____ _____ _ _ _ __ __ _____
| | | | / _ \ / ___||_ _|| \ | | / \ | \/ || ____|
| |_| || | | |\___ \ | | | \| | / _ \ | |\/| || _|
| _ || |_| | ___) | | | | |\ | / ___ \ | | | || |___
|_| |_| \___/ |____/ |_| |_| \_|/_/ \_\|_| |_||_____|
These servers are borked:
Sat Feb 8 21:37:58 AEDT 2020 resolving sub.mydomain.com: fail
Sat Feb 8 21:37:58 AEDT 2020 resolving gov.site.gov.au: fail
Sat Feb 8 21:37:58 AEDT 2020 resolving host.localdomain: fail
As long as results-out can be read by the user when they log in
(maybe put it somewhere in /var or /opt,) assuming your system is setup
to use an motd on login, they will be greated by a list of all the
broken things when they ssh in.
Now, there are two ways to do this, I’ve used awk because if I was
to use grep, I would have to use cut or something to get just the
hostname, and then using cut I had to do a bunch of stuff to make sure
my /etc/hosts, which I got from someonewhocares, was formated the
same so cut actually worked, which meant piping it through tr.
for x in $(seq 10)
do
{ time ( for x in $(seq 1000)
do
awk '/^[0-9a-f:]/ { print $2 }' /etc/hosts > /dev/null
done )
} 2>&1 | grep real
donereal 0m22.339s real 0m21.570s real 0m21.494s real 0m21.677s real 0m21.635s real 0m21.733s real 0m21.758s real 0m21.795s real 0m21.809s real 0m21.699s
for i in $(seq 10)
do
{ time ( for x in $(seq 1000)
do
grep '^[0-9a-f:]' /etc/hosts |\
tr -s ' \t' |\
tr '\t' ' ' |\
cut -f 2 -d ' ' > /dev/null
done )
} 2>&1 | grep real
donereal 0m19.194s real 0m19.175s real 0m19.276s real 0m19.710s real 0m19.915s real 0m20.038s real 0m20.224s real 0m20.344s real 0m20.288s real 0m20.399s
After finishing, I realised I didn’t time sed.
for i in $(seq 10)
do
{ time ( for x in $(seq 1000)
do
sed -e 's/#.\+//' \
-e 's/[0-9a-f\.:]\+[ \t]\+//' \
-e 's/[ \t].\+//' /etc/hosts > /dev/null
done )
} 2>&1 | grep real
donereal 0m27.520s real 0m27.552s real 0m28.276s real 0m27.814s real 0m28.321s real 0m28.517s real 0m28.282s real 0m27.662s real 0m27.735s real 0m27.486s
In this one I get rid of comments and newlines with grep and then
everything but the hostname with sed
for i in $(seq 10)
do
{ time ( for x in $(seq 1000)
do
grep '^[0-9a-f:]' /etc/hosts |\
sed -e 's/[0-9a-f\.:]\+[ \t]\+//' \
-e 's/[ \t].\+//' > /dev/null
done )
} 2>&1 | grep real
donereal 0m28.837s real 0m28.857s real 0m29.033s real 0m28.841s real 0m29.072s real 0m29.234s real 0m29.310s real 0m30.214s real 0m29.295s real 0m29.701s
sed is rather slow, so i’ve redone it and awk.
for i in $(seq 10)
do
{ time ( for x in $(seq 10)
do
grep '^[0-9a-f:]' /etc/hosts |\
sed -e 's/[0-9a-f\.:]\+[ \t]\+//' \
-e 's/[ \t].\+//' > /dev/null
done )
} 2>&1 | grep real
donereal 0m0.561s real 0m0.567s real 0m0.594s real 0m0.565s real 0m0.526s real 0m0.582s real 0m0.612s real 0m0.526s real 0m0.562s real 0m0.567s
for i in $(seq 10)
do
{ time ( for x in $(seq 10)
do
grep '^[0-9a-f:]' /etc/hosts |\
sed -e 's/^[0-9a-f\.:]\+[ \t]\+\([^ \t]\+\).*/\1/' > /dev/null
done )
} 2>&1 | grep real
donereal 0m3.057s real 0m3.143s real 0m3.217s real 0m3.337s real 0m3.174s real 0m3.299s real 0m3.474s real 0m3.299s real 0m3.325s real 0m3.122s
for x in $(seq 10)
do
{ time ( for x in $(seq 10)
do
awk '/^[0-9a-f:]/ { print $2 }' /etc/hosts > /dev/null
done )
} 2>&1 | grep real
donereal 0m0.365s real 0m0.457s real 0m0.401s real 0m0.396s real 0m0.446s real 0m0.438s real 0m0.399s real 0m0.396s real 0m0.378s real 0m0.434s