Skip to content

Instantly share code, notes, and snippets.

@aruruka
Last active June 30, 2021 17:59
Show Gist options
  • Save aruruka/cc11797ff4e747353476c54b2bd8152f to your computer and use it in GitHub Desktop.
Save aruruka/cc11797ff4e747353476c54b2bd8152f to your computer and use it in GitHub Desktop.
Delete something to create some disk space for Squadron hosts.
#!/usr/bin/env bash
# Purpose: Delete something to create some disk space for Squadron hosts.
# Prerequisite:
# Must run on c3669-node1.coelab.cloudera.com as root.
# Before running this script, make sure to stop all services including Cloudera Management Services,
# but only leave Zookeeper and HDFS running, because we'll use hdfs command to delete something on HDFS.
# This shell script runs Python script - "clean_up_squadron_host_disk.py", on every Squadron CDP cluster host.
# Author: ShounenG
# Usage:
# cd /root && \
# curl -L "https://gist.githubusercontent.com/aruruka/cc11797ff4e747353476c54b2bd8152f/raw/d268351af75dc5bd115b746687d9847da4ea5e96/make_disk_space_squadron_hosts.sh" -O && \
# mkdir -pv /root/mytools/python && \
# cd $_ && \
# curl -L "https://gist.githubusercontent.com/aruruka/3e9be112bcfb5d458a83263981fb9ca1/raw/330f802ee8b9c8ed47941e5076bea66547720ea0/clean_up_squadron_host_disk.py" -O &&\
# cd /root && \
# sh ./make_disk_space_squadron_hosts.sh
# --------------------------------------
set -o errexit
set -o nounset
set -o pipefail
REMOTE_SERVERS=(
c3669-node1.coelab.cloudera.com
c3669-node2.coelab.cloudera.com
c3669-node3.coelab.cloudera.com
c3669-node4.coelab.cloudera.com
)
for SERVER in ${REMOTE_SERVERS[@]} ; do
ssh \
-o StrictHostKeyChecking=no \
root@${SERVER} \
<< EOF 2>/dev/null
echo -e "Executing commands on ${SERVER}:\n";
echo "mkdir -pv /root/mytools/python";
mkdir -pv /root/mytools/python;
echo -e "\nCommands execution on ${SERVER} is finished.\n";
EOF
done
for SERVER in ${REMOTE_SERVERS[@]} ; do
echo -e "scp -o StrictHostKeyChecking=no /root/mytools/python/clean_up_squadron_host_disk.py root@${SERVER}:/root/mytools/python"
scp -o StrictHostKeyChecking=no /root/mytools/python/clean_up_squadron_host_disk.py root@${SERVER}:/root/mytools/python 2>/dev/null
done
for SERVER in ${REMOTE_SERVERS[@]} ; do
echo -e "Executing commands on ${SERVER}:\n";
echo "python /root/mytools/python/clean_up_squadron_host_disk.py"
ssh \
-o StrictHostKeyChecking=no \
root@${SERVER} \
<< EOF 2>/dev/null
python /root/mytools/python/clean_up_squadron_host_disk.py
EOF
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment