Bash
October 09, 2019https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash
http://goo.gl/ZThKCj - printing strings help
http://goo.gl/HnPkiq - arithmetic operator
http://goo.gl/g6xtca - arrays
http://goo.gl/xRHo3u - parameter expansion
https://github.com/aperezdc/perezdecastro.org/blob/master/stash/using-boolean-variables-in-bash.markdown - flag variables in bourne shell
https://kvz.io/blog/2013/11/21/bash-best-practices/ - boiler plate and best practice guide
https://explainshell.com/explain - amazing visual explanation
Bash 4
http://clubmate.fi/upgrade-to-bash-4-in-mac-os-x/
Post upgrade to use bash 4 in script. change shebang line
#!/usr/local/bin/bash
Configure terminal to use it
# Add the new shell to the list of allowed shells
sudo bash -c 'echo /usr/local/bin/bash >> /etc/shells'
# Change to the new shell
chsh -s /usr/local/bin/bash
sed
appending newlines with mac: https://stackoverflow.com/questions/16576197/how-to-add-new-line-using-sed-mac
echo foo | sed 's/f/f\'$'\n/'
tokens=$(echo "$tokens" | sed ' /^[[:space:]]*}/a \
\
')
tokens=$(echo "${tokens}" | sed ' /^\s*$/d;s/},/}/;s/}/}\'$'\n\\\n/; ')
https://lists.freebsd.org/pipermail/freebsd-questions/2009-March/194500.html - example to safe replace with newlines
a="This
is
the
input
from the
web server"
b=`echo $a`
sed "s/foo/$b/"
https://taoofmac.com/space/cli/sed - examples
https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed - more whitespacing issues
awk
echo "${tokens}" | awk 'BEGIN { RS=""; FS=/,/; }; $0 ~ "jane@github.com" { print }'
escaping quotes
wrap with single quote and expand variable
grep '^'"${APP_NAME}"''
args and help
http://spf13.com/post/easy-bash-scripting-with-shflags/ - documentation module
shflags
https://github.com/kward/shflags
Install
-
Download
-
link to path
ln ~/data/tools/shflags-1.2.2/shflags /usr/local/bin/shflags -
execute
lint/static analysis
https://github.com/koalaman/shellcheck
brew install shellcheck
execute
shellcheck script.sh
list directory structure
ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'
outputs
|-jobs
|---PCF-Upgrade
|-----jobs
|-------Bulk-Build-Pipelines
|---------jobs
count directories
ls -R | grep ":$" | wc -l
show structure
ls -la | xargs ls -la \{\}
ls -lrt -d -1 $PWD/* | xargs ls -la \{\}
find by dir name
find . -type d -name builds -exec rm -rf {} \;
find string in directory
grep -rnw /var/lib/jenkins -e 'utils/util'
delete fast
https://www.kinamo.be/en/support/faq/efficiently-remove-a-zillion-files-on-linux-servers
rsync -a --delete empty_dir/ dir_to_delete
move vs. copy vs. delete
cp -> slow
mv -> faster
delete -> fast as mv
mv (rename in place) -> fastest
cli inputs / help
https://github.com/kward/shflags/wiki/Documentation12x
styleguide
https://google.github.io/styleguide/shell.xml#Builtin_Commands_vs._External_Commands
https://www.chromium.org/chromium-os/shell-style-guidelines#TOC-Default-Assignments
colors
https://misc.flogisoft.com/bash/tip_colors_and_formatting
use python to get json value
DOMAIN_NAME=$(curl -s http://localhost:4040/api/tunnels/jenkins | python -c 'import sys, json; print json.load(sys.stdin)["public_url"]' | sed 's/http:\/\/\(.*\)/\1/')
colors
https://misc.flogisoft.com/bash/tip_colors_and_formatting
rerun with sudo
sudo !!
Generating a Sequence for Output
I needed to execute several scripts in terminal tabs, this made that easier.
#!/usr/bin/env bash
set -o nounset
set -o errexit
set -o pipefail
DEBUG="${DEBUG:-false}"
[[ ${DEBUG} == true ]] && set -o xtrace
start_num=50000
end_num=90000
increment=10000
for ((i=start_num; i<=end_num; i+=increment)); do
finish=$((i+increment))
# echo "start: $i, finish ${finish}"
echo "time bundle exec rails 'one_time:preprocess_active_storage_for_users_and_companies[${i},${finish},100]'"
done