Re-nice everyone

Sometimes when someone wants to get past a throttle of some sort they place the niceness of a process in the negative stage. The higher the negative number the higher the more priority.. This is a burden on servers that meant to have shared allocated resources.

Well, the solution. If this is a VPS and they all of their processes are set to -15 but the default is 0, issue this command:

for i in `cat /etc/passwd | awk -F: {'print $1'}`; do renice -n 0 -u $i; done

VirtFS

Do you see a bunch of /home/virtfs mounts? Well there is a way to clear this from command line:


/scripts/clear_orphaned_virtfs_mounts --clearall

OpenVZ simfs to ploop

There are many advantages to using ploop rather than simfs. The advantage of course I choose is file security. While simfs places all the files directly on the hostname for ease of use and managing, ploop creates a virtual disk file and then put files inside this disk. When the virtual machine is started vzctl has to mount the ploop disk. Versus with simfs, there is no mount involved.

vzctl convert  [--layout ploop[:mode]] 
vzctl convert --layout ploop <CT_ID>

MAKE A BACKUP FIRST

Cerbot Free SSL

Certbot has finally given instructions for majority of the Linux distros with various web-servers. You can find the all the instructions at this magical link: 

https://certbot.eff.org/all-instructions  

Here is an example of installing Certbot on a NagiosXI server:

[root@nagios ~]# wget https://dl.eff.org/certbot-auto
--2019-04-23 16:32:45-- https://dl.eff.org/certbot-auto
Resolving dl.eff.org… 151.101.0.201, 151.101.64.201, 151.101.128.201, …
Connecting to dl.eff.org|151.101.0.201|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 63564 (62K) [application/octet-stream]
Saving to: `certbot-auto'
100%[================================================================================>] 63,564 --.-K/s in 0.003s
2019-04-23 16:32:45 (17.8 MB/s) - `certbot-auto' saved [63564/63564]
[root@nagios ~]# sudo mv certbot-auto /usr/local/bin/certbot-auto
[root@nagios ~]# sudo chown root /usr/local/bin/certbot-auto
[root@nagios ~]# sudo chmod 0755 /usr/local/bin/certbot-auto
[root@nagios ~]# cert
certbot-auto certutil
[
root@nagios ~]# certbot-auto
Bootstrapping dependencies for RedHat-based OSes that will use Python3… (you can skip this with --no-bootstrap)
yum is /usr/bin/yum
yum is hashed (/usr/bin/yum)
Loaded plugins: fastestmirror
Setting up Install Process
Loading mirror speeds from cached hostfile
base: mirror.steadfastnet.com
epel: fedora-epel.mirror.lstn.net
extras: mirror.jaleco.com
updates: mirror.hackingand.coffee
Package gcc-4.4.7-23.el6.x86_64 already installed and latest version
Package openssl-1.0.1e-57.el6.x86_64 already installed and latest version
Package openssl-devel-1.0.1e-57.el6.x86_64 already installed and latest version
Package redhat-rpm-config-9.0.3-51.el6.centos.noarch already installed and latest version
Package ca-certificates-2018.2.22-65.1.el6.noarch already installed and latest version
Package 1:mod_ssl-2.2.15-69.el6.centos.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package augeas-libs.x86_64 0:1.0.0-10.el6 will be installed
---> Package libffi-devel.x86_64 0:3.0.5-3.2.el6 will be installed
---> Package python34.x86_64 0:3.4.8-1.el6 will be installed
--> Processing Dependency: python34-libs(x86-64) = 3.4.8-1.el6 for package: python34-3.4.8-1.el6.x86_64
--> Processing Dependency: libpython3.4m.so.1.0()(64bit) for package: python34-3.4.8-1.el6.x86_64
---> Package python34-devel.x86_64 0:3.4.8-1.el6 will be installed
--> Processing Dependency: python-rpm-macros for package: python34-devel-3.4.8-1.el6.x86_64
--> Processing Dependency: python3-rpm-macros for package: python34-devel-3.4.8-1.el6.x86_64
---> Package python34-tools.x86_64 0:3.4.8-1.el6 will be installed
--> Processing Dependency: python34-tkinter = 3.4.8-1.el6 for package: python34-tools-3.4.8-1.el6.x86_64
--> Running transaction check
---> Package python-rpm-macros.noarch 0:3-14.el6 will be installed
--> Processing Dependency: python-srpm-macros for package: python-rpm-macros-3-14.el6.noarch
---> Package python3-rpm-macros.noarch 0:3-14.el6 will be installed
---> Package python34-libs.x86_64 0:3.4.8-1.el6 will be installed
---> Package python34-tkinter.x86_64 0:3.4.8-1.el6 will be installed
--> Processing Dependency: libtcl8.5.so()(64bit) for package: python34-tkinter-3.4.8-1.el6.x86_64
--> Processing Dependency: libtk8.5.so()(64bit) for package: python34-tkinter-3.4.8-1.el6.x86_64
--> Running transaction check
---> Package python-srpm-macros.noarch 0:3-14.el6 will be installed
---> Package tcl.x86_64 1:8.5.7-6.el6 will be installed
---> Package tk.x86_64 1:8.5.7-5.el6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================================================================================
Package Arch Version Repository Size
Installing:
augeas-libs x86_64 1.0.0-10.el6 base 314 k
libffi-devel x86_64 3.0.5-3.2.el6 base 18 k
python34 x86_64 3.4.8-1.el6 epel 50 k
python34-devel x86_64 3.4.8-1.el6 epel 186 k
python34-tools x86_64 3.4.8-1.el6 epel 426 k
Installing for dependencies:
python-rpm-macros noarch 3-14.el6 epel 6.6 k
python-srpm-macros noarch 3-14.el6 epel 5.8 k
python3-rpm-macros noarch 3-14.el6 epel 5.4 k
python34-libs x86_64 3.4.8-1.el6 epel 8.4 M
python34-tkinter x86_64 3.4.8-1.el6 epel 336 k
tcl x86_64 1:8.5.7-6.el6 base 1.9 M
tk x86_64 1:8.5.7-5.el6 base 1.4 M
Transaction Summary
Install 12 Package(s)
Total download size: 13 M
Installed size: 41 M
Is this ok [y/N]: y
Downloading Packages:
(1/12): augeas-libs-1.0.0-10.el6.x86_64.rpm | 314 kB 00:00
(2/12): libffi-devel-3.0.5-3.2.el6.x86_64.rpm | 18 kB 00:00
(3/12): python-rpm-macros-3-14.el6.noarch.rpm | 6.6 kB 00:00
(4/12): python-srpm-macros-3-14.el6.noarch.rpm | 5.8 kB 00:00
(5/12): python3-rpm-macros-3-14.el6.noarch.rpm | 5.4 kB 00:00
(6/12): python34-3.4.8-1.el6.x86_64.rpm | 50 kB 00:00
(7/12): python34-devel-3.4.8-1.el6.x86_64.rpm | 186 kB 00:00
(8/12): python34-libs-3.4.8-1.el6.x86_64.rpm | 8.4 MB 00:00
(9/12): python34-tkinter-3.4.8-1.el6.x86_64.rpm | 336 kB 00:00
(10/12): python34-tools-3.4.8-1.el6.x86_64.rpm | 426 kB 00:00
(11/12): tcl-8.5.7-6.el6.x86_64.rpm | 1.9 MB 00:00
(12/12): tk-8.5.7-5.el6.x86_64.rpm | 1.4 MB 00:00
Total 11 MB/s | 13 MB 00:01
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : python34-libs-3.4.8-1.el6.x86_64 1/12
Installing : python34-3.4.8-1.el6.x86_64 2/12
Installing : 1:tcl-8.5.7-6.el6.x86_64 3/12
Installing : 1:tk-8.5.7-5.el6.x86_64 4/12
Installing : python34-tkinter-3.4.8-1.el6.x86_64 5/12
Installing : python-srpm-macros-3-14.el6.noarch 6/12
Installing : python-rpm-macros-3-14.el6.noarch 7/12
Installing : python3-rpm-macros-3-14.el6.noarch 8/12
Installing : python34-devel-3.4.8-1.el6.x86_64 9/12
Installing : python34-tools-3.4.8-1.el6.x86_64 10/12
Installing : augeas-libs-1.0.0-10.el6.x86_64 11/12
Installing : libffi-devel-3.0.5-3.2.el6.x86_64 12/12
Verifying : python-rpm-macros-3-14.el6.noarch 1/12
Verifying : 1:tcl-8.5.7-6.el6.x86_64 2/12
Verifying : python34-tkinter-3.4.8-1.el6.x86_64 3/12
Verifying : python34-3.4.8-1.el6.x86_64 4/12
Verifying : python3-rpm-macros-3-14.el6.noarch 5/12
Verifying : python34-libs-3.4.8-1.el6.x86_64 6/12
Verifying : libffi-devel-3.0.5-3.2.el6.x86_64 7/12
Verifying : python-srpm-macros-3-14.el6.noarch 8/12
Verifying : augeas-libs-1.0.0-10.el6.x86_64 9/12
Verifying : 1:tk-8.5.7-5.el6.x86_64 10/12
Verifying : python34-devel-3.4.8-1.el6.x86_64 11/12
Verifying : python34-tools-3.4.8-1.el6.x86_64 12/12
Installed:
augeas-libs.x86_64 0:1.0.0-10.el6 libffi-devel.x86_64 0:3.0.5-3.2.el6 python34.x86_64 0:3.4.8-1.el6
python34-devel.x86_64 0:3.4.8-1.el6 python34-tools.x86_64 0:3.4.8-1.el6
Dependency Installed:
python-rpm-macros.noarch 0:3-14.el6 python-srpm-macros.noarch 0:3-14.el6 python3-rpm-macros.noarch 0:3-14.el6
python34-libs.x86_64 0:3.4.8-1.el6 python34-tkinter.x86_64 0:3.4.8-1.el6 tcl.x86_64 1:8.5.7-6.el6
tk.x86_64 1:8.5.7-5.el6
Complete!
Creating virtual environment…
Installing Python packages…
Installation succeeded.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Enter email address (used for urgent renewal and security notices) (Enter 'c' to
cancel):


Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory

(A)gree/(C)ancel: A

Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.

(Y)es/(N)o: Y
No names were found in your configuration files. Please enter in your domain
name(s) (comma and/or space separated) (Enter 'c' to cancel): nagios.xxx.com
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for nagios.xxx.com
Cleaning up challenges
Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80.
IMPORTANT NOTES:
Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
-------------------------
So at this point I needed to add a <VirtualHost> entry in the /etc/apache/conf.d/nagios.conf to get this to work.
----------------------------

$certbot-auto
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Which names would you like to activate HTTPS for?

1: nagios.xxx.com

Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): Obtaining a new certificate
Performing the following challenges:
http-01 challenge for nagios.xxx.com
Waiting for verification…
Cleaning up challenges
Created an SSL vhost at /etc/httpd/conf.d/nagios-le-ssl.conf
Deploying Certificate to VirtualHost /etc/httpd/conf.d/nagios-le-ssl.conf
Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.

1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.

Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 1

Congratulations! You have successfully enabled https://nagios.xxx.com
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=nagios.xxx.com

IMPORTANT NOTES:
Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/nagios.xxx.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/nagios.xxx.com/privkey.pem
Your cert will expire on 2019-07-22. To obtain a new or tweaked
version of this certificate in the future, simply run certbot-auto
again with the "certonly" option. To non-interactively renew all
of your certificates, run "certbot-auto renew"
If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
[root@nagios conf.d]# service httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]
[root@nagios conf.d]#


Apache Won’t Start

  • Clear out your active semaphores

Semaphores? What the heck is a semaphore? Well, it’s actually an apparatus for conveying information by means of visual signals. But, when it comes to programming, semaphores are used for communicating between the active processes of a certain application. In the case of Apache, they’re used to communicate between the parent and child processes. If Apache can’t write these things down, then it can’t communicate properly with all of the processes it starts.

ipcs -s

If you see a list of semaphores, Apache has not cleaned up after itself, and some semaphores are stuck. Clear them out with this command:

for i in ipcs -s | awk '/httpd/ {print $2}'; do (ipcrm -s $i); done

Now, in almost all cases, Apache should start properly. If it doesn’t, you may just be completely out of available semaphores. You may want to increase your available semaphores, and you’ll need to tickle your kernel to do so. Add this to /etc/sysctl.conf:

kernel.msgmni = 1024
kernel.sem = 250 256000 32 1024
And then run sysctl -p to pick up the new changes.

Cool Bash One Liners

See more cool one liners at http://www.bashoneliners.com

Check if a text snippet is valid C code
$ gcc -fsyntax-only -xc – <<< “text snippet”
Feb. 10, 2019, 8:12 a.m. — Janos

Inspect the HTTP headers of a website
$ curl -I amazon.com
Feb. 8, 2019, 10:56 p.m. — Janos

Search man pages and present a PDF
$ man -k . | awk ‘{ print $1 ” ” $2 }’ | dmenu -i -p man | awk ‘{ print $2 ” ” $1 }’ | tr -d ‘()’ | xargs man -t | ps2pdf – – | zathura –
Dec. 18, 2018, 11:31 a.m. — Jab2870


Find all log files modified 24 hours ago, and zip them
$ find . -type f -mtime +1 -name “*.log” -exec zip -m {}.zip {} \; >/dev/null
Nov. 9, 2018, 10:04 a.m. — TrongTan124


List IP addresses connected to your server on port 80
$ netstat -tn 2>/dev/null | grep :80 | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -nr | head
Sept. 26, 2018, 11:10 p.m. — Goeks1


Random 6-digit number
$ python -c ‘import random; print(random.randint(0,1000000-1))’
Sept. 19, 2018, 10:42 p.m. — johntellsall


Very fast history search with Ripgrep
$ rh() { rg “$1” ~/.bash_history }
Sept. 18, 2018, 7 p.m. — johntellsall


While loop to pretty print system load (1, 5 & 15 minutes)
$ while :; do date; awk ‘{printf “1 minute load: %.2f\n”, $1; printf “5 minute load: %.2f\n”, $2; printf “15 minute load: %.2f\n”, $3}’ /proc/loadavg; sleep 3; done
Sept. 5, 2018, 8:41 p.m. — Janos


Scan entire Git repo for dangerous Amazon Web Service IDs
$ git grep -Ew ‘[A-Z0-9]{20}’
Sept. 5, 2018, 8:30 p.m. — Janos


Scan entire Git repos for dangerous Amazon Web Service IDs
$ git ls-tree –full-tree -r –name-only HEAD | xargs egrep -w ‘[A-Z0-9]{20}’
Aug. 31, 2018, 10:29 p.m. — johntellsall


While loop to pretty print system load (1, 5 & 15 minutes)
$ while [ 1 == 1 ]; do cat /proc/loadavg | awk ‘{printf “1 minute load: %.2f\n”, $(NF-5)}’ && cat /proc/loadavg |awk ‘{printf “5 minute load: %.2f\n”, $(NF-3)}’ && cat /proc/loadavg |awk ‘{printf “15 minute load: %.2f\n”, $(NF-2)}’; sleep 3; date; done
Aug. 30, 2018, 8:54 a.m. — peek2much3


Dump all AWS IAM users/roles to a Terraform file for editing / reusing in another environment
$ echo iamg iamgm iamgp iamip iamp iampa iamr iamrp iamu iamup | AWS_PROFILE=myprofile xargs -n1 terraforming
Aug. 28, 2018, 12:38 a.m. — johntellsall


Organise image by portrait and landscape
$ mkdir “portraits”; mkdir “landscapes”; for f in ./*.jpg; do WIDTH=$(identify -format “%w” “$f”)> /dev/null; HEIGHT=$(identify -format “%h” “$f”)> /dev/null; if [[ “$HEIGHT” > “$WIDTH” ]]; then mv “$f” portraits/ ; else mv “$f” landscapes/ ; fi; done
Aug. 23, 2018, 2:09 p.m. — Jab2870


Create a txt files with 10000 rows
$ for FILE in .full ; do split -l 100000 $FILE; mv -f xaa echo "$FILE" | cut -d'.' -f1.txt; rm -f x; done
Aug. 22, 2018, 2:02 p.m. — Kifli88


List open processes ordered by it’s number of open files
$ ps -ef |awk ‘{ print $2 }’ \ |tail -n +2 \ |while read pid; do echo “$pid $(lsof -p $pid |wc -l)”; done \ |sort -r -n -k 2 \ |while read pid count; do echo “$pid $count $(ps -o command= -p $pid)”; done
Aug. 22, 2018, 1:21 p.m. — cddr


Remove all container from an specific network (docker)
$ docker ps -a -f network=$NETWORK –format='{{.ID}}’ | xargs docker rm -f
Aug. 17, 2018, 4:38 p.m. — gatero


Up all docker services as detached mode over all immediate subdirectories
$ for dir in $(ls -d */); do eval $(cd $PWD/$dir && docker-compose up -d && cd ..); done;
Aug. 17, 2018, 4:31 p.m. — gatero


Find and replace string inside specific files
$ grep -ril ‘$SEARCH_PATTERN’ src | sed -i ‘s/$FIND_PATTERN/$REPLACE_PATTERN/g’
Aug. 17, 2018, 4:18 p.m. — gatero


Puppet/Bash: test compare json objects.
$ unless => “client_remote=\”$(curl localhost:9200/_cluster/settings | python -c \”import json,sys;obj=json.load(sys.stdin);print(obj[‘persistent’][‘search’][‘remote’])\”)\”; new_remote=\”$( echo $persistent_json | python -c \”import json,sys;obj=json.load(sys.stdin);print(obj[‘persistent’][‘search’][‘remote’])\”)\”; [ \”$client_remote\” = \”$new_remote\” ]”,
July 27, 2018, 8:37 p.m. — cjedwa


Print wifi access points sorted by signal
$ iw dev IFACE scan | egrep “SSID|signal” | awk -F “:” ‘{print $2}’ | sed ‘N;s/\n/:/’ | sort
June 16, 2018, 5:37 a.m. — kazatca


Kill a process running on port 8080
$ lsof -i :8080 | awk ‘{l=$2} END {print l}’ | xargs kill
June 15, 2018, 4:18 a.m. — jamestomasino


Take values from a list (file) and search them on another file
$ for ITEM in cat values_to_search.txt; do (egrep $ITEM full_values_list.txt && echo $ITEM found) | grep “found” >> exit_FOUND.txt; done
May 16, 2018, 3:20 p.m. — ManuViorel


Delete all untagged Docker images
$ docker rmi $(docker images -f “dangling=true” -q)
April 27, 2018, 2:50 p.m. — stefanobaghino


Have script run itself in a virtual terminal
$ tty >/dev/null || { urxvt -e /bin/sh -c “tty >/tmp/proc$$; while test x; do sleep 1; done” & while test ! -f /tmp/proc$$; do sleep .1; done; FN=$(cat /tmp/proc$$); rm /tmp/proc$$; exec >$FN 2>$FN <$FN; }