Allow Me To Save You Some Time: Some Useful Shortcuts
We aren’t machines, we are not supposed to keep everything inside of our heads; that means we often forget things, that’s why we need to better organize the things we need.
In this article, I’ll share my collection of shortcuts I gathered in past few years in order to save time, and now … yours! I hope you’ll find it useful.
Backup all MySQL Databases to individual files
Code:
fordb in $(mysql -e ‘show databases’ -s –skip-column-names); do mysqldump $db | gzip> “/backups/mysqldump-$(hostname)-$db-$(date +%Y-%m-%d-%H.%M.%S).gz”; done
Copy a MySQL Database to a new Server via ssh with one command
Code:
mysqldump –add-drop-table –extended-insert –force –log-error=error.log -uUSER -pPASS OLD_DB_NAME | ssh -C user@newhost “mysql -uUSER -pPASS NEW_DB_NAME”
Run complex remote shell cmd over ssh, without escaping quotes
Code:
ssh host -l user $(<cmd.txt)
Check site ssl certificate date
Code:
echo | openssls_client-connect Google 2>/dev/null |openssl x509 -dates -noout
Synchronize date and time with a server over ssh
Code:
date –set=”$(sshuser@server date)”
Duplicate installed packages from one machine to the other (RPM-based systems)
Code:
sshroot@remote.host “rpm -qa” | xargs yum -y install
Find SUID & SGID files
Code:
# find / ( -perm -4000 -o -perm -2000 ) -print
Find open ports
Code:
# netstat –listen
To display open ports and established TCP connections
Code:
# netstat -vatn
To display only open UDP ports try the following command
Code:
# netstat -vaun
# netstat -tulpn
If you want to see FQDN (full dns hostname), try removing the -n flag
Code:
$ netstat -vat
To display all open IPv4 network files in use by the process whose PID is 9255
Code:
# lsof -i 4 -a -p 9255
Lsof command examples
Code:
lsof -i :portNumber
lsof -itcp:portNumber
lsof -iudp:portNumber
lsof -i :80
lsof -i :80 | grep LISTEN
Sample outputs:
Code:
apache2 1607 root 3u IPv4 6472 0t0 TCP *:www (LISTEN)
apache2 1616 www-data 3u IPv4 6472 0t0 TCP *:www (LISTEN)
apache2 1617 www-data 3u IPv4 6472 0t0 TCP *:www (LISTEN)
apache2 1618 www-data 3u IPv4 6472 0t0 TCP *:www (LISTEN)
apache2 1619 www-data 3u IPv4 6472 0t0 TCP *:www (LISTEN)
apache2 1620 www-data 3u IPv4 6472 0t0 TCP *:www (LISTEN)
Find out the processes PID that opened TCPport **22**
Code:
# fuser22/tcp
Comparison of CPU utilization
The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the count and interval parameters. For example, to display the comparison of CPU utilization, 2 seconds apart and 5 times, use:
Code:
# sar -u 2 5
Output
Code:
Linux 2.6.18-53.el5 (slv0451i) 01/25/2011
09:55:02 AM CPU %user %nice %system %iowait %steal %idle
09:55:04 AM all 16.75 0.00 83.25 0.00 0.00 0.00
09:55:06 AM all 20.45 0.00 79.55 0.00 0.00 0.00
09:55:08 AM all 24.31 0.00 75.69 0.00 0.00 0.00
09:55:10 AM all 19.95 0.00 80.05 0.00 0.00 0.00
09:55:12 AM all 21.00 0.00 79.00 0.00 0.00 0.00
Average: all 20.49 0.00 79.51 0.00 0.00 0.00
Where
-u 12 5: Report CPU utilization. The following values are displayed:
o%user: Percentage of CPU utilization that occurred while executing at the user level (application).
o %nice: Percentage of CPU utilization that occurred while executing at the user level with nice priority.
o%system: Percentage of CPU utilization that occurred while executing at the system level (kernel).
o%iowait: Percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.
o%idle: Percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.
To get multiple samples and multiple reports, set an output file for the sar command. Run the sar command as a background process using the following code:
Code:
#sar -o output.file 12 8 >/dev/null 2>&1 &
You should alsouse the nohup command so that you can logout and check back report later on:
Code:
# nohupsar -o output.file 12 8 >/dev/null 2>&1 &
All the data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed by using the sar command with the -f option.
Code:
# sar -f data.file
Find CPU usage with mpstat (package sysstat)
Code:
# mpstat
Output
Code:
Linux 2.6.18-53.el5 (slv0451i) 01/25/2011
09:49:21 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
09:49:21 AM all 11.33 0.00 14.72 0.54 0.08 0.22 0.00 73.11 1184.08
The mpstat command display activities for each available processor; processor 0 being the first one. Global average activities among all processors are also reported.
The mpstat command can be used on both, SMP and UP machines, but in the latter, only global average activities will be printed:
Code:
# mpstat -P ALL
Output
Code:
Linux 2.6.18-53.el5 (slv0451i) 01/25/2011
09:52:14 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
09:52:14 AM all 11.33 0.00 14.73 0.54 0.08 0.22 0.00 73.10 1184.08
09:52:14 AM 0 13.15 0.00 14.71 0.65 0.12 0.31 0.00 71.06 1136.45
09:52:14 AM 1 9.50 0.00 14.74 0.43 0.05 0.14 0.00 75.11 47.63
Create an ISO image
Code:
# dd if=/dev/cdrom of=/file.iso
Clone
Code:
$ find . -path ./mnt -prune -o -path ./lost+found -prune -o -path ./sys -prune -o -path ./proc -prune -o -print |
cpio -pumd /destination &&mkdir /destination/mnt/ &&mkdir /destination/proc&&mkdir /destination/sys
Clone the current directory into /destination verboselyC
Code:
$ find . | cpio -pumdv /destination
Bash function to decompress archives
Code:
extract () {
if [ -f $1 ] ; then
case $1 in
*.tar.bz2) tar xvjf $1 ;;
*.tar.gz) tar xvzf $1 ;;
*.bz2) bunzip2 $1 ;;
*.rar) unrar x $1 ;;
*.gz) gunzip $1 ;;
*.tar) tar xvf $1 ;;
*.tbz2) tar xvjf $1 ;;
*.tgz) tar xvzf $1 ;;
*.zip) unzip $1 ;;
*.Z) uncompress $1 ;;
*.7z) 7z x $1 ;;
*) echo “‘$1’ cannot be extracted via >extract
esac
else
echo “‘$1’ is not a valid file”
fi
}
Monitor open TCP connections
Code:
$ watch -n 1 “netstat -tpanl | grep ESTABLISHED”
OR
$ lsof -i -T -n
Transfer files with netcat
-On client machine
Code:
$ nc -lpvv port > file
-On server machine
Code:
$ nc -vvclientip port < file
Example
Code:
Client: $ nc -l -p 6868 > file.txt
Server: $ nc 31.41.59.26 6868 < file.txt
Moving large directories
Code:
$ tar -C /usr -cf – home | tar -C /storage/export -xvf –
Or, to copy to a remote machine
Code:
$ tar -C /usr -cf – home | sshuser@somemachine tar -C /storage/export -xvf –
Get an ordered list of subdirectory sizes
Code:
$ du -sk ./* | sort -n | awk ‘BEGIN{ pref[1]=”K”; pref[2]=”M”; pref[3]=”G”;}
{ total = total + $1; x = $1; y = 1; while( x > 1024 ) { x = (x + 1023)/1024; y++; }
printf(“%g%st%sn”,int(x*10)/10,pref[y],$2); } END { y = 1; while( total > 1024 )
{ total = (total + 1023)/1024; y++; } printf(“Total: %g%sn”,int(total*10)/10,pref[y]); }’
List files opened by a PID
Code:
$ lsof -p 15857
Viewing Top Processes according to cpu, mem, swap size, etc…
Code:
$ pswwo pid,user,group,vsize:8,size:8,sz:6,rss:6,pmem:7,pcpu:7,time:7,wchan,sched=,stat,flags,comm,args k -vsz -A|sed -u ‘/^ *PID/d;10q’
List all files opened by a particular command
Code:
$ lsof -c dhcpd
Run a file system check on your next boot
Code:
$ touch /forcefsck
Show system installation date
Code:
$ tune2fs -l $(df -P / | tail -n1 | cut -d’ ‘ -f1 ) | grep ‘Filesystem created:
Turn off auto hard disc boot scanning for ext3
Code:
$ tune2fs -c -1 -i 0 /dev/VG0/data
Don’t spam root; log your cronjob output to syslog
Code:
$ */5 * * * * root /usr/local/nagios/sbin/nsca_check_disk 2>&1 |/usr/bin/logger -t nsca_check_disk
Create random numbers within range for conjob usage
Code:
$ H=$(until ([ $i -le 6 -a $i -gt 0 -o $i -le 23 -a $i -gt 21 ] ); do i=$(date +%N |
cut -c8-10); done ; echo $i) ; M=$(until [ $i -le 59 ]; do i=$(date +%N |
cut -c8-10); done ; echo $i) ; echo $M $H * * * backup-rsync-push.sh
List the CPU model name
Code:
$ grep ‘model|MHz’ /proc/cpuinfo |tail -n 2
Turn off all services in specific runlevel
Code:
$ fori in $(chkconfig –list | grep “4:on” | awk {‘print $1’}); do chkconfig –level 4 “$i” off; done
Find the device when you only know the mount point
Code:
$ df /media/mountpoint |egrep -o ‘^[/a-z0-9]*’
OR
$ df | grep -w ‘/media/armadillo’ | cut -d ” ” -f 1
Display any UDO/TCP connections by process name or by process id
Code:
$ lsof -nP -c COMMAND | egrep -o ‘(TCP|UDP).*$’ | sort -u
$ lsof -c apache2 | egrep -o ‘TCP.*ESTABLISHED.*$’
-nP flags are optional and UDP is irrelevant for established connections
Similar but using the process id
Code:
$ lsof -nP -p PID | egrep -o ‘(TCP|UDP).*$’
Detect your computer’s harddisk read speed without disk cache speed
Code:
$ cat /dev/sda | pv -r > /dev/null
Check the reserved block percentage of an Ext2/3 filesystem
You are probably aware that some percent of disk space on an ext2/ext3 file system is reserved for root (typically 5%). As documented elsewhere, this can be reduced to 1% with the following:
Code:
$ dumpe2fs -h /dev/sdX
$ tune2fs -m 1 /dev/sdX (where X = drive/partition, like /dev/sda1)
$ dumpe2fs -h /dev/sdX
Rsync directory tree including only files that match a certain find result
‘-mtime -10’ syncs only files newer than 10 days (-mtime is just one example, use whatever find expressions you need)
printf %P: File’s name with the name of the command line argument under which it was found removed.
This way, you can use any src directory; no need to cd into your src directory first.
Using in printf and a corresponding –from0 in rsync ensures that even filenames with newline characters work (thanks syssyphus for #3808).
Both, #1481 and #3808 just work if you either copy the current directory (.), or the filesystem root (/); otherwise, the output from find and the source dir from rsync just don’t match. #7685 works with an arbitrary source directory.
Code:
$ find /src/dir/ -mtime -10 -printf %P |rsync –files-from=- –from0 /src/dir/ /dst/dir/
Find the biggest file in current folder
Code:
$ ls -S|head -1
Find files that contain foo, but not bar
Code:
$ grep -l foo *cl*.log | xargsgrep -lL bar
Run local bash script on a remote server
Code:
$ ssh -T user@server< script.sh
Create a copy of the bootload and partition table
Code:
$ dd if=/dev/sda of=/home/sam/MBR.imagebs=512 count=1
Find files in multiple TAR files
Code:
$ find . -type f -name “*.tar” -printf [%f]n -exec tar -tf {} ; | grep -iE “[[]|”
Create a tar of directory structure only
Code:
$ tar -cf ~/out.tar –no-recursion –files-from
Backup a file with a date-time stamp
Code:
$ buf() { cp -v $1 ${1/${1%%.*}/$f-$(date +”%Y%m%d_%H%M%S”)};}
Use them wisely. There will be more to come!