Notas de Luis

Tabla de Contenidos

Commandline - Fu

Referencia: Commandlinefu

Random

Pick a random line from a file

head -$(($RANDOM % $(wc -l < file.txt) +1 )) file.txt | tail -1

Generates random texts

tr -dc a-z1-4 </dev/urandom|tr 1-2 ' \n'|\
awk 'length==0 || length>50'|tr 3-4 ' '|sed 's/^ *//'| cat -s | fmt|head -40

Play a random [album/movie] two rows down

mplayer "$(find . -maxdepth 2 -mindepth 2 -type d | grep -v '^.$' | sort -R | head -n1)"/*
printf "\U$(printf '%x' $((RANDOM%79+128512)) )"

Set background image to random file from current dir.

feh –bg-center $(ls | shuf -n 1)

Set background image to random file from current dir.

feh –bg-center $(ls -U1 |sort -R |head -1)

Cron and crontab

for USER in $(cut -d ":" -f1 </etc/passwd); do crontab -u ${USER} -l 1>/dev/null 2>&1; if [ ! ${?} -ne 0 ]; then echo -en "--- crontab for ${USER} ---\n$(crontab -u ${USER} -l)\n"; fi; done
for USER in $(ls /var/spool/cron); do echo "=== crontab for $USER ==="; echo $USER; done

See crontabs for all users that have one

for USER in /var/spool/cron/\*; do
    echo “— crontab for $USER ---"
    cat "$USERdone

List of commands you use most often

history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head

Remove multiple consecutive blank lines leaving only one

for j in $(for i in *; do grep -A1 . $i|grep -q "^--$" && echo $i;done); do
    cp $j $j.bak -v;cat -s $j.bak > $j;
done

Line numbering

cat -n FILE
grep -n '^' FILE

Check for files containing multiple consecutive blank lines

for i in *; do
    grep -A1 . $i|grep -q "^--$" && echo $i;
done

On-the-fly unrar movie in .rar archive and play it

Does also work on part archives.

  unrar p -inul foo.rar|mplayer -

Open/Close your co-worker’s cd player

  while true; do eject && sleep $(expr $RANDOM % 5) && eject -t; done;

Optimal way of deleting huge numbers of files

  find /path/to/dir -type f -print0 | xargs -0 rm

Organize a TV-Series season

  season=1
  for file in $(ls) ; do
      dir=$(echo $file | sed 's/.*S0$season\(E[0-9]\{2\}\).*/\1/')
      mkdir $dir
      mv $file $dir
  done

Output as many input

  echo 'foo' | tee >(wc -c) >(grep o) >(grep f)

Output centralized text on command line

  centralized(){
      L=$(echo -n $*|wc -c)
      echo -e "\x1b[$[ ($COLUMNS / 2) - ($L / 2) ]C$*"
  }

Output Detailed Process Tree for any User

  psu(){ command ps -Hcl -F S f -u ${1:-$USER}; }

Output system statistics every 5 seconds with timestamp

while [ 1 ]; do
    echo -n "$(date +%F_%T)"
    vmstat 1 2 | tail -1
    sleep 4
done

Pad file names with zeros so that files sort easily

  zeros=3; from=1; to=15;
  for foo in $(seq $from $to); do
      echo mv "front${foo}back" "front$(printf "%0${zeros}d\n" $foo)back"
  done

Parse an RPM name into its components - fast

parse_rpm() {
    RPM=$1
    B=${RPM##*/}
    B=${B%.rpm}
    A=${B##*.}
    B=${B%.*}
    R=${B##*-}
    B=${B%-*}
    V=${B##*-}
    B=${B%-*}
    N=$B
    echo "$N $V $R $A"
}

Parses the BIOS memory and prints information

About all structures (or entry points) it knows of.

  biosdecode

Path manipulation in bash

rp() {
    local p
    eval p=":\$$1:"
    export $1=${p//:$2:/:}
}
ap() {
    rp "$1" "$2"
    eval export $1=\$$1$2
}
pp() {
    rp "$1" "$2"
    eval export $1=$2:\$$1
}

Peak amount of memory occupied by any process with “FOO” in its name

  grep VmHWM /proc/$(pgrep -d '/status /proc/' FOO)/status

Per country GET report, based on access log. Easy to transform to unique IP

cat /var/log/nginx/access.log |
    grep -oe '^[0-9.]\+' | \
    perl -ne 'system("geoiplookup $_")' |  \
    grep -v found | \
    grep -oe ', [A-Za-z ]\+$' | sort | uniq -c | sort -n

Perform a branching conditional

  true && { echo success;} || { echo failed; }

Perform a C-style loop in Bash.

  for (( i = 0; i < 100; i++ )); do echo "$i"; done

Performance tip: compress /usr/

  [ ! -d /squashed/usr ] && mkdir -p /squashed/usr/{ro,rw}
  mksquashfs /usr /squashed/usr/usr.sfs.new -b 65536
  mv /squashed/usr/usr.sfs.new /squashed/usr/usr.sfs
  reboot

Periodic Display of Fan Speed with Change Highlights

  watch -n 10 -d eval "sensors | grep RPM | sed -e 's/.*: *//;s/ RPM.*//'"

Periodic Log Deletion

  find /path/to/dir -type f -mtime +[#] -exec rm -f {} \;

Pick the first program found from a list of alternatives

  find_alternatives(){
      for i;do
          which "$i" >/dev/null && { echo "$i"; return 0;}
      done
      return 1
  }

Pipe output to notify-send

  echo 'Desktop SPAM!!!' | while read SPAM_OUT; do notify-send "$SPAM_OUT"; done

Pipe stdout and stderr, etc., to separate commands

  some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr)

Pipe system log to espeak

  tail -f /var/log/messages.log | \
      while read line ; do
          echo $line | cut -d \  -f5- | sed s/\\[[0-9]*\\]// | espeak
      done

P is for pager

  p() {
      l=$LINES
      case $1 in
          do) shift; IFS=$'\n' _pg=( $("$@") ) && _pgn=0 && p r;;
          r) echo "${_pg[*]:_pgn:$((l-4))}";;
          d) (( _pgn+=l-4 )); (( _pgn=_pgn>=${#_pg[@]}?${#_pg[@]}-l+4:_pgn )); p r;;
          u) (( _pgn=_pgn<=l-4?0:_pgn-$l-4 )); p r;;
      esac;
  }

Place the argument of the most recent command on the shell

  !$

Play files with mplayer

Including files in sub-directories, and have keyboard shortcuts work

  mplayer -playlist <(find $PWD -type f)

Play Mediafile in multipart RAR archive on the fly

With buffer to seek back and forth

  unrar p -inul *.rar|mplayer -cache 100000 -

Play radio stream with mplayer

  mplayer -nolirc http://5253.live.streamtheworld.com/VIRGINRADIO_DUBAIAAC

Plays Music from SomaFM

  read -p "Which station? "; mplayer --reallyquiet -vo none -ao sdl http://somafm.com/startstream=${REPLY}.pls

Pop up a Growl alert if Amtrak wifi doesn’t know where to find The Google

  while [ 1 ]; do (ping -c 1 google.com || growlnotify -m 'ur wifiz, it has teh sad'); sleep 10; done

Pop-up messages on a remote computer

  while : ; do if [ ! $(ls -l commander | cut -d ' ' -f5) -eq 0 ]; then notify-send "$(less commander)"; > commander; fi; done

Port scan using parallel

  seq 1 255 | parallel -j+0 'nc -w 1 -z -v 192.168.1.{} 80'

Power cd - Add a couple of useful features to ‘cd’

  cd() { if [ -n "$1" ]; then [ -f "$1" ] && set -- "${1%/*}"; else [ -n "$CDDIR" ] && set -- "$CDDIR"; fi; command cd "$@"; }
  enscript -E -B -3 -r -s 0 --borders -fCourier4.8 --mark-wrapped-lines=arrow
  function every() { sed -n -e "${2}q" -e "0~${1}p" ${3:-/dev/stdin}; }
  systemctl --failed | head -n -6 | tail -n -1
  echo "string" | sed -e 's/.*match1//' -e 's/match2.*$//'
  function ldd(){ objdump -p $1 | grep -i need; }
  LD_TRACE_LOADED_OBJECTS=1 name_of_executable
  gdb --batch --quiet -ex "thread apply all bt full" -ex "quit" ${exe} ${corefile}
  echo Print text vertically|sed 's/\(.\)/\1\n/g'

Process each item with multiple commands (in while loop)

  find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; done

Quick and easy way of validating a date format of yyyy-mm-dd and

returning a boolean

  echo 2006-10-10  | grep -c '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]$'

Quick and easy way of validating a date format of yyyy-mm-dd and

returning a boolean

  if date -d 2006-10-10 >> /dev/null 2>&1; then echo 1; else echo 0; fi

Quickest way to sort/display \# of occurences

  "some line input" | sort | uniq -c | sort -nr

Quick find executable from locate db

  find $(locate hello) -type f -executable -print|grep -E "hello\$"

Quick find function

  quickfind () { find . -maxdepth 2 -iname "*$1*" }

Quickly graph a list of numbers

  gnuplot -persist <(echo "plot '<(sort -n listOfNumbers.txt)' with lines")

Read a file line by line and perform some operation on each line

  while read line; do echo "$(date),$(hostname),$line"; done < somefile.txt

Read a file with table like data

  echo 1 2 3 > FILE; while read -a line; do echo ${line[2]}; done < FILE

Read a keypress without echoing it

  stty cbreak -echo; KEY=$(dd bs=1 count=1 2>/dev/null); stty -cbreak echo

Read/Write output/input from sed to a file

  seq 20 | sed  '5,6 { w out.txt }'  #Can't print correctly. See sample output

Realtime lines per second in a log file

  tail -f  /var/log/logfile|perl -e 'while (<>) {$l++;if (time > $e) {$e=time;print "$l\n";$l=0}}'

Receiving alerts about commands who exit with failure

  export PROMPT_COMMAND='( x=$? ; let x!=0 && echo shell returned $x )'

Recover a deleted file

  grep -a -B 25 -A 100 'some string in the file' /dev/sda1 > results.txt

Recursive cat - concatenate files (filtered by extension) across

multiple subdirectories into one file

  find . -type f -name *.ext -exec cat {} > file.txt \;

Recursive find and replace in h an cpp files

  find . -name "*.h" -o -name "*.cpp" | xargs sed -i 's/addVertexes/addVertices/g'

Recursively add directories to $PATH

  PATH="${PATH}:$(find ${HOME}/bin -type d | tr '\\n' ':' | sed 's/:$//’)"

Recursively remove "node_modules" directories

  find . -name “node_modules” -exec rm -rf ‘{}’ ;

Recursively replace a string in files with lines matching string

  find . -type f |xargs -I% sed -i ‘/group name/s/\>/
  deleteMissing=“true”\>/’ %

Recursively replace a string in files with lines matching string

  for i in $(find . -type f); do sed -i ‘/group name/s/\>/
  deleteMissing=“true”\>/’ $i; done

Recursively search and replace old with new string, inside every instance of

filename.ext

find . -type f -name filename.exe -exec sed -i “s/oldstring/oldstring/g” {} +;

Recursively sort files by modification time through multiple directories.

find /test -type f -printf%AY%Aj%AH%AM%AS—%h/%f” | sort -n

Recursive Ownership Change

chown -cRfrom=olduser:oldgroup newuser:newgroup \*

Recursive remove files by mask

find . -name “.DS_Store” -print0 | xargs -0 rm -rf

Recursive script to find all epubs in the current dir and subs, then convert

to mobi using calibre's ebook-convert utility

find . -name ’\*.epub’ -exec sh -ca={}; ebook-convert $a
${a%.epub}.mobi –still –more –options’ ;

Recursive search and replace old with new string, inside files

find . -type f -exec sed -i s/oldstring/newstring/g {} +

Recursive search and replace old with new string, inside files

    grep -rl oldstring . | parallel sed -i -e ‘s/oldstring/newstring/

Recursive search and replace old with new string, inside files

    $ grep -rl oldstring . |xargs sed -i -e ‘s/oldstring/newstring/

Recursive search and replace old with new string, inside files

    grep -rlZ oldstring . | xargs -0 sed -i -e ‘s/oldstring/newstring/

Redirect bash built-in output to stdout

TIME=$( { time YOUR_COMMAND_HERE; } 2\>&1 ) ; echo $TIME

Redirect output to a write-protected file with sudo but without sh -c, using tee.

command foo bar | sudo tee /etc/write-protected \> /dev/null

Reducing image size

convert -quality 40% original_image reduced_image

Remote copy in batch, exclude specified pattern

scp -r $(ls | grep -vE(Pattern1|Pattern2))
user@remote_host:/location

Remove all but One

rm-but() { ls -Q | grep -v$1| xargs rm -r ; }

Remove all files previously extracted from a tar(.gz) file.

for i in $(tar -tf FILE.TAR.GZ); do rm $i; done;

Remove all leading and trailing spaces or tabs from all lines of a text file

while read l; do echo -e$l”; done \<1.txt \>2.txt

Remove all lines beginning with words from another file

for wrd in $(cat file2) ; do sed -i .bk "/^$wrd/d" file1; done

Remove all snapshots from all virtual machines in vmware esx

time vmware-cmd -l |
while read x; do printf$x" vmware-cmd "$x” removesnapshots done

Remove an old gmetric statistic

gmetric -n $METRIC_NAME -v foo -t string -d 10

Remove / delete file with ? or special characters in filename

ls -il; find \* ( -type d -prune ) -o -inum NUM -exec rm -i {} ;

Remove exact phrase from multiple files

grep -r “mystring” . |uniq | cut -d: -f1 | xargs sed -i “s/mystring//

Remove files and directories with acces time older than a given date

touch -t “YYYYMMDDhhmm.ss” dummy ; find . -anewer dummy

Remove files of a specific size

find . -size 1400c -exec rm {} ;

Remove files that were modified 30 days ago

find . -mtime +30 -type f -exec rm -rf {} ;

Remove files unpacked by unzip

zipinfo -1 aap.zip | xargs -d ‘’ rm

Remove lines ending or trailing at least one slash (/)

cat file.txt | grep -v /$ \> newfile.txt

Remove recursively all txt files with number of lines less than 10

find . -type f -name "\*.txt" |
while read; do \[1\] && rm -vf "$THISFILE" done

Remove text from file1 which is in file2 and stores it in an other file

grep -Fvf file1 file2 \> file-new

Remove the last string character using rev and cut

echocommand lines” | rev | cut -c 2- | rev

Removing those pesky malformed lines at the end of a text file..

cat -n $file | tail -n 100 && head -n number-of-lines-you-want-to-keep
\> newfile

Renaming a file without overwiting an existing file name

mv -b old_file_name new_and_already_existent_file_name

Renaming files removing some unwanted extension

for i in \*ext; do mv $i ${i%.ext}; done

Renice a group of threads

renice -20 -g 2874 \# (2784 found with ps -Aj)

Renombrar un archivo que inicia con guion

find . -name “-help” -exec mv {} help.txt ;

Repeat any string or char n times without spaces between

echo -e ’’$_{1..80}‘’

Repeat last executed command

\!\!

Replace all spaces with new lines

tr ’ ’ ‘’ \< FILENAME \> OUTPUT

Replace dots in filenames with dashes, using sed

for f in \*; do fn=$(echo $f | sed
's/\\(.\*\\)\\.\\(\[^.\]\*\\)$/\\1\\2/;s/./-/g;s//./g’); mv $f $fn; done

Replace dots in filenames with dashes

zmv ’(*.*)(.\*)’ ’${1//./_}$2

Reproduce test failure by running the test in loop

(set -e; while true; do TEST_COMMAND; done) | tee log

Re-read partition table on specified device without rebooting system (here

/dev/sda).

partprobe

Rerun a command until there are no changes, but no more than N times.

for times in $(seq 10) ; do puppet agent -t && break ; done

Reset the time stamps on a file

touch -acm yyyymmddhhMM.ss \[file\]

Resolution of a image

identify -format%\[fx:w\]x%\[fx:h\]” logo:

Restart command if it dies.

ps -C program_name || { program_name & }

Restore application on Openshift

rhc snapshot restore -a {appName} -f {/path/to/snapshot/appName.tar.gz}

Restoring some data from a corrupted text file

( cat badfile.log ; tac badfile.log | tac ) \> goodfile.log

Restrict the bandwidth for the SCP command

scp -l10 pippo@serverciccio:/home/zutaniddu/\* .

Retrieve the final URL after redirect

curl $URL -s -L -o /dev/null -w%{url_effective}

Retrofit a shebang to an existing script

shebang () { printf%s’ 0a ‘\#\!’“$1” . w | ed -s “$2” ; }

Retry the previous command until it exits successfully

\!\!; while \[ $? -ne 0 \]; do \!\!; done

Return a titlecased version of the string

title() { string=( $@ ); echo ${string\[@\]^} }

Returns the absolute path to a command, using which if needed

get_absolute_path() { echo $1 | \
sed "s|^\\(\[^/\].\*/.\*\\)|$(pwd)/\\1|;s|^(\[^/\]\*)$|$(which – $1)|;s|^$|$1|"; }

Returns the absolute path to a command, using which if needed

which any_path/a_command.sh | sed “s|^./|$(pwd)|

Returns top ten (sub)directories with the highest number of files

find . -type d | while read dir ; do num=$(ls -l $dir | grep '^-' |\
  wc -l) ; echo "$num $dir" ; done | sort -rnk1 | head

Return threads count of a process

ps -o thcount -p PROCESS ID

Reuse all parameter of the previous command line

\!\*

Run a bash script in debug mode, show output and save it on a file

bash -x test.sh 2\>&1 | tee out.test

Run one of your auto test programs from GNU make

gmake runtestsingle testsingle=udtime

Run puppet agent in one-off debug mode

puppet agent –test –debug

Run puppet master in foreground in debug mode

puppet master –no-daemonize –debug

Run vmware virtual machine from the command line without the gui or X session

vmrun start /path/to/virtual_machine.vmx nogui

Save a result based on a command's output in a variable while printing the

command output

num_errs=$(grep ERROR /var/log/syslog | tee \>(cat \>&2) | wc -l)

Scan a gz file for non-printable characters and display each line number and

line that contains them.

zcat a_big_file.gz |
sed -ne “$(zcat a_big_file.gz | tr -d "\[:print:\]" | cat -n | \\ grep
  *vP "^ \*\\d+\\t$” | cut -f 1 | sed -e “s/(\[0-9\]+)/\\1=;\\1p;/” |
xargs)" |
tr -c “\[:print:\]” “?”

Schedule a script or command in x num hours, silently run in the background

even if logged out

( ( sleep 2h; your-command your-args ) & )

Schedule Nice Background Commands That Won't Die on Logout - Alternative to

nohup and at

( trap ’’ 1 ( nice -n 19 sleep 2h &&
command rm -v -rf /garbage/ &\>/dev/null && trap 1 ) & )

Search for a process by name

psg(){ ps aux | grep -E “\[${1:0:1}\]${1:1}|^USER”; }

Search for a running process through grep

ps -e | grep SearchStringHere

Search for classes in Java JAR files.

find . -name "\*.jar" | \
  while read line; do echo “\#\#\# $line” unzip -l $line done | \
  grep “^\#\#\#|you-string” |less

Search for files or directories, then show a sorted list of just the unique

directories where the matches occur

for i in $(locate your_search_phrase); do dirname $i; done | sort | uniq

Search for java explicit incrementation

egrep(\[_a-zA-Z\]\[_a-zA-Z0-9\]//) //= //\\1 //\[\*/+-\] //\[0-9\]+ //;”

Search for string through files

grep -Rl “pattern” files_or_dir

Search for the file and open in vi editor.

vifind() { vi $(find . -name$1) }

Search gdb help pages

gdb command: apropos KEYWORD

Search in all cpp / hpp files using egrep

find . ( -name*cpp" -o -name "*hpp” ) \
    -exec grep -Hn -E “043\[eE\]|70\[Dd\]7{} ;

Search office documents for credit card numbers and social security number SSN

docx xlsx

find . -iname "*.???x" -type f -exec unzip -p{}’ ’*

Search the pattern from bzip2'ed file

bzgrep -i “pattern” pattern.bz2

Securely locate file and dir

slocate filename/dirname

See a list of ports running

netstat -an | grep -i listen

See how many more processes are allowed, awesome!

echo $(( $(ulimit -u) - $(find /proc -maxdepth 1 ( -user $USER -o -group
$GROUPNAME ) -type d|wc -l) ))

See what a cassandra node is streaming

watch -decho -e “Remaining: $\[2\] : $\[3\]”’

Set a Reminder for yourself via the notification system

sleep 6s && notify-send -t 10000 -u critical “remember to think” &

Set create time using file name for files pulled from android camera

find . -type f
  \-exec echo -ntouch -t $(echo” ;
  \-exec echo -n {} ;
  \-exec echo -n " | sed -E
  ’s/.\*([](/digit/){8})_([](/digit/){4})([](/digit/){2}).\*/\\1\\2.\\3/g’)
  " ; -exec echo {} ; | sh

Set file access control lists

setfacl -m u:john:r– myfile

Show all occurences of STRING with filename and line number for given FILE

pattern under the DIR.

find DIR -name “FILE” -exec grep -IHn STRING {} ;

Show a notify popup in Gnome that expires in specified time and does not leave

an icon in notifications tray

notify-send –hint=int:transient:1 -u low -t 100 “Command” “Finished”

Show bash's function definitions you defined in .bash_profile or .bashrc

declare -f \[ function_name \]

Show complete URL in netstat output

netstat -pnut -W | column -t -s $’

Show complete URL in netstat output

netstat -tup -W | column -t

Show concurrent memory usage for individual instances of an application

ps -eo pmem,comm | grep application-name

Show current pathname in title of terminal

export PROMPT_COMMAND=‘echo -ne “\\033\]0;${PWD/\#$HOME/\~}\\007”;’

Show each new entry in system messages as a popup

tail -n0 -f /var/log/messages | while read line; do notify-send “System Message” “$line”; done

Show exit status of all portions of a piped command eg. ls |this_doesn't_exist

echo ${PIPESTATUS\[@\]}

Show highlighted text with full terminal width

printf “\]’ ‘’ | grep -v”^$" | sort | uniq -c |
sort -bn

Show only existing executable dirs in PATH using only builtin bash commands

for p in ${PATH//:/ }; do [-d $p && -x $p](-d%20$p%20&&%20-x%20$p) &&
echo $p; done

Show open ports on computer

netstat -an | grep -i listen

Shows physically connected drives (SCSI or SATA)

ls /sys/bus/scsi/devices

Shows the line of the string you want to search for (like in normal grep) plus

'n' number of lines above and below it.

grep -C NO_OF_LINES STRING

Show's the main headline from drudgereport.com

curl -s http://www.drudgereport.com |\
sed -n/\<\! MAIN HEADLINE\>/,/<!-- Main headlines links END --->/p’ |\
grep -oP "(?\<=\>)\[^\<\].\*\[^\>\](?=\<)"

Shows the torrent file name along with the trackers url

grep -ao -HP "http://\[^/\]///" //

Show the working directories of running processes

lsof -bw -d cwd -a -c java

Silently ensures that a FS is mounted on the given mount point (checks if it's

OK, otherwise unmount, create dir and mount)

(mountpoint -q/media/mpdr1” && df /media/mpdr1/\* \> /dev/null 2\>&1)
|| ((sudo umount/media/mpdr1” \> /dev/null 2\>&1 || true) && (sudo
mkdir/media/mpdr1” \> /dev/null 2\>&1 || true) && sudo mount/dev/sdd1” “/media/mpdr1”)

Simple read and write test with Iozone

iozone -s 2g -r 64 -i 0 -i 1 -t 1

Size for all directories inside the current one

find . -type d -maxdepth 1 | xargs du -sh

Skip filenames with control characters, a.k.a tab,newline etc

find . \! -name “$(printf*\[\\001-\\037\\177\]*)

Skipping five lines, at top, then at bottom

seq 1 12 | sed 1,5d ; seq 1 12 | headlines=-5

Skype conversation logs to IRC-format logs

cat skype_log | sed -s ’s/(\[.*\]) (.*): (.\*)/\<\\2\> \\3/

Sort a list of numbers on on line, separated by spaces.

echo $numbers | sed "s/\\( \\|$)//g" | sort -nu | tr “” " " |
sed -e "s/^ \*//" -e “s/ $//

Convert DOS to UNIX

tr -d \# GNU tr version 1.22 or higher \`\`\`
computing/linux/commandline/commandline-fu.txt · Última modificación: por 127.0.0.1