WEB command line
See where a shortened url takes you before click
check(){ curl -sI $1 | sed -n 's/Location: *//p';}
Resolve short urls
resolve(){ curl -Is $1 | egrep "Location" | sed "s/Location: \(.*\)/\1/g"; }
Google URL shortener
python -c 'import googl; print googl.Googl("<your_google_api_key>").shorten("'$someurl'")[u"id"]'
Short URLs with ur1.ca
ur1() { curl -s --url http://ur1.ca/ -d longurl="$1" | sed -n -e '/Your ur1/!d;s/.*<a href="\(.*\)">.*$/\1/;p' ; }
Down for everyone or just me?
down4me() { wget -qO - "http://www.downforeveryoneorjustme.com/$1" | sed '/just you/!d;s/<[^>]*>//g' ; }
Trace http requests with tshark
tshark -i en1 -z proto,colinfo,http.request.uri,http.request.uri -R http.request.uri
Decode uri
csi -R uri-common -p '(uri-decode-string (read-line))'
Extract domain from URl
ruby -ruri -e 'u=URI(ARGV[0]).host.split("."); puts u[(u[-2] =~ /^com?$/ ? -3 : -2)..-1].join(".")' http://test.example.com
Get URL’s from a webpage
curl url | egrep -o '(http|https)://[a-z0-9]*\.[a-z0-9]*\.[a-z0-9]*'|sort |uniq
Encode file path to URL
convert_path2uri () { echo -n 'file://'; echo -n "$1" | perl -pe 's/([^a-zA-Z0-9_\/.])/sprintf("%%%.2x", ord($1))/eg' ;} #convert2uri '/tmp/a b' ### convert file path to URI
Python Cgi server for tests.
Start a HTTP server which serves Python docs
pydoc -p 8888 & gnome-open http://localhost:8888
Web server in Ruby
ruby -rwebrick -e'WEBrick::HTTPServer.new(:Port => 3000, :DocumentRoot => Dir.pwd).start'
perl6 Download a webpage
perl6 -M HTTP::UserAgent -e 'say HTTP::UserAgent.new.get("google.com").content'
perl6 Download a webpage and strip out the HTML
wget -O - "http://perl6.org" | perl6 -ne 's:g/\<.+?\>//.say'
perl6 Download a webpage, strip out and decode the HTML
wget -O - "http://perl6.org" | perl6 -MHTML::Strip -ne 'strip_html($_).say'
Perl6 Launch a simple web server
perl6 -M HTTP::Server::Simple -e 'HTTP::Server::Simple.new.run'