Tips, tricks and tools for Linux/Unix

[I completely updated this article in early 2014. In particular, I included links directly to blog posts and books that discuss each tool.]

Unix is a different way -- a different philosophy -- of computing.

This is a compendium of all the useful Unix tools I've used over the years:

Text, word and document processing.
Networking, security and system administration.
Window managers.
Web browsing and web development.
Mail and messaging.
Command-line control.
Programming languages and software development.
Text manipulation.
Digital media viewing and manipulation.
Dealing with windows.

If you're new to Unix/Linux, you should check out my related posts:

Text, word and document processing

emacs: An instance of the text-editor-as-operating-system thesis.

WhizzyTeX: WYSIWYG LaTeX editing for emacs.
AUCTeX: Power-editing LaTeX inside emacs.
TNT: Instant-messaging inside emacs.
Aquamacs: An OS X-enhanced port of emacs.
Related book: Learning GNU Emacs

vim: A text editor that brutally minimizes the number of keystrokes.

NERDtree: a file browser for vim.
Related book:
Learning the vi and Vim Editors

TeX and LaTeX: High-quality scientific and technical publishing tools.

qtree: A package for drawing parse trees.
Active-DVI: A feature-rich DVI viewer.
ArabTeX: Write arabic script in LaTeX.
Xy-pic: A graph-rendering language for LaTeX.
Bibtex: Effortless bibliography and citation management.
pstricks: Graphics in LaTeX.
TikZ: Beautiful diagrams in LaTeX.
eepic: Graphics in LaTeX.
HeVeA: A powerful LaTeX-to-HTML converter.
Related books: Guide to LaTeX. For low-level TeX-hacking, Knuth's original is the only guide that helped me write programs in raw TeX: The TeXbook

gnuplot: Graph data of all kinds; import them into LaTeX.
Xfig: GUI tool for drawing diagrams for LaTeX. Old, but still useful.
DOT: For rendering large graphs/networks/charts automatically.

GraphViz: An excellent OS X interface for DOT files.

gnumeric + OpenOffice + catdoc and xls2csv: For dealing with Microsoft Office documents.

Related post: Read/edit/create MS Office without windows

Google Docs/Drive: Makes cross-platform, multi-user documents easy; it has some support for Microsoft Office documents

Networking, security and system administration

ssh, scp, sftp, ssh-agent, etc. + putty: The swiss-army knives of secure remote system management.

Related post: SSH hacks.

iptables: Flexible firewall and network management.

ufw: A simple front end for iptables in Ubuntu.

NFS + samba: Set up a Unix box as a home file server.
Unison: A file sync daemon, i.e., your own DropBox.
BIND: Run your own DNS server.
Apache: A flexible HTTPD.

htpasswd + .htaccess: Password protect pages.
VirtualHost: Run multiple web sites off the same server.

nginx: A high-performance yet still versatile HTTPD.

Related post: Scaling a web site.

exim + vexim + courier + null webmail: Run your own mail server.
GPG: Cryptography for the (educated) masses.

Related post: HOWTO: Encryption basics

VNC: Connect to X remotely.
MediaWiki | MoinMoin: Run your own wiki.
RootKit hunter + chkrootkit: Check if your machine has been compromised.
I learned a lot of my admin hacks from these two books:
Linux Server Hacks and Linux Server Hacks, Volume Two.

Window managers

ion: The only innovative window manager ever.

Web browsing and web development

Firefox: Great for browsing, great for development.

AdBlock: No more ads.
Firebug: JavaScript debugging on steroids.
Greasemonkey: Add per site JavaScript customization.
leechblock: Block/restrict time-wasting web sites.

lynx | links: Browse the web from the command line.
wget | curl: Scrape the web from the command line.
nodejs: A non-blocking implementation of JavaScript geared toward writing daemons.

Related post: Continuation-passing style in JavaScript.

Ruby on Rails | Perl | PHP: For dynamic pages.

GD: Dynamic image generation.
CGI: For web devolpment in Perl.

MySQL | PostgreSQL: Relational database servers.
CouchDB: A NoSQL database server based in JavaScript.

Mail and messaging

fetchmail + procmail + mutt + spamassassin + msmtp: A complete tool-chain for mail management.
irssi | X-Chat: Command-line/GUI IRC chat.
bip: An IRC proxy that lets you stay connected around the clock.

Command-line control

bash: An ugly but useful shell-scripting language.

bash completions: Smart tab-completion for bash.
Related post: Settling into Unix.

zsh: A better bash.
screen | tmux: Leave a console process running; re-connect to it later.
bc: A command-line calculation language.
sudo: Delegate partial super-user privileges.
safe-rm: Eliminate the risk of an accidental rm -rf /.
ducks: Hunt down space hogs.
autojump / j: Automatically track/jump to frequent directories.

Related post: Console productivity hack.

Programming languages and software development

Scheme Shell + Racket: Robust, hacker-friendly Scheme environments, interpreters and compilers.

SXML: A nifty encoding of XML into S-Expressions.
A guide to learning Racket, motivated by video games: Realm of Racket

(MLton + SML/NJ) | OCaml: High-performance, high-reliability functional programming.

Related book: Real World OCaml

Scala: A strictly superior replacement for Java.

My favorite guide to Scala, written by its creator: Programming in Scala

haskell: A lazy, purely functional programming language.

Related books: Learn You a Haskell for Great Good! and Real World Haskell.

Java SDK: Beause it runs everywhere.

javadoc: Source-code documentation management.
ANTLR | (JLex + CUP): Lexer/parser generators for Java.

gnu toolchain: Because C is still around.
lex + yacc: Lexer/parser generators for C.

Related post: Standalone lexers.

make: Project compilation management.

Related post: An intro to make.

subversion: Centralized source-code version control. (Works great with LaTeX too.)
git | mercurial: Decentralized source-code version control.
SDL + OpenGL: A mature cross-platform 2D and 3D graphics libraries.
WebGL: A JavaScript API for 3D graphics.
Squeak: An instance of the programming-language-as-operating-system thesis.

Text manipulation

m4 + awk + sed: Macros and regular expressions.

Related post: Sculpting text.

diff + patch: Study file changes; make patches.

Digital media viewing and manipulation

(VLC | MPlayer) + MEncoder + DeCSS tools: View DVDs and transfer them to a compressed file.
xmms | (mpg123 + mp3blaster): MP3 playback and management.
ImageMagick: Handy command-line tools for image processing.
gimp: Photoshop for Linux.
lame: Encode MP3s.
X-CD-Roast: Burn CDs under X.

Dealing with Windows

Wine: Run Windows programs under Linux.
Parallels: Run Windows (or Linux) on Mac OS X.