Useless Use of...
This and That






Jan Schaumann <jschauma@netmeister.org>

99CE 1DC7 770A C5A8 09A6   0DCD 66CE 4FE9 6F6B D3D7

whoami

$ ME=$(id -un) 
$ grep ${ME} /etc/passwd | cut -d: -f5 
Jan Schaumann 
$ 

whoami

$ ME=$(id -un) 
$ grep ${ME} /etc/passwd | cut -d: -f5 
Jan Schaumann 
$ groups ${ME}
etsy netbsd sa stevens
$ 

In the beginning...

Search and Destroy

Subject: Search and Destroy
Date: 1995/09/06
Message-ID: <42k9bf$ja9@ixnews2.ix.netcom.com>
newsgroups: comp.unix.shell


I have a bunch of text files that contain strings containing /u. I no
longer want them to contain /u ex: /u/appl/ should be /appl/....

Should I use awk? sed? Help!

Search and Destroy

Subject: Re: Search and Destroy
Date: 1995/09/06
Message-ID: <42l1b0$sk0@flood.xnet.com>
references: <42k9bf$ja9@ixnews2.ix.netcom.com>
newsgroups: comp.unix.shell


> I have a bunch of text files that contain strings containing /u. I no
> longer want them to contain /u ex: /u/appl/ should be /appl/....
>
> Should I use awk? sed? Help!

cat file | sed -e "s!/u/!/!"

Enter Merlyn


Headshot of Randal Schwartz by Chris Marquardt

Search and Destroy

Subject: This week's Useless Use of Cat Award goes to... (was Re: Search and Destroy)
Date: 1995/09/06
Message-ID: <ukspm9r5lf.fsf_-_@linda.teleport.com>
newsgroups: comp.unix.shell

This week's Useless Use of Cat Award goes to...

"Hemant" == Hemant Shah <s...@flood.xnet.com> who writes:

Hemant> cat file | sed -e "s!/u/!/!"

which, using the simple rule that:

        cat fred | barney junk junk junk ...

can nearly always be rewritten as:

        <fred barney junk junk junk ...

would look like this:

        <file sed -e "s!/u/!/!"
 

Search and Destroy

Subject: This week's Useless Use of Cat Award goes to... (was Re: Search and Destroy)
Date: 1995/09/06
Message-ID: <ukspm9r5lf.fsf_-_@linda.teleport.com>
newsgroups: comp.unix.shell

This week's Useless Use of Cat Award goes to...

"Hemant" == Hemant Shah <s...@flood.xnet.com> who writes:

Hemant> cat file | sed -e "s!/u/!/!"

which, using the simple rule that:

        cat fred | barney junk junk junk ...

can nearly always be rewritten as:

        <fred barney junk junk junk ...

would look like this:

        <file sed -e "s!/u/!/!"

Useful Use of Cat?

The obvious examples first:

cat file | grep pattern


cat file | awk '{ print $2; }'
 

Useless Use of Cat!

The obvious examples first:

cat file | grep pattern
grep pattern <file

cat file | awk '{ print $2; }'
awk '{ print $2; }' <file

Useless Use of Cat!

The obvious examples first:

cat file | grep pattern
grep pattern <file

cat file | awk '{ print $2; }'
awk '{ print $2; }' <file



I would also accept:

grep pattern file

awk '{ print $2; }' file

Useless Use of Cat!

The obvious examples first:

cat file | grep pattern
grep pattern <file

cat file | awk '{ print $2; }'
awk '{ print $2; }' <file



I would also accept:

grep pattern file

awk '{ print $2; }' file



Or:

<file grep pattern
<file awk '{ print $2; }'

Useful Use of Cat?

cat file1 file2 file3 | wc -l



cat file1 file2 file3 | wc -w
 

Useless Use of Cat!

cat file1 file2 file3 | wc -l
awk 'END { print NR }' file1 file2 file3


cat file1 file2 file3 | wc -w
awk '{w = w + NF} END { print w }' file1 file2 file3

Useless Use of Cat! - TMTOWTDI

cat file1 file2 file3 | wc -l
awk 'END { print NR }' file1 file2 file3
sed -n -e '$=' file1 file2 file3

cat file1 file2 file3 | wc -w
awk '{w = w + NF} END { print w }' file1 file2 file3
...left as an exercise...

Even More Useless Use of Cat

cat file1 file2 file3 | grep pattern



if [ $(cat files | grep -c pattern) -gt 0 ]; then
 

Even More Useless Use of Cat

cat file1 file2 file3 | grep pattern
grep -h pattern file1 file2 file3


if [ $(cat files | grep -c pattern) -gt 0 ]; then
if [ -n "$(grep -l pattern files)" ]; then

Even More Useless Use of Cat - TMTOWTDI

cat file1 file2 file3 | grep pattern
grep -h pattern file1 file2 file3
awk '/pattern/ { print }' file1 file2 file3

if [ $(cat files | grep -c pattern) -gt 0 ]; then
if [ -n "$(grep -l pattern files)" ]; then
if grep pattern files >/dev/null 2>&1; then

Useful Use of Cat

Useful Use of Cat

Useful Use of Cat

Useful Use of Cat

Useful Use of Cat


Cat5 (now actually less useful)


Cat6

Why bother?

Why bother?



Why bother?



Why bother?

Why bother?

So.... cats are stupid. What else?



Useless Use of Grep

Of course, all use of grep(1) is useless!

echo g/RE/p | ed -s file

Useful Use of Grep?

host hostname | grep 'has address' | awk '{ print $NF }'



echo ${string} | grep 'pattern' | sed -e 's/foo/bar/'
 

Useless Use of Grep!

host hostname | grep 'has address' | awk '{ print $NF }'
host hostname | awk '/has address/ { print $NF}'


echo ${string} | grep 'pattern' | sed -e 's/foo/bar/'
echo ${string} | sed -n -e '/pattern/ s/foo/bar/p'

Useful Use of Grep?

grep pattern1 file | grep -v pattern2


grep pattern1 file | grep -v ^# | grep -v pattern2
 

Useless Use of Grep!

grep pattern1 file ... | grep -v pattern2
awk '/pattern1/ && !/pattern2/ { print }' file

grep pattern1 file | grep -v ^# | grep -v pattern2
awk '/pattern1/ && !/(^#)|(pattern2)/ { print }

Useless Use of Grep!

grep pattern1 file ... | grep -v pattern2
awk '/pattern1/ && !/pattern2/ { print }' file

grep pattern1 file | grep -v ^# | grep -v pattern2
awk '/pattern1/ && !/(^#)|(pattern2)/ { print }

ps waux | grep cmd | grep -v grep

Useless Use of Grep!

grep pattern1 file ... | grep -v pattern2
awk '/pattern1/ && !/pattern2/ { print }' file

grep pattern1 file | grep -v ^# | grep -v pattern2
awk '/pattern1/ && !/(^#)|(pattern2)/ { print }

ps waux | grep cmd | grep -v grep
ps waux | grep [c]md

Useful Use of Grep!

Useful Use of Sed?

for p in $(echo ${PATH} | sed -e 's/:/ /'); do
        ls ${p}
done
 

Useless Use of Sed!

for p in $(echo ${PATH} | sed -e 's/:/ /'); do
IFS=":"; for p in ${PATH}; do
        ls ${p}
done

ls /usr/share/ikea

ls /usr/share/ikea

ls /usr/share/ikea

ls /usr/share/ikea

ls /usr/share/ikea

Useful Use of Sed!

s/[0-9]/<&/g
s/0//g; s/1/|/g; s/2/||/g; s/3/|||/g; s/4/||||/g; s/5/|||||/g;
s/6/||||||/g
s/7/|||||||/g; s/8/||||||||/g; s/9/|||||||||/g
: tens
s/|</<||||||||||/g
t tens
s/<//g
s/+//g
: minus
s/|-|/-/g
t minus
s/-$//
: back
s/||||||||||/</g
s/<\([0-9]*\)$/<0\1/
s/|||||||||/9/; s/||||||||/8/; s/|||||||/7/; s/||||||/6/; s/|||||/5/;
s/||||/4/
s/|||/3/; s/||/2/; s/|/1/
s/</|/g
t back
http://is.gd/m62ijc

Useful Use of Sed?

VAR="foo-bar-baz"
VAR_A=$(echo ${VAR} | sed -e 's/-.*//')
VAR_B=$(echo ${VAR} | sed -e 's/[^-]*-\([^-]*\)-.*/\1/')
VAR_C=$(echo ${VAR} | sed -e 's/.*-//')
 

Useful Use of Sed?

VAR="foo-bar-baz"
VAR_A=$(echo ${VAR} | sed -e 's/-.*//')
VAR_B=$(echo ${VAR} | sed -e 's/[^-]*-\([^-]*\)-.*/\1/')
VAR_C=$(echo ${VAR} | sed -e 's/.*-//')

VAR_A=${VAR%%-*}
 

Useful Use of Sed?

VAR="foo-bar-baz"
VAR_A=$(echo ${VAR} | sed -e 's/-.*//')
VAR_B=$(echo ${VAR} | sed -e 's/[^-]*-\([^-]*\)-.*/\1/')
VAR_C=$(echo ${VAR} | sed -e 's/.*-//')

VAR_A=${VAR%%-*}

VAR_C=${VAR##*-}
 

Useful Use of Sed?

VAR="foo-bar-baz"
VAR_A=$(echo ${VAR} | sed -e 's/-.*//')
VAR_B=$(echo ${VAR} | sed -e 's/[^-]*-\([^-]*\)-.*/\1/')
VAR_C=$(echo ${VAR} | sed -e 's/.*-//')

VAR_A=${VAR%%-*}
VAR_B=${${VAR%-*}#*-} # Wait, does this work?
VAR_C=${VAR##*-}
 

Useful Use of Sed?

VAR="foo-bar-baz"
VAR_A=$(echo ${VAR} | sed -e 's/-.*//')
VAR_B=$(echo ${VAR} | sed -e 's/[^-]*-\([^-]*\)-.*/\1/')
VAR_C=$(echo ${VAR} | sed -e 's/.*-//')

VAR_A=${VAR%%-*}
VAR_B=${${VAR%-*}#*-}
sh: Syntax error: Bad substitution
VAR_C=${VAR##*-}
 

Useless Use of Sed!

VAR="foo-bar-baz"
VAR_A=$(echo ${VAR} | sed -e 's/-.*//')
VAR_B=$(echo ${VAR} | sed -e 's/[^-]*-\([^-]*\)-.*/\1/')
VAR_C=$(echo ${VAR} | sed -e 's/.*-//')

IFS=-
set -- ${VAR}
VAR_A="${1}"
VAR_B="${2}"
VAR_C="${3}"

IFS is teh Rox0r!!1!

Look, Ma: reading a CSV with shell Only!

$ IFS="," 
$ while read -r field1 waste field3 field4 waste; do 
        echo "${field1}: ${field4} --> ${field3}" 
done <file
$ 

Useless Use of IFS!

$ cat >script.sh
IFS=","
while read -r field1 waste field3 field4 waste; do
        echo "${field1}: ${field4} --> ${field3}"
done <file
^D
$ wc -l file
       111061 file
$ time sh script.sh >/dev/null
    12.33s real 3.30s user 8.54s system
$

Useful Use of AWK

$ cat >script2.sh 
awk -F"," '{ print $1 ": " $4 " --> " $3 }' <file 
^D 
$ time sh script2.sh >/dev/null 
    1.08s real 1.08s user 0.00s system 
$ 

Useful Use of ls?

for file in $(ls *pattern*); do
        the_needful
done
 

Useless Use of ls!

for file in $(ls *pattern*); do
for file in *pattern*; do
        the_needful
done

Useful Use of wc?

VAR="$(cat file | wc -l | sed -e 's/ *//g')"  

Useful Use of wc?

VAR="$(cat file | wc -l | sed -e 's/ *//g')"
VAR="$(wc -l <file | sed -e 's/ *//g')"

Useless Use of wc and sed!

VAR="$(cat file | wc -l | sed -e 's/ *//g')"
VAR="$(wc -l <file | sed -e 's/ *//g')"
VAR="$(awk 'END { print NR }' file)"

Useful Use of wc!

VAR="$(cat file | wc -l | sed -e 's/ *//g')"
VAR="$(wc -l <file | sed -e 's/ *//g')"
VAR="$(awk 'END { print NR }' file)"
VAR="$(wc -l <file)"
echo "${VAR}"
echo ${VAR}

Useful Use of head?

command1 | head -1 | sed -e 's/pattern/string/'


command1 | head -10 | sed -e 's/pattern/string/'
 

Useless Use of head!

command1 | head -1 | sed -e 's/pattern/string/'
command1 | sed -e 's/pattern/string/;q'

command1 | head -10 | sed -e 's/pattern/string/'
command1 | sed -e 's/pattern/string/;10q'

Useful Use of tail?

command1 | tail -1 | sed -e 's/pattern/string/'


command1 | tail -10 | sed -e 's/pattern/string/'
 

Useless Use of tail!

command1 | tail -1 | sed -e 's/pattern/string/'
command1 | sed -n -e '$s/pattern/string/p'

command1 | tail -10 | sed -e 's/pattern/string/'

Useful Use of tail!

command1 | tail -1 | sed -e 's/pattern/string/'
command1 | sed -n -e '$s/pattern/string/p'

command1 | tail -10 | sed -e 's/pattern/string/'

Shell Performance

Shell Performance

Shell Performance

Shell Performance

Shell Performance

Shell Performance

How do we know which tool performs well?

How do we know which tool performs well?



Count HTTP status codes

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

sed -e 's/.*GET.*HTTP[^ ]*" \([^ ]*\).*/\1/' | \
        grep ^2 | wc -l
 

Count HTTP status codes

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

sed -e 's/.*GET.*HTTP[^ ]*" \([^ ]*\).*/\1/' | \
        grep ^2 | wc -l

sed -e 's/.*GET.*HTTP[^ ]*" \([^ ]*\).*/\1/' | grep -c ^2

Count HTTP status codes

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

sed -e 's/.*GET.*HTTP[^ ]*" \([^ ]*\).*/\1/' | \
        grep ^2 | wc -l

sed -e 's/.*GET.*HTTP[^ ]*" \([^ ]*\).*/\1/' | grep -c ^2
awk '/HTTP\/[^ ]*" 200 / { c++; } END { print c }'

MEASURE ALL THE THINGS!

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

$ time sed -n -e '/HTTP\/[^ ]*" 2/p' | wc -l >/dev/null # TMTOWTDI

real     4m16.154s
user     4m13.461s
sys      0m3.955s
 

MEASURE ALL THE THINGS!

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

$ time sed -n -e '/HTTP\/[^ ]*" 2/p' | wc -l >/dev/null # TMTOWTDI

real     4m16.154s
user     4m13.461s
sys      0m3.955s


$ time awk '/HTTP\/[^ ]*" 2/ { c++; } END { print c }' >/dev/null

real     1m3.647s
user     0m57.873s
sys      0m2.234s
 

MEASURE ALL THE THINGS!

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

$ time perl -lne 'if (m/HTTP\/[^ ]*" 2/) { $n++; } END { print $n; }'
 

Wut? Perl? Surely you jest!

$ ldd `which awk` `which sed` `which perl`
/usr/bin/awk:
        -lc.12 => /usr/lib/libc.so.12
        -lm.0 => /usr/lib/libm.so.0
/usr/bin/sed:
        -lc.12 => /usr/lib/libc.so.12
/usr/pkg/bin/perl:
        -lc.12 => /usr/lib/libc.so.12
        -lm.0 => /usr/lib/libm.so.0
        -lcrypt.0 => /usr/lib/libcrypt.so.0
        -lpthread.0 => /usr/lib/libpthread.so.0
        -lperl => /usr/pkg/lib/perl5/5.14.0/x86_64-netbsd-thread-multi/CORE/libperl.so
$ du -h /usr/bin/awk /usr/bin/sed /usr/pkg/bin/perl \
        /usr/pkg/lib/perl5/5.14.0/x86_64-netbsd-thread-multi/CORE/libperl.so
154K    /usr/bin/awk
34K     /usr/bin/sed
12K     /usr/pkg/bin/perl
1.5M    /usr/pkg/lib/perl5/5.14.0/x86_64-netbsd-thread-multi/CORE/libperl.so

MEASURE ALL THE THINGS!

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

$ time perl -lne 'if (m/HTTP\/[^ ]*" 2/) { $n++; } END { print $n; }'

real    0m22.146s
user    0m15.827s
sys     0m1.662s

Useful Use of Perl!

207.38.139.33 - - [18/Apr/2012:18:41:11 -0400] "GET /slides/nycbug201205/pics/grep-eyes.png HTTP/1.1" 200 191432 "http://www.netmeister.org/slides/nycbug201205/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"

$ time perl -lne 'if (m/HTTP\/[^ ]*" 2/) { $n++; } END { print $n; }'

real    0m22.146s
user    0m15.827s
sys     0m1.662s


92% performance increase compared to sed
65% performance increase compared to awk

Performance vs. Readability

grep ${ME} /etc/passwd | cut -d: -f5




cat file1 file2 file2 | wc -w
 

Performance vs. Readability

grep ${ME} /etc/passwd | cut -d: -f5
awk -F: -v ME="${ME}" '{ if ($0 ~ ME) { print $5 }}' \
        /etc/passwd


cat file1 file2 file2 | wc -w

Performance vs. Readability

grep ${ME} /etc/passwd | cut -d: -f5
awk -F: -v ME="${ME}" '{ if ($0 ~ ME) { print $5 }}' \
        /etc/passwd

awk -F: "/${ME}/ { print \$5 }" /etc/passwd

cat file1 file2 file2 | wc -w

Performance vs. Readability

grep ${ME} /etc/passwd | cut -d: -f5
awk -F: -v ME="${ME}" '{ if ($0 ~ ME) { print $5 }}' \
        /etc/passwd

awk -F: "/${ME}/ { print \$5 }" /etc/passwd

cat file1 file2 file2 | wc -w
awk '{w = w + NF} END { print w }' file1 file2 file3

Performance vs. Readability

grep ${ME} /etc/passwd | cut -d: -f5
awk -F: -v ME="${ME}" '{ if ($0 ~ ME) { print $5 }}' \
        /etc/passwd

awk -F: "/${ME}/ { print \$5 }" /etc/passwd

cat file1 file2 file2 | wc -w
awk '{w = w + NF} END { print w }' file1 file2 file3

The Prime Directive


Worse is Better

  • Simplicity -- most important
  • Correctness -- slightly better to be simple than correct
  • Consistency -- better to drop less common use cases than to introduce complexity or inconsistency
  • Completeness -- completeness can be sacrificed in favor of any other quality

http://is.gd/4DRphb

Remember how a web server used to work?

Remember how a web server used to work?

Client: "Hello, I'd like to get the contents of this file, please."
   GET / HTTP/1.0\n\n

Remember how a web server used to work?

Client: "Hello, I'd like to get the contents of this file, please."
   GET / HTTP/1.0\n\n

Server: "Why, certainly, here you go."
   cat file > socket

Remember how a web server used to work?

Client: "Hello, I'd like to get the contents of this file, please."
   GET / HTTP/1.0\n\n

Server: "Why, certainly, here you go."
   cat file > socket

HTTP today...

Client: "Hello, I'd like to get the contents of this file on this server, please."
   GET / HTTP/1.1\n
   Host: this server\n\n

HTTP today...

Client: "Hello, I'd like to get the contents of this file on this server, please."
   GET / HTTP/1.1\n
   Host: this server\n\n


Server: "I'm sorry, Node.js just crashed."
   HTTP/1.1 500 Internal Server Error
   Server: Apache-Coyote/1.1


HTTP today...

Client: "Uhm... Node.js? I thought you said you're an Apache server?"

Server: "Oh, sure, but I hand off LESS compilation to Node.js, which is faster than Rhino."

HTTP today...

Client: "Uhm... Node.js? I thought you said you're an Apache server?"

Server: "Oh, sure, but I hand off LESS compilation to Node.js, which is faster than Rhino."

Client: "Uhm... rhino? So you're running Javascript inside Java?"

HTTP today...

Client: "Uhm... Node.js? I thought you said you're an Apache server?"

Server: "Oh, sure, but I hand off LESS compilation to Node.js, which is faster than Rhino."

Client: "Uhm... rhino? So you're running Javascript inside Java?"

Server: "Oh, sure, I'm a Tomcat server!"

HTTP today...

Client: "Uhm... Node.js? I thought you said you're an Apache server?"

Server: "Oh, sure, but I hand off LESS compilation to Node.js, which is faster than Rhino."

Client: "Uhm... rhino? So you're running Javascript inside Java?"

Server: "Oh, sure, I'm a Tomcat server!"

Client: "So I'm talking to... your Coyote 'HTTP connector', which hands my requests off to your Catalina servlet container, which..."

Server: "...looks up stuff in various databases, triggers actions in a message broker and hands the LESS compilation off to Node.js..."

HTTP today...

Client: "Uhm... Node.js? I thought you said you're an Apache server?"

Server: "Oh, sure, but I hand off LESS compilation to Node.js, which is faster than Rhino."

Client: "Uhm... rhino? So you're running Javascript inside Java?"

Server: "Oh, sure, I'm a Tomcat server!"

Client: "So I'm talking to... your Coyote 'HTTP connector', which hands my requests off to your Catalina servlet container, which..."

Server: "...looks up stuff in various databases, triggers actions in a message broker and hands the LESS compilation off to Node.js..."

Client: "Uhm... LESS? 'Compilation'? Why don't you just serve the plain old CSS?"

HTTP today...

Client: "Uhm... Node.js? I thought you said you're an Apache server?"

Server: "Oh, sure, but I hand off LESS compilation to Node.js, which is faster than Rhino."

Client: "Uhm... rhino? So you're running Javascript inside Java?"

Server: "Oh, sure, I'm a Tomcat server!"

Client: "So I'm talking to... your Coyote 'HTTP connector', which hands my requests off to your Catalina servlet container, which..."

Server: "...looks up stuff in various databases, triggers actions in a message broker and hands the LESS compilation off to Node.js..."

Client: "Uhm... LESS? 'Compilation'? Why don't you just serve the plain old CSS?"

Server: "Don't be silly. 'All problems in computer science can be solved by another level of indirection.' Now where was I?"

HTTP today...

Client: "Uhm... Node.js? I thought you said you're an Apache server?"

Server: "Oh, sure, but I hand off LESS compilation to Node.js, which is faster than Rhino."

Client: "Uhm... rhino? So you're running Javascript inside Java?"

Server: "Oh, sure, I'm a Tomcat server!"

Client: "So I'm talking to... your Coyote 'HTTP connector', which hands my requests off to your Catalina servlet container, which..."

Server: "...looks up stuff in various databases, triggers actions in a message broker and hands the LESS compilation off to Node.js..."

Client: "Uhm... LESS? 'Compilation'? Why don't you just serve the plain old CSS?"

Server: "Don't be silly. 'All problems in computer science can be solved by another level of indirection.' Now where was I?"

Server: "Oh, right: ...hands the LESS compilation off to Node.js, which just crashed."

HTTP today...

Client: "My head hurts. How did I get here?"

So... what about those layers?

All problems in computer science can be solved by another level of indirection.

David Wheeler

Layers are bad...

...when all they do is provide indirection, obfuscation and increase complexity.

$ echo moo > 1
$ for i in `jot 100`; do
>        ln -s $i $(($i + 1))
done
$ cat 100
cat: 100: Too many levels of symbolic links
$ 

Layers are good...

...when they provide security.

Layers are good...

...when they provide abstraction.

Layers are good...

...when they make things taste better.

Complexity is the Enemy

  1. essential complexity: inherent to the problem; cannot be reduced

    An HTTP server has to, at some point, shuffle data from somewhere to the client.

Complexity is the Enemy

  1. essential complexity: inherent to the problem; cannot be reduced

    An HTTP server has to, at some point, shuffle data from somewhere to the client.

  2. accidental complexity: inadvertently and often indirectly introduced

    The data can be static. Making it dynamic adds complexity.

Complexity is the Enemy

  1. essential complexity: inherent to the problem; cannot be reduced

    An HTTP server has to, at some point, shuffle data from somewhere to the client.

  2. accidental complexity: inadvertently and often indirectly introduced

    The data can be static. Making it dynamic adds complexity.


Software complexity increases non-linearly, frequently exponentially!

Complexity is the Enemy

  1. essential complexity: inherent to the problem; cannot be reduced

    An HTTP server has to, at some point, shuffle data from somewhere to the client.

  2. accidental complexity: inadvertently and often indirectly introduced

    The data can be static. Making it dynamic adds complexity.


Software complexity increases non-linearly, frequently exponentially!

Wait... the same could be said about:

Useless Use of Message Queues

SysV and "message-oriented middleware" (insert your MOM joke here)

Wait... the same could be said about:

Useless Use of Message Queues

SysV and "message-oriented middleware" (insert your MOM joke here)

Useless Use of Complex Formats

XML makes me YAML.

Wait... the same could be said about:

Useless Use of Message Queues

SysV and "message-oriented middleware" (insert your MOM joke here)

Useless Use of Complex Formats

XML makes me YAML

Useless Use of DSL

Greenspun's 10th Rule of Programming

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

Philip Greenspun

Wait... the same could be said about:

Useless Use of Message Queues

SysV and "message-oriented middleware" (insert your MOM joke here)

Useless Use of Complex Formats

XML makes me YAML

Useless Use of DSL

Greenspun's 10th Rule of Programming

Useless Use of NoSQL, Cassandra, Hadoop, ...

Old and busted vs. new hotness ("Oooh, shiny!")

Wait... the same could be said about:

...all sorts of things.

Wait... the same could be said about:

...all sorts of things.



Precisely.

Wait... the same could be said about:

...all sorts of things.



Precisely.




http://is.gd/5kx8mL
Down With the Fancy Pants

Worse is Better

Simplicity

The design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface. Simplicity is the most important consideration in a design.

Worse is Better

Simplicity

The design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface. Simplicity is the most important consideration in a design.

...but...

Useless Use of Time?

Booo complexity, hooray simplicity!

Hooray beer!

jschauma@netmeister.org

/