Usenet is still a strange place

Jan Schaumann


Contents

Introduction

Who ever said that Usenet is dead? It most certainly is not. Even though Usenet has changed it's face over the last, say ten years, the simple idea of exchanging ideas and having discussions among a large number of people by posting articles on a global message-board has pertained. Yes, for those of you who still don't know what this is all about, we are talking about Newsgroups, an important part of the Internet, albeit heavily neglected by the media, who usually equate ``Internet'' with ``World Wide Web''. The spread of the Internet, the increased availability to more and more households has not only influenced the way we shop, trade stocks etc. but has also (trans)formed the old message-board system of Usenet. But rest assured, no matter how much things have changed over the last few years, Usenet is still a strange place.1

This ``strange'' place has, for a long time, been regarded as inhibited only by full-time geeks, nerds with coke-bottle thick glasses trying to download obviously fake pictures of naked celebrities2 when they are not busy discussing the technological details of the Starship Enterprise's Warp-Drive3.

But while these people certainly exist, there is another side to Usenet: with more and more people getting on the Internet it is true that on the one hand the signal-to-noise ratio has been lowered, which is why some people might say that ``Usenet is dead''. On the other hand, more and more people are exchanging ideas, helping each other out and plenty of highly interesting discussions take place every day in this community. If you invest a little time, I'm sure I can convince you that Usenet is quit alive.

History

The history of Usenet is so closely linked to the history of the Internet itself and the history of Unix (and thus Linux!), that it is worth to spend some time finding out how things started. Let us take a trip back into the year 1976 (a good year, I might add4) - the place, how could it be any different, AT&T Bell Laboratories, where a new utility called ``Unix To Unix Copy Program'' (or short: UUCP) had just been developed. UUCP was designed as a simple and efficient way of copying files between computers via phone-lines, and while nowadays UUCP has been superseded by TCP/IP based protocols, it still lay the foundation for what became known as ``the poor man's ARPANet''.

The poor man's ARPANet

The ARPANet (Advanced Research Projects Agency Network), created by the US Department of Defense in 1969, was originally limited to computer scientists with Department of Defense contracts - not everybody could just go ``willy-nilly over the ARPANet''. Aside from these connections, in order to join ARPANet, one needed quite some cash - assumed numbers ranged up to $100,000.

In 1979, two Duke University grad students started using UUCP to enable people to exchange information by uploading (``posting'') a message to a designated subject-area called a ``newsgroup''. Following a simple bulletin-board approach, subsequent messages would appear in the same ``newsgroup'', messages regarding other topics would appear in a different ``newsgroup''. This message-system became known as Usenet News (Unix Users Network) - using UUCP, everybody, even poor students were able to use this system, without being connected to the ARPANet.

The shell-scripts used by these two students, Tom Truscott and Jim Ellis, were later re-written in C for public distribution and named the ``A'' release of news. If you consider how much traffic goes through an average full-feed news-server nowadays5, it is amusing to know that this ``A'' release was designed for not more than a few articles per group per day.

This small number of messages posted each day evolved into what became its own culture of sharing information and support as more and more groups of core sites were linked together. In the year 1981, UUCP and ARPANet, up until then still independent, were linked together by Berknet (at the University of California at Berkeley). Other universities and colleges developed their own networks, such as BITnet (Because It's Time net - we see, a geek's affirmation for witty acronyms has a long tradition), which was based on the IBM protocol and developed by Yale and the City University of New York[4].

As these various different networks were linked together, Usenet had the form of a graph - and ``Network maps'' were drawn.

Figure 1: An early map of Usenet, ca 1980
\includegraphics[width=1.0\textwidth]{map.eps}

The news software was re-written and became the ``B'' release, networks were developed and linked, in brief, things were changing quickly. More and more forums were established, and it became clear that more structure was needed - it was time for ``The Great Renaming''[3].

The Great Renaming

The first forums, or newsgroups, to be established were net.xxxx, and dept.xxxx; several mailing-lists from the ARPANet fed into this Unix Users Network formed the ``fa''-hierarchy. As traffic increased, several hierarchies were instantiated, and Usenet was now connecting people all over the globe. However,transmission was still very expensive, so that Universities in Europe were not willing to pay for ``the fluff groups like net.religion and net.flame''[3]. A new hierarchy, the ``talk.*''-hierarchy was proposed, and the Great Renaming began.

In the time between 1986 and 1987, the main top-hierarchies were created, including comp, misc, news, rec, sci, soc and talk. Brian Reid, as many other Usenet users, were still not quite happy with this structure - the Backbone Cabal6 for example was unwilling to create rec.drugs and planned on dropping net.flame. In 1987, Brian Reid came up with the idea for an ``alternative'' distribution system that would not use these backbone links, creating the new top-level hierarchy ``alt.*''. alt.sex and alt.rock-n-roll followed a year later:

``To end the suspense, I have just created alt.sex. That meant that the alt network now carried alt.sex and alt.drugs. It was therefore artistically necessary to create alt.rock-n-roll, which I have also done. I have no idea what sort of traffic it will carry. If the bizzarroids take it over I will rmgroup it or moderate it; otherwise I will let it be.'' - Brian Reid [5]

Eventually, with the advent of the TCP/IP based Network News Transport Protocol (NNTP) all these groups were also transmitted to Europe and all over the world. Fewer people were using UUCP, NNTP became more and more popular, the Internet itself spread around the globe - Backbone Cabal became more and more silent and eventually was declared dead. Usenet, however, was quite alive and thriving, attracting more and more people every day - ultimately leading to what some old-timers refer to as ``The September that never ended''.

The September that never ended

During the late 80's and early 90's Usenet had become quite popular, and even more so, quite understandably, among students. Every year in September when Semesters started and Freshmen got access to the Internet, many so-called ``newbies'' did not bother to follow the netiquette but started posting to the various groups in rude and inexperienced ways. In the days before 1993 the majority of people on Usenet were regulars, following the newsgroups for years and (sometimes more, sometimes less) willing to help out newcomers, teaching them netiquette. But in 1993 ([6]) AOL gave Usenet-Access to its customers, opening the doors for a flood of technically inexperienced newbies to swamp the forums demonstrating (sometimes more, sometimes less) firmly that they did not know how to deal with this new medium and, much worse, a limited willingness to learn. Some people insist that ever since that September in 1993 the overall quality of discussions in Usenet has gone down.7

It is hard to deny that the merging of two such different groups of people like the tech-savvy geeks and the average Joe-Sixpack with an AOL-account lead to conflicts and changed the way of the discussions held in the various newsgroups. However, this did not mean the end of Usenet, as some old-timers would have it - in the end, all it means is a larger variety of opinions to be discussed. More groups were created, old silent groups deleted, new structures laid in. For every possible topic there is a newsgroup nowadays, all one needs to do is take a look at the impressive hierarchy of Usenet, to get an idea how information is flowing around the globe and into everybody's home.

Hierarchy

As mentioned earlier, Usenet is structured hierarchically. The main top-level hierarchies are alt, comp, misc, news, rec, sci, soc and talk. Furthermore, there are hierarchies for each country, the country code of which precedes the international equivalent. For example, de.comp.os.linux.misc would be the German speaking equivalent to comp.os.linux.misc8, just as fr.comp.lang.c is the French equivalent to comp.lang.c. In addition to the country (or rather language) specific newsgroups there are now a large variety of other top-level hierarchies such as free.*9, gnu.* and news.*, as well as commercial or ``company-related'' newsgroups.

The ``Master List of all Newsgroup Hierarchies'' is available at
http://faqs.org/faqs/usenet/hierarchy-list

Restructuring

These hierarchies are highly functional and very well structured. Occasionally, however, some restructuring is necessary. If one discussion group for example becomes overwhelmed with too many question about a certain topic, it may be decided to split the group into foo.general (or foo.misc) and foo.bar. If such a decision needs to be made, a posting is sent into this group as well as related groups to call for a vote. Every person reading the newsgroup can then cast his or her vote for or against the creation of another, more specific group - a very democratic process.

Different kinds of Newsgroups

Since every newsgroup has its own Charta, in which the purpose and the topic of this group is clearly stated, it is easy to find the right group to which post your article. Furthermore, this Charta specifies the rules that apply to this group. For example, it is generally frowned upon to send HTML-formatted postings into Usenet - however, the Charta may indicate that this behavior is acceptable or even desired in this (and only this) newsgroup.

Some Newsgroups are ``moderated'', meaning that the posts are not posted to the group directly, but rather to a person (or a group of people) who is in charge of filtering out noise and inappropriate posts. Anything that is not on topic in this newsgroup, anything that is of offensive nature or anything that already has been answered in the newsgroup or in its Charta or FAQ is weeded out by the moderator(s) and will not appear in the group.

The third kind of newsgroups are the private newsgroup. These may or may not be moderated, or have specific rules, but in general the only difference to the ``normal'' newsgroups is that they are not available on all news-servers, or only certain people have access to these groups. Most universities have some local newsgroups that can only be accessed from within the local network - the same goes for some ISP's. Other companies for example sell a certain service and restrict the access to their full feed to paying customers only.

Newsreader

Since network news articles certainly resemble mail messages (or e-mail), many people think of them as being the same, which is why they often request a program that handles Mail and News instead of looking for one single program for each of these rather different tasks10. A news article is broadcasted to all interested hosts by the news server, which keeps one local copy for a certain amount of time, before it gets expired. This article contains a lot of important information in its headers, among them the unique Message-ID identifying each article and the Message-ID's of the articles it refers to (References), if any.

GNKSA

The structure of Usenet and the way these articles are copied from news server to news server require the posting agent to be as standards-conform as possible. To ensure that a newsreader follows the standards, ``The Good Net-Keeping Seal of Approval'', a document intended to describe minimal standards for decent net-behavior, was created. It specifies the requirements a good news-client should meet, some of which are rather essential, others which are ``merely a Very Good Idea, Really''[9].

Note, however, that this is no ``official'' document, and that there are plenty of newsreaders out there which do not carry this Seal, and may still be functional. In general, a newsreader should follow all o GNKSA's ``MUST'''s - otherwise broken postings might be generated.

While the content of the article is certainly in the hands of the user, the posting agent must ensure that it does everything possible to create standards-compliant postings and does not encourage behavior that is considered to be bad netiquette (see below).

List of Newsreaders

The question as to which is the ``best'' newsreader can not be answered - it would be equivalent to ask, which linux-distribution is the ``best''. It all depends on your personal preference: Some people prefer a GUI, while others are die-hard command-line-interface fans. The following is an aplhabetical list of the most common newsreaders available for Linux:

To evaluate each and every one of these newsreaders would certainly go beyond the scope of this article - however, I feel that I should point out a few brief details about some of these applications. For a more detailed discussion of the various newsreaders, please see [8].

Gnus

For all you hard-core (X)Emacs fans out there, you probably already made your decision: this is the Newsreader for you. From all of the above listed applications, this is the only one that combines News and Mail in one application and can still be recommended without any reservations. It is highly configurable and written in Emacs LISP for GNU Emacs and XEmacs.

Figure 2: A Screenshot of Gnus
\includegraphics[width=0.75\textwidth]{gnus.eps}

Pan

Pan is a very powerful gtk/gnome-based newsreader. It can be used in off-line mode as well as in online-mode, which makes it very attractive to the poor souls out there not using DSL or a Cable-Modem. It recently scored the highest GNKSA-rating of all Unix/Linux newsreaders and simplifies downloading binaries by automatically grouping together multi-part binary attachments and support of multiple connections.

The developers at Superpimp are focusing on creating a functional GUI-newsreader which still can be operated entirely by use of keyboard. The compliance with all of GNKSA's rules (by the time of this writing, Pan would receive a perfect score if re-evaluated) ensures that the user does not accidently create a broken posting.

One of the most important things about Pan is it's exemplary development: daily CVS-snapshots are available for everybody who is interested; developers are actively following the users mailing-list and submitted bug-reports and patches are considered and/or applied within days. It is a great example of how and why Open Source works.

Figure 3: A Screenshot of Pan
\includegraphics[width=0.75\textwidth]{pan.eps}

Slrn

Slrn (``s-lang read news'') is a text-interfaced newsreader, very similar in appearance to mutt11. It has been ported to a large variety of Operating Systems and can easily be extended using the s-lang macro language. If you don't need or don't want to use a GUI, this is most certainly the fastest and most functional console-newsreader you can get. The many configuration-options offered through the s-lang macro language make it more customizable than Pan for example.

Figure 4: A Screenshot of Slrn
\includegraphics[width=0.75\textwidth]{slrn.eps}

Netiquette

``On the Internet, nobody knows you're a dog.'' - While this is certainly true, there are a few simple rules that everybody should follow. These rules are referred to as the ``Netiquette'' (from ``network etiquette''), and they ensure that Usenet does not drown in noise. Even though you may ask yourself in the beginning why things should be a certain way, rest assured that once you spend some time on Usenet, you will understand and appreciate these rules.

One thing to always keep in mind is that Usenet is not a free support-service, but that there are people having discussions in their free time. If you have a question and you get it answered quickly, remember this.

What it boils down to is basically common sense and common courtesy. One of the core-rules of netiquette is to always remember the human on the other side [10]. All other rules follow logically as long as you keep this in mind.

Finding the right Newsgroup

Of course it is important to find the correct newsgroup for your question - not only would it be quite unlikely that somebody in ``japan.animal.penguin'' might know the answer to a networking-related question, it is also considered rude to post ``Off-Topic'' messages into a newsgroup.

In order to make it easier for people to find the right newsgroup to post their message to, almost all newsreaders are able to show you a brief summary, a one line description of most groups when you subscribe to a newsgroup. However, occasionally you might want to subscribe to a newsgroup without having to search through the list of all groups available on your server. Since all Linux-Newsreaders use the same file-format to keep track of the available and subscribed groups, it is easy to search through this file (~/.newsrc in general) for a specific newsgroup using grep or egrep.

For example, say you wish to find a list of all Linux-specific groups from the ``alt'' or ``comp'' hierarchy:

egrep '^(comp|alt).*linux' ~/.newsrc | more

If you want to see the list of groups regarding news-related software and announcements, you might try:

egrep '^news(soft|announce)' ~/.newsrc | more

Since egrep supports regular expressions, you can refine your search as narrowly or widely as needed. This way of searching for specific newsgroups may come in handy if you want to hint to another poster that he might be getting more qualified responses in another newsgroup, one that you are not subscribed to.

Before you start posting into a newsgroup, it is generally considered a Good Idea (tm) to

This way you get an idea of what the general tone in this newsgroup is, or if your question has not been answered already or is even part of the FAQ. In addition, it is also generally expected that you searched through a news-archive like Deja.com before you post into the group. Remember, Usenet is not free tech support!

``Subject'', ``From'' and Quoting

It is important to remember that some people are reading hundreds and hundreds of messages each day - they are most likely the most qualified to help you out, so you want to make it easy for them to read your posts. Many people just read through the Subject-lines and determine from there on, if they are going to bother to read the entire article, so try to keep the subject line short, but significant. "HELP!!!!!!!" is bad, bad example. Remember that writing in all caps is considered to be screaming; furthermore, ``multiple exclamation marks are a sure sign of a sick mind.''12.

As mentioned above, nobody knows who you are, and sometimes you may not even want to reveal your identity. However, many people prefer answering questions when asked politely by somebody with a real name. Your name is like your outer appearance in cyberspace - if you post under the name "K3wl d00d", your postings might not be taken seriously. Likewise, if you choose a rather offensive name, people might killfile you right away, without even looking at your posts. Just use your real name and your real e-mail address.

When you reply to another message, quote only the relevant portions of the message you are replying to. Don't quote the whole message if not necessary, especially deleting previous signatures. Be careful to not misattribute something. Leave in the attribution line that most newsreaders provide. Observe the attribution lines and read posts carefully. Replied text goes below the quoted text! For a more detailed discussion on the ``Art of Quoting'', please see [11]

Attachments

Many people are still using a dial-up connection to connect to the Internet. These people are literally paying for every byte of information that goes through their modem13. Let this be just one reason why you should never ever post an attachment into a newsgroup that does not specifically state in its Charta that it is desired. Usually, attachments are only appropriate in *binaries*-newsgroups.

Signatures

Once you have spent some time on Usenet, you will see that many people have a signature which is appended to every posting. Some people use it to provide, for example, a link to their website, others have a funny quote in it. If you want to have a signature, keep in mind that there are rules for these, too14: first of all, a signature should not be any longer than four lines. This is again an argument of bandwidth. As your article is copied hundreds and hundreds of times, one more line does make a difference. Why four? Why not five? Well, basically, because. The original purpose of the signature file was to provide your business information. Now there has to be a limit somewhere, and since generally speaking all information such as address, phone number and email-address does fit into four lines, it's four.

Another thing about the signature is the correct delimiter. Usually, your newsreader will provide the correct delimiter itself, but in case you want to supply your own, please notice that the only correct delimiter is ``- '' 15. The delimiter (or ``sigdash'') allows other programs to recognize when the body of your message is finished and where the sig begins - this is useful for automatically removing the sig when quoting for a followup, as well as coloring it a different color, archiving, and various other things.

Randomized Signatures

Many people use their signature not to provide their contact information, but rather to express their individuality to a certain extent, after all, it's called their signature. While you may often find short ASCII-art, or a link to the owners homepage in the signature, many people use shell scripts ranging from very simple to rather elaborate to generate a unique and/or funny signature for every post they send. You will always be able to impress a newbie by having your signature for example display the current uptime of your linux-box, or feed it with some quotes from the popular "fortune"-program:

#!/bin/bash

echo -e "Cheers, $\backslash$n Yourname$\backslash$n$\backslash$n- " > ~/.signature

uptime » ~/.signature

fortune -s » ~/.signature

Just make sure you use the -s switch for ``fortune'' - otherwise your signature might get too long. To make your signature even more individual, you might want to create a few fortune-files yourself16 and specify them in the script. Then put this script into your crontab and run it every, say, five minutes. Voilà - you have your own personal sig.

Spam Protection

Unfortunately, not all of the people posting to Usenet read the Netiquette or follow them. Every now and then you will find that some people enjoy lurking around in newsgroups and posting offensive material and/or rude replies17. It's usually a good idea to simply ignore these people - a task that can easily be automatically accomplished by your newsreader. Most (if not all) Linux-based newsreaders provide the capability of assigning different levels of importance to the different articles, depending on various criteria such as Author, Number of Lines, Subject etc. The act of deleting a certain persons articles so that you will not even see them is referred to as ``killfiling''18, while other actions (such as highlighting, sorting by importance, marking as read while still displaying the article etc.) are referred to as ``scoring''. Again Gnus, Pan and Slrn stand out among the other readers with respect to this capability.

Email-Munging

A more severe drawback from posting to newsgroups is that you expose your email-address to the world, and special spiders or bots are ``harvesting'' high-traffic newsgroups for email-address, resulting in increased Spam in your mailbox. It is important to provide a valid email-address in some way when posting to Usenet, so that people can reply to you if you desire so. Many people are ``munging'' their email-address, or are simply entering an invalid address in the From-Header and a short note in their signature, explaining how to retrieve the original valid email-address.

However, one should not simply use an invalid email-address, since this way it either puts more load on your system19 or on the originating mailserver20. In general it is considered polite to use only the suffix ``.invalid'' for your email-address, so as to make it clear that your address is in fact not a genuine one.

From here on

As you can, did and/or will see, Usenet is a very interesting place to spend some (plenty? all?) of your free time on. But it is also particularly large place, a topic that can hardly be covered adequately within limited space. In this article I tried to give an overview over the main aspects of Usenet, touching various topics only briefly.

Some topics that are of particular interest and that need further, more detailed attention include real spam-protection through a mail-filtering program such as procmail, and the installation and maintenance of a local newsserver such as leafnode (for a small local network) or inn (for a full-blown professional network or newsservice). Since these topics are much too extensive to even be touched here, I might elaborate on them in future articles.

In the mean time, I would like to encourage the reader to follow the links referenced in here and, of course, to explore Usenet through their own experience. But remember:

``This game lends itself to certain abuses.''
-- Bill Watterson's Calvin

Bibliography

1
Angela Gunn, Charles Pappas A Brief History of the Net, http://www.zdnet.com/yil/content/mag/9809/history.html

2
Various, Bell Laboratories History, http://www.bell-labs.com/history

3
Lee S. Bumgarner, The Great Renaming FAQ, http://www.vrx.net/usenet/history/rename.html

4
Unknown, What is the Internet? A Brief History of ARPANet, UUCP, BITNET, NSFNet http://ramat-negev.org.il/tutorials/luc/history.html

5
Henry Edward Hardy, The History of the Net, http://www.vrx.net/usenet/thesis/hardy.html

6
Various, The New Hacker's Dictionary, http://www.tuxedo.org/ esr/jargon/jargon.html

7
Henry Spencer, News Article Format and Transmission, RFC1036-SON

8
Various Newsreaders.com, http://www.newsreaders.com

9
Jeroen Scheerder, GNKSA * The Good Net-Keeping Seal of Approval, http://www.gnksa.org

10
Virginia Shea, Netiquette Home Page, http://www.albion.com/netiquette/index.html

11
Dirk Nimmich, How do I quote correctly in Usenet?, http://www.netmeister.org/news/learn2quote.html

About this document ...

Usenet is still a strange place

This document was generated using the LaTeX2HTML translator Version 99.2beta6 (1.42)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -split 1 usenet.tex

The translation was initiated by Jan Schaumann on 2001-02-20


Jan Schaumann 2001-02-20