Usenet?

[Admiral Nelson]

I've searched on wikipedia and on google, and I still don't really understand what the "Usenet" network is.

So far I have only gleaned that it involves some sort of discussion, so is it like a really big forum you have to pay for? What is it useful for? Why would people pay for this service?

[pannonian]

Usenet is a kind of global, decentralised bulletin board. Google groups is one portal for Usenet, but you might want to try others. Different forums (aka newsgroups) cater for different interests, and these forums are organised into hierarchies to make finding a suitable forum easier. For instance, rec.sport.tennis is one of the livelier tennis forums around.

While web forums such as TWC have centralised control of data, Usenet posts get sent out into the ether, where they are stored in various Usenet servers. Your ISP or a specialist Usenet provider will give you access to these servers, and you can download the posts from selected newsgroups to your local machine. In the traditional newsgroups, these posts are text only, and you can view them in whatever format is to your taste. Google groups is one method, Outlook Express is another, while other specialist newsreaders that format the text for easier reading exist.

Note: because control of the forum is decentralised, you do not have a centrally controlled account like Soviet Warlord here, nor can you edit posts and stuff like that. Once you send your post into the outside world, it is beyond your control. As you can imagine, this allows for great freedom of speech, as, except for rare exceptions, there will be no moderators to control what you can write. Note that others will be similarly unrestrained though, so it is best to lurk for a few weeks, getting used to the atmosphere of the group, before you dip your toe in. Otherwise, if you rush in and lose all credibility from the start, the regulars will just killfile you - a similar feature to the Ignore List here, except they won't even be aware of your existence once you're in their killfile. As you can imagine, being ignored by everyone isn't much fun.

Despite all that, the Usenet crowd is generally older and more mature than those in web forums, so if you want to listen and learn (as I did back then), it's a good place to go. If you are polite and behave yourself, you should be accepted by the regulars within a few months, until you're considered one of them yourself.

[Admiral Nelson]

Thanks for the info!

Though why are some usenet groups free, whereas others you must pay a monthly fee to use? (such as this one)

[pannonian]

Thanks for the info!

Though why are some usenet groups free, whereas others you must pay a monthly fee to use? (such as this one)

Because of the huge amount of traffic, Usenet servers clear out old data every so often - in Giganews' case, stuff older than 200 days is deleted. Remember this isn't just a single forum we're talking about, but data from every forum in the Usenet forum network, so we're probably talking about tens or hundreds of terabytes of data being stored. This is exacerbated by other factors which I'm not sure I can discuss here. The Usenet service provided by ISPs is patchy at best, and have a considerably shorter retention time (30 days, if that, sometimes a week). Despite not having used it in some time, I still retain a subscription with Berlin University (I think), which provides excellent access to text-only newsgroups and very good retention at a barely noticeable annual cost. 20 euros (IIRC) a year ain't exactly pocket-burning.

[Simetrical]

For Google, on the other hand, tens or hundreds of terabytes can be provided by scrounging for spare disks under the sofa. I'd assume that if their free Usenet coverage is not absolutely complete, this is due to access issues rather than disk space. I don't know much about Usenet, so what are the advantages to a paid service instead of Google (unless you really hate ads)?

Incidentally, a million messages at 1 KB each compressed would be only a gigabyte. Even if Usenet has averaged a million messages a day over its lifetime, it wouldn't have quite reached ten terabytes yet, especially not compressed. Ten terabytes of storage would today cost something like a couple thousand dollars, hardly a huge price by itself. TWC has half a terabyte of usable storage, and that's on pretty fast disks with redundancy (RAID-10 on four SATAs). Twenty euros times hundreds or thousands of subscriptions seems like way more than would be needed to retain the entire history of Usenet and serve it quickly upon request.

(Of course, if the other factors you're alluding to involve the transmission of large files, that's different in terms of storage requirements. But you remark that your newsgroup service for one is text-only, and I'm assuming that also excludes large files masquerading as Base64 or somesuch.)

[pannonian]

Unlike most web forums, Usenet posts aren't collected on a single server which can control the flow of data, but are distributed as is to the big internet servers (if I've grasped the tech right), and from there they are distributed to any other intervening servers that might demand them. AFAIK there isn't any collective effort to check whether or not the data has been transmitted or received successfully at any stage beyond your initial posting, so the network is unreliable. The way that Usenet posts get distributed to the wider network is called propagation, and there's a host of theories on how to get around its inherent problems. AFAIK Berlin University, the Usenet provider I use, accesses a variety of sources or something and compiles the mass of data that ensues to create what's probably the most comprehensive collation of text-only Usenet posts anywhere. In contrast, Usenet provision comes only as an afterthought to most ISPs, and little resources are set aside for its use, so barebones Usenet as seen by those who stick to what their ISPs provide is extremely poor in all aspects.

About the terabytes: the answer to your last paragraph is yes. I'm not going answer in any more detail without going into areas that are explicitly forbidden here.

What's wrong with Google Groups? Its interface is abysmal, it encourages users to see Usenet as something it isn't, and its propagation AFAICS is one of the worse I've seen.

Specialised newsreaders, or even the much maligned Outlook Express (aka Virus Express) can format the text in a post to suit the user, and have tools to make browsing newsgroups easier. Google groups used to have a tolerable reading window, but they've recently dispensed even with that, they now have an interface that combines the worst features of every forum I've ever seen, all unadjustable.

There's a running joke about the introduction of AOLers to the world of Usenet, and how this led to the September that never ended. AOL has discontinued Usenet access, but Google Groups has more than ably taken up the baton. I've flamed my fair share of noobs in the past for ignorance of Usenet conventions, thanks to how Outlook Express is set up, but Google Groups takes this to another level, actively encouraging users to blur the difference between web and Usenet forums. Gah.

The Google Groups archives were preceded by something called Dejanews, which was one of the first (possibly the first) to compile the entire history of Usenet, as far as could be compiled. When Google took over, one of the first things I noticed was that a particularly infamous thread I took part in was missing some 5-10% of its posts. Searching for keywords didn't bring much joy either. Since then, I've dipped into my former groups from time to time, and noticed that threads felt disjointed as posts were missing all over the place.

Finally, Google Groups misses what's probably the most important feature necessary for keeping one's sanity in that place - the killfile, aka the trash can, plonkfile or bozo bin. It works a tad like the Ignore List, but you won't even see the header of the post that's been blocked - it would seem to you as if the post never existed in the first place. With it, one can exercise a certain amount of control over the anarchy that is Usenet. Without it, one has to endure all the flames, trolls, spam and all those other unsavoury terms that had their origins in Usenet history.

Still, for seekers of knowledge, there are few better places to go on the internet than the appropriate newsgroup.

[Simetrical]

Interesting.

[Admiral Nelson]

So can Usenet be used as a sort of high-speed Bittorrent where uploading to others isn't necessary? (Note this does not necessarily involve copyright infringement)

[Simetrical]

It would probably be slower than Bittorrent, I'd imagine, unless you paid for a high-bandwidth Usenet subscription. The whole reason Bittorrent is relatively fast and cheap is because it's peer-to-peer and requires uploading to others (thus making more bandwidth available for others to download).

[Kanaric]

Usenet is not that slow, I used to use it before Bittorrent was out for years. Its slower than if you use a private tracker with servers hosting files, but sometimes faster than if your downloading something from torrentspy that doesn't have many seeds. It really depends.

Files, at least then, were always in segments though and you had to download them one at a time. Ever see a torrent file with a bunch of .rar .r00 .r01 .r02? That is because its from usenet.


Another good place, at least before torrent, to download from used to be IRC. I used that to get files and the speed of that depended on the host of the files. Usually this was faster than bittorrent but these irc channels were hard to find since you would need both the server AND the channel so someone would had to of posted it.