|
Usenet, what is it, how does it work?
Newsgroups began in 1979 as a forum in which UNIX users could
communicate with each other, and the concept grew steadily from
there into what is now a global assemblage of people sharing
information on virtually every topic imaginable.
Originally, news servers exchanged articles using UNIX-to-UNIX Copy
Protocol (UUCP), which involves direct modem dial-up over long-distance
phone lines. In 1986, the Network News Transport Protocol (NNTP) was
released, allowing news to be transported via TCP/IP connection over
the Internet. Most modern newsgroups use the NNTP protocol.
Newsgroups are scattered on servers around the world, and the rough
network used to carry newsgroup bandwidth is generally referred to as
Usenet. Usenet is a virtual anarchy that has become so large and diverse
that a simple definition cannot possibly do justice.
Basically, the administrator of your news server determines which news
feeds you will have access to. Feeds are passed along to the server from
adjacent servers, providing a decidedly decentralized structure to Usenet.
Each server maintains a list of message IDs to ensure that new articles are
received at a given server only once. An individual server can control which
feeds it propagates, although the interconnectivity of Usenet servers ensures
that a lone server has little or no control of the overall distribution.
Thus, the authority of a news server is generally limited to what clients
(that would be you) can access and what kind of material those clients can post.
Likewise, the decentralization of servers means that an article you post may take
hours—or even days—to circulate among all other news servers.
Censorship, Big Brother, and NNTP Servers
News feeds are censored for a variety of reasons. For example, your company's server
may restrict feeds from alt., rec., and talk. groups to reduce the number of work hours
lost to employee abuse or simply to reduce bandwidth. Many other servers restrict feeds
that contain pornographic content for both legal and moral reasons.
Even if your news server provides a relatively unrestricted news feed, you should exercise care
when deciding which articles you download from the server. Virtually all servers maintain logs of
the activities of each login account. This means that your service provider can track which articles
you download, and in most cases these logs can be subpoenaed and used against you in court.
In other words, Big Brother might be watching you download porn, bomb making instructions,
and bootleg copies of "Go tell dat". Be especially paranoid if you access a company
news server; hours spent receiving otherwise legal content such as fruit cake recipes, Bill Gates
jokes, and the like could still land you in hot water if the boss is logging your online activities.
Locating News Servers
Many ISPs and companies provide news server accounts to their Internet users, but you still might
find yourself looking for a server on your own. This may be the case even if you have a news account
available to you; some service providers censor the news content that is available, and if you want
uncensored or anonymous access to news, you must rely on a different source.
Note
The terms newsgroup and Usenet are used almost interchangeably in today's
online world, but it is useful to know that newsgroup refers to individual
groups, whereas Usenet refers to the entire network of groups as a whole.
submitted by Goth
PAR file what is it?
PAR files use Reed-Solomon error-correction codes, to create redundant data (PAR file).
In the case of loss or corruption of data in the original data files, the
data may be reconstructed from the PAR files.
To more fully understand PAR files we need to understand their origins. Irving S. Reed and Gustave Solomon, in 1960 while staff
members at MIT's Lincoln Laboratory, wrote a five-page paper that appeared in 1960 in the
Journal of the Society for Industrial and Applied Mathematics, that introduced ideas that form the core of current error-correcting techniques
for everything from computer hard disk drives to CD players. Reed-Solomon codes (plus a lot of engineering wizardry,
of course) made possible the stunning pictures of the outer planets sent back by Voyager II. They make it possible to
scratch a compact disc and still enjoy the music. And in the not-too-distant future, they will enable the profit mongers
of cable television to squeeze more than 500 channels into their systems.
The payoff of their paper was a coding system based on groups of bits--such as bytes--rather than individual 0s and 1s.
That feature makes Reed-Solomon codes particularly good at dealing with "bursts" of errors: Six consecutive
bit errors, for example, can affect at most two bytes. Thus, even a double-error-correction version of a Reed-
Solomon code can provide a comfortable safety factor. (Current implementations of Reed-Solomon codes in CD
technology are able to cope with error bursts as long as 4000 consecutive bits.)
Conceptually, the Reed-Solomon code specifies a polynomial by "plotting" a large number of points. And just as
the eye can recognize and correct for a couple of "bad" points in what is otherwise clearly a smooth parabola, the
Reed-Solomon code can spot incorrect values of P and still recover the original message. A modicum of combinatorial
reasoning (and a bit of linear algebra) establishes that this approach can cope with up to s errors, as long as m, the
message length, is strictly less than N - 2s.
A great feature of PAR files is the versatility of the files. In many ways,
PAR files are "chameleon" files, having the ability to reconstruct any one
of the missing files- whether part 1 or part 71 is missing- as long as you
have as many PAR files as missing files, the data can be reconstructed!
Creating PAR files for use with archives that are to be posted is simple
with a program like SmartPar or WinRar(V3.0 or higher).
Reed, now a professor of electrical engineering at the University of Southern California, is still working on problems in
coding theory. Solomon, recently retired from the Hughes Aircraft Company, consults for the Jet Propulsion Laboratory.
Reed was among the first to recognize the significance of abstract algebra as the basis for error-correcting codes.
read more about Reed-Solomon:
http://www.siam.org/siamnews/mtc/mtc193.htm
submitted by Goth
What is CRC/CRC32?
CRC stands for Cyclic Redundancy Check. The 32 comes from the fact it calculates a 32-bit checksum.
CRC32 is an algorithm for calculating a unique identifier for a file.
It's used in programs like yEnc32 and PKZIP to identify files and make sure that they are original.
A CRC performs a mathematical calculation on a block of data and returns a
number that represents the content and organization of that data. The idea
is to have the CRC return a number that uniquely identifies the data. You
can think of CRC as being the operation that generates a "fingerprint" for
a block of data. The actual number, or fingerprint, that is used to identify
the data is called a checksum. The following illustrates the flow of a CRC.
Data ---> CRC = Checksum
So, why perform CRCs on data? Well, by comparing the checksum of a block of
data to another block of data's checksum, you can determine if the data is
an exact match or not. CRCs are mostly performed when transferring files
from one location to another. Depending on the medium by which files are
transferred, errors to data may occur during the transmission. In mission
critical applications, it may be especially important to know that files
are valid and reliable. Most networking protocols use CRCs to verify data
received is the same as the data that was sent.
Verifying transmitted information, sending and receiving records,
modifying files and records, and verifying emails,
are a few of the reasons for which a person would want to use CRC.
submitted by Goth
What is yEnc?
The yEnc format is a new encoding method and an alternative to UUencode and Base64 encoding. It's primarily used in Usenet newsgroup posts but can also be
used for eMail and other applications. yEncoded
messages are up to 40% smaller than UUencoded/Base64 equivalents, and include built-in
CRC error checking and multipart file support.
News and Mail transfer require that a binary attachment is "encoded" before it is sent.
And they are "decoded" after they have been received. Normally all this is done by your
newsreader or mail-program (you don't see it, it's transparent).
The encoding is necessary because the special methods for the transfer of news & mail
protocols require it. A message with a binary which is not encoded is corrupted during transmission - or not transmitted at all.
To understand yEnc and why you might want to use it requires a little history. Transport of messages by
News and Mail was restricted to US-ASCII characters when the protocols were written 20 years ago.
These services have been created to transport only plain US-text. Special characters (control-characters,
symbols, non-US-characters) were forbidden - and used for special purposes. But because people wanted
to send also binary attachments by News and Mail some 'tricks' were implemented: The binary was changed
to "allowed US-ASCII-characters" before transmission (encoding) - and back to a binary after transmission (decoding).
The usual encoding methods are still respecting these old limitations - and are used everywhere.
Unfortunately there is a price for this 'trick': Encoding makes a message longer. And not just a little,
but 33%-40% longer than the original attachments. This results in 33%-40% more bytes for a message -
33%-40% more time for the transmission - 33-40% more diskspace on the harddisk where there
messages are stored (on news- and mail-servers).
Meanwhile Usenet is able to to transport more than "US-ASCII", it could also transport other characters.
Just a few special characters are still forbidden. Unfortunately the encodings were never changed. We are
all still using BASE64, BinHex, UUencode. So if you post large binaries you are wasting bandwidth, time, diskspace
and in the case of those that pay by volume, money.
yEnc is now a proposed encoding method which is using the fact that news-servers can today transport
binaries more efficient. On eMail the situation is far more complicated because there are a lot of older
programs and computers involved. But also there would be potential for savings.
submitted by Goth
| 
