Dupe checks first appeared on BBSes as a way for sysops to nuke uploads. When files get moved offline, the DIZ files are kept as a record of the releases' existence.
Midsorrow (40 gig BBS sysop)
I've always pretty much just relied on my memory to let me know which new uploads are dupes of something already online, but even then, dupes do sometimes slip through the cracks (and on a 40 gig system like the one I run, it's getting QUITE difficult to dupe-check solely by memory).
Life on the BBS was a bit slower, so it was possible for a sysop to keep it all straight. As people moved to the ftp sites, things sped up considerably. Fast connections spread files quicker, and the growth of the software industry meant there were more releases to spread. With around 60 releases a day now, it's almost impossible for sysops, couriers, and release groups to keep it all straight in their heads.
Enter the modern dupe check. As of this writing, there are three ways to check for dupes. The first, and the simplest to use, is the irc dupe check. Several varieties exist, with the most popular dupe checks being run out of #releases and #thescene. #releases's bot "Dupe" responds to queries in the form of "!dupe" followed by a search string, either in the channel or in a private message to the bot. Dupe will respond with a message containing titles that match your query and the dates that they were released. Limitations of IRC prevent dupe bots from returning all hits to the searcher, though ops in #releases are excluded from this limitation.
Release records for these dupe bots come from prominent ftp sites. When a courier uploads a release to one of those ftp sites, the system automatically adds the information about the upload to the database and makes it available to the irc bot. Search strings can be part of the name, the group that released it, or even the date that the program was released on.
...
About a month ago katman, a siteop on QC, released a beta for a new kind of dupe check. Katman's dupe check, named "WinDupe," is a native win95 program that connects the user to an SQL database. Although the user is required to run a separate program, WinDupe offers greater speed and flexibility over irc dupe checkers because of its direct link to an SQL database. WinDupe is faster then irc dupe checks, returns all hits for queries, and allows users to manipulate those returns a little easier. WinDupe is presently in its 6th beta, which can be downloaded here.
...
The last alternative for dupe checkers is the web. DOD gets credit for creating the first web dupe check, which seems roughly comparable in quality to WinDupe. www.nrg2000.com offers an alternative to DOD's, though the page is a bit more disorganized.
Scenelink itself will enter the somewhat crowded field of dupe checkers in the near future. Our "Dupelink" will be web based, and similar to the existing web dupe checks. The major differences will be that entries in Dupelink will include links to nfos, a total size, and a listing of every file included in the release.
- mead