Illustration/Eric Frommelt

DIG Sheds Light onto the Dark Web’s Shadows, Connects the Dots on Gun Traffickers

The Dark Web provides the anonymity gun traffickers crave

It isn’t at all surprising that so much of the illegal weapons trade now takes place on the Dark Web, the shadowy parallel Internet system where criminals operate vast, untraceable, anonymous marketplaces for outlawed goods and services. Gunrunners, whether global or just down the block, thrive in anonymity. They usually steer clear of meeting customers in person and never take even remotely traceable payments.

Until now, it was nearly impossible for law enforcement agencies to fight the booming illegal gun market, but that is about to change.


Craig Knoblock and Pedro Szekely of USC Viterbi’s Information Sciences Institute and the Department of Computer Science have developed DIG, or Domain-specific Insight Graphs. This groundbreaking, cloud-based analytical tool is part of Memex, a special browser developed by DARPA to help law enforcement enter the heretofore unsearchable Dark Web to find, extract and correlate the hidden information gunrunners and other traffickers inadvertently leave behind.

Memex is now being tested by six police departments and law enforcement agencies, which, according to Knoblock, could expand to as many as 200 by the end of 2016. Memex’s early successes have led Manhattan District Attorney Cyrus Vance Jr. to describe it to “60 Minutes” as “Google on steroids.”

“The DIG software is being used by several law enforcement agents to investigate human trafficking cases, but its potential is equally strong in thwarting and catching gunrunners,” Szekely said.

Memex needs to be powerful because the full Internet, beyond the “surface web” that most of us never get beyond, is a vast ocean of data, where conventional indexing and searching doesn’t work. That’s because the data is either too obscure, too vast or, in the case of the Dark Web, because it is encrypted and kept inside a private “anonymity network.”

There are approximately 300 million guns in the U.S., according to a recent Bureau of Alcohol, Tobacco and Firearms study. Each year, half a million are stolen. They used to get sold in backrooms and empty parking lots. Now they’re sold on the Dark Web, experts say.

Behind most gunrunning is a host of thousands of small, unlicensed gun dealers operating out of their garages or basements. The weapons are easy to conceal, particularly as power tools, and ship abroad. With the Dark Web, American gunrunners suddenly have global reach. Hunting them means bringing together dissimilar pieces of information from different places and finding how they connect.

Using DIG, Memex can automatically cross-reference huge global databases, allowing international law enforcement agencies to work together to track down gunrunning rings. However, Memex can also work on a local level by connecting the guns used in violent crimes with the “straw buyers” who initially legally bought them.

“Let’s say there’s a news story about a firearms store being broken into or antique guns being stolen from somebody’s house,” Szekely said. “You try to see if they can be found on the Dark Web. Is somebody selling similar guns? Are they the same guns or is it a coincidence? The idea is to be able to cross-reference information we get from the open web to information that we find from the Dark Web.”


Every day, hundreds of individuals, mostly women and children, disappear into the twilight world of human trafficking. Once in, they become objects of abuse, degradation and sexual commerce. Few come out alive. Finding them is difficult because their identities are discarded and they are constantly moved about. Even though they are recruited and marketed openly via Internet forums, chats and job postings, much of this activity is fragmented and takes place in the “Deep Web,” the vast regions of the Internet not indexed by conventional search engines.

That is about to change. Craig Knoblock and Pedro Szekely of the USC Viterbi School of Engineering’s Information Sciences Institute (ISI) and Department of Computer Science have developed a groundbreaking, cloud-based analytics tool designed to reveal the identities of sex traffickers. Domain-specific Insight Graphs, or DIG, uses open-source software to identify, extract, correlate and then visually express vast amounts of information taken from these hidden sources on the Dark Web and reveals patterns that could lead police to the location of sex traffickers and their victims. DIG has already assembled a database of more than 50 million web pages and is growing at the rate of 5,000 pages each hour.

DIG was developed as part of the DARPA’s Memex initiative, which seeks to develop software and search technologies for countering human trafficking, the illegal arms trade and terrorism. It is currently being tested by six police departments and could expand to as many as 200 this year. “There is a lot of interest from law enforcement agencies,” Szekely said.