A visitors’ guide to the Dark Web

A visitors’ guide to the Dark Web

Whether for privacy or profit, much of the internet is hidden away – we reveal the places that Google won’t take you.

Over the past two decades, we’ve created a whole new world that’s visible only through a screen. Visibility and transparency have long been hallmarks of the online world. We’re used to being able to see anything posted on the web, and we’re used to the idea that anything we do there may be visible to anyone else.

That’s not true though. To understand why, consider how you find things online. It’s tempting to think that anything anyone has ever published can be magicked to your screen with the right keywords. That’s partly true – if Google or Bing has indexed it, and your internet service provider (ISP) lets you go there, you’ll find it. But those “ifs” are bigger than they may seem. It turns out that an even greater proportion of the online world is dark than of the physical universe: it is estimated that just 4 per cent of the web is searchable.

Like dark matter and dark energy, we know the dark web is out there, but we can’t see it directly. Entire uncharted realms lie beyond the reach of the crawlers that tirelessly catalogue the web on behalf of search giants: databases, internet relay chat records, and the raw data behind research papers. Much of it is unsearchable, because most people aren’t searching for it. And if they do search for it, and don’t find it, they assume it’s not there to be found. “There’s this underlying assumption that if it’s not online it doesn’t exist,” says Ken Varnum, a systems manager at the University of Michigan library in Ann Arbor.

But it’s not just obscurities that may be overlooked. Search engines and online stores exist to make money, and to do so they make constant, almost imperceptible suggestions about what you should be looking at. In some ways, this is benign: online shops prefer to direct you to products like those you’ve previously shown interest in. But the same algorithms can steer you away from things you might like to see. And then there are things the online giants insist you see. For example, last year Google banned an ad blocker program from its app store.

What’s visible and invisible online is not necessarily up to you: commercial interests can dictate what’s allowed on your screen. Legitimate businesses aren’t the only players in the invisibility game any more, though. Privacy enthusiasts have developed a variety of services that let their users tap into the dark web. One such service, Tor, covers your online tracks by blending your internet traffic into data from many servers worldwide to make you functionally invisible. It has notoriously been used by criminals, allowing, for example, the Silk Road black market to flourish for almost three years before the FBI took it down. But it wasn’t invented by those looking to break laws, nor is it their exclusive domain: it’s widely used by activists and others who are persecuted or living under surveillance. That latter epithet could be applied to an increasing number of us.

In the past year, we’ve learned that government agencies and marketing firms are able to keep amazingly detailed records of our online activities. This might be for the purposes of national security; or it might be to sell us more stuff. The problem is that most of the data collected this way is visible to those who collect it – but not to us. Fed up, in 2012 Caleb James DeLisle started Hyperboria, an alternative to the normal web. Like Tor, it binds people together in a network that severs the link between them and their IP addresses. Its denizens have built analogues of Twitter and Facebook there, allowing them to communicate and socialise without fear of snoopers. But Hyperboria is hard work, requiring far more than just a browser and an internet connection. You have to access your command prompt, install something called the cjdns protocol, and find someone willing to vouch for you and become the bridge that lets your computer connect to Hyperboria. This may explain why the user headcount still languishes in the hundreds.

Other anonymous networks are even less popular, although they’re growing in the wake of Edward Snowden’s revelations about NSA surveillance. All of these new darkling networks rely on the existing cables and servers that make up the standard internet, where ISPs are free to block traffic to them. To become credible alternatives, then, they will need their own wires. Like dark energy, we know the dark web is out there, but we can’t see it directly and switches to carry their traffic. This is beginning to happen.

Across the US, small groups of dedicated hackers are building their own wireless networks to run Hyperboria on. In Portland, Oregon, you can now send an email across town without ever connecting to the wider internet. But can these alternatives win over the path of least resistance? Perhaps. “Expediency is a very powerful thing,” says Varnum. Technologies are being developed to make invisibility easier. The Enigmabox, designed to run on any internet connection, automatically encrypts its users’ email. Users don’t need to understand the €283 box; they just plug it into their router. Two hundred people are now on the network, says Enigmabox founder Claude Hohl, based in Switzerland. While Enigmabox doesn’t yet talk to the other independent networks – such as the Portland meshnet or Hyperboria – they’re working on it. “When every alternative network connects to all the others, that’s when you start to have a global network,” he says: a true alternative to the internet. Once it gets big enough, this dark internet will no doubt attract attention from the very forces that have driven people into its depths. But DeLisle says Hyperboria won’t be as quick to succumb to centralised control as today’s internet, “which requires central authorities controlling it”.

Much more than our surfing habits depends on the outcome of this struggle. The online world is increasingly converging with the physical world. Apps annotate the space around us, revealing places to eat and people to meet. We can look into the past with apps that overlay history on what we see; or the future. Through our screens, we can see the flow of city traffic, or the positions of stars behind a cloudy sky – and those screens may soon be perched on our noses: before our very eyes. Technology may soon let us see almost anything we can imagine. Whether we will depends on whether we fulfil the decades-old promise of the internet – to make the world more visible, not less so.