31.1.15

The Deep and Dark Web



In my free time I like to browse the Internet - just for fun. Perhaps I watch a short movie on YouTube, or I tune into a German television network to watch the evening news. I like to browse on StumpleUpon, which suggests random websites based on my interests. 

Or I simply have a drink and listen to some tunes.

However, I didn't know that that which in my innocence I considered to be the Internet makes up only 20 % of all the content that's available.

You read that right, all those giant sites and networks like Google, Facebook, Twitter, all the media outlets, all that which YOU are familiar with makes up only twenty percent of available content. The rest is 'hidden' on the Deep Web.

I find that staggering.

Here are a couple of facts:
  • General public info on the deep Web is actually 4 hundred to 5 hundred times larger compared to that which is generally characterized as the World-Wide-Web
  • The deep Web consists of 6,500 terabytes of information in comparison to 20 terabytes of information that is accessible in the surface Web.
  • The deep Web is made up of closely 550 billion unique records and documents in contrast to the 1 billion from the surface Web.
  • A lot more than 200,000 deep Internet sites currently are present.
  • 60 of the largest deep-Websites mutually consist of 750 terabytes of data — enough by themselves in order to surpass the size of the public Web by 40 times
  • The Deep Web is actually the largest expanding division of fresh information on the Internet

Origin
The Deep Web was created originally by the US government . In the nineties, under President Bill Clinton,  security analysts in the White House started to ask themselves how they could protect government information which was not meant for the general public from hackers.

They devised a way by which Internet traffic could flow not through major servers, but through a network of personal computers that participated in the project.

How does it work?
When you are browsing the Deep Web, your query goes to a random computer that is part of the network (a 'node'), and is encrypted on the way. From that computer it goes to another computer, maybe in South Africa, and it gets another layer of encryption. And off it goes from there maybe to Australia, and is encrypted once again.

The sum of the encryption is like the layers of an onion, and that's why the whole network sometimes is called 'the onion network'.

Once your query is at its end-point, it gets decrypted, and you can view the content anonymously.

The Aspect of Anonymity
You are probably aware that absolutely everything you do on the Internet is being recorded (at least temporarily) and can be traced back to you, via your IP address.

That's very handy on the one hand for Government security agencies like the NSA in the United States, its equivalent in the United Kingdom, the GCHQ, and security agencies all over the world, like in China or Russia.

But let's just assume that they are not interested in you.

However, the big sites on the Surface Web (that which is commonly, and wrongly, thought of as THE Internet) are very, very much interested in which sites you browse, which search terms you use on Google or other search engines. They are generally interested in you, your habits, your likes and dislikes.

Why is that? Simple, they want to know which advertisements to show you, because that's where sites like Google and Facebook get their revenue from. Yes, these sites are all FREE, great! However, for you to use them you have to at least redefine your definition of privacy.

Let's take this young lady in Ireland who searches innocently for the term 'pregnancy test'. That information is immediately sold on special sites to advertisers. It goes, in fact, to the highest bidder, usually those who buy this kind of information in bulk.

And minutes after this young lady made her search for 'pregnancy test', she is being shown all kinds of advertising for baby clothes, eBooks on how to raise a child, how to feed a child, how to keep your child healthy, you name it.

And it is all done legally. In fact, that's how the Surface Web works. You and your browsing habits become the target. And make no mistake, they don't need to know your name or birthday. A whole lot of information about you can be gotten just by looking on what you do on the Internet.

It works about the same way as if a government agency is tapping a certain cell phone. They do need a court order for them to listen to the calls. However, they can simply record the numbers that are being dialed from that phone, and from which numbers that phone is receiving  calls. And in most countries they do not need a court order, general legislation takes care of it.

By simply cross-referencing the calls that person makes, at what time the calls are made, how frequently they are made and from which location, they can garner lots of information about that individual.

The same with your browsing habits. Again, this is not about the police or some security agency shadowing you, but multimillion dollar Internet companies.

The Tor Network
The Tor network, which is one of the portals to the Deep Web, was and still is used by millions of users who don't want the world to know what they are doing on the Internet. Think of people like Edward Snowden or the folks at WikiLeaks. They couldn't do what they are doing by sending their kinds of documents via Gmail!

And, most importantly, it is being used by activists in countries with repressive regimes, like China, Russia and Syria. In fact, the Tor Network played a decisive role during the Arab Spring in 2010.

The activists can't organize themselves and communicate with each other over Facebook obviously,  because that leaves a trail via their IP address, which the police of the respective country would be only too happy to follow, with dire consequences for the user. In countries like Syria, for example, it would mean torture and death.

The Dark Side of the Deep Web
Since it is impossible to trace any activity on the Deep Web, it contains thousands of sites with a criminal purpose. We are talking here about sites that show and sell child pornography, for example, and a whole lot of other unsavory content.

One of them is the Silk Road, where you can buy drugs, hire prostitutes, buy PayPal accounts with their login details, you name it.

So here my WARNING, if you want to check out the network, and be it just for academic purposes, there are a lot of sites which YOU DO NOT WANT TO SEE.

So don't open them. I have been warned against it and I'm heeding the warning.

But since I spend a very great deal of my time on the Internet, I felt that I had to at least check it out.

And I believe we all should at least know what's really going on.

How to access the Deep Web.
First of all, you have to download the Tor browser, which you can do here.

Then you can do a little test. In your normal browser check your IP address here. The site will give you your IP address and your general location.

Now copy and paste the site which lets you find your IP address into the Tor browser. And oops, suddenly you appear to be in Switzerland or Papua New Guinea.

The best known directory for content on the Deep Web is Hidden Wiki 
(http://zqktlwi4fecvo6ri.onion/wiki/index.php/Main_Page ) 
You have to copy and paste that, since it's part of the onion network and doesn't show on the Surface Web.

You will see that browsing the Tor network will remind you of the beginning of the Internet (if you are old enough, that is). It's pretty slow, images take their time to load, the sites are pretty basic, without graphics - it's like taking a time machine back to the beginning of the nineties.

Conclusion
I myself downloaded the browser and had a look around. I wanted to basically confirm that I'm spending my time in a tiny niche, the Surface Web, which, again, makes up only twenty percent of generally available content.

But my thirst for information has been stilled, and I'm not using it. I believe, however, that it is an excellent tool for political activists all over the world who are risking their lives to bring freedom to their countries.

1 comment:

  1. Georg, that's pretty disturbing stuff. Takes me back to a work companion of 1997 asking seriously, "would you really go on the internet?" That the threat to privacy was enormous. I replied that if the choice was between isolation and information, I was prepared to take the risk. (He was not. Or at least if he's changed his mind he has not told me.)

    But what I HAVE found since, is that in a one to one internet relationship, exchanging information is like squeezing blood out of the proverbial stone. Users are reticent to offer or accept what-ever is said that differs from their conventional wisdom, and that doesn't include the overwhelming problem of international comprehension. For example having discovered there are TWO Phrenic Nerves that control the diaphragm, and mapped their pathes, custom made this impossible for broad range of people to accept. That put a stop to efforts to prove whether or not the diaphragm halves could be independantly controlled.

    So now I wonder at the validity of topic association gathered by Google and the rest. The rampant rigidity and confusion factor must generate some wierd conclusions.

    ReplyDelete