I was recently asked, in the context of the ongoing Phorm debacle, and with other interested parties, to meet with members of the UK government and discuss deep packet inspection technologies, and their impact on the web. I’m still organizing my thoughts on the subject – I’ve put some here, but I’d love to know where else you think I should look to ensure I have considered the relevant angles.
Brief Background
Phorm‘s technology hooks in at the ISP level, watches and logs user traffic, and uses it to assemble a comprehensive profile for targeting advertising. While an opt-out mechanism was provided, many users have complained that there was no notice, or that it was insufficiently clear what was going on. NebuAd, another company with a similar product, has apparently used its position at the ISP level to not only observe, but also to inject content into the pages before they reached the user. It’s hard to get unbiased information here, but this is what I understand of the situation.
Thoughts
1. Deep packet inspection, in the general case, is a neutral technology. Some technologies are malicious by design (virus code, for instance), but I think DPI has as many positive uses as negative. DPI can let an ISP make better quality of service decisions, and can be done with the full knowledge and support of its users. I don’t think DPI, as a technology, should be treated as insidious.
2. Using deep packet inspection to assemble comprehensive browsing profiles of users without explicit opt-in is substantially more questionable. My browsing history and habits are things I consider private in aggregate, even though any single visit is obviously visible to the site I’m browsing.
It’s possible that I will choose to allow this surveillance in exchange for other things I value, but it must be a deliberate exchange. I would want to have that choice in an explicit way, and not to be opted in by default, even for aggregate data. Moreover, given the complexity of this technology, I would want a great deal of care to go into the quality of the explanation. Explaining this well to non-technical users might be so difficult as to be impossible, which is why it’s so important that it be opt-in.
3. Using deep packet inspection in conjunction with software that modifies the resultant pages to include, for instance, extra advertising content, is profoundly offensive and undermines the web. The content provider and the user have a reasonable expectation that no one else is modifying the content, and a typical user should not be expected to understand the mechanics of the web sufficiently to be able to anticipate such modifications.
Solutions
As a browser, we do some things to help our users here, but we can’t solve the problem. https resists this kind of surveillance and tampering well, but requires sites to provide 100% of their content over SSL. Technologies like signed http content would prevent tampering, if not surveillance, but once again assume that sites (and browsers!) will support the technology. Ad blockers can turn off any injected ads, tools like NoScript can de-fang any injected javascript but, fundamentally, http content is not tamper-proof, and no plaintext protocol is eavesdropping proof.
People trust their ISPs with a huge amount of very personal data. It’s fine to say that customers should vote with their feet if their ISP is breaking that trust, but in many areas, the list of available ISPs is small, and so the need for prudence on the part of ISPs is large.
That’s what I’m thinking so far, what am I missing?
meandering wildly 
A couple podcasts covering the details and politics of it.
That these interactions, once the profile is extracted, are not limited to the Web, and can have arbitrary realworld consequences depending on who the data gets shared with and what those people’s intentions are.
Actually, NoScript cannot defang injected JavaScript, since it works on a per-URL basis, and injected JS doesn’t necessarily have an independent URL. In other words, if the JS is directly injected into a page you trust, the JS will be executed, no matter how harmful.
people with deep pockets (read, music / movie majors) want this to see if you wouldn’t be “pirating” their prized crap, by chance…
then they contend with getting the state to lock up your internet access up to a year so that you, evil “pirate” don’t do it again, which may get you to buy the crap instead of downloading it (yeah, fat chance)
it’s called “riposte graduee” here in france, and about to go to the house for discussion…
see http://www.laquadrature.net/ for more information
Its a popular misconception that web pages are somehow broadcast.
Web pages are delivered over a private personal connection with a web site, in a similar way to a phone call. Both use a shared public data communication network. Both are unencrypted. Both are private ‘conversations’.
Consequently, a web site may selectively decline or accept requests. The sueqnce content delivered is specific and personal to a given person. The content itself may also be specific and personal to a given personal (for example ecommerce, forums, webapps, webmail).
There are profound implications for anyone who publishes online, any community online, and electronic commerce in particular. UK ISPs/communication companies risk sacrificing their status as trusted couriers of data. (The concept of ‘common carrier’).
A significant amount of commercial intelligence can be gleaned from monitoring a person’s interaction with even a simplistic web site. It is this economic intelligence which is used to sell competitive advertising. Monitoring in this way is mass industrial espionage, identifying customers or prospective customers of a given web site/business, and selling competitor products/services instead.
The only available response is complete encryption, and/or selectively blocking untrusted ISPs from your site.
Even encryption doesn’t prevent your customers being identified by IP address.
Using DPI for marketing will radically change the future of the internet, and digital communication. For example, there are security implications for the wider community; do we really want a world where the only circumstance in which anyone can communicate privately requires strong encryption? How will that affect the need to conduct legal interception if you can’t decode the traffic in real time?
Longer term, there are implications for all forms of unencrypted B2C/C2C/B2B digital communication. Email. VoIP. SMS. P2P. PSTN. Once you concede interception of a given method of private communication, it is very difficult to make a coherent argument for not intercepting other communication methods (it is all ones and zeros on a wire crossing a public communcation network).
Very much looking forward to meeting you in London.
Your description of Phorm’s activity shows the classic question regarding technology. You nail it when you say “Deep packet inspection, in the general case, is a neutral technology.” All innovations can be used as a tool or a weapon. It just depends on who is wielding it and why. A knife’s usefulness or threat level depends on who carries it — the same object can be a tool for chopping vegetables or a murder weapon. DPI is no different.
If a user explicitly grants permission, then DPI is a tool, possibly a very helpful tool. If a user would like to see ads for products and services that they might actually use and enjoy, this can be very helpful. It could even possibly reduce the appearance of unwanted ads for a user. For example, some people complain about steamy, even pornographic ads that they see online. In theory, if a person’s browsing history reflects no interest in online dating or porn, over time they would be less likely to be presented with ads for that part of the Web that doesn’t interest them.
Without an explicit opt-in process, this technology is an assault on the user’s privacy. Nobody wants someone spying on them, and without inviting DPI, the helpful little program scribbling down your every move turns into a creepy digital stalker. It becomes even more concerning when the fate of all of this collected information is unclear. Can it ever be sold? Viewed by third parties? Compromised by identity theives? A program outside of a user’s control that assembles a specific profile of that person’s online habits presents a substantial liability, and it should be up to the user to determine if that level of risk is worth the convenience of targeted advertising.
DPI is a highly technical concept with no brief explanation possible to a person who doesn’t understand anything more complex about software than “point and click.” It’s extremely unethical to push an optional, risky technology on users without allowing them to understand what they are signing on for. Advertisers can still get their ads out there without cyberstalking users.
Fighting for an opt-in process seems even more important in a country like Britain, where there are far fewer options when it comes to ISPs. British Internet users don’t have as much flexibility as the U.S. when it comes to switching services if they disapprove of what they’re getting. With BT centrally holding the reins on most Web traffic in the UK, someone has to protect the users from losing control over their own information.