Whirlpool

Exetel etc ?

At the ISP end how do they know what traffic is used by P2P clients or servers ?

Cheers,
Rich

Have a read of this
en.wikipedia.org/wiki/BitTorrent
Bookmark it and read some more from time to time.

Its about your IP address and the Hosts (of the tracker) IP Address.

BitTorrent does not offer its users anonymity. It is possible to obtain the IP addresses of all current, and possibly previous, participants in a swarm from the tracker.

then take a gander at this:
www.azureuswiki.com/index.php/Encryption

then
www.azureuswiki.com/index.php/SafePeer

and this:
www.afterdawn.com/softwa...peerguardian.cfm

I have trouble answering my own question from the article :P

How would they keep a table of all well-known trackers ?

They couldn't filter by port, or could they ?

fons writes...

filter by port

yup

edit: more like throttle

fons writes...

How would they keep a table of all well-known trackers ?

they wouldnt waste there time doing that

They couldn't filter by port, or could they ?

Yes sir they COULD but once again you wouldnt waste your time doing that either... :)

read this: www.webopedia.com/quick_ref/OSI_Layers.asp

fons writes...

At the ISP end how do they know what traffic is used by P2P clients or servers ?

Not only do they do it basedon port, but they do it based on the traffic as well.

For example, some nice Cisco or Allot boxes will inspect the traffic and shape it:)

Take a look at this post.

BBD writes...

Take a look at this post.

you'd be supprised how that even in a VPN a decent box can see it as a VPN :)

Plus if you were VPNing you'd lose alot of your speed or have to have a host fast enough to VPN to and send you the data quickly - then pay for the bandwidth for this host too.

Muad'Dib writes...

Packet sniffing.

Its not packet sniffing, its packet inspection, very simply done on a cisco :)

ISP: Hmm, why would that customer be maintaining 120 half-open TCP connections...

pesken writes...

very simply done on a cisco :)

We refer to it as "Deep Packet Analysis"

when you think about it, its actually very clever, :)

pesken writes...

Its not packet sniffing, its packet inspection, very simply done on a cisco

Introduction to Packet Sniffing

netsecurity.about.com/cs...s/a/aa121403.htm

a better way would be to simply look at the make up of the packet and decide from that.

Can be done on the fly and be very fast, encryption would stop that from happening though.

AAPT reduced my download speed on BitTorrent. Is that possible?

Muad'Dib writes...

netsecurity.about.com/cs...s/a/aa121403.htm

I suggest you read your own article.

Packet Sniffing is done with apps such as Wiretap (formally Ethereal), and it captures the packets, for either automated or manual analysis later.

Packet Inspection is done in real time on the fly, usually with a router, firewall, proxy, and a rule based decision is made on what to do with the packet. The packet is not saved, as it is with packet sniffing.

DJ_J@@P writes...

AAPT reduced my download speed on BitTorrent. Is that possible?

You say they did then ask if it's possible. sheesh....
AAPT do not throttle P2P unless you've used your download quota of course then everything is throttled.

Its more than just throttling/shaping by port.

You can perform content inspection on the packets to determine what is IN them (so its no use trying to hide bit torrent data in a packet designed to look like a HTTP packet), and prioritise or shap based on that.

That does, of course, require a LOT of processing power, so its not an overly feasibly solution at this present stage.

There are appliances you can buy that are dedicated to traffic shaping etc, but theres only so far they can go.

Read up on NBAR for Cisco in particular to get an idea.

Tom S writes...

That does, of course, require a LOT of processing power, so its not an overly feasibly solution at this present stage.

No, it takes bugger all processing. You only have to look at the first couple of bits of the first packet to work out what it is. I do it on my home network to prioritise VOIP, p2p, ftp, http etc. This is all done on a PC with a 350mhz processor and 40megs of ram. I would be lucky to use more then 10% of the processor.

Even if they do throttle the bandwidth, users still use up their download quota...in the end it's the same effect (users downloading that much data)

FrankO writes...

No, it takes bugger all processing ... I would be lucky to use more then 10% of the processor.

Yeah, bugger all for one person ... What about the [tens of] thousands that an ISP would have?

Tom S writes...

Yeah, bugger all for one person .

6 people
If you have a router / gateway that can handle thousands of connections then it can handle thousands of packet inspections. A router has to inspect a packet to see what the destination is. Looking to see what sort of protocol it is does not take much more processor time. It is a simple match which computers are designed to do.

Tom S writes...

What about the [tens of] thousands that an ISP would have?

A $100,000 cisco will do, hmm, about a gigabit worth of traffic? :)

FrankO writes...

No, it takes bugger all processing.

In an ISP environment with 100s of megabits of data transfer, things change a little...

FrankO writes...

6 people

Sigh.

If you have a router / gateway that can handle thousands of connections then it can handle thousands of packet inspections

You clearly have no experience in an ISP environment if you think that.

Looking to see what sort of protocol it is does not take much more processor time. It is a simple match which computers are designed to do.

So all of my experience as a network engineer, and in ISP environments in general is all wrong? Maybe I'll just forget it all and listen to you, since you seem to know best.

Tom S writes...

So all of my experience as a network engineer, and in ISP environments in general is all wrong? Maybe I'll just forget it all and listen to you, since you seem to know best.

I do have a bachelor of Technology management majoring in computer science and software development if that helps.

FrankO writes...

I do have a bachelor of Technology management majoring in computer science and software development if that helps

Well no it doesnt. What does "technology management" and software development have to do with network engineering?

What sort of experience do you have in a production ISP environment that backs up your claims that packet content inspection has little to no impact on CPU load?

If we're talking Cisco, the box that does packet shaping/quota management is this sucker:

www.cisco.com/en/US/prod...s6151/index.html

5 Dedicated ASICS which each handle a configurable IP range (i.e. you break the users down by IP range and spread them across each processor in the device)

Each box is capable of handling up to 200 000 users and 2gb/s of traffic.

That's the hardware, and there is other software that does subscriber management and reporting/statistics collection which runs on high end servers running Sun or Linux.

This isn't a router. It sits between your subcribers and the outside network as a "bump in the wire" and it totally transparent - except for the affect on users :)

1 or 2 ISP's in Australia are using this solution, and it's getting quite common in Asia.

The technology is way beyond me, but I think that the underlying principle is all wrong. People are paying for a broadband internet connection, they buy a plan with an allotment of data included, I think it is unethical to employ these kinds of technologies to restrict the customers ability to use their purchased connection.

It's made even worse in the instances where the restriction is done without the user's knowledge and/or approval.

I think ISP's in general need to concentrate more on delivering a quality overall experience to customers, rather than cutting everything back to facilitate the lowest price in the market. At the end of the day in a service industry price wars and price slashing actually hurt the customer because the service is unsustainable and therefore degrades.

Surely these sums of money could be better spent in actually serving customers?!

uVelocity writes...

People are paying for a broadband internet connection, they buy a plan with an allotment of data included, I think it is unethical to employ these kinds of technologies to restrict the customers ability to use their purchased connection.


Expect to see a change in the way that broadband plans are advertised and sold in the next few years.

Right now, it's all about restrictions, but this hardware is designed to offer service level guarentees on top of the existing data.

e.g

10gb plan, with a service guarentee that VoIP and Media Streaming (e.g. RTSP) traffic will ALWAYS have available bandwidth and P2P traffic will be a lower priority than Web or Email.

The market up to this point has been designed around over subscription and relatively low bandwidth. With the advent of DSL2 and higher speed cable and FTTH, there is a worldwide shift with Service Providers starting to focus on the services that run on the network, and right now, most of the aggregation points can't keep up.

At least 70% of Internet traffic right now is P2P.. and I'm not exaggerating at all.

QoS to the home user is now becoming a requirement, as once providers start pushing IPTV and VoIP, people will not be happy if the movie they are watching starts buffering half way through.

Eventually, it will go more the other way - Everything will run at full speed, but once congestion occurs, protocols like P2P will be the first things to get dropped, in order to keep other traffic like VoIP from being dropped.

I'm not saying I agree with any of this - just letting you know the rationale behind the shaping products. :)

The Derro writes...

The market up to this point has been designed around over subscription and relatively low bandwidth. With the advent of DSL2 and higher speed cable and FTTH, there is a worldwide shift with Service Providers starting to focus on the services that run on the network, and right now, most of the aggregation points can't keep up.

The shift from dial up to ADSL provided a similar situation - people getting hold of faster connection, which moved them from BBS's to forums, text based pages to multimedia pages, ICQ to Skype, etc...

This move didn't cause any major issues in the scheme of things. I can't see the move to ADSL2+ causing any more issues that the original move to broadband. Sure, some ISP's will have issue's (as they have in the past), but I don't think it will be a major issue.

I could be wrong though. We will just have to wait and see.

FrankO writes...

I do it on my home network to prioritise VOIP, p2p, ftp, http etc. This is all done on a PC with a 350mhz processor and 40megs of ram

Your filtering what? 5 users?

were talking about 50,000 users

There are clauses in many ISPs contracts that stipulate that the service cannot be used for illegal purposes. A LOT of P2P traffic is downloading and/or sharing copyrighted material, which for 99.9999% of cases the parties involved probably dont have the copyright holders permission to do so.

Therefore in that case I think it is fair.

Also, a lot of contracts stipulate that the service should not be used in a way that it has a detrimental effect on the performance/usability/etc of the network. Downloading 24/7 at full speed will cause issues if a lot of people do it.

So in that case it is also fair.

But I think you are right in that ISPs offer a certain amount of data to customers, and customers are entitled to use that quota, since they are paying for it.

In the case of "unlimited" plans, well there is no set minimum or maximum that a customer can use, its just you get whatever you get.

I guess if ISPs dont want to encounter these issues, the obvious solution is dont offer plans with huge download limits in the first place. If I was running an ISP, I wouldnt, and if I did, there would be a price. Heavy users arent economical, place a huge strain on costly resources (support staff, network, bandwidth, etc), and are generally subsidised by light users that dont use all of their "share".

Heres some interestig comments made by an Internode rep in the Exetel shaping thread, some have to do with P2P shaping.

forum-replies.cfm?t=606144&ux=11318

John Lindsay writes...

Shaping peer to peer traffic is all about working out what isn't peer to peer and NOT shaping that.

It's an inexact science but there are vendors who employ salespeople who claim the products are able to do it.

Interesting. Sounds hit and miss to me.

Tom S writes...

Heavy users arent economical, place a huge strain on costly resources (support staff, network, bandwidth, etc), and are generally subsidised by light users that dont use all of their "share".

Its sounds like your saying that ISPs aren't happy providing the service to their customers that their customers are paying them to provide, relying on the fact that other customers are paying for a service and not having to provide it for them to stay in business. I really find this hard to believe and if it is the case they deserve to go broke.

Danos writes...

I really find this hard to believe and if it is the case they deserve to go broke.

Why?

It's like an all you can eat buffet. Places that put them on expect people to eat a reasonable amount - an average amount if you like. Sure, there will be some people that eat only a little, and others that eat a lot, but overall the average is maintained. In this model, essentially the cost of the big eaters is covered by the little eaters, based on the cost of the overall average amount.

The same is true in the capped phone plans - while they offer lots and lots, most people won't use all of it, and those that do are funded by those that don't...

C.

Interesting thread,
I didnt think ISPs would go to such an extent as Packet Inspection because:

Tom S writes...

That does, of course, require a LOT of processing power, so its not an overly feasibly solution at this present stage.

Are there any privacy issues, as to how deep an ISP can inspect a packet?

prdgy writes...

Are there any privacy issues, as to how deep an ISP can inspect a packet?

dunno but really the isp doesnt log what the packet is they are just tagging it to sort it as they want there is a part of the packet called a TOS this is there for low latancy based traffic its to help sort traffic so that httpd doesnt overrun say sip traffic.. tos isnt really that useful as alof of programs dont set the TOS bit so another way to mark packets is to check out the layer 7 patten this is what the isp is doing ... its not like the isp is dumping your traffic they have no idea what your sending/reciving inless they start dumping it... but i guess they wouldnt with out a court order telling them to.

they just do

There doesnt seem to be a plausible way. From my limited knowledge, at the TCP/IP level, all that ISP's can do is filter/throttle by port. That does not seem feasible when the default port of these apps can be changed.

If they filter/throttle by packet sniffing, don't the "sniffers" need to know signatures of all p2P client/servers ?

fons writes...

If they filter/throttle by packet sniffing, don't the "sniffers" need to know signatures of all p2P client/servers ?

Yes. its packet inspection - the cisco unit linked on a previous page can do so quite effectively.

You can also do it by traffic profiling. P2P traffic has a particular signature - outgoing traffic going to many ports on many IPs, with incoming traffic coming from many ports and many IPS to one IP / port. Through this method, it is possible to de-prioritise traffic with such a profile, and prioritise other traffic.

C.

Don't some p2p clients come with randomization port usage?

You could just randomize the port and setup uPnP. I assume the trafic analysis would require a minimum of time of unfettered usage to analyse the traffic before shaping and probably even a latency of time AFTER analysis and determination for shaping to occur.

Setup your p2p application to randomize port usage every 15mins?

Alternatively they would have to UNTHROTTLE the port after a certain period on inactivity. You could just alternate ports after a certain time period and cycle through them. Only requiring X number of ports open and a cycle method between them. Then you could just setup a script as a manual process to change port numbers.

Tom S writes...

There are clauses in many ISPs contracts that stipulate that the service cannot be used for illegal purposes. A LOT of P2P traffic is downloading and/or sharing copyrighted material, which for 99.9999% of cases the parties involved probably dont have the copyright holders permission to do so.

Therefore in that case I think it is fair.

Without getting into a massive debate on copyright infringement, that standard clause is there to cover the ISPs arse in case the customer does use the connection for illegal purposes and the ISP gets named in a suit along with the naughty customer. The ISP is simply absolving themselves of responsibility because there is no possible way for them to check the legality of you online actions in real time.

If they're shaping/throttling data just because they think p2p is illegal they're 1) wrong and 2) using a very dodgy quasi-legal excuse for providing a bad service. i.e.:

"Its not our fault if you download copyrighted material, and we're not going to stop you, but we're going to ensure that if you do, you can't do it very fast and we're still going to chage you full price as well."

Also, a lot of contracts stipulate that the service should not be used in a way that it has a detrimental effect on the performance/usability/etc of the network. Downloading 24/7 at full speed will cause issues if a lot of people do it.

So in that case it is also fair.

I think it's pretty unfair to expect Joe Customer to have any kind of insight into the inner workings of an ISPs network architechure and QoS policies. If you're paying for a 1.5Mb connection, you expect to be able to use it at 1.5Mb. How is the customer supposed to know that using the service they're paying for is detrimental to the network? If using the service is such a terrible thing, why is it being sold?

That said, if there is some kind of massive load on the network then it is perfectly reasonable for the ISP to cut back the torrent traffic until the surge has passed, but if the ISP's standard practice is to limit p2p then they simply need to get more bandwidth.

BTW - I started this thread to ask if ISP's p2p traffic shaping policies can be made available on Whirlpool. If you like the idea, give it a +1.

Thread: forum-replies.cfm?t=608207

saudukar writes...

Setup your p2p application to randomize port usage every 15mins?

Alternatively they would have to UNTHROTTLE the port after a certain period on inactivity. You could just alternate ports after a certain time period and cycle through them. Only requiring X number of ports open and a cycle method between them. Then you could just setup a script as a manual process to change port numbers.

That would create huge problems for the p2p networks. Changing ports at 15 minute intervals would mean a HUGE increase in network overhead as other clients would have to be constantly re-negotiating how to connect to you.

A number of clients will get annoyed at you if you keep playing hide-and-seek with your port numbers and are likely to ban your IP and/or username.

saudukar writes...

Don't some p2p clients come with randomization port usage?

Yes, and this is part of the problem.

Setup your p2p application to randomize port usage every 15mins?

This is where content inspection comes into play.

It cares not for the destination or source port of the packet, it looks at the contents of the actual packet, the payload, the data that the client will inevitably receive, and bases its decision on that. This requires looking at the content and matching it against rules defined in a "database" or definition file I guess you could call it.

On Cisco's, NBAR can be used for this purpose, and there are also dedicated appliances which can do it, as has been mentioned.

fons writes...

From my limited knowledge, at the TCP/IP level, all that ISP's can do is filter/throttle by port. That does not seem feasible when the default port of these apps can be changed.

The contents can be inspected at any stage along the way, as long as the contents arent encrypted. Obviously if you try to inspect the contents of a packet that contains an encrypted payload theres not a lot you can do because you cant tell whats in it or what it is.

But you can still look at what it is in the packet in terms of its payload. The payload could be anything.

And for the reasons above, this is why it is a drain on CPU, because when you consider the amount of packets an ISP will be processing on a single router, and how many different types of traffic they need to identify, its not a "bugger all" situation.

Alex12 writes...

that standard clause is there to cover the ISPs arse in case the customer does use the connection for illegal purposes and the ISP gets named in a suit along with the naughty customer.

It is there principally to allow the ISP to break contract with users performing illegal activities, and so they can claim common carrier status when it comes to any law suits brought due to the illegal activities of their users. By definition, a common carrier does not look at what is being sent across their network, and only intervenes when informed by a third party of a breach.

C.

Corey writes...

Why?

It's like an all you can eat buffet. Places that put them on expect people to eat a reasonable amount - an average amount if you like. Sure, there will be some people that eat only a little, and others that eat a lot, but overall the average is maintained. In this model, essentially the cost of the big eaters is covered by the little eaters, based on the cost of the overall average amount.

If i was at an all you can eat buffet and i was eating lots and lots of prawns and some one come out and said you can keep eating but no more prawns i would get a bit upset. Its says all you can eat i should be able to eat what ever is on the buffet, i've paid for it.

Yes it is like an all you can eat buffet, you've paid them to eat all you can now they dont have the right to slow you down from eating what you want.

I was just saying if they've set their business up so if nearlly everyone goes close to using their limit they will be in trouble. Thats not a smart way to run a business.

Danos writes...

I was just saying if they've set their business up so if nearlly everyone goes close to using their limit they will be in trouble.

That's how most service provider businesses operate.

Thats not a smart way to run a business.

The market would suggest otherwise...

C.

uVelocity writes...

I think it is unethical to employ these kinds of technologies to restrict the customers ability to use their purchased connection.

I think its unethical to use P2P networking to rip off movies, music and software.

Danos writes...

If i was at an all you can eat buffet and i was eating lots and lots of prawns and some one come out and said you can keep eating but no more prawns i would get a bit upset.

So would I.

When I go to the buffet I paid just like everyone else to eat from the selection available. Some selections cost more and are more sort after. I eat as much prawns/mussels/seafood as I can get. I mean I paid for it there is no rule in place that says we have to equally consume food items.

Same with p2p shaping. How do they determine what constitutes p2p? Does my SQL2000 service that services 50 or so concurrent connections with large client downloads fit that bill? Does my ftp rsync fit that bill? Does my BT client that I use to pull legit patch and OS upgrades and distributes count?

The constitution here is that p2p traffic is a bit broad a determination for a product that you pay for. I mean you pay for data. If the ISP doesn't like providing that data THEN LOWER QUOTAS OR INSTIGATE THEM.

I am afraid if the ISP doesn't like the volume of traffic being used then stipulate a limit. Don't whinge when people actually USE that limit to download stuff.

Besides it is not the ISPs place to determine that p2p applications constitute something used to download illegal material. It is the polices place to enforce laws on copyright.

If you advertise that you can download to a certain limit that that limit should NOT be inpeeded by ANY technology.

FiberGuru writes...

I think its unethical to use P2P networking to rip off movies, music and software.

I agree with you that it is unethical, and it's also illegal. If you can determine what is copyright and what is not in a P2P file transfer, feel free to share your discovery with the world. Legality of content however isn't the issue, restriction of a paid service is.

ie. Don't throw out the baby with the bathwater....

saudukar writes...

Same with p2p shaping. How do they determine what constitutes p2p? Does my SQL2000 service that services 50 or so concurrent connections with large client downloads fit that bill? Does my ftp rsync fit that bill? Does my BT client that I use to pull legit patch and OS upgrades and distributes count?

It's pretty much all done via pattern matching/protocol signatures.

There has been cases in the past where the Cisco product I linked to earlier was classifying an Online Game as P2P traffic (eMule if I recall correctly), so a new protocol pack was released with an updated signature for that specific application.

For most traffic, port based classifcation is sufficient. However these devices are capable of looking as deep as you want, so if providers get complaints from customer that a specific program they use is being mis-classified, an updated protocol pack can be developed to cover that specific program.

Unfortunately, the bottom line is that traffic shaping is going to become more and more common as broadband links are used for different applications (VoD, VoIP etc).

I can tell you now that P2P users are in the minority - The vast majority of users don't care about P2P, but they DO care when there VoD which they have paid for is being slowed down.

Once again - I'm not advocating a side on this, and I think that providers should provision enough bandwidth for it not to be an issue, but the reality is that this is going to be much more common in the future.

throttling p2p traffic is gay because downloading linux iso is now going be throttle

now thats just great!!

AZNTieN writes...

downloading linux iso

ohnoes how can we ever survive without our "linux isos".

You can get them via FTP anyway.

ssj writes...

ohnoes how can we ever survive without our "linux isos".

You can get them via FTP anyway.

i suppose and its also on pipe free download w00t

The Derro writes...

This isn't a router. It sits between your subcribers and the outside network as a "bump in the wire" and it totally transparent - except for the affect on users :)

Wow, so Cisco have finally invented the Packeteer Packetshaper.

So in about five years time it'll probably start working properly. Cool.

joe onemillion writes...

Wow, so Cisco have finally invented the Packeteer Packetshaper.

The SCE product came from an acquisition. It's a little different at a packet level to Packeteer, but yes, they are very similar products in terms of what they actually do.

www.azureuswiki.com/index.php/Bad_ISPs

Two that block BT and P2P altogether.

Unwired and Iburst.

Rather than block they prioritise P2P.

Muad'Dib writes...

www.azureuswiki.com/index.php/Bad_ISPs

Lets get the list updated for Aust. If your ISP blocks/throttles/prioritises p2p traffic, let the azureus guys know on their IRC channel #azureus-wiki and they will update the webpage.

DDL

DrDoLittle writes...

let the azureus guys know on their IRC channel #azureus-wiki and they will update the webpage.

exetel not listed on that site, someone should tell em !

sonix writes...

etel not listed on that site, someone should tell em !

Extel. Is a Reseller of Iburst

I was under the impression that, originally, ISP's used to shape well known P2P ports then moved to packet sniffing to shape the ports after P2P programs allowed for different (non-standard) ports to be used. Now, several P2P programs come with protocol obfuscation which, as I understand it, has been programmed into the software to get around ISP packet sniffing and port shaping.

I'm not sure of this, havn't really read up, but don't most P2P clients now have protocol encrytpion. Which is random, so the ISP can't use packet sniffing to find P2P data. I know utorrent has this option, and I have it turned on.

packet sniffy is a breach of contrat in a lot of ISP conrtracs

Morphias writes...

I'm not sure of this, havn't really read up, but don't most P2P clients now have protocol encrytpion. Which is random, so the ISP can't use packet sniffing to find P2P data. I know utorrent has this option, and I have it turned on.

I was thinking about encryption before as well. Some private trackers encourage their participants to enable encryption or upgrade to a BT client that uses it.

Does using encryption beat these cisco packet inspecting systems and does it have a negative effect such as increased data requirements (resulting in more download quota being chewed) ?

Alex12 writes...

ISP: Hmm, why would that customer be maintaining 120 half-open TCP connections...

Bingo.

Also, I think you guys need to stop thinking in absolutes. If an ISP can throttle 80% of its p2p customers using something like packet inspection then it doesn't matter if 20% of the customers use encryption or whatever magic feature to get round it. It has already achieved a significant reduction in traffic. ISPs tend to have a lot of experience in averages games like this :)

Of course, when the bypass technology reaches a majority of customers then new ways to shape it are needed.

Something I wrote in another forum a while ago about how p2p shaping works:

A user utilizing a couple of megabits worth of FTP will be seen as one flow between the user's IP address and the FTP server's IP address which will carry a couple of megabits.

A user running a web server which is utilizing a couple of megabits worth of traffic will be seen as lots of short-lived flows between the user's IP address and semi-random "other" IP addresses where each flow runs as fast as it can and occurs on fairly predictable port numbers.

A user utilizing a couple of megabits worth of P2P will be seen as hundreds of flows between the user's IP address and a heap of semi-random "other" IP addresses where each flow is reasonably long-lived, has semi-random port numbers, and runs reasonably slowly because there'll be a bandwidth constraint somewhere on the network which the shaping box can't see.

Writing software to detect those heuristics isn't hard. And you'll note that it doesn't really matter what port number you run your P2P app on, or whether your P2P app happens to be eDonkey or BitTorrent, or whether your P2P app is using encryption. The heuristics will be kinda-sorta the same.

The encryption in p2p apps isn't very good at obscuring the fact that you're using the apps. It is, however, good at obscuring the file you're downloading/uploading. That's great for you if your "opponent" is RIAA, but it doesn't help you at all if you're interested in preventing your ISP from shaping you, because your ISP doesn't care what you're downloading and doesn't need to know in order to work out that you're a P2P user.

- mark

Thanks for clearing that up Mark.

Mark Newton writes...

Something I wrote in another forum a while ago about how p2p shaping works:

That should win node a fair few customers from Exetel & co as soon as someone over there points out the futility of pursuing avoidance strategies :)

Mark Newton writes...

A user utilizing a couple of megabits worth of P2P will be seen as hundreds of flows between the user's IP address and a heap of semi-random "other" IP addresses where each flow is reasonably long-lived, has semi-random port numbers, and runs reasonably slowly because there'll be a bandwidth constraint somewhere on the network which the shaping box can't see.

Doesn't this only apply to public torrents, where there are thousands of clients connected??

For example, a private torrent, which will have a MUCH better speed, doesn't have the rollercoaster life of a public torrent. Trackers I frequent have users with 10-100Mbit connections, and often one seed will 100% fill my 1500 connection.

Does that mean that a Download-Accelerator (ie: Getright) could be choked, being mistaken for a P2P download with 10 uploaders, for example?

Wanderlei Silva writes...

Doesn't this only apply to public torrents, where there are thousands of clients connected??

Pretty much, yes.

But for what it's worth, the overwhelming majority of P2P users don't use private trackers. An ISP who is interested in doing this stuff only needs to grab the low-hanging fruit to make a difference to their bandwidth consumption. It's big-picture irrelevent if a minority of users do something that'll escape the detection regime.

(on the other hand, if a majority of users do something that escapes the detection regime, the ISP will have a serious problem, and that's when the whole house of cards will probably come crashing down. Think of the economic situation such an ISP would be in: They've implemented shaping because they can't afford their bandwidth and they want to buy less of it; Their customers have bypassed the shaping; So they're using the same bandwidth they've always used; AND they've invested anything up to $100,000 in shaping equipment, software licensing, and network re-engineering over and above their bandwidth bills. Probably not a happy place to be in, eh?)

Does that mean that a Download-Accelerator (ie: Getright) could be choked, being mistaken for a P2P download with 10 uploaders, for example?

Depends on how smart the heuristics are.

The fact that all the connections would be on ports 20 / 21 would probably be a bit of a giveaway.

- mark

Mark Newton writes...

But for what it's worth, the overwhelming majority of P2P users don't use private trackers.....

Thanks for the info Mark, it's all of interest to myself & others. It's nice to know what we're getting/not-getting, and what we're paying for etc.

I agree completely, and heck, it's all good news for me :)

I'm going to the trouble not to be one of the crowd anyway, so it's win-win for me :)

They've implemented shaping because they can't afford their bandwidth and they want to buy less of it

These things happen every year I s'pose, in one shape or another.

A dilemma people find I spose, keeping-them-honest vs. chasing value for money.

The fact that all the connections would be on ports 20 / 21 would probably be a bit of a giveaway.

A bit more info please? I understand what you're saying, but not the comparison.

If some software is sniffing packets to find P2P traffic, it'll see a difference between a download accelerator (which tends to fire off gazillions of connections on the ftp-data port TCP/20 and the ftp port TCP/21) and a P2P app (which tends to fire off gazillions of connections to some reasonably non-deterministic number of other ports)

- mark

Mark Newton writes...

The encryption in p2p apps isn't very good at obscuring the fact that you're using the apps.

It is if they are using signatures instead of heuristics. Particularly if you are saying using SSL.

On a slightly unrelated note. I believe an ISP should be able to shape whatever they like AS LONG AS THEY TELL YOU IN ADVANCE CLEARLY.

There is very little technical reasons to shape P2P traffic however there are massive business reasons to do it as it moves away from Client server models which the infrastructure has been built for to P2P. However changing the infrastructure to suit the new needs ain't cheap and well shaping CAN be cheaper.

Not the real Monty writes...

AAPT reduced my download speed on BitTorrent. Is that possible?

Very improbably as they dont throttle p2p n never have !!!

Is there a database or wiki of Australian ISP's that shape or throttle data such as P2P?

If not, wouldn't it be a good idea to make one?

dj007 writes...

Is there a database or wiki of Australian ISP's that shape or throttle data such as P2P?

yes

www.azureuswiki.com/inde...d_ISPs#Australia

forum.exetel.com.au/viewtopic.php?p=169285

Packet sniffing and packet inspection.

Tom S writes...

But I think you are right in that ISPs offer a certain amount of data to customers, and customers are entitled to use that quota, since they are paying for it.

Its like buying a Ferrari, you pay for it, but you cannot actually utilise what it can do.
You are limited to a goverment regulation (speeding laws) same with the internet.

But life goes on,

tacoflavour writes...

Its like buying a Ferrari, you pay for it, but you cannot actually utilise what it can do.

Actually you can, you can go as fast as you like in your Ferrari just like i should be able to do what ever i want with my quota at full speed.

someone already mentioned Allot, so I reckon some of these would do the trick:

www.allot.com/index.php?...&id=45&Itemid=44

Theravadin writes...

someone already mentioned Allot, so I reckon some of these would do the trick:

www.allot.com/index.php?...&id=45&Itemid=44

+1

Any news on the p2p packages/hardware Australian ip are getting?
Better encryption detection?
Upgrades as p2p apps change?

Did they go for the cheaper end?
Or did a few of them go for a smarter active encryption hunt?