linkstate.org

Due to the number of comments and questions I’ve received about my previous post on the Service Control Engine and how we use it to manage international transit, I’ve put together a bit of a followup piece. There have also been a number of changes to the implementation recently that are worth sharing as well.

Since launching our UCLL network earlier in the year, we’ve seen pretty huge growth in the volume of traffic we’re carrying. I presented a paper recently where I shared a few statistics on the network and recent trends. For example:

- Our UCLL subscribers achieve an average linespeed of just over 10Mb.
- The lower quartile linespeed sits at around 5Mb - which is slightly higher than the average linespeed we used to see on the UBS network.
- Our DSL customers alone consume over 15Tb of data per day.
- Our DSL customers consumed over 10Tb of Olympics streaming coverage in ten days.
- Customers on the Orcon+ network (UCLL) consume nearly twice as much data per month on average than our UBA (Telecom wholesale) customers.
- Streaming video now accounts for at least 30% of international traffic. Peer to peer uses approx another 30%, web 30%, and the remaining 10% by all other applications.

So bandwidth requirements in our network are growing at an exponential rate. Its driving a few architectural changes within our network, such as:

- We’re moving to 10Gb Ethernet for our metro backhaul network.
- Many exchanges are being moved to Gigabit Ethernet.
- We’re moving our transit links to 10Gb Ethernet and upgrading our DPI platforms to cope with the volumes in traffic.

The last point is a particularly relevant one for the purposes of this article. As covered off in my earlier post on the subject, our approach until recently has been to group all subscribers into a single pool based on their connection type - e.g. UBS, UCLL, dialup, etc. The result has been chunks of bandwidth that look like this:

This has changed recently in that each and every subscriber now has an individual policy of their own. This policy does two things:

- It sets a peak rate. For ADSL2+ this is 24Mb.
- It sets a priority for that subscriber.
- It sets a priority for P2P vs everything else for that subscriber.

This has the effect of distributing bandwidth fairly across all subscribers. It means that heavy users won’t unduly impact the experience for lighter users by consuming a proportionally larger amount of bandwidth. It’s important to note thought that users aren’t classified as “heavy” or “light” - it simply means that a user downloading an email or web page from overseas (e.g. light and bursty usage) won’t be negatively impacted by those who are streaming video or downloading a P2P file (e.g. sustained and high bandwidth usage).

In graphical form:

On a related topic - one of the most commonly asked question so far has been whether there is any difference between ADSL and ADSL2+ connections in how they are treated on the SCE?

In short, no, but there are some differences in broadband products which are worth mentioning. There are two sub-policies to the broadband “pool” of bandwidth: one for UCLL, and the other for UBA. Our retail UCLL services are dimensioned roughly 3 times higher than UBA services, and that flows through to the SCE config. UCLL services are only ADSL2+; UBA are a mix of ADSL and ADSL2+. The UBS policies don’t differentiate between ADSL and ADSL2+ subscribers - and in fact we have no way of knowing which type of line a subscriber is on without asking Telecom.

The dimensioning rules are a direct result of the backhaul capacity allocated within the respective networks: the Telecom UBA network (basic UBA) is dimensioned at 32kbps per subscriber averaged over 15 minutes. The Orcon+ network is dimensioned over three times that.

Feel free to post comments; if any further questions come up I’ll do another followup piece.

PS - I’ve closed off comments on the previous post due to the high volume of spam I was receiving; I’ll leave comments on this thread open as long as I can.