Re: Routed optical networks

[Mark Tinka <mark@tinka.africa>]

On 5/11/23 14:15, Jared Mauch wrote

> We have seen a continued trend of the privatization of traffic and localization of that over time.

CDN's, submarine cables and exchange points are decentralizing the "core 
of the Internet", and relegating IP Transit providers to whom CDN's 
subscribe as sources of origin content, if they are unable to build 
their own backbones.

Only in markets where CDN's are not as rife do you still see huge growth 
and stable pricing for IP Transit traffic.


>    I’ve watched all the big carriers retreat from their global network reaches to be more of regionalized networks.  A 
> decade ago you would have seen European national incumbents peering and with market in Asia, and the complete global 
> networks continue to shrink.

Yeah - unless you serve a very tight niche of Enterprise customers that 
require stable (not necessarily fast) connectivity to some part of the 
word, it makes little sense, nowadays, for the large telco's of old to 
go poking in other markets that aren't theirs, something the top global 
exchange points are going to learn the hard way.


> Meanwhile you have a mix of the content and cloud providers continue to build their business-purpose networks expanding 
> into markets that the uppercase Internet may not need to reach.

At the moment, the only motivation I see for large telco's trying to 
enter markets they can't be strong in is their "global brand". Often 
times, they can't do very well in that market because a) there is 
sufficient local content and peering, b) they can't build a network as 
robust as what the local telco's can, and c) all they have to offer is 
access to traffic several milliseconds away that is not any faster than 
what the local telco's can, which forms less than 20% of what the local 
customers chase after anyway.

Such telco's usually, then, turn to targeting global Enterprise 
customers with "worldwide service contracts" accompanied by marked-up 
pricing for "private global branch connectivity". It always sounds and 
looks good on PowerPoint slides, but 12 months after the big launch, a 
broken champagne bottle and some photos, the PoP is gathering quite a 
bit of dust in the data centre.


> You can look at the proposals in the EU about fees, and I have dual thoughts on this which are MY OWN and don’t 
> represent my employer or otherwise, but if you read this post from Petra Arts 
> -https://blog.cloudflare.com/eu-network-usage-fees/  - it speaks around major interconnection points like Frankfurt, 
> which are important but double as problematic.  The number of people that need to go to the near market (eg: Chicago, 
> while I’m in Detroit area) for good connectivity is an issue, meanwhile there’s a robust need to keep traffic within 
> the state of Michigan and a halfway decent ecosystem for that via Detroit IX - (disclaimer, I’m on the board).  There 
> need to be some aggregation points, so not everyone needs to be in Detroit, but also not everyone needs to be in 
> Frankfurt - and content localization needs to continue to happen, but is also very regionalized in popularity.

Us telco's didn't learn our lesson 23 years ago when the content folk 
tried to do a deal with us. This will be deja vu, although this time, 
with some help from governments that we rather won't like.

Telco's continue to strong-arm content because we own the customer. But, 
it's 2023... I'm not sure we want to see a battle of the customer 
between content and network. It might not end well. We already see 
glimpses of it with QUIC, so...


> How to do this all and not have it all route via Chicago or Frankfurt is a challenge, but also not everyone will be in 
> Berlin, Munich or these other markets.  This is where having a robust optical network capability (or backbone) can come 
> into play, that you can deliver deeper from those hub points, but at the same time, I’ve been in meetings where 
> companies have their own challenges accepting that content in those downstream locations as their network was also 
> built to get to/from the major hub cities, or IP space wasn’t allocated in a way that can provide consistent routing 
> results or behaviors.  (This is where IPv6 can be super helpful, it gives the chance to possibly Greenfield, aka not 
> screw it up - at least initially).

As content continues to grow, PoP's built in far-flung locations away 
from a telco's core area of business will only remain as relevant as 
traffic that has not yet migrated to a syndicated cloud. As that tapers 
off- which it will - there will be more pressure to either shut those 
PoP's down, or compete for local IP Transit. The latter is much less likely.


> There’s huge volumes of IP traffic exchanged, but the largest volumes are being moved over private interconnects or a 
> localized IX to those eyeball networks with the historical global backbones playing more of the long-distance carrier 
> role, which is critical as you want a path to deliver those bits, without it following the ITU-style sender pays model, 
> as the majority of IP traffic is actually requested by the customer of the end-user network.  (All of it if you remove 
> network scans, ddos, web bots/crawlers).

I'd say traffic growth is in the content space in descending order of scale:

 * DC-to-DC
 * PNI's in the data centre.
 * Public peering at exchange points.
 * Origin fetch.


> Most networks have no SLA once things cross an unpaid boundary (SFI, or even private peering) - and if they are a 
> customer and that path is congested, it’s up to the customer to upgrade that path.

The relevance of QoS in a cloud-based Internet, and the rise of QUIC, is 
essentially gone. I doubt any serious operator is buying line cards on 
the basis of their deep and fleshy QoS queues, anymore :-).

Mark.