Metro Ethernet

*** see also the MEF (Metro Ethernet Forum) http://www.metroethernetforum.org/ 

Metro Ethernet is very difficult to explain, because it encompasses so much !!  It is an extremely complex set of recommendations (not standardized yet)

Traditional Ethernet is used to connect devices and allow Layer 2 data communication - forming a LAN.  This allowed distributed processing and peer-to-peer networking.  It worked well and for the most part, replaced much of the previous centric network model (i.e. Mainframes and Dumb terminals).  But companies needed to communicate with their "sister sites" in different cities.  After numerous methods have been used for many years to connect Ethernet LAN's - what was missing was standards to address the problems of speed, QoS, and scalability.  Metro Ethernet is an answer to those missing standards.  First, a few important points:  

Example - two MEN's Connected by a WAN Link

MEN (Metro Ethernet Networks) Definition

Metro Ethernet areas that the MEF is Working On

MEF Metro Ethernet "Service Areas"

Ethernet services Definition - Phase 1

Uses 2 models . . .  E-Line, a point-to-point link, and E-LAN, an any-to-any LAN interconnection similar to Transparent LAN Service (TLS), but in a multipoint configuration. At the business-user level, the MEF is positioning E-Line and E-LAN as the basis for applications such as LAN extension, dedicated Internet access, and Ethernet over Virtual Private LAN Service

E-Line

 

E-LAN

*** for the other 3 service areas - Traffic Management, CES, and PDH  -  see the MEF Technical White Paper.(8/2004 -  for most recent goto MEF website)

 

History of Metro Ethernet

  1. the Telecom companies had a solution for connecting Ethernet (and Token Ring, SNA, etc) LAN's - private lines (dedicated Point-to-Point circuits).  Geographically separated LAN's connected together into one large WAN - a router for each LAN interconnected them via TCP/IP across private lines
  2. private lines were very expensive, especially for full mesh (the more sites, the more private lines required) - and were replaced by relatively inexpensive Layer 2 protocols such as Frame Relay or ATM.  These protocols were priced using Flat-Rates, and allowed cheap PVC's to be purchased and emulated private lines
  3. for small companies with limited needs, ISDN BRI connections were made available - the LAN administrator would dial up to a remote site via a 64k or 128k BRI, and transfer files, run applications, etc - then disconnect when finished.  The fees were usage-based and expensive.
  4. some companies connected their LAN's via the Internet - but the Internet was very slow for connecting LAN's and insecure - so Telecom companies came out with Layer 3 "pure" private IP networks.  These were available for long distance transport, but for most companies were too expensive and not preferred
  5. Internet connectivity became secure with encryption protocols such as IPsec, and Firewall enhancements.  But still, the connection was slow at times and had the overhead of the IP headers.  
  6. the Internet speed increased dramatically as telecom companies spent billions on infrastructure upgrades.  With IPsec VPN boxes, Firewalls, and speed - the Internet was now an excellent choice for interconnection of Ethernet LAN's.
  7. but weak spots of Internet LAN connectivity remained  .  .  .  VPN boxes were very expensive, and hackers could still potentially create havoc, since no firewall can stop everything.  MPLS (RFC 2547) private IP networks solved the problem.  They offered secure VPN's and the customer did not have to purchase or maintain VPN IPsec boxes.  Today - this seems to be a perfect solution to LAN interconnectivity.
  8. at the same time that MPLS was being developed
  9. the MEF is formed and they begin working on Metro Ethernet standards

 

Traditional TDM ways of getting to the customer, with traditional SDH/SONET-based transmission, carriers pay a high price in operational complexity, cost and provisioning delay. Ethernet's advantages include fast provisioning, fine-grained bandwidth granularity (inherent in packet technologies) and a scalability from kbps to Gbps. The customer also expects a lower cost service, although carrier pricing remains volatile, partially from fear of cannibalising their existing connectivity revenues.

Generic Framing Procedure (GFP) 

Carriers with already deployed SONET/SDH networks naturally consider how to use them efficiently to carry Ethernet. The problem of mapping continuously-scalable packet flows into the lumpy SONET/SDH bandwidth hierarchy is well-described. Generic Framing Procedure (GFP) is becoming an increasingly popular adaptation layer between Ethernet (and other packet protocols such as PPP, Fiber Channel, FICON/ESCON) and SDH/SONET, implemented via the evolution of SONET/SDH devices into Multi-Service Provisioning Platforms (MSPPs). The bandwidth mismatches are addressed via Virtual Concatenation (VCAT) and Link Capacity Adjustment Scheme (LCAS).

Halabi's "Metro Ethernet" Book Summary

Perhaps the best book on Metro Ethernet is summarized here:

Ethernet-over-SONET/SDH as just described is a pure transport mechanism. To create an Ethernet analogue of add-drop multiplexing and to support traffic aggregation, L2 switching functionality needs to be added to the basic SDH/SONET box. (This is a well-worn path for transmission vendors - the same model was proposed for ATM). Different customers' Ethernet streams need to be identifiable, and carrier VLAN tagging is a possibility, although MPLS provides a more scalable solution.

In L2 switching in ring topologies, bandwidth fairness and efficient protection switching is difficult to achieve. The new "Resilient Packet Ring" (RPR) MAC protocol was developed to address these issues, and RPR can be run over GFP, and therefore supported in SONET/SDH devices which understand the RPR protocol. Of course, one can dispense with SONET/SDH equipment altogether (especially if you are a new operator and never installed it). Halabi briefly touches on the deployment and management of Gigabit Ethernet switches with direct interconnect.

Ethernet services must be understood, such as L2 switching, MAC learning, flooding, broadcast/multicast, VLANs and spanning tree protocol, and then we can get down to the services.

The Metro Ethernet Forum has defined two Ethernet service types: Ethernet Line Service (ELS) and Ethernet LAN service (E-LAN). Point-to-point vs. multipoint-to-multipoint, or transport vs. transport-and-switching if you prefer. ELS issues include traffic and performance management, class-of-service, VLAN support. Additional issues for E-LAN services focus on mechanisms for customer-separation, address-management and scalability.

Halabi identifies a number of issues along the way: with the VLAN tag length restricting operators to 4,096 customer-id values, operational services cannot scale; Ethernet does not have the kind of embedded OA&M facilities which allow carrier services such as SONET/SDH to be monitored and provisioned; the spanning tree protocol for loop-prevention does not scale and is inefficient; VPN configuration is hard to scale.

Chapter 4: "hybrid L2 and L3 IP/MPLS networks" unveils the solution. In a nutshell it is to adapt Ethernet to MPLS at the network edge, and use the power of BGP/MPLS VPN technology to scale the service. Halabi starts by reviewing standard L3 VPNs, both IP tunnels (GRE, not IPsec) and BGP/MPLS rfc 2547.

He then notes that Ethernet can be carried over MPLS via the IETF Pseudowire standard - this is fine for Ethernet Line Services. An E-LAN service such as Virtual Private LAN Service requires more work from the CE/PE devices, however. Specifically, the CEs think they are talking to an Ethernet switch on their link to the PE. The PE needs the additional functionality of a VLAN switch, and maps a specific MPLS label to each VLAN on a per-customer basis. This "broadcast domain" identification inner label is then augmented by an outer traffic-engineering label to forward traffic to the correct destination PE across the Service Provider network. Halabi describes in detail the mechanisms, which are similar to rfc 2547 VPNs at L3.

In practice there are still scaling issues, and the concept of "Decoupled Transparent LAN Service" is introduced. This creates an additional customer-premises PE which specialises in L2 MAC address management and customer segmentation, while the network POP PE can specialise in L3 MPLS tunnel and connectivity management. Given the detailed technical treatment, this is one of the harder chapters in the book. However, this is the last chapter actually devoted to Ethernet.

With Chapter 5, we enter part II of the book, which is more focused on traffic engineering and GMPLS. This chapter is a fast review of MPLS for traffic engineering of IP networks. Chapter 6 extends this discussion to cover the details of RSVP-TE for LSP establishment, specifically for fast-reroute. In Chapter 7 and the final Chapter 8, we see how MPLS (specifically GMPLS) can be used as a generalised control plane for virtual circuit management in the SONET/SDH and the optical layers.

The Internet is awash with white papers on all aspects of metro Ethernet. This book was published, by Cisco Press, in September 2003 so it's hot off the press. But sections still appear to be slightly dated in what is an incredibly fast-moving area.

Why buy it? Because Halabi knows what he's talking about, and gets down into the detail of how everything works with great intellectual clarity. Although it can be hard to see the wood from the trees in this book, it is the ideal "in one place" reference for both services and technologies for carrier Ethernet. I consider chapters 5-8 as an MPLS bonus, as they actually have nothing specifically to do with carrier Ethernet.