PANET Working Group Shankar Raman INTERNET-DRAFT Balaji Venkat Venkataswami Intended Status: Experimental RFC Kamakoti Veezhinathan Gaurav Raina Expires: May 2013 November 5, 2012 TCAM power reduction and optimization in Routers draft-mjsraman-panet-tcam-power-efficiency-00 Abstract This idea entails enabling power and performance management in Routers (multi-chassis and single chassis) with respect to TCAM / SRAM usage on intelligent router line cards that implement Virtual Aggregation of routes. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Copyright and License Notice Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of Shankar Raman et.al Expires May 2013 [Page 1] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Using Aggregate routes . . . . . . . . . . . . . . . . . . . 7 2.2 Switching off the TCAM banks which are unused . . . . . . . 7 2.3 Using Aggregate routes . . . . . . . . . . . . . . . . . . . 7 2.4 Advantages . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 Security Considerations . . . . . . . . . . . . . . . . . . . . 8 4 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8 5 References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.1 Normative References . . . . . . . . . . . . . . . . . . . 8 5.2 Informative References . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 8 Shankar Raman et.al Expires May 2013 [Page 2] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 1 Introduction This idea / draft entails enabling power and performance management in Routers (multi-chassis and single chassis) with respect to TCAM / SRAM usage on intelligent router line cards that implement Virtual Aggregation. Distributed line cards in a routers have intelligence to forward packet without interference from the central control plane, once their TCAMs and associated SRAMs are populated. Since memory and TCAM are the most power hungry components in these line cards, we should effectively manage the line card power usage by optimally using their SRAM memory and TCAM banks. When a packet enters a distributed line card, the packet switching logic extracts information from the header. It then looks up the entry in the appropriate TCAM (CAM in L2 switches or both in Multi- layer switches), and then passes the packet to the outgoing line card. The packet is then placed onto the outgoing port. The TCAM banks used in the line cards typically carry the entire forwarding table as tabulated by the central control processor card/cards (when in plurality used for redundancy). All line cards do not need the entire forwarding table as each line card may serve a set of sources and destinations. This is true especially in smaller networks but may also apply to routers in the internet, especially ASBRs and POP border routers facing the customers of the ISP. Switching on the entire set of TCAMs (in the worst case) and downloading the entire forwarding table atop each of the line cards in the chassis leads to a sub-optimal way of switching packets Current status quo on this (apart from VPN route localization) is a proof that as far as internet destinations go, all line cards on the backbone routers or even within a small campus or a medium sized ISP, carry the entire forwarding table built by the routing table manager on these routers. In order to obviate the necessity of carrying routes which are unused (by this term. we mean those that are unreferenced for making forwarding decisions) and to reduce TCAM space (applicable to switching as well which involves CAMs), we suggest a solution where a couple of Linecards are considered to be Designated and others are called Backup. Designated Linecards are filled with all the entries from the control plane. The rest of the Linecards are provided with entries leftover from aging other entries which were not used over a period of time. An entry may not be available in these Linecards after they have been aged out (because of not being referenced and/or modified), these non- DLC linecards (as they are called), refer to the DLC/BDLC linecards for the first packet of a flow and retrieve the prefixes Shankar Raman et.al Expires May 2013 [Page 3] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 associated with the hit on the DLC/BDLC. They populate these retrieved entries in the TCAM. Periodically on the non-DLC linecards a Route aggregation similar to the S-VA or the simple Virtual Aggregate with exception entries is generated to further reduce the TCAM occupation. The TCAM banks which are not occupied as a result of this scheme may be switched off. This inturn can switch off their associated SRAM banks as well which contain the rewrite information. The paper also opens up the idea to simulations which help strengthen the argument for this scheme. 1.1 Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. Methodology Distributed line cards in a routers have intelligence to forward packet without interference from the central control plane, once their TCAMs and associated SRAMs are populated. Since memory and TCAM are the most power hungry components in these line cards, we should effectively manage the line card power usage by optimally using their SRAM memory and TCAM banks. When a packet enters a distributedline card, the packet switching logic extracts information from the header. It then looks up the entry in the appropriate TCAM (CAM in L2 switches or both in Multi- layer switches), and then passes the packet to the outgoing line card. It is then placed onto the outgoing port. The TCAM banks used in the line cards typically carry the entire forwarding table as tabulated by the central control processor card/cards (when in plurality used for redundancy). All line cards do not need the entire forwarding table as each line card may serve a set of sources and destinations. This is true especially in smaller networks but may also apply to routers in the internet, especially ASBRs and POP border routers facing the customers of the ISP. Switching on the entire set of TCAMs (in the worst case) and downloading the entire forwarding table atop each of the line cards in the chassis leads to a sub-optimal way of switching packets. Current status quo on this (apart from VPN route localization) is a proof that as far as internet destinations go, all line cards on the backbone routers or even within a small campus or a medium sized ISP, carry the entire forwarding table built by the routing table manager on these routers. In order to obviate the necessity of Shankar Raman et.al Expires May 2013 [Page 4] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 carrying routes which are unused (by this term. we mean those that are unreferenced for making forwarding decisions) and to reduce TCAM space (applicable to switching as well which involves CAMs), we suggest the following solution: a) At initialization time the entire forwarding table that is constructed in the control processor is downloaded to all line cards. This is the current implementation in most routers today. b) Begin a clock algorithm on these routes using a referenced bit and modified bit that is appended to each TCAM entry in the line card. a. When a TCAM entry is accessed for forwarding decision in the router on a specific line card, the referenced bit associated with that entry is set. b. When a TCAM entry is modified as a result of a routing entry the modified bit is set. c. A timer called the ReferenceTimer is started with a suitable interval. d. When the ReferenceTimer clock ticks down to 0, the TCAM entries in the line card is scanned and those that have the following values are not disturbed. 1. Referenced bit = 1, Modified bit = 1 2. Referenced bit = 0, Modified bit = 1 e. A timer called ModifiedDecayTimer is also started using a suitable time interval. f. The ModifiedDecayTimer is a single global timer which is started whenever a route change that modifies the nexthop to a set of route entries is downloaded to the line card (which is the status quo as of now). Each line card has its own respective ModifiedDecayTimer. g. On expiry of the ModifiedDecayTimer the TCAM entries with modified bit = 1 are cleared to 0. h. Similarly each TCAM entry has a 32 bit counter that counts down to 0, which is in effect a ReferencedDecayTimer. A suitable value may be appended to this counter at the time of initialization and when the TCAM entry is referenced for forwarding. This counter is active only for the active banks of TCAM A suitable alternative with or without the counter would be an LRU policy that removes and adds entries as appropriate. Shankar Raman et.al Expires May 2013 [Page 5] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 i. When the ReferencedDecayTimer counts down to 0, the referenced bit for that TCAM entry is cleared. j. Continuing from (d) the TCAM entries with Modified bit = 0 and Referenced bit = 0 are removed from the TCAM when the ReferencedTimer expires. Further optimization on the TCAM may be done at regular intervals whenever the ReferencedTimer expires. This would involve re-arranging the TCAM to optimize on storage. This could be a parallel thread in hardware. We will consider the initial condition on how these entries get populated. If a packet enters a line card on one of its ports and the switching logic finds that the TCAM entry is absent or has been cleared from its entry in the TCAM. We use a mechanism by which such a packet is first forwarded to one of the DLC (or designated Line cards which may be elected from the set of line cards in the chassis) where the entire forwarding table is always stored. There may be a Primary DLC and a backup DLC for redundancy purposes. When the packet hits the ingress line card which does not have the required TCAM entry for forwarding, it routes the packet within the chassis to the DLC. The DLC receives the packet or just the packet header and does the following: i) Looks up its full forwarding table, ii) Picks up the result and along with it the TCAM entry or entries which accumulate to all the set of related routes (perhaps all the routes with the same suffix) and iii) sends them across to the ingress line card in question. The ingress line card uses the result to forward the packet and populates the TCAM with the group of entries dispatched to it by the DLC or the Backup DLC. One could use per-packet loadbalancing within the ingress line card to distribute the result in such a way that the load is effectively shared by picking to the DLC and the Backup DLC. Once the set of routes sent back from the DLC is accumulated and loaded into the TCAM, it may be periodically scanned and compressed even at a later point in time. Now the ingress line card has the required entries. We assume that the flows that go to the desired destinations from then on would hit the TCAM entries that are populated according to the method discussed above. Thus, long lived TCP / UDP flows would have TCAM entries populated for the duration of their existence in the respective scheme-deployed linecards. Smaller timed flows too would have entries populated but Shankar Raman et.al Expires May 2013 [Page 6] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 this could be throttled by doing inspection deeper into the packet (for example, to see if it is DNS). With respect to multicast routes, all multicast routing entries and their respective unicast companions are left on the line card and are not affected by this scheme. This relates to unicast routing alone and TCAM entries that relate to it. It is obvious to see because the OIL is dynamic for the multicast routes. Applying this technique to multicast would lead to thrashing of entries in the TCAM. Additionally if a set threshold called ResultQueryThreshold is configured on the DLC slots. If this is exceeded in terms of rate of packets coming to DLCs, a periodic full download of the entire forwarding table is done to the those linecards which deploy this scheme. A further purge could be done using the Timers mentioned above. It is also possible to think of deploying this scheme selectively on a set of line cards. In this method, the rest of the line cards can act as TCAM entry route servers. Such an arrangement can help in load balancing the packets on the scheme-deployed line cards to transfer to each of them in a round robin fashion for DLC lookup. 2.1 Using Aggregate routes Also periodically on the non-DLC linecards a Route aggregation similar to the S-VA or the simple Virtual Aggregate with exception entries is generated further to reduce the TCAM occupation. 2.2 Switching off the TCAM banks which are unused The TCAM banks which are not occupied as a result of this scheme may be switched off completely along with their associated SRAM banks as well which contain the rewrite information. The paper also opens up the idea to simulations which help strengthen the argument for this scheme. Further examples of this will be dealt with in the future versions of 2.3 Using Aggregate routes Also periodically on the non-DLC linecards a Route aggregation similar to the S-VA or the simple Virtual Aggregate with exception entries is generated further to reduce the TCAM occupation. The TCAM banks which are not occupied as a result of this scheme may be switched off completely along with their associated SRAM banks as well which contain the rewrite information. The paper also opens up the idea to simulations which help strengthen the argument for this scheme. Shankar Raman et.al Expires May 2013 [Page 7] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 Further examples of this will be dealt with in the future versions of this document. 2.4 Advantages Though it seems to be complex, the TCAM banks unused as a result of this scheme save power when they are empty. The switching logic may not use them since they are empty. Power consumption of related SRAM banks that map to these TCAM banks , which store datum connected to route entries represented in the respective TCAM entries are also saved from being refreshed. A finer granularity of TCAM banks w.r.t to their size and the number of TCAM entries that occupy these banks can be achieved to derive maximum benefit from this scheme. 3 Security Considerations This section of the document will be duly filled in the later versions of the document. 4 IANA Considerations No IANA considerations need to be considered as part of this document. 5 References 5.1 Normative References TBD 5.2 Informative References TBD Authors' Addresses Shankar Raman et.al Expires May 2013 [Page 8] INTERNET DRAFT TCAM power reduction methods in Routers November 2012 Shankar Raman Department of Computer Science and Engineering I.I.T Madras, Chennai - 600036 TamilNadu, India. EMail: mjsraman@cse.iitm.ac.in Balaji Venkat Venkataswami Department of Electrical Engineering, I.I.T Madras, Chennai - 600036, TamilNadu, India. EMail: balajivenkat299@gmail.com Prof.Kamakoti Veezhinathan Department of Computer Science and Engineering, I.I.T Madras, Chennai - 600036, TamilNadu, India. Email: kama@cse.iitm.ac.in Prof.Gaurav Raina Department of Electrical Engineering, I.I.T Madras, Chennai - 600036, TamilNadu, India. EMail: gaurav@ee.iitm.ac.in Shankar Raman et.al Expires May 2013 [Page 9]