Broadcom’s use of SONiC switches in their network infrastructure is a great example of how open-source solutions, combined with modern network orchestration tools, can redefine data center operations.

Key takeaways

Broadcom operates 9 data centers worldwide, largest located in Las Vegas, housing up to 4,500 network devices, and potential expansion to support up to 6,000 switches.
A bespoke orchestration tool allowed them to reduce operational costs significantly
SONiC helped Broadcom achieve requirements in scalability, robustness, cost-effectiveness, secure tenant segmentation, high performance, and automation

As Broadcom absorbed VMware’s operations, the company needed to replace a legacy three-tier architecture. The reason was simple – they were looking to expand, scale-up and improve their data center operations. Broadcom was looking for a solution that could scale easily, reduce costs, and increase overall operational effectiveness.

Who is Broadcom?

A global tech-leader specializing in semiconductor and infrastructure software solutions. The company invests approximately $5 billion annually in research and development, supporting its extensive operations across 9 data centers. With over 40,000 employees, Broadcom’s innovations power a wide range of applications, from smartphones and data center networking and industrial automation.

The clear choice was SONiC, an open-source, community-driven network operating system. In the end, SONiC allowed Broadcom to achieve the outlined goals, creating an efficient and scalable network. Other sectors should draw inspiration from building a resilient, future-proof infrastructure.

But why SONiC?

Broadcom’s previous network was a conventional 3-tier model, which relied heavily on vendor-specific solutions. This mainly meant high licensing and maintenance costs.

But let’s break down what a 3-tier architecture looks like:

Core layer: handles most of the traffic between devices
Aggregation/distribution layer: connects the core & access layer, while filtering, routing and controlling traffic. Broadcom used this layer to support failover and redundancy.
Access layer: primarily provides a direct connection to network devices.

Broadcom mentioned it was drawn to SONiC for its open-source flexibility, automation potential, and scalability. This radical step made them avoid vendor-locked and pricey platforms and enjoy everything SONiC has to offer.

But what does SONiC offer?

Vendor independence = lower costs

One of SONiC’s greatest appeals for Broadcom was the opportunity to break free from vendor lock-in. Their legacy system required proprietary vendor intervention, which not only came with recurring licensing fees but also limited their ability to adopt alternative, more cost-effective hardware options.

SONiC’s adaptability also allowed Broadcom to integrate and scale its infrastructure quickly at a critical moment – as it acquired VMware’s data centers.

Automation and even more reduced costs

Network automation was a central goal in Broadcom’s migration to SONiC. The previous system relied heavily on manual configuration, which was labor-intensive and prone to errors. With SONiC, Broadcom initially leveraged Ansible scripts to automate configuration tasks.

The bespoke orchestration system would be rolled out by then, thus reducing the operational overhead. – Tobin Hawkshaw, Network Architect, Broadcom

As they became more advanced, they moved to a bespoke orchestration platform, significantly reducing manual interventions. Automation allowed Broadcom to minimize operational complexity, enabled fast deployment, updates, and streamlined maintenance. By automating repetitive tasks, you are freeing staff to focus on higher-value activities, and improve service response times.

Robustness agrees with growing demand

Broadcom expanded its data center drastically. Currently, they operate 9 data centers all over the world. The largest being the Las Vegas DC, with up to 4500 network devices.

Source: Broadcom OCP SONiC Summit 2024

It became critical to support high-density traffic with a network architecture that could scale dynamically. The legacy system, with its 3-tier design, had limited flexibility, creating bottlenecks as traffic loads increased.

SONiC enabled Broadcom to scale up or down based on demand, with efficient horizontal and vertical expansion options. SONiC’s scalability ensured that networks could expand without extensive re-architecture or increased operational costs.

Improved network resilience with reduced downtime

The migration to SONiC included adopting EVPN VXLAN, which enabled Broadcom to isolate network traffic into distinct virtual segments, enhancing both security and reliability. By implementing these advancements in SONiC, Broadcom contributed back to the community, readying the platform for broader enterprise adoption.

Introducing these technologies boosted Broadcom’s team confidence in delivering uninterrupted services. Operations remained seamless even during hardware failures or maintenance windows.

Enhanced network security

SONiC provided Broadcom with the ability to secure its network through a decentralized architecture. Instead of a single centralized firewall, which can be a point of failure, Broadcom adopted a distributed firewall strategy. This ensured that traffic across various segments could be monitored and controlled closer to its source. This decentralized approach allows for a more secure network design with fewer vulnerabilities.

Building a future-ready infrastructure

However, this is not the end of Broadcom’s SONiC endeavor.

Broadcom plans to improve the bespoke orchestration system currently used, as well as upgrade to 1.6 Tbps to support its high-speed East-West traffic.

Broadcom’s ongoing efforts to expand SONiC’s footprint into its WAN and campus networks illustrate the potential to create a standardized, end-to-end infrastructure that is automated, flexible, and highly reliable.

When, if not today?

Broadcom’s adoption of SONiC is a great example of how open-source solutions can transform traditional network infrastructure. By introducing a flexible, automated, and resilient network approach, Broadcom has set a new trend for modern data centers, with significant cost savings, scalability, and operational efficiency.