Cisco ACI

What do you like best?

The rest API makes it easy to automate if you have the right talent that knows how to actually interact with APIs (Which is not most of the network engineers that I've met/dealt with even at Fortune 500 companies there were very few people out of a community of 50+ network engineers there were 3 people that actually understood ACI and how to interact with it programatically)

What do you dislike?

Very few people actually understand what's going on.

Working on this at multiple companies I've consistently run into scaling issues. The first company scaled wide and had a number of 100+ leaf fabrics that we had numerous issues with. The second company and my current employer scaled deep and put Cisco Fabric Interconnects hanging off the ACI Leafs and we've run into numerous scaling issues as well. In both cases Cisco support has recommended the opposite way of scaling and I've now seen both scaling methods cause numerous issues that caused impact and outages for our clients. In several of those instances the only way to recover was to acidiag touch clean & reload which forces the switch to pull down a fresh copy of it's entire config. In most of these cases it took escalating within TAC to get a senior engineer engaged to actually get a resolution. This was after spending several hours on the phone with TAC while our clients were being impacted and in some cases in complete outages.

Recommendations to others considering the product:

I would strongly recommend against it, under most circumstances. Even if you've already invested in the hardware I would strongly advise you to consider using a tool like Ansible and running the switches in Native mode to accomplish much of what the APICs are able to do by configuring the accomplish. About the only thing you wouldn't be able to do that ACI allows you to do is the Application Centric Model and the Tenants/App Profiles/End Point Groups, but we've run into so many issues along the way with ACI at multiple companies with large and talented networking teams that I feel like it's a small trade off. The Application Centric Model of networking is useless if the network is constantly down/having major issues/impacts to you and your customers.

What problems are you solving with the product? What benefits have you realized?

Automating network deployments is fairly straight forward once you have the original recipe down, but the same can be done with other automation tools such as ansible on traditional networking equipment or even the Cisco Nexus 9000 series switches in native vxlan mode. The benefits aren't really anything compared to what I've also been able to do using Ansible and traditional networking models. The argument is often made that it gives you an additional layer of security using tenants, app profiles, end point groups, and policy. But the unfortunate reality is that the resources allocated for the security policy are extremely limited and again I've run into scaling issues here. You can carve us the CAM differently to address this by switching to a different "Forwarding Scale Profile", but you only have so much CAM to reallocate for the various tasks (routing IPv4, IPv6, & Security Policies)