In this article, Josh Stephens, CTO at BackBox, explores nine best practices for approaching network infrastructure upgrades and highlights common mistakes to avoid along the way.
Last year, malicious cyber actors exploited older software vulnerabilities more frequently than recently disclosed vulnerabilities and targeted unpatched, internet-facing systems, according to CISA. A fact that won’t surprise many in the IT world is that the top exploited vulnerability was fixed more than four years ago. In a survey we commissioned earlier this year, 92% of respondents said they couldn’t keep up with network update velocity, and 53% (rising to 68% for larger companies) only upgraded network and security devices quarterly or less frequently.
Network infrastructure upgrades play a critical role in maintaining the operational effectiveness and security of an organization’s network. However, approaching these upgrades requires careful planning, the involvement of key stakeholders, and the implementation of robust automation workflows. Here are nine strategic best practices to follow.
Not all upgrades are equal. It’s important to understand where you have high-severity common vulnerabilities and exposures (CVEs), devices at or approaching end-of-life (EOL), and devices running custom or non-standard code. Then you can prioritize upgrades to focus on vulnerabilities that matter most to your organization, assess whether it’s time to retire a device and upgrade to a new device, and ensure you have the expertise in place to handle custom upgrades manually. With a methodical, risk-based process, teams will be able to decrease risk while doing updates more rapidly.
Before initiating a network infrastructure upgrade, it is essential to document and clearly understand the scope of the project. Identify whether it falls under urgent or regularly scheduled maintenance and outline the specific workflows involved. Determine the number of devices, their current operating system levels, redundancies in place, and whether hardware, configurations, and bandwidth upgrades are required. A comprehensive understanding of the upgrade scope serves as the foundation for successful planning and execution.
A well-defined upgrade plan should include key goals to guide the project. Strive for zero downtime or, at the very least, minimal impact on the organization. Establish goals for each phase of the project, from audit and discovery to post-upgrade analysis and debriefing. Additionally, for larger upgrade projects, set daily and weekly goals for the number of devices to be upgraded and success ratios for automation-driven upgrades.
The involvement of key stakeholders is crucial for successful planning and execution. Depending on the upgrade scope, include the project leader, a technical leader or architect, the VP or director of network operations, the NOC leader, and, if applicable, a representative from the CISO’s organization. Ensuring a diverse range of perspectives and expertise will result in a more comprehensive and effective upgrade plan.
To maintain normal network operations during the upgrade, it is essential to have proper redundancies in place. Automation workflows can be leveraged to reroute traffic, change IP addresses, and mitigate operational impacts. For upgrades that may result in downtime, carefully schedule them to minimize disruption to the organization, considering periods of low network usage or implementing temporary workarounds.
Several common mistakes can hinder the success of network infrastructure upgrades. First and foremost, never attempt upgrades directly on the production network without thorough testing in a lab environment. Procrastination and neglecting upgrades under the assumption that they are not important can have serious consequences for network security and performance. Recognizing the significance of upgrades and implementing a well-architected automation strategy to manage them efficiently is vital.
Individual device upgrades can be time-consuming, expensive, and prone to errors. Embracing automation as a central component of the upgrade strategy allows for expedient, reliable, and cost-effective upgrades. Organizations can streamline the upgrade process by implementing a well-designed automation strategy, freeing up valuable time for teams to focus on more strategic and interesting work.
See More: A Guide to Converting Network Automation Skeptics
Often, there is no patch available for critical CVEs in devices at or approaching EOL, so the only choice is upgrading to something newer. Automating the tracking of EOL data for hardware, software, and even specific components helps organizations stay on top of device lifecycle management, enabling proactive budgeting and planning for the retirement of old devices and procurement of new technology.
Automation can also simplify and accelerate the upgrade process to the new devices, including rerouting traffic to avoid downtime, restoring the configuration from the old device to the new one, and even translating configuration changes as needed if there are inconsistencies between devices.
Organizations can also strategically leverage network automation to upgrade for redundancy, for example, by adding a second link. Many of the same upgrade processes and best practices previously discussed to address vulnerabilities also apply to resiliency upgrades, including validating that the second device is working correctly in a lab environment and then moving traffic over and load balancing.
Redundancy also makes OS upgrades smoother and easier in the future, as there’s no need to plan for downtime or disruptions. Traffic can be switched over so that it all goes through one device. Then, automation workflows can streamline upgrading the other device that is not in use and validate it is working. The workflow is repeated with the second device, and traffic is load-balanced between the devices once they are successfully upgraded.
As long as organizations continue to fall behind in addressing critical network upgrades, threat actors will continue to use older vulnerabilities to disrupt operations and steal sensitive information. Approaching network infrastructure upgrades with a strategic mindset and leveraging automation workflows is essential for ensuring a smooth and successful process to close gaps attackers use to their advantage.
By documenting the upgrade scope, setting key goals, involving the right stakeholders, maintaining normal network operations, avoiding common mistakes, embracing automation, and addressing device lifecycle management and resiliency, organizations can upgrade their network infrastructure efficiently, enhance security, and unlock the benefits of enhanced features and performance. Remember, a well-executed upgrade plan is crucial for maintaining your network’s operational effectiveness and security.
Are you following any of the discussed best practices for your network infrastructure upgrades? Share with us on Facebook, X, and LinkedIn. We’d love to hear from you!
Image Source: Shutterstock
/cloudfront-us-east-1.images.arcpublishing.com/gray/TPPPA3BVIBGHJIET2KYDUDJE44.jpeg)
TOPEKA, Kan. (WIBW) - Kansas Governor Laura Kelly, along with the Kansas Infrastructure Hub (KIH), announced on Tuesday, Sept. 19, the launch of a $200 million
ST. PAUL (WJON News) -- A number of small Minnesota communities will benefit from $24-million in Small Cities Development Program grants to upgrade public faci
On Tuesday, MEPs in the Industry, Research and Energy Committee adopted their negotiating position on draft legislation that would reduce the costs of deploying
On a recent Sunday afternoon, as water pooled in my driveway, I received a text from a friend that Dean Street in Providence had not only flooded, but had a cur
