DevOps has been all the rage lately, and for good reason: it allows an organization to provide a more consistent environment for development, test, and production, decreasing configuration errors, all while reducing development time. Another name given to this technique is “Infrastructure as code”. This is generally good for security, as many technical attacks exploit inconsistencies in an environment, plus it reduces the time to roll out security improvements. Unfortunately, it also means that new security issues can be rolled out much faster. This has led to the notion of integrating security into the DevOps process; at the same time there is the notion of applying DevOps for security infrastructure. These notions are alternatively titled “DevSecOps” and “SecDevOps”. In this post, we’ll look at the benefits and risks of DevOps, how it can be secured and applied to security, and how that should inform the use of the terms DevSecOps and SecDevOps.
The term DevOps derives from the core idea that operations (infrastructure) should be developed as code. This has been implemented in numerous frameworks such as Ansible (which we use internally at BuboWerks), Puppet, and Chef. In these frameworks one can create configuration files which, when run, setup or modify an environment (be that a bare metal server, local VM, or IaaS instance) to match that configuration. This allows infrastructure development to live in tandem with code development; in particular, it allows the configuration to be managed along with the code in a repository such as Git. It also makes it much easier for development teams to work on a true mirror of production, which they can create on the fly and tear down when they’re done with it — a huge improvement over the days when we used to spend days setting up development, test, and production racks, often cannibalizing from each other depending on where we were in the development cycle. It also means teams can set up multiple environments instead of stepping on each other. In this age of increasing cloud adoption, it also means lower infrastructure costs as developers only need to run instances during the time they are actively using them.
These advantages combine together to mean faster development time and better security. The faster development time comes from the increased speed developers have access to infrastructure, less time spent dealing with contention for infrastructure, and less time debugging issues that result from inconsistent environments. DevOps also encourages faster development, test, and production cycles, such that a fix developed this morning can be tested and pushed out to production this afternoon with very little risk due to the high degree of assurance that the development and test environments matched production; or production will automatically update to the proper configuration. Less bugs with faster fixes means less security issues. The increased consistency between environments also improves security. No longer do you have a backdoor that lingers on production for months or years from a developer who only needed it there for debugging for a few hours.
Despite the many advantages, there are some disadvantages, particularly from a security perspective. Historically, application security (AppSec) programs depended on a long development cycle where security reviews could be performed between design and development, and again between development and production, usually in tandem with other code reviews or testing. The increased pace of development means that there is usually not a formal design review, code reviews tend to be frequent, small affairs — often involving only one other developer — and testing is fast and automated.
We still need a way to integrate security into this process (add Sec into DevOps); fortunately, there are many potential hooks. First off, knowledgeable developers are the best defense against bugs in application code. If you want to perform like the top organizations on the planet and deploy code frequently, make sure you your developers know the OWASP Top Ten — provide them any training they need so they take it to heart. Second, always conduct code reviews on anything pushed to production, and make sure at least one of the reviewers is knowledgeable about application security. Include someone with specialized knowledge such as an application security specialist from the security department or a consultant such as BuboWerks if you are touching any security sensitive code such as authentication, or session management. Third, make sure your developers know the security functionality in the frameworks you are using; any code changes that introduce new packages should receive extra scrutiny. Fourth, integrate static code security analysis and dynamic application security analysis into your development process such that changes must pass before they can be pushed to production.
The DevOps phenomenon can also be useful to security teams: all the advantages they bring to development teams can be realized in rolling out security controls. This can cover everything from integrating in the desktop rollout procedure to automatically include the organization’s standard EDR agent, to using Ansible to roll out controls such as Bro, to managing the configurations on security appliances such as web proxies. While it might seem like a large time investment for an already overworked security team to try to automate their controls deployment, in the long-term it will be a time savings as less time will be spent on management of existing controls.
So where does this leave us with terminology? Is integrating security into DevOps make it SecDevOps, or DevSecOps? We propose that DevSecOps is the appropriate term, and this does seem to align with what others are starting to use predominately. One can think of it as making security the core of DevSecOps. This conveniently allows us to use the term SecDevOps for Security teams using DevOps for their own infrastructure. In summary:
DevSecOps: Integrating security into the core of DevOps through informed developers, reviews, and security analysis.
SecDevOps: Applying DevOps principals for the rollout of security controls and associated infrastructure.
Whichever one of these applies to your situation, there is much to be gained by taking a DevOps approach. If you need help with either DevSecOps or SecDevOps, contact us and we’ll discuss how we can help!