Securing DevOps Pipelines from Code to Cloud
Cycode is a detection and control platform for source code management (SCM) systems. It delivers full visibility into enterprise development infrastructure and detects and responds to anomalies in access, movement, and usage of your source code. Cycode hardens your SCM system by detecting secrets leaked to the code and preventing code tampering. Cycode also scans your CI/CD settings and Infrastructure-as-Code (IaC) for security misconfiguration and prevents source code tampering in the cloud by identifying drift between production systems’ IaC configurations. As a comprehensive platform for SDLC security, governance, and integrity, Cycode prevents theft and leakage of your code and is a single source of truth for the governance of your DevOps pipeline.
Hours and days spent on log-based debugging across 70 different microservices
Cycode’s platform comprises about 70 different microservices. The highly distributed architecture makes debugging in production even more complex than usual. Cycode would struggle to determine the root cause of errors with complex code execution flows across several microservices from initiation to manifestation of the error.
Before using Ozcode, Cycode’s primary method for debugging in production was to add logs, rebuild and redeploy the application, reproduce the error and analyze the new log entries to try and determine the root cause. However, this method could be both ineffective and inefficient. Ineffective because in some cases, several iterations of this cycle were required before Cycode’s developers understood what caused the error. Inefficient because even though Cycode has an extremely efficient CI/CD pipeline, it would still take an hour to deploy a new build so the developers could analyze the data in the new log entries. Consequently, resolving an error would take anything from several hours to days of going through the arduous, repetitive cycles of “log-only builds.”
Easy onboarding and instant results with Ozcode
After evaluating different offerings, Cycode selected Ozcode because of the extensive support for .NET (including abstract classes and functions which Cycode uses extensively) and a highly responsive support team that helped Cycode in the adoption process. Getting started with Ozcode was smooth and simple. By adding a few lines of code into the dockerfile of their base image, Cycode could redeploy their application to include the Ozcode agent in all their Docker microservices. Connecting to the agent in the microservices from the Ozcode front end was completely straightforward. Within minutes, Cycode was able to add tracepoints and start using dynamic logs to debug their application in a live production environment.
Bugs were diminished by an order of magnitude
While Cycode’s CI/CD pipeline was very efficient, deploying a new build still took about an hour to complete. Some bugs, especially complex ones, could require several CI/CD cycles to determine the root cause of the error and fix it. Ozcode’s dynamic logging with tracepoints completely changed the debugging paradigm for Cycode. Instead of going through a complete CI/CD cycle for each iteration of adding logs, Cycode’s developers could add tracepoints anywhere in the code and just let the application run. Log output was immediately available for analysis. By eliminating arduous and time-consuming CI/CD cycles, Cycode’s developers could quickly drill down into their application’s state anywhere in the code to determine the root cause of errors.
By eliminating hours of repeated CI/CD cycles, Cycode cut the time taken to fix production errors by 70 – 80% on average.
Preventing faulty deployments from reaching production
Cycode discovered unexpected benefits of using tracepoints.
Feature validation and edge cases
Cycode’s developers discovered that by adding dynamic log entries at strategic points in the code, they could validate new features that were introduced into their platform. By examining dynamic log output in their staging environment, Cycode validated that the application flow of a new feature was behaving as expected for different inputs and producing the correct output at each step. The developers also found this to be an extremely efficient way to validate the application behavior in different edge cases. Instead of having to deploy a build to test new log entries along an obscure application pathway, Cycode developers added dynamic log entries on the fly and validated all the edge cases very quickly.
By validating new features and edge cases with tracepoints and dynamic logs in Ozcode, Cycode prevents logical errors from being deployed to production.
Detecting a misconfigured CI/CD pipeline
Some live debuggers use symbols to connect a live system to the source code used to build it. Ozcode uses a different paradigm and shows you code that was decompiled directly from the live system being debugged. This enabled Cycode to discover bugs in their CI/CD pipeline.
Validating a new feature using Ozcode revealed to Cycode’s developers that a certain commit was not deployed to staging as expected due to a misconfiguration of the CI/CD pipeline. Without Ozcode’s decompiled code, Cycode’s developer would not have any straightforward way to understand which code was actually deployed. Even with a sophisticated versioning mechanism in place, the scenario is unexpected and could easily have been missed allowing a seemingly correct build to be deployed to production only to display confounding production errors that would be extremely difficult to resolve. With Ozcode, it was easy to see that the expected fix was not in the build deployed.
Ozcode helped Cycode uncover a misconfiguration of their CI/CD pipeline that would have caused a production error that was extremely difficult to resolve.
Cycode resolved errors in their production and pre-production environments using log-based debugging. However, even with a very efficient CI/CD pipeline that could deploy a build in an hour, the debugging process was long and arduous because typically, resolving a bug required several iterations of adding logs and running a new build.
Cycode investigated different live debugging alternatives. The main reasons they chose Ozcode were:
- Ozcode has the most comprehensive support for .NET.
- Ozcode works with code that is decompiled from the live application being debugged.
- Ozcode’s support team was highly responsive, enabling a rapid time to value.
After a short and smooth onboarding process, Cycode immediately saw the benefits of dynamic logging with tracepoints. Cycode’s developers:
- Slashed their debugging time by 80% by eliminating repetitive CI/CD cycles required from their traditional log-based debugging.
- Increased developer velocity by validating new features and edge cases.
- Prevented faulty deployments from reaching production by identifying misconfigurations in their CI/CD pipeline.