As a .NET developer, no doubt you’re familiar with NuGet Gallery, the most comprehensive of all public NuGet repositories providing open source NuGet packages on the web. NuGet Gallery is extremely popular, delivering about one billion NuGet packages to developers every week.
If you’re reading this post, you’ve probably downloaded a package or three in your time. And my question to you is when you downloaded those packages, where did store them? Where did your colleagues store the packages they downloaded? Did any of your builds ever get stuck because NuGet Gallery was down, or there was a problem with your internet connection? Where do you store NuGet packages that you build yourself? How did you share them with your teammates working on the same project? How do you control who has access to them? These are all questions you need to consider when working with NuGet (or any other package format for that matter).
Let me introduce you to JFrog Artifactory, a DevOps solution that provides end-to-end automation and management of binary artifacts. It hosts the packages you build in-house and is a proxy for any remote resource you want to access to download packages from outside your organization. i.e., instead of pointing your NuGet client at NuGet Gallery, you point it at Artifactory.
Artifactory supports all major package formats, including NuGet, and provides an answer to all those questions I asked above.
Now, you may ask why I’m devoting a whole post to Artifactory on Ozcode’s blog. It’s because (and here’s the full disclosure) I worked at JFrog a while back and saw up close and personal how Artifactory developed, and how it supports developers in their daily work. If you use NuGet, you should definitely give it a try. In this post, I’ll show you why by describing five ways in which Artifactory boosts your .NET development with NuGet (and I’ll mention a few more).
Don’t go down with Nuget Gallery
We’ve established that Nuget Gallery is the ultimate resource for open-source NuGet packages. But like all remote resources, it can (and does) go down. This screen capture is taken directly from NuGet Gallery.
If one of your local builds requires a new dependency, and NuGet Gallery is down…bummer, let’s go make coffee and hope it’s back up soon. For that matter, the same can happen if there’s a problem with your external network. That’s one level of annoyance. But what about your build server. It’s running builds all the time. If builds get held up, releases get held up, and if there’s a deadline to meet, everyone has to stay late.
@#$%^!!! No NETFLIX tonight!
Artifactory remote repositories can put NETFLIX back on the evening’s agenda in these cases. An Artifactory remote NuGet repository acts as a caching proxy for NuGet gallery. The first time anyone connected to Artifactory needs a dependency, Artifactory will download it from NuGet Gallery and cache it. From then on, that dependency is locally available to anyone who needs it. So even if NuGet Gallery (or your external network) goes down, your dependencies are still available for your build to succeed.
Share dependencies and don’t let the network be a bottleneck
Remote NuGet repositories in Artifactory have another cool advantage.
Imagine there’s two of you working on a project together. Does it make sense for both of you to download the same dependency separately? Well, if it’s really only two of you, why bother even asking the question? But what if there are tens or hundreds of you working on the same project? You collectively need the same hundreds or even thousands of dependencies. That can amount to significant network traffic if everyone is downloading redundantly. And to build servers grinding builds out all day long, downloading dependencies over the network can make each build take a very long time.
Think big release, think midnight, think NETFLIX.
So, getting back to Artifactory remote repositories, if your build servers are accessing all those dependencies locally, there’s no download time to consider, and builds are going to succeed much faster.
Your proprietary NuGet packages all neatly stored in local NuGet Repositories
While most of your application is probably assembled from different open-source (and perhaps some commercial) packages, some of the code is indeed your own, the packages that define the unique business logic of your application. You need a place to store them; somewhere everyone in your organization knows to look when they need something. That’s what Artifactory local NuGet repositories are for. Once everybody points their NuGet client to Artifactory, that’s where they’ll get your proprietary NuGet packages. And even if you copy the same package to multiple repositories, Artifactory automatically deduplicates the package by using a checksum-based storage mechanism (which has a bunch of other benefits, but I’ll leave you to check that out for yourself).
Control who gets access to what
I’ve talked about remote NuGet repositories, which are caching proxies for remote resources like NuGet Gallery. But there are other remote NuGet resources from which developers may be downloading packages. You never know which of those resources harbor nefarious packages. I also mentioned local NuGet repositories where you upload your own NuGet packages. The question is, should anyone be able to upload anything to any repository? Should they have access to ALL the local NuGet repositories? So, let me now introduce virtual repositories.
A virtual repository is an aggregation of local and remote repositories in Artifactory. It provides a single URL through which developers and build tools access NuGet repositories to upload and download packages. As an Artifactory admin, you can decide which remote NuGet repositories are included in this virtual aggregation, and thereby ensure that only trusted resources like NuGet Gallery are used to download dependencies. You can also specify which local repositories are included as resources for your proprietary packages. Let’s see this in action in a simple example.
Dev and QA are working on a project.
Dev has a local repository for their exclusive use.
QA has a local repository for their exclusive use.
There’s also a shared local repository that both Dev and QA can use.
Both teams also need access to a shared remote NuGet repository that proxies NuGet Gallery.
Here’s how that setup might look.
Now, access control doesn’t end there; it’s not all or nothing for a repository. You can define what are called include and exclude patterns to exercise control over which specific paths within a virtual repository any user can access. And I’ve only scratched the surface. Artifactory’s security model includes several layers, including Users and Groups, Permissions, Access Tokens, and more, all of which enable fine-grained access control allowing you (or rather Artifactory admins) to decide exactly who can access what – from the level of a whole repository down to a single artifact.
Reproduce the build, not the bug
Ozcode Production Debugger has relegated the need to reproduce Production issues to the pages of debugging history. Instead, to help you debug an error, Ozcode autonomously captures and records the error execution flow and lets you go back in time, step-by-step, with code-level visibility to determine what went wrong. Once you’ve figured that out, you’ll want to implement a fix. Here’s where Ozcode Production Debugger and Artifactory meet.
You may want to fix the exact build in which the error occurred. But builds are complex creatures, with possibly thousands of artifacts (each with their own version) and configurations. By the time your error occurs, you may be many builds beyond, and reproducing the exact build with the error can get very tricky. Artifactory has a solution for this.
When your build server runs through Artifactory (downloading dependencies from Artifactory, and uploading the resulting build), it stores exhaustive build information including specific package versions, dependencies, system properties, environment variables, user information, timestamps and more. This extensive “bill of materials” enables you to exactly recreate the specific build that displayed the error you just debugged.
One size fits all
Polyglot programming is the norm in most software development organizations. While you may have a local NuGet expert, and a Docker guru, your organization as a whole, and indeed, most developers, work with several development technologies at the same time. In fact, in the 2020 Stack Overflow Developer Survey, 55% of respondents identified as full-stack developers. So just like Artifactory is a great tool to manage your NuGet packages, the same goes for your Docker images, NPM packages, Python packages, and more. In fact, Artifactory supports all major package formats in common use today. That means you manage all your packages using the same methodology, same APIs, same admin, same everything – all your binaries in one comprehensive mothership under one roof.
And there’s more…much more
I’ve only touched the surface on how Artifactory helps accelerates your .NET development with NuGet. This great product offers many more features like advanced search for packages, high availability, user plugins for custom behavior, repository replication and more. But Artifactory is just a good place to start. If you check it out, you’ll find that it’s just the leading player in the larger JFrog Platform that is a much larger solution covering the development, build, security and distribution of the binary artifacts that make up your application. But if you want to hear more about that, talk to the frogs.