Scan a repository for secrets with HCP Vault Radar

8min
|
HCP
Vault

In this tutorial, you will follow the HashiCups teams as they set up a proof-of-concept (POC) implementation of HCP Vault Radar.

Scenario introduction

After seeing the demonstration of HCP Vault Radar, the HashiCups teams have decided to move forward with a POC to test out the features and functionality in their own environment.

For the initial phase of the POC, the engineering teams have decided to focus on HCP Vault Radar's ability to identify secrets in their source code.

Alice and the architect team have reviewed the documentation and selected GitHub to use for the POC. Danielle from the development team is going to create a sample repository and commit different types of data they hope to find with HCP Vault Radar.

Oliver (operations) and Steve (SRE) will be standing by to monitor the results using the HCP Portal.

Prerequisites

Access to the HCP Portal with a user assigned the admin role.
Access to a GitHub account with an organization configured.
Permission to create a GitHub personal access token.
You do not need to be familiar with the Go programming language to follow this tutorial.

Create a test repository

(Persona: developer)

Note

If you do not have have access to a GitHub organization, create one before proceeding with the tutorial.

Organization names are unique across GitHub. Replace references to the radar-example organization with the name of your organization.

Log in to your GitHub account and access the https://github.com/hashicorp-education/hcp-vault-radar-foundations example repository.
Create a fork of the repository and select the desired organization.
Review the main.go file. Lines 11 and 12 include a username and password. This username and password represents secrets that HashiCups does not want included in their source code.
Secrets such as passwords, API keys, and other sensitive information should not be committed to source.
Click on the docs directory and review index.mdx file.
Line 18 includes an example API key. Documentation may include example secrets to help direct users on how to use the source code or application. Consider using an example secret that is clearly identified such as EXAMPLEKey or EXAMPLEpassword.

Add repository to HCP Vault Radar

(Persona: operations)

Open a new browser tab, log in to your HCP organization and select the project HCP Vault Radar is enabled for.
Click Get started with Vault Radar.
Click on Settings and then click Data Sources.
Select HCP Cloud Scan and click Continue.
Click GitHub Cloud.
Enter the name of your GitHub organization.
Click the Generate a GitHub token with appropriate scopes... link, a new browser/tab will open.
On the New Personal Access Token page, enter hcp-vault-radar in the note field and select an expiration date.
Scroll to the bottom of the New personal access token form and click Generate token.
Copy the personal access token and enter it in the Personal Access Token text box on the HCP Vault Radar Configure GitHub Cloud page.
Click Next and select the Select repositories to monitor radio button.
Select the hcp-vault-radar-foundations repository and click Finish (1 selected). HCP Vault Radar will perform an initial scan of the selected repository.
When the initial scan completes, Steve (SRE) and Oliver (operations) review the findings on Overview dashboard.
Click Events in the left navigation menu and use the Filters pull down menu to select All events.
Click the accordion button to expand the details for the MEDIUM severity Password assignment item.
HCP Vault Radar provides details about the findings from the initial scan including:
1. Context or repository the reported item was found
2. The author from the item committed
3. The Commit Hash to help you locate the line where the item was found
4. A Unique Secret ID that was generated for the secret

Filter events

(Persona: SRE)

Steve and Oliver are able to see the secret that was visible on line 18 of the main.go file. However they have asked why they are unable to see the API secret key from /docs/index.mdx.

From the Events page, expand the State menu.
Check the Not Important checkbox.
Click the accordion button to expand the details for the LOW severity Secret assignment item.
Hover the cursor over the Secret in example file notification.
HCP Vault Radar found, and through its heuristics engine, was able to determine this secret is likely a false positive.

Create a global ignore rule

(Persona: operations)

HCP Vault Radar scanned and identified both secrets in the GitHub repository used for the HashiCups POC. The engineering teams would like to see how to exclude specific items to help further limit false positive findings beyond the native HCP Vault Radar heuristics engine.

From the Events page, click Settings in the left navigation menu and then click Global Ignore Rules.
Global ignore rules allow you ignore items found by HCP Vault Radar based on paths, specific secret examples, or secret types. You can also create per repository rules based on those items so that they are only ignored on specific repositories.
Copy the ignore rule to prevent HCP Vault Radar from scanning the docs path in the hcp-vault-radar-foundations repository and paste the rule in the Global Ignore Rules text box.
```
- paths:
  - docs/index.mdx
```
Click Update. The ignore rule is added.
Click Resources in the left navigation menu and then click hcp-vault-radar-foundations.
Makes note of the Scan Status time and click Schedule a rescan to perform a new scan of the repository.
When the Scan Status time updates, click Events in the left navigation menu.
Select the All events filter from the pulldown menu and click the Not Important checkbox.
The event is now flagged as having an active ignore rule.

Commit secrets to GitHub

(Persona: developer)

Danielle from the development team would like to set up a PR check to prevent pull requests from being merged when HCP Vault Radar detects sensitive information.

From the Events page, click Settings in the left navigation menu and then click PR Checks.
The default settings are to fail the check on a pull request if any secrets are found, which matches Danielle's requirements for the development team.
Note
To prevent merging a pull request with a failed check, you must enable branch protection for each repository in GitHub. Refer to the GitHub documentation on managing branch protection rules.
Click the Install the HashiCorp Vault Radar app for GitHub Checks link, a new tab will open for the HCP Vault Radar GitHub app.
Click Install and select the organization where you forked the hcp-vault-radar-foundations repository to.
Click the Only select repositories radio button.
Select the hcp-vault-radar-foundations repository from the Select repositories pulldown menu and click Install.
Click the Repositories tab and click the hcp-vault-radar-foundations repository.
Click the main.go file and then click the pencil icon to use the GitHub editor.
Change the const password value to b3stp@stw00rd3vA!!! and click Commit changes....
Click the Create a new branch radio button and click Propose changes.
Click Create pull request (if prompted, click Create pull request again).
HCP Vault Radar will start a pull request scan.
When the pull request scan completes, the HCP Vault Radar Secret Scan will change status to Failed.
Click Details to review why the pull request check failed.
The pull request scan details show the status, as well as the file path and line number where the sensitive information was detected.

Summary

In this tutorial you learned how to add GitHub as a data source for HCP Vault Radar. You then performed an initial scan of a repository, added a global ignore rule, and installed the HCP Vault Radar GitHub app to scan pull requests.

Next steps

In the next tutorial, the operations and SRE teams at HashiCups will enable an alerting integration to receive real time alerts instead of relying on the HCP Portal for notifications.

Additional resources

What is secret scanning

Set up alert integration