Skip to content

Add > Guide > Intelligent Assessment, Intelligent Fixes, Policy Pack Summary and restructure AI section > Draft #352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Jun 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2b17fbb
Add > Guide > intelligent Assessment > Draft
RahulSrivastav14 Jun 23, 2025
172a70f
Updates
rajlearner17 Jun 23, 2025
1b07ef3
Update > Images and Sidebar
RahulSrivastav14 Jun 23, 2025
48a7a22
Update sidebar
RahulSrivastav14 Jun 23, 2025
1ecd269
Update using guardrails section
RahulSrivastav14 Jun 23, 2025
5878050
Updates to satisfy more informational need
rajlearner17 Jun 24, 2025
56ad65c
Updates to separate the sidebar and landing page
rajlearner17 Jun 24, 2025
0cac0f7
Updates and refactored the AI segment to consolidate all AI related f…
rajlearner17 Jun 24, 2025
cf51d09
Updates
rajlearner17 Jun 25, 2025
76ccab2
Updates
rajlearner17 Jun 25, 2025
7d0faf8
Updates
rajlearner17 Jun 25, 2025
814b79c
Updates
rajlearner17 Jun 25, 2025
0ead4a9
Updates
rajlearner17 Jun 25, 2025
6c9df20
Updates
rajlearner17 Jun 25, 2025
6cb130f
Tidy up
rajlearner17 Jun 25, 2025
1168f9f
Updates
rajlearner17 Jun 25, 2025
f62d295
Add Policy Pack Development with AI Tools
rajlearner17 Jun 25, 2025
046f1e4
Update # Guardrail AI landing page
rajlearner17 Jun 25, 2025
8cead8f
Updates
rajlearner17 Jun 25, 2025
f1a22fc
Updates
rajlearner17 Jun 26, 2025
53b5630
Updates
rajlearner17 Jun 26, 2025
0980786
Updates feedback-loop-2
rajlearner17 Jun 26, 2025
e013212
- Updated existing guides with new images and removed outdated ones f…
rajlearner17 Jun 27, 2025
fd49767
Remove outdated validator prompts and base style guide
rajlearner17 Jun 27, 2025
1f4f2de
Add concepts doc for Intelligent Assessment guardrail (#360)
SumitPopat Jun 27, 2025
46f519c
Merge branch 'main' into add-ai-assessment-guide
rajlearner17 Jun 27, 2025
4094a22
Updates as per copilot review comments
rajlearner17 Jun 27, 2025
46f73b0
Update & tidy up Enable Intelligent Assessment
rajlearner17 Jun 27, 2025
bd8905f
Updates
rajlearner17 Jun 27, 2025
caf1043
Tidy up
rajlearner17 Jun 27, 2025
b605aa7
Tidyup - copilot review
rajlearner17 Jun 27, 2025
57b71a1
Tidy up
rajlearner17 Jun 27, 2025
4eb4e79
Tidy up and update Turbot AI Configuration
rajlearner17 Jun 30, 2025
32a623a
Tidy up Policy Pack Development with AI Tools
rajlearner17 Jun 30, 2025
b76d387
Tidy up Enable Intelligent Assessment
rajlearner17 Jun 30, 2025
333fe3f
Tidy up formats and content
rajlearner17 Jun 30, 2025
17a223d
Tidy up Enable Policy Pack Summary
rajlearner17 Jun 30, 2025
a913bf7
reorganize sidebar for ai and developers sections
dboeke Jun 30, 2025
1f87b14
Update sidebar and guides to include links to 'Developing guardrails …
rajlearner17 Jun 30, 2025
c628a7b
Updates
rajlearner17 Jun 30, 2025
20f5a29
Update landing page
rajlearner17 Jun 30, 2025
5f3bae8
Updates and tidyup
rajlearner17 Jun 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/developers/developers-sidebar.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"type": "category",
"id": "developers",
"link": "developers",
"items": [
"developers/install-mcp",
"developers/policy-pack-development",
"developers/graphql",
"developers/nunjucks",
"developers/fix-calc-policy-evaluation-errors"
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,55 +22,55 @@ However, calculated policies can sometimes encounter errors due to misconfigurat

Log into the Guardrails console with provided local credentials or by using any SAML based login and Select **Policies** from the top navigation menu.

![Navigate to Reports](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-select-policies.png)
![Navigate to Reports](./guardrails-select-policies.png)

## Step 2: Select Policy Value

Select the calculated policy in an error state that needs to be resolved. This redirects to the Policy Value page, where the error message is displayed.

Here, the error occurs due to `TypeError: Cannot read properties of undefined (reading 'toString')`, indicating that the referenced property is undefined and cannot be converted to a string.

![Select Calculated Policy](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-select-calc-policy-in-error.png)
![Select Calculated Policy](./guardrails-select-calc-policy-in-error.png)

## Step 3: Select Calculated Policy

Select the **Calculated** policy, with an ✅ `EFFECTIVE SETTING`.

![Effective Setting](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-select-effective-calc-policy.png)
![Effective Setting](./guardrails-select-effective-calc-policy.png)

## Step 4: Edit Policy Setting

Select **Edit** from the top right corner.

![Select Edit](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-select-edit.png)
![Select Edit](./guardrails-select-edit.png)

Choose **Launch calculated policy builder**.

![Launch Calculated Policy Builder](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-launch-policy-builder.png)
![Launch Calculated Policy Builder](./guardrails-launch-policy-builder.png)

This displays the `GraphQL` query and `Jinja2/Nunjucks` template used in the calculated policy, providing insight into how the policy value is generated.

![Calculated Policy Builder Page](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/calc-policy-builder-page.png)
![Calculated Policy Builder Page](./calc-policy-builder-page.png)

## Step 5: Resolve Calculated Policy

Select the `Test Resource`, update the corrected Jinja2/Nunjucks template, and view the real-time output to verify if the fix is successful. Choose **Update**.

![Resolve Error](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-resolve-cal-policy.png)
![Resolve Error](./guardrails-resolve-cal-policy.png)

Select **Update** from the Update Policy Setting page.

![Select Update](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-update-policy.png)
![Select Update](./guardrails-update-policy.png)

## Step 6: Review

- [ ] Verify that the policy value transitions to an `OK` state, confirming the issue has been resolved successfully.

![Policy Value State](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-policy-value-ok.png)
![Policy Value State](./guardrails-policy-value-ok.png)

- [ ] Verify that the affected control transitions to an `OK` state.

![Control State](/images/docs/guardrails/guides/using-guardrails/troubleshooting/fix-calc-policy-evaluation-errors/guardrails-control-ok-state.png)
![Control State](./guardrails-control-ok-state.png)

## Troubleshooting

Expand Down
16 changes: 16 additions & 0 deletions docs/developers/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
title: Developers
sidebar_label: Developers
---

# Developing guardrails for Guardrails

This section covers the various AI-powered features available in Guardrails.

| Section | Description
| - | -
| [Install Guardrails MCP](/guardrails/docs/developers/install-mcp) | Learn how Guardrails Model Context Protocol (MCP) server is enabled.
| [Policy Pack Development](/guardrails/docs/developers/policy-pack-development) | Learn how to use AI to develop and validate policy packs.
| [GraphQL](/guardrails/docs/developers/graphql) | Tips and Tricks for GraphQL
| [Nunjucks](/guardrails/docs/developers/nunjucks) | Tips and tricks for using Nunjucks, including troubleshooting
| [Resolve Calculated Policy Errors](/guardrails/docs/developers/fix-calc-policy-evaluation-errors) | Troubleshooting calc policy errors
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: AI Tools (MCP)
sidebar_label: AI Tools (MCP)
title: Install Guardrails MCP
sidebar_label: Install MCP for AI
---

# Configure Guardrails MCP Server
# Install the Guardrails' MCP Server for AI Assisted Development

In this guide, you will:

Expand Down
205 changes: 205 additions & 0 deletions docs/developers/policy-pack-development/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
---
title: Policy Pack Development with AI Tools
sidebar_label: Using AI for Policy Pack Development
---

# Policy Pack Development with AI Tools

In this guide, you will learn how to:

- Use the MCP server's AI assistant to generate a production-ready Turbot policy pack
- Configure Terraform files, policy settings, and README according to Turbot best practices
- Test and validate your policy pack before deploying to production

This step-by-step guide will help you streamline policy pack development using AI, ensuring all required components are created and validated efficiently.

## Prerequisites

Before you begin, ensure you have the following:

- Guardrails MCP server installed and configured.
Follow the [Install Guardrails MCP guide](/guardrails/docs/guides/using-guardrails/ai/install-mcp) to set up the Model Context Protocol (MCP) server.
- Access to your Guardrails workspace URL (e.g., `https://your-workspace.cloud.turbot.com`).
- Familiarity with the [Guardrails console](https://turbot.com/guardrails/docs/getting-started/).
- A clear policy objective (for example, `Enforce EKS Public Endpoint Access`).
- Resource IDs available for testing your policy pack.

## Step 1: Clone *guardrails-samples* Repository

Before starting, clone the [guardrails-samples](https://github.com/turbot/guardrails-samples) repository locally. This repository contains example policy packs, recommended directory structure, and a README template. Use this as your working directory for creating and testing your new policy pack.

```bash
git clone https://github.com/turbot/guardrails-samples.git
cd guardrails-samples/policy_packs
```
>[!NOTE]
> Create a new branch to make sure you use this branch to raise PR against the main branch.

Let AI create your new policy pack as a subdirectory here, following the structure and examples provided.

## Step 2: Define Goal

Clearly state the policy objective. For example:

- Enforce EKS endpoint access security:
- Public access is disabled by default.
- If public access is enabled, restrict to specific CIDRs (e.g., `203.0.113.0/24`, `10.0.0.0/8`).
- Private access must always be enabled.

### Identify Policies

List the relevant Turbot policies to include in your policy pack. You can explore the [Guardrails Hub](https://hub.guardrails.turbot.com/) to find policies specific to your chosen cloud provider's mod.

For example, consider the following policies:

- [AWS > EKS > Cluster > Endpoint Access](https://hub.guardrails.turbot.com/mods/aws/policies/aws-eks/clusterEndpointAccess)
- [AWS > EKS > Cluster > Endpoint Access > CIDR Ranges](https://hub.guardrails.turbot.com/mods/aws/policies/aws-eks/clusterEndpointAccessCidrRanges)

## Step 3: Prepare LLM Prompt

Use the template below to guide the LLM when working in your new branch within the Cursor AI IDE or any AI-assisted development environment.

```
Goal:
- Enforce EKS endpoint access security:
- Public access should be disabled by default.
- If public access is enabled, restrict it to specific CIDRs (e.g., 203.0.113.0/24, 10.0.0.0/8).
- Private access must always be enabled.
- Policies must be visible in the Guardrails UI as part of the policy pack.
- Support a production workflow where customers attach the policy pack manually.

Instructions:
1. Create a policy pack (Terraform) for AWS EKS endpoint access, following Turbot best practices and directory structure.
2. Define the policies as part of the policy pack (not as standalone policy settings). Use `turbot_policy_setting` resources with `resource = turbot_policy_pack.<name>.id`.
3. For testing:
- Add a `turbot_policy_pack_attachment` resource to attach the policy pack to a real resource (provide a test resource ID).
- After a successful test, prompt the user to validate in the Guardrails console that the policies are visible and correct.
- Once the user confirms, remove the `turbot_policy_pack_attachment` resource for production readiness.
4. For production:
- The policy pack should NOT include any `turbot_policy_pack_attachment` resource.
- Customers will attach the policy pack to their desired resources manually via the Guardrails console.
5. README:
- Generate a README for the policy pack following the rules in https://github.com/turbot/guardrails-samples/blob/main/policy_packs/README.md (categories, primary_category, type, usage, etc.).
- Validate the README and policy pack structure against all best practices. If any rule fails, revise and revalidate until all are green/OK.
6. Test:
- Initialize and apply the policy pack using Terraform, attaching it to the provided test resource.
- Confirm that the policies are visible in the Guardrails UI under the policy pack and that the resource is compliant.
- After user validation, remove the attachment for production.

Configuration Input:
- Approved CIDR ranges: `["203.0.113.0/24", "10.0.0.0/8"]`
- Test Resource ID: (provide your real resource ID, e.g., `355421285155896`)

Example Terraform Structure:
- `main.tf`: Defines the `turbot_policy_pack` (and, for testing only, the `turbot_policy_pack_attachment`).
- `policies.tf`: Defines `turbot_policy_setting` resources with `resource = turbot_policy_pack.<name>.id`.
- `providers.tf`: Provider block.
- `variables.tf`: Variable for the target resource.
- `README.md`: As per best practices.

Do not proceed if no real resource is available.
Do not use standalone policy settings attached directly to the resource.
All policies must be visible in the UI as part of the policy pack.

Workflow:
1. Develop and test with the attachment.
2. Prompt user to validate in the Guardrails console.
3. After confirmation, remove the attachment for production readiness.
```

## Step 4: Optimize the Feedback Loop

Work collaboratively with the AI assistant to review, validate, and improve your policy pack. Use an iterative approach: check the generated files, provide clear feedback, and request specific changes until the policy pack meets your requirements and best practices.

### Review the generated files

- `README.md`
- `main.tf`
- `policies.tf`
- `providers.tf`
- `variables.tf`

### Checklist

- Do the policy settings match your objectives?
- Are all required files present and following the recommended structure?
- Is the documentation clear and complete?
- Are there any errors or missing configurations?

### How to provide feedback

You can use prompts like:

```
Please update the following:
- Add more detail to the usage section in README.md
- Change the approved CIDR ranges in variables.tf to ["198.51.100.0/24"]
- Add tags to the policy pack in main.tf
- Ensure all policies are visible in the Guardrails UI
- Fix the resource reference in policies.tf to use the correct variable
```
### Iterate as needed
Repeat the review and feedback process until you are satisfied with the results. Don't hesitate to ask the AI assistant for clarifications, best practice checks, or additional examples.

## Step 5: Actionable Execution

Take your validated policy pack and apply it in a test environment to ensure everything works as expected.

### Apply the Policy Pack

1. **Plan the Deployment:**

Let's initialize and run terraform plan.

- Example prompt:
```
- Initialize the Terraform configuration
- Run terraform plan and wait for me to confirm terraform apply
```

2. **Apply to a Test Resource:**
- Run `terraform apply` to deploy the policy pack, attaching it to your test resource (e.g., `355421285155896`).
- Example prompt:
```
- Apply the policy pack to resource 355421285155896
```
#### Example Prompts

In case any error or further validation use below

```
- Help me troubleshoot a Terraform apply error
- Confirm that the policy pack is visible in the Guardrails UI
- Check if the resource is compliant after applying the policy pack
```

### Review

Manually review to make sure execution is successful

- [ ] Terraform apply completed without errors.
- [ ] The policy pack and its policies are visible in the Guardrails Console.
- [ ] Resource compliance status is as expected.
- [ ] No unexpected alarms or errors.

## Step 6: Finalize for Production

After you have successfully tested your policy pack and validated the results, it's important to prepare your configuration for release by removing any test-specific resources or attachments.

### Remove Test Attachments

- Instruct the AI assistant (or manually update your Terraform files) to remove the `turbot_policy_pack_attachment` resource used for testing.
- Ensure your policy pack is now ready for release, where customers will attach it to their own resources via the Guardrails Console.

#### Example Prompts

```
- Remove the test attachment for the generated policy pack.
- Update documentation for production use.
- Verify final structure matches existing policy pack structure <provide relative path of existing policy pack>
```
### Review

- [ ] All test attachments have been removed from your Terraform configuration.
- [ ] The policy pack structure matches the recommended format and best practices.
- [ ] Documentation is updated for release.
Loading