docs: AP-5783 add QuickSight user docs

source/data/curated-databases/data-documentation/index.html.md.erb

@@ -1,6 +1,6 @@
 ---
 title: Database Documentation
-weight: 110
+weight: 7
 last_reviewed_on: 2023-03-10
 review_in: 1 year
 show_expiry: true

source/documentation/tools/index.md

@@ -15,6 +15,10 @@ Allows you to browse the databases that are available on the Analytical Platform
 ### [GitHub](https://github.com/)
 Online hosting platform for git. Git is a distributed version control system that allows you to track changes in files, while GitHub hosts the Analytical Platform's code.
 
+## Data Visualisation
+### [QuickSight](quicksight)
+Business Intelligence (BI/MI) tool for visualising / dashboarding data.
+
 ## Data Ingest
 
 ### [Data Extractor](https://github.com/ministryofjustice/data-engineering-data-extractor)

source/documentation/tools/quicksight/01-index.md

@@ -0,0 +1,35 @@
+# QuickSight
+
+⚠️ This service is in beta ⚠️
+
+QuickSight is a data visualisation service that allows you to create interactive dashboards and reports from data available to the Analytical Platform.
+
+We’re still in beta so we'd love to get some of you using QuickSight to get your feedback to help guide data visualization best practices in the Ministry of Justice and make sure we can continue to improve the user experience.
+
+## [Overview](/tools/quicksight/overview.html)
+- [What is QuickSight](/tools/quicksight/overview.html#what-is-quicksight)
+- [What data is in QuickSight?](/tools/quicksight/overview.html#what-data-is-in-quicksight)
+
+## [Getting Started](/tools/quicksight/getting-started.html)
+- [Prerequisites](/tools/quicksight/getting-started.html#prerequisites-to-using-quicksight-in-analytical-platform)
+- [Accessing QuickSight](/tools/quicksight/getting-started.html#accessing-quicksight)
+
+## [Working with QuickSight](/tools/quicksight/working-with-quicksight.html)
+- [Finding datasets](/tools/quicksight/working-with-quicksight.html#finding-datasets)
+- [External QuickSight resources](/tools/quicksight/working-with-quicksight.html)
+- [Publishing dashboards](/tools/quicksight/working-with-quicksight.html)
+- [Sharing dashboards](/tools/quicksight/working-with-quicksight.html)
+
+## [FAQs](/tools/quicksight/faqs.html)
+
+## [Best Practices](/tools/quicksight/best-practices.html)
+
+## [Common Issues and Troubleshooting](/tools/quicksight/troubleshooting.html)
+
+## Resources
+- [aws resources]
+- [aws example analysis] (_dataset requires file upload_)
+
+<!-- External links -->
+[aws resources]: https://aws.amazon.com/quicksight/resources/
+[aws example analysis]: https://docs.aws.amazon.com/quicksight/latest/user/example-analysis.html

source/documentation/tools/quicksight/02-overview.md

@@ -0,0 +1,18 @@
+# Overview
+
+## What is QuickSight?
+QuickSight is a data visualisation service that allows you to create interactive dashboards and reports from data available on the Analytical Platform; straight to your browser, in a reproducible way. All you’ll need to do is connect your data sources and start building your visualizations using QuickSight's interface. QuickSight is packed full of features for analyzing data using various visualization types, so you'll need to get familiar with certain bits of QuickSight functionality. To learn more about QuickSight, take a look at [the aws documentation]. Some of the basics about working with QuickSight in this user guide, but you can also sign up and work through [QuickSight's own examples].
+
+We’re still in beta so we'd love to get some of you using QuickSight to get your feedback to help guide data visualization best practices in the Ministry of Justice and make sure we can continue to improve the user experience.
+
+## What data is available in QuickSight?
+QuickSight is connected to [the `mojap-derived-table` domains] published through [Create a Derived Table](/tools/create-a-derived-table). 
+
+If the output from Create a Derived Table goes to the `mojap_derived_tables/prod/models` path in S3, it should be available in QuickSight.
+
+
+<!-- External links -->
+
+[the aws documentation]: https://docs.aws.amazon.com/quicksight/latest/user/welcome.html
+[QuickSight's own examples]: https://docs.aws.amazon.com/quicksight/latest/user/quickstart-createanalysis.html
+[the `mojap-derived-table` domains]: https://github.com/moj-analytical-services/create-a-derived-table/tree/main/mojap_derived_tables/models

source/documentation/tools/quicksight/03-getting-started.md

@@ -0,0 +1,23 @@
+# Getting Started
+
+## Prerequisites to using QuickSight in Analytical Platform
+- [ ] Permission to read datasets available in QuickSight
+- [ ] QuickSight enabled for your user within Analytical Platform Control Panel
+
+### Permissions
+You'll need permissions to databases, tables, or domains for datasets within [the `mojap-derived-table` domains] produced by [Create a Derived Table](/tools/create-a-derived-table).
+These permissions are defined in [the data-engineering-database-access GitHub repository].
+
+### Enable QuickSight in Analytical Platform Control Panel
+[Raise a support request] to enable QuickSight for your user
+
+## Accessing QuickSight
+- Complete [prerequisites](#prerequisites-to-using-quicksight-in-analytical-platform)
+- Navigate to [QuickSight UI within Control Panel]
+- Dashboard to your heart’s content / contractual obligation
+
+<!-- External links -->
+[the `mojap-derived-table` domains]: https://github.com/moj-analytical-services/create-a-derived-table/tree/main/mojap_derived_tables/models
+[the data-engineering-database-access GitHub repository]: https://github.com/moj-analytical-services/data-engineering-database-access/?tab=readme-ov-file#access-to-curated-databases
+[Raise a support request]: https://github.com/ministryofjustice/data-platform-support/issues/new/choose
+[QuickSight UI within Control Panel]: https://controlpanel.services.analytical-platform.service.justice.gov.uk/quicksight/

source/documentation/tools/quicksight/04-working-with-quicksight.md

@@ -0,0 +1,44 @@
+# Working with QuickSight
+
+## Finding datasets
+Datasets are available via the `Athena` data source in QuickSight: 
+
+>Datasets > New Dataset (top right) > Athena > New Athena data source.
+
+When adding a new dataset via Athena you'll be prompted for a `Data source name`. 
+This is just a descriptive name for the new data source tile for **you** in **your** QuickSight interface.
+
+Here's [the advice from aws] about naming your Data source:
+
+> _This name displays on the Amazon QuickSight list of existing data sources, which is at the bottom of the 'Create a Dataset' view. 
+Use a name that makes it easy to distinguish your data sources from other similar data sources._
+
+Unless specified elsewhere for your data, use the default `Athena workgroup`.
+
+Data will be in the default Catalog (`AwsDataCatalog`), and you should only see Databases that your user has access to via the Database Access Repo.
+
+Adding a dataset in this way enables you to add a single table at a time.
+
+You'll be asked if you want to 'Import to [SPICE] for quicker analytics'.  
+This imports the data into memory, rather than querying the data where it lives in S3. Whether to use SPICE or directly query the data is a judgement call, and will likely come down to:
+
+- how complex the analysis is
+- the size, format and structure of the underlying data
+- how likely users are to drill down into the dashboards
+- user tolerance for load times while making changes to dashboards
+
+
+## External QuickSight resources
+- [aws resources]
+- [aws example analysis] (_dataset requires file upload_)
+
+## Publishing Dashboards
+
+## Sharing Dashboards
+
+<!-- External links -->
+
+[the advice from aws]: https://docs.aws.amazon.com/quicksight/latest/user/create-a-data-source.html
+[SPICE]: https://docs.aws.amazon.com/quicksight/latest/user/managing-spice-capacity.html
+[aws resources]: https://aws.amazon.com/quicksight/resources/
+[aws example analysis]: https://docs.aws.amazon.com/quicksight/latest/user/example-analysis.html

source/documentation/tools/quicksight/best-practices.md

@@ -0,0 +1,33 @@
+# Best Practices
+
+
+<!-- relevant once data expands beyond CaDeT
+## Data Preparation
+- Clean and preprocess your data before importing it into QuickSight.
+- Use meaningful names for datasets and fields. 
+-->
+
+## Dashboard Design
+- Keep dashboards simple and focused.
+- Use appropriate visualization types for your data.
+- Limit the number of visuals on a single dashboard to improve performance.
+
+## Performance Optimization
+- see [this issue in the troubleshooting guide](/tools/quicksight/troubleshooting.html#issue-slow-dashboard-performance).
+- Optimize your queries to reduce load times.
+- Load datasets into SPICE.
+
+## Collaboration
+- Share dashboards with relevant stakeholders. 
+- Use descriptive names and annotations for visuals. 
+
+## Security
+- Ensure proper permissions are set for datasets and dashboards.
+- Regularly review and update access controls.
+
+## Maintenance
+- Regularly update datasets and dashboards to reflect changes in data.
+- Monitor dashboard performance and make necessary adjustments.
+
+<!-- External links -->
+

source/documentation/tools/quicksight/faqs.md

@@ -0,0 +1,48 @@
+# Frequently Asked Questions
+
+## How do I know if I should be able to see a particular table/database/domain in QuickSight?  
+
+Databases and Tables should be available via the `Athena` data source in QuickSight provided that:
+
+- your Analytical Platform `alpha_user` has access to the data via [the data-engineering-database-access GitHub repository]
+- the data is within one of [the `mojap-derived-table` domains] in Create a Derived Table
+
+## Can I share the dashboards that I create?
+
+### For consumers
+
+### For collaborators / editor
+
+## I’m already a user of another version of QuickSight on the AP, what should I do if:
+
+### I can do all of my work on the new QuickSight (all of my tables are create-a-derived-table tables)
+
+Export dashboards from other versions of QuickSight
+
+Import dashboards into QuickSight via Control Panel
+
+### I can’t do all of my work on the new QuickSight (some / all of my tables are outside create-a-derived-table)
+
+## When should I use SPICE?
+
+Importing a dataset into [SPICE] imports the data into memory, rather than querying the data where it lives in S3. Whether to use SPICE or directly query the data is a judgement call, and will likely come down to:
+
+- how complex the analysis is
+- the size, format and structure of the underlying data
+- how likely users are to drill down into the dashboards
+- user tolerance for load times while making changes to dashboards
+
+## What should I do if I encounter an error while using QuickSight?
+
+Check [the Troubleshooting guide](/tools/quicksight/troubleshooting), and [raise a support request] if you're unable to resolve your issue there.
+
+## How can I optimize the performance of my QuickSight dashboards?
+
+- see [this issue in the troubleshooting guide](/tools/quicksight/troubleshooting.html#issue-slow-dashboard-performance)
+
+<!-- External links -->
+
+[the `mojap-derived-table` domains]: https://github.com/moj-analytical-services/create-a-derived-table/tree/main/mojap_derived_tables/models
+[the data-engineering-database-access GitHub repository]: https://github.com/moj-analytical-services/data-engineering-database-access/?tab=readme-ov-file#access-to-curated-databases
+[SPICE]: https://docs.aws.amazon.com/quicksight/latest/user/managing-spice-capacity.html
+[Raise a support request]: https://github.com/ministryofjustice/data-platform-support/issues/new/choose

source/documentation/tools/quicksight/troubleshooting.md

@@ -0,0 +1,29 @@
+# Common Issues and Troubleshooting
+
+## Common Issues
+
+### Issue: Unable to see a dataset in QuickSight
+- Ensure you have the necessary permissions to access the dataset.
+- Verify that the dataset is within the `mojap-derived-table` domains.
+- Check if the dataset is available via the `Athena` data source.
+
+See [this question in FAQs](/tools/quicksight/faqs.html#how-do-i-know-if-i-should-be-able-to-see-a-particular-tabledatabasedomain-in-quicksight) for more information 
+
+### Issue: Slow dashboard performance
+- Optimize your queries
+   - Remove unnecessary data from queries
+   - Optimise joins and data structure
+- Check out [the QuickSight optimisation tips page] for specific tips
+- Load data into SPICE
+
+
+### Issue: Error while importing data
+- Check the data format and structure
+   - Can you read and query the data via the Athena console (in the Ireland region within the data account) or via `boto3`/`pydbtools`?
+- Ensure the data source is correctly configured in QuickSight (see [Finding datasets](/tools/quicksight/04-working-with-quicksight.html#finding-datasets))
+- [Raise a support request] providing details of the problem and what you've tried.
+
+<!-- External links -->
+
+[the QuickSight optimisation tips page]: https://aws.amazon.com/blogs/big-data/tips-and-tricks-for-high-performant-dashboards-in-amazon-quicksight/
+[Raise a support request]: https://github.com/ministryofjustice/data-platform-support/issues/new/choose

source/tools/control-panel.html.md.erb

@@ -1,6 +1,6 @@
 ---
 title: Control Panel
-weight: 10
+weight: 5
 last_reviewed_on: 2022-07-06
 review_in: 1 year
 show_expiry: true

source/tools/data-processing/index.html.md.erb

@@ -1,6 +1,6 @@
 ---
 title: Tools for Data Processing
-weight: 5
+weight: 8
 last_reviewed_on: 2024-07-11
 review_in: 1 year
 show_expiry: true

source/tools/quicksight/best-practices.html.md.erb

@@ -0,0 +1,11 @@
+---
+title: Best Practices
+weight: 30
+tags: quicksight
+last_reviewed_on: 2024-12-11
+review_in: 2 months
+owner_slack: "#analytical-platform-support"
+owner_slack_workspace: "mojdt"
+---
+
+<%= partial 'documentation/tools/quicksight/best-practices' %>

source/tools/quicksight/faqs.html.md.erb

@@ -0,0 +1,11 @@
+---
+title: FAQs
+weight: 25
+tags: quicksight
+last_reviewed_on: 2024-12-11
+review_in: 2 months
+owner_slack: "#analytical-platform-support"
+owner_slack_workspace: "mojdt"
+---
+
+<%= partial 'documentation/tools/quicksight/faqs' %>

source/tools/quicksight/getting-started.html.md.erb

@@ -0,0 +1,11 @@
+---
+title: Getting Started
+weight: 15
+tags: quicksight
+last_reviewed_on: 2024-12-11
+review_in: 2 months
+owner_slack: "#analytical-platform-support"
+owner_slack_workspace: "mojdt"
+---
+
+<%= partial 'documentation/tools/quicksight/03-getting-started' %>

source/tools/quicksight/index.html.md.erb

@@ -0,0 +1,11 @@
+---
+title: QuickSight
+weight: 6
+tags: quicksight
+last_reviewed_on: 2024-12-11
+review_in: 2 months
+owner_slack: "#analytical-platform-support"
+owner_slack_workspace: "mojdt"
+---
+
+<%= partial 'documentation/tools/quicksight/01-index' %>

source/tools/quicksight/overview.html.md.erb

@@ -0,0 +1,11 @@
+---
+title: QuickSight
+weight: 10
+tags: quicksight
+last_reviewed_on: 2024-12-11
+review_in: 2 months
+owner_slack: "#analytical-platform-support"
+owner_slack_workspace: "mojdt"
+---
+
+<%= partial 'documentation/tools/quicksight/02-overview' %>

source/tools/quicksight/troubleshooting.html.md.erb

@@ -0,0 +1,11 @@
+---
+title: Common Issues and Troubleshooting
+weight: 35
+tags: quicksight
+last_reviewed_on: 2024-12-11
+review_in: 2 months
+owner_slack: "#analytical-platform-support"
+owner_slack_workspace: "mojdt"
+---
+
+<%= partial 'documentation/tools/quicksight/troubleshooting' %>

source/tools/quicksight/working-with-quicksight.html.md.erb

@@ -0,0 +1,11 @@
+---
+title: Working with QuickSight
+weight: 20
+tags: quicksight
+last_reviewed_on: 2024-12-11
+review_in: 2 months
+owner_slack: "#analytical-platform-support"
+owner_slack_workspace: "mojdt"
+---
+
+<%= partial 'documentation/tools/quicksight/04-working-with-quicksight' %>