docs: add docs for upgrading the project when using Helm (#582)

* docs: adding upgrade notes for Helm

* chore: adding new ignore

* docs: add in cmd to check for stuck runners

* docs: better format

* docs: removing superfluous steps

* docs: moved location of docs

Co-authored-by: Callum James Tait <callum.tait@photobox.com>

.github/RELEASE_NOTE_TEMPLATE.md

@@ -21,7 +21,7 @@ v{{ .Version }}: {{ .TitlesOfImportantChanges }}
 **Body**:
 
 ```
-**CAUTION:** If you're using the Helm chart, beware to review changes to CRDs and do manually upgrade CRDs! Helm installs CRDs only on installing a chart. It doesn't automatically upgrade CRDs. Otherwise you end up with troubles like #427, #467, and #468
+**CAUTION:** If you're using the Helm chart, beware to review changes to CRDs and do manually upgrade CRDs! Helm installs CRDs only on installing a chart. It doesn't automatically upgrade CRDs. Otherwise you end up with troubles like #427, #467, and #468. Please refer to the [UPGRADING](charts/actions-runner-controller/docs/UPGRADING.md) docs for the latest process.
 
 This release includes the following changes from contributors. Thank you!
 

.github/workflows/on-push-lint-charts.yml

@@ -5,6 +5,7 @@ on:
     paths:
       - 'charts/**'
       - '.github/**'
+      - '!.github/*.md'
   workflow_dispatch:
 
 env:

.github/workflows/on-push-master-publish-chart.yml

@@ -8,6 +8,7 @@ on:
     paths:
       - 'charts/**'
       - '.github/**'
+      - '!.github/*.md'
   workflow_dispatch:
 
 env:

charts/actions-runner-controller/.helmignore

@@ -21,3 +21,5 @@
 .idea/
 *.tmproj
 .vscode/
+# Docs
+docs/

charts/actions-runner-controller/docs/UPGRADING.md

@@ -0,0 +1,29 @@
+## Upgrading
+
+This project makes extensive use of CRDs to provide much of its functionality. Helm unfortunately does not support [managing](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/) CRDs by design:
+
+```
+There is no support at this time for upgrading or deleting CRDs using Helm. This was an explicit decision after much 
+community discussion due to the danger for unintentional data loss. Furthermore, there is currently no community 
+consensus around how to handle CRDs and their lifecycle. As this evolves, Helm will add support for those use cases.
+```
+
+Helm will do an initial install of CRDs but it will not touch them afterwards (update or delete).
+
+Additionally, because the project leverages CRDs so extensively you **MUST** run the matching controller app container with its matching CRDs i.e. always redeploy your CRDs if you are changing the app version.
+
+Due to the above you can't just do a `helm upgrade` to release the latest version of the chart, the best practice steps are recorded below:
+
+## Steps
+
+1. Uninstall the chart
+2. Manually delete the CRDs:
+
+```shell
+# Delete the CRDs
+kubectl get crds | grep actions.summerwind. | awk '{print $1}' | xargs kubectl delete crd
+# Confirm the CRDs are gone
+kubectl get crds | grep actions.summerwind.
+```
+
+3. Install the chart following the documentation