Enable User Workload Monitoring Stack

Configuration

To enable the stack edit the cluster-monitoring ConfigMap like so:

oc -n openshift-monitoring edit configmap cluster-monitoring-config

Set the enableUserWorkload to true like so:

apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-monitoring-config
  namespace: openshift-monitoring
data:
  config.yaml: |
    enableUserWorkload: true
    prometheusK8s:
      retention: 30d
      volumeClaimTemplate:
        spec:
          storageClassName: ocs-storagecluster-ceph-rbd
          resources:
            requests:
              storage: 100Gi
    alertmanagerMain:
      volumeClaimTemplate:
        spec:
          storageClassName: ocs-storagecluster-ceph-rbd
          resources:
            requests:
              storage: 50Gi

Save the configmap changes. Monitor the rollout progress of the User Workload Monitoring Stack with the following:

oc -n openshift-user-workload-monitoring get pod
NAME                                   READY   STATUS        RESTARTS   AGE
prometheus-operator-6f7b748d5b-t7nbg   2/2     Running       0          3h
prometheus-user-workload-0             4/4     Running       1          3h
prometheus-user-workload-1             4/4     Running       1          3h
thanos-ruler-user-workload-0           3/3     Running       0          3h
thanos-ruler-user-workload-1           3/3     Running       0          3h

At this point we can create a ConfigMap to configure the User Workload Monitoring stack in the openshift-user-workload-monitoring namespace.

oc create configmap user-workload-monitoring-config -n openshift-user-workload-monitoring

Then edit this ConfigMap:

oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config

Save the following configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: user-workload-monitoring-config
  namespace: openshift-user-workload-monitoring
data:
  config.yaml: |
    prometheus:
      retention: 30d
      volumeClaimTemplate:
        spec:
          storageClassName: ocs-storagecluster-ceph-rbd
          resources:
            requests:
              storage: 100Gi
    thanosRuler:
      volumeClaimTemplate:
        spec:
          storageClassName: ocs-storagecluster-ceph-rbd
          resources:
            requests:
              storage: 50Gi

To provide access to users to create PrometheusRule and ServiceMonitor and PodMonitor objects see [2]. To allow access to the User Workload Monitoring Stack dashboard see [3].