Fixing Insecure COTS Helm Charts on OpenShift Without Forking Them

The Problem Every OpenShift Team Hits

You’re adopting a commercial product that ships as a Helm chart. You install it on OpenShift and every pod fails to schedule. The reason: the vendor’s chart runs containers as root, adds unnecessary Linux capabilities, and skips seccomp profiles. OpenShift’s Security Context Constraints (SCC) admission controller — specifically the restricted-v2 SCC — rejects all of it.

You ask the vendor to fix their chart. They say “we don’t support OpenShift” or “just disable SCC.” Neither is acceptable. You can’t fork their chart either — you’d have to maintain the fork through every vendor update.

So how do you fix a Helm chart you don’t own, in a GitOps-friendly way, without forking it?

I tried three approaches that work and one that doesn’t. Here’s what I learned.

What’s Actually Wrong

The typical COTS chart produces Deployments with some combination of these security violations:

# Pod-level
securityContext:
  runAsUser: 0      # root
  runAsGroup: 0     # root group

# Container-level
securityContext:
  allowPrivilegeEscalation: true
  capabilities:
    add: [NET_BIND_SERVICE]
resources: {}        # no requests or limits

What OpenShift’s restricted-v2 SCC requires:

# Pod-level
securityContext:
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

# Container-level
securityContext:
  allowPrivilegeEscalation: false
  runAsNonRoot: true
  capabilities:
    drop: [ALL]
resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 256Mi

The gap isn’t subtle. Every field is wrong.

Why Server-Side Apply Doesn’t Work

My first instinct was ArgoCD Server-Side Apply (SSA). Create a second ArgoCD Application with ServerSideApply=true and Force=true that applies partial Deployment manifests containing only the corrected securityContext. SSA would merge the fixes into the vendor’s Deployments.

It failed for five cascading reasons:

ArgoCD blocks shared resources. Two Applications can’t manage the same Deployment without explicit tracking annotations.
Partial manifests fail validation. The override Application’s manifests lack required fields like selector and image. To work around this, you end up duplicating most of the vendor’s manifest — defeating the purpose of “partial.”
Child apps revert your fixes. Vendor charts often use an app-of-apps pattern: a parent chart creates ArgoCD Application CRs, each with selfHeal: true. When SSA modifies a Deployment, the child app detects drift and re-syncs within seconds, wiping your changes.
ignoreDifferences can’t save you. You’d need to SSA-patch the child Application CRs themselves to inject ignoreDifferences, but that requires RespectIgnoreDifferences=true in syncOptions — which the vendor’s chart doesn’t set.
Sync wave deadlock. The base install can’t become healthy without the security fixes (SCC blocks pods), so any sync ordering that waits for health creates a deadlock.

But the fundamental reason is simpler: SSA can add fields but cannot remove them. When the vendor sets runAsUser: 0, SSA can add runAsNonRoot: true alongside it, but it cannot delete runAsUser: 0. Both fields coexist, and SCC still rejects the pod.

I’ve preserved the SSA manifests in the demo repo under gitops-ssa-archive/ so you don’t have to repeat the experiment.

The Key Insight: JSON Patch

What works is JSON Patch (RFC 6902). Its replace operation overwrites an entire object — it doesn’t merge alongside existing fields. When you replace /spec/template/spec/securityContext, the vendor’s runAsUser: 0 is gone, replaced entirely by your compliant context.

All three working methods use JSON Patch. They differ in where the patch is applied.

Method 1: Kyverno Mutating Webhook (Recommended)

A Kyverno ClusterPolicy intercepts all Deployments created in the vendor’s namespace at admission time. Before the Deployment is persisted to etcd, Kyverno applies JSON Patch operations:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: mutate-cots-platform-security
spec:
  rules:
    - name: fix-security-context
      match:
        resources:
          kinds: [Deployment]
          namespaces: [cots-platform]
      mutate:
        patchesJson6902: |-
          - op: replace
            path: /spec/template/spec/securityContext
            value:
              runAsNonRoot: true
              seccompProfile:
                type: RuntimeDefault
          - op: replace
            path: /spec/template/spec/containers/0/securityContext
            value:
              allowPrivilegeEscalation: false
              runAsNonRoot: true
              capabilities:
                drop: [ALL]
          - op: add
            path: /spec/template/spec/containers/0/resources
            value:
              requests:
                cpu: 100m
                memory: 128Mi
              limits:
                cpu: 500m
                memory: 256Mi

What’s great: The vendor’s entire app-of-apps topology is preserved. The parent chart creates child Applications, child Applications deploy child charts, Kyverno fixes every Deployment as it arrives. Zero changes to the vendor’s Helm structure. New components added by the vendor in future chart versions are automatically fixed.

The tradeoff: Child apps show OutOfSync in ArgoCD because the live state (post-mutation) differs from what Helm rendered. This is cosmetic — the pods are running correctly — but it can confuse operators who expect everything green.

Method 2: Kustomize Direct (Simplest)

A single ArgoCD Application points at a Kustomize directory. The kustomization.yaml renders the vendor’s child charts directly via helmCharts — bypassing the vendor’s parent chart and app-of-apps pattern entirely — then applies JSON Patch:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

helmGlobals:
  chartHome: ../../fake-cots-charts

helmCharts:
  - name: cots-frontend
    releaseName: cots-frontend
    namespace: cots-platform
    valuesFile: values.yaml
  - name: cots-api
    releaseName: cots-api
    namespace: cots-platform
    valuesFile: values.yaml
  - name: cots-worker
    releaseName: cots-worker
    namespace: cots-platform
    valuesFile: values.yaml

patches:
  - target:
      kind: Deployment
      name: cots-frontend
    patch: |-
      - op: replace
        path: /spec/template/spec/securityContext
        value:
          runAsNonRoot: true
          seccompProfile:
            type: RuntimeDefault
      # ... same pattern for container securityContext and resources

What’s great: One Application, everything shows Synced and Healthy, simplest to understand and maintain.

The tradeoff: You bypass the vendor’s app-of-apps topology entirely. If the vendor’s parent chart does other things besides creating child Applications (namespace setup, RBAC, etc.), you need to account for that separately.

Method 3: Kustomize Redirect (Preserves Vendor Topology)

This is the most sophisticated approach. It uses two layers of JSON Patch:

Layer 1: Render the vendor’s parent chart via Kustomize, then JSON Patch the resulting ArgoCD Application CRs to redirect each child app to a local overlay:

patches:
  - target:
      kind: Application
      name: cots-frontend
    patch: |-
      - op: replace
        path: /spec/source/path
        value: kustomize-redirect/child-frontend
      - op: remove
        path: /spec/source/helm

Layer 2: Each local overlay renders the vendor’s child chart and applies the security JSON Patches.

What’s great: Preserves the vendor’s multi-Application topology (separate ArgoCD Applications per component), everything shows Synced, no Kyverno dependency.

The tradeoff: Highest maintenance. Vendor charts must be copied locally because Kustomize’s security sandbox doesn’t allow chartHome to reference parent directories. In production, use helmCharts.repo to pull from the vendor’s Helm registry instead.

Which Method to Use

	Kyverno	Kustomize Direct	Kustomize Redirect
Vendor topology preserved	Yes	No	Yes
ArgoCD sync status	OutOfSync (cosmetic)	Synced	Synced
Auto-covers new components	Yes	No (manual)	No (manual)
External dependency	Kyverno operator	None	None
Maintenance burden	Low	Low	High

My recommendation: Kyverno for production. It’s the lowest maintenance, automatically covers new vendor components, and the OutOfSync status is a cosmetic tradeoff worth accepting. If you can’t run Kyverno, use Kustomize Direct for simplicity or Kustomize Redirect if you need the vendor’s multi-app topology.

The Bigger Lesson

The real insight isn’t about any specific tool. It’s that strategic merge and SSA fundamentally cannot remove fields — they can only add or update. When a vendor’s insecure defaults need to be deleted rather than overridden alongside, you need JSON Patch’s replace operation.

This applies beyond security contexts. Any time you need to replace an entire object in a vendor’s Helm output — tolerations, node selectors, environment variables — the same pattern works: render the chart, JSON Patch the output, deploy the result.

The full demo repo with all three methods (plus the archived SSA failure) is at ultraJeffOrg/cots-override-demo.