Replacing Internal Certificates
NeuVector versions 5.4.2 and later must have users generate/replace internal certificates before using NeuVector. After March 2025, NeuVector versions prior to 5.4.2 must have users generate/replace internal certificates before using NeuVector.
Internal Communication and Certificatesβ
NeuVector includes default self-signed certificates for encryption for the Manager (console/UI access), Controller (REST API, internal), Enforcer (internal), and Scanner (internal) communications.
These certificates can be replaced by your own to further harden communication. For replacing certificates used by external access to NeuVector (i.e, browser to the Manager, or REST API to the Controller), please see this section. See below for replacing the certificates used in internal communication between NeuVector containers.
WARNING: Replacing certificates is recommended to be performed only during initial deployment of NeuVector. Replacing on a running cluster (even with rolling upgrade) may result in an unstable state where NeuVector pods are unable to communicate with each other due to a mismatch in certificates, and DATA LOSS may occur.
Replacing Certificates Used in Internal Communications of NeuVectorβ
Replace the internal encryption files ca.crt
, tls.key
, tls.crt
as follows:
- Create a new
ca.cfg
file with your favorite editor:
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = California
L = San Jose
O = NeuVector Inc.
OU = Neuvector
CN = Neuvector
[v3_req]
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = Neuvector
For additional information on ca.cfg
, see https://open-docs.neuvector.com/configuration/console/replacecert.
- Choose your scenario from the three options below:
- New certificate
- Update current certificate with SANs
- Regenarate certificate files and add SANs
- Regenerate certificate when built-in certificate is being used
If your certificate is about to expire and you need to generate a new one, follow the steps below:
- Delete the old
ca.crt
,tls.key
,tls.crt
, kubernetes secret, and generate new ones:
kubectl delete secret internal-cert -n neuvector
openssl genrsa -out ca.key 2048
openssl req -x509 -sha256 -new -nodes -key ca.key -days 3650 -out ca.crt
openssl genrsa -out tls.key 2048
openssl req -new -key tls.key -sha256 -out cert.csr -config ca.cfg
openssl req -in cert.csr -noout -text
openssl x509 -req -sha256 -in cert.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 3650 -extensions 'v3_req' -extfile ca.cfg
openssl x509 -in tls.crt -text
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key --from-file=tls.crt --from-file=ca.crt
Then edit the Controller, Enforcer, and Scanner deployment yamls, adding:
containers:
- name: neuvector-controller/enforcer/scanner-pod
volumeMounts:
- mountPath: /etc/neuvector/certs/internal/cert.key
name: internal-cert
readOnly: true
subPath: tls.key
- mountPath: /etc/neuvector/certs/internal/cert.pem
name: internal-cert
readOnly: true
subPath: tls.crt
- mountPath: /etc/neuvector/certs/internal/ca.cert
name: internal-cert
readOnly: true
subPath: ca.crt
volumes:
- name: internal-cert
secret:
defaultMode: 420
secretName: internal-cert
Then proceed to deploy NeuVector as before. You can also shell into the controller/enforcer/scanner pods to confirm that the ca.crt, tls.key, tls.crt files are the customized ones and that the NeuVector communications are working using the new certificates.
Sample patch commands for controller (change namespace to cattle-neuvector-system if needed, and modify for use on enforcer, scanner):
NAMESPACE=neuvector
kubectl patch deployment -n ${NAMESPACE} neuvector-controller-pod --type='json' -p='[{"op": "add", "path": "/spec/template/spec/volumes/-", "value": {"name": "internal-cert", "secret": {"defaultMode": 420, "secretName": "internal-cert"}} } ]'
kubectl patch deployment -n ${NAMESPACE} neuvector-controller-pod --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/volumeMounts", "value": [{"mountPath": "/etc/neuvector/certs/internal/cert.key", "name": "internal-cert", "readOnly": true, "subPath": "cert.key"}, {"mountPath": "/etc/neuvector/certs/internal/cert.pem", "name": "internal-cert", "readOnly": true, "subPath": "cert.pem"}, {"mountPath": "/etc/neuvector/certs/internal/ca.cert", "name": "internal-cert", "readOnly": true, "subPath": "ca.cert"} ] } ]'
If your certificate files were created before NeuVector version 5.3, you need to update the certificate with at least one Subject Alternative Name, or SAN for short.
If you still have the files ca.key
and ca.crt
accessible, run the commands as follows:
kubectl delete secret internal-cert -n neuvector
openssl genrsa -out tls.key 2048
openssl req -new -key tls.key -sha256 -out cert.csr -config ca-new.cfg
openssl req -in cert.csr -noout -text
openssl x509 -req -sha256 -in cert.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 3650 -extensions 'v3_req' -extfile ca-new.cfg
openssl x509 -in tls.crt -text
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key --from-file=tls.crt --from-file=ca.crt
Once the certificate files have been updated, restart the deployments in order to use the updated certificate:
kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod
If your certificate files were created before NeuVector version 5.3, you need to update the certificate with at least one Subject Alternative Name, or SAN for short.
If you donΒ΄t have the files ca.key
and ca.crt
anymore, follow the steps below:
- Backup your original certificate
kubectl get secret internal-cert -o yaml > internal-cert.yaml
- Export the existing internal-cert
kubectl get secret internal-cert -o json | jq -r '.data."ca.crt"' | base64 -d > old-ca.crt
kubectl get secret internal-cert -o json | jq -r '.data."tls.crt"' | base64 -d > old-tls.crt
kubectl get secret internal-cert -o json | jq -r '.data."tls.key"' | base64 -d > old-tls.key
- Create new certificate files and internal certificates
openssl genrsa -out ca.key 2048
openssl req -x509 -sha256 -new -nodes -key ca.key -days 3650 -out ca.crt
openssl genrsa -out tls.key 2048
openssl req -new -key tls.key -sha256 -out cert.csr -config ca.cfg
openssl req -in cert.csr -noout -text
openssl x509 -req -sha256 -in cert.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 3650 -extensions 'v3_req' -extfile ca.cfg
openssl x509 -in tls.crt -text
- Merge the old and new
ca.crt
files
cat old-ca.crt > /tmp/ca.crt
cat ca.crt >> /tmp/ca.crt
- Update the Kubernetes secret with the merged
ca.crt
kubectl delete secret internal-cert -n neuvector
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key=old-tls.key --from-file=tls.crt=old-tls.crt --from-file=ca.crt=/tmp/ca.crt
- Restart the deployments in order to use the updated certificate
kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod
- Wait for the restart to complete
kubectl rollout status deployment neuvector-controller-pod
kubectl rollout status deployment neuvector-scanner-pod
kubectl rollout status deployment neuvector-registry-adapter-pod
kubectl rollout status ds neuvector-enforcer-pod
-
Make sure the console can be accessed and controllers are all online.
-
Update the Kubernetes secret with the new
tls.key
kubectl delete secret internal-cert -n neuvector
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key=tls.key --from-file=tls.crt=tls.crt --from-file=ca.crt=/tmp/ca.crt
- Restart the deployments in order to use the updated certificate
kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod
- Wait for the restart to complete
kubectl rollout status deployment neuvector-controller-pod
kubectl rollout status deployment neuvector-scanner-pod
kubectl rollout status deployment neuvector-registry-adapter-pod
kubectl rollout status ds neuvector-enforcer-pod
-
Make sure the console can be accessed and controllers are all online.
-
Update the Kubernetes secret with the new
ca.crt
kubectl delete secret internal-cert -n neuvector
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key=tls.key --from-file=tls.crt=tls.crt --from-file=ca.crt=ca.crt
- Restart the deployments in order to use the updated certificate
kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod
- Wait for the restart to complete
kubectl rollout status deployment neuvector-controller-pod
kubectl rollout status deployment neuvector-scanner-pod
kubectl rollout status deployment neuvector-registry-adapter-pod
kubectl rollout status ds neuvector-enforcer-pod
- Make sure the console can be accessed and controllers are all online.
If you didn't replace the internal certificate before and want to migrate to a new set of certificates, follow the steps below:
- Check if you already have the internal certificate generated automatically.
kubectl get secret internal-cert -o yaml
If you see tls.key
, tls.crt
and ca.crt
there, that means you've been using the automatically generated certificate and you can skip this section.
If you can see the secret, but cannot find these secrets, consider enabling internal.autoRotateCert
in the helm charts override. This option will generate and rotate your internal certificate automatically.
If you don't use the automatically generated internal certificate and can't do so, follow the steps below:
- Follow the steps in the
New certificate
tab to use a Kubernetes secret to manage the internal certificate. Instead of generating a new certificate, use the certificate,old-ca.crt
,old-tls.crt
andold-tls.key
, retrieved below:
docker run -it --entrypoint=bash neuvector/scanner:3.654 -c "cat /etc/neuvector/certs/internal/ca.cert" > old-ca.crt
docker run -it --entrypoint=bash neuvector/scanner:3.654 -c "cat /etc/neuvector/certs/internal/cert.pem" > old-tls.crt
docker run -it --entrypoint=bash neuvector/scanner:3.654 -c "cat /etc/neuvector/certs/internal/cert.key" > old-tls.key
-
Make sure that all components are running without errors.
-
After that, follow the steps in
Regenerate certificate files and add SANs
tab and migrate to your own certificate.
Updating/Deploying with Helmβ
As of Helm chart 2.4.1
we can now manage the internal certificate install. The chart values.yaml should be reviewed for all the settings. The below example uses RKE2, standard Ingress and installer certificates.
# add chart
helm repo add neuvector https://neuvector.github.io/neuvector-helm/
# update chart
helm repo update
# add domain for ingress
export domain=awesome.sauce
# run the helm
helm upgrade -i neuvector -n neuvector neuvector/core --create-namespace --set imagePullSecrets=regsecret --set k3s.enabled=true --set k3s.runtimePath=/run/k3s/containerd/containerd.sock --set manager.ingress.enabled=true --set manager.ingress.host=neuvector.$domain --set manager.svc.type=ClusterIP --set controller.pvc.enabled=true --set controller.pvc.capacity=500Mi --set controller.internal.certificate.secret=internal-cert --set cve.scanner.internal.certificate.secret=internal-cert --set enforcer.internal.certificate.secret=internal-cert