I am managing a Kubernetes cluster with ArgoCD, which includes an ingress-nginx. We are trying to obtain a certificate for HTTPS on a domain using cert-manager, but we are encountering a series of errors during the process.
The challenge fails with the following error:
Failed 10s cert-manager-challenges Accepting challenge authorization failed: acme: authorization error for example.org: 403 urn:ietf:params:acme:error:unauthorized: 2001:8d8:100f:f000::200: Invalid response from http://example.org/.well-known/acme-challenge/ZzU4jDSzvVCHwPHwPMsUleJDwf-K3URomZwuhQgNZOo: 204. The challenge generates a cm-acme-http-solver with the following logs:
I0712 10:37:33.303185 1 solver.go:39] "cert-manager/acmesolver: starting listener" expected_domain="example.org" expected_token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" expected_key="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY.2DhKIbjC5D1b_jnc9Katl9vWzWPu9HPi-bGtAm8wLnw" listen_port=8089 I0712 10:37:42.095947 1 solver.go:64] "cert-manager/acmesolver: validating request" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" I0712 10:37:42.095987 1 solver.go:72] "cert-manager/acmesolver: comparing host" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" expected_host="example.org" I0712 10:37:42.096005 1 solver.go:79] "cert-manager/acmesolver: comparing token" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" expected_token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" I0712 10:37:42.096030 1 solver.go:87] "cert-manager/acmesolver: got successful challenge request, writing key" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" ... Error: http: Server closed Usage: acmesolver [flags] Flags: --domain string the domain name to verify -h, --help help for acmesolver --key string the challenge key to respond with --listen-port int the port number to listen on for connections (default 8089) --token string the challenge token to verify against E0712 10:37:53.692260 1 main.go:39] "cert-manager: error executing command" err="http: Server closed" Additionally, the cert-manager-webhook logs show these errors:
W0712 11:33:35.571985 1 client_config.go:618] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I0712 11:33:35.657545 1 webhook.go:128] "cert-manager: using dynamic certificate generating using CA stored in Secret resource" secret_namespace="cert-manager" secret_name="cert-manager-webhook-ca" I0712 11:33:35.657833 1 server.go:133] "cert-manager/webhook: listening for insecure healthz connections" address=":6080" I0712 11:33:35.657899 1 server.go:197] "cert-manager/webhook: listening for secure connections" address=":10250" I0712 11:33:36.662043 1 dynamic_source.go:255] "cert-manager/webhook: Updated cert-manager webhook TLS certificate" DNSNames=["cert-manager-webhook","cert-manager-webhook.cert-manager","cert-manager-webhook.cert-manager.svc"] I0712 11:33:54.925304 1 logs.go:59] http: TLS handshake error from 10.221.113.195:52358: remote error: tls: bad certificate I0712 11:33:59.817534 1 logs.go:59] http: TLS handshake error from 10.216.164.3:46654: EOF ... Our ClusterIssuer configuration is as follows:
apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-example spec: acme: email: email@example.com preferredChain: "" privateKeySecretRef: name: lets-encrypt-ionos-issuer-account-key server: https://acme-v02.api.letsencrypt.org/directory solvers: - selector: dnsZones: - example.org http01: ingress: ingressClassName: nginx And the Ingress configuration:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: cert-manager.io/cluster-issuer: letsencrypt-example name: example namespace: example spec: ingressClassName: nginx rules: - host: example.org http: paths: - backend: service: name: example port: number: 8080 path: / pathType: Prefix tls: - hosts: - example.org secretName: example-dev-tls Has anyone experienced similar issues or have any insights on how to resolve these errors? Any help would be greatly appreciated.
I tried to add some ssl-redirect=false annotations and edit the dnskube config on kube-system.