Users in Okta to 1password groups not syncing

Hi @varun118 ,

I'm sorry you're experiencing these issues. I'm looking into this with the team.

In the meantime I'm hoping you could answer a couple questions. Are you saying you have only started seeing these issues after trying to upgrade the SCIM bridge? The changelog you mentioned is referencing a feature introduced in 2.2.0 but that log line showing the error is running 2.1.0. What steps did you take prior to encountering the error?

What errors are you seeing that make you think the Let's Encrypt changes are related?

Thanks for posting, hoping to get all the issues resolved quickly.
Chas

Hi
these are errors which we notice when we upgrade to 2.2.0

6:33AM ERR failed to GetTLSConfig, retrying after backoff delay error="Network: (could not obtain Let's Encrypt certificate), 1password-scim.internal.icims.io: obtaining certificate: [1password-scim.internal.icims.io] Obtain: [1password-scim.internal.icims.io] solving challenges: 1password-scim.internal.icims.io: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/264427320/36502081140) (ca=https://acme-v02.api.letsencrypt.org/directory)" application=op-scim build=202001 version=2.2.0
6:33AM INF TLS attempting to acquire certificate application=op-scim build=202001 domain=1password-scim.internal.icims.io version=2.2.0
6:33AM ERR failed to GetTLSConfig, retrying after backoff delay error="Network: (could not obtain Let's Encrypt certificate), 1password-scim.internal.icims.io: obtaining certificate: [1password-scim.internal.icims.io] Obtain: [1password-scim.internal.icims.io] solving challenges: 1password-scim.internal.icims.io: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/264427320/36502102450) (ca=https://acme-v02.api.letsencrypt.org/directory)" application=op-scim build=202001 version=2.2.0
6:33AM INF TLS attempting to acquire certificate application=op-scim build=202001 domain=1password-scim.internal.icims.io version=2.2.0
6:33AM ERR failed to GetTLSConfig, retrying after backoff delay error="Network: (could not obtain Let's Encrypt certificate), 1password-scim.internal.icims.io: obtaining certificate: [1password-scim.internal.icims.io] Obtain: [1password-scim.internal.icims.io] solving challenges: 1password-scim.internal.icims.io: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/264427320/36502169590) (ca=https://acme-v02.api.letsencrypt.org/directory)" application=op-scim build=202001 version=2.2.0
6:35AM INF TLS attempting to acquire certificate application=op-scim build=202001 domain=1password-scim.internal.icims.io version=2.2.0
6:35AM ??? [ERROR] TLS-ALPN challenge server: handshake: no certificate available for '172.18.0.3' application=op-scim build=202001 version=2.2.0
6:35AM ERR failed to GetTLSConfig, retrying after backoff delay error="Network: (could not obtain Let's Encrypt certificate), 1password-scim.internal.icims.io: obtaining certificate: [1password-scim.internal.icims.io] Obtain: [1password-scim.internal.icims.io] solving challenges: 1password-scim.internal.icims.io: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/264427320/36502548970) (ca=https://acme-v02.api.letsencrypt.org/directory)" application=op-scim build=202001 version=2.2.0

Port 80 is open and is listening

netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1029/sshd
tcp6 0 0 :::80 :::* LISTEN 6727/docker-proxy-c
tcp6 0 0 :::22 :::* LISTEN 1029/sshd
tcp6 0 0 :::3002 :::* LISTEN 6740/docker-proxy-c
tcp6 0 0 :::443 :::* LISTEN 6708/docker-proxy-c
udp 0 0 0.0.0.0:68 0.0.0.0:* 821/dhclient
udp 0 0 127.0.0.1:323 0.0.0.0:* 546/chronyd
udp6 0 0 ::1:323 :::* 546/chronyd

Hi @varun118 ,

We haven't been able to reproduce the Let's Encrypt issues you are seeing, even on v2.2.0.

Just to clarify:

On 2.1.0, you noticed Okta issues, so you attempted to upgrade to 2.2.x. But on 2.2.x, Let's Encrypt is now failing, correct?

Are you using any sort of HTTPS rewrite functionality in your AWS DNS? We've seen Cloudfare DNS cause some problems before, but your set up looks ok from what you listed. My other thought is that perhaps you have run into a rate limit with attempting to acquire a certificate for your domain.

We will continue to investigate and get back to you as quickly as we can.

We have a similar issue. We deployed a SCIM test bridge in Azure Kubernetes and receive below error. Public IP allocated, DNS zone available, port 80 opened. Any idea or solution identified?

ERR failed to GetTLSConfig, retrying after backoff delay error="Network: (could not obtain Let's Encrypt certificate), tst1pscim-dns-b06af904.hcp.germanywestcentral.azmk8s.io: obtaining certificate: [tst1pscim-dns-b06af904.hcp.germanywestcentral.azmk8s.io] Obtain: [tst1pscim-dns-b06af904.hcp.germanywestcentral.azmk8s.io] solving challenges: tst1pscim-dns-b06af904.hcp.germanywestcentral.azmk8s.io: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01]remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/268659220/37271842900) (ca=https://acme-v02.api.letsencrypt.org/directory)" application=op-scim build=202011 version=2.2.1

we have spunned up a docker container with scimbridge on ec2 instance which has a ELB with aws certs on it
but there are no specific redirection rules present on it

Hi

Do you have any update on this

Thanks

Hi @varun118.

Thank you for providing the additional information. You mentioned that you have port 80 open but I wanted to ask if you have port 443 open?

With the release of version 2.2.0 (and later) we have moved to a TLS-ALPN-01 challenge for Let's Encrypt. This means that a direct connection using port 443 is possible, which is one of the main advantages. This means that port 80 is no longer required for obtaining a certificate from Let's Encrypt.

Hi @fdietrich.

Thanks for sharing the error log.

Could you also check your configuration to ensure that port 443 is open?

Hi @DeVille_1P both port 80 and 443 were open
but still having this issue.

Hi @DeVille_1P the ports were opened. In our case the problem was with the DNS. We we're using as fqdn and DNS record the kubernetes API generated from Azure, but it required an additional custom domain to be registered.