Avoiding Credential Chaos: Authenticating With No Secrets

Avoiding Credential Chaos: Authenticating With No Secrets

Speakers: Chitra Dhar Rajan, VP of Security and Privacy Engineering, Okta; Steve Jarvis, Security Architect, Okta

Conference: BSides Las Vegas

YouTube: https://www.youtube.com/watch?v=v9CcGjlbrwQ

Overview

Chitra Dhar Rajan and Steve Jarvis deliver a paired talk that reframes enterprise authentication and automation around a deliberately provocative golden rule: “Thou shalt not have the burden of any secrets.” They immediately qualify it: if secrets must exist for bootstrapping or similar, they belong in HSM or KMS with automated rotation as routine operations work—not as a panic response after incidents. The session is structured as a journey from a “red bad / blue good” current-state diagram of an imaginary company to a target state where engineers, CI/CD, services, and cross-cloud workloads rely on federated identity, short-lived tokens, and PKI rather than long-lived passwords and API keys. The speakers anchor motivation in breach economics (citing Thomson Reuters figures described as a global average data breach cost around $4.88 million in 2024 and a higher US average around $10 million) and the observation that many breaches involve lost, stolen, or harvested credentials.

Background

▶ Watch: Golden rule stated: no burden of secrets; caveat for HSM/KMS and automated ro... (4:00)

The talk paints a familiar enterprise collage: employees authenticate to SaaS (examples named include Confluence, Slack, GitHub), CI/CD deploys to cloud infrastructure, engineers reach servers via SSH keys, and microservices exchange API tokens. The speakers argue this everyday picture hides systemic risk—passwords where SSO should exist, SSH keys that may live indefinitely on laptops, cloud config files with long-lived keys, and service-to-service shared secrets that can be logged or leaked symmetrically on client and server.

Steve introduces a staged remediation plan across four areas: (1) engineer access to SaaS and servers, (2) GitHub workflows deploying to cloud (GitHub doubling as version control and deployment in their example), (3) services calling an API, and (4) cross-cluster service communication. The narrative is example-heavy toward AWS and Microsoft Entra patterns but explicitly states other clouds offer analogous primitives.

Key Findings

▶ Watch: WebAuthn, passkeys, and phishing resistance; keys stay on device. (12:00)

Passwordless user authentication. The session promotes WebAuthn-style flows where the device generates a private key and user presence/biometrics gates signing. A historical pain point—keys bound to one device—is described as largely addressed by passkeys and cross-device sync, restoring usability. Security upside: the private key does not leave the device, reducing phishing and server-side credential database exposure (attackers might steal public keys, which do not enable impersonation of the user to the relying party in the described model).

Server access without SSH keys to the internet. For incident response access to hosts, the speakers propose chaining IdP authentication to temporary AWS credentials via role assumption combined with AWS Systems Manager Session Manager, eliminating traditional SSH exposure and enabling hosts without open inbound shell ports—called out as a hardening win alongside credential reduction.

OIDC from CI/CD instead of static cloud secrets. GitHub Actions can act as an OIDC issuer, minting ID tokens for workflows. Cloud roles (AWS IAM roles, Entra service principals in the talk) trust those tokens, replacing long-lived access keys or client secrets stored in GitHub Secrets with configuration (role ARNs, trust policies). The speakers emphasize tight binding to repository, branch, and similar constraints when defining trust.

Private-key JWT for service-to-API access. Instead of indefinite API tokens that are identical on client and server and prone to log leakage, services present JWT assertions signed with a private key to an IdP, obtain short-lived access tokens, and call APIs with those. Benefits claimed: reduced blast radius if a token leaks, plus richer authorization via scopes and custom claims embedded at the IdP rather than opaque shared secrets.

Service mesh and mutual TLS vs pragmatic token sealing. For cross-cluster communication, ideal state is PKI with mutual TLS using workload certificates, long-lived trust anchored in a root CA in HSM, and frequent rotation of leaf certificates (potentially every deployment). The speakers acknowledge service mesh and attestation prerequisites make this heavy for many shops. A lighter improvement uses a pre-shared key not sent on the wire to build Iron-style encrypted/signed tokens (@hapi/iron module named), yielding a modest but low-cost upgrade over raw shared secrets.

Cross-cloud federation demo narrative. A live troubleshooting segment shows AKS and EKS workloads attempting cross-cloud role assumption. Failures illustrated include stale OIDC issuer identifiers tied to Azure subscription and cluster ID that change on redeploy, misconfigured IAM role trust for IRSA-style assumptions, and Amazon Cognito token issuer formatting (Kubernetes-issued tokens include an https:// prefix Cognito rejects until stripped). AWS Cognito is positioned as a stable issuer decoupled from ephemeral cluster identities—useful when Entra must trust an issuer that survives cluster rebuilds.

Technical Deep Dive

▶ Watch: GitHub as OIDC issuer to assume IAM roles and Entra service principals withou... (16:00)

The WebAuthn discussion connects usability and security: syncing passkeys addresses cross-device login while preserving phishing resistance. The Session Manager pattern shifts trust to AWS APIs and IAM session credentials with tight lifetime rather than persistent host keys.

The GitHub OIDC → cloud role pattern is the CI/CD centerpiece: trust policies must encode least privilege at the workflow granularity the organization accepts. The speakers argue this removes an entire category of leaked static keys from pipeline secrets stores.

For private-key JWT, the talk stresses separation between signing key material (held by the service, ideally in KMS/HSM) and bearer access tokens presented to APIs—reducing symmetric secret duplication.

The mTLS section describes a realistic PKI hierarchy: root in HSM, intermediates, workload certificates rotated with deployments so rotation becomes boring rather than a quarterly fire drill.

The Iron token mitigation is framed honestly: it does not remove the pre-shared key, but keeps it off the wire and wraps short-lived payloads.

The demo debugging covers Terraform layers (identity providers, roles), Kubernetes manifests (service account annotations for allowed role ARNs), and application code adjustments—reflecting how federation breaks in practice when infrastructure is redeployed without updating trust anchors.

Audience Q&A on Iron clarifies it is not “just a certificate”: it derives keys from the shared secret material to provide integrity and confidentiality for the encapsulated token. That matters for teams evaluating stopgaps—understand what cryptography you are buying with the simpler pattern versus mTLS.

Demo / Proof of Concept

▶ Watch: Private-key JWT to IdP for short-lived API access tokens with scopes and claims. (18:00)

Steve runs a live demo with split panes showing applications in Azure AKS and AWS EKS. Initial failures include “no OpenID Connect provider found for this issuer” when the Azure cluster ID changed, fixed via Terraform apply updating the IAM OIDC provider mapping. A second failure—“not authorized to perform sts:AssumeRoleWithWebIdentity”—traces to a wrong role ARN in a service account annotation, corrected with kubectl deployment refresh. A third issue involves Cognito rejecting a malformed issuer string; trimming the https:// prefix in Python application code is shown as the fix. Container rebuild/push to ACR/ECR is mentioned when code changes require new images. The demo intentionally exposes misconfiguration classes teams hit during blue/green cluster operations.

Defensive Implications

▶ Watch: Live demo: OIDC issuer mismatch after AKS redeploy and Terraform fix for IAM ... (30:00)

Program design: prioritize passwordless where humans log in; eliminate SSH keys for cloud access where SSM or cloud-native shell channels exist; adopt OIDC federation for pipelines; shift services to OAuth2-style short-lived tokens with scoped claims; plan mTLS where mesh maturity allows, otherwise apply token sealing improvements.

For security architecture review boards, the session offers a checklist-friendly narrative: each connection in the reference diagram should have an explicit credential type, rotation mechanism, storage location, and blast-radius story if logged or leaked.

Operational reality: federation trust must be automated alongside cluster lifecycle—issuer IDs that embed cluster identifiers will drift on rebuild. The demo is a template for runbooks: when a cluster is recreated, update OIDC providers, annotations, and any Cognito integrations in the same change window.

Cryptographic hygiene: long-lived secrets that remain necessary should live in HSM/KMS and rotate on a cadence tied to deployments or calendar rhythm (“rain or shine”).

Key Takeaways

• Treat static credentials as liabilities; default architecture should push humans and automation toward federated, short-lived trust.
• Passkeys and WebAuthn reduce password and phishing risk while improving MFA posture when implemented with modern device sync.
• GitHub OIDC can remove cloud access keys from CI if trust policies are precise to repo and branch.
• Private-key JWT exchanges move services from opaque API keys to auditable scopes and TTL-bound bearer tokens.
• Cross-cloud and IRSA-style setups break when cluster issuers rotate; introduce a stable IdP (example: Cognito) when federation partners cannot tolerate churn.
• Where mTLS is too heavy, Iron-style constructions can still keep PSKs off the wire.
• Golden rule + caveat: minimize secrets; for the remainder, HSM/KMS + automated rotation is non-negotiable.

About the Speaker(s)

Chitra Dhar Rajan is introduced as VP of Security and Privacy Engineering at Okta, focused on building high-performance global teams and security transformations, advising Bay Area startups, and advocating security as an enabler rather than a gatekeeper. Steve Jarvis is introduced as a Security Architect at Okta with a long background in software engineering for network and security products; his stated mantra is that the secure path must be the simple path. Personal details mentioned include Steve’s cycling and family; Chitra jokes about Steve’s strong opinions on bicycle tires. Any additional titles or histories beyond the introductions are not detailed in the transcript.