Security - AWX

Overview

AWX implements multiple layers of security including credential encryption, role-based access control, authentication providers, and secure communication. Understanding these features is essential for maintaining a secure automation platform.

Credential Encryption

AWX encrypts all sensitive credential data before storing it in the database.

Encryption Method

AWX uses AES-256 in CBC mode with the following characteristics:

256-bit encryption key
PKCS7 padding
HMAC using SHA-256 for authentication
Unique initialization vector (IV) per encrypted value

Encrypted fields include:

Passwords and passphrases
SSH private keys
API tokens and secret keys
Cloud service credentials
Custom credential type secret fields
External service authentication tokens

Secret Key Management

The SECRET_KEY is the master encryption key for all sensitive data. Location:

Traditional: /etc/tower/SECRET_KEY
Kubernetes: Secret named awx-secret-key

Never commit the SECRET_KEY to version control or expose it in logs. Loss of the SECRET_KEY makes all encrypted credentials permanently unrecoverable.

Rotating the Secret Key

To rotate the SECRET_KEY:

Generate new key

openssl rand -base64 32 > NEW_SECRET_KEY

Decrypt with old key

Use awx-manage to decrypt credentials with the current key:

kubectl exec -it -n awx deployment/awx-task -c awx-task -- \
  awx-manage shell_plus

from awx.main.utils import decrypt_field, encrypt_field
from awx.main.models import Credential

# Document current credentials before rotation
for cred in Credential.objects.all():
    print(f"Credential {cred.id}: {cred.name}")

Update SECRET_KEY

Kubernetes:

kubectl create secret generic awx-secret-key \
  --from-file=secret_key=./NEW_SECRET_KEY \
  -n awx --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout restart deployment/awx-web -n awx
kubectl rollout restart deployment/awx-task -n awx

Traditional:

cp NEW_SECRET_KEY /etc/tower/SECRET_KEY
chown awx:awx /etc/tower/SECRET_KEY
chmod 600 /etc/tower/SECRET_KEY
systemctl restart awx

Re-encrypt credentials

All credentials must be updated with the new key. This requires re-entering sensitive values through the UI or API.

Secret key rotation requires re-creating all credentials. Plan this during a maintenance window and have credential values ready to re-enter.

Extracting Encrypted Values

If you need to extract credentials for migration or debugging:

# Access AWX shell
kubectl exec -it -n awx deployment/awx-task -c awx-task -- awx-manage shell_plus

# Decrypt a credential field
from awx.main.utils import decrypt_field
from awx.main.models import Credential

cred = Credential.objects.get(name="my-credential")
decrypted_password = decrypt_field(cred, "password")
print(decrypted_password)

# Decrypt a setting
from awx.conf.models import Setting
setting = Setting.objects.get(key='SOCIAL_AUTH_GITHUB_SECRET')
decrypted_value = decrypt_field(setting, 'value')

Only extract credentials in secure environments. Never log or expose decrypted values.

External Secret Management

AWX can retrieve secrets from external secret management systems instead of storing them encrypted in the database.

Supported Systems

HashiCorp Vault KV — Key/Value secret engine
HashiCorp Vault SSH — SSH certificate signing
Microsoft Azure Key Vault
CyberArk Conjur
Thycotic Secret Server

HashiCorp Vault Integration

Setup
Link to credential

Create a HashiCorp Vault credential:

curl -X POST https://awx.example.org/api/v2/credentials/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "HashiCorp Vault",
    "credential_type": 15,
    "inputs": {
      "url": "https://vault.example.org",
      "token": "hvs.XXXXXX",
      "api_version": "v2"
    }
  }'

Link a credential field to Vault:

curl -X POST https://awx.example.org/api/v2/credentials/1/input_sources/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "source_credential": 2,
    "input_field_name": "password",
    "metadata": {
      "secret_path": "/kv/data/my-secret",
      "secret_key": "password"
    }
  }'

External secrets are fetched on-demand when jobs run. They are never stored in the AWX database, providing enhanced security for sensitive credentials.

Custom Credential Plugins

You can write custom credential plugins to integrate with additional secret management systems:

# Example custom credential plugin
def my_secret_backend(**kwargs):
    url = kwargs.get('url')
    token = kwargs.get('token')
    secret_path = kwargs.get('secret_path')
    
    # Fetch secret from your system
    response = requests.get(
        f"{url}/{secret_path}",
        headers={"Authorization": f"Bearer {token}"}
    )
    return response.json()['secret']

Authentication Methods

AWX supports multiple authentication backends for user login.

Local Authentication

Default username/password authentication using the AWX database. Enable local authentication:

kubectl exec -it -n awx deployment/awx-task -c awx-task -- \
  awx-manage enable_local_authentication

Create a local user:

curl -X POST https://awx.example.org/api/v2/users/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "username": "analyst",
    "password": "secure_password",
    "email": "analyst@example.org",
    "is_superuser": false
  }'

LDAP Authentication

Configure LDAP for centralized user management:

# /etc/tower/conf.d/ldap.py
AUTH_LDAP_SERVER_URI = 'ldap://ldap.example.org'
AUTH_LDAP_BIND_DN = 'cn=awx,ou=services,dc=example,dc=org'
AUTH_LDAP_BIND_PASSWORD = 'ldap_password'
AUTH_LDAP_USER_SEARCH = LDAPSearch(
    'ou=users,dc=example,dc=org',
    ldap.SCOPE_SUBTREE,
    '(uid=%(user)s)'
)
AUTH_LDAP_GROUP_SEARCH = LDAPSearch(
    'ou=groups,dc=example,dc=org',
    ldap.SCOPE_SUBTREE,
    '(objectClass=groupOfNames)'
)
AUTH_LDAP_USER_ATTR_MAP = {
    'first_name': 'givenName',
    'last_name': 'sn',
    'email': 'mail'
}

Apply via the API:

curl -X PATCH https://awx.example.org/api/v2/settings/ldap/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "AUTH_LDAP_SERVER_URI": "ldap://ldap.example.org",
    "AUTH_LDAP_BIND_DN": "cn=awx,ou=services,dc=example,dc=org",
    "AUTH_LDAP_BIND_PASSWORD": "ldap_password"
  }'

SAML Authentication

Configure SAML 2.0 for single sign-on:

curl -X PATCH https://awx.example.org/api/v2/settings/saml/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "SOCIAL_AUTH_SAML_ENABLED_IDPS": {
      "myidp": {
        "entity_id": "https://idp.example.org",
        "url": "https://idp.example.org/sso",
        "x509cert": "MIIDXTCCAkWgAwIBAgIJ..."
      }
    },
    "SOCIAL_AUTH_SAML_SP_ENTITY_ID": "https://awx.example.org",
    "SOCIAL_AUTH_SAML_SP_PUBLIC_CERT": "MIIDXTCCAkWgAwIBAgIJ...",
    "SOCIAL_AUTH_SAML_SP_PRIVATE_KEY": "-----BEGIN PRIVATE KEY-----..."
  }'

OAuth 2.0 / OpenID Connect

Supported providers:

GitHub
Google OAuth2
Azure AD
Generic OIDC

GitHub example:

curl -X PATCH https://awx.example.org/api/v2/settings/github/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "SOCIAL_AUTH_GITHUB_KEY": "github_oauth_client_id",
    "SOCIAL_AUTH_GITHUB_SECRET": "github_oauth_client_secret",
    "SOCIAL_AUTH_GITHUB_ORGANIZATION_MAP": {
      "Default": {
        "users": true
      },
      "Engineering": {
        "admins": ["my-github-org"]
      }
    }
  }'

Role-Based Access Control (RBAC)

AWX uses django-ansible-base for role-based access control. RBAC determines what users can view, create, edit, or delete.

Role Types

System Roles

System Administrator — Full control over AWX
System Auditor — Read-only access to all objects

Object Roles

Admin — Full control over an object and its children
Execute — Can launch jobs or workflows
Update — Can modify object configuration
Use — Can use but not modify (credentials, inventories)
Read — Read-only access to an object

Assigning Roles

Via API:

# Grant user execute permission on a job template
curl -X POST https://awx.example.org/api/v2/job_templates/5/access_list/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "user": 12,
    "role_definition": "JobTemplate Execute"
  }'

# Grant team admin permission on an organization
curl -X POST https://awx.example.org/api/v2/organizations/3/access_list/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "team": 8,
    "role_definition": "Organization Admin"
  }'

Via awx CLI:

awx role grant --user analyst --job-template "Deploy App" --role execute
awx role grant --team developers --organization "Engineering" --role admin

RBAC Hierarchy

Permissions inherit down the object hierarchy:

Organization (Admin)
  ├── Inventories (Inherited Admin)
  │   └── Hosts (Inherited Admin)
  ├── Projects (Inherited Admin)
  └── Job Templates (Inherited Admin)

Grant roles at the highest appropriate level to simplify permission management. Organization admins automatically have admin rights to all child objects.

Checking User Permissions

# List all roles for a user
curl -X GET https://awx.example.org/api/v2/users/12/roles/ \
  -H "Authorization: Bearer <token>"

# Check if user can execute a job template
curl -X GET https://awx.example.org/api/v2/job_templates/5/ \
  -H "Authorization: Bearer <token>" \
  | jq '.summary_fields.user_capabilities.start'

HTTPS and TLS Configuration

Kubernetes with Ingress

Configure TLS using Kubernetes Ingress:

awx-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: awx-ingress
  namespace: awx
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - awx.example.org
    secretName: awx-tls-cert
  rules:
  - host: awx.example.org
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: awx-service
            port:
              number: 80

Traditional Deployment with Nginx

Configure Nginx as a reverse proxy with TLS:

server {
    listen 443 ssl http2;
    server_name awx.example.org;

    ssl_certificate /etc/ssl/certs/awx.crt;
    ssl_certificate_key /etc/ssl/private/awx.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    location / {
        proxy_pass http://localhost:8052;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /websocket {
        proxy_pass http://localhost:8052/websocket;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

server {
    listen 80;
    server_name awx.example.org;
    return 301 https://$server_name$request_uri;
}

Certificate Management

Using cert-manager (Kubernetes):

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: awx-tls
  namespace: awx
spec:
  secretName: awx-tls-cert
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
  - awx.example.org

Manual certificate:

# Generate self-signed certificate (development only)
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout awx.key -out awx.crt \
  -subj "/CN=awx.example.org"

# Create Kubernetes secret
kubectl create secret tls awx-tls-cert \
  --cert=awx.crt --key=awx.key -n awx

Never use self-signed certificates in production. Use certificates from a trusted CA or Let’s Encrypt.

Network Security

Firewall Configuration

Required ports:

Port	Protocol	Purpose
443	TCP	HTTPS web interface
80	TCP	HTTP (redirect to HTTPS)
27199	TCP	Receptor mesh (between AWX instances)
5432	TCP	PostgreSQL (internal only)

Kubernetes network policies:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: awx-network-policy
  namespace: awx
spec:
  podSelector:
    matchLabels:
      app: awx
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: ingress-nginx
    ports:
    - protocol: TCP
      port: 8052
  egress:
  - to:
    - podSelector:
        matchLabels:
          app: postgres
    ports:
    - protocol: TCP
      port: 5432

Database Connection Security

Configure PostgreSQL to require SSL:

# /etc/tower/conf.d/postgres.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'awx',
        'USER': 'awx',
        'PASSWORD': os.environ.get('AWX_DB_PASSWORD'),
        'HOST': 'postgres.example.org',
        'PORT': '5432',
        'OPTIONS': {
            'sslmode': 'require',
            'sslrootcert': '/etc/ssl/certs/ca-bundle.crt'
        }
    }
}

Security Best Practices

Principle of least privilege

Grant users only the minimum permissions needed
Use team-based permissions instead of individual user permissions
Regularly audit user roles and remove unnecessary access
Avoid granting System Administrator role unless absolutely necessary

Credential management

Use external secret management (Vault, Azure Key Vault) for sensitive credentials
Rotate credentials regularly (every 90 days minimum)
Never hardcode credentials in playbooks or job templates
Use separate credentials for each environment (dev, staging, production)
Enable credential input on launch only when necessary

Authentication security

Enforce multi-factor authentication (MFA) through your identity provider
Use SSO (SAML/OAuth) instead of local passwords when possible
Set strong password policies for local accounts
Implement session timeouts: SESSION_COOKIE_AGE = 3600 (1 hour)
Disable unused authentication backends

Network security

Always use HTTPS/TLS for web interface access
Restrict AWX access to internal networks or VPN
Use firewall rules to limit database access to AWX hosts only
Enable network policies in Kubernetes
Use private container registries for execution environments

Audit and monitoring

Enable activity stream logging for all object changes
Export logs to a centralized logging system (Splunk, ELK, etc.)
Monitor failed authentication attempts
Alert on privilege escalation (user becomes admin)
Review audit logs regularly for suspicious activity

System hardening

Keep AWX updated to the latest stable version
Apply security patches promptly
Minimize installed packages in execution environments
Run AWX containers as non-root users
Enable SELinux or AppArmor where applicable
Regular vulnerability scanning of container images

Security Auditing

Activity Stream

All object changes are logged in the activity stream:

# View all activity
curl -X GET https://awx.example.org/api/v2/activity_stream/ \
  -H "Authorization: Bearer <token>"

# View changes to a specific credential
curl -X GET https://awx.example.org/api/v2/credentials/5/activity_stream/ \
  -H "Authorization: Bearer <token>"

# Filter by action type
curl -X GET "https://awx.example.org/api/v2/activity_stream/?operation=delete" \
  -H "Authorization: Bearer <token>"

Export Audit Logs

# Export last 30 days of activity
curl -X GET "https://awx.example.org/api/v2/activity_stream/?timestamp__gte=$(date -d '30 days ago' -Iseconds)" \
  -H "Authorization: Bearer <token>" \
  | jq '.results[] | {timestamp, operation, object_type, changes}' \
  > audit_log_$(date +%Y%m%d).json

Compliance Reporting

Generate compliance reports:

# Access AWX shell
kubectl exec -it -n awx deployment/awx-task -c awx-task -- awx-manage shell_plus

# Report on users with admin access
from django.contrib.auth.models import User
from awx.main.models import Role

admins = User.objects.filter(
    roles__role_field='admin_role'
).distinct()

for user in admins:
    print(f"{user.username}: {user.email}")

# Report on credentials not used in 90 days
from datetime import timedelta
from django.utils import timezone
from awx.main.models import Credential

cutoff = timezone.now() - timedelta(days=90)
unused_creds = Credential.objects.filter(
    jobtemplates__isnull=True,
    created__lt=cutoff
)

for cred in unused_creds:
    print(f"Unused: {cred.name} (created {cred.created})")

Incident Response

In case of a security incident:

Contain the incident

Immediately revoke compromised credentials
Disable affected user accounts
Block suspicious IP addresses at firewall level

# Disable a user
curl -X PATCH https://awx.example.org/api/v2/users/12/ \
  -H "Authorization: Bearer <token>" \
  -d '{"is_active": false}'

Assess the impact

Review activity stream for unauthorized actions
Check job history for suspicious playbook runs
Identify what systems were accessed
Determine if credentials were exfiltrated

Eradicate the threat

Rotate all potentially compromised credentials
Update the SECRET_KEY if database access was compromised
Apply security patches if a vulnerability was exploited
Reset passwords for affected accounts

Recover and restore

Re-enable systems after verification
Monitor closely for recurring issues
Document the incident timeline
Update security procedures

Post-incident review

Conduct root cause analysis
Update incident response procedures
Implement additional security controls
Train team on lessons learned

​Overview

​Credential Encryption

​Encryption Method

​Secret Key Management

​Rotating the Secret Key

​Extracting Encrypted Values

​External Secret Management

​Supported Systems

​HashiCorp Vault Integration

​Custom Credential Plugins

​Authentication Methods

​Local Authentication

​LDAP Authentication

​SAML Authentication

​OAuth 2.0 / OpenID Connect

​Role-Based Access Control (RBAC)

​Role Types

System Roles

Object Roles

​Assigning Roles

​RBAC Hierarchy

​Checking User Permissions

​HTTPS and TLS Configuration

​Kubernetes with Ingress

​Traditional Deployment with Nginx

​Certificate Management

​Network Security

​Firewall Configuration

​Database Connection Security

​Security Best Practices

​Security Auditing

​Activity Stream

​Export Audit Logs

​Compliance Reporting

​Incident Response

​See Also

Overview

Credential Encryption

Encryption Method

Secret Key Management

Rotating the Secret Key

Extracting Encrypted Values

External Secret Management

Supported Systems

HashiCorp Vault Integration

Custom Credential Plugins

Authentication Methods

Local Authentication

LDAP Authentication

SAML Authentication

OAuth 2.0 / OpenID Connect

Role-Based Access Control (RBAC)

Role Types

Assigning Roles

RBAC Hierarchy

Checking User Permissions

HTTPS and TLS Configuration

Kubernetes with Ingress

Traditional Deployment with Nginx

Certificate Management

Network Security

Firewall Configuration

Database Connection Security

Security Best Practices

Security Auditing

Activity Stream

Export Audit Logs

Compliance Reporting

Incident Response

See Also