Managing Argocd Permissions in Multi-cluster Eks Environments

Granting ArgoCD cluster-specific permissions in EKS requires explicit IAM roles and careful RBAC configuration to avoid.

JR

2 minute read

Granting ArgoCD cluster-specific permissions in EKS requires explicit IAM roles and careful RBAC configuration to avoid overprivilege.

When deploying ArgoCD in Amazon EKS, even for clusters where ArgoCD is deployed, you must explicitly grant permissions via IAM roles. This stems from EKS’s integration with IAM, which enforces IAM policies for Kubernetes API access. Skipping this step leads to permission denied errors during sync or apply operations.

Workflow: Secure ArgoCD Permissions in EKS

  1. Identify Required Permissions

    • ArgoCD needs eks:DescribeCluster, iam:AttachRolePolicy, and Kubernetes RBAC permissions (get, list, create on core resources).
    • For GitOps workflows, include s3:GetObject if using private repositories.
  2. Create IAM Role for ArgoCD

    • Use AWS IAM to create a role with a trust policy allowing the ArgoCD service account in its namespace to assume it.
    • Example trust policy:
      {  
        "Version": "2012-10-17",  
        "Statement": [  
          {  
            "Effect": "Allow",  
            "Principal": {  
              "Service": "eks.amazonaws.com"  
            },  
            "Action": "sts:AssumeRoleWithWebIdentity",  
            "Condition": {  
              "StringLike": {  
                "eks:awsserviceaccount.name": "argoCD:argoCD-controller"  
              }  
            }  
          }  
        ]  
      }  
      
  3. Attach IAM Policies to the Role

    • Use AWS managed policies like AmazonEKSClusterPolicy or create custom policies scoped to specific clusters.
    • Example inline policy for a single cluster:
      {  
        "Version": "2012-10-17",  
        "Statement": [  
          {  
            "Effect": "Allow",  
            "Action": [  
              "eks:DescribeCluster",  
              "iam:AttachRolePolicy"  
            ],  
            "Resource": "arn:aws:eks:region:account:cluster/my-cluster"  
          }  
        ]  
      }  
      
  4. Configure ArgoCD to Use the Role

    • Set the assumeRole field in ArgoCD’s cluster provider configuration:
      clusters:  
      - name: my-cluster  
        providerConfig:  
          name: aws  
          assumeRole: arn:aws:iam::account:role/ArgoCD-Cluster-Role  
      
  5. Validate Access

    • Run kubectl get clusters in ArgoCD’s namespace to confirm connectivity.
    • Check AWS CloudTrail for IAM permission denied errors.

Tooling

  • AWS CLI:

    aws iam create-role --role-name ArgoCD-Cluster-Role --assume-role-policy-document file://trust-policy.json  
    aws iam attach-role-policy --role-name ArgoCD-Cluster-Role --policy-arn arn:aws:iam::aws:policy/AmazonEKSClusterPolicy  
    
  • kubectl:

    kubectl auth can-i get pods -n argocd --as=system:serviceaccount:argoCD:argoCD-controller  
    

Tradeoff: Granularity vs. Maintenance

Explicit per-cluster roles reduce blast radius but increase overhead in environments with many clusters. For large-scale deployments, consider automation (e.g., Terraform modules) to generate roles and policies.

Troubleshooting

  • Permission Denied Errors:

    • Verify the IAM role’s trust policy matches the ArgoCD service account’s namespace and name.
    • Check that the role’s policies include required actions (e.g., eks:DescribeCluster).
  • Role Assumption Failures:

    • Confirm the AWS region in ArgoCD’s provider config matches the cluster’s region.
    • Use aws sts get-federated-token to test role assumption manually.
  • RBAC Misconfigurations:

    • Ensure the ArgoCD service account has cluster-admin or scoped Role/ClusterRole bindings in the target cluster.

By following this approach, you enforce least privilege while maintaining operational flexibility. Avoid wildcards in IAM policies unless absolutely necessary—specificity reduces risk.

Source thread: Weekly: This Week I Learned (TWIL?) thread

comments powered by Disqus