Terraform Multi-Team Resource Example with Tag-Based Preconditions

.gitignore

# Local .terraform directories
**/.terraform/*

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log
crash.*.log

# Exclude all .tfvars files, which are likely to contain sensitive data
*.tfvars
*.tfvars.json

# Ignore override files as they are usually used to override resources locally
override.tf
override.tf.json
*_override.tf
*_override.tf.json

# Include override files you do wish to add to version control using negated pattern
# !example_override.tf

# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
*tfplan*

# Ignore CLI configuration files
.terraformrc
terraform.rc

# Ignore Mac system files
.DS_Store

# Ignore lock files (optional - some teams prefer to commit this)
# .terraform.lock.hcl

README.md

Terraform Multi-Team Resource Example with Tag-Based Preconditions

This example demonstrates how to use Terraform data sources and preconditions to enable multiple teams to run the same Infrastructure as Code (IaC) while ensuring resources are only created once.

Overview

This pattern is useful when:

• Multiple teams need to deploy shared infrastructure
• You want to prevent duplicate resource creation
• Resources should be identified by specific tags
• Teams should be able to detect existing resources before attempting creation

How It Works

1. Team 1 (team-alpha) runs the Terraform code first

   • No matching resources exist
   • Precondition passes
   • S3 bucket is created with required tags

2. Team 2 (team-beta) runs the same Terraform code

   • Data source queries all S3 buckets
   • Filters buckets by required tags
   • Finds the bucket created by Team 1
   • Precondition detects existing resource
   • No new bucket is created
   • Outputs confirm the existing resource was found

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Terraform Execution                      │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  1. Data Source: aws_s3_buckets                              │
│     └─> Lists all S3 buckets in the account                 │
│                                                               │
│  2. Data Source: aws_s3_bucket (for_each)                   │
│     └─> Retrieves tags for each bucket                      │
│                                                               │
│  3. Local Value: matching_buckets                            │
│     └─> Filters buckets by required_tags                    │
│                                                               │
│  4. Lifecycle Precondition                                   │
│     └─> Checks if matching_buckets is empty                 │
│         ├─ Empty: Create resource                           │
│         └─ Not Empty: Skip creation, output existing        │
│                                                               │
└─────────────────────────────────────────────────────────────┘

File Structure

.
├── main.tf           # Main Terraform configuration with data sources and resources
├── variables.tf      # Input variables for customization
├── outputs.tf        # Outputs showing creation status and found resources
├── team1.tfvars      # Configuration for Team 1 (team-alpha)
├── team2.tfvars      # Configuration for Team 2 (team-beta)
└── README.md         # This file

Prerequisites

• Terraform >= 1.5.0
• AWS CLI configured with appropriate credentials
• AWS permissions to:
  • List S3 buckets
  • Read S3 bucket tags
  • Create S3 buckets (for Team 1)

Usage

Team 1: First Deployment (Creates Resource)

# Initialize Terraform
terraform init

# Plan the deployment
terraform plan -var-file="team1.tfvars"

# Apply the configuration
terraform apply -var-file="team1.tfvars"

Expected Output:

bucket_created = true
bucket_name = "shared-resource-dev-team-alpha"
existing_buckets_found = []
resource_exists = false
summary = "Team 'team-alpha' created new bucket: shared-resource-dev-team-alpha"

Team 2: Subsequent Deployment (Detects Existing Resource)

# Initialize Terraform (if not already done)
terraform init

# Plan the deployment
terraform plan -var-file="team2.tfvars"

# Apply the configuration
terraform apply -var-file="team2.tfvars"

Expected Output:

bucket_created = false
bucket_name = "shared-resource-dev-team-beta"
existing_buckets_found = ["shared-resource-dev-team-alpha"]
resource_exists = true
summary = "Team 'team-beta' found existing bucket(s): shared-resource-dev-team-alpha. No new resource created."

Key Components

Data Sources

aws_s3_buckets: Lists all S3 buckets in the AWS account

data "aws_s3_buckets" "tagged" {
  count = var.check_existing ? 1 : 0
}

aws_s3_bucket: Retrieves detailed information including tags for each bucket

data "aws_s3_bucket" "check_tags" {
  for_each = var.check_existing ? toset(try(data.aws_s3_buckets.tagged[0].ids, [])) : []
  bucket   = each.value
}

Tag Filtering Logic

locals {
  matching_buckets = var.check_existing ? [
    for bucket_name, bucket_data in data.aws_s3_bucket.check_tags :
    bucket_name
    if alltrue([
      for tag_key, tag_value in var.required_tags :
      try(bucket_data.tags[tag_key], "") == tag_value
    ])
  ] : []
  
  resource_exists = length(local.matching_buckets) > 0
  should_create   = var.check_existing ? !local.resource_exists : true
}

Precondition

lifecycle {
  precondition {
    condition     = !var.check_existing || !local.resource_exists
    error_message = <<-EOT
      Resource already exists with matching tags!
      Found bucket(s): ${join(", ", local.matching_buckets)}
      Team ${var.team_name} should not create this resource as it already exists.
    EOT
  }
}

Configuration Variables

Variable	Description	Default	Required
team_name	Name of the team running the code	-	Yes
project_name	Name of the project	shared-resource	No
environment	Environment name (dev, staging, prod)	dev	No
aws_region	AWS region for resources	us-east-1	No
check_existing	Enable existence checking	true	No
required_tags	Tags to match for resource identification	See below	No

Default Required Tags

{
  Project     = "shared-resource"
  Environment = "dev"
  Purpose     = "multi-team-example"
}

Outputs

Output	Description
bucket_created	Boolean indicating if a new bucket was created
bucket_name	Name of the bucket (created or would-be)
bucket_id	ID of the created bucket (empty if not created)
bucket_arn	ARN of the created bucket (empty if not created)
existing_buckets_found	List of existing buckets matching required tags
resource_exists	Boolean indicating if matching resource exists
team_name	Name of the team that ran the configuration
summary	Human-readable summary of the operation

Testing Scenarios

Scenario 1: Fresh Deployment

# Team 1 creates the resource
terraform apply -var-file="team1.tfvars" -auto-approve

# Verify bucket was created
aws s3 ls | grep shared-resource-dev-team-alpha

Scenario 2: Duplicate Prevention

# Team 2 attempts to create (should detect existing)
terraform apply -var-file="team2.tfvars" -auto-approve

# Verify no duplicate bucket
aws s3 ls | grep shared-resource-dev-team-beta  # Should not exist

Scenario 3: Override Behavior

# Force creation without checking (not recommended)
terraform apply -var-file="team2.tfvars" -var="check_existing=false" -auto-approve

Cleanup

# Team 1 destroys their resources
terraform destroy -var-file="team1.tfvars" -auto-approve

# If Team 2 created resources with check_existing=false
terraform destroy -var-file="team2.tfvars" -auto-approve

Best Practices

1. Consistent Tagging: Ensure all teams use the same required_tags values
2. Tag Strategy: Use meaningful tags that uniquely identify shared resources
3. State Management: Use remote state backends (S3 + DynamoDB) for team collaboration
4. Permissions: Implement least-privilege IAM policies
5. Documentation: Keep team-specific tfvars files well-documented

Extending This Pattern

This pattern can be adapted for other AWS resources:

• EC2 Instances: Filter by instance tags
• RDS Databases: Check for existing DB instances by tags
• VPCs: Identify shared network infrastructure
• IAM Roles: Prevent duplicate role creation
• Lambda Functions: Check for existing functions by tags

Example: Adapting for EC2

data "aws_instances" "existing" {
  filter {
    name   = "tag:Project"
    values = [var.required_tags["Project"]]
  }
  
  filter {
    name   = "tag:Environment"
    values = [var.required_tags["Environment"]]
  }
  
  instance_state_names = ["running", "stopped"]
}

locals {
  instance_exists = length(data.aws_instances.existing.ids) > 0
}

Troubleshooting

Issue: "Bucket already exists" error

Solution: Another team or process created a bucket with the same name but different tags. Update bucket_name logic or tags.

Issue: Precondition not detecting existing resource

Solution: Verify that:

• AWS credentials have permission to list buckets and read tags
• required_tags match exactly (case-sensitive)
• check_existing is set to true

Issue: State drift between teams

Solution: Use remote state backend and state locking:

terraform {
  backend "s3" {
    bucket         = "terraform-state-bucket"
    key            = "shared-resource/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}

Security Considerations

• Encryption: All buckets are encrypted with AES256 by default
• Public Access: Public access is blocked on all created buckets
• Versioning: Bucket versioning is enabled for data protection
• IAM: Follow least-privilege principle for AWS credentials
• Tags: Avoid including sensitive information in tags

License

This example is provided as-is for educational purposes.

Contributing

Contributions and improvements are welcome! Please ensure:

• Code follows Terraform best practices
• Documentation is updated
• Examples are tested

References

• Terraform Lifecycle Meta-Arguments
• AWS S3 Data Sources
• Terraform Preconditions and Postconditions

main.tf

terraform {
  required_version = ">= 1.5.0"
  
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

# Data source to check if a bucket with the specified tags already exists
data "aws_s3_bucket" "existing" {
  count  = var.check_existing ? 1 : 0
  bucket = local.bucket_name

  # This will fail if bucket doesn't exist, which we'll handle with lifecycle
  lifecycle {
    postcondition {
      condition     = self.id != "" || !var.enforce_precondition
      error_message = "Bucket ${local.bucket_name} does not exist and enforce_precondition is true."
    }
  }
}

# Alternative: Use aws_s3_buckets data source to list and filter by tags
data "aws_s3_buckets" "tagged" {
  count = var.check_existing ? 1 : 0
}

# Check each bucket for matching tags
data "aws_s3_bucket" "check_tags" {
  for_each = var.check_existing ? toset(try(data.aws_s3_buckets.tagged[0].ids, [])) : []
  bucket   = each.value
}

locals {
  bucket_name = "${var.project_name}-${var.environment}-${var.team_name}"
  
  # Find buckets that match our required tags
  matching_buckets = var.check_existing ? [
    for bucket_name, bucket_data in data.aws_s3_bucket.check_tags :
    bucket_name
    if alltrue([
      for tag_key, tag_value in var.required_tags :
      try(bucket_data.tags[tag_key], "") == tag_value
    ])
  ] : []
  
  # Determine if we should create the resource
  resource_exists = length(local.matching_buckets) > 0
  should_create   = var.check_existing ? !local.resource_exists : true
}

# S3 Bucket with conditional creation based on tag filtering
resource "aws_s3_bucket" "example" {
  count = local.should_create ? 1 : 0
  
  bucket = local.bucket_name
  
  tags = merge(
    var.required_tags,
    {
      Name        = local.bucket_name
      Team        = var.team_name
      Environment = var.environment
      ManagedBy   = "Terraform"
      CreatedBy   = var.team_name
    }
  )
  
  lifecycle {
    precondition {
      condition     = !var.check_existing || !local.resource_exists
      error_message = <<-EOT
        Resource already exists with matching tags!
        Found bucket(s): ${join(", ", local.matching_buckets)}
        Team ${var.team_name} should not create this resource as it already exists.
        Set check_existing=false to override this behavior.
      EOT
    }
  }
}

# Bucket versioning configuration
resource "aws_s3_bucket_versioning" "example" {
  count  = local.should_create ? 1 : 0
  bucket = aws_s3_bucket.example[0].id
  
  versioning_configuration {
    status = "Enabled"
  }
}

# Bucket encryption configuration
resource "aws_s3_bucket_server_side_encryption_configuration" "example" {
  count  = local.should_create ? 1 : 0
  bucket = aws_s3_bucket.example[0].id
  
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

# Public access block
resource "aws_s3_bucket_public_access_block" "example" {
  count  = local.should_create ? 1 : 0
  bucket = aws_s3_bucket.example[0].id
  
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

outputs.tf

output "bucket_created" {
  description = "Whether a new bucket was created by this run"
  value       = local.should_create
}

output "bucket_name" {
  description = "Name of the bucket (created or existing)"
  value       = local.bucket_name
}

output "bucket_id" {
  description = "ID of the created bucket (empty if not created)"
  value       = try(aws_s3_bucket.example[0].id, "")
}

output "bucket_arn" {
  description = "ARN of the created bucket (empty if not created)"
  value       = try(aws_s3_bucket.example[0].arn, "")
}

output "existing_buckets_found" {
  description = "List of existing buckets that match the required tags"
  value       = local.matching_buckets
}

output "resource_exists" {
  description = "Whether a resource with matching tags already exists"
  value       = local.resource_exists
}

output "team_name" {
  description = "Name of the team that ran this configuration"
  value       = var.team_name
}

output "check_performed" {
  description = "Whether existence check was performed"
  value       = var.check_existing
}

output "summary" {
  description = "Summary of the operation"
  value = local.should_create ? (
    "Team '${var.team_name}' created new bucket: ${local.bucket_name}"
    ) : (
    "Team '${var.team_name}' found existing bucket(s): ${join(", ", local.matching_buckets)}. No new resource created."
  )
}

team1.tfvars

# Team 1 Configuration
# This team will create the resource on first run

team_name       = "team-alpha"
project_name    = "shared-resource"
environment     = "dev"
aws_region      = "us-east-1"
check_existing  = true

required_tags = {
  Project     = "shared-resource"
  Environment = "dev"
  Purpose     = "multi-team-example"
}

team2.tfvars

# Team 2 Configuration
# This team will NOT create the resource if Team 1 already created it

team_name       = "team-beta"
project_name    = "shared-resource"
environment     = "dev"
aws_region      = "us-east-1"
check_existing  = true

required_tags = {
  Project     = "shared-resource"
  Environment = "dev"
  Purpose     = "multi-team-example"
}

terraform.tfvars.example

# Example Terraform Variables Configuration
# Copy this file to terraform.tfvars and customize for your team

# Required: Your team's name
team_name = "your-team-name"

# Project configuration
project_name = "shared-resource"
environment  = "dev"
aws_region   = "us-east-1"

# Enable checking for existing resources
check_existing = true

# Tags used to identify shared resources
# All teams must use the same tags for the pattern to work
required_tags = {
  Project     = "shared-resource"
  Environment = "dev"
  Purpose     = "multi-team-example"
}

validate.sh

#!/bin/bash
# Validation script for Terraform multi-team example

set -e

echo "=================================="
echo "Terraform Multi-Team Example Validator"
echo "=================================="
echo ""

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

# Check if Terraform is installed
echo "Checking prerequisites..."
if ! command -v terraform &> /dev/null; then
    echo -e "${RED}✗ Terraform is not installed${NC}"
    echo "Please install Terraform: https://www.terraform.io/downloads"
    exit 1
fi
echo -e "${GREEN}✓ Terraform is installed${NC}"

# Check Terraform version
TF_VERSION=$(terraform version -json | grep -o '"terraform_version":"[^"]*' | cut -d'"' -f4)
echo "  Version: $TF_VERSION"

# Check if AWS CLI is installed
if ! command -v aws &> /dev/null; then
    echo -e "${YELLOW}⚠ AWS CLI is not installed (optional but recommended)${NC}"
else
    echo -e "${GREEN}✓ AWS CLI is installed${NC}"
    
    # Check AWS credentials
    if aws sts get-caller-identity &> /dev/null; then
        echo -e "${GREEN}✓ AWS credentials are configured${NC}"
        ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
        echo "  Account ID: $ACCOUNT_ID"
    else
        echo -e "${RED}✗ AWS credentials are not configured${NC}"
        echo "Please configure AWS credentials: aws configure"
        exit 1
    fi
fi

echo ""
echo "Validating Terraform configuration..."

# Initialize Terraform
echo "Running: terraform init"
if terraform init > /dev/null 2>&1; then
    echo -e "${GREEN}✓ Terraform initialized successfully${NC}"
else
    echo -e "${RED}✗ Terraform initialization failed${NC}"
    exit 1
fi

# Validate configuration
echo "Running: terraform validate"
if terraform validate > /dev/null 2>&1; then
    echo -e "${GREEN}✓ Terraform configuration is valid${NC}"
else
    echo -e "${RED}✗ Terraform validation failed${NC}"
    terraform validate
    exit 1
fi

# Format check
echo "Running: terraform fmt -check"
if terraform fmt -check > /dev/null 2>&1; then
    echo -e "${GREEN}✓ Terraform files are properly formatted${NC}"
else
    echo -e "${YELLOW}⚠ Some files need formatting${NC}"
    echo "Run: terraform fmt"
fi

echo ""
echo "Testing Team 1 configuration..."
if terraform plan -var-file="team1.tfvars" -out=/dev/null > /dev/null 2>&1; then
    echo -e "${GREEN}✓ Team 1 configuration plan succeeded${NC}"
else
    echo -e "${RED}✗ Team 1 configuration plan failed${NC}"
    terraform plan -var-file="team1.tfvars"
    exit 1
fi

echo ""
echo "Testing Team 2 configuration..."
if terraform plan -var-file="team2.tfvars" -out=/dev/null > /dev/null 2>&1; then
    echo -e "${GREEN}✓ Team 2 configuration plan succeeded${NC}"
else
    echo -e "${RED}✗ Team 2 configuration plan failed${NC}"
    terraform plan -var-file="team2.tfvars"
    exit 1
fi

echo ""
echo "=================================="
echo -e "${GREEN}All validation checks passed!${NC}"
echo "=================================="
echo ""
echo "Next steps:"
echo "1. Review the configuration files"
echo "2. Run: terraform apply -var-file=\"team1.tfvars\""
echo "3. Run: terraform apply -var-file=\"team2.tfvars\""
echo "4. Observe that Team 2 detects Team 1's resource"
echo ""

variables.tf

variable "aws_region" {
  description = "AWS region where resources will be created"
  type        = string
  default     = "us-east-1"
}

variable "project_name" {
  description = "Name of the project"
  type        = string
  default     = "shared-resource"
}

variable "environment" {
  description = "Environment name (e.g., dev, staging, prod)"
  type        = string
  default     = "dev"
}

variable "team_name" {
  description = "Name of the team running this Terraform code"
  type        = string
  
  validation {
    condition     = length(var.team_name) > 0
    error_message = "Team name must not be empty."
  }
}

variable "check_existing" {
  description = "Whether to check for existing resources with matching tags before creating"
  type        = bool
  default     = true
}

variable "enforce_precondition" {
  description = "Whether to enforce precondition checks (set to false for testing)"
  type        = bool
  default     = true
}

variable "required_tags" {
  description = "Tags that must match on existing resources for them to be considered the same"
  type        = map(string)
  default = {
    Project     = "shared-resource"
    Environment = "dev"
    Purpose     = "multi-team-example"
  }
  
  validation {
    condition     = length(var.required_tags) > 0
    error_message = "At least one required tag must be specified."
  }
}