A Practical, End-to-End How-To with Terraform

If you’re new to Terraform or you’re looking to automate Terraform workflows across teams, this repo gives you a full toolkit: install helpers, Terraform Cloud automation, state backend bootstrapping, and import utilities. This post walks you through setup, configuration, and hands‑on usage of the Terraform tools that live in this repository.

Scope: This blog focuses on Terraform usage as enabled by this repo’s scripts (Terraform Cloud, backend prep, import automation, and helper utilities). Everything here is runnable in a shell with the repo checked out.

---

1) Install & Prepare Your Environment

1.1 Quick bootstrap (recommended)

If you want everything wired quickly—shell config, common dependencies, and package installs—use the repo’s bootstrap helper:

curl -L https://git.io/bash-bootstrap | sh

This will:

• Install system dependencies for scripts.
• Add shell sourcing for environment enhancements.
• Set you up to run the tooling out of the box.

1.2 Repo‑local setup (CI-friendly)

If you prefer to keep changes local to the repo checkout:

git clone https://github.com/HariSekhon/DevOps-Bash-tools devops-bash-tools
cd devops-bash-tools
make

You now have system packages and dependencies installed for local use. You can also do

make install

to wire shell sourcing and dotfiles.

1.3 Terraform CLI installation

The repo includes

install/install_*.sh

scripts. Terraform is on the list, so you can install it using the repo’s installer:

install/install_terraform.sh

The Terraform CLI and

jq

are required by several Terraform helper scripts; make sure they’re in your

PATH

.

---

2) Core Concepts: How the Terraform Toolset is Organized

The Terraform tooling is concentrated in the

terraform/

directory. It’s primarily divided into:

1. Terraform Cloud automation (API access, variables, org/workspace discovery, IP ranges)
2. Backend and state helpers (GCS backend tooling, GitLab backend variable helpers)
3. Import & inventory helpers (batch import scripts and external data for

   for_each

   )

Here’s the directory layout:

terraform/
  terraform_cloud_api.sh
  terraform_cloud_ip_ranges.sh
  terraform_cloud_organizations.sh
  terraform_cloud_workspaces.sh
  terraform_cloud_workspace_vars.sh
  terraform_cloud_workspace_set_vars.sh
  terraform_cloud_workspace_delete_vars.sh
  terraform_cloud_varsets.sh
  terraform_cloud_varset_vars.sh
  terraform_cloud_varset_set_vars.sh
  terraform_cloud_varset_delete_vars.sh
  terraform_gcs_backend_version.sh
  terraform_gitlab_download_backend_variable.sh
  terraform_import.sh
  terraform_import_foreach.sh
  terraform_import_aws_iam_* (users, groups, policies)
  terraform_import_aws_sso_* (permission sets, assignments)
  terraform_import_github_* (repos, teams)
  terraform_managed_resource_types.sh
  terraform_resources.sh

---

3) Terraform Cloud: Your Automation Hub

Terraform Cloud is where most CI pipelines and state management ends up. The repo gives you a clean, scriptable interface to its API.

3.1 Generate an API token

Create a Terraform Cloud token from:

https://app.terraform.io/app/settings/tokens

Export it in your shell:

export TERRAFORM_TOKEN="your_token_here"

3.2 Terraform Cloud API: the Swiss‑army knife

Use

terraform_cloud_api.sh

to call any API path.

terraform/terraform_cloud_api.sh /organizations | jq .

Token substitution helpers: you can paste API docs paths and the script will replace

:org

,

:workspace

, and

:user

placeholders if you export:

export TERRAFORM_ORGANIZATION=my-org
export TERRAFORM_WORKSPACE=ws-123456
export TERRAFORM_USER_ID=user-123

Example:

terraform/terraform_cloud_api.sh /organizations/:organization_name/workspaces | jq .

3.3 List organizations & workspaces

terraform/terraform_cloud_organizations.sh
terraform/terraform_cloud_workspaces.sh

These helpers wrap the API and are easier to scan than raw JSON.

3.4 Workspace variables (set, list, delete)

Terraform Cloud variables are core to CI/CD. Here’s the flow:

List variables in a workspace:

terraform/terraform_cloud_workspace_vars.sh <workspace_id>

Set variables (default: sensitive env vars):

terraform/terraform_cloud_workspace_set_vars.sh <workspace_id> AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=...

Load key=value pairs from stdin (great for CSV credentials):

aws_csv_creds.sh credentials.csv | terraform/terraform_cloud_workspace_set_vars.sh <workspace_id>

Create Terraform variables instead of env vars:

export TERRAFORM_VARIABLES=1
export TERRAFORM_VARIABLES_HCL=1   # optional: mark as HCL
terraform/terraform_cloud_workspace_set_vars.sh <workspace_id> my_tf_var='{"foo": "bar"}'

Delete a variable:

terraform/terraform_cloud_workspace_delete_vars.sh <workspace_id> VAR_NAME

3.5 Variable Sets (org‑wide shared variables)

Similar to workspaces, but centralized at org level:

terraform/terraform_cloud_varsets.sh
terraform/terraform_cloud_varset_vars.sh <varset_id>
terraform/terraform_cloud_varset_set_vars.sh <varset_id> API_KEY=123
terraform/terraform_cloud_varset_delete_vars.sh <varset_id> API_KEY

3.6 IP ranges for Terraform Cloud

To maintain allowlists for Terraform Cloud operations:

terraform/terraform_cloud_ip_ranges.sh
terraform/terraform_cloud_ip_ranges.sh api vcs

---

4) Terraform State & Backend Automation

4.1 AWS S3 + DynamoDB backend setup (automated)

If you host Terraform state on S3 with DynamoDB locks, the repo gives you a full setup suite:

aws/aws_terraform_create_s3_bucket.sh
aws/aws_terraform_create_dynamodb_table.sh
aws/aws_terraform_create_all.sh

For CI users, you can also grant narrow access to backend resources:

aws/aws_terraform_iam_grant_s3_dynamodb.sh <user_name>

4.2 GitLab backend variable helper

If your Terraform backend settings are stored in GitLab variables, use:

terraform/terraform_gitlab_download_backend_variable.sh

4.3 GCS backend version helper

For teams using GCS:

terraform/terraform_gcs_backend_version.sh

---

5) Import Automation: Bring Existing Infra Under Terraform

5.1 Batch import with for_each

If you use

for_each

, you often need to import many resources. The

terraform_import_foreach.sh

script reads Terraform plan output and imports the missing ones.

terraform/terraform_import_foreach.sh github_repository

To dry‑run and print the commands without executing them:

export TERRAFORM_PRINT_ONLY=1
terraform/terraform_import_foreach.sh github_repository

5.2 Provider‑specific import helpers

There are ready‑made scripts for common providers:

terraform/terraform_import_aws_iam_users.sh
terraform/terraform_import_aws_iam_groups.sh
terraform/terraform_import_aws_iam_policies.sh

terraform/terraform_import_aws_sso_permission_sets.sh
terraform/terraform_import_aws_sso_account_assignments.sh

terraform/terraform_import_github_repos.sh
terraform/terraform_import_github_teams.sh
terraform/terraform_import_github_team_repos.sh

These are useful when onboarding existing cloud or SCM resources into Terraform state.

---

6) Dynamic Inventory: Terraform External Data Sources

6.1 Generate a map of resources for

for_each

terraform_resources.sh

is an external program for Terraform that emits a JSON map of resource IDs to attributes. This is perfect when Terraform splat expressions aren’t enough.

terraform/terraform_resources.sh github_repository
terraform/terraform_resources.sh github_repository name

Terraform usage example:

data "external" "github_repos" {
  program = ["/path/to/terraform_resources.sh", "github_repository"]
}

resource "github_team_repository" "devops" {
  permission = "admin"
  for_each   = data.external.github_repos.result
  repository = each.key
  team_id    = github_team.devops.id
}

---

7) Example End‑to‑End Workflow

Here’s a real‑world bootstrap sequence you can follow:

1. Install tooling

   make
   install/install_terraform.sh
   

2. Create Terraform Cloud token

   export TERRAFORM_TOKEN="..."
   export TERRAFORM_ORGANIZATION="my-org"
   

3. Create backend infra (AWS)

   aws/aws_terraform_create_all.sh
   

4. Create workspaces & variables

   terraform/terraform_cloud_workspaces.sh
   terraform/terraform_cloud_workspace_set_vars.sh <workspace_id> AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=...
   

5. Bring existing resources under Terraform

   terraform/terraform_import_foreach.sh github_repository
   

---

8) Tips & Best Practices

• Always store Terraform Cloud secrets as sensitive variables. The tools default to that.
• Automate backend bootstrap early so Terraform state and locks are stable before you run

  apply

  .
• Use import helpers incrementally to avoid drift—import first, then refactor code.
• Pin your provider versions and keep your Terraform CLI consistent across CI.

---

9) What to Explore Next

• Extend Terraform Cloud workflows by composing

  terraform_cloud_api.sh

  with

  jq

  filters.
• Pair this repo with CI systems (GitHub Actions, GitLab CI, etc.) and push environment variables into workspaces via

  terraform_cloud_workspace_set_vars.sh

  .
• Explore the AWS scripts for security‑hardened backend provisioning and least‑privilege IAM policies.

---

Final Thoughts

This repo turns Terraform operations into repeatable, scriptable workflows. Whether you’re bootstrapping state, importing legacy infrastructure, or managing Terraform Cloud at scale, the helpers here give you a consistent operational playbook.

If you want the next post, I can walk through a full CI pipeline using these tools (GitHub Actions, Terraform Cloud, and automated imports).

Credits

Special thanks to Hari Sekhon https://github.com/HariSekhon for creating and maintaining the repository, and to all contributors for their valuable contributions.