Using Terraform with AWS

These are the slides from my meetup talk at http://www.meetup.com/AWS-Munich/events/227474245/.

Materials are here: https://github.com/sethvargo/aws-munich-meetup-demo.

Transcript

1. AWS Meetup Munich Using Terraform with AWS
2. Seth Vargo Director of Evangelism, HashiCorp @sethvargo
3. Agenda 1. Introduction to Terraform 2. Infrastructure as Code 3. Variables, outputs, and meta-parameters 4. State 5. Modules
4. Introduction
5. Terraform Provisions Infrastructure Resources
6. Terraform's Goals Unify the view of resources using infrastructure as code Support the modern data center (IaaS, PaaS, SaaS) Expose a way to safely and predictably change infrastructure Provide a workflow that is technology agnostic
7. Terraform vs. Other Tools Provides a high-level abstraction of infrastructure (IaC) Allows for composition and combination Supports parallel management of resources (graph, fast) Separates planning from execution (dry-run)
8. Glossary
9. Glossary Provider A provider is an abstraction of the API/service provider such as AWS, GCP, DNSimple, or Fastly. Providers typically require some sort of configuration data such as an API key or credential file.
10. Glossary Resource A resource represents a component of a provider such as an "AWS instance", "DNSimple Record", or "Fastly service". Resources have both arguments (inputs) and attributes (outputs) which are specific to the resource. Resources also have meta-parameters such as count and lifecycle.
11. Glossary (Resource) Argument An argument is an input or configuration option to a resource. An AWS EC2 instance accepts ami as an input parameter. This makes ami an argument to the aws_instance resource.
12. Glossary (Resource) Attribute An attribute is an output or computed value available only after resource creation. An AWS EC2 instance provides public_ip as an output parameter. This makes public_ip an attribute to the aws_instance resource. This makes sense, because an instance's IP address is assigned during creation.
13. Glossary Graph The graph is the internal structure for Terraform's resource dependencies and order. The graph implements a directed acyclic graph (DAG) which allows Terraform to optimize for parallelism while adhering to dependency ordering. It is possible to generate the graph as a DOT file for human viewing.
14. Glossary (Remote/Local) State Terraform stores the last-known arguments and attributes for all resources. These contents known as "state" can be stored locally as a JSON file (local state) or stored in a remote shared location like Atlas (remote state).
15. Glossary Module A module is a self-contained package of Terraform configurations. Modules are like abstract classes that are imported into other Terraform configurations. Parallels: Chef Cookbook, Puppet Module, Ruby gem
16. Glossary Variable A variable is a user or machine-supplied input in Terraform configurations. Variables can be supplied via environment variables, CLI flags, or variable files. Combined with modules, variables help make Terraform flexible, sharable, and extensible.
17. Glossary Interpolation Terraform includes a built-in syntax for referencing attributes of other resources. This technique is called interpolation. Terraform also provides built-in functions for performing string manipulations, evaluating math operations, and doing list comprehensions.
18. Glossary HashiCorp Configuration Language (HCL) Terraform's syntax and interpolation are part of an open source language and specification called HCL.
19. Infrastructure as Code
20. Infrastructure as Code Provide a codified workflow to create infrastructure Expose a workflow for managing updates to existing infrastructure Integrate with application code workflows (Git, SCM, Code Review) Provide modular, sharable components for separation of concerns
21. Infrastructure as Code (Terraform) Human-readable configuration (HCL) is designed for human consumption so users can quickly interpret and understand their infrastructure configuration. HCL is fully JSON-compatible for machine-generated configurations.
22. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } main.tf
23. Infrastructure as Code (Terraform) Configuration format is very VCS friendly with support for multi- line lists, trailing commas, and auto-formatting.
24. resource "aws_instance" "web" { ami = "ami-9a562df2" - instance_type = "t2.micro" + instance_type = "m1.small" } main.tf
25. Infrastructure as Code (Terraform) Reference values from other resources, building the implicit dependency graph.
26. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } resource "dnsimple_record" "web" { domain = "hashicorp.com" name = "web" ttl = "3600" type = "A" value = "${aws_instance.web.public_ip}" } main.tf
27. Infrastructure as Code (Terraform) Configuration can be in a single file or split across multiple files. Terraform will merge all files in the current working directory which end in .tf or .tf.json.
28. Terminal $ ls main.tf outputs.tf variables.tf
29. Terminal $ ls instances.tf load-balancers.tf shared.tf
30. Terminal $ ls everything.tf
31. Configuration Format
32. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } main.tf
33. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } main.tf resource
34. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } main.tf "aws_instance"
35. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } main.tf "web"
36. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } resource "aws_instance" "web" { ami = "ami-6b563df1" instance_type = "t2.micro" } main.tf "web"
37. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } resource "aws_instance" "web-2" { ami = "ami-6b563df1" instance_type = "t2.micro" } main.tf "web"
38. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } resource "digitalocean_droplet" "web" { ami = "ami-6b563df1" instance_type = "t2.micro" } main.tf "web"
39. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } main.tf ami instance_type
40. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } main.tf
41. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } resource "dnsimple_record" "web" { domain = "hashicorp.com" name = "web" ttl = "3600" type = "A" value = "${aws_instance.web.public_ip}" } "${aws_instance.web.public_ip}" main.tf
42. resource "aws_instance" "web" { ami = "ami-9a562df2" instance_type = "t2.micro" } resource "dnsimple_record" "web" { domain = "hashicorp.com" name = "web" ttl = "3600" type = "A" value = "${aws_instance.web.public_ip}" } # This is a comment main.tf // This is a comment as well
43. Syntax Highlighting Plugins for HCL exist for most major editors If not: - "javascript" if using // comments
 - "ruby" if using # comments Community leans toward # comments
44. Resource Graph
45. Resource Graph The resource graph is an internal representation of all resources and their dependencies. A human-readable graph can be generated using the
 terraform graph command. Can optionally draw cycles (advanced).
46. Demo: Run terraform graph Run the terraform graph command against the sample Terraform configurations.
47. Terraform Graph Useful for visualizing infrastructure and dependencies Builds upon existing visualization technologies and open formats such as DOT
48. Command Line Interface
49. Command Line Interface All interactions with Terraform occur via the CLI. Terraform is a local tool (runs on the current machine).
50. Terminal $ terraform help Available commands are: apply Builds or changes infrastructure destroy Destroy Terraform-managed infrastructure fmt Rewrites config files to canonical format get Download and install modules for the configuration graph Create a visual graph of Terraform resources init Initializes Terraform configuration from a module output Read an output from a state file plan Generate and show an execution plan push Upload this Terraform module to Atlas to run refresh Update local state file against real resources remote Configure remote state storage show Inspect Terraform state or plan taint Manually mark a resource for recreation untaint Manually unmark a resource as tainted validate Validates the Terraform files version Prints the Terraform version
51. Terminal $ terraform help Available commands are: apply Builds or changes infrastructure destroy Destroy Terraform-managed infrastructure fmt Rewrites config files to canonical format get Download and install modules for the configuration graph Create a visual graph of Terraform resources init Initializes Terraform configuration from a module output Read an output from a state file plan Generate and show an execution plan push Upload this Terraform module to Atlas to run refresh Update local state file against real resources remote Configure remote state storage show Inspect Terraform state or plan taint Manually mark a resource for recreation untaint Manually unmark a resource as tainted validate Validates the Terraform files version Prints the Terraform version
52. Command: terraform plan The plan shows you what will happen You can save plans to guarantee what will happen Plans show reasons for certain actions (such as re-create) Prior to Terraform, users had to guess change ordering, parallelization, and rollout effect
53. Demo: Run terraform plan
54. Command: terraform apply Executes changes in order based on the resource graph Parallelizes changes when possible Handles and recovers transient errors
55. Demo: Run terraform apply
56. Command: terraform apply Updates existing resources when updates are allowed Re-creates existing resources when updates are not allowed
57. Demo: In-place Upate Edit the main Terraform file an add tags to the AWS instance.
58. Variables, Outputs, & Meta Parameters
59. Variables Define the parameterization of Terraform configurations Can have defaults, be provided with a variables file, asked for at execution, or overridden via the CLI Values can be strings or maps Must be defined before used
60. variable "aws_access_key" {} provider "aws" { access_key = "..." secret_key = "..." region = "us-east-1" } resource "aws_instance" "web" { ami = "ami-50759d3d" instance_type = "t2.micro" } main.tf
61. Variables Variables are available in the interpolation syntax.
62. variable "aws_access_key" {} variable "aws_secret_key" {} variable "aws_region" { default = "us-east-1" } provider "aws" { access_key = "${var.aws_access_key}" secret_key = "..." region = "us-east-1" } resource "aws_instance" "web" { ami = "ami-50759d3d" instance_type = "t2.micro" } main.tf
63. Demo: Show tfvars file Reads from tfvars by default Prompts for input if not provided
64. Outputs define values that will be highlighted to the user when Terraform applies. Outputs can be queried using the terraform output command. Outputs
65. variable "aws_access_key" {} provider "aws" { access_key = "..." secret_key = "..." region = "us-east-1" } resource "aws_instance" "web" { ami = "ami-50759d3d" instance_type = "t2.micro" } output "public_ip" { value = "${aws_instance.web.public_ip}" } main.tf
66. Demo: Query Output Use the terraform output command to for all outputs. Use the terraform output command to query for a single output.
67. Meta-Parameters Meta-parameters allow for higher-level control flow and lifecycle management in Terraform.
68. Meta-Parameters Count The count attribute allows for N number of identical resources to be created. This removes the need for complex variable iteration with "for" or "while" loops.
69. Meta-Parameters Depends On The depends_on attribute allows for declaration of explicit dependencies. This is useful where interpolation is not required, but explicit ordering is desired.
70. Meta-Parameters Lifecycle The lifecycle attribute allow explicit configuration of resource lifecycle such as preventing destruction or ignoring property chanegs. This is an advanced option and is not recommended for users who are getting started with Terraform.
71. Demo: Change count Increase the count parameter on the AWS instance to two (5). Run terraform plan and verify the output. Run terraform apply to apply the changes.
72. State
73. Terraform stores the state of your managed infrastructure from the last time Terraform was run. Terraform uses this state to create plans and make changes to your infrastructure. It is critical that this state is maintained appropriately so future runs operate as expected. State
74. Terminal $ cat terraform.tfstate { "version": 1, "serial": 14, "modules": [ { "path": [ "root" ], "outputs": { "public_ip": "54.164.13.128" }, "resources": { "aws_instance.example": { "type": "aws_instance", "primary": { "id": "i-d829670c", "attributes": {
75. Local State State is stored locally on one machine in JSON format Generally acceptable for individuals and small teams Does not scale to large teams Resolving JSON git-diffs is hard
76. Remote State State is stored on a shared remote source such as Atlas or Consul Remote storage is responsible for handling merging and locking Unnecessary overhead for small teams Best-suited for large or distributed teams
77. Transitioning from Local to Remote Transitioning is a one-time operation After configured, Terraform will no longer store local state
78. Modules
79. Modules Portable Terraform configurations (packages) Allow separation of concerns and responsibilities among teams Parallels: Chef Cookbook, Puppet Module, Ruby gem
80. Modules Spoiler Alert! Modules are just Terraform configurations inside a folder - there's nothing special about them.
81. Modules Variables => Arguments Outputs => Attributes Configuration is sandbox/blackbox
82. Demo: Consul Module Suppose I want an entire Consul cluster, but I know nothing about best practices for setting up Consul.
83. Questions & More Information Terraform Guides & Docs
 terraform.io Terraform OSS (Golang)
 github.com/hashicorp/terraform Best Practices Repo
 github.com/hashicorp/best-practices