AWS Fundamentals - Part 10: Using the AWS CLI20 Feb 2019
Way back in part 3, I briefly discussed the AWS CLI as a way to interact with the AWS platform. In this post, I want to spend a bit more time looking at the CLI, as once you start to become more comfortable with AWS, it’s likely that you’ll find the console a bit limiting - mastering the CLI will ultimately make it easier and faster for you when using the platform.
Before I go any further, the first thing you need to do is actually install the CLI - the documentation explaining how to install the CLI on your local machine is available here. It’s easy to follow so I won’t duplicate it here.
Once you have the CLI installed, you’ll need to configure it with credentials to talk to your AWS account, as well as specifying the region to use. If you’re using your local machine with the CLI installed, have your access key ID and secret access key associated with your AWS account to hand. To configure credentials, run
Once you’ve configured this, you should be able to access your AWS account via the CLI. Note that the config and credentials are stored in a couple of files in the ‘~/.aws’ directory:
If I take a look at my credentials file, I see the following (actual credentials removed!!):
What are the different sections for (default, eks-test-2, nocreds)? These are profiles that I have set up. Each of these profiles contains a different set of credentials (the ‘nocreds’ profile doesn’t contain any credentials at all due to some testing I was doing). To use these profiles, you can set an environment variable that specifies the profile - export AWS_DEFAULT_PROFILE=nocreds. Alternatively, you can specify the profile on each command that you run, for example:
As you can see, I’ve specified the ‘nocreds’ profile in the above S3 command, which - as you would expect - returns an error as that profile doesn’t contain any credentials.
As I mentioned in part 3, AWS CLI commands generally take the following form: firstly, the ‘aws’ command, followed by the name of the service you want to interact with (e.g. ec2, s3, etc) and then the specific action you want to take. Some examples are as follows:
aws s3 ls: lists all S3 buckets in your account
aws ec2 describe-instances: describes every EC2 instance in your account
aws ec2 start-instances –instance-ids i-079db50f95285d315: starts the instance with the instance ID specified
Changing the Output Format
By default, the AWS CLI outputs everything in JSON format. Using JSON as the output makes it easy to filter using the –query parameter (desrcibed in the next section) or using other tools such as jq. However, there might be times when you want more of a ‘human readable’ format, such as a table or just plain text.
Specifying –output=table after the CLI command returns the output as an ASCII table, for example:
Alternatively, you could use –output=text to return the information as plain text:
You might find it easier to use the text output type if using Linux tools such as grep, sed, awk and so on.
Using –query to Filter Output
Taking the example in the section above - aws ec2 describe-instances, this potentially returns a lot of data, especially if you are running a lot of EC2 instances in your account. It’s highly likely that you are looking for more specific information - for example, you might be looking for the private IP address associated with one specific instance. Fortunately, this is easy to achieve through the use of CLI queries.
Query strings use the JMESpath query specification. To demonstrate how it works, let’s look at some examples.
Firstly, let’s say I want to find out what the instance type is for each of my EC2 instances. I can get this information using the following command:
Let’s look at how that query works. Firstly, the ‘Reservations.Instances’ causes the query to iterate over all the instances in the list. The second half of the query (‘[InstanceId,InstanceType]’) then returns only the two elements (InstanceID and InstanceType) from each instance. Finally, I specified that I wanted the output to be in table format. I can make the table even more friendly:
This time, I tweaked the command so that the table would have the headings ‘Instance’ and ‘Type’ to make it a bit more readable. Here’s another example - what if I wanted to find the private IP address of a specific instance? Here’s how I could do it:
In this example, I’ve searched for a specific instance ID in the first half of the query (using ?InstanceId==
i-0f77eeba0d2835444) and then in the second half of the query I’ve asked for only the ‘PrivateIpAddress’ element to be returned.
Generating a CLI Skeleton
The AWS CLI has the ability to accept input from a file containing all of the parameters required, using the option –cli-input-json. However, it can be difficult to know which parameters you need and what they are called. To help with that, the CLI provides a way to generate a ‘skeleton’ set of parameters for a given command, which then allows you to fill it in and use it as input via the –cli-input-json option.
Here’s an example: I want to create a new VPC using the CLI - to generate the CLI skeleton, I can do the following:
Now, all I have to do is create a file containing the information in the skeleton with the values filled in (and with ‘DryRun’ set to false), and then refer to that file in the command:
Ok, that hopefully gives you a decent overview of the AWS CLI. Thanks for reading!