ARC332: Inventory and Patch Management Using AWS Systems Manager
In the cloud, you can apply the same engineering discipline that you use for application code to your entire environment. You can define your entire workload (applications, infrastructure, etc.) as code and update it with code. You can script your operations procedures and automate their execution by triggering them in response to events. By performing operations as code, you limit human error and enable consistent execution of operations activities.
In this lab you will apply the concepts of Infrastructure as Code and Operations as Code to the following activities:
- Deployment of Infrastructure
- Estate Management
- Patch Management
Included in the lab guide are bonus sections that can be completed if you have time or later if interested.
- Creating Maintenance Windows and Scheduling Automated Operations Activities
- Create and Subscribe to a Simple Notification Service Topic
Note: At the end of the lab guide there is an additional section on how to remove all resources you created. Remove the resources you created. Otherwise you will be charged for any resources that are not covered in the AWS Free Tier.
1. Setup
Requirements
You will need the following to be able to perform this lab:
- Your own device for console access
- An AWS account that you are able to use for testing, that is not used for production or other purposes
- An available region within your account with capacity to add 2 additional VPCs
NOTE: You will be billed for any applicable AWS resources used if you complete this lab that are not covered in the AWS Free Tier.
User and Group Management
When you first create an Amazon Web Services (AWS) account, you begin with a single sign-in identity that has complete access to all AWS services and resources in the account. This identity is called the AWS account root user and is accessed by signing in with the email address and password that you used to create the account.
We strongly recommend that you do not use the root user for your everyday tasks, even the administrative ones. Instead, adhere to the best practice of using the root user only to create your first IAM user. Then securely lock away the root user credentials and use them to perform only a few account and service management tasks. To view the tasks that require you to sign in as the root user, see AWS Tasks That Require Root User.
IAM Users & Groups
As a best practice, do not use the AWS account root user for any task where it's not required. Instead, create a new IAM user for each person that requires administrator access. Then make those users administrators by placing the users into an "Administrators" group to which you attach the AdministratorAccess managed policy.
Thereafter, the users in the administrators group should set up the groups, users, and so on, for the AWS account. All future interaction should be through the AWS account's users and their own keys instead of the root user.
1.1 Create Administrator IAM User and Group
To create an administrator user for yourself and add the user to an administrators group:
- Use your AWS account email address and password to sign in as the AWS account root user to the IAM console at https://console.aws.amazon.com/iam/.
- In the IAM navigation pane, choose Users and then choose Add user.
- In Set user details for User name, type a user name for the administrator account you are creating. The name can consist of letters, digits, and the following characters: plus (+), equal (=), comma (,), period (.), at (@), underscore (_), and hyphen (-). The name is not case sensitive and can be a maximum of 64 characters in length.
- In Select AWS access type for Access type, select the check box next to AWS Management Console access, select Custom password, and then type your new password in the text box. If you're creating the user for someone other than yourself, you can leave Require password reset selected to force the user to create a new password when first signing in. Clear the box next to Require password reset and then choose Next: Permissions.
- In set permissions for user ensure Add user to group is selected.
- Under Add user to group choose Create group.
- In the Create group dialog box, type a Group name for the new group, such as Administrators. The name can consist of letters, digits, and the following characters: plus (+), equal (=), comma (,), period (.), at (@), underscore (_), and hyphen (-). The name is not case sensitive and can be a maximum of 128 characters in length. In the policy list, select the check box next to AdministratorAccess and then choose Create group.
- Back at Add user to group, in the list of groups, ensure the check box for your new group is selected. Choose Refresh if necessary to see the group in the list. choose Next: Review to see the list of group memberships to be added to the new user. When you are ready to proceed, choose Create user.
- At the confirmation screen you do not need to download the user credentials for programmatic access at this time. You can create new credentials at any time.
You can use this same process to create more groups and users and to give your users access to your AWS account resources. To learn about using policies that restrict user permissions to specific AWS resources, see Access Management and Example Policies. To add additional users to the group after it's created, see Adding and Removing Users in an IAM Group.
1.2 Log in to the AWS Management Console using your administrator account
- You can now use this administrator user instead of your root user for this AWS account. Choose the link https://<yourAccountNumber>.signin.aws.amazon.com/console and log in with your administrator user credentials.
-
Select the region you will use for the lab from the the list in the upper right corner.
- Verify that you have 2 available VPCs (3 or less in use) in the selected region by navigating to the VPC Console (https://console.aws.amazon.com/vpc/) and in the Resources section reviewing the number of VPCs.
1.3 Create an EC2 Key Pair
Amazon EC2 uses public-key cryptography to encrypt and decrypt login information. Public-key cryptography uses a public key to encrypt a piece of data, such as a password, then the recipient uses the private key to decrypt the data. The public and private keys are known as a key pair. To log in to the Amazon Linux instances we will create in this lab, you must create a key pair, specify the name of the key pair when you launch the instance, and provide the private key when you connect to the instance.
- Use your administrator account to access the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
- In the IAM navigation pane under Network & Security, choose Key Pairs and then choose Create Key Pair.
- In the Create Key Pair dialog box, type a Key pair name such as
OELab2018
and then choose Create.
-
Save the
keyPairName.pem
file for optional later use accessing the EC2 instances created in this lab.
2. Deploy an Environment Using Infrastructure as Code
Tagging
We will make extensive use of tagging throughout the lab. The CloudFormation template for the lab includes the definition of multiple tags against a variety of resources.
AWS enables you to assign metadata to your AWS resources in the form of tags. Each tag is a simple label consisting of a customer-defined key and an optional value that can make it easier to manage, search for, and filter resources. Although there are no inherent types of tags, commonly adopted categories of tags include technical tags (e.g., Environment, Workload, InstanceRole, and Name), tags for automation (e.g., Patch Group, and SSMManaged), business tags (e.g., Owner), and security tags (e.g., Confidentiality).
Apply the following best practices when using tags:
- Always use a standardized, case-sensitive format for tags, and implement it consistently across all resource types
- Consider tag dimensions that support the following:
- Managing resource access control with IAM
- Cost tracking
- Automation
- AWS console organization
- Implement automated tools to help manage resource tags. The Resource Groups Tagging API enables programmatic control of tags, making it easier to automatically manage, search, and filter tags and resources.
- Err on the side of using too many tags rather than too few tags.
-
Develop a tagging strategy.
Remember that it is easy to modify tags to accommodate changing business requirements; however, consider the ramifications of future changes, especially in relation to tag-based access control, automation, or upstream billing reports.
Note: Patch Group is a reserved tag key used by Systems Manager Patch Manager that is case sensitive with a space between the two words.
AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS. You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation provisions and configures those resources for you. AWS CloudFormation enables you to use a template file to create and delete a collection of resources together as a single unit (a stack).
There is no additional charge for AWS CloudFormation. You pay for AWS resources (such as Amazon EC2 instances, Elastic Load Balancing load balancers, etc.) created using AWS CloudFormation in the same manner as if you created the resources manually. You only pay for what you use, as you use it; there are no minimum fees and no required upfront commitments.
2.1 Deploy the Lab Infrastructure
- While logged into your administrator account click on this link to launch the cloud formation script that will install the lab.
- It will take you to the CloudFormation console Create stack window and prepopulate the path to the CloudFormation template in the Specify an Amazon S3 template URL text box.
CloudFormation Designer
AWS CloudFormation Designer is a graphic tool for creating, viewing, and modifying AWS CloudFormation templates. With Designer, you can diagram your template resources using a drag-and-drop interface, and then edit their details using the integrated JSON and YAML editor. AWS CloudFormation Designer can help you quickly see the interrelationship between a template's resources and easily modify templates.
- On the Select Template page next to Specify an Amazon S3 template URL, choose the link to View/Edit template in Designer.
- Take a moment to look at the graphical representation of the environment we are about to create, and the template in the JSON and YAML formats. The template was written in JSON and this is a method to convert it to YAML if that is your preference.
-
Choose the Create Stack icon (a cloud with an arrow) to return to the Select Template page.
- On the Select Template page, choose Next.
A CloudFormation template is a JSON or YAML formatted text file that describes your AWS infrastructure containing both optional and required sections. In the next steps, we will provide a name for our stack and parameters that will be passed into the template to help define the resources that will be implemented.
- In the Specify Details section, define a Stack name, such as
OELabStack1
.
- In the Parameters section:
- Leave InstanceProfile blank as we have not yet defined an instance profile.
- Leave InstanceTypeApp and InstanceTypeWeb as the default free-tier-eligible t2.micro value.
- Select the EC2 KeyName you defined earlier from the list.
- In a browser window, go to http://checkip.amazonaws.com/ to get your IP and then enter your IP address in SSHLocation in CIDR notation (i.e., ending in /32).
- Define the Workload Name as
Test
. - Choose Next.
- On the Options page under Tags, define a Key of Owner, with Value set to the username you choose for your administrator. You can optionally define additional keys. The CloudFormation template will create all the example tags given in the discussion on tagging above.
- Leave all other sections unmodified. Scroll to the bottom of the page and choose Next.
- On the Review page, review your choices and then choose Create.
- On the CloudFormation console page, click the refresh button (circular arrow) in the top right if your stack name is not displayed.
-
Check the box next to your Stack Name to see its details. You may need to choose the refresh button again to get details.
- Choose the Events tab for your selected workload to see the activity log from the creation of your CloudFormation stack.
When the Status of your stack in filter list is CREATE_COMPLETE you will have just created a representation of a typical lift and shift 2-tier application migrated to the cloud.
- Navigate to the EC2 console to view the deployed systems.
- Choose Instances.
- Select a server and review the details under its Description and Tag tabs.
- Optional Exercise: choose Security Groups and select the Security Group whose name begins with the name of your stack. Examine the inbound rules.
- Optional Exercise: Navigate to the VPC console and examine the configuration of the VPC you just created.
The impact of Infrastructure as Code
With infrastructure as code, if you can deploy one environment, you can deploy any number of copies of that environment. In this example we have created a Test
environment. Later, we will repeat these steps to deploy a Prod
environment. The ability to on-demand, dynamically deploy temporary environments enables parallel experimentation, development, and testing efforts; duplication of environments to recreate and analyze errors; and the cut-over deployment of production systems using blue-green methodologies. These all contribute to reduced risk and increased operations effectiveness and efficiency.
3. Estate Management using Operations as Code
AWS Systems Manager is a collection of features that enable IT Operations in the cloud that we will explore throughout this lab.
There are set up tasks and pre-requisites that must be satisfied prior to using Systems Manager to manage your EC2 instances or on-premises systems in hybrid environments.
- You must be using a supported operating system. Windows, Amazon Linux, Ubuntu Server, RHEL, and CentOS all have supported versions.
- The SSM Agent must be installed. The SSM Agent for Windows also requires PowerShell 3.0 or later for some capabilities.
- Your EC2 instances must have outbound internet access or you must configure a VPC endpoint for SSM
- You must access Systems Manager in a supported region.
- Systems Manager requires an IAM role for instances that will process commands and a separate role for users executing commands.
SSM Agent is installed, by default, on Amazon Linux base AMIs dated 2017.09 and later. The SSM Agent is installed by default on Windows Server 2016 instances and instances created from Windows Server 2003-2012 R2 AMIs published in November 2016 or later.
There is no additional charge for AWS Systems Manager. You only pay for your underlying AWS resources managed or created by AWS Systems Manager (e.g., Amazon EC2 instances or Amazon CloudWatch metrics). You only pay for what you use, as you use it; there are no minimum fees and no upfront commitments.
3.1 Setting up Systems Manager
- Use your administrator account to access the Systems Manager console at https://console.aws.amazon.com/systems-manager/.
- Choose Managed Instances from the navigation bar. If you have not satisfied the pre-requisites for Systems Manager, you will arrive at the AWS Systems Manager Managed Instances page.
- Navigate to the IAM console to create an Instance Profile for Systems Manager managed instances.
- In the navigation pane, choose Roles, and then choose Create role.
- In the Select type of trusted entity section, verify that the default AWS service is selected.
- In the Choose the service that will use this role section, choose EC2 within the field of services to select it. This will open the Select your use case section.
- In the Select your use case section, choose EC2 Role for Simple Systems Manager to select it
- Then choose Next: Permissions.
- Under Attached permissions policy, verify that AmazonEC2RoleforSSM is listed, and then choose Next: Review.
- In the Review section:
- Enter a Role name, such as
ManagedInstancesRole
- Accept the default in the Role description.
- Choose Create role.
- You must apply this role to the instances you wish to manage with Systems Manager.
- Navigate to the EC2 Console and choose Instances.
- Select the first instance and then choose Actions, Instance Settings, and Attach/Replace IAM Role.
- Under Attach/Replace IAM Role, select ManagedInstancesRole from the drop down list and choose Apply.
- After you receive confirmation of success, choose Close.
- Repeat this process, assigning ManagedInstancesRole to each of the 3 remaining instances.
- Return to the Systems Manager console and choose Managed Instances from the navigation bar. Periodically choose Managed Instances until your instances begin to appear in the list. Over the next couple of minutes your instances will be populated into the list as managed instances.
- Click on this link to create a second VPC with the following changes.
- In the Specify Details section,
- define a Stack name, such as
OELabStack2
.
- In the Parameters section:
- Specify the InstanceProfile using the
ManagedInstancesRole
you defined. - Leave InstanceTypeApp and InstanceTypeWeb as the default free-tier-eligible t2.micro value.
- Select the EC2 KeyName you defined earlier from the list.
- In a browser window, go to http://checkip.amazonaws.com/ to get your IP and then enter your IP address in SSHLocation in CIDR notation (i.e., ending in /32).
- Define the Workload Name as
Prod
. - Choose Next.
- On the Options page under Tags, define a Key of Owner, with Value set to the username you choose for your administrator.
- Leave all other sections unmodified. Scroll to the bottom of the page and choose Next.
- On the Review page, review your choices and then choose Create.
https://s3-us-west-2.amazonaws.com/aws-well-architected-labs/Operations/OE_Single_VPC_2-Tier_Application_Lab.json
https://s3-us-west-2.amazonaws.com/aws-well-architected-labs/Operations/OE_Single_VPC+_2-Tier_Application_Lab.json
Systems Manager: Inventory
You can use AWS Systems Manager Inventory to collect operating system (OS), application, and instance metadata from your Amazon EC2 instances and your on-premises servers or virtual machines (VMs) in your hybrid environment. You can query the metadata to quickly understand which instances are running the software and configurations required by your software policy, and which instances need to be updated.
3.3 Using Systems Manager Inventory to Track Your Instances
- Under Insights in the AWS Systems Manager navigation bar, choose Inventory.
- Scroll down in the window to the Corresponding managed instances section. Inventory currently contains only the instance data available from the EC2.
- Choose the InstanceID of one of your systems.
- Examine each of the available tabs of data under the Instance ID heading.
- Inventory collection must be specifically configured and the data types to be collected must be specified.
- Choose Inventory in the navigation bar.
- Choose Setup Inventory in the top left corner of the window.
- In the Setup Inventory screen, you can define targets for inventory. You can select all managed instances in this account, ensuring that all managed instances will be inventoried. You can constrain inventoried instances to those with specific tags, such as Environment or Workload. Or you can manually select specific instances for inventory.
- Under Specify targets by, select Specifying a tag.
- Under Tags specify
Environment
for the key and OELab2018
for the value. - Alternatively you can select Selecting all managed instances in this account.
- You can schedule the frequency with which inventory is collected. The default and minimum period is 30 minutes.
- For Collect inventory data every, accept the default 30 Minute(s).
- Note: depending on when the instances from your second VPC are launched your inventory collection may include them. If they are not present in your inventory after it completes they will be picked up in 30 minutes during the next collection.
- Under parameters, you can specify what information to collect with the inventory process.
- Review the options and select the defaults.
-
Optional Exercise: If desired, you can check the box next to Sync inventory execution logs to an S3 bucket under the Advanced options and provide an S3 bucket name, and optional S3 bucket prefix, for the inventory execution logs. You will need to create a bucket prior to proceeding.
- Choose Setup Inventory.
- Note: it can take up to 10 minutes to deploy a new inventory policy to an instance.
- You can create multiple Inventory specifications. They will each be stored as associations within Systems Manager State Manager.
- To create a new inventory policy, from Inventory, choose Setup inventory.
- To edit an existing policy, from State Manager in the left navigation menu, select the association and choose Edit.
Systems Manager: State Manager
In State Manager, an association is the result of binding configuration information that defines the state you want your instances to be in to the instances themselves. This information specifies when and how you want instance-related operations to run that ensure your Amazon EC2 and hybrid infrastructure is in an intended or consistent state.
An association defines the state you want to apply to a set of targets. An association includes three components:
- a document that defines the state
- target(s)
- a schedule
Note: optionally you can also specify runtime parameters.
When you performed the Setup Inventory actions, you created an association in State Manager.
3.4 Review Association Status
- Under Actions in the navigation bar, select State Manager. At this point in time the Status may reflect the fact that the inventory activity has not yet completed.
- Choose the single Association id that is the result of your Setup Inventory action.
- Examine each of the available tabs of data under the Association ID heading.
- Choose Edit.
- Enter a name under Name - optional to provide a more user friendly label to the association, such as
InventoryAllInstances
. Note that white space is not permitted in an association name.
Inventory is accomplished through the activities defined in the AWS-GatherSoftwareInventory command document.
The parameters provided in the Parameters section are passed to the document at execution. The targets are defined in the Targets section. In this example there is a single target, which is a wildcard that matches all instances. The schedule for this activity is defined under Specify schedule and Specify with to use a CRON/Rate expression on a 30 minute interval. Finally there is the option to specify Output options. If you change the command document, the Parameters section will change to be appropriate to that command document.
- Navigate to Managed Instances under Shared Resources in the navigation bar.
- Note that an Association Status has been established for the inventoried instances under management.
- Choose one of the Instance ID links to go to the inventory of the instance. Note that the Inventory tab is now populated and you can track associations and their last activity under the Associations tab.
- Navigate to Compliance under Insights in the navigation bar. Here you can view the overall compliance status of your managed instances in the Compliance Summary and the individual compliance status of systems in the Corresponding managed instances section below.
Note: The inventory activity can take up to 10 minutes to complete. While waiting for it to complete you can proceed with the next section.
Systems Manager: Compliance
You can use AWS Systems Manager Configuration Compliance to scan your fleet of managed instances for patch compliance and configuration inconsistencies. You can collect and aggregate data from multiple AWS accounts and Regions, and then drill down into specific resources that aren’t compliant. By default, Configuration Compliance displays compliance data about Systems Manager Patch Manager patching and Systems Manager State Manager associations. You can also customize the service and create your own compliance types based on your IT or business requirements. You can also port data to Amazon Athena and Amazon QuickSight to generate fleet-wide reports.
4. Patch Management
Systems Manager: Patch Manager
AWS Systems Manager Patch Manager automates the process of patching managed instances with security related updates. For Linux-based instances, you can also install patches for non-security updates. You can patch fleets of Amazon EC2 instances or your on-premises servers and virtual machines (VMs) by operating system type. This includes supported versions of Windows, Ubuntu Server, Red Hat Enterprise Linux (RHEL), SUSE Linux Enterprise Server (SLES), and Amazon Linux. You can scan instances to see only a report of missing patches, or you can scan and automatically install all missing patches. You can target instances individually or in large groups by using Amazon EC2 tags.
Important Notes:
-
AWS does not test patches for Windows or Linux before making them available in Patch Manager .
-
If any updates are installed by Patch Manager the patched instance is rebooted.
-
Always test patches thoroughly before deploying to production environments.
Patch Baselines
Patch Manager uses patch baselines, which include rules for auto-approving patches within days of their release, as well as a list of approved and rejected patches. Later in this lab we will schedule patching to occur on a regular basis using a Systems Manager Maintenance Window task. Patch Manager integrates with AWS Identity and Access Management (IAM), AWS CloudTrail, and Amazon CloudWatch Events to provide a secure patching experience that includes event notifications and the ability to audit usage. Note: The operating systems supported by Patch Manager may vary from those supported by the SSM Agent.
4.1 Create a Patch Baseline
- Under Actions in the AWS Systems Manager navigation bar, choose Patch Manager.
- Choose the default patch baselines link in the Patch you Instances box.
- Choose Create patch baseline
- On the Create patch baseline page in the Provide patch baseline details section:
- Enter a Name for your custom patch baseline, such as
AmazonLinuxSecAndNonSecBaseline
- Optionally enter a description, such as
Amazon Linux patch baseline including security and non-security patches
. - Select Amazon Linux from the list.
- In the Approval rules section:
- Examine the options in the lists but leave Product, Classification, and Severity at their default of All.
- Leave the Auto approval delay at its default of 0 days.
- Change the value of Compliance level - optional to Critical.
- Choose Add another rule.
- In the new rule, change the value of Compliance level - optional to Medium.
- Check the box under Include non-security updates to include all Amazon Linux updates when patching.
If an approved patch is reported as missing, the option you choose in Compliance level, such as Critical
or Medium
, determines the severity of the compliance violation reported in System Manager Compliance.
- In the Patch exceptions section in the Rejected patches - optional text box, enter
system-release.*
This will reject patches to new Amazon Linux releases that may advance you beyond the Patch Manager supported operating systems prior to your testing new releases.
- For Linux operating systems, you can optionally define an alternative patch source repository. Choose the X in the Patch sources area to remove the empty patch source definition.
- Choose Create patch baseline and you will go to the Patch Baselines page where the AWS provided default patch baselines, and your custom baseline, are displayed.
Patch Groups
A patch group is an optional method to organize instances for patching. For example, you can create patch groups for different operating systems (Linux or Windows), different environments (Development, Test, and Production), or different server functions (web servers, file servers, databases). Patch groups can help you avoid deploying patches to the wrong set of instances. They can also help you avoid deploying patches before they have been adequately tested.
You create a patch group by using Amazon EC2 tags. Unlike other tagging scenarios across Systems Manager, a patch group must be defined with the tag key: Patch Group
. Note that the key is case sensitive. You can specify any value, for example, "web servers," but the key must be Patch Group
.
Note: An instance can only be in one patch group.
After you create a patch group and tag instances, you can register the patch group with a patch baseline. By registering the patch group with a patch baseline, you ensure that the correct patches are installed during the patching execution. When the system executes the task to apply a patch baseline to an instance, the service checks to see if a patch group is defined for the instance.
- If the instance is assigned to a patch group, the system then checks to see which patch baseline is registered to that group.
- If a patch baseline is found for that group, the system applies that patch baseline.
- If an instance isn't configured for a patch group, the system automatically uses the currently configured default patch baseline.
4.2 Assign a Patch Group
- Choose the Baseline ID of your newly created baseline to enter its details screen.
- Choose Actions in the top right of the window and select Modify patch groups.
- In the Modify patch groups window under Patch groups, enter
Critical
, choose Add, and then choose Close to be returned to the Patch Baseline details screen.
AWS-RunPatchBaseline
AWS-RunPatchBaseline is a command document that enables you to control patch approvals using patch baselines. It reports patch compliance information that you can view using the Systems Manager Compliance tools, such as which instances are missing patches and what those patches are. For Linux operating systems, compliance information is provided for patches from both the default source repository configured on an instance and from any alternative source repositories you specify in a custom patch baseline. AWS-RunPatchBaseline supports both Windows and Linux operating systems.
AWS Systems Manager: Document
An AWS Systems Manager document defines the actions that Systems Manager performs on your managed instances. Systems Manager includes many pre-configured documents that you can use by specifying parameters at runtime, including AWS-RunPatchBaseline. Documents use JavaScript Object Notation (JSON) or YAML, and they include steps and parameters that you specify.
All AWS provided Automation and Run Command documents can be viewed in AWS Systems Manager Documents. You can create your own documents or launch existing scripts using provided documents to implement custom operations as code activities.
4.3 Examine AWS-RunPatchBaseline in Documents
- In the AWS Systems Manager navigation bar under Shared Resources, choose Documents.
- Click in the search box, select Document name prefix, and then Equal.
- Enter
AWS-Run
into the text field and press enter.
- Select AWS-RunPatchBaseline and choose View details.
- Review the content of each tab in the details page of the document.
AWS Systems Manager: Run Command
AWS Systems Manager Run Command lets you remotely and securely manage the configuration of your managed instances. Run Command enables you to automate common administrative tasks and perform ad hoc configuration changes at scale. You can use Run Command from the AWS Management Console, the AWS Command Line Interface, AWS Tools for Windows PowerShell, or the AWS SDKs.
4.4 Scan Your Instances with AWS-RunPatchBaseline via Run Command
- Under Actions in the AWS Systems Manager navigation bar, choose Run Command. In the Run Command dashboard, you will see previously executed commands including the execution of AWS-RefreshAssociation, which was performed when you set up inventory.
-
(Optional) choose a Command ID from the list and examine the record of the command execution.
- Choose Run Command in the top right of the window.
- In the Run a command window, under Command document:
- Choose the search icon and select
Platform
, and then choose Linux
to display all the available commands that can be applied to Linux instances. - Choose AWS-RunPatchBaseline in the list.
- In the Command parameters box, leave the Operation value as the default Scan.
- In the Targets box, you can choose either Specify a tag or Manually selecting instances and use the check box at the top of the list to select all instances displayed, or select them individually. Note: there are multiple pages of instances and selections on each page must be selected specifically.
- Under Specify targets by, choose Specifying a tag to reveal the Tags sub-section.
- Under Enter a tag key, enter
Workload
, and under Enter a tag value, enter Test
. - Choose Add to commit the key value pair.
- In the Rate control box accept the defaults.
- Optionally you may specify Rate control, limiting Concurrency to a specific number of targets or a calculated percentage of systems, and specify an Error threshold by count or percentage of systems after which the command execution will end.
- In the Output options box uncheck the box next to Enable writing to an S3 bucket
- Note: Only the last 2500 characters of a command's output are displayed in the console. If you need additional detail you can log to a preconfigured S3 bucket and optional S3 key prefix.
- In the SNS notifications box accept the default of not enabling SNS notifications.
- Alternatively you can specify an SNS Topic for notification on all events, or on a specific event type, for either the entire command, or on a per-instance basis. Note: This requires Amazon SNS to be preconfigured.
- Expand the AWS command line interface command to view the command as it would appear if executed within the AWS Command Line Interface.
- Choose Run to execute the command and return to its details page.
- Scroll down to Targets and outputs to view the status of the individual targets that were selected through your tag key and value pair. Refresh your page to update the status.
- Choose an Instance ID from the targets list to view the Output from command execution on that instance.
- Choose Step 1 - Output to view the first 2500 characters of the command output from Step 1 of the command, and choose Step 1 - Output again to conceal it.
- Choose Step 2 - Output to view the first 2500 characters of the command output from Step 2 of the command. Note that this execution step PatchWindows was skipped as it did not apply to your Amazon Linux instance.
- Choose Step 1 - Output again to conceal it.
4.5 Review Initial Patch Compliance
- Under Insights in the the AWS Systems Manager navigation bar, choose Compliance.
- On the Compliance page in the Compliance Summary, you will now see that there are 4 systems that have critical severity compliance issues. In the Corresponding managed instances list, you will see the individual compliance status and details.
4.6 Patch Your Instances with AWS-RunPatchBaseline via Run Command
- Under Actions in the AWS Systems Manager navigation bar, choose Run Command.
- Choose Run Command in the top right of the window.
- In the Run a command window, under Command document:
- Choose the search icon, select
Platform
, and then choose Linux
to display all the available commands that can be applied to Linux instances. - Choose AWS-RunPatchBaseline in the list.
- In the Command parameters box, change the Operation value to Install.
- In the Targets box:
- Under Specify targets by, choose Specifying a tag to reveal the Tags sub-section.
- Under Enter a tag key, enter
Workload
and under Enter a tag value enter Test
.
- In the Rate control box:
- For Concurrency, leave the default targets selected and specify
1
. - For Error threshold, leave the default errors selected and specify
1
. - Note: If any updates are installed by Patch Manager, the patched instance is rebooted.
- Note: Limiting concurrency will stagger the application of patches and the reboot cycle, however, to ensure that your instances are not rebooting at the same time, create separate tags to define target groups (for example, primary and secondary) and schedule the application of patches at separate times to avoid a self inflicted outage.
- In the Output options box uncheck Enable writing to an S3 bucket
- Choose Run to execute the command and to go to its details page.
- Refresh the page to view updated status and proceed when the execution is successful.
4.7 Review Patch Compliance After Patching
- Under Insights in the the AWS Systems Manager navigation bar, choose Compliance.
- On the Compliance page, change the Compliance Type: to
Patch
. The Compliance Summary will now show that there are 4 systems that have satisfied critical severity patch compliance.
In the optional Scheduling Automated Operations Activities section of this lab you can set up Systems Manager Maintenance Windows and schedule the automated application of patches.
The Impact of Operations as Code
In a traditional environment you would have had to set up the systems and software to perform these activities. You would require a server from which to execute your scripts. You would need to manage authentication credentials across all of your systems. To access the state of your systems may require access to multiple systems.
Operations as code: reduces the resources, time, risk, and complexity of performing operations tasks and ensures consistent execution. You can take operations as code and automate operations activities by using scheduling and event triggers. Through integration at the infrastructure level you avoid "swivel chair" processes that require multiple interfaces and systems to complete a single operations activity.
Bonus Content: Creating Maintenance Windows and Scheduling Automated Activities
AWS Systems Manager: Maintenance Windows
AWS Systems Manager Maintenance Windows let you define a schedule for when to perform potentially disruptive actions on your instances such as patching an operating system (OS), updating drivers, or installing software. Each Maintenance Window has a schedule, a duration, a set of registered targets, and a set of registered tasks. With Maintenance Windows, you can perform tasks like the following:
- Installing applications, updating patches, installing or updating SSM Agent, or executing PowerShell commands and Linux shell scripts by using a Systems Manager Run Command task.
- Building Amazon Machine Images (AMIs), boot-strapping software, and configuring instances by using Systems Manager Automation.
- Executing AWS Lambda functions that trigger additional actions such as scanning your instances for patch updates.
- Running AWS Step Function state machines to perform tasks such as removing an instance from an Elastic Load Balancing environment, patching the instance, and then adding the instance back to the Elastic Load Balancing environment.
- Note: to register Step Function tasks you must use the AWS CLI.
Creating Maintenance Windows
To create a Maintenance Window, you must do the following:
- Create the window and define its schedule and duration.
- Assign targets for the window.
- Assign tasks to run during the window.
After you complete these steps, the Maintenance Window runs according to the schedule you defined and runs the tasks on the targets you specified. After a task is finished, Systems Manager logs the details of the execution.
5.1 Create a Patch Maintenance Window
First you must create the window and define its schedule and duration.
- Open the AWS Systems Manager console.
- In the navigation pane, choose Maintenance Windows and then choose Create a Maintenance Window.
- In the Provide maintenance window details box:
- In the Name field, type a descriptive name to help you identify this Maintenance Window, such as
PatchTestWorkloadWebServers
. - (Optionally), you may enter a description in the Description field.
- Choose Allow unregistered targets if you want to allow a Maintenance Window task to run on managed instances, even if you have not registered those instances as predefined targets.
- Note: If you choose this option, then you can choose the unregistered instances (by instance ID) when you register a task with the Maintenance Window.
- Note: if you don't choose this option, then you must choose previously-registered targets when you register a task with the Maintenance Window.
- Specify a schedule for the Maintenance Window by using one of the scheduling options.
- Under Specify with, accept the default Cron schedule builder.
- Under Window starts, choose the third option, specify Every Day at, and select a time 10 minutes in the future, such as
02:00
. - Note: if you do not complete the following steps prior to your maintenance window you will be able to update the start time.
- In the Duration field, type the number of hours the Maintenance Window should run, such as '3' hours.
- In the Stop initiating tasks field, type the number of hours before the end of the Maintenance Window that the system should stop scheduling new tasks to run, such as
1
hour before the window closes. - Set the Schedule timezone - optional to the local timezone.
- Choose Create maintenance window. The system returns you to the Maintenance Window page. The state of the Maintenance Window you just created is Enabled.
5.2 Assigning Targets to Your Patch Maintenance Window
After you create a Maintenance Window, you assign targets where the tasks will run.
- On the Maintenance windows page, choose the Window ID of your maintenance window to enter its Details page.
- Choose Actions in the top right of the window and select Register targets.
- On the Register target page under Maintenance window target details:
- In the Target Name field, enter a name for the targets, such as
TestWebServers
. - (Optionally) Enter a description in the Description field.
- (Optionally) Specify a name or work alias in the Owner information field.
- Note: Owner information is included in any CloudWatch Events that are raised while running tasks for these targets in this Maintenance Window.
- In the Targets section, under Select Targets by:
- Choose the default Specifying tags to target instances by using Amazon EC2 tags that were previously assigned to the instances.
- Under Tags, enter 'Workload' as the key and
Test
as the value. The option to add and additional tag key/value pair will appear. - Add a second key/value pair using
InstanceRole
as the key and WebServer
as the value.
- Choose Register target at the bottom of the page to return to the maintenance window details page.
If you want to assign more targets to this window, choose the Targets tab, and then choose Register target to register new targets. With this option, you can choose a different means of targeting. For example, if you previously targeted instances by instance ID, you can register new targets and target instances by specifying Amazon EC2 tags.
5.3 Assigning Tasks to Your Patch Maintenance Window
After you assign targets, you assign tasks to perform during the window.
- From the details page of your maintenance window, choose Actions in the top right of the window and select Register Run command task.
- On the Register Run command task page:
- In the Name field, enter a name for the task, such as
PatchTestWorkloadWebServers
. - (Optionally) Enter a description in the Description field.
- In the Command document section:
- Choose the search icon, select
Platform
, and then choose Linux
to display all the available commands that can be applied to Linux instances. - Choose AWS-RunPatchBaseline in the list.
- Leave the Task priority at the default value of 1.
- Note 1 is the highest priority.
- Tasks in a Maintenance Window are scheduled in priority order, with tasks that have the same priority scheduled in parallel.
- In the Targets section:
- For Target by, select Selecting registered target groups.
- Select the group you created from the list.
- In the Rate control section:
- For Concurrency, leave the default targets selected and specify
1
. - For Error threshold, leave the default errors selected and specify
1
.
- In the Role section, accept the default Use the service-linked role for Systems Manager
- In Output options, leave Enable writing to S3 unchecked.
- (Optionally) Specify Output options to record the entire output to a preconfigured S3 bucket and optional S3 key prefix
- Note: Only the last 2500 characters of a command's output are displayed in the console.
- In SNS notifications, leave Enable SNS notifications unchecked.
- (Optionally) Specify SNS notifications to a preconfigured SNS Topic on all events or a specific event type for either the entire command or on a per-instance basis.
- In the Parameters section, under Operation, select Install.
- Choose Register Run command task to complete the task definition and return to the details page.
5.4 Review Maintenance Window Execution
- Navigate to the AWS Systems Manager console and verify that the Next execution time is in the future.
- If not choose your maintenance window, and the choose Edit on the maintenance window details page.
- update the CRON/Rate expression to a time at least 2 minutes in the future and choose Save changes
- After allowing enough time for your maintenance window to have completed choose the Window ID for your new maintenance window.
- On the Maintenance window ID details page, choose History.
- On the View execution history page in the Task Invocations box select the Windows execution ID and choose View details.
- On the Command ID details page, scroll down to the Targets and outputs section, select an Instance ID, and choose View output.
- Choose Step 1 - Output and review the output.
- Choose Step 2 - Output and review the output.
You have now configured a maintenance window, assigned targets, assigned tasks, and validated successful execution. The same procedures can be used to schedule the execution of any AWS Systems Manager Document.
Bonus Content: Creating a Simple Notification Service Topic
Amazon Simple Notification Service (Amazon SNS) coordinates and manages the delivery or sending of messages to subscribing endpoints or clients. In Amazon SNS, there are two types of clients: publishers and subscribers. These are also referred to as producers and consumers. Publishers communicate asynchronously with subscribers by producing and sending a message to a topic, which is a logical access point and communication channel. Subscribers (i.e., web servers, email addresses, Amazon SQS queues, AWS Lambda functions) consume or receive the message or notification over one of the supported protocols (i.e., Amazon SQS, HTTP/S, email, SMS, Lambda) when they are subscribed to the topic.
6.1 Create and Subscribe to an SNS Topic
- Navigate to the SNS console at https://console.aws.amazon.com/sns/.
- Choose Create topic.
- In the Create new topic window:
- In the Topic name field, enter
AdminAlert
. - In the Display name field, enter
AdminAlert
. - Choose Create topic.
- On the Topic details: AdminAlert page, choose Create subscription.
- In the Create subscription window:
- Select Email from the Protocol list.
- Enter your email address in the Endpoint field.
- Choose Create subscription.
- You will receive an email request for confirmation. Your Subscription ID will remain PendingConfirmation until you confirm your subscription by clicking through the link to Confirm subscription in the email.
- Refresh the page after confirming your subscription to view the populated Subscription ARN.
You can now use this SNS topic to send notifications to your Administrator user.
7 Removing Lab Resources
Note: when the lab is complete, remove the resources you created. Otherwise you will be charged for any resources that are not covered in the AWS Free Tier.
- Navigate to the CloudFormation dashboard at https://console.aws.amazon.com/cloudformation/:
- Select your first stack.
- Choose Actions and choose delete stack.
- Select your second stack.
- Choose Actions and choose delete stack .
- Navigate to Systems Manager console at https://console.aws.amazon.com/systems-manager/:
- Choose State Manager.
- Select the association you created.
- Choose Delete.
- If you created an S3 bucket to store detailed output, delete the bucket and associated data:
- If you created the optional SNS Topic, delete the SNS topic:
- If you created a Maintenance Window, delete the Maintenance Window:
- If you do not intend to continue to use the Administrator account you created, delete the account:
- Navigate to the IAM console at https://console.aws.amazon.com/iam/.
- Choose Users.
- Select your user from the list.
- Choose Delete user.
- Select the check box next to "One or more of these users have recently accessed AWS. Deleting them could affect running systems. Check the box to confirm that you want to delete these users.".
- Choose Yes, delete.
- When next you navigate within the console you will be returned to the account login page.
- If you do intend to continue to use the Administrator account you created, we strongly suggest you enable MFA.
Thank you for using this lab.