Solved RDS : Access denied; you need the SUPER privilege for this operation

Access denied; you need the SUPER privilege for this operation

You may get this error while trying to set values for RDS AURORA MySQL from the command line. It can be setting for long running queries or slow queries or many others.

If, you are sure you are trying to execute these changes using the master user then you can’t set these from command line.

For RDS Aurora you will have to make these changes through Parameter groups of DB and Cluster.

  •  To make the change, login to your AWS RDS console.
  • On the left side panel click on Parameter Groups and select the group associated with your RDS Cluster and node.
  • Make changes in the parameter groups.
  • Once you have saved the changes in parameter group it will start applying to your RDS cluster.

Some parameter changes will require reboot of your cluster while others can be done without reboot.  You will see pending-reboot in your cluster if it needs reboot to change the parameter.  For more details about parameter groups refer this AWS doc.

Solved: AWS Inspector issue : Service ‘Amazon Web Services Agent’ (AWSAgent) could not be stopped. Verify that you have sufficient privileges to stop system services.

AWS Inspector issue
“Service ‘Amazon Web Services Agent’ (AWSAgent) could not be stopped. Verify that you have sufficient privileges to stop system services.”

Solution:-

First check that you are running the AWS inspector installation as administrator. But if you are still getting error then it can be because the most recent Amazon Windows AMIs released on February 23rd include a driver that uses the same service name as the Amazon Inspector Agent. This causes Inspector Agent installations to fail with the above error message. Impacted versions of the Windows AMIs include Windows Server 2008 R2, Windows Server 2012, Windows Server 2012 R2, and Windows Server 2016.

Try fixing it with the remove script provided in below forum link after taking all the required backups.

https://forums.aws.amazon.com/ann.jspa?annID=5505

But if you are still getting error “EC2 Windows Utility Device’ not found” when you execute the remove script follow below steps.

  • Take snapshot image of the instance.
  • After taking snapshot image login to the instance and execute below command in powershell as an administrator to fix it. It will need reboot of instance.
$agentService = Get-WmiObject -class win32_systemdriver | Where-Object {$_.Name -eq 'awsagent'}
$agentService.Delete()
  • After running these commands reboot the instance and try installation of AWS Inspector again.

Solved: How to reset RDS master user password

In this post we will show you how to reset  the master user password of an RDS DB instance. The new password after reset will be effective immediately.

  • Go to your AWS RDS console.
  • Select the DB instance whose master password you want to change. Do note that the DB instance should be in available state and not in backing up or any other state.
  • Once you have selected the instance, select Modify from Instance Actions dropdown.

  • It will take you to a new page  :- Modify DB Instance: cloudvedasdb
  • Scroll down to the Setting section and look for New master password.
  • Enter the new password in the text box and at the bottom click on continue.
  • Finally click on Modify DB instance.
  • The new password will be effective almost immediately.

Must have skills for DevOps Engineer and Free methods to learn them

Many people have asked us how to become a DevOps engineer so in this post we will show you the skills and technologies which can get you there.

Before we start we should know what is “DevOps”. As you can see the word is combination of “Developer” and “Operations”. So a DevOps engineer is suppose to be a bridge between the development and operations processes.

Let’s see which skills do you need to become a DevOps engineer. In this post we will also discuss how you can acquire these skills for free. We have also mentioned some advanced paid courses which can help you acquire these skills quickly.

Skill 1 :-  Make Linux your friend.

Nowadays whole world is moving towards opensource, and Linux is the poster boy of opensource world.  Most of the new enterprise level tools and services are now compatible with Linux and people are adopting them fast. So, as a devops engineer you will definitely have to work on/with linux in your job. Even Microsoft has come up with their own Linux version. So, you can understand the importance of Linux. You can learn Linux for free from Linux Foundation courses. But, if you are short of time you can quickly learn Linux through this course Learn Linux in 5 days .

Skill 2 :- Learn scripting

Automation is a big part of a DevOps engineer job and you will be expected to automate mundane tasks. This will not only help you work efficiently but also improve the overall maintainability of your company’s setup. Easiest way to automate things is through scripting. You should learn either shell scripting on Linux/Unix or powershell on windows. Free source to learn scripting is Learn Shell. Else, you can check out this around 6 hours course on Shell Scripting to solve real-world problems like a pro.

Skill 3 :- Understand the Cloud

Companies are shifting to cloud like never before. Applications are being re-built to leverage the advantage of cloud computing. So it’s always good that you know cloud really well. AWS, Azure and GCP are three market leaders and it will be good to know about at least one of them. All these vendors have tons of free material on their sites which can help you learn their technology. Else you can checkout these awesome Udemy courses for AWS, Azure or GCP .

Skill 4:- Source code Management

Tools like git are used to manage the version of code base and avoid any conflicts in code. As a devops engineer you should have good understanding of tools like git or bitbucket since they will be useful in build phase. Later you will use them to build a codepipeline. One of the good source to learn git for free is Try Git. To become zero to hero of Git in hours, one of the best course is Git complete.

Skill 5 :- Configuration management

Configuration management is used to manage the state of your infrastructure. You can use them to patch your linux servers or manage the services.  Some of the tools in config management are Ansible, Chef and Puppet. To get basic understanding of Ansible the best free course is from Redhat called Ansible essentials. If you want to get deeper understanding you can try Mastering Ansible.

Skill 6: – Continuous Integration/Continuous Delivery

CI/CD is an important aspect in DevOps engineers job profile. As we are moving towards cloud automated deployments are now essential. They avoid human errors and are fast. You can learn tools like Jenkins for CI/CD. Jenkins will specially be very useful in building a codepipeline, you can learn it for free here. Since CI/CD consists of multiple tools we think that it’s better you learn it from a single source. We found course from Edward Learn Devops: Continuously Deliver Better Software  to have covered most of the topics.

Skill 7 :- Infra as a Code

The latest “in thing” in cloud is Infra as a Code. Yes you read it right, gone are the days where you have to buy the server, install OS and other softwares on it. With tools like Terraform or Cloudformation you can automate the build of your entire infrastructure in cloud right from VPC, Security groups,  servers etc. to application installation and configuration. To get basic free understanding of terraform you can learn it from Katacoda . If you want to deploy AWS Infrastructure using Terraform we found Edward’s another course on Terraform to be really useful. For cloudformation we found the free AWS cloudformation tutorials to be useful. If you are looking for advance practical guidance AWS CloudFormation Master Class is a good course.

Skill 8:- Containers

Servers are now shrinking every day. With the advent of VMs, hardware anyways became less important. Dockers, a containerization technology is taking it to one level up. Now with Dockers you can have a miniature copy of your OS image with the application code. Docker will only run for a few mili seconds when a query is executed and once it’s job is done it will just stop. If you want to learn Docker for free the best source is Play with Docker Classroom. To gain more advanced skills you can learn from Docker Mastery.

Skill 9 :-  Container orchestration

Tools like Kubernetes or Elastic container service (ECS) are used to manage cluster of Docker containers. These tools provides you mechanisms for deploying, maintaining, and scaling application. You can learn about Kubernetes with practical labs for free from Katacoda. For advanced knowledge of Kubernetes try the complete Kubernetes course.

Skill 10 : – Serverless

Technologies like AWS lambda are removing the need to manage the servers at all. You just have to upload you code in the AWS Lambda and set your run time environment. Rest all the issues of memory, CPU, OS patching will be taken care by AWS Lambda. It’s important as a DevOps engineer that you understand how the lambda works and how you can set it up. You can learn about AWS Lambda for free from this AWS Lambda Tutorial. For more advanced real world example you can refer how to build a serverless App with AWS Lambda .

We hope this post is helpful to you in understanding the skills which are important for a DevOps engineer and how you can master these skills. You can expect questions related to these skills in the interview of a DevOps Engineer.

How to Import SSL certificate in AWS Certificate Manager and apply on ELB

In this post we will show you how to apply an SSL certificate on an AWS ELB.

Image source aws.amazon.com

To use the certificate in ELB first we will have to import it in AWS Certificate Manager(ACM).

Currently ACM needs PEM files namely Certificate Body, Certificate Private Key and  Certificate Chain. You can check ACM SSL certificate pre-requisites here.

If you have the SSL certificate in .pem formats you can directly go to Step 7. Else if you have certifcate in .pfx or other format please follow from Step 1.

Step 1  As mentioned earlier ACM expects the certificates in PEM format. So we will first convert the certificates from pfx to pem format using openssl tool.

Step 2  Download the openssl binaries for windows of Linux from the links on openssl site.

Step 3 Once downloaded, install it. In our case we have installed the exe on windows.

Step 4 Go to the location where the OpenSSL is installed. In the bin folder you will find the openssl.exe . We will use this exe for conversion as per below commands. (You may need the import password you used while generation of SSL certificate)

Execute below command in windows command prompt(CMD) as an administrator.

C:\OpenSSL-Win64\bin>openssl pkcs12 -in "C:\OpenSSL-Win64\bin\test-cloud.cloudvedas.com.pfx" -nodes -out "C:\OpenSSL-Win64\bin\test\test-cloud.cloudvedas.com.pem"

Enter Import Password:

Step 5 In the above command with “-in” you provide location where your .pfx certificate is kept. In “-out” you define the destination where the newly created .pem file should be kept. You can create a pem file from other certificate formats like .cer, .der, .p7b etc. also using openssl. Just explore its help options and execute command as we have done above.

Step 6 Once the .pem file is created you can open it with notepad. It will have multiple certificates and a key in it.

Step 7 Now go to AWS ACM console and click on Import a certificate.

Step 8 In the  Certificate body enter the complete certificate with all digits as below. In it’s description it will have something like friendlyName.

-----BEGIN CERTIFICATE-----

hdshgdhsgjjdghweuiw123hjhd8

-----END CERTIFICATE-----

Step 9 Similarly enter the private key contents as below.

-----BEGIN PRIVATE KEY-----

deuiryfmvfv7682376ruifn3487tdfi58fvnf8g9

-----END PRIVATE KEY-----

Step 10 Similarly enter the Certificate Chain .

-----BEGIN CERTIFICATE-----

yeuiwye78689ywhyeyds85d76ctd7cx7c56x8

-----END CERTIFICATE-----

Step 11 Finally review and save the certificate.

Step 12 Now go to Route 53 > Hosted Zone and create a record set. Check this link if you want to create a new Hosted Zone.

Step 13 Once inside the hosted zone create a record set where you will basically map your friendly URL name (test-cloud.cloudvedas.com) to the load balancer with a CNAME. (refer image below)

Step 14 Now go to Load Balancer section and click on your ELB.

Step 15 Go to listeners tab and click on Edit.  Click “Add” and in the Load Balancer protocol select HTTPS. In the same window click “Change” option below SSL certificate.

Step 16 In the new window select Choose an existing certificate from AWS Certificate Manager (ACM), and then select the certificate that you created, from Certificate list. Finally click on Save.

Congrats! You have now applied a new SSL certificate to the load balancer. Do let us know in comments section if you have any query.

How to prepare for AWS Certified SysOps Administrator – Associate

In one of our earlier post we have detailed about which AWS certification is suitable for you ?

If you are from System Admin or DevOps background the  AWS Certified SysOps Administrator – Associate certification will be a good plus for you.

If you are  an absolute beginner on AWS you can start with free labs from AWS . To practice further you can create free AWS account. These two actions will get you started on AWS.

Beware that if you go beyond free tier limits you will be billed. Best practice is to create a billing alert.  This alert can save you from unexpected bill shocker.

If you want to learn further you can opt for either classroom course or online course. The classroom course is generally expensive and range between USD 600 to 2000. While the online course can cost you anything between USD 10 to USD 300 depending on which course you choose.

Our personal opinion is that you should go for online courses as they are cheaper and if you follow their labs honestly(yeah not just seeing him do it but actually doing the labs yourself 😉 ) they can be as good as classroom training.

In online courses we found courses from two providers acloudguru and linux academy  to be good. Earlier the Linux Academy course was only available through their site using a monthly plan but the same course is now available on Udemy too. The acloudguru course is already available on Udemy from longtime.  Though both these courses are available to purchase from their respective site under monthly subscription but, if you buy these courses from Udemy you pay only once and get lifetime access to same courses.  And many times Udemy provide heavy discounts on courses which can get you a good bargain.

The acloudguru course is delivered by Ryan who is enthusiastic and teach you really well. Though sometime he can get a bit click happy and can quickly zip past few topics. But, you always have the option to rewind and go through the topic again 🙂 .

The labs in the course are very useful and helps you get deep understanding of topic. The course also has quizzes to check your knowledge.

Overall we found the acloudguru course to be beneficial in getting you exam ready.

However do note that the course alone is not enough to clear the exam. You should go through whitepapers and FAQs of atleast below services .

  • EC2
  • S3
  • VPC
  • Route 53
  • CloudWatch
  • OpsWorks
  • Billing

Exam pattern

Exam has Multiple-choice and multiple-answer questions. Exam is of 80 minutes. You can download the exam blueprint here.

Practice Exam Questions

To get a good evaluation of your preparation you can go through another course  for sample exam questions on Udemy  .

Exam Cost

Exam will cost you USD 150. AWS also gives you an option to book a practice exam for USD 20 before you go for actual exam.

How to book exam

To book an exam you will have to create an account in AWS Training and Certification Portal .

Passing score

AWS doesn’t reveal minimum passing score and it keeps on changing. But we have observed that generally people who score above 70% passed the exam. You will immediately see the score on you screen once you finish the test and will also get a report on your mail within 1 hour.

Exam Tips

  • Get good sleep and keep calm during the exam.
  • You won’t get more than 3 mins per question.
  • You may find some very long questions in exam. Best strategy to tackle them is to read the answer options first and then check for relevant info in question.
  • Since it’s an AWS exam so look for AWS related options in the answers.  Chances are high that  Non-AWS related option in answer will be wrong.
  • AWS exams generally don’t focus on mugging their datasheets. So you won’t get a question like “How much RAM does a C3.xlarge offer?” .

That’s all folks! Best of luck for the exam!

Do let us know in comments section if you have any query.

How to prepare for AWS Certified Developer – Associate certification exam

In one of our earlier post we have detailed about which AWS certification is suitable for you ?

If you have decided to go for AWS Certified Developer – Associate certification this post is for you.

If you are  an absolute beginner you can start with free labs from AWS . To practice further you can create free AWS account. These two actions will get you started on AWS.

Beware that if you go beyond free tier limits you will be billed. Best practice is to create a billing alert.  This alert can save you from unexpected bill shockers.

To further hone your skills you can either go for AWS classroom training or go for online courses. The classroom training will cost you from USD 800 to USD 2000. While the online courses can cost you from USD 10 to USD 300, depending on which course you choose.

Our personal opinion is that you should go for online courses as they are cheaper and if you follow their labs honestly they can be as good as classroom training.

In online courses we recommend two providers acloudguru and Linux Academy.  Both have monthly plans to buy the courses but they can become bit costly if you can’t complete the course in 1 month.

We have observed that acloudguru also provide the same course on Udemy where you get lifetime access to course with just one time payment. Also, Udemy provide heavy discounts on courses during sale which can get you a good bargain.

Thus, we recommend this AWS Certified Developer – Associate course  of acloudguru on Udemy . Also, once you buy this course on Udemy you will get access of the same course on acloudguru website also.

The acloudguru course instructor Ryan is an industry expert and deliver the course really well. The course will cover almost all the main topics which are asked in exam.  (Though we have observed that as of Apr-18 it was missing a session on AWS lambda for which the questions have started appearing in exam. Hope they update the course soon.) Currently you can learn about lambda from Ryan’s other course on AWS Lambda .

It’s a good idea to follow all the labs with the instructor and once you get confidence redo the labs independently. Don’t forget to complete the practice quizzes to check your knowledge.

This course will give you good base for the exam. But, the course itself is not enough to clear the exam.  You should go through whitepapers and FAQs of atleast below service .

  • EC2
  • S3
  • SQS
  • RDS
  • DynamoDB
  • Lambda

Exam pattern

Exam has Multiple-choice and multiple-answer questions. Exam is of 80 minutes.

Practice Exam Questions

To get a good evaluation of your preparation you can go through another course of  acloudguru sample exam questions on Udemy  .  Many test takers have said that they got similar questions in exam.

Exam Cost

Exam will cost you USD 150. AWS also gives you an option to book a practice exam for USD 20 before you go for actual exam.

How to book exam

To book an exam you will have to create an account in AWS Training and Certification Portal .

Passing score

AWS doesn’t reveal minimum passing score and it keeps on changing. But we have observed that generally people who score above 80% passed the exam. You will immediately see the score on you screen once you finish the test and will also get a report on your mail within 1 hour.

Exam Tips

  • Get good sleep and keep calm during the exam.
  • You won’t get more than 3 mins per question.
  • You may find some very long questions in exam. Best strategy to tackle them is to read the answer options first and than check for relevant info in question.
  • Since it’s an AWS exam so look for AWS related options in the answers.  Chances are high that  Non-AWS related option in answer will be wrong.
  • AWS exams generally don’t focus on mugging their datasheets. So you won’t get a question like “How much RAM does a C3.xlarge offer?” .

That’s all folks! Best of luck for the exam!

Do let us know in comments section if you have any query.

Solved: Restore root disk of EC2 without changing IP or Hostname

If the root volume of your EC2 instance got corrupt, instance won’t come up. Since you don’t have access to console of EC2 instance so you can’t do much.

In this post we will discuss options to restore an EC2 instance from snapshot backup. Prerequisite for the below guide is that you already have snapshot of the volume which you want to restore.

Option 1 – Different IP and Hostname 

Easiest option to restore an EC2 instance is to launch a new instance with the available snapshot. Refer this AWS doc to launch an instance from backup.

But, an instance launched this way will have both hostname and private IP different from original instance. If this is a problem for you go to option 2.

Option 2 – Same IP different Hostname

So, as per option 1 you have created and instance with an AMI but now you need the IP to be same as old one. To go around this you can detach the network interface of old instance and attach it to new instance.

This option will give you same private IP as old one but you will still have new hostname.

If you need both hostname and IP to be same go to option 3.

Option 3 – Same IP and Hostname

In this option we will discuss how you can restore an EC2 instance and keep both hostname and IP same. This can very important if your EC2 instance is in Active Directory(AD) domain, as change in IP and hostname mapping will cause conflict in domain. Because of this conflict the domain server can block login to the EC2 instance.

Let’s see how we can get around this.

Prerequisites:-

  • You already have snapshot of the root and other volumes of instance.
  • Keep a screenshot of your Instance description from your AWS console, this can be used to refer instance configuration later.

Plan

For the eager one’s, plan is to follow the below steps.

  • Stop the instance
  • Detach the current root EBS
  • Create a new volume from the old snapshot
  • Attach the new volume to instance
  • Boot the instance

Now let’s see the steps in detail.

Stop Instance

Stop the instance if it’s up.

Detach the current root EBS 

  • Select the root volume(/dev/sda1) mentioned as “Root device” from instance description and click on EBS id of the volume.
  • You will now come in “Volumes” window.
  • From the “Actions” drop down select “Detach Volume”

Create a new volume from the old snapshot

  • Create a volume from the snapshot you have taken earlier.
  • Select the snapshot of the volume and from “Actions” drop down select “Create Volume”.
  • In the “Create Volume” window ensure that you select the Availability Zone to be same as the AZ in which your instance is located.
  • Leave the other options as default. (refer image)

  • Finally hit “Create Volume”.

Attach the new volume

  • Once your volume is created select it.
  • From the “Actions” drop down select “Attach”.
  • While attaching the volume in the device field mention volume name as /dev/sda1 since we are attaching it as root volume. This is same as your old root volume name.

  • Hit “Attach” .

Boot instance

  • Once the volume is attached start the instance normally. You should now see that the instance has the data from your old backup. Also, it’s hostname and private IP will remain same as before.

Note:- If you want to keep the public IP to be fixed, you need to assign an Elastic IP to the instance. Public IP assigned by default, by AWS keeps on changing with every reboot.

That’s all folks!

Solve: How to add an EBS volume to a Windows EC2 instance and configure it

This post is divided in two sections. In the first section you will see how to create an EBS volume and in the next section we will show you how to configure the EBS volume in windows instance.

Create EBS Volume

  • Go to AWS Console > EC2
  • In the left panel select “Volumes” .
  • Once in the “Volume” screen select “Create Volume”

  • In the “Create Volume” window specify the size of disk and the Availability Zone  in which you want the disk to be created.

Tip:- The disk should be in same AZ as your EC2 instance.

  • Now in the left pane again select the “Volume” to see all your volumes.
  • Select the volume you just created and after that in the upper menu click on “Actions” and select “Attach volume”

  • In “Attach volume” window select the instance to which you want to attach the volume and click on “Attach”.

Configure EBS volume in Windows

  • Login to your windows EC2 instance using RDP. Once inside the instance, from the Start menu go to  “Computer Management” as mentioned below.

Start > Control Panel > System and Security > Administrative Tools > Computer Management

  • Click on Disk Management on the left pane.
  • Here we can see the new disk but it’s still offline. Right click on the new disk and select “Online”.

  • Once the disk is online right click again on the disk and select “Initialize Disk”.

  • If disk is below 2TB, select MBR and click OK

  • Finally right click on the pane where size is shown(refer image below). Select “New Simple Volume”

  • Leave other options as default and click “Next” till you come to “Assign Drive Letter or Path”.  Here we have assigned the drive letter E .

  • Leave everything else as default in next windows and click on finish.

  • Now if we go to “This PC/ My computer” we should see the new disk.

So here we have attached an EBS volume to the Windows EC2 instance. Do let us know in comments section if you have any query.