Solved: How to lock Terraform provider version

While working with terraform you would have noticed that every time you execute a terraform plan it will download the latest version of terraform available for that provider.

While this is good if you are testing as you get the latest features but, it can create trouble in production if a buggy version gets deployed. So, it is always recommended that you lock down the version of provider. In this post we will show you how to do that.

It’s really very simple to lock down the provider version. You just have to add a snippet like below in your main.tf file .

provider "aws" {

     region="us-east-1"
 
     version="<=2.6.0"

}

In the above example we have specified that version 2.6.0 or older can be used.

The version argument value may either be a single explicit version or a version constraint string. Constraint strings use the following syntax to specify a range of versions that are acceptable:

>= 2.4.0: version 2.4.0 or newer
<= 2.4.0: version 2.4.0 or older
~> 2.4.0: any non-beta version >= 2.4.0 and < 2.5.0, e.g. 2.4.X
~> 2.4: any non-beta version >= 2.4.0 and < 3.0.0, e.g. 2.X.Y
>= 2.0.0, <= 3.0.0: any version between 2.0.0 and 3.0.0 inclusive

Give it a try and let us know if you have any query or suggestion.

Solved: How to configure Terraform backend on AWS S3

Terraform is a very useful tool for IaaS. As you would have already known that it create a .tfstate file to save the status of infra. If you are doing testing you can save the .tfstate locally on your laptop. But, if you are working in prod environment with team then it’s best that you save the .tfstate remotely so that it’s secure and can be used by other team members.

Here we will show you two ways of configuring AWS S3 as backend to save the .tfstate file.

  1. First way of configuring .tfstate is that you define it in the main.tf file. You will just have to add a snippet like below in your main.tf file.
terraform {

      backend "s3" {

          bucket="cloudvedas-test123"

          key="cloudvedas-test-s3.tfstate"

          region="us-east-1"

      }

}

Here we have defined following things.

bucket = The S3 bucket in which the .tfstate should be saved

key = The name of the .tfstate file

region = The region in which S3 backend bucket exists.

2 Another way of specifying the S3 backend is that you define it when you initialize the terraform using the init command. This can be useful when you want to invoke the terraform from a jenkins file.

  • Here is an example that you can  execute in windows command prompt. This will do the same thing as we did in first example.
terraform init -no-color -reconfigure -force-copy -backend-config="region="us-east-1"" -backend-config="bucket="cloudvedas-test123"" -backend-config="key="cloudvedas-test1-win-s3.tfstate""
  • If you want to execute from a linux shell use below syntax.
 terraform init -no-color -reconfigure -force-copy \
-backend-config="region=us-east-1" \
-backend-config="bucket=cloudvedas-test123" \
-backend-config="key=cloudvedas-test-s3.tfstate"

Give it a try and let us know in comments section if you have any query or suggestion.

Solved RDS : Access denied; you need the SUPER privilege for this operation

Access denied; you need the SUPER privilege for this operation

You may get this error while trying to set values for RDS AURORA MySQL from the command line. It can be setting for long running queries or slow queries or many others.

If, you are sure you are trying to execute these changes using the master user then you can’t set these from command line.

For RDS Aurora you will have to make these changes through Parameter groups of DB and Cluster.

  •  To make the change, login to your AWS RDS console.
  • On the left side panel click on Parameter Groups and select the group associated with your RDS Cluster and node.
  • Make changes in the parameter groups.
  • Once you have saved the changes in parameter group it will start applying to your RDS cluster.

Some parameter changes will require reboot of your cluster while others can be done without reboot.  You will see pending-reboot in your cluster if it needs reboot to change the parameter.  For more details about parameter groups refer this AWS doc.

Review of 70-532 Developing Microsoft Azure Solutions Certification preparation course by Scott Duffy

In our earlier post we have described steps about how you can prepare for Architecting Microsoft Azure Solutions Certification. But, if you have just started your journey in Azure than it will be a good idea to start with the Azure developer certification which is 70-532 Developing Microsoft Azure Solutions .

If you are an absolute beginner you can start with free Azure account.  It will help give a good understanding of Azure. But, be careful while using the resources because as we get free hand access to resources, we tend to spin up lot of instances and forget them running. This may end up with a huge bill. Best practice is that you setup a billing alert to avoid any shocks at month end.

To further hone your skills you can either go for Azure classroom training or go for online courses. The classroom training will cost you from USD 800 to USD 2000. While the online courses can cost you from USD 10 to USD 300, depending on which course you choose.

As mentioned in the last post since my experience with online courses was good as it give you good foundation, so I prefer to go the online way.

When I started my search for a suitable course I zeroed in on 70-532 Developing Microsoft Azure Solutions Certification  course by Scott Duffy on Udemy.

Scott himself is a certified architect and got a rich industry experience. Going through the course content I found that it covered almost all exam topics. Also Scott keeps on updating the content as the syllabus changes. And if you buy from Udemy you get free life time access to the course so I went with this one.

As of Dec-18 the course contents 10 hours of video and a practice test. Also, you get lifetime free access to the course on Udemy.

It’s a good idea to follow all the labs with the instructor and once you get confidence redo the labs independently. Don’t forget to complete the practice quizzes to check your knowledge.

You can also supplement your preparation with practice Tests for 70-532 Developing MS Azure Solutions  .

Solved : How to simply install python pip on windows?

The latest version of python is shipped with pip. But, if you are using older version of python 2(<2.7.9) or python 3(<3.4) and still need pip on windows please follow the instruction below.

If pip is not installed or the path is not set correctly you will get an error like below while invoking it in windows command prompt.

pip install sklearn
'pip' is not recognized as an internal or external command,
operable program or batch file.

Here are the steps to get pip in your windows box.

  • Download get-pip.py . Copy and paste the contents of the link in a notepad and save it as get-pip.py, remember not in .txt format but .py format.
  • Next install it.
python get-pip.py
  • Find the recently installed pip.exe in your machine. It’s generally in C:\Python27\Scripts or similar folder depending on the python version you installed. You can set the path variable in windows using these instructions.

– Search for “Advanced System Settings” in your windows machine.
– Click on the tab “Advanced”. In it click on Environment Variables.
– In the new window click on System Variables.  Select “Path” and click on “Edit”.
– Click on new and enter C:\Python27\Scripts\ in the space.
– Finally click ok to all windows and re-open command prompt.

  • If you have followed the instructions correctly you will now be able to install packages using pip.
pip install sklearn

That’s all you need to install pip in windows.

Tip:- Some people make mistake of executing pip from Python interpreter and get the below error.

>>> pip install sklearn
File "<stdin>", line 1
pip install sklearn
^
SyntaxError: invalid syntax
>>>

But, from the above tutorial you now know that pip is installed and executed from command prompt and not from the interpreter.

Hope you find this simple tutorial useful. Let us know in comment section if you face any issue.

Solved : How to check disk usage in windows like Linux

Identifying the files or directories which are hogging space in your windows machine can be very difficult specially if you have hidden files or they are in nested directories.

If you have worked on Linux you will know it has a very useful command du -sh. When i work on windows  I really miss the Linux command which is very handy.

Thankfully you can find a utility at sysinternals on microsoft site which do the same job as du command in linux.

The name of utility is simply “du” and you can download it from here .

Once you download the utility you just have unzip it and no installation needed.

After that open the command prompt as admin user and go to the directory where you have unzipped the “du” and execute it as below to find the usage of any directory.

du -l 1 "C:\Program Files\Microsoft SQL Server"

It will show you usage of even the hidden files. The usage is shown in KB.

Solved : Check without telnet if port is open on remote host – Linux

Earlier many system admins were using telnet to check if a port is open on remote machine. But, because of security concerns telnet is now not recommended to be installed on most linux boxes.

But hey we still need to check the remote port. This is part of normal troubleshooting in a linux admin’s life.

So, instead of using telnet you can use  nc.  netcat or nc is a  simple tool which is generally installed by default in linux distributions or you can install it with instructions below.

Ubuntu

$ sudo apt-get update

$ sudo apt-get install netcat

Redhat or CentOS

yum install nc.x86_64

Amazon Linux

 sudo yum install nc

Once you are done with installation simply run it as below:-

nc -z 100.4.78.39 22
Connection to 100.4.78.39 22 port [tcp/ldaps] succeeded!

Here we can see that port 22 is open on remote server.

Tip :- If you want to check the same from windows machine, follow this post on How to check remote port status in windows without telnet.

nc is a swiss knife kind of tool. It comes with very useful options.  You can explore the below options also.

Tag Description
4 Forces nc to use IPv4 addresses only.
6 Forces nc to use IPv6 addresses only.
D Enable debugging on the socket.
d Do not attempt to read from stdin.
h Prints out nc help.
i interval
Specifies a delay time interval between lines of text sent and received. Also causes a delay time between connections to multiple ports.
k Forces nc to stay listening for another connection after its current connection is completed. It is an error to use this option without the –l option.
l Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host. It is an error to use this option in conjunction with the –p , –s , or –z options. Additionally, any timeouts specified with the –w option are ignored.
n Do not do any DNS or service lookups on any specified addresses, hostnames or ports.
p source_port
Specifies the source port nc should use, subject to privilege restrictions and availability. It is an error to use this option in conjunction with the –l option.
r Specifies that source and/or destination ports should be chosen randomly instead of sequentially within a range or in the order that the system assigns them.
S Enables the RFC 2385 TCP MD5 signature option.
s source_ip_address
Specifies the IP of the interface which is used to send the packets. It is an error to use this option in conjunction with the –l option.
T ToS
Specifies IP Type of Service (ToS) for the connection. Valid values are the tokens “lowdelay”, “throughput”, “reliability”, or an 8-bit hexadecimal value preceded by “0x”.
t Causes nc to send RFC 854 DON’T and WON’T responses to RFC 854 DO and WILL requests. This makes it possible to use nc to script telnet sessions.
U Specifies to use Unix Domain Sockets.
u Use UDP instead of the default option of TCP.
v Have nc give more verbose output.
w timeout
If a connection and stdin are idle for more than timeout seconds, then the connection is silently closed. The –w flag has no effect on the –l option, i.e. nc will listen forever for a connection, with or without the –w flag. The default is no timeout.
X proxy_version
Requests that nc should use the specified protocol when talking to the proxy server. Supported protocols are “4” (SOCKS v.4), “5” (SOCKS v.5) and “connect” (HTTPS proxy). If the protocol is not specified, SOCKS version 5 is used.
x proxy_address[: port]
Requests that nc should connect to hostname using a proxy at proxy_address and port. If port is not specified, the well-known port for the proxy protocol is used (1080 for SOCKS, 3128 for HTTPS).
z Specifies that nc should just scan for listening daemons, without sending any data to them. It is an error to use this option in conjunction with the –l option.

Real shell scripting interview questions

Q What is Shell?

Ans: Shell is a command interpreter, which interprets the command which the user gives to the kernel. It can also be defined as an interface between a user and operating system.

Q How to debug the problems encountered in shell script/program?

Ans: Two options
1) Execute the script as “sh -x script.sh”
2) Put “set -x” in the script

Q Which is the symbol used for comments in bash shell scripting ?
Ans:

#

Q What is the difference between = and ==?

Ans:

=      -> It is used for assigning value to the variable.

==    -> It is used for string comparison.

Q How to get 4th element from each line of a file ?

Ans:

awk '{print $4}'

Q What needs to be done before you can execute a shell script?

Ans:
You need to make the shell script executable using the chmod command.

This chmod command makes the shell script file “file1” executable for the user (owner) only:
$ chmod u+x file1

Below syntax makes it executable for all (everyone):

$ chmod a+x file1

Q How to pass argument to a script ?
Ans:

./script argument

Q How do you terminate an if statement?

Ans: Using “fi” . Check example below.

Q Give an example of if else statement
Ans:

#Testfile.sh script to test if the file exists
#!/bin/ksh
cvfile=$1
if [ -f $cvfile ]
then
echo "$cvfile exists"
else
"$cvfile does not exists"
fi
exit 0

So you will execute the script as “./Testfile.sh file1” .

Q How to check if a directory exists?

Ans:

if [ -d $mydir ] 
then
echo "Directory exists"
fi

Q How to calculate number of passed arguments ?

Ans:

$#

Q How to check if previous command run successful ?

Ans:

echo $?

If exit code is 0 it means command ran successfully

Q How to get last line from a file ?

Ans:

tail -1

Q How to redirect stdout and stderr streams to log.txt file from inside the script ?
Ans:

Add “exec >log.txt 2>&1” put this as the first command in the script

Q How to remove blank lines from a file?

Ans :

grep -v '^

Q Write a command to find all the files modified in less than 3 days and print the record count of each?

find . –mtime -3 –exec wc –l {} \;

Q How to find a process name from process ID?

ps -p PID

 testcv.txt > testcv2.txt

Q Write a command to find all the files modified in less than 3 days and print the record count of each?

find . –mtime -3 –exec wc –l {} \;

Q How to find a process name from process ID?

ps -p PID

Solved: AWS Inspector issue : Service ‘Amazon Web Services Agent’ (AWSAgent) could not be stopped. Verify that you have sufficient privileges to stop system services.

AWS Inspector issue
“Service ‘Amazon Web Services Agent’ (AWSAgent) could not be stopped. Verify that you have sufficient privileges to stop system services.”

Solution:-

First check that you are running the AWS inspector installation as administrator. But if you are still getting error then it can be because the most recent Amazon Windows AMIs released on February 23rd include a driver that uses the same service name as the Amazon Inspector Agent. This causes Inspector Agent installations to fail with the above error message. Impacted versions of the Windows AMIs include Windows Server 2008 R2, Windows Server 2012, Windows Server 2012 R2, and Windows Server 2016.

Try fixing it with the remove script provided in below forum link after taking all the required backups.

https://forums.aws.amazon.com/ann.jspa?annID=5505

But if you are still getting error “EC2 Windows Utility Device’ not found” when you execute the remove script follow below steps.

  • Take snapshot image of the instance.
  • After taking snapshot image login to the instance and execute below command in powershell as an administrator to fix it. It will need reboot of instance.
$agentService = Get-WmiObject -class win32_systemdriver | Where-Object {$_.Name -eq 'awsagent'}
$agentService.Delete()
  • After running these commands reboot the instance and try installation of AWS Inspector again.