# DevOps Interview Questions
:information_source: This repository contains interview questions on various DevOps related topics
:bar_chart: There are currently **170** interview questions
:warning: Some answers might be only partial and shouldn't be used as they are in interviews
:exclamation: You don't need to know how to answer all these questions here. DevOps is not about specific set of tools or techs
:exclamation: Some questions are very specific and will be asked only if you are being interviewed to specialized roles
:pencil: You can add more questions & answers by submitting pull requests :)
****
## DevOps
#### :baby: Beginner
What is Continuous Integration?
A development practice where developers integrate code into a shared repository frequently. It can range from a couple of changes every day or week to a couple of changes in one hour in larger scales.
Each piece of code (change/patch) is verified, to make the change is safe to merge. Today, it's a common practice to test the change using an automated build that makes sure the code can integrated. It can be one build which runs several tests in different levels (unit, functional, etc.) or several separate builds that all or some has to pass in order for the change to be merged into the repository.
What is Continuous Deployment?
What is Continuous Delivery?
What DevOps helps us to achieve?
What do you consider as best practices for CI/CD?
What are the anti-patterns of DevOps?
What systems and/or tools are you using for the following?:
* CI/CD
* Provisioning infrastructure
* Configuration Management
* Monitoring & alerting
* Logging
* Code review
* Code coverage
* Tests
* CI/CD - Jenkins, Circle CI, Travis
* Provisioning infrastructure - Terraform, CloudFormation
* Configuration Management - Ansible, Puppet, Chef
* Monitoring & alerting - Prometheus, Nagios
* Logging - Logstash, Graylog, Fluentd
* Code review - Gerrit, Review Board
* Code coverage - Cobertura, Clover, JaCoCo
* Tests - Robot, Serenity, Gauge
What are you taking into consideration when choosing a tool/technology?
You may use one or all of the following:
* mature vs. cutting edge
* community size
* architecture aspects - agent vs. agentless, master vs. masterless, etc.
What is the difference between SQL and NoSQL?
What the difference between VPN and VPS?
What is the difference between SSH and SSL?
What scripting language are you familiar with? why specifically this one?
How long do you think it would take you to learn another language?
Explain mutable vs. immutable infrastructure
In mutable infrastructure paradigm, changes applied on top of the existing infrastructure and over time
the infrastructure builds up a history of changes. Ansible, Puppet and Chef are examples to tools which
follow mutable infrastructure paradigm.
In immutable infrastructure paradigm, every change is actually new infrastructure. So a change
to a server will result in a new server instead of updating it. Terraform is an example of technology
which follows the mutable infrastructure paradigm.
What ways are you familiar with to deliver a software?
* Archive - collect all your app files into one archive (e.g. tar) and deliver it to the user.
* Package - depends on the OS, you can use your OS package format (e.g. in RHEL/Fefodra it's RPM) to deliver your software with a way to install, uninstall and update it using the standard packager commands
* Images - Either VM or container images where your package is included with everything it needs in order to run successfully.
What is caching? How it works? Why is it important?
Explain stateless vs. stateful
What is HTTP and how it works?
#### :star: Advanced
Tell me how you perform plan capacity for your CI/CD resources (e.g. servers, storage, etc.)
How would you structure/implement CD for an application which depends on several other applications?
How do you manage dependencies?
Explain what are design patterns. Which design patterns are you familiar with?
How do you measure your CI/CD quality? Are there any metrics you are using?
What is a configuration drift? What problems is it causing?
Configuration drift happens when in an environment of servers with the exact same configuration and software, a certain server
or servers are being applied with updates or configuration which other servers don't get and over time these servers become
slightly different than all others.
This situation might lead to bugs which hard to identify and reproduce.
How to deal with configuration drift?
Do you have experience with testing cross-projects changes? (aka cross-dependency)
Note: cross-dependency is when you have two or more changes to separate projects and you would like to test them in mutual build instead of testing each change separately.
In what scenarios would you prefer to use SQL?
* Homogeneous data, no changes anticipated
* ACID compliance is important
## Jenkins
#### :baby: Beginner
Why are you using Jenkins? What are the advantages of Jenkins over its competitors?
What is a plugin?
What plugins are you using in Jenkins? Which do you consider to be the most useful?
Explain CI/CD and how you implemented it in Jenkins
What type of jobs there are? what is the advantage of each type?
What ways are you familiar with to notify users on build results?
How to secure Jenkins?
#### :star: Advanced
How to acquire multiple slaves for one specific build?
There are four teams in your organization. How to prioritize the builds of each team? So the jobs of team x will always run before team y
Do you have experience with developing a Jenkins plugin? Can you describe this experience?
If you are managing a dozen of jobs, you can probably use the Jenkins UI. How do you manage the creation and deletion of hundreds of jobs every week/month?
How would you implement an option of a starting a build from a certain stage and not from the beginning?
What are some of Jenkins limitations?
## AWS
#### :baby: Beginner
##### Global Infrastructure
Explain the following
* Availability zone
* Region
* Edge location
##### S3
Explain what is S3 and what is it used for
What is a bucket?
True or False? a bucket name must be globally unique
True
What objects in S3 consists of?
* Another way to ask it: explain key, value, version id and metadata in context of objects
Explain data consistency
Can you host dynamic websites on s3? what about static websites?
What security measures have you taken in context of S3?
##### CloudFront
Explain what is CloudFront and what is it used for
Explain the following
* Origin
* Edge location
* Distribution
What delivery methods available for the user with CDN?
True or False? objects are cached for the life of TTL
##### EC2
What type of instances have you created?
How to increase RAM for a given EC2 instance?
Stop the instance, the type of the instance to match the desired RAM and start the instance.
## Network
#### :baby: Beginner
Explain the OSI model. What layers there are? What each layer is responsible for?
Application: user end (HTTP is here)
Presentation: establishes context between application-layer entities (Encryption is here)
Session: establishes, manages and terminates the connections
Transport: transfers variable-length data sequences from a source to a destination host (TCP & UDP are here)
Network: transfers datagrams from one network to another (IP is here)
Data link: provides a link between two directly connected nodes (MAC is here)
Physical: the electrical and physical spec the data connection (Bits are here)
What delivery schemes are you familiar with?
Unitcast: One to one communication where there is one sender and one reciever.
Broadcast: Sending a message to everone in the network. The address ff:ff:ff:ff:ff:ff is used for broadcasting.
Two common protocols which use broadcast are ARP and DHCP.
Multicast: Sending a message to a group of subscribers. It can be one-to-many or many-to-many.
What is CSMA/CD? Is it used in modern ethernet networks?
CSMA/CD stands for Carrier Sense Multiple Access / Collision Detection.
Its primarily focus it to manage access to shared medium/bus where only one host can transmit at a given point of time.
CSMA/CD algorithm:
1. Before sending a frame, it checks whether another host already transmitting a frame.
2. If no one transmitting, it starts transmitting the frame.
3. If two hosts transmitted at the same time, we have a collision.
4. Both hosts stop sending the frame and they send to everyone a 'jam signal' notifying everyone that a collision occured
5. They are waiting for a random time before sending again
6. Once each host waited for a raondom time, they try to send the frame again and so the
Describe the following network devices and the difference between them: router, switch and hub
What is NAT?
What is the difference between TCP and UDP?
What is ARP? How it works?
What is DHCP? How it works?
What is SSL tunneling? How it works?
What is a socket?
What is IPv6? Why should we consider using it if we have IPv4?
What is VLAN?
What is MTU?
What is SDN?
What is ICMP?
What is NAT? How it works?
## Linux
#### :baby: Beginner
Explain what each of the following commands does and given an example on how to use it:
* ls
* rm
* rmdir (can you achieve the same result by using rm?)
* grep
* wc
* df
How to make sure a service will start on a OS of your choice?
How do you schedule tasks periodically?
You can use the commands cron and at.
With cron, tasks are scheduled using the following format:
The tasks are stored in a cron file.
How to change the permissions of a file?
What does the following permissions mean?:
* 777
* 644
* 750
How to add a new user to the system without providing him the ability to log-in into the system?
What commands are you using for troubleshooting issues? specifically:
* Disk issues
* Memory, CPU issues
* Networking issues
What is the difference between Linux and Unix?
What is a Linux kernel module and how do you load a new module?
What is KVM?
Explain redirection
What is an exit code? What exit codes are you familiar with?
An exit code (or return code) represents the code returned by a child process to its
parent process.
0 is an exit code which represents success while anything higher than 1 represents error.
Each number has different meaning, based on how the application was developed.
I consider this as a good blog post to read more about it: https://shapeshed.com/unix-exit-codes
How to grep two strings?
What is the different between a soft link and hard link?
hard link is the same file, using the same inode.
soft link is a shortcut to another file, using a different inode.
soft links can be created between different file systems while
hard link can be created only within the same file system.
What RAID is used for? Can you explain the differences between RAID 0, 1, 5 and 10?
Fix the following commands:
* sed "s/1/2/g' /tmp/myFile
* find . -iname \*.yaml -exec sed -i "s/1/2/g" {} ;
Explain what is stored in each of the following paths and if there is anything unique about it:
* /tmp
* /var/log
* /bin
* /proc
* /usr/local
How to run a process in the background and why to do that in the first place?
You can achieve that by specifying & at end of the command.
As to why, since some commands/processes can take a lot of time to finish
execution or run forever
What signal is used when you run 'kill '?
The default signal is SIGTERM (15). This signal kills
process gracefully which means it allows it to save current
state configuration.
What signals are you familiar with?
SIGTERM - default signal for terminating a process
SIGHUP - common usage is for reloading configuration
SIGKILL - a signal which cannot caught or ignored
To view all available signals run `kill -l`
In what state a process in Linux can be?
Ready
Running
Blocked
Terminated
Zombie
Find all files which end with '.yml' and replace the number 1 in 2 in each file
find /some_dir -iname \*.yml -exec sed -i "s/1/2/g" {} \;
How to check how much free memory a system has? How to check memory consumption by each process?
You can use the commands top and free
How would you split a 50 lines file into 2 files of 25 lines each?
You can use the split command this way: split -l 25 some_file
What is a file descriptor? What file descriptors are you familiar with?
File descriptor, also known as file handler, is a unique number which identifies an open file in the operating system.
In Linux (and Unix) the first three file descriptors are:
* 0 - the default data stream for input
* 1 - the default data stream for output
* 2 - the default data stream for output related to errors
This is a great article on the topic: https://www.computerhope.com/jargon/f/file-descriptor.htm
What's an inode?
For each file (and directory) in Linux there is an inode, a data structure which stores metadata
related to the file like its size, owner, permissions, etc.
How to list ports which being currently used?
DNS: What is a "A record"?
DNS: What is a PTR?
While an A record points a domain name to an IP address, a PTR record does the opposite and resolves the IP address to a domain name.
DNS: What is a MX record?
DNS: is it using TCP or UDP?
#### :star: Advanced
How to delete the last word from each line in a file?
sed "s/\s*\w\+\s*$//" file
When you run ip a you see there is a device called 'lo'. What is it and why do we need it?
What traceroute command does? How it works?
What is the routing table? How do you view it?
What are cgroups? In what scenario would you use them?
How to create a file of a certain size?
There are a couple of ways to do that:
* dd if=/dev/urandom of=new_file.txt bs=2MB count=1
* truncate -s 2M new_file.txt
* fallocate -l 2097152 new_file.txt
## Ansible
Describe each of the following components in Ansible, including the relationship between them:
* Task
* Module
* Play
* Playbook
* Role
Task – a call to a specific Ansible module
Module – the actual unit of code executed by Ansible on your own host or a remote host. Modules are indexed by category (database, file, network, …) and also referred as task plugins.
Play – One or more tasks executed on a given host(s)
Playbook – One or more plays. Each play can be executed on the same or different hosts
Role – Ansible roles allows you to group resources based on certain functionality/service such that they can be easily reused. In a role, you have directories for variables, defaults, files, templates, handlers, tasks, and metadata. You can then use the role by simply specifying it in your playbook.
What is an inventory file and how you define one?
An inventory file defines hosts and/or groups of hosts on which Ansible tasks executed upon.
An example of inventory file:
192.168.1.2
192.168.1.3
192.168.1.4
[web_servers]
190.40.2.20
190.40.2.21
190.40.2.22
What is a dynamic inventory file? When you would use one?
A dynamic inventory file tracks hosts from one or more sources like cloud providers and CMDB systems.
You should use one when using external sources and especially when the hosts in your environment are being automatically
spun up and shut down, without you tracking every change in these sources.
You want to run Ansible playbook only on specific minor version of your OS, how would you achieve that?
Write a task to create the directory ‘/tmp/new_directory’
```
- name: Create a new directory
file:
path: "/tmp/new_directory"
state: directory
```
What would be the result of the following play?
```
---
- name: Print information about my host
hosts: localhost
gather_facts: 'no'
tasks:
- name: Print hostname
debug:
msg: "It's me, {{ ansible_hostname }}"
```
When given a written code, always inspect it thoroughly. If your answer is “this will fail” then you are right. We are using a fact (ansible_hostname), which is a gathered piece of information from the host we are running on. But in this case, we disabled facts gathering (gather_facts: no) so the variable would be undefined which will result in failure.
Write a playbook to install ‘zlib’ and ‘vim’ on all hosts if the file ‘/tmp/mario’ exists on the system.
```
---
- hosts: all
vars:
mario_file: /tmp/mario
package_list:
- 'zlib'
- 'vim'
tasks:
- name: Check for mario file
stat:
path: "{{ mario_file }}"
register: mario_f
- name: Install zlib and vim if mario file exists
become: "yes"
package:
name: "{{ item }}"
state: present
with_items: "{{ package_list }}"
when: mario_f.stat.exists
```
Write a playbook to deploy the file ‘/tmp/system_info’ on all hosts except for controllers group, with the following content
```
I'm and my operating system is
```
replace and with the actual data for the specific host you are running on
The playbook to deploy the system_info file
```
---
- name: Deploy /tmp/system_info file
hosts: all:!controllers
tasks:
- name: Deploy /tmp/system_info
template:
src: system_info.j2
dest: /tmp/system_info
```
The content of the system_info.j2 template
```
# {{ ansible_managed }}
I'm {{ ansible_hostname }} and my operating system is {{ ansible_distribution }
```
The variable 'whoami' defined in the following places:
* role defaults -> whoami: mario
* extra vars (variables you pass to Ansible CLI with -e) -> whoami: toad
* host facts -> whoami: luigi
* inventory variables (doesn’t matter which type) -> whoami: browser
According to variable precedence, which one will be used?
The right answer is ‘toad’.
Variable precedence is about how variables override each other when they set in different locations. If you didn’t experience it so far I’m sure at some point you will, which makes it a useful topic to be aware of.
In the context of our question, the order will be extra vars (always override any other variable) -> host facts -> inventory variables -> role defaults (the weakest).
A full list can be found at the link above. Also, note there is a significant difference between Ansible 1.x and 2.x.
## Terraform
#### :baby: Beginner
Can you explain what is Terraform? How it works?
Read [here](https://www.terraform.io/intro/index.html#what-is-terraform-)
What benefits infrastructure-as-code has?
- fully automated process of provisioning, modifying and deleting your infrastructure
- version control for your infrastructure which allows you to quickly rollback to previous versions
- validate infrastructure quality and stability with automated tests and code reviews
- makes infrastructure tasks less repetitive
Why Terraform and not other technologies? (e.g. Ansible, Puppet, CloufFormation)
A common *wrong* answer is to say that Ansible and Puppet are configuration management tools
and Terraform is a provisioning tool. While technically true, it doesn't mean Ansible and Puppet can't
be used for provisioning infrastructure. Also, it doesn't explains why Terraform should be used over
CloudFormation if at all.
The benefits of Terraform over the other tools:
* it follows the immutable infrastructure approach which has benefits like avoiding a configuration drift over time
* Ansible and Puppet are more procedural (you mention what to execute in each step) and Terraform is declartive since you describe the overall desired state and not per resource or task. You can give the example of going from 1 to 2 servers in each tool. In terrform you specify 2, in Ansible and puppet you have to only provision 1 additional server
Explain what the following commands do:
* terraform init
* terraform plan
* terraform apply
terraform init scans your code to figure which providers are you using and download them.
terraform plan will let you see what terraform is about to do before actually doing it.
terraform apply will provision the resources specified in the .tf files.
How to write down a variable which changes by an external source or during terraform apply?
You use it this way: variable “my_var” {}
## Docker
#### :baby: beginner
How containers are different from VMs?
The primary difference between containers and VMs is that containers allow you to virtualize
multiple workloads on the operating system while in the case of VMs the hardware is being virtualized to
run multiple machines each with its own OS.
In which scenarios would you use containers and in which you would prefer to use VMs?
You should choose VMs when:
* you need run an application which requires all the resources and functionalilies of an OS
* you need full isolation and security
You should choose containers when:
* you need a lightweight solution that quickly starts
* Running multiple versions or instances of a single application
What happens when you run `docker run hello-world`?
Docker CLI passes your request to Docker daemon.
Docker daemon downloads the image from Docker Hub
Docker daemon creates a new container by using the image it downloaded
Docker daemon redirects output from container to Docker CLI which redirects it to the standard output
How do you run a container?
What do you see when you run `docker ps`?
What `docker commit` does? when will you use it?
How would you transfer data from one container into another?
What is the difference between ADD and COPY in Dockerfile?
What is the difference between CMD and RUN in Dockerfile?
Explain what is Docker compose and what is it used for
What are the differences between Docker compose, Docker swarm and Kuberenets?
Explain Docker interlock
What is the difference between Docker Hub and Docker cloud?
Docker Hub is a native Docker registry service which allows you to run pull
and push commands to install and deploy Docker images from the Docker Hub.
Docker Cloud is built on top of the Docker Hub so Docker Cloud provides
you with more options/features compared to Docker Hub. One example is
Swarm management which means you can create new swarms in Docker Cloud.
Explain image layers
## Kubernetes
What is Kubernetes?
Why Docker isn't enough? Why do we need Kubernetes?
Describe the architecture of Kuberenets
How do you monitor your Kuberenets?
What is kubectl? How do you use it?
What is kubconfig? What do you use it for?
How do you create users? Where user information is stored?
## Python
#### :baby: Beginner
What data type supported in Python and which of them are mutable?
What function can you use to show that a certain data type is mutable?
The mutable data types are:
List
Dictionary
Set
The immutable data types are:
Numbers (int, float, ...)
String
Bool
Tuple
The id function can be used to check if a given variable is mutable or not.
What is PEP8? Give an example of 5 style guidelines
PEP8 is a list of coding conventions and style guidelines for Python
5 style guidelines:
1. Limit all lines to a maximum of 79 characters.
2. Surround top-level function and class definitions with two blank lines.
3. Use commas when making a tuple of one element
4. Use spaces (and not tabs) for indentation
5. Use 4 spaces per indentation level
Write a program which will revert a string (e.g. pizza -> azzip)
```
Shortest way is str[::-1]
"Classic" way:
```
What _ is used for in Python?
1. Translation lookup in i18n
2. Hold the result of the last executed expression or statement
3. As a general purpose "throwaway" variable name. For example: x, y, _ = get_data() (x and y are used but since we don't care about third variable, we "threw it away").
Sort a list of lists by the second item of each nested list
```
li = [[1, 4], [2, 1], [3, 9], [4, 2], [4, 5]]
sorted(x, key=lambda l: l[1])
```
You have the following list: [{'name': 'Mario', 'food': ['mushrooms', 'goombas']}, {'name': 'Luigi', 'food': ['mushrooms', 'turtles']}]
Extract all type of foods. Final output should be: {'mushrooms', 'goombas', 'turtles'}
```
set([food for bro in x for food in bro['food']])
```
Explain what is GIL
## Prometheus
Describe the following Prometheus components:
- Prometheus server
- Push Gateway
- Alert Manager
Prometheus server responsible for scraping the storing the data
Push gateway is used for short-lived jobs
Alert manager is responsible for alerts ;)
What is an exporter? What is it used for?
## Git
#### :baby: Beginner
What is the difference between git pull and git fetch?
Shortly, git pull = git fetch + git merge
When you run git pull, it gets all the changes from the remote or central
repository and attaches it to your corresponding branch in your local reposistory.
git fetch gets all the changes from the remote repository, stores the changes in
a separate branch in your local repository
Explain the following: git directory, working directory and staging area
The Git directory is where Git stores the metadata and object database for your project. This is the most important part of Git, and it is what is copied when you clone a repository from another computer.
The working directory is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.
The staging area is a simple file, generally contained in your Git directory, that stores information about what will go into your next commit. It’s sometimes referred to as the index, but it’s becoming standard to refer to it as the staging area.
This answer taken from [git-scm.com](https://git-scm.com/book/en/v1/Getting-Started-Git-Basics#_the_three_states)
How to resolve git merge conflicts?
First, you open the files which are in conflict and identify what are the conflicts.
Next, based on what is accepted in your company or team, you either discuss with your
colleagues on the conflicts or resolve them by yourself
After resolving the conflicts, you add the files with `git add `
Finally, you run `git rebase --continue`
What is the difference between git reset and git revert?
`git revert` creates a new commit which undoes the changes from last commit.
`git reset` depends on the usage, can modify the index or change the commit which the branch head
is currently pointing at.
You would like to move forth commit to the top. How would you achieve that?
Using git rebase> command
In what situations are you using git rebase?
What merge strategies are you familiar with?
Mentioning two or three should be enough and it's probably good to mention that 'recursive' is the default one.
recursive
resolve
ours
theirs
This page explains it the best: https://git-scm.com/docs/merge-strategies
#### :star: Advanced
Explain Git octopus merge
Probably good to mention that it's:
* It's good for cases of merging more than one branch (and also the default of such use cases)
* It's primarily meant for bundling topic branches together
This is a great article about Octopus merge: http://www.freblogg.com/2016/12/git-octopus-merge.html
## Go
#### :baby: Beginner
What are some characteristics of the Go programming language?
* Strong and static typing - the type of the variables can't be changed over time and they have to be defined at compile time
* Simplicity
* Fast compile times
* Built-in concurrency
* Garbage collected
* Platform independent
* Compile to standalone binary - anything you need to run your app will be compiled into one binary. Very useful for version management in run-time.
Go also has good community.
What is the difference between var x int = 2 and x := 2?
The result is the same, a variable with the variable 2.
with var x int = 2 we are setting the variable type to integer while with x := 2 we are letting Go figure out by itself the type.
What libraries of Go have you used?
This should be answered based on your usage but some examples are:
* fmt - formatted I/O
Write an "hello world" program?
```
package main
import "fmt"
func main() {
fmt.Println("Hello World")
}
```
## Mongo
#### :baby: Beginner
In what scenarios would you prefer to use NoSQL/Mongo over SQL?
* Heterogeneous data which changes often
* Data consistency and integrity is not top priority
What is a document? What is a collection?
What is an aggregator?
## OpenShift
#### :baby: Beginner
What is OpenShift? Did you use it? If yes, how?
Can you explain the difference between OpenShift and Kubernetes?
Define Pods and explain what are stateful pods
What types of build strategies are you familiar with?
Explain what are labels and what they are used for
Explain what are annotations and how they are different from labels
Explain what is Downward API
## Shell Scripting
#### :baby: Beginner
Tell me about your experience with shell scripting
What this line in scripts mean?: #!/bin/bash
What do you tend to include in every script you write?
True or False?: when a certain command/line fails, the script, by default, will exit and will no keep running
Depends on the language and settings used but in Bash for example, by default the script will keep running.
Today we have tools and technologies like Ansible. Why would someone still use shell scripting?
Explain what would be the result of each command:
* echo $0
* echo $?
* echo $$
* echo $@
* echo $#
How do you debug shell scripts?
How do you get input from the user in shell scripts?
Explain conditionals and how do you use them
What is a loop? What types of loops are you familiar with?
Explain continue and break. When do you use them if at all?
How do you check variable length?
## Scenarios
Scenarios are questions which combine several subjects together. Some scenarios will
require from you to design, plan and implement environments with different constraints
and considerations.
* [Elasticsearch & Kibana on AWS](scenarios/elk_kibana_aws.md)
* [Ansible, Minikube and Docker](scenarios/ansible_minikube_docker.md)
* [Cloud Slack bot](scenarios/cloud_slack_bot.md)
* [Writing Jenkins Scripts](scenarios/jenkins_scripts.md)
* [Writing Jenkins Pipelines](scenarios/jenkins_pipelines.md)
