* You would like to run a certain application in a container on multiple different locations and sync changes across all of them, no matter where they run
* Performing updates and changes across hundreds of containers
- If you are a small team (like less than 20 engineers) running less than a dozen of containers, Kubernetes might be an overkill (even if you need scale, rolling out updates, etc.). You might still enjoy the benefits of using managed Kubernetes, but you definitely want to think about it carefully before making a decision on whether to adopt it.
<summary>What are some of Kubernetes features?</summary><br><b>
- Self-Healing: Kubernetes uses health checks to monitor containers and run certain actions upon failure or other type of events, like restarting the container
- Load Balancing: Kubernetes can split and/or balance requests to applications running in the cluster, based on the state of the Pods running the application
- Operators: Kubernetes packaged applications that can use the API of the cluster to update its state and trigger actions based on events and application state changes
- Automated Rollout: Gradual updates roll out to applications and support in roll back in case anything goes wrong
- Scaling: Scaling horizontally (down and up) based on different state parameters and custom defined criteria
- Secrets: you have a mechanism for storing user names, passwords and service endpoints in a private way, where not everyone using the cluster are able to view it
Kubectl is the Kubernetes command line tool that allows you to run commands against Kubernetes clusters. For example, you can use kubectl to deploy applications, inspect and manage cluster resources, and view logs.
<summary>What is a Kubernetes Cluster?</summary><br><b>
Red Hat Definition: "A Kubernetes cluster is a set of node machines for running containerized applications. If you’re running Kubernetes, you’re running a cluster.
At a minimum, a cluster contains a worker node and a master node."
Read more [here](https://www.redhat.com/en/topics/containers/what-is-a-kubernetes-cluster)
</b></details>
<details>
<summary>What is a Node?</summary><br><b>
A node is a virtual or a physical machine that serves as a worker for running the applications.<br>
It's recommended to have at least 3 nodes in a production environment.
</b></details>
<details>
<summary>What the master node is responsible for?</summary><br><b>
The master coordinates all the workflows in the cluster:
<summary>You are managing multiple Kubernetes clusters. How do you quickly change between the clusters using kubectl?</summary><br><b>
`kubectl config use-context`
</b></details>
<details>
<summary>How do you prevent high memory usage in your Kubernetes cluster and possibly issues like memory leak and OOM?</summary><br><b>
Apply requests and limits, especially on third party applications (where the uncertainty is even bigger)
</b></details>
<details>
<summary>Do you have experience with deploying a Kubernetes cluster? If so, can you describe the process in high-level?</summary><br><b>
1. Create multiple instances you will use as Kubernetes nodes/workers. Create also an instance to act as the Master. The instances can be provisioned in a cloud or they can be virtual machines on bare metal hosts.
2. Provision a certificate authority that will be used to generate TLS certificates for the different components of a Kubernetes cluster (kubelet, etcd, ...)
1. Generate a certificate and private key for the different components
3. Generate kubeconfigs so the different clients of Kubernetes can locate the API servers and authenticate.
4. Generate encryption key that will be used for encrypting the cluster data
If you are a Kubernetes beginner you should know that this is not a common way to run Pods. The common way is to run a Deployment which in turn runs Pod(s).
Pods are usually indeed not created directly. You'll notice that Pods are usually created as part of another entities such as Deployments or ReplicaSets.
If a Pod dies, Kubernetes will not bring it back. This is why it's more useful for example to define ReplicaSets that will make sure that a given number of Pods will always run, even after a certain Pod dies.
</b></details>
<details>
<summary>How many containers can a pod contain?</summary><br><b>
A pod can include multiple containers but in most cases it would probably be one container per pod.
There are some patterns where it makes to run more than one container like the "side-car" pattern where you might want to perform logging or some other operation that is executed by another container running with your app container in the same Pod.
* Succeeded - Every container in the Pod terminated with success
* Unknown - Pod's state could not be obtained
* Pending - Containers are not yet running (Perhaps images are still being downloaded or the pod wasn't scheduled yet)
</b></details>
<details>
<summary>True or False? By default, pods are isolated. This means they are unable to receive traffic from any source</summary><br><b>
False. By default, pods are non-isolated = pods accept traffic from any source.
</b></details>
<details>
<summary>True or False? The "Pending" phase means the Pod was not yet accepted by the Kubernetes cluster so the scheduler can't run it unless it's accepted</summary><br><b>
False. "Pending" is after the Pod was accepted by the cluster, but the container can't run for different reasons like images not yet downloaded.
<summary>How to confirm a container is running after running the command <code>kubectl run web --image nginxinc/nginx-unprivileged</code></summary><br><b>
* When you run `kubectl describe pods <POD_NAME>` it will tell whether the container is running:
`Status: Running`
* Run a command inside the container: `kubectl exec web -- ls`
</b></details>
<details>
<summary>After running <code>kubectl run database --image mongo</code> you see the status is "CrashLoopBackOff". What could possibly went wrong and what do you do to confirm?</summary><br><b>
"CrashLoopBackOff" means the Pod is starting, crashing, starting...and so it repeats itself.<br>
There are many different reasons to get this error - lack of permissions, init-container misconfiguration, persistent volume connection issue, etc.
One of the ways to check why it happened it to run `kubectl describe po <POD_NAME>` and having a look at the exit code
```
Last State: Terminated
Reason: Error
Exit Code: 100
```
Another way to check what's going on, is to run `kubectl logs <POD_NAME>`. This will provide us with the logs from the containers running in that Pod.
</b></details>
<details>
<summary>Explain the purpose of the following lines
```
livenessProbe:
exec:
command:
- cat
- /appStatus
initialDelaySeconds: 10
periodSeconds: 5
```
</summary><br><b>
These lines make use of `liveness probe`. It's used to restart a container when it reaches a non-desired state.<br>
In this case, if the command `cat /appStatus` fails, Kubernetes will kill the container and will apply the restart policy. The `initialDelaySeconds: 10` means that Kubelet will wait 10 seconds before running the command/probe for the first time. From that point on, it will run it every 5 seconds, as defined with `periodSeconds`
</b></details>
<details>
<summary>Explain the purpose of the following lines
```
readinessProbe:
tcpSocket:
port: 2017
initialDelaySeconds: 15
periodSeconds: 20
```
</summary><br><b>
They define a readiness probe where the Pod will not be marked as "Ready" before it will be possible to connect to port 2017 of the container. The first check/probe will start after 15 seconds from the moment the container started to run and will continue to run the check/probe every 20 seconds until it will manage to connect to the defined port.
</b></details>
<details>
<summary>What does the "ErrImagePull" status of a Pod means?</summary><br><b>
It wasn't able to pull the image specified for running the container(s). This can happen if the client didn't authenticated for example.<br>
More details can be obtained with `kubectl describe po <POD_NAME>`.
</b></details>
<details>
<summary>What happens when you delete a Pod?</summary><br><b>
1. The `TERM` signal is sent to kill the main processes inside the containers of the given Pod
2. Each container is given a period of 30 seconds to shut down the processes gracefully
3. If the grace period expires, the `KILL` signal is used to kill the processes forcefully and the containers as well
</b></details>
<details>
<summary>Explain liveness probes</summary><br><b>
Liveness probes is a useful mechanism used for restarting the container when a certain check/probe, the user has defined, fails.<br>
For example, the user can define that the command `cat /app/status` will run every X seconds and the moment this command fails, the container will be restarted.
You can read more about it in [kubernetes.io](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes)
<summary>Why it's common to have only one container per Pod in most cases?</summary><br><b>
One reason is that it makes it harder to scale when you need to scale only one of the containers in a given Pod.
</b></details>
<details>
<summary>True or False? Once a Pod is assisgned to a worker node, it will only run on that node, even if it fails at some point and spins up a new Pod</summary><br><b>
[Kubernetes.io](https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/): "Static Pods are managed directly by the kubelet daemon on a specific node, without the API server observing them. Unlike Pods that are managed by the control plane (for example, a Deployment); instead, the kubelet watches each static Pod (and restarts it if it fails)."
</b></details>
<details>
<summary>True or False? The same as there are "Static Pods" there are other static resources like "deployments" and "replicasets"</summary><br><b>
False.
</b></details>
<details>
<summary>What are some use cases for using Static Pods?</summary><br><b>
One clear use case is running Control Plane Pods - running Pods such as kube-apiserver, scheduler, etc. These should run and operate regardless of whether some components of the cluster work or not and they should run on specific nodes of the cluster.
</b></details>
<details>
<summary>How to identify which Pods are Static Pods?</summary><br><b>
The suffix of the Pods is the same as the name of the nodes on which they are running
TODO: check if it's always the case.
</b></details>
<details>
<summary>Which of the following is not a static pod?:
* kube-scheduler
* kube-proxy
* kube-apiserver
</summary><br><b>
kube-proxy - it's a DaemonSet (since it has to be presented on every node in the cluster). There is no one specific node on which it has to run.
</b></details>
<details>
<summary>Where static Pods manifests are located?</summary><br><b>
Most of the time it's in /etc/kubernetes/manifests but you can verify with `grep -i static /var/lib/kubelet/config.yaml` to locate the value of `statisPodsPath`.
It might be that your config is in different path. To verify run `ps -ef | grep kubelet` and see what is the value of --config argument of the process `/usr/bin/kubelet`
The key itself for defining the path of static Pods is `staticPodPath`. So if your config is in `/var/lib/kubelet/config.yaml` you can run `grep staticPodPath /var/lib/kubelet/config.yaml`.
<summary>How to check to which worker node the pods were scheduled to? In other words, how to check on which node a certain Pod is running?</summary><br><b>
<summary>You try to run a Pod but it's in "Pending" state. What might be the reason?</summary><br><b>
One possible reason is that the scheduler which supposed to schedule Pods on nodes, is not running. To verify it, you can run `kubectl get po -A | grep scheduler` or check directly in `kube-system` namespace.
<summary>Create a static pod with the image <code>python</code> that runs the command <code>sleep 2017</code></summary><br><b>
First change to the directory tracked by kubelet for creating static pod: `cd /etc/kubernetes/manifests` (you can verify path by reading kubelet conf file)
Now create the definition/manifest in that directory
[Kubernetes.io](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/): "Labels are key/value pairs that are attached to objects, such as pods. Labels are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system. Labels can be used to organize and to select subsets of objects. Labels can be attached to objects at creation time and subsequently added and modified at any time. Each object can have a set of key/value labels defined. Each Key must be unique for a given object."
</b></details>
<details>
<summary>Explain selectors</summary><br><b>
[Kubernetes.io](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors): "Unlike names and UIDs, labels do not provide uniqueness. In general, we expect many objects to carry the same label(s).
Via a label selector, the client/user can identify a set of objects. The label selector is the core grouping primitive in Kubernetes.
The API currently supports two types of selectors: equality-based and set-based. A label selector can be made of multiple requirements which are comma-separated. In the case of multiple requirements, all must be satisfied so the comma separator acts as a logical AND (&&) operator."
</b></details>
<details>
<summary>Provide some actual examples of how labels are used</summary><br><b>
* Can be used by the scheduler to place certain Pods (with certain labels) on specific nodes
* Used by replicasets to track pods which have to be scaled
</b></details>
<details>
<summary>What are Annotations?</summary><br><b>
[Kubernetes.io](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/): "You can use Kubernetes annotations to attach arbitrary non-identifying metadata to objects. Clients such as tools and libraries can retrieve this metadata."
</b></details>
<details>
<summary>How annotations different from labels?</summary><br><b>
[Kuberenets.io](Labels can be used to select objects and to find collections of objects that satisfy certain conditions. In contrast, annotations are not used to identify and select objects. The metadata in an annotation can be small or large, structured or unstructured, and can include characters not permitted by labels.): "Labels can be used to select objects and to find collections of objects that satisfy certain conditions. In contrast, annotations are not used to identify and select objects. The metadata in an annotation can be small or large, structured or unstructured, and can include characters not permitted by labels."
<summary>What is a "Deployment" in Kubernetes?</summary><br><b>
AKubernetes Deploymentis used to tellKuberneteshow to create or modify instances of the pods that hold a containerized application.
Deploymentscan scale the number of replica pods, enable rollout of updated code in a controlled manner, or roll back to an earlierdeploymentversion if necessary.
A Deployment is a declarative statement for the desired state for Pods and Replica Sets.
This command lists all the Deployment objects created and exist in the cluster. It doesn't mean the deployments are readt and running. This can be checked with the "READY" and "AVAILABLE" columns.
"An abstract way to expose an application running on a set of Pods as a network service." - read more [here](https://kubernetes.io/docs/concepts/services-networking/service)
<summary>What is the default service type in Kubernetes and what is it used for?</summary><br><b>
The default is ClusterIP and it's used for exposing a port internally. It's useful when you want to enable internal communication between Pods and prevent any external access.
It exposes a ReplicaSet by creating a service called 'replicaset-svc'. The exposed port is 2017 (this is the port used by the application) and the service type is NodePort which means it will be reachable externally.
</b></details>
<details>
<summary>True or False? the target port, in the case of running the following command, will be exposed only on one of the Kubernetes cluster nodes but it will routed to all the pods
False. It will be exposed on every node of the cluster and will be routed to one of the Pods (which belong to the ReplicaSet)
</b></details>
<details>
<summary>How to verify that a certain service configured to forward the requests to a given pod</summary><br><b>
Run `kubectl describe service` and see if the IPs from "Endpoints" match any IPs from the output of `kubectl get pod -o wide`
</b></details>
<details>
<summary>Explain what will happen when running apply on the following block
```
apiVersion: v1
kind: Service
metadata:
name: some-app
spec:
type: NodePort
ports:
- port: 8080
nodePort: 2017
protocol: TCP
selector:
type: backend
service: some-app
```
</summary><br><b>
It creates a new Service of the type "NodePort" which means it can be used for internal and external communication with the app.<br>
The port of the application is 8080 and the requests will forwarded to this port. The exposed port is 2017. As a note, this is not a common practice, to specify the nodePort.<br>
The port used TCP (instead of UDP) and this is also the default so you don't have to specify it.<br>
The selector used by the Service to know to which Pods to forward the requests. In this case, Pods with the label "type: backend" and "service: some-app".<br>
</b></details>
<details>
<summary>How to turn the following service into an external one?
```
spec:
selector:
app: some-app
ports:
- protocol: TCP
port: 8081
targetPort: 8081
```
</summary><br><b>
Adding `type: LoadBalancer` and `nodePort`
```
spec:
selector:
app: some-app
type: LoadBalancer
ports:
- protocol: TCP
port: 8081
targetPort: 8081
nodePort: 32412
```
</b></details>
<details>
<summary>What would you use to route traffic from outside the Kubernetes cluster to services within a cluster?</summary><br><b>
Ingress
</b></details>
<details>
<summary>True or False? When "NodePort" is used, "ClusterIP" will be created automatically?</summary><br><b>
True
</b></details>
<details>
<summary>When would you use the "LoadBalancer" type</summary><br><b>
Mostly when you would like to combine it with cloud provider's load balancer
</b></details>
<details>
<summary>How would you map a service to an external address?</summary><br><b>
Using the 'ExternalName' directive.
</b></details>
<details>
<summary>Describe in detail what happens when you create a service</summary><br><b>
1. Kubectl sends a request to the API server to create a Service
2. The controller detects there is a new Service
3. Endpoint objects created with the same name as the service, by the controller
4. The controller is using the Service selector to identify the endpoints
5. kube-proxy detects there is a new endpoint object + new service and adds iptables rules to capture traffic to the Service port and redirect it to endpoints
6. kube-dns detects there is a new Service and adds the container record to the dns server
</b></details>
<details>
<summary>How to list the endpoints of a certain app?</summary><br><b>
`kubectl get ep <name>`
</b></details>
<details>
<summary>How can you find out information on a Service related to a certain Pod if all you can use is <code>kubectl exec <POD_NAME> -- </code></summary><br><b>
You can run `kubectl exec <POD_NAME> -- env` which will give you a couple environment variables related to the Service.<br>
Variables such as `[SERVICE_NAME]_SERVICE_HOST`, `[SERVICE_NAME]_SERVICE_PORT`, ...
</b></details>
<details>
<summary>Describe what happens when a container tries to connect with its corresponding Service for the first time. Explain who added each of the components you include in your description</summary><br><b>
- The container looks at the nameserver defined in /etc/resolv.conf
- The container queries the nameserver so the address is resolved to the Service IP
- Requests sent to the Service IP are forwarded with iptables rules (or other chosen software) to the endpoint(s).
Explanation as to who added them:
- The nameserver in the container is added by kubelet during the scheduling of the Pod, by using kube-dns
- The DNS record of the service is added by kube-dns during the Service creation
- iptables rules are added by kube-proxy during Endpoint and Service creation
<summary>Describe in high level what happens when you run <code>kubctl expose deployment remo --type=LoadBalancer --port 8080</code></summary><br><b>
1. Kubectl sends a request to Kubernetes API to create a Service object
2. Kubernetes asks the cloud provider (e.g. AWS, GCP, Azure) to provision a load balancer
3. The newly created load balancer forwards incoming traffic to relevant worker node(s) which forwards the traffic to the relevant containers
</b></details>
<details>
<summary>After creating a service that forwards incoming external traffic to the containerized application, how to make sure it works?</summary><br><b>
You can run `curl <SERIVCE IP>:<SERVICE PORT>` to examine the output.
<summary>An internal load balancer in Kubernetes is called <code>____</code> and an external load balancer is called <code>____</code></summary><br><b>
An internal load balancer in Kubernetes is called Service and an external load balancer is Ingress
From Kubernetes docs: "Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource."
Read more [here](https://kubernetes.io/docs/concepts/services-networking/ingress/)
</b></details>
<details>
<summary>Complete the following configuration file to make it Ingress
```
metadata:
name: someapp-ingress
spec:
```
</summary><br><b>
There are several ways to answer this question.
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: someapp-ingress
spec:
rules:
- host: my.host
http:
paths:
- backend:
serviceName: someapp-internal-service
servicePort: 8080
```
</b></details>
<details>
<summary>Explain the meaning of "http", "host" and "backend" directives
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: someapp-ingress
spec:
rules:
- host: my.host
http:
paths:
- backend:
serviceName: someapp-internal-service
servicePort: 8080
```
</summary><br><b>
host is the entry point of the cluster so basically a valid domain address that maps to cluster's node IP address<br>
the http line used for specifying that incoming requests will be forwarded to the internal service using http.<br>
backend is referencing the internal service (serviceName is the name under metadata and servicePort is the port under the ports section).
</b></details>
<details>
<summary>Why using a wildcard in ingress host may lead to issues?</summary><br><b>
The reason you should not wildcard value in a host (like `- host: *`) is because you basically tell your Kubernetes cluster to forward all the traffic to the container where you used this ingress. This may cause the entire cluster to go down.
</b></details>
<details>
<summary>What is Ingress Controller?</summary><br><b>
An implementation for Ingress. It's basically another pod (or set of pods) that does evaluates and processes Ingress rules and this it manages all the redirections.
There are multiple Ingress Controller implementations (the one from Kubernetes is Kubernetes Nginx Ingress Controller).
</b></details>
<details>
<summary>What are some use cases for using Ingress?</summary><br><b>
* Multiple sub-domains (multiple host entries, each with its own service)
* One domain with multiple services (multiple paths where each one is mapped to a different service/application)
</b></details>
<details>
<summary>How to list Ingress in your namespace?</summary><br><b>
kubectl get ingress
</b></details>
<details>
<summary>What is Ingress Default Backend?</summary><br><b>
It specifies what do with an incoming request to the Kubernetes cluster that isn't mapped to any backend (= no rule to for mapping the request to a service). If the default backend service isn't defined, it's recommended to define so users still see some kind of message instead of nothing or unclear error.
</b></details>
<details>
<summary>How to configure a default backend?</summary><br><b>
Create Service resource that specifies the name of the default backend as reflected in `kubectl describe ingress ...` and the port under the ports section.
<summary>True or False? When configuring Ingress with TLS, the Secret component must be in the same namespace as the Ingress component</summary><br><b>
True
</b></details>
<details>
<summary>Which Kubernetes concept would you use to control traffic flow at the IP address or port level? </summary><br><b>
You can speciy any other number, given that your application knows how to scale.
</b></details>
### ReplicaSets
<details>
<summary>What is the purpose of ReplicaSet?</summary><br><b>
[kubernetes.io](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset): "A ReplicaSet's purpose is to maintain a stable set of replica Pods running at any given time. As such, it is often used to guarantee the availability of a specified number of identical Pods."
In simpler words, a ReplicaSet will ensure the specified number of Pods replicas is running for a selected Pod. If there are more Pods than defined in the ReplicaSet, some will be removed. If there are less than what is defined in the ReplicaSet then, then more replicas will be added.
It defines a replicaset for Pods whose type is set to "backend" so at any given point of time there will be 2 concurrent Pods running.
</b></details>
<details>
<summary>What will happen when a Pod, created by ReplicaSet, is deleted directly with <code>kubectl delete po ...</code>?</summary><br><b>
The ReplicaSet will create a new Pod in order to reach the desired number of replicas.
</b></details>
<details>
<summary>True or False? If a ReplicaSet defines 2 replicas but there 3 Pods running matching the ReplicaSet selector, it will do nothing</summary><br><b>
False. It will terminate one of the Pods to reach the desired state of 2 replicas.
</b></details>
<details>
<summary>Describe the sequence of events in case of creating a ReplicaSet</summary><br><b>
* The client (e.g. kubectl) sends a request to the API server to create a ReplicaSet
* The Controller detects there is a new event requesting for a ReplicaSet
* The controller creates new Pod definitions (the exact number depends on what is defined in the ReplicaSet definition)
* The scheduler detects unassigned Pods and decides to which nodes to assign the Pods. This information sent to the API server
* Kubelet detects that two Pods were assigned to the node it's running on (as it constantly watching the API server)
* Kubelet sends requests to the container engine, to create the containers that are part of the Pod
* Kubelet sends a request to the API server to notify it the Pods were created
</b></details>
<details>
<summary>How to list ReplicaSets in the current namespace?</summary><br><b>
<summary>Is it possible to delete ReplicaSet without deleting the Pods it created?</summary><br><b>
Yes, with `--cascase=false`.
`kubectl delete -f rs.yaml --cascade=false`
</b></details>
<details>
<summary>What is the default number of replicas if not explicitly specified?</summary><br><b>
1
</b></details>
<details>
<summary>What the following output of <code>kubectl get rs</code> means?
NAME DESIRED CURRENT READY AGE
web 2 2 0 2m23s
</summary><br><b>
The replicaset `web` has 2 replicas. It seems that the containers inside the Pod(s) are not yet running since the value of READY is 0. It might be normal since it takes time for some containers to start running and it might be due to an error. Running `kubectl describe po POD_NAME` or `kubectl logs POD_NAME` can give us more information.
</b></details>
<details>
<summary>True or False? Pods specified by the selector field of ReplicaSet must be created by the ReplicaSet itself</summary><br><b>
False. The Pods can be already running and initially they can be created by any object. It doesn't matter for the ReplicaSet and not a requirement for it to acquire and monitor them.
</b></details>
<details>
<summary>True or False? In case of a ReplicaSet, if Pods specified in the selector field don't exists, the ReplicaSet will wait for them to run before doing anything</summary><br><b>
False. It will take care of running the missing Pods.
</b></details>
<details>
<summary>In case of a ReplicaSet, Which field is mandatory in the spec section?</summary><br><b>
The field `template` in spec section is mandatory. It's used by the ReplicaSet to create new Pods when needed.
</b></details>
<details>
<summary>You've created a ReplicaSet, how to check whether the ReplicaSet found matching Pods or it created new Pods?</summary><br><b>
`kubectl describe rs <ReplicaSet Name>`
It will be visible under `Events` (the very last lines)
</b></details>
<details>
<summary>True or False? Deleting a ReplicaSet will delete the Pods it created</summary><br><b>
True (and not only the Pods but anything else it created).
True. When the label, used by a ReplicaSet in the selector field, removed from a Pod, that Pod no longer controlled by the ReplicaSet and the ReplicaSet will create a new Pod to compensate for the one it "lost".
</b></details>
<details>
<summary>How to scale a deployment to 8 replicas?</code></summary><br><b>
<summary>ReplicaSets are running the moment the user executed the command to create them (like <code>kubectl create -f rs.yaml</code>)</summary><br><b>
False. It can take some time, depends on what exactly you are running. To see if they are up and running, run `kubectl get rs` and watch the 'READY' column.
</b></details>
<details>
<summary>How to expose a ReplicaSet as a new service?</summary><br><b>
[Kubernetes.io](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset): "A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created."
</b></details>
<details>
<summary>What's the difference between a ReplicaSet and DaemonSet?</summary><br><b>
A ReplicaSet's purpose is to maintain a stable set of replica Pods running at any given time.
A DaemonSet ensures that all Nodes run a copy of a Pod.
</b></details>
<details>
<summary>What are some use cases for using a DaemonSet?</summary><br><b>
* Monitoring: You would like to perform monitoring on every node part of cluster. For example datadog pod runs on every node using a daemonset
* Logging: You would like to having logging set up on every node part of your cluster
* Networking: there is networking component you need on every node for all nodes to communicate between them
</b></details>
<details>
<summary>How DaemonSet works?</summary><br><b>
Historically, up 1.12, it was done with NodeName attribute.
Starting 1.12, it's achieved with regular scheduler and node affinity.
</b></details>
#### DaemonSet - Commands
<details>
<summary>How to list all daemonsets in the current namespace?</summary><br><b>
StatefulSet is the workload API object used to manage stateful applications. Manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods.[Learn more](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/)
A directory accessible by the containers inside a certain Pod and containers. The mechanism responsible for creating the directory, managing it, ... mainly depends on the volume type.
</b></details>
<details>
<summary>What volume types are you familiar with?</summary><br><b>
* emptyDir: created when a Pod assigned to a node and ceases to exist when the Pod is no longer running on that node
* hostPath: mounts a path from the host itself. Usually not used due to security risks but has multiple use-cases where it's needed like access to some internal host paths (`/sys`, `/var/lib`, etc.)
2. Storage in containers is ephemeral - it usually doesn't last for long. For example, when a container crashes, you lose all on-disk data. Certain volumes allows to manage such situation by persistent volumes
<summary>Provide at least one use-case for each of the following volume types:
* emptyDir
* hostPath
</summary><br><b>
* EmptyDir: You need a temporary data that you can afford to lose if the Pod is deleted. For example short-lived data required for one-time operations.
* hostPath: You need access to paths on the host itself (like data from `/sys` or data generated in `/var/lib`)
[kubernetes.io](https://kubernetes.io/docs/concepts/services-networking/network-policies): "NetworkPolicies are an application-centric construct which allow you to specify how a pod is allowed to communicate with various network "entities"..."
In simpler words, Network Policies specify how pods are allowed/disallowed to communicate with each other and/or other network endpoints.
<summary>True or False? If no network policies are applied to a pod, then no connections to or from it are allowed</summary><br><b>
False. By default pods are non-isolated.
</b></details>
<details>
<summary>In case of two pods, if there is an egress policy on the source denining traffic and ingress policy on the destination that allows traffic then, traffic will be allowed or denied?</summary><br><b>
Denied. Both source and destination policies has to allow traffic for it to be allowed.
</b></details>
<details>
<summary>Where Kubernetes cluster stores the cluster state?</summary><br><b>
Namespaces allow you split your cluster into virtual clusters where you can group your applications in a way that makes sense and is completely separated from the other groups (so you can for example create an app with the same name in two different namespaces)
</b></details>
<details>
<summary>Why to use namespaces? What is the problem with using one default namespace?</summary><br><b>
When using the default namespace alone, it becomes hard over time to get an overview of all the applications you manage in your cluster. Namespaces make it easier to organize the applications into groups that makes sense, like a namespace of all the monitoring applications and a namespace for all the security applications, etc.
Namespaces can also be useful for managing Blue/Green environments where each namespace can include a different version of an app and also share resources that are in other namespaces (namespaces like logging, monitoring, etc.).
Another use case for namespaces is one cluster, multiple teams. When multiple teams use the same cluster, they might end up stepping on each others toes. For example if they end up creating an app with the same name it means one of the teams overridden the app of the other team because there can't be too apps in Kubernetes with the same name (in the same namespace).
<summary>True or False? When a namespace is deleted all resources in that namespace are not deleted but moved to another default namespace</summary><br><b>
False. When a namespace is deleted, the resources in that namespace are deleted as well.
</b></details>
<details>
<summary>What special namespaces are there by default when creating a Kubernetes cluster?</summary><br><b>
* default
* kube-system
* kube-public
* kube-node-lease
</b></details>
<details>
<summary>What can you find in kube-system namespace?</summary><br><b>
Resource quota provides constraints that limit aggregate resource consumption per namespace. It can limit the quantity of objects that can be created in a namespace by type, as well as the total amount of compute resources that may be consumed by resources in that namespace.
Start by inspecting the pods status. we can use the command `kubectl get pods` (--all-namespaces for pods in system namespace)<br>
If we see "Error" status, we can keep debugging by running the command `kubectl describe pod [name]`. In case we still don't see anything useful we can try stern for log tailing.<br>
In case we find out there was a temporary issue with the pod or the system, we can try restarting the pod with the following `kubectl scale deployment [name] --replicas=0`<br>
Setting the replicas to 0 will shut down the process. Now start it with `kubectl scale deployment [name] --replicas=1`
</b></details>
<details>
<summary>What happens what pods are using too much memory? (more than its limit)</summary><br><b>
They become candidates to for termination.
</b></details>
<details>
<summary>Describe how roll-back works</summary><br><b>
</b></details>
<details>
<summary>True or False? Memory is a compressible resource, meaning that when a container reach the memory limit, it will keep running</summary><br><b>
False. CPU is a compressible resource while memory is a non compressible resource - once a container reached the memory limit, it will be terminated.
"Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components. Operators follow Kubernetes principles, notably the control loop."
<summary>Why do we need Operators?</summary><br><b>
The process of managing stateful applications in Kubernetes isn't as straightforward as managing stateless applications where reaching the desired status and upgrades are both handled the same way for every replica. In stateful applications, upgrading each replica might require different handling due to the stateful nature of the app, each replica might be in a different status. As a result, we often need a human operator to manage stateful applications. Kubernetes Operator is suppose to assist with this.
This also help with automating a standard process on multiple Kubernetes clusters
</b></details>
<details>
<summary>What components the Operator consists of?</summary><br><b>
1. CRD (Custom Resource Definition) - You are fanmiliar with Kubernetes resources like Deployment, Pod, Service, etc. CRD is also a resource, but one that you or the developer the operator defines.
It uses the control loop used by Kubernetes in general. It watches for changes in the application state. The difference is that is uses a custom control loop.
In addition, it also makes use of CRD's (Custom Resources Definitions) so basically it extends Kubernetes API.
<summary>Describe in detail what is the Operator Lifecycle Manager</summary><br><b>
It's part of the Operator Framework, used for managing the lifecycle of operators. It basically extends Kubernetes so a user can use a declarative way to manage operators (installation, upgrade, ...).
<summary>Would you use Helm, Go or something else for creating an Operator?</summary><br><b>
Depends on the scope and maturity of the Operator. If it mainly covers installation and upgrades, Helm might be enough. If you want to go for Lifecycle management, insights and auto-pilot, this is where you'd probably use Go.
The main difference relies on the moment when you want to configure storage. For instance, if you need to pre-populate data in a volume, you choose static provisioning. Whereas, if you need to create volumes on demand, you go for dynamic provisioning.
RBAC in Kubernetes is the mechanism that enables you to configure fine-grained and specific sets of permissions that define how a given user, or group of users, can interact with any Kubernetes object in cluster, or in a specific Namespace of cluster.
<summary>Explain what are "Service Accounts" and in which scenario would use create/use one</summary><br><b>
[Kubernetes.io](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account): "A service account provides an identity for processes that run in a Pod."
An example of when to use one:
You define a pipeline that needs to build and push an image. In order to have sufficient permissions to build an push an image, that pipeline would require a service account with sufficient permissions.
</b></details>
<details>
<summary>What happens you create a pod and you DON'T specify a service account?</summary><br><b>
The pod is automatically assigned with the default service account (in the namespace where the pod is running).
</b></details>
<details>
<summary>Explain how Service Accounts are different from User Accounts</summary><br><b>
- User accounts are global while Service accounts unique per namespace
- User accounts are meant for humans or client processes while Service accounts are for processes which run in pods
</b></details>
<details>
<summary>How to list Service Accounts?</summary><br><b>
[kubernetes.io](https://kubernetes.io/docs/tasks/configure-pod-container/security-context): "A security context defines privilege and access control settings for a Pod or Container."
r, the application would not exist. In addition to the application container, there is a sidecar container.
In simpler words, when you have a Pod and there is more than one container running in that Pod that supports or complements the application container, it means you use the sidecar pattern.
A CronJob creates Jobs on a repeating schedule. One CronJob object is like one line of a crontab (cron table) file. It runs a job periodically on a given schedule, written in Cron format.
<summary>What possible issue can arise from using the following spec and how to fix it?
```
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: some-cron-job
spec:
schedule: '*/1 * ** *'
startingDeadlineSeconds: 10
concurrencyPolicy: Allow
```
</summary><br><b>
If the cron job fails, the next job will not replace the previous one due to the "concurrencyPolicy" value which is "Allow". It will keep spawning new jobs and so eventually the system will be filled with failed cron jobs.
To avoid such problem, the "concurrencyPolicy" value should be either "Replace" or "Forbid".
</b></details>
<details>
<summary>What issue might arise from using the following CronJob and how to fix it?
```
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: "some-cron-job"
spec:
schedule: '*/1 * ** *'
jobTemplate:
spec:
template:
spec:
restartPolicy: Never
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
```
</summary><br><b>
The following lines placed under the template:
```
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
```
As a result this configuration isn't part of the cron job spec hence the cron job has no limits which can cause issues like OOM and potentially lead to API server being down.<br>
To fix it, these lines should placed in the spec of the cron job, above or under the "schedule" directive in the above example.
<summary>Explain what Kubernetes Service Discovery means</summary><br><b>
</b></details>
<details>
<summary>You have one Kubernetes cluster and multiple teams that would like to use it. You would like to limit the resources each team consumes in the cluster. Which Kubernetes concept would you use for that?</summary><br><b>
Namespaces will allow to limit resources and also make sure there are no collisions between teams when working in the cluster (like creating an app with the same name).
<summary>What "Resources Quotas" are used for and how?</summary><br><b>
</b></details>
<details>
<summary>Explain ConfigMap</summary><br><b>
Separate configuration from pods.
It's good for cases where you might need to change configuration at some point but you don't want to restart the application or rebuild the image so you create a ConfigMap and connect it to a pod but externally to the pod.
Overall it's good for:
* Sharing the same configuration between different pods
* Storing external to the pod configuration
</b></details>
<details>
<summary>How to use ConfigMaps?</summary><br><b>
1. Create it (from key&value, a file or an env file)
2. Attach it. Mount a configmap as a volume
</b></details>
<details>
<summary>True or False? Sensitive data, like credentials, should be stored in a ConfigMap</summary><br><b>
False. Use secret.
</b></details>
<details>
<summary>Explain "Horizontal Pod Autoscaler"</summary><br><b>
The term cloud native refers to the concept of building and running applications to take advantage of the distributed computing offered by the cloud delivery model.
[Gatekeeper docs](https://open-policy-agent.github.io/gatekeeper/website/docs): "Gatekeeper is a validating (mutating TBA) webhook that enforces CRD-based policies executed by Open Policy Agent"
</b></details>
<details>
<summary>Explain how Gatekeeper works</summary><br><b>
On every request sent to the Kubernetes cluster, Gatekeeper sends the policies and the resources to OPA (Open Policy Agent) to check if it violates any policy. If it does, Gatekeeper will return the policy error message back. If it isn't violates any policy, the request will reach the cluster.
As a concept it's quite common and can be found in many platforms and services. Think for example on package managers in operating systems. If you use Fedora/RHEL that would be dnf. If you use Ubuntu then, apt. If you don't use Linux, then a different question should be asked and it's why? but that's another topic :)
<summary>Why do we need Helm? What would be the use case for using it?</summary><br><b>
Sometimes when you would like to deploy a certain application to your cluster, you need to create multiple YAML files/components like: Secret, Service, ConfigMap, etc. This can be tedious task. So it would make sense to ease the process by introducing something that will allow us to share these bundle of YAMLs every time we would like to add an application to our cluster. This something is called Helm.
A common scenario is having multiple Kubernetes clusters (prod, dev, staging). Instead of individually applying different YAMLs in each cluster, it makes more sense to create one Chart and install it in every cluster.
Another scenario is, you would like to share what you've created with the community. For people and companies to easily deploy your application in their cluster.
Helm Charts is a bundle of YAML files. A bundle that you can consume from repositories or create your own and publish it to the repositories.
</b></details>
<details>
<summary>It is said that Helm is also Templating Engine. What does it mean?</summary><br><b>
It is useful for scenarios where you have multiple applications and all are similar, so there are minor differences in their configuration files and most values are the same. With Helm you can define a common blueprint for all of them and the values that are not fixed and change can be placeholders. This is called a template file and it looks similar to the following
```
apiVersion: v1
kind: Pod
metadata:
name: {[ .Values.name ]}
spec:
containers:
- name: {{ .Values.container.name }}
image: {{ .Values.container.image }}
port: {{ .Values.container.port }}
```
The values themselves will in separate file:
```
name: some-app
container:
name: some-app-container
image: some-app-image
port: 1991
```
</b></details>
<details>
<summary>What are some use cases for using Helm template file?</summary><br><b>
* Deploy the same application across multiple different environments
* CI/CD
</b></details>
<details>
<summary>Explain the Helm Chart Directory Structure</summary><br><b>
Helm allows you to upgrade, remove and rollback to previous versions of charts. In version 2 of Helm it was with what is known as "Tiller". In version 3, it was removed due to security concerns.
<summary>Running <code>kubectl get pods</code> you see Pods in "Pending" status. What would you do?</summary><br><b>
One possible path is to run `kubectl describe pod <pod name>` to get more details.<br>
You might see one of the following:
* Cluster is full. In this case, extend the cluster.
* ResourcesQuota limits are met. In this case you might want to modify them
* Check if PersistentVolumeClaim mount is pending
If none of the above helped, run the command (`get pods`) with `-o wide` to see if the node is assigned to a node. If not, there might be an issue with scheduler.
</b></details>
<details>
<summary>Users unable to reach an application running on a Pod on Kubernetes. What might be the issue and how to check?</summary><br><b>
One possible path is to start with checking the Pod status.
1. Is the Pod pending? if yes, check for the reason with `kubectl describe pod <pod name>`
Istio is an open source service mesh that helps organizations run distributed, microservices-based apps anywhere. Istio enables organizations to secure, connect, and monitor microservices, so they can modernize their enterprise apps more swiftly and securely.
[Kubernetes.io](https://kubernetes.io/docs/concepts/architecture/controller): "In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state."
</b></details>
<details>
<summary>Name two controllers you are familiar with</summary><br><b>
1. Node Contorller: manages the nodes of a cluster. Among other things, the controller is responsible for monitoring nodes' health - if the node is suddenly unreachable it will evacuate all the pods running on it and will mark the node status accordingly.
2. Replication Controller - monitors the status of pod replicas based on what should be running. It makes sure the number of pods that should be running is actually running
</b></details>
<details>
<summary>What process is responsible for running and installing the different controllers?</summary><br><b>
<summary>Using node affinity, set a Pod to schedule on a node where the key is "region" and value is either "asia" or "emea"</summary><br><b>
`vi pod.yaml`
```yaml
affinity:
nodeAffinity:
requiredDuringSchedlingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: region
operator: In
values:
- asia
- emea
```
</b></details>
<details>
<summary>Using node affinity, set a Pod to never schedule on a node where the key is "region" and value is "neverland"</summary><br><b>
`vi pod.yaml`
```yaml
affinity:
nodeAffinity:
requiredDuringSchedlingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: region
operator: NotIn
values:
- neverland
```
</b></details>
<details>
<summary>True of False? Using the node affinity type "requiredDuringSchedlingIgnoredDuringExecution" means the scheduler can't schedule unless the rule is met</summary><br><b>
True
</b></details>
<details>
<summary>True of False? Using the node affinity type "preferredDuringSchedlingIgnoredDuringExecution" means the scheduler can't schedule unless the rule is met</summary><br><b>
False. The scheduler tries to find a node that meets the requirements/rules and if it doesn't it will schedule the Pod anyway.
<summary>Create a taint on one of the nodes in your cluster with key of "app" and value of "web" and effect of "NoSchedule". Verify it was applied</summary><br><b>
<summary>You applied a taint with <code>k taint node minikube app=web:NoSchedule</code> on the only node in your cluster and then executed <code>kubectl run some-pod --image=redis</code>. What will happen?</summary><br><b>
The Pod will remain in "Pending" status due to the only node in the cluster having a taint of "app=web".
</b></details>
<details>
<summary>You applied a taint with <code>k taint node minikube app=web:NoSchedule</code> on the only node in your cluster and then executed <code>kubectl run some-pod --image=redis</code> but the Pod is in pending state. How to fix it?</summary><br><b>
`kubectl edit po some-pod` and add the following
```
- effect: NoSchedule
key: app
operator: Equal
value: web
```
Exit and save. The pod should be in Running state now.
</b></details>
<details>
<summary>Remove an existing taint from one of the nodes in your cluster</summary><br><b>
<summary>What taint effects are there? Explain each one of them</summary><br><b>
`NoSchedule`: prevents from resources to be scheduled on a certain node
`PreferNoSchedule`: will prefer to shcedule resources on other nodes before resorting to scheduling the resource on the chosen node (on which the taint was applied)
`NoExecute`: Appling "NoSchedule" will not evict already running Pods (or other resources) from the node as opposed to "NoExecute" which will evict any already running resource from the Node
<summary>Explain why one would specify resource limits in regards to Pods</summary><br><b>
* You know how much RAM and/or CPU your app should be consuming and anything above that is not valid
* You would like to make sure that everyone can run their apps in the cluster and resources are not being solely used by one type of application
</b></details>
<details>
<summary>True or False? Resource limits applied on a Pod level meaning, if limits is 2gb RAM and there are two container in a Pod that it's 1gb RAM each</summary><br><b>
False. It's per container and not per Pod.
</b></details>
#### Resources Limits - Commands
<details>
<summary>Check if there are any limits on one of the pods in your cluster</summary><br><b>
`kubectl describe po <POD_NAME> | grep -i limits`
</b></details>
<details>
<summary>Run a pod called "yay" with the image "python" and resources request of 64Mi memory and 250m CPU</summary><br><b>
`kubectl run yay --image=python --dry-run=client -o yaml > pod.yaml`
`vi pod.yaml`
```
spec:
containers:
- image: python
imagePullPolicy: Always
name: yay
resources:
requests:
cpu: 250m
memory: 64Mi
```
`kubectl apply -f pod.yaml`
</b></details>
<details>
<summary>Run a pod called "yay2" with the image "python". Make sure it has resources request of 64Mi memory and 250m CPU and the limits are 128Mi memory and 500m CPU</summary><br><b>
`kubectl run yay2 --image=python --dry-run=client -o yaml > pod.yaml`
<summary>What monitoring solutions are you familiar with in regards to Kubernetes?</summary><br><b>
There are many types of monitoring solutions for Kubernetes. Some open-source, some are in-memory, some of them cost money, ... here is a short list:
* metrics-server: in-memory open source monitoring
* datadog: $$$
* promethues: open source monitoring solution
</b></details>
<details>
<summary>Describe how the monitoring solution you are working with monitors Kubernetes and </summary><br><b>
This very much depends on what you chose to use. Let's address some of the solutions:
* metrics-server: an open source and free monitoring solution that uses the cAdvisor component of kubelet to retrieve information on the cluster and its resources and stores them in-memory.
Once installed, after some time you can run commands like `kubectl top node` and `kubectl top pod` to view performance metrics on nodes, pods and other resources.
<summary>An engineer form your organization told you he is interested only in seeing his team resources in Kubernetes. Instead, in reality, he sees resources of the whole organization, from multiple different teams. What Kubernetes concept can you use in order to deal with it?</summary><br><b>
Namespaces. See the following [namespaces question and answer](#namespaces-use-cases) for more information.
<summary>An engineer in your team runs a Pod but the status he sees is "CrashLoopBackOff". What does it means? How to identify the issue?</summary><br><b>
The container failed to run (due to different reasons) and Kubernetes tries to run the Pod again after some delay (= BackOff time).
Some reasons for it to fail:
- Misconfiguration - mispelling, non supported value, etc.
- Resource not available - nodes are down, PV not mounted, etc.
Some ways to debug:
1.`kubectl describe pod POD_NAME`
1. Focus on `State` (which should be Waiting, CrashLoopBackOff) and `Last State` which should tell what happened before (as in why it failed)
2. Run `kubectl logs mypod`
1. This should provide an accurate output of
2. For specific container, you can add `-c CONTAINER_NAME`
<summary>An engineer form your organization asked whether there is a way to prevent from Pods (with cretain label) to be scheduled on one of the nodes in the cluster. Your reply is:</summary><br><b>
Yes, using taints, we could run the following command and it will prevent from all resources with label "app=web" to be scheduled on node1: `kubectl taint node node1 app=web:NoSchedule`
<summary>You would like to limit the number of resources being used in your cluster. For example no more than 4 replicasets, 2 services, etc. How would you achieve that?</summary><br><b>
