Tasks
Install Tools
Configure Pods and Containers
Assign Memory Resources to Containers and Pods
Assign CPU Resources to Containers and Pods
Configure Quality of Service for Pods
Assign Extended Resources to a Container
Configure a Pod to Use a Volume for Storage
Configure a Pod to Use a PersistentVolume for Storage
Configure a Pod to Use a Projected Volume for Storage
Configure a Security Context for a Pod or Container
Configure Service Accounts for Pods
Pull an Image from a Private Registry
Configure Liveness and Readiness Probes
Assign Pods to Nodes
Configure Pod Initialization
Attach Handlers to Container Lifecycle Events
Configure Containers Using a ConfigMap
Use ConfigMap Data in Pods
Translate a Docker Compose File to Kubernetes Resources
Inject Data Into Applications
Define a Command and Arguments for a Container
Define Environment Variables for a Container
Expose Pod Information to Containers Through Environment Variables
Expose Pod Information to Containers Through Files
Distribute Credentials Securely Using Secrets
Inject Information into Pods Using a PodPreset
Run Applications
Run a Stateless Application Using a Deployment
Run a Single-Instance Stateful Application
Run a Replicated Stateful Application
Update API Objects in Place Using kubectl patch
Upgrade from PetSets to StatefulSets
Scale a StatefulSet
Delete a Stateful Set
Force Delete StatefulSet Pods
Perform Rolling Update Using a Replication Controller
Horizontal Pod Autoscaler
Horizontal Pod Autoscaler Walkthrough
Specifying a Disruption Budget for your Application
Run Jobs
Access Applications in a Cluster
Web UI (Dashboard)
Accessing Clusters
Configure Access to Multiple Clusters
Use Port Forwarding to Access Applications in a Cluster
Provide Load-Balanced Access to an Application in a Cluster
Use a Service to Access an Application in a Cluster
Connect a Front End to a Back End Using a Service
Create an External Load Balancer
Configure Your Cloud Provider's Firewalls
List All Container Images Running in a Cluster
Communicate Between Containers in the Same Pod Using a Shared Volume
Configuring DNS for a Cluster
Monitor, Log, and Debug
Core metrics pipeline
Tools for Monitoring Compute, Storage, and Network Resources
Get a Shell to a Running Container
Monitor Node Health
Logging Using Stackdriver
Events in Stackdriver
Logging Using Elasticsearch and Kibana
Determine the Reason for Pod Failure
Debug Init Containers
Debug Pods and Replication Controllers
Debug Services
Troubleshoot Clusters
Troubleshoot Applications
Debug a StatefulSet
Application Introspection and Debugging
Auditing
Developing and debugging services locally
Use Explorer to Examine the Runtime Environment
Extend Kubernetes
Use an HTTP Proxy to Access the Kubernetes API
Extend the Kubernetes API with CustomResourceDefinitions
Extend the Kubernetes API with ThirdPartyResources
Migrate a ThirdPartyResource to CustomResourceDefinition
Configure the aggregation layer
Setup an extension API server
Install Service Catalog using Helm
Install Service Catalog using SC
Administer a Cluster
Manage Memory, CPU, and API Resources
Configure Default Memory Requests and Limits for a Namespace
Configure Default CPU Requests and Limits for a Namespace
Configure Minimum and Maximum Memory Constraints for a Namespace
Configure Minimum and Maximum CPU Constraints for a Namespace
Configure Memory and CPU Quotas for a Namespace
Configure a Pod Quota for a Namespace
Configure Quotas for API Objects
Advertise Extended Resources for a Node
Control CPU Management Policies on the Node
Access Clusters Using the Kubernetes API
Access Services Running on Clusters
Securing a Cluster
Encrypting data at rest
Operating etcd clusters for Kubernetes
Static Pods
Cluster Management
Cluster Management Guide for Version 1.6
Upgrading kubeadm clusters from 1.6 to 1.7
Upgrading kubeadm clusters from 1.7 to 1.8
Share a Cluster with Namespaces
Namespaces Walkthrough
Autoscale the DNS Service in a Cluster
Safely Drain a Node while Respecting Application SLOs
Configure Out Of Resource Handling
Reserve Compute Resources for System Daemons
Guaranteed Scheduling For Critical Add-On Pods
Declare Network Policy
Reconfigure a Node's Kubelet in a Live Cluster
Set Kubelet parameters via a config file
Install Network Policy Provider
Change the Reclaim Policy of a PersistentVolume
Limit Storage Consumption
Change the default StorageClass
Kubernetes Cloud Controller Manager
Developing Cloud Controller Manager
Set up High-Availability Kubernetes Masters
Configure Multiple Schedulers
IP Masquerade Agent User Guide
Configure private DNS zones and upstream nameservers in Kubernetes
Federation - Run an App on Multiple Clusters
Cross-cluster Service Discovery using Federated Services
Set up Cluster Federation with Kubefed
Set up CoreDNS as DNS provider for Cluster Federation
Set up placement policies in Federation
Federated Cluster
Federated ConfigMap
Federated DaemonSet
Federated Deployment
Federated Events
Federated Horizontal Pod Autoscalers (HPA)
Federated Ingress
Federated Jobs
Federated Namespaces
Federated ReplicaSets
Federated Secrets
Manage Cluster Daemons
Manage GPUs
Manage HugePages
Extend kubectl with plugins
Debug Pods and Replication Controllers
Debugging pods
The first step in debugging a pod is taking a look at it. Check the current state of the pod and recent events with the following command:
$ kubectl describe pods ${POD_NAME}
Look at the state of the containers in the pod. Are they all Running? Have
there been recent restarts?
Continue debugging depending on the state of the pods.
My pod stays pending
If a pod is stuck in Pending it means that it can not be scheduled onto a
node. Generally this is because there are insufficient resources of one type or
another that prevent scheduling. Look at the output of the kubectl describe
... command above. There should be messages from the scheduler about why it
can not schedule your pod. Reasons include:
Insufficient resources
You may have exhausted the supply of CPU or Memory in your cluster. In this case you can try several things:
-
Add more nodes to the cluster.
-
Terminate unneeded pods to make room for pending pods.
-
Check that the pod is not larger than your nodes. For example, if all nodes have a capacity of
cpu:1, then a pod with a request ofcpu: 1.1will never be scheduled.You can check node capacities with the
kubectl get nodes -o <format>command. Here are some example command lines that extract just the necessary information:kubectl get nodes -o yaml | grep '\sname\|cpu\|memory' kubectl get nodes -o json | jq '.items[] | {name: .metadata.name, cap: .status.capacity}'The resource quota feature can be configured to limit the total amount of resources that can be consumed. If used in conjunction with namespaces, it can prevent one team from hogging all the resources.
Using hostPort
When you bind a pod to a hostPort there are a limited number of places that
the pod can be scheduled. In most cases, hostPort is unnecessary; try using a
service object to expose your pod. If you do require hostPort then you can
only schedule as many pods as there are nodes in your container cluster.
My pod stays waiting
If a pod is stuck in the Waiting state, then it has been scheduled to a
worker node, but it can’t run on that machine. Again, the information from
kubectl describe ... should be informative. The most common cause of
Waiting pods is a failure to pull the image. There are three things to check:
- Make sure that you have the name of the image correct.
- Have you pushed the image to the repository?
- Run a manual
docker pull <image>on your machine to see if the image can be pulled.
My pod is crashing or otherwise unhealthy
First, take a look at the logs of the current container:
$ kubectl logs ${POD_NAME} ${CONTAINER_NAME}
If your container has previously crashed, you can access the previous container’s crash log with:
$ kubectl logs --previous ${POD_NAME} ${CONTAINER_NAME}
Alternately, you can run commands inside that container with exec:
$ kubectl exec ${POD_NAME} -c ${CONTAINER_NAME} -- ${CMD} ${ARG1} ${ARG2} ... ${ARGN}
Note that -c ${CONTAINER_NAME} is optional and can be omitted for pods that
only contain a single container.
As an example, to look at the logs from a running Cassandra pod, you might run:
$ kubectl exec cassandra -- cat /var/log/cassandra/system.log
If none of these approaches work, you can find the host machine that the pod is running on and SSH into that host.
Debugging Replication Controllers
Replication controllers are fairly straightforward. They can either create pods or they can’t. If they can’t create pods, then please refer to the instructions above to debug your pods.
You can also use kubectl describe rc ${CONTROLLER_NAME} to inspect events
related to the replication controller.