ChatGPT解决这个技术问题 Extra ChatGPT

How to use local docker images with Minikube?

I have several docker images that I want to use with minikube. I don't want to first have to upload and then download the same image instead of just using the local image directly. How do I do this?

Stuff I tried: 1. I tried running these commands (separately, deleting the instances of minikube both times and starting fresh)

kubectl run hdfs --image=fluxcapacitor/hdfs:latest --port=8989
kubectl run hdfs --image=fluxcapacitor/hdfs:latest --port=8989 imagePullPolicy=Never

Output:

NAME                    READY     STATUS              RESTARTS   AGE
hdfs-2425930030-q0sdl   0/1       ContainerCreating   0          10m

It just gets stuck on some status but never reaches the ready state.

2. I tried creating a registry and then putting images into it but that didn't work either. I might've done that incorrectly but I can't find proper instructions to do this task.

Please provide instructions to use local docker images in local kubernetes instance. OS: ubuntu 16.04 Docker : Docker version 1.13.1, build 092cba3 Kubernetes :

Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.3", GitCommit:"029c3a408176b55c30846f0faedf56aae5992e9b", GitTreeState:"clean", BuildDate:"2017-02-15T06:40:50Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"linux/amd64"}

If someone could help me get a solution that uses docker-compose to do this, that'd be awesome.

Edit:

Images loaded in eval $(minikube docker-env):

REPOSITORY                                            TAG                 IMAGE ID            CREATED             SIZE
fluxcapacitor/jupyterhub                              latest              e5175fb26522        4 weeks ago         9.59 GB
fluxcapacitor/zeppelin                                latest              fe4bc823e57d        4 weeks ago         4.12 GB
fluxcapacitor/prediction-pmml                         latest              cae5b2d9835b        4 weeks ago         973 MB
fluxcapacitor/scheduler-airflow                       latest              95adfd56f656        4 weeks ago         8.89 GB
fluxcapacitor/loadtest                                latest              6a777ab6167c        5 weeks ago         899 MB
fluxcapacitor/hdfs                                    latest              00fa0ed0064b        6 weeks ago         1.16 GB
fluxcapacitor/sql-mysql                               latest              804137671a8c        7 weeks ago         679 MB
fluxcapacitor/metastore-1.2.1                         latest              ea7ce8c5048f        7 weeks ago         1.35 GB
fluxcapacitor/cassandra                               latest              3cb5ff117283        7 weeks ago         953 MB
fluxcapacitor/apachespark-worker-2.0.1                latest              14ee3e4e337c        7 weeks ago         3.74 GB
fluxcapacitor/apachespark-master-2.0.1                latest              fe60b42d54e5        7 weeks ago         3.72 GB
fluxcapacitor/package-java-openjdk-1.8                latest              1db08965289d        7 weeks ago         841 MB
gcr.io/google_containers/kubernetes-dashboard-amd64   v1.5.1              1180413103fd        7 weeks ago         104 MB
fluxcapacitor/stream-kafka-0.10                       latest              f67750239f4d        2 months ago        1.14 GB
fluxcapacitor/pipeline                                latest              f6afd6c5745b        2 months ago        11.2 GB
gcr.io/google-containers/kube-addon-manager           v6.1                59e1315aa5ff        3 months ago        59.4 MB
gcr.io/google_containers/kubedns-amd64                1.9                 26cf1ed9b144        3 months ago        47 MB
gcr.io/google_containers/kube-dnsmasq-amd64           1.4                 3ec65756a89b        5 months ago        5.13 MB
gcr.io/google_containers/exechealthz-amd64            1.2                 93a43bfb39bf        5 months ago        8.37 MB
gcr.io/google_containers/pause-amd64           

V
Victor D.

As the README describes, you can reuse the Docker daemon from Minikube with eval $(minikube docker-env).

So to use an image without uploading it, you can follow these steps:

Set the environment variables with eval $(minikube docker-env) Build the image with the Docker daemon of Minikube (eg docker build -t my-image .) Set the image in the pod spec like the build tag (eg my-image) Set the imagePullPolicy to Never, otherwise Kubernetes will try to download the image.

Important note: You have to run eval $(minikube docker-env) on each terminal you want to use, since it only sets the environment variables for the current shell session.


Very important to remember to run eval $(minikube docker-env) after closing the terminal you're working in BEFORE you try to rebuild images... just burned 6 hours fighting with an image that was not updating in minikube... looked like a package was not updating... really just not updating the image that minikube was referencing.
If u wanna back or exit env from minikube.. eval $(minikube docker-env -u)
How can I "Set the imagePullPolicy to Never" using kubectl ?
Nowdays, you can also use minikube cache add imagename:tag to push the image to the minikube - be sure to include the tag as well. Docs
"minikube cache" will be deprecated in upcoming versions, please switch to "minikube image load" - just got it from my terminal.
R
Robert J

What worked for me, based on the solution by @svenwltr:

# Start minikube
minikube start

# Set docker env
eval $(minikube docker-env)             # unix shells
minikube docker-env | Invoke-Expression # PowerShell

# Build image
docker build -t foo:0.0.1 .

# Run in minikube
kubectl run hello-foo --image=foo:0.0.1 --image-pull-policy=Never

# Check that it's running
kubectl get pods

You can find the yml version of the above command line (in regards to the imagePullPolicy) here : kubernetes.io/docs/concepts/containers/images
On Windows eval $(minikube docker-env) was not working for me. minikube docker-env | Invoke-Expression seems to work in PowerShell. In other terminals one needs to read the last line returned by minikube docker-env. For example, in IntelliJ (on Windows) it is @FOR /f "tokens=*" %i IN ('minikube -p minikube docker-env') DO @%i You have to do this in any new terminal/session always before building the docker image.
I am getting following error while doing eval $(minikube docker-env). "'none' driver does not support 'minikube docker-env' command" It is also logged on github it seems. github.com/kubernetes/minikube/issues/2443
K
Kamafeather

Notes:

This Answer isnt limited to minikube!

If wanting to create the registry on minikube's Docker then run eval $(minikube docker-env) first (to make docker available on the host machine's terminal). Otherwise enter in the virtual machine via minikube ssh, and then proceed with the following steps

depending on your operative system, minikube will automatically mount your homepath onto the VM.

as Eli stated, you'll need to add the local registry as insecure in order to use http (may not apply when using localhost but does apply if using the local hostname) Don't use http in production, make the effort for securing things up.

Use a local registry:

docker run -d -p 5000:5000 --restart=always --name local-registry registry:2

Now tag your image properly:

docker tag ubuntu localhost:5000/ubuntu

Note that localhost should be changed to dns name of the machine running registry container.

Now push your image to local registry:

docker push localhost:5000/ubuntu

You should be able to pull it back:

docker pull localhost:5000/ubuntu

Now change your yaml file to use the local registry.

Think about mounting volumes at appropriate location, to persist the images on the registry.


| Now change your yaml file to use the local registry. Are you able to explain this a little bit? I pushed to the local registry (cool trick) but I have the same problem that I can't get minikube to connect to it.
@ZachEstela change the image name in the yaml to <registryIP>:5000/ubuntu
@FarhadFarahi If I give my laptop to you, how would you find out? I just want to know it. I followed docker tutorial steps to get docker for windows running.
@FarhadFarahi: Please add to your answer that you'll need to add the local registry as insecure in order to use http: docs.docker.com/registry/insecure (may not apply when using localhost but does apply if using the local hostname).
Failed to pull image "localhost:5000/src_interface:latest" according to my minikube dashboard, this doesn't seem to work. How would kubectl even access port 5000?
M
Mikeb

There is one easy and effective way to push your local Docker image directly to minikube, which will save time from building the images in minikube again.

minikube image load <image name>

(minikube cache add <image name> - old deprecated way, for reference)

More details here

All possible method to push images to minikube are mention here: https://minikube.sigs.k8s.io/docs/handbook/pushing/


This answer should be higher in the list, it is the more up-to-date solution.
This is what solved the problem for me and yes should have been higher
glad i scrolled down to this answer since it is super easy, thanks !
welcome @hocikto
You are a legend.
N
Nithin

Adding to to @Farhad 's answer based on this answer,

This are the steps to setup a local registry.

Setup in local machine

Setup hostname in local machine: edit /etc/hosts to add this line

docker.local 127.0.0.1

Now start a local registry (remove -d to run non-daemon mode) :

docker run -d -p 5000:5000 --restart=always --name registry registry:2

Now tag your image properly:

docker tag ubuntu docker.local:5000/ubuntu

Now push your image to local registry:

docker push docker.local:5000/ubuntu

Verify that image is pushed:

curl -X GET http://docker.local:5000/v2/ubuntu/tags/list

Setup in minikube

ssh into minikube with: minukube ssh

edit /etc/hosts to add this line

docker.local <your host machine's ip>

Verify access:

curl -X GET http://docker.local:5000/v2/ubuntu/tags/list

Now if you try to pull, yo might get an http access error.

Enable insecure access:

If you are always planning to use minkube with this local setup then create a minikube to use insecure registry by default (wont work on existing cluster).

minikube start --insecure-registry="docker.local:5000"

else follow below steps:

systemctl stop docker

edit the docker serice file: get path from systemctl status docker

it might be :

/etc/systemd/system/docker.service.d/10-machine.conf or /usr/lib/systemd/system/docker.service

append this text (replace 192.168.1.4 with your ip)

--insecure-registry docker.local:5000 --insecure-registry 192.168.1.4:5000

to this line

ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 10.0.0.0/24

systemctl daemon-reload
systemctl start docker

try pulling:

docker pull docker.local:5000/ubuntu

Now change your yaml file to use local registry.

containers: - name: ampl-django image: dockerhub/ubuntu

to

containers: - name: ampl-django image: docker.local:5000/nymbleup

Don't use http in production, make the effort for securing things up.


A
Artem Kozlenkov

one thing to remember regarding 'minikube' is that minikube's host is not the same as your local host, therefore, what i realized, that in order to use local images for testing with minikube you must build your docker image first locally or pull it locally and then add it using the command bellow into the minikube context which is, nothing else as another linux instance.

 minikube cache add <image>:<tag>

yet, don't forget to set the imagePullPolicy: Never in your kubernetes deployment yamls, as it will ensure using locally added images instead of trying pull it remotely from the registry.

update: minikube cache will be deprecated in upcoming versions, please switch to minikube image load


While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value.
C
Chris Stryczynski

One approach is to build the image locally and then do:

docker save imageNameGoesHere | pv | (eval $(minikube docker-env) && docker load)

minikube docker-env might not return the correct info running under a different user / sudo. Instead you can run sudo -u yourUsername minikube docker-env.

It should return something like:

export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/home/chris/.minikube/certs"
export DOCKER_API_VERSION="1.23"
# Run this command to configure your shell:
# eval $(minikube docker-env)

The correct command is docker save imageNameGoesHere > pv | (eval $(minikube docker-env) && docker load)
docker save imageNameGoesHere | (eval $(minikube docker-env) && docker load) worked for me
C
Community

In addition to the accepted answer, you can also achieve what you originally wanted (creating a deployment using the run command) with the following command:

kubectl run hdfs --image=fluxcapacitor/hdfs:latest --port=8989 --generator=run-pod/v1 

I found the information about the generator on the Kubernetes-dev forum:

If you're using kubectl run, it generates a manifest for you that happens to have imagePullPolicy set to Always by default. You can use this command to get an imagePullPolicy of IfNotPresent, which will work for minikube: kubectl run --image= --generator=run-pod/v1 Dan Lorenc

https://groups.google.com/forum/#!topic/kubernetes-dev/YfvWuFr_XOM


N
Nithish Thomas

Newer versions of minikube allows you to load image from the local docker instance by running

 minikube image rm image <imagename>:<version>  
 minikube image load <imagename>:<version> --daemon

the load command might show an error but the image still gets loaded to your minikube instance


this seems to be a great answer ... do you have any sources ?
just found reference here:minikube.sigs.k8s.io/docs/handbook/pushing/… ... the --daemon option is not really needed.
How long does it take? Its running since 20 minutes for 645MB image
C
Chamila Adhikarinayake

If anyone is looking to come back to the local environment after setting the minikube env, use following command.

eval $(docker-machine env -u)

Would be eval $(minikube docker-env -u) for minikube
A
Adam Wise

A simpler method that answers the original question "How to use local docker images with Minikube?", is to save the image to a tar file and load it into minikube:

# export the docker image to a tar file
docker save --output my-image.tar the.full.path.to/the/docker/image:the-tag
# set local environment variables so that docker commands go to the docker in minikube
eval $(minikube docker-env)
# or if on windows: @FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i
# import the docker image from the tar file into minikube
docker load --input my-image.tar
# cleanup - put docker back to normal
eval $(minikube docker-env -u)
# or if on windows: @FOR /f "tokens=*" %i IN ('minikube docker-env -u') DO @%i

Then running the image involves a command like the following. Make sure to include the "--image-pull-policy=Never" parameter.

kubectl run my-image --image=the.full.path.to/the/docker/image:the-tag --image-pull-policy=Never --port=80

Well explained, worked like a charm. I only had to call docker save with sudo, and then set sudo chmod 664 my-image.tar to make it available for my current user.
J
Jason

From the kubernetes docs:

https://kubernetes.io/docs/concepts/containers/images/#updating-images

The default pull policy is IfNotPresent which causes the Kubelet to skip pulling an image if it already exists. If you would like to always force a pull, you can do one of the following: set the imagePullPolicy of the container to Always; use :latest as the tag for the image to use; enable the AlwaysPullImages admission controller.

Or read the other way: Using the :latest tag forces images to always be pulled. If you use the eval $(minikube docker-env) as mentioned above, then either don't use any tag, or assign a tag to your local image you can avoid Kubernetes trying to forcibly pull it.


M
Milkncookiez

One idea would be to save the docker image locally and later load it into minikube as follows:

Let say, for example, you already have puckel/docker-airflow image.

Save that image to local disk - docker save puckel/docker-airflow > puckel_docker_airflow.tar Now enter into minikube docker env - eval $(minikube docker-env) Load that locally saved image - docker load < puckel_docker_airflow.tar

It is that simple and it works like a charm.


You still need the accepted answer's tip of Set the imagePullPolicy to Never. If your image is tagged with an address e.g. us.icr.io/mydiv/my-service:v0.0.1 then a deploy will try to remote pull this image. Since you've already manually copied the image, you need to suppress k8s from pulling the image from an address (container registry) it can't access.
@colm.anseo The steps above worked for me but I had to replace the latest tag in the image with a specific version and select this version in the create deploy command. The imagePullPolicy was automatically set to IfNotPresent and the image was loaded correctly without further changes.
M
Michael Böckling

There is now a Minikube Registry addon, this is probably the easiest way. Here is how to use it: https://minikube.sigs.k8s.io/docs/tasks/registry/insecure/

Note that I had DNS issues, might be a bug.


C
Chris Sattinger
minikube addons enable registry -p minikube

💡 Registry addon on with docker uses 32769 please use that instead of default 5000
📘 For more information see: https://minikube.sigs.k8s.io/docs/drivers/docker

docker tag ubuntu $(minikube ip -p minikube):32769/ubuntu
docker push $(minikube ip -p minikube):32769/ubuntu

OR

minikube addons enable registry
docker tag ubuntu $(minikube ip):32769/ubuntu
docker push $(minikube ip):32769/ubuntu

The above is good enough for development purpose. I am doing this on archlinux.


How do u refer to the image in the k8s specs? By localhost:32769/ubuntu or with the minikube ip or the registry dns name?
J
Julien Nyambal

To add to the previous answers, if you have a tarball image, you can simply load it to you local docker set of images docker image load -i /path/image.tar .Please remember to run it after eval $(minikube docker-env), since minikube does not share images with the locally installed docker engine.


н
никта

Other answers suppose you use minikube with VM, so your local images are not accessible from minikube VM.

In case if you use minikube with --vm-driver=none, you can easily reuse local images by setting image_pull_policy to Never:

kubectl run hello-foo --image=foo --image-pull-policy=Never

or setting imagePullPolicy field for cotainers in corresponding .yaml manifests.


M
Maciek Leks

For minikube on Docker:

Option 1: Using minikube registry

Check your minikube ports docker ps

You will see something like: 127.0.0.1:32769->5000/tcp It means that your minikube registry is on 32769 port for external usage, but internally it's on 5000 port.

Build your docker image tagging it: docker build -t 127.0.0.1:32769/hello . Push the image to the minikube registry: docker push 127.0.0.1:32769/hello Check if it's there: curl http://localhost:32769/v2/_catalog Build some deployment using the internal port: kubectl create deployment hello --image=127.0.0.1:5000/hello

Your image is right now in minikube container, to see it write:

eval $(minikube -p <PROFILE> docker-env)
docker images

caveat: if using only one profile named "minikube" then "-p " section is redundant, but if using more then don't forget about it; Personally I delete the standard one (minikube) not to make mistakes.

Option 2: Not using registry

Switch to minikube container Docker: eval $(minikube -p docker-env) Build your image: docker build -t hello . Create some deployment: kubectl create deployment hello --image=hello

At the end change the deployment ImagePullPolicy from Always to IfNotPresent:

kubectl edit deployment hello


V
VonC

In addition of minikube image load <image name>, check out the latest (Nov 2021 at the time of writing) release of Minikube.

v1.24.0

Add --no-kubernetes flag to start minikube without Kubernetes See PR 12848, for

That gives you:

mk start --no-kubernetes
minikube v1.24.0-beta.0 on Darwin 11.6 (arm64)
Automatically selected the docker driver
Starting minikube without Kubernetes minikube in cluster minikube
Pulling base image ...
Creating docker container (CPUs=2, Memory=1988MB) ...
Done! minikube is ready without Kubernetes!

Things to try without Kubernetes "minikube ssh" to SSH into minikube's node. "minikube docker-env" to build images by pointing to the docker inside minikube "minikube image" to build images without docker


d
dididothat

what if you could just run k8s within docker's vm? there's native support for this with the more recent versions of docker desktop... you just need to enable that support.

https://www.docker.com/blog/kubernetes-is-now-available-in-docker-desktop-stable-channel/ https://www.docker.com/blog/docker-windows-desktop-now-kubernetes/

how i found this out:

while reading the docs for helm, they give you a brief tutorial how to install minikube. that tutorial installs minikube in a vm that's different/separate from docker.

so when it came time to install my helm charts, i couldn't get helm/k8s to pull the images i had built using docker. that's how i arrived here at this question.

so... if you can live with whatever version of k8s comes with docker desktop, and you can live with it running in whatever vm docker has, then maybe this solution is a bit easier than some of the others.

disclaimer: not sure how switching between windows/linux containers would impact anything.


I think I also had to set the imagePullPolicies to IfNotPresent as well
A
Anurag Manikkame

setup minikube docker-env again build the same docker image (using minikube docker-env) change imagePullPolicy to Never in your deployment

actually what happens here , your Minikube can't recognise your docker daemon as it is independent service.You have to first set your minikube-docker environment use below command to check

 "eval $(minikube docker-env)"

If you run below command it will show where your minikube looks for docker.

~$ minikube docker-env
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.37.192:2376"
export DOCKER_CERT_PATH="/home/ubuntu/.minikube/certs"
export MINIKUBE_ACTIVE_DOCKERD="minikube"

**# To point your shell to minikube's docker-daemon, run:**
# eval $(minikube -p minikube docker-env)

You have to again build images once you setup minikube docker-env else it will fail.


n
noob_to_so

steps to run local docker images in kubernetes 1. eval $(minikube -p minikube docker-env) 2. in the artifact file , under spec section -> containers add imagePullPolicy: IfNotPresent or imagePullPolicy: Never

apiVersion: "v1"
kind: Pod
metadata:
    name: web
    labels:
        name: web
        app: demo
spec:
    containers:
        - name: web
          image: web:latest
          imagePullPolicy: IfNotPresent
          ports:
              - containerPort: 5000
                name: http
                protocol: TCP


3. then run kubectl create -f <filename>


A
Alex Cline

building off the earlier answer to use eval $(minikube docker-env) in order to load up minikube's docker environment, for an easier toggle, add the following function to your shell rc file:

dockube() {
  if [[ $1 = 'which' ]]; then
    if [[ $MINIKUBE_ACTIVE_DOCKERD = 'minikube' ]]; then
      echo $MINIKUBE_ACTIVE_DOCKERD
    else
      echo 'system'
    fi
    return
  fi

  if [[ $MINIKUBE_ACTIVE_DOCKERD = 'minikube' ]]; then
    eval $(minikube docker-env -u)
    echo "now using system docker"
  else
    eval $(minikube -p minikube docker-env)
    echo "now using minikube docker"
  fi
}

dockube with no argument will toggle between the system and minikube docker environment, and dockube which will return which one is in use.


A
Abdallah Alhaddad

You should know that docker in your local machine is separated from the docker in your minikube cluster.

So you should load/copy a Docker image from your local machine into the minikube cluster:

minikube image load <IMAGE_NAME>

or alternatively when working with minikube, you can build images directly inside it:

#instead of:
docker image build -t <IMAGE_NAME> .
#do:
minikube image build -t <IMAGE_NAME> .

J
Jackie

you can either reuse the docker shell, with eval $(minikube docker-env), alternatively, you can leverage on docker save | docker load across the shells.


K
Kartoch

On minikube 1.20, minikube cache add imagename:tag is deprecated.

Instead use minikube image load imagename:tag


j
jevenus

i find this method from ClickHouse Operator Build From Sources and it helps and save my life!

docker save altinity/clickhouse-operator | (eval $(minikube docker-env) && 
docker load)