It seems like .dockerignore deletes files containeed within the file locally upon
docker build, so I created a temporary test dir to make sure the
--ignore flag works. We make sure it exists before building docker and
kaniko for Dockerfile_test_ignore, and then delete it after the builds
have completed.
Added a --ignore flag to ignore packages and files in the build context.
This should mimic the .dockerignore file. Before starting the build, we
go through and delete ignored files from the build context.
* comments
* initial commit for persisent volume caching
* cache warmer works
* general cleanup
* adding some debugging
* adding missing files
* Fixing up cache retrieval and cleanup
* fix tests
* removing auth since we only cache public images
* simplifying the caching logic
* fixing logic
* adding volume cache to integration tests. remove auth from cache warmer image.
* add building warmer to integration-test
* move sample yaml files to examples dir
* small test fix
To add layer caching to kaniko, I added two flags: --cache and
--use-cache.
If --use-cache is set, then the cache will be used, and if --cache is
specified then that repo will be used to store cached layers. If --cache
isn't set, a cache will be inferred from the destination provided.
Currently, caching only works for RUN commands. Before executing the
command, kaniko checks if the cached layer exists. If it does, it pulls
it and extracts it. It then adds those files to the snapshotter and
append a layer to the config history. If the cached layer does not exist, kaniko executes the command and
pushes the newly created layer to the cache.
All cached layers are tagged with a stable key, which is built based off
of:
1. The base image digest
2. The current state of the filesystem
3. The current command being run
4. The current config file (to account for metadata changes)
I also added two integration tests to make sure caching works
1. Dockerfile_test_cache runs 'date', which should be exactly the same
the second time the image is built
2. Dockerfile_test_cache_install makes sure apt-get install can be
reproduced
In #251 we are investigating test flakes due to layer offsets not
matching, this change will give us a bit more context so we can be sure
which image has which number of layers, and it will also include the
digest of the image, since kaniko always pushes images to a remote repo,
so if the test fails we can pull the digest and see what is up.
Also updated reproducible Dockerfile to be built with reproducible flag,
which I think was the original intent (without this change, there is no
difference between how `kaniko-dockerfile_test_copy_reproducible` and
`kaniko-dockerfile_test_copy` are built.
To allow contributors to run the integration tests with their own GCS
buckets and image repos (since not all contributors will have accesss to
the projects used by the kaniko maintainers) this updates the
integration tests so that these can be provided on the command line.
This allows tests to be run individually, without using `make
integration-test`. Previously, part of the test setup was done
in the shell script (creating the context tarball that is required
for the tests that build images with context). Instead it will be
done in the test iself, so we can use `go test` to run tests
individually if we want to.
If we are running only one individual test, we don't want to build
all of the images, so this commit creates a builder which tracks which
images it has built and can be used by a tests to check if it should
build an image before running, or it will use the images that have
already been built by a previous test.
The name of the context tarball has also been made unique (it includes
the unix timestamp) to avoid potential test flakes if two tests using
the same GCS bucket run simultaneously.
Priya Wadhwa
Sharif Elgamal
Kartik Verma
Christie Wilson