To add layer caching to kaniko, I added two flags: --cache and
--use-cache.
If --use-cache is set, then the cache will be used, and if --cache is
specified then that repo will be used to store cached layers. If --cache
isn't set, a cache will be inferred from the destination provided.
Currently, caching only works for RUN commands. Before executing the
command, kaniko checks if the cached layer exists. If it does, it pulls
it and extracts it. It then adds those files to the snapshotter and
append a layer to the config history. If the cached layer does not exist, kaniko executes the command and
pushes the newly created layer to the cache.
All cached layers are tagged with a stable key, which is built based off
of:
1. The base image digest
2. The current state of the filesystem
3. The current command being run
4. The current config file (to account for metadata changes)
I also added two integration tests to make sure caching works
1. Dockerfile_test_cache runs 'date', which should be exactly the same
the second time the image is built
2. Dockerfile_test_cache_install makes sure apt-get install can be
reproduced
gometalinter is broken @ HEAD, and I looked into why that was. During
that process, I remembered that we took the linting scripts from
skaffold, and found that in skaffold gometalinter was replaced with
GolangCI-Lint:
https://github.com/GoogleContainerTools/skaffold/pull/619
The change made linting in skaffold faster, so I figured instead of
fixing gometalinter it made more sense to remove it and replace it with
GolangCI-Lint for kaniko as well.
I added a KanikoStage to hold each stage of the Dockerfile along with
information about each stage that would be useful later on.
The new KanikoStage type holds the stage itself, along with some
additional information:
1. FinalStage -- whether the current stage is the final stage
2. BaseImageStoredLocally/BaseImageIndex -- whether the base image for
this stage is stored locally, and if so what the index of the base image
is
3. SaveStage -- whether this stage needs to be saved for use in a future
stage
This is the first part of a larger refactor for building stages, which
will later make it easier to add layer caching.
In this refactor I:
1. Created KanikoOptions to make it easier to pass around arguments
passed in through the command line
2. Reorganized executor.go by putting the logic for pushing the image in
a new file push.go
3. Made some error messages clearer
4. Fixed a mistake in the README for pushing to AWS
5. Marked the --bucket flag as hidden since we want people to use
--context instead, and marked an aws flag as hidden which is set in a
vendored directorya
The flag, `--no-push`, is added to allow building a container image
without pushing to a container registry. It can be common, especially
with multi-stage builds and `--target`, to build enough to run the tests,
and then perform a push in a separate CI step. This will facilitate these
workflows.
* added switch to extract different sources as build context
* first rough implementation of aws s3
* added buildcontext package and interface
* added GetBuildContext func to buildcontext.go
added fallback to gcs
renamed GC struct to GCS
* improved the default behavior of build context retrieval
* renamed gc:// to gs:// in order to follow common standards
* renamed struct File to Dir and some cleanup work
* moved context.tar suffix to the buildcontext processors where it is needed
* added buildcontext retrieval as struct variable
added fallback if prefix in bucket specifier is present
* cleanup if structures
* added prefix to s3
* WIP
* Fixed build context bugs
* refactored build context
* adding reproducible test
* newer version of go-containerregistry
* new ImageOptions
* switch reproducible flag to default to false
* small fixes
* update dep
* Vendor changes for go-containerregistry switch.
* Manual changes for go-containerregistry switch.
The biggest change is refactoring the tarball unpacking.
* Pull more of container-diff out.
* More vendor removals.
* More unit tests.
Priya Wadhwa
Sebastian Jackel
David McKay
Guilherme Rezende
Christian Jantz
Sharif Elgamal
jesusofsuburb1a
dlorenc
Nicolas Byl
Carlos Sanchez
Matt Rickard