* Add support for configurable compression algorithm (gzip, zstd) and compression level
We want to make the layer compression in kaniko configurable, so we have added two optional command line arguments “--compression” and “--compression-level”. The former allows the user to specify a compression algorithm (zstd, gzip) and the latter can be used to specify the compression level.
Depending on the selected compression algorithm and level we modify the set of layerOptions that are used to create tarball layers in `push.go` and `build.go`.
The actual implementation of the zstd support can be found in our fork of the go-containerregistry package for which we have filed this PR: google/go-containerregistry#1487
The changes should be fully backwards compatible.
* Restrict inputs for compression flag to gzip and zstd
This change will ensure that users can only specify supported compression algorithms (`zstd`, `gzip`) to the `--compression` flag.
* Fix incorrect type for switch statements on config.Compression
- Adds a new option, InitialFSUnpacked
- When opts.InitialFSUnpacked is true, the first stage builder will
skip unpacking the file system; later stages are unaffected
Signed-off-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* fix: Correct flatten function in layers
- Added a test.
- Cache current image, track deletes in `whiteouts` as well as normal adds in `layers`.
- Fix ugly delete behavior of `layerHashCache`.
Delete it when crerating a new snapshot.
- Slight cleanup in `snapshot.go`.
- Format ugly `WalkFS` function.
* fix: Add symbolic link changes to Hasher and CacheHasher
* fix: Better log messages
* fix(ci): Integration tests
* fix(ci): Add `--no-cache` to docker builds
* fix(ci): Pass credentials for error integration test
* np: Missing .gitignore in `hack`
* np: Capitalize every log message
- Correct some linting.
* fix: Key function
- Merge only last layer onto `currentImage`.
* fix: Remove old obsolete `cacheHasher`
* remove configurables from constants
* add configurables to config init
* add kaniko dir flag
* cleanup pkg constants
* replace buildcontext constants pkg occurrences with config
* add KanikoDir to KanikoOptions
* replace executor constants pkg occurrences with config
* remove redundant KanikoDir assignment
* replace constants to config for IntermediateStagesDir
* fix imports
* add default kaniko directory into constants
* add check for kanikoDir on use
* update init to use default path constant
* update executor kanikoDir check
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* alter checkKanikoDir parameter
* add TestKanikoDir func
* update error handling style
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* remove shorthand flag usage from test
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* add docstring to integration test
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* remove shorthand flag from kaniko-dir
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* Remove tarball.WithCompressedCaching flag to resolve OOM Killed error
Large images cannot be build as the kaniko container will be killed due to an OOM error. Removing the tarball compression drastically reduces the memory required to push large image layers. Fixes#1680
This change may increase the build time for smaller images. Therefore a command line option to trigger the compression or a more intelligent behaviour may be useful.
* Add new command line flag to toggle compressed caching
* Add unittest for build with --compressed-caching command line flag set to false
which means we can now:
- set up one or more mirrors
- set up registries certificates
- skip TLS verify
- use plain HTTP
using the same set of flags that are defined for the executor
Fixes#1473
The initial implementation of the registry mirror only allowed a single mirror, and if pulling from the mirror failed, the build would fail.
This change introduces:
- multiple registry mirrors instead of a single one
- fallback if an image can't be pulled from a registry
This is the same behavior as the docker daemon and will allow using a registry mirror such as `mirror.gcr.io` which is incomplete and doesn't have all the content that the default registry on docker.io has.
Note that there are no changes in the CLI flags, the `--registry-mirror` flag is still valid. But now it can be used multiple times to set up more than one registry mirror.
Co-authored-by: Tejal Desai <tejaldesai@google.com>
Fixed#296.
The output manifests may have `application/vnd.docker.distribution.manifest.v2+json`
as their media types instead of `application/vnd.oci.image.manifest.v1+json`.
This changes allow to use kaniko-warmer multiple times without unnecessary docker image downloads.
To check image presence in cache directory I'm using existing cache function that is used by kaniko-executor.
I've considered building separate function to only check image presence, but it will have pretty much the same code.
Questionable decision is to embed CacheOptions type to KanikoOptions and WarmerOptions. Probably this should be resolved by creating interface providing needed options and implement it both mentioned structs. But I've struggled to get a meaningfull name to it.
To replicate previous behaviour of downloading regardless of cache state I've added --force(-f) option.
This changes provides crucial speed-up when downloading images from remote registry is slow.
Closes#722
This flag, when set, takes a file in the container and writes the image digest to it. This can be used to extract the exact digest of the built image by surrounding tooling without having to parse the logs from Kaniko, for example by pointing the file to a mounted volume or to a file used durint exit status, such as with Kubernetes' [Termination message policy](https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/)]
When the flag is not set, the digest is not written to file and the executor behaves as before. The digest is also written to file in case of a tarball or a `--no-push`.
Closes#654
This change calculates the exact files and directories needed between
stages used in the COPY command. Instead of saving the entire
stage as a tarball, we now save only the necessary files.
* parse arg commands at the top of dockerfiles
* fix pointer reference bug and remove debugging
* fixing tests
* account for meta args with no value
* don't take fs snapshot if / is the only changed path
* move metaArgs inside KanikoStage
* removing unused property
* check for any directory instead of just /
* remove unnecessary check
Lavrenti Frobeen
Natalie Arellano
Gabriel Nützi
Ramy
Jack
Janosch Maier
Tejal Desai
Rhianna
Jose Donizetti
Patrick Barker
Jon Friesen
Sascha Schwarze
Or Geva
Vincent Behar
mickkael
Carlos Alexandro Becker
Tejal Desai
Jordan Goasdoué
Ben Einaudi
Yoan Blanc
Cole Wippern
Will Ripley
chhsia0
v.rul
Gijs
dlorenc
Daisuke Taniwaki
Sharif Elgamal