* Use canonical image architectures. Closes#1995.
Prior to this change, Kaniko was not using the platform's canonical value.
* Update platform unit test.
* Correctly handle platforms that include CPU variants
Prior to this change, Kaniko would only select the platform-specific
image from a multi-platform image using the OS and architecture to
select the image. This leads to problems when there are two platforms
that are indistinguishable using only this information (e.g.,
linux/arm/v5 and linux/arm/v7).
This change more explicitly selects the right image from a
multi-platform image, taking CPU variant into account (v5 vs v7), using
containerd's CPU variant detection logic.
This also moves platform defaulting up as early as possible in execution
as it can go, so that malformed platform values are detected as soon as
possible.
* set platform in unit test
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Remove tarball.WithCompressedCaching flag to resolve OOM Killed error
Large images cannot be build as the kaniko container will be killed due to an OOM error. Removing the tarball compression drastically reduces the memory required to push large image layers. Fixes#1680
This change may increase the build time for smaller images. Therefore a command line option to trigger the compression or a more intelligent behaviour may be useful.
* Add new command line flag to toggle compressed caching
* Add unittest for build with --compressed-caching command line flag set to false
* chore: add workflows for pr tests
* fix unit tests
* fix formatting
* chore: fix gobuild
* change minikube script
* chore: fix lint install script
* chore: ignore and fix tests
* fix lint and run gofmt
* lint fixes
* k8s executor image only
* fix Makefile
* fix travis env variables
* more info on k8s tests
* fix travis run
* fix
* fix
* fix
* fix log
* some more changes
* increase timeout
* delete travis.yml and fix multiple copy tests
* fix registry mirror
* fix lint
* add concurency
* last attemot to fix k8 integrations
* diff id for diff workflows
* Fix composite cache key for multi-stage copy command (#1706)
PR #1518 reintroduced COPY layers caching using the `--cache-copy-layers`
flag. Unfortunately, this PR also introduced a bug by not including the
stage digest into the caching key of the COPY command when the
`--cache-copy-layers` flag was not set. As a result, kaniko would use
any previous (possibly stalled) layer from the cache because the digest
of the "COPY --from" command would never change.
PR author probably expected Go to fallthrough in the switch just like C
does. However, this is not the case. Go does not fallthrough in
switch-statements by default and requires the fallthrough keyword to be
used. Note that this keyword is not available in type-switches though,
because it wouldn't work properly with typings.
* refactor: add an abstract copy command interface to avoid code duplication
* fix typo in error message
Co-authored-by: Tejal Desai <tejal29@gmail.com>
* Extend .dockerignore integration test with copies in later stages
.dockerignore should continue to apply when copying from the build context in later stages, but it currently doesn't
* Replace excluded global with passed along FileContext struct
This new FileContext struct allows much cleaner handling of context specific file exclusions.
The global excluded file state is no longer needed.
Additionally this also fixes the issue where excluded files aren't being applied for build context copies in later build stages.
Cached COPY layers are expensive in that they both need to be retrieved
over the network and occupy space in the layer cache.
They are unnecessary in that we already have all resources needed to
execute the COPY locally, and doing so is a trivial file-system
operation. This is in contrast to RUN layers, which can do
arbitrary and unbounded work.
The end result is that cached COPY commands were more expensive when
cached, not less. Remove them.
Resolves#1357
* Update cached copy command to return the same result for
files used from context so that cached and uncached copy
commands produce the same cache key
* Update tests for fix
* Add test for cached run command key consistency
* use the cachekey of the src stage rather than the digest
for COPY --from commands as they are reproducible unlike digests
* track digest to cache keys and stage indexes to digest
* add extra debug logging for troubleshooting cachekey building issues
* convert Sha256 hashes to hex encoded strings rather than plain strings
for easier human reading
This change calculates the exact files and directories needed between
stages used in the COPY command. Instead of saving the entire
stage as a tarball, we now save only the necessary files.
We previously had an optimization that would skip snapshotting mutli-stage images
when in an intermediate stage, until the very end.
This conflicted with another optimization to avoid snapshotting when no files had changed.
Ramy
Diego Gonzalez
Jason Hall
Eng Zer Jun
Dawei Ma
Janosch Maier
Gilbert Gilb's
Tejal Desai
Tinjo Schöni
Ian Kerins
Tejal Desai
Jordan Goasdoue
Tom Prince
Dani Raznikov
Yoan Blanc
Cole Wippern
dlorenc