* Removed block on use --cache-copy-layers with multistage builds
* Removed using digest in composite key with command COPY --from
* COPY --from command uses src as file context (only changed files will be reason for change hash)
* ARG and ENV changed before COPY dont change composite key
* Add and fix some tests
* Caching work same as caching in docker buildx
Co-authored-by: Sergei Kraev <skraev@tradingview.com>
* fix: getUIDandGID is able to resolve non-existing users and groups
A common pattern in dockerfiles is to provide a plain uid and gid number, which doesn't neccesarily exist inside the os.
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* test: add chown dockerfile
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: format
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: add comment
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* tests: fix chown dockerfile
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* refactor: split up getIdsFromUsernameAndGroup func
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* fix: implement raw uid logic for LookupUser
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* test: add dockerfiles for integration test
* fix: lookup user error message
* test: add dockerfiles for non-existing user testcase
* fix: forgot error check
* tests: fix syscall credentials test
* chore: add debug output for copy command
* tests: set specific gid for integration dockerfile
* tests: fix syscall credentials test
github runner had the exact uid that i was testing on, so the groups were not empty
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* tests: fix test script
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: apply golangci lint checks
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* fix: reset file ownership in createFile if not root owned
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: logrus.Debugf missed format variable
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore(test-script): remove go html coverage
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* test(k8s): increase wait timeout
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* fix: Correct flatten function in layers
- Added a test.
- Cache current image, track deletes in `whiteouts` as well as normal adds in `layers`.
- Fix ugly delete behavior of `layerHashCache`.
Delete it when crerating a new snapshot.
- Slight cleanup in `snapshot.go`.
- Format ugly `WalkFS` function.
* fix: Add symbolic link changes to Hasher and CacheHasher
* fix: Better log messages
* fix(ci): Integration tests
* fix(ci): Add `--no-cache` to docker builds
* fix(ci): Pass credentials for error integration test
* np: Missing .gitignore in `hack`
* np: Capitalize every log message
- Correct some linting.
* fix: Key function
- Merge only last layer onto `currentImage`.
* fix: Remove old obsolete `cacheHasher`
* chore: add workflows for pr tests
* fix unit tests
* fix formatting
* chore: fix gobuild
* change minikube script
* chore: fix lint install script
* chore: ignore and fix tests
* fix lint and run gofmt
* lint fixes
* k8s executor image only
* fix Makefile
* fix travis env variables
* more info on k8s tests
* fix travis run
* fix
* fix
* fix
* fix log
* some more changes
* increase timeout
* delete travis.yml and fix multiple copy tests
* fix registry mirror
* fix lint
* add concurency
* last attemot to fix k8 integrations
* diff id for diff workflows
* Fix composite cache key for multi-stage copy command (#1706)
PR #1518 reintroduced COPY layers caching using the `--cache-copy-layers`
flag. Unfortunately, this PR also introduced a bug by not including the
stage digest into the caching key of the COPY command when the
`--cache-copy-layers` flag was not set. As a result, kaniko would use
any previous (possibly stalled) layer from the cache because the digest
of the "COPY --from" command would never change.
PR author probably expected Go to fallthrough in the switch just like C
does. However, this is not the case. Go does not fallthrough in
switch-statements by default and requires the fallthrough keyword to be
used. Note that this keyword is not available in type-switches though,
because it wouldn't work properly with typings.
* refactor: add an abstract copy command interface to avoid code duplication
* fix typo in error message
Co-authored-by: Tejal Desai <tejal29@gmail.com>
* Extend .dockerignore integration test with copies in later stages
.dockerignore should continue to apply when copying from the build context in later stages, but it currently doesn't
* Replace excluded global with passed along FileContext struct
This new FileContext struct allows much cleaner handling of context specific file exclusions.
The global excluded file state is no longer needed.
Additionally this also fixes the issue where excluded files aren't being applied for build context copies in later build stages.
Cached COPY layers are expensive in that they both need to be retrieved
over the network and occupy space in the layer cache.
They are unnecessary in that we already have all resources needed to
execute the COPY locally, and doing so is a trivial file-system
operation. This is in contrast to RUN layers, which can do
arbitrary and unbounded work.
The end result is that cached COPY commands were more expensive when
cached, not less. Remove them.
Resolves#1357
Update caching run and copy commands to use the new
GetFSFromLayers method and include the whiteout option so that
whiteout files are extracted and included in extractedFiles
* Update cached copy command to return the same result for
files used from context so that cached and uncached copy
commands produce the same cache key
* Update tests for fix
* Add test for cached run command key consistency
This change calculates the exact files and directories needed between
stages used in the COPY command. Instead of saving the entire
stage as a tarball, we now save only the necessary files.
This PR adds support for the dockerignore file. Previously when kaniko
had support for the dockerignore file, kaniko first went through the
build context and deleted files that were meant to be ignored. This
resulted in a really bad bug where files in user mounted volumes would
be deleted (my bad).
This time around, instead of modifying the build context at all, kaniko
will check if a file should be excluded when executing ADD/COPY
commands. If a file should be excluded (based on the .dockerignore) it
won't be copied over from the buildcontext and shouldn't end up in the
final image.
I also added a .dockerignore file and Dockerfile as an integration test,
which should fail if the dockerignore is not being processed correctly or if files aren't being excluded correctly.
Also, I removed all the integration testing from the previous version of the
dockerignore support.
JeromeJu
Kraev Sergei
Lukas
Gabriel Nützi
Gilbert Gilb's
Tejal Desai
Tinjo Schöni
Ian Kerins
Dani Raznikov
Yoan Blanc
xanonid
Cole Wippern
Cole Wippern
Wietse Muizelaar
ohchang-kwon
Don McCasland
dlorenc
Priya Wadhwa