* Create directories with the right UID/GID
* Forgot to create the actual directory
* Integration test creation of intermediate files with correct ownership
* ADD version of the test
filepath.Clean shows up in profiles as a hot spot, and there seem to be
many redundant calls, particularly in ignorelist handling. We can avoid
these redundant calls by pre-cleaning entries in the ignore list, and
providing fast paths when we know we're already dealing with a cleaned
candidate path.
Before:
580ms 3.03% 72.35% 590ms 3.08% path/filepath.(*lazybuf).append (inline)
390ms 2.03% 74.39% 990ms 5.16% path/filepath.Clean
After:
0.13s 0.69% 84.01% 0.17s 0.91% path/filepath.(*lazybuf).append (inline)
0.13s 0.69% 84.70% 0.31s 1.65% path/filepath.Clean
* Removed block on use --cache-copy-layers with multistage builds
* Removed using digest in composite key with command COPY --from
* COPY --from command uses src as file context (only changed files will be reason for change hash)
* ARG and ENV changed before COPY dont change composite key
* Add and fix some tests
* Caching work same as caching in docker buildx
Co-authored-by: Sergei Kraev <skraev@tradingview.com>
* somehow now the only thing that doesnt work is devices.Device
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* this gets rid of all the compiler errors in the vendored code
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fixed some things but a bunch of tests and maybe some compiler steps are still failing
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* all the things i figured out how to fix so far
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* guess i had to redo go mods after rebasing again
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* update docker constants to be SHOUTY CASE now
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* include DestPath in resolveEnv
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fix one mistake in Docker lib upgrade and some typos/deprecations in the file
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* last changes (hopefully) to update to new docker libs
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
---------
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fix(ci): Bump golangci-lint to 1.51.1
* chore(lint): fix gofmt and goimport issues
* chore(lint): fix linter issues
- Adapted error comparison according to linter recommendation
- Disabled noctx linting for http request where canceling makes no sense
- Disabled nilerror linting where nil error is returned on purpose
- Disabled makezero linter where slice is explicitly deepcopied
* chore(ci): Update go version in tests workflows
* fix(ci): Allow boilerplate years from 2000-2099
Previously the regex only allowed the copyright notice to contain the
years 2018,2019,2020,2021, or 2022. This commit widens to regex to
20\d\d allowing any year in the range [2000-2099]
* feat(ci): Replace minikube with k3s for intregration tests
The existing setup for minikube is very complicated, replicating most of
the setup steps for a full kubernetes cluster in an only partially
supported minikube configuration (driver=none). Furthermore the existing
setup has been broken for sometime, likely, at least in part due to the
changes to CNI and CRI in recent kubernetes versions.
Since what we actually need is only a running Kubernetes cluster on the
node and access to a registry on localhost:5000, we can switch the
extremely complicated minikube setup for a lightweight cluster using
k3s. Minikube came with a default addon for running a registry on every
node, but the same is not the case for k3s, instead we make use of the
package helm controller and its HelmChart CR to deploy twuni/docker-registry.helm
and expose it on localhost using the integrated LoadBalancer controller.
* fix(test-684): pin base container version
The dockerfile for the regression test connected to issue 684 used a
rolling tag as base image, making it flaky and fail since it was
introduced.
This commit pins the base image to the digest of bionic-20200219, which,
based on the date of the commit that introduced to the dockerfile would
be the most newest ubuntu build and likely what the "rolling" tag
resolved to back then. Since this also an image from the pre-oci days of
ubuntu, this circumvents a bug in container-diff as well
(https://github.com/GoogleContainerTools/container-diff/issues/389)
WORKDIR ignores the currently set USER and creates the new directories
with the root user ownership.
This changes that, by executing a chown after the mkdir if needed, and
also handle the case where the provided USER is an uid and the passwd
file is not available to resolve to the username.
Fixes#2259
Signed-off-by: Aris Buzachis <buzachis.aris@gmail.com>
Signed-off-by: Aris Buzachis <buzachis.aris@gmail.com>
* fix: getUIDandGID is able to resolve non-existing users and groups
A common pattern in dockerfiles is to provide a plain uid and gid number, which doesn't neccesarily exist inside the os.
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* test: add chown dockerfile
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: format
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: add comment
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* tests: fix chown dockerfile
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* refactor: split up getIdsFromUsernameAndGroup func
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* fix: implement raw uid logic for LookupUser
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* test: add dockerfiles for integration test
* fix: lookup user error message
* test: add dockerfiles for non-existing user testcase
* fix: forgot error check
* tests: fix syscall credentials test
* chore: add debug output for copy command
* tests: set specific gid for integration dockerfile
* tests: fix syscall credentials test
github runner had the exact uid that i was testing on, so the groups were not empty
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* tests: fix test script
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: apply golangci lint checks
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* fix: reset file ownership in createFile if not root owned
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore: logrus.Debugf missed format variable
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* chore(test-script): remove go html coverage
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* test(k8s): increase wait timeout
Signed-off-by: Höhl, Lukas <lukas.hoehl@accso.de>
* fix: Correct flatten function in layers
- Added a test.
- Cache current image, track deletes in `whiteouts` as well as normal adds in `layers`.
- Fix ugly delete behavior of `layerHashCache`.
Delete it when crerating a new snapshot.
- Slight cleanup in `snapshot.go`.
- Format ugly `WalkFS` function.
* fix: Add symbolic link changes to Hasher and CacheHasher
* fix: Better log messages
* fix(ci): Integration tests
* fix(ci): Add `--no-cache` to docker builds
* fix(ci): Pass credentials for error integration test
* np: Missing .gitignore in `hack`
* np: Capitalize every log message
- Correct some linting.
* fix: Key function
- Merge only last layer onto `currentImage`.
* fix: Remove old obsolete `cacheHasher`
* Fix resolving argument over mulit-stage build
- Building multi-stage Dockerfiles did not correctly resolve
ARG commands over different stages. Now, each stage depends on the
build arguments set by the stage before.
Closes: #1911
* Fix resolving empty arg commands
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Bump github.com/docker/docker to latest release
* Pin deps to docker/docker's requirements
fix build breakages
fix unit test
Trying to see whether am-i-in-a-container checks are broken for integration tests...
* chore: add workflows for pr tests
* fix unit tests
* fix formatting
* chore: fix gobuild
* change minikube script
* chore: fix lint install script
* chore: ignore and fix tests
* fix lint and run gofmt
* lint fixes
* k8s executor image only
* fix Makefile
* fix travis env variables
* more info on k8s tests
* fix travis run
* fix
* fix
* fix
* fix log
* some more changes
* increase timeout
* delete travis.yml and fix multiple copy tests
* fix registry mirror
* fix lint
* add concurency
* last attemot to fix k8 integrations
* diff id for diff workflows
* Fix composite cache key for multi-stage copy command (#1706)
PR #1518 reintroduced COPY layers caching using the `--cache-copy-layers`
flag. Unfortunately, this PR also introduced a bug by not including the
stage digest into the caching key of the COPY command when the
`--cache-copy-layers` flag was not set. As a result, kaniko would use
any previous (possibly stalled) layer from the cache because the digest
of the "COPY --from" command would never change.
PR author probably expected Go to fallthrough in the switch just like C
does. However, this is not the case. Go does not fallthrough in
switch-statements by default and requires the fallthrough keyword to be
used. Note that this keyword is not available in type-switches though,
because it wouldn't work properly with typings.
* refactor: add an abstract copy command interface to avoid code duplication
* fix typo in error message
Co-authored-by: Tejal Desai <tejal29@gmail.com>
* Extend .dockerignore integration test with copies in later stages
.dockerignore should continue to apply when copying from the build context in later stages, but it currently doesn't
* Replace excluded global with passed along FileContext struct
This new FileContext struct allows much cleaner handling of context specific file exclusions.
The global excluded file state is no longer needed.
Additionally this also fixes the issue where excluded files aren't being applied for build context copies in later build stages.
Cached COPY layers are expensive in that they both need to be retrieved
over the network and occupy space in the layer cache.
They are unnecessary in that we already have all resources needed to
execute the COPY locally, and doing so is a trivial file-system
operation. This is in contrast to RUN layers, which can do
arbitrary and unbounded work.
The end result is that cached COPY commands were more expensive when
cached, not less. Remove them.
Resolves#1357
Currently the default WorkingDir in test is "/", while in reallife it's
empty.
This change the tests to reflect reel life and fix the case where
First WorkingDir is relative.
Manish Giri
Adrià Garriga-Alonso
Quan Zhang
JeromeJu
Aaron Lehmann
Kraev Sergei
Joe Kimmel
Joël Pepper
Aris Buzachis
Ramy
Lukas
Gabriel Nützi
Eng Zer Jun
Jason Hall
Gilbert Gilb's
Tejal Desai
Tinjo Schöni
Ian Kerins
Tejal Desai
aca
Mehdi Abaakouk
Dani Raznikov
Thomas Stromberg