* Removed block on use --cache-copy-layers with multistage builds
* Removed using digest in composite key with command COPY --from
* COPY --from command uses src as file context (only changed files will be reason for change hash)
* ARG and ENV changed before COPY dont change composite key
* Add and fix some tests
* Caching work same as caching in docker buildx
Co-authored-by: Sergei Kraev <skraev@tradingview.com>
* somehow now the only thing that doesnt work is devices.Device
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* this gets rid of all the compiler errors in the vendored code
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fixed some things but a bunch of tests and maybe some compiler steps are still failing
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* all the things i figured out how to fix so far
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* guess i had to redo go mods after rebasing again
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* update docker constants to be SHOUTY CASE now
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* include DestPath in resolveEnv
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fix one mistake in Docker lib upgrade and some typos/deprecations in the file
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* last changes (hopefully) to update to new docker libs
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
---------
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fix(ci): Bump golangci-lint to 1.51.1
* chore(lint): fix gofmt and goimport issues
* chore(lint): fix linter issues
- Adapted error comparison according to linter recommendation
- Disabled noctx linting for http request where canceling makes no sense
- Disabled nilerror linting where nil error is returned on purpose
- Disabled makezero linter where slice is explicitly deepcopied
* chore(ci): Update go version in tests workflows
* fix(ci): Allow boilerplate years from 2000-2099
Previously the regex only allowed the copyright notice to contain the
years 2018,2019,2020,2021, or 2022. This commit widens to regex to
20\d\d allowing any year in the range [2000-2099]
* feat(ci): Replace minikube with k3s for intregration tests
The existing setup for minikube is very complicated, replicating most of
the setup steps for a full kubernetes cluster in an only partially
supported minikube configuration (driver=none). Furthermore the existing
setup has been broken for sometime, likely, at least in part due to the
changes to CNI and CRI in recent kubernetes versions.
Since what we actually need is only a running Kubernetes cluster on the
node and access to a registry on localhost:5000, we can switch the
extremely complicated minikube setup for a lightweight cluster using
k3s. Minikube came with a default addon for running a registry on every
node, but the same is not the case for k3s, instead we make use of the
package helm controller and its HelmChart CR to deploy twuni/docker-registry.helm
and expose it on localhost using the integrated LoadBalancer controller.
* fix(test-684): pin base container version
The dockerfile for the regression test connected to issue 684 used a
rolling tag as base image, making it flaky and fail since it was
introduced.
This commit pins the base image to the digest of bionic-20200219, which,
based on the date of the commit that introduced to the dockerfile would
be the most newest ubuntu build and likely what the "rolling" tag
resolved to back then. Since this also an image from the pre-oci days of
ubuntu, this circumvents a bug in container-diff as well
(https://github.com/GoogleContainerTools/container-diff/issues/389)
* fix: Correct flatten function in layers
- Added a test.
- Cache current image, track deletes in `whiteouts` as well as normal adds in `layers`.
- Fix ugly delete behavior of `layerHashCache`.
Delete it when crerating a new snapshot.
- Slight cleanup in `snapshot.go`.
- Format ugly `WalkFS` function.
* fix: Add symbolic link changes to Hasher and CacheHasher
* fix: Better log messages
* fix(ci): Integration tests
* fix(ci): Add `--no-cache` to docker builds
* fix(ci): Pass credentials for error integration test
* np: Missing .gitignore in `hack`
* np: Capitalize every log message
- Correct some linting.
* fix: Key function
- Merge only last layer onto `currentImage`.
* fix: Remove old obsolete `cacheHasher`
* Bump github.com/docker/docker to latest release
* Pin deps to docker/docker's requirements
fix build breakages
fix unit test
Trying to see whether am-i-in-a-container checks are broken for integration tests...
update stage code so that when comparing the BaseName of
a stage against the recorded, lowercase version of a Stage name
the BaseName is also lowercased.
The third library moby/buildkit lowers the image alias used in 'FROM .. AS' instruction.
This change fixes this issue by making the resolve of dependencies agnostic to case.
Fixes#592Fixes#770
This change calculates the exact files and directories needed between
stages used in the COPY command. Instead of saving the entire
stage as a tarball, we now save only the necessary files.
This PR adds support for the dockerignore file. Previously when kaniko
had support for the dockerignore file, kaniko first went through the
build context and deleted files that were meant to be ignored. This
resulted in a really bad bug where files in user mounted volumes would
be deleted (my bad).
This time around, instead of modifying the build context at all, kaniko
will check if a file should be excluded when executing ADD/COPY
commands. If a file should be excluded (based on the .dockerignore) it
won't be copied over from the buildcontext and shouldn't end up in the
final image.
I also added a .dockerignore file and Dockerfile as an integration test,
which should fail if the dockerignore is not being processed correctly or if files aren't being excluded correctly.
Also, I removed all the integration testing from the previous version of the
dockerignore support.
Right now kaniko only supports COPY --from=<another stage>.
This commit adds support for the case where the referenced image is a remote image
in a registry that has not been used as a stage yet in the build.
* parse arg commands at the top of dockerfiles
* fix pointer reference bug and remove debugging
* fixing tests
* account for meta args with no value
* don't take fs snapshot if / is the only changed path
* move metaArgs inside KanikoStage
* removing unused property
* check for any directory instead of just /
* remove unnecessary check
I improved handling of the .dockerignore file by:
1. Using docker's parser to parse the .dockerignore and using their
helper functions to determine if a file should be deleted
2. Copying the Dockerfile we are building to /kaniko/Dockerfile so that
if the Dockerfile is specified in .dockerignore it won't be deleted, and
if it is specified in the .dockerignore it won't end up in the final
image
3. I also improved the integration test to create a temp directory with
files to ignore, and updated the .dockerignore to include exclusions (!)
gometalinter is broken @ HEAD, and I looked into why that was. During
that process, I remembered that we took the linting scripts from
skaffold, and found that in skaffold gometalinter was replaced with
GolangCI-Lint:
https://github.com/GoogleContainerTools/skaffold/pull/619
The change made linting in skaffold faster, so I figured instead of
fixing gometalinter it made more sense to remove it and replace it with
GolangCI-Lint for kaniko as well.
I added a KanikoStage to hold each stage of the Dockerfile along with
information about each stage that would be useful later on.
The new KanikoStage type holds the stage itself, along with some
additional information:
1. FinalStage -- whether the current stage is the final stage
2. BaseImageStoredLocally/BaseImageIndex -- whether the base image for
this stage is stored locally, and if so what the index of the base image
is
3. SaveStage -- whether this stage needs to be saved for use in a future
stage
This is the first part of a larger refactor for building stages, which
will later make it easier to add layer caching.
This change should fix the bug in #294, where kaniko wasn't recognizing
that a stage would be used in a later build and so wasn't saving it as a
tarball.
Each stage of the Dockerfile has a Name and a BaseName (FROM BaseName as
Name), but if a Name isn't specified then it's set to the same value as
BaseName. Our test cases weren't complete enough to catch this
distinction, which is why this bug occurred.
I added more test cases to the unit tests to make sure this fix works.
Mick van Gelderen
Aaron Prindle
Manish Giri
geekvest
Kraev Sergei
Joe Kimmel
Joël Pepper
dmr
Gabriel Nützi
François JACQUES
Florian Apolloner
Jason Hall
Tejal Desai
Tejal Desai
Lukasz Jakimczuk
Jordan Goasdoué
Dani Raznikov
cvgw
Ben Einaudi
Taylor Barrella
dlorenc
linuxshokunin
Priya Wadhwa
Sharif Elgamal
Benjamin Nørgaard