* Write parent directories to tar before whiteout files
Fixes#1149
The OCI image spec does not specify this order but it's a good idea and Docker
does the same.
When manually comparing layers created by Docker and Kaniko there are still
some differences (that container-diff does not show):
* Kaniko adds / to layers
* For `mkdir /test`, docker adds `/test` and an opaque whiteout file
`/test/.wh..wh..opq`. Kaniko only adds `/test/` (and /).
* snapshot_test: cleanup
Fix typos and use listFilesInTar() where possible
* Fix missing file permissions on multi-stage build
Fixes#2075
When a file with the setuid bit is copied from one stage
to another, the permissions were not copied over properly after
setting ownership on directory and the file itself.
* Update pkg/util/fs_util.go
Co-authored-by: Jason Hall <jason@chainguard.dev>
* Adding boilerplate to dockerfile
* Add bash check to bail with exit code 1 if setuid not present
Co-authored-by: Jason Hall <jason@chainguard.dev>
* fix: Correct flatten function in layers
- Added a test.
- Cache current image, track deletes in `whiteouts` as well as normal adds in `layers`.
- Fix ugly delete behavior of `layerHashCache`.
Delete it when crerating a new snapshot.
- Slight cleanup in `snapshot.go`.
- Format ugly `WalkFS` function.
* fix: Add symbolic link changes to Hasher and CacheHasher
* fix: Better log messages
* fix(ci): Integration tests
* fix(ci): Add `--no-cache` to docker builds
* fix(ci): Pass credentials for error integration test
* np: Missing .gitignore in `hack`
* np: Capitalize every log message
- Correct some linting.
* fix: Key function
- Merge only last layer onto `currentImage`.
* fix: Remove old obsolete `cacheHasher`
In Dockerfile, if there is something like:
```
RUN setcap cap_net_raw=+ep /path/to/binary
```
kaniko won't detect that there is a change on file `/path/to/binary` and
thus discards this layer. This patch allows the hasher function to
actually look at `security.capability` extended attributes.
Fix#1936
Kubernetes was not being detected by files not by /proc/?/cgroup
contents. Now it detects the kubernetes runtime if any of those
conditions are met:
* /var/run/secrets/kubernetes.io/serviceaccount exists
* /proc/mounts has the mount for "/" with fs type "overlay"
This simplifies usage of Kaniko in GitLab CI environments and
means that it's not longer necessary to manually cobble together
the config with the right values from the environment in
error-prone shell commands.
* Use canonical image architectures. Closes#1995.
Prior to this change, Kaniko was not using the platform's canonical value.
* Update platform unit test.
* remove configurables from constants
* add configurables to config init
* add kaniko dir flag
* cleanup pkg constants
* replace buildcontext constants pkg occurrences with config
* add KanikoDir to KanikoOptions
* replace executor constants pkg occurrences with config
* remove redundant KanikoDir assignment
* replace constants to config for IntermediateStagesDir
* fix imports
* add default kaniko directory into constants
* add check for kanikoDir on use
* update init to use default path constant
* update executor kanikoDir check
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* alter checkKanikoDir parameter
* add TestKanikoDir func
* update error handling style
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* remove shorthand flag usage from test
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* add docstring to integration test
Co-authored-by: Jason Hall <jasonhall@redhat.com>
* remove shorthand flag from kaniko-dir
Co-authored-by: Jason Hall <jasonhall@redhat.com>
Fixed the codeql issue
```
Sanitizing untrusted URLs is an important technique for preventing attacks such as request forgeries and malicious redirections. Often, this is done by checking that the host of a URL is in a set of allowed hosts.
If a regular expression implements such a check, it is easy to accidentally make the check too permissive by not escaping regular-expression meta-characters such as ..
Even if the check is not used in a security-critical context, the incomplete check may still cause undesirable behavior when it accidentally succeeds.
```
Prior to this change, Kaniko disabled logging for its ecr-login
dependency by redirecting the logger's output to ioutil.Discard.
However, since that used the logrus standard logger, this had the effect
of completely disabling (redirecting to Discard) _all_ logging done by
Kaniko.
This caused integratiion test failures, and left any user of Kaniko
without any logging.
This fix updates the dependency on ecr-login to include a fix for this
issue, and updates to the new API introduced in that fix.
Previously the ecr-login cred helper we compile into the executor/warmer
binaries would log a warning when they were asked for credentials for a
non-ECR image. This takes advantage of recent work in ecr-login to let
consumers like kaniko disable that logging.
* Fix resolving argument over mulit-stage build
- Building multi-stage Dockerfiles did not correctly resolve
ARG commands over different stages. Now, each stage depends on the
build arguments set by the stage before.
Closes: #1911
* Fix resolving empty arg commands
* Correctly handle platforms that include CPU variants
Prior to this change, Kaniko would only select the platform-specific
image from a multi-platform image using the OS and architecture to
select the image. This leads to problems when there are two platforms
that are indistinguishable using only this information (e.g.,
linux/arm/v5 and linux/arm/v7).
This change more explicitly selects the right image from a
multi-platform image, taking CPU variant into account (v5 vs v7), using
containerd's CPU variant detection logic.
This also moves platform defaulting up as early as possible in execution
as it can go, so that malformed platform values are detected as soon as
possible.
* set platform in unit test
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Bump github.com/docker/docker to latest release
* Pin deps to docker/docker's requirements
fix build breakages
fix unit test
Trying to see whether am-i-in-a-container checks are broken for integration tests...
When os.Stat returns an error different from ErrNotExist,
mkdirAllWithPermissions may panic with a nil pointer
derefence due to insufficient error checking.
Avoid the panic by bailing out, returning the error to the
caller.
* git: accept explicit commit hash for git context
When checking out code from non-github repositories, the typical
assumptions may not be valid, e.g. that the only interesting
non-branch commits have ref names starting with refs/pull. A specific
example is fetching an un-merged commit from a gerrit repository by
commit hash.
This change just looks at the second part of the git context path and
checks if it's a SHA commit hash, and if so, will fetch and check out
this commit after cloning the repository.
Sample context argument:
https://github.repo/project#e1772f228e06d15facdf175e5385e265b57068c0
* ci: fix test script to recognize any non-zero exit as an error
hack/linter.sh didn't properly install golangci-lint in hack/bin as I
already have another version of golangci-lint on my PATH, but then it
failed to execute because it was looking for it specifically in
hack/bin.
When the executable is not found, the exit code is 127 instead of 1,
and so test.sh ignored the error.
Two fixes:
1. `test.sh`:
- Use `if (script) ...` instead of assigning / checking a result
variable to determine if each validation script passed or failed.
2. `hack/linter.sh`:
- Instead of checking for golangci-lint on the path, just
specifically check for an executable file (`test -x`) in the
expected location.
Co-authored-by: Wade Carpenter <wwade@users.noreply.github.com>
* Remove tarball.WithCompressedCaching flag to resolve OOM Killed error
Large images cannot be build as the kaniko container will be killed due to an OOM error. Removing the tarball compression drastically reduces the memory required to push large image layers. Fixes#1680
This change may increase the build time for smaller images. Therefore a command line option to trigger the compression or a more intelligent behaviour may be useful.
* Add new command line flag to toggle compressed caching
* Add unittest for build with --compressed-caching command line flag set to false
* chore: add workflows for pr tests
* fix unit tests
* fix formatting
* chore: fix gobuild
* change minikube script
* chore: fix lint install script
* chore: ignore and fix tests
* fix lint and run gofmt
* lint fixes
* k8s executor image only
* fix Makefile
* fix travis env variables
* more info on k8s tests
* fix travis run
* fix
* fix
* fix
* fix log
* some more changes
* increase timeout
* delete travis.yml and fix multiple copy tests
* fix registry mirror
* fix lint
* add concurency
* last attemot to fix k8 integrations
* diff id for diff workflows
* Fix composite cache key for multi-stage copy command (#1706)
PR #1518 reintroduced COPY layers caching using the `--cache-copy-layers`
flag. Unfortunately, this PR also introduced a bug by not including the
stage digest into the caching key of the COPY command when the
`--cache-copy-layers` flag was not set. As a result, kaniko would use
any previous (possibly stalled) layer from the cache because the digest
of the "COPY --from" command would never change.
PR author probably expected Go to fallthrough in the switch just like C
does. However, this is not the case. Go does not fallthrough in
switch-statements by default and requires the fallthrough keyword to be
used. Note that this keyword is not available in type-switches though,
because it wouldn't work properly with typings.
* refactor: add an abstract copy command interface to avoid code duplication
* fix typo in error message
Co-authored-by: Tejal Desai <tejal29@gmail.com>
Lukas
Hingbong Lo
Andreas Fleig
Tony De La Nuez
Gabriel Nützi
Jason D'Amour
zhouhaibing089
Víctor
François JACQUES
ePirat
Diego Gonzalez
Jack
Igor Scheller
Naveen
Jason Hall
Florian Apolloner
Eng Zer Jun
Dawei Ma
Andrei Kvapil
Guillaume Calmettes
Lars Seipel
ygelfand
Travis DePrato
Tejal Desai
wwade
Janosch Maier
Gilbert Gilb's
Rhianna
nihilo
Silvano Cirujano Cuesta
Jose Donizetti
Patrick Barker