* chore: go get -u ./...
* chore: go mod tidy
* chore: go mod vendor
* chore: fix compilation for buildkit >= 0.15.0
* chore: upgrade to Go 1.24
* chore: upgrade the Debian container used in an integration test
* when using wildcard hitting ResolveEnvAndWildcards ignore error when there is no file match
* adding unit test
* fix unit test wrong check in expected
* fix unit test for make sure there is no file copied
* copy with wildcard warning and success when no match file
* fix Test_IsSrcsValid since its changes the behaviour
* For each registry mapping, represent it by a new instance of Repository and
create a new Reference containing it.
* Improve registry mapping parser
* Add more unit tests to cover more use cases
* Fix#3032: Remove query parameters in ADD command when the destination is a directory
* fixing linter URL sorry forget to lint
* add error in extractFilename and realize that ResolveEnvironmentReplacement better execute before getting the filename
* add /path possible in registry maps and/or mirror
* Fixing Unit test Test_ExtractPathFromRegistryURL
* fix typo library
* fix unit test on ExtractPathFromRegistryURL
* fixing go lint url
* fix typo s/ectract/extract
If git clone context is a private self-signed repository, we allow user
to add --git insecure-skip-tls=true flag in the option. The value is
default to false, this behavior is in accordance with the go-git
package.
* Add flag to remap registries for any registry mirror
The purpose of this PR is to add an option to remap registries, a kind of generalized `--registry-mirror`.
This is helpful for air-gapped environments and/or when local registry mirrors are available (not limited to docker.io).
This allows user to reference any images without having to change their location.
It also permit to separate infra related configuration (the mirrors) from CI/CD pipeline definition by using an environment variable for example (the reason behind the early return if flag provided but empty).
Therefore you can have a pipeline calling kaniko with `--registry-map=$REGISTRY_MAP` and have the `REGISTRY_MAP` populated via the runner's env by another team, and the absence of env wouldn't trigger a failure, it makes the pipeline env independent.
I've also considered the option of environment variables directly but it doesn't seems to be in kaniko's philosophy.
This makes quite some duplicated code :/ One option to keep the mirror flag and behavior would be to use only one codebase and convert `--registry-mirror=VALUE` to `--registry-map=index.docker.io=VALUE` internally. Suggestions welcome!
* Configure logging config sooner to be able to use it in flag parsing
* Replace registry mirrors by maps logic and use env var
* Add env vars to README.md
* Fix test
* Prevent extra snapshot when using new run
* Add unit tests for initializing snapshotter
There should be no snapshot for RunV2. Added a test for SingleSnapshot
as well to prove that the test actually works (rather than `initialized`
just not being read or set properly).
* Add an integration test to reproduce #2892
* Fix go compilation
* Fix docker run cmd
* Fixing entrypoint
* Test warmer with cache in a volume.
* Add missing comma
* Fix imports
* Fix dir
* Add logs
* fix
* Use test framework to log
* Fix warmer failing if image already in cache.
* Fix format.
This commit adds the skip option for otiai10.Copy to skip the /kaniko
directory when the root is being copied. The files under /kaniko dir
should be ignored and thus this shall not cause any loss of information.
fixes: GoogleContainerTools#2033
* Fix warmer memory leak. Write down images directly into a temp file. Add a script to test warmer in boxed memory conditions. Fixes: #2754
* Document usage of boxed_warm_in_docker.sh integration test.
* Create directories with the right UID/GID
* Forgot to create the actual directory
* Integration test creation of intermediate files with correct ownership
* ADD version of the test
* feat: add a retry with result function enabled by --image-download-retry (#2853)
* impl: add a retry with result function
* fix ci errs
* test: add unit tests
* gofmt
* make debian a const
* update param description
This feature allows one to specify an https URL for any of the
digest-file options, resulting in an HTTP PUT to the provided
URL. This could for example be a (pre-signed) URL to S3 or GCS.
Currently the final digest is only written to the local filesystem,
which disappears and is not accessible when Kaniko is run in a
managed container service like AWS ECS.
By supporting https a single implementation supports all storage
services, without the need for special code for S3, GCS, etc..
`sync` system call triggers a full page cache sync which may not always
work, especially in kubernetes environment where it is easy to be
interfered by others. I have seen several cases where a broken nfs mount
is blocking kaniko from doing its job.
With `syncfs`, it only writes cache back to disk for the current
filesystem that is used by kaniko which is supposed to be more reliable.
Ensure zstd compression only gets applied to oci images.
When adding a layer to an image ensure that they are compatable if not convert them.
Create function to convert mediatypes between oci and docker types.
This commit changes the condition check for the behavior when no-push is
set to true while destinations are needed. Prior this change, users would
have to set destinations even when noPush option is set to true. More
specifically, a workaround for tar files to be generated when --no-push is
true and destinations is empty is provided where a dummy destination would be
set.
filepath.Clean shows up in profiles as a hot spot, and there seem to be
many redundant calls, particularly in ignorelist handling. We can avoid
these redundant calls by pre-cleaning entries in the ignore list, and
providing fast paths when we know we're already dealing with a cleaned
candidate path.
Before:
580ms 3.03% 72.35% 590ms 3.08% path/filepath.(*lazybuf).append (inline)
390ms 2.03% 74.39% 990ms 5.16% path/filepath.Clean
After:
0.13s 0.69% 84.01% 0.17s 0.91% path/filepath.(*lazybuf).append (inline)
0.13s 0.69% 84.70% 0.31s 1.65% path/filepath.Clean
* Allow to disable the fallback to the default registry on image pull
When one or more registry mirror(s) are deffined with the 'registry-mirror' argument, if none of those mirrors include the image,
the current behavior is to fallback to the default registry.
If a whitelist (or some image restriction) is applied at the mirror side, fallbacking to the default registry makes that restriction useless.
This new argument allows to skip the fallback and abort the build if the mirror rejects an image.
If it is not set, is completelly transparent.
* fix typo on command help
If a non-empty directory gets replaced with something other than a
directory (e.g. file or symlink), the files in that directory also get
deleted. However, there should not be any whiteout files for them in the
layer. If there were, the resulting tar file would not be extractable.
Fixes#2308
* feat: cache dockerfile images through warmer
* Fix logical error in conditional statement
* Addressed review feedback
1. Updated help text for the --build-arg flag to indicate it should be used with the dockerfile flag.
2. Updated the documentation to include the optional --build-arg flag.
3. Added unit tests for `ParseDockerfile`, covering scenarios for missing Dockerfile, invalid Dockerfile, single stage Dockerfile, multi-stage Dockerfile and Args Dockerfile
---------
Co-authored-by: 连奔驰 <benchi.lian@thoughtworks.com>
* Rename IgnoreListPath to MountInfoPath in config & constants
The string points to /proc/self/mountinfo
* fs_util_test.go: fix tests failing when /tmp mountpoint present
The tests
* Test_GetFSFromLayers_ignorelist
* Test_GetFSFromLayers_with_whiteouts_include_whiteout_disabled
* Test_GetFSFromLayers_with_whiteouts_include_whiteout_enabled
were failing on systems with a /tmp mountpoint:
fs_util.InitIgnoreList() adds all mountpoints to the ignore list,
but the tests were expecting file operations in a /tmp subdirectory.
This change provides an empty mountinfo list for the affected tests.
Fixes#1779
* Removed block on use --cache-copy-layers with multistage builds
* Removed using digest in composite key with command COPY --from
* COPY --from command uses src as file context (only changed files will be reason for change hash)
* ARG and ENV changed before COPY dont change composite key
* Add and fix some tests
* Caching work same as caching in docker buildx
Co-authored-by: Sergei Kraev <skraev@tradingview.com>
Michael Plump
Mick van Gelderen
Aaron Prindle
Leo Palmer Sunmo
Prima Adi Pradana
Marc Lallaouret
Samarth08
Matthias Schneider
Alessandro Bitocchi
Jérémie Augustin
Matheus Pimenta
schwannden
Damien Degois
Kraev Sergei
Asher
Bob Du
Maxime BOSSARD
JeromeJu
Manish Giri
Adrià Garriga-Alonso
Anna Levenberg
tal66
Lio李歐
Anna Levenberg
Quan Zhang
zhouhaibing089
Vishal Khot
Logan Price
guangwu
geekvest
Diego Gonzalez
Julian
Aaron Lehmann
Fernando Giannetti
Andreas Fleig
alexezio
Kraev Sergei