filepath.Clean shows up in profiles as a hot spot, and there seem to be
many redundant calls, particularly in ignorelist handling. We can avoid
these redundant calls by pre-cleaning entries in the ignore list, and
providing fast paths when we know we're already dealing with a cleaned
candidate path.
Before:
580ms 3.03% 72.35% 590ms 3.08% path/filepath.(*lazybuf).append (inline)
390ms 2.03% 74.39% 990ms 5.16% path/filepath.Clean
After:
0.13s 0.69% 84.01% 0.17s 0.91% path/filepath.(*lazybuf).append (inline)
0.13s 0.69% 84.70% 0.31s 1.65% path/filepath.Clean
* Rename IgnoreListPath to MountInfoPath in config & constants
The string points to /proc/self/mountinfo
* fs_util_test.go: fix tests failing when /tmp mountpoint present
The tests
* Test_GetFSFromLayers_ignorelist
* Test_GetFSFromLayers_with_whiteouts_include_whiteout_disabled
* Test_GetFSFromLayers_with_whiteouts_include_whiteout_enabled
were failing on systems with a /tmp mountpoint:
fs_util.InitIgnoreList() adds all mountpoints to the ignore list,
but the tests were expecting file operations in a /tmp subdirectory.
This change provides an empty mountinfo list for the affected tests.
Fixes#1779
* somehow now the only thing that doesnt work is devices.Device
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* this gets rid of all the compiler errors in the vendored code
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fixed some things but a bunch of tests and maybe some compiler steps are still failing
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* all the things i figured out how to fix so far
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* guess i had to redo go mods after rebasing again
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* update docker constants to be SHOUTY CASE now
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* include DestPath in resolveEnv
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* fix one mistake in Docker lib upgrade and some typos/deprecations in the file
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* last changes (hopefully) to update to new docker libs
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
---------
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* avoid deleting whiteouts if they are in ignore list
* fix snapshotter ignore list
* include filesystem mounts in ignorelist of snapshotter
* clean up ignore list logic
* add unit and integration tests for #1652
* fix tests and ignore list updates
* Extend .dockerignore integration test with copies in later stages
.dockerignore should continue to apply when copying from the build context in later stages, but it currently doesn't
* Replace excluded global with passed along FileContext struct
This new FileContext struct allows much cleaner handling of context specific file exclusions.
The global excluded file state is no longer needed.
Additionally this also fixes the issue where excluded files aren't being applied for build context copies in later build stages.
This allows those using Kaniko as a library to add ignored dirs and
files without needs to modify /proc/self/mountinfo or doing other
strange things with mount -t tmpfs
This refactoring reversed the order of the "ParentDirectories" function: ffc372a63b (diff-d36eb675aa49a7b471e3a2be77005b18R465)
As a side-effect, parent directories weren't added in lexicographical
order, which broke some tests. We now ensure in unit test that the order
of the ParentDirectories function is stable.
The zero value of time.Time is not a valid argument to os.Chtimes
because of the syscall that os.Chtimes calls. Instead we update the zero
value of time.Time to the zero value of Unix Epoch
Add a new option additonal-whitelist which defaults
to a single entry, "/var/run". This will allow users to
remove "/var/run" from the whitelist or retain the current
behavior with no change.
Previously kaniko would compute the cache key for any copy command by computing
the combined hash of all files in a directory, even if they were listed
as ignored.
With this change, the cache key creation was updated to be aware of ignored
files.
Related issues:
* https://github.com/GoogleContainerTools/kaniko/issues/594
Certain utilities like Apt depend on modtime
for certain files. Kaniko was not setting modtime when
extracting files and so this broke things like apt.
Kaniko now sets the file mod time to the value from the tar
header.
Update caching run and copy commands to use the new
GetFSFromLayers method and include the whiteout option so that
whiteout files are extracted and included in extractedFiles
* add util.GetFSFromLayers
* GetFSFromImage delegates to GetFSFromLayers
* add FSOpts and FSConfig for GetFSFromLayers
* add tests for GetFSFromLayers
* add gomock for test support
* add mock_v1 for layers
When using the COPY command, if the source and destination have the same
the file should be skipped rather than copied. This is to prevent the
file from being overwritten and therefore producing an empty file.
fixes#904
Matthias Schneider
Manish Giri
tal66
Aaron Lehmann
Andreas Fleig
Joe Kimmel
Eng Zer Jun
Andrei Kvapil
Kamal Nasser
Jon Friesen
Tinjo Schöni
Joe Kutner
Tejal Desai
Tejal Desai
Gilbert Gilb's
cvgw
xanonid
Moritz Wanzenböck
Cole Wippern
Logan.Price
poy