Commit Graph

947 Commits

Author SHA1 Message Date
PhoenixMage aa6a3de316
Fix 32bit overflow
math.UaxUint32 will actually overflow on a 32 bit platform (tested on armv7)

This will resolve that issue.
2020-03-30 23:37:34 +11:00
Gilbert Gilb's e5585fded8 Always add parent directories of files to snapshots.
During a snapshot, when a file changed and not its parent directories,
the parent directories weren't added to the layer. This is inconsistent
with Docker's behavior which always add parent directories to the layer.
In some edge-cases, it could lead to problems with docker considering
that parent directories where owned by root in forthcoming layers
although they shouldn't (see #1163).

Also, Docker seems to be POSIX compliant regarding the name of
directories in the archive, which always have a slash appended. This
commit also fixes this.

Fixes #1163
2020-03-29 18:25:37 +02:00
Thomas Stromberg a6504769e3 Add secondary group impersonation w/ !cgo support 2020-03-28 22:01:20 -07:00
Thomas Stromberg 3426a24929 Add support for impersonating secondary groups 2020-03-28 12:10:37 -07:00
Jordan Goasdoué ed883b9015 feat: can now resolves args from stage 2020-03-27 18:42:46 +01:00
Tejal Desai 340ca79fbb lint 2020-03-25 11:01:12 -07:00
Tejal Desai 1978f1e08e
Merge pull request #1130 from Vlatombe/set_image_platform
Set image platform for any build
2020-03-25 10:10:41 -07:00
Tejal Desai 9567b755dd fix unit tests 2020-03-25 10:09:33 -07:00
Tejal Desai 2ea28fc7f5 revert back to previous file to save logic 2020-03-25 09:43:59 -07:00
Tejal Desai caaf6c8adf fix tests 2020-03-24 10:50:59 -07:00
Sam Stoelinga 1b662e7834
Merge pull request #1140 from samos123/fix-1122-public-gcr
Add ability to use public GCR repos without being authenticated
2020-03-24 10:31:58 -07:00
Vincent Latombe 336bcd3a9a
Merge branch 'master' into set_image_platform 2020-03-24 09:51:11 +01:00
Tejal Desai 5da32d3f18
Merge pull request #1147 from tejal29/small_perf
small perf optimizing. Only remove whiteout path if it needs to be included in base image
2020-03-23 18:12:39 -07:00
Tejal Desai ffc372a63b refactor to add unit tests 2020-03-23 17:48:49 -07:00
Sam Stoelinga 1fb1eddd8c Test: Remove DockerConf after CheckPushPermission 2020-03-23 10:51:09 -07:00
Shihab Hasan 99b08059c4 Checkout a specific git commit 2020-03-23 12:12:52 +06:00
Sam Stoelinga c2393df7c3 Cleanup push_test CheckPushPermission unit tests 2020-03-22 17:11:08 -07:00
Sam Stoelinga c56f16b163 Create constant for Docker conf location 2020-03-21 16:45:04 -07:00
Sam Stoelinga 8518b37ca1 Add unit tests for CheckPushPermissions 2020-03-21 16:15:34 -07:00
Jordan Goasdoué d08a767454 feat: allow injecting through stdin tar.gz on kaniko 2020-03-18 12:29:49 +01:00
Yoan Blanc 03fcc537c3
Merge remote-tracking branch 'origin/master' into fix-chown 2020-03-18 09:01:15 +01:00
Tejal Desai 67db51810b small perf optimizing. Only remove whiteout path if it needs to be included in the tar 2020-03-18 00:43:55 -07:00
Tejal Desai 0302e5150c
Merge pull request #1085 from tejal29/rm_buildargs_from_cache_key
remove build args from composite key and replace all build args
2020-03-17 16:02:59 -07:00
Tejal Desai a173bf020b
Merge pull request #1115 from JordanGoasdoue/allow-build-context-from-local-targz
feat: add support of local '.tar.gz' file inside the kaniko container
2020-03-17 15:24:10 -07:00
Tejal Desai 0185818bd8
Merge pull request #1134 from dani29/master
Add support to --chown flag to ADD command (Issue #57)
2020-03-17 15:07:02 -07:00
Sam Stoelinga e1e6c55386
Merge pull request #1128 from zhangyuheng/fix/typo
fix: credential typo
2020-03-17 11:26:36 -07:00
Sam Stoelinga 66268690b3 Add comments 2020-03-17 09:11:50 -07:00
Sam Stoelinga 8a020010b7 Add ability to use public GCR image
Kaniko by default used to configure the GCR credential helper however
this caused Kaniko to fail when trying to use a base image from a public
GCR image. This patch makes it possible to use public GCR images as base
image when using docker even when you're not authenticated to GCR.

Co-authored-by: Nate Williams <nate.williams@files.com>
2020-03-17 09:09:41 -07:00
Yoan Blanc 3ff2ee3ba5
copy: unit test for chown
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-17 15:19:00 +01:00
Dani Raznikov c136f886a9 Add support to --chown flag to ADD command 2020-03-14 16:24:40 +02:00
Jordan GOASDOUE fda7ed4f74 feat: add support of local '.tar.gz' file inside the kaniko container 2020-03-12 22:51:30 +01:00
Tejal Desai 32decf5798
Merge pull request #1075 from greut/label-flag
executor: add --label flag
2020-03-12 13:47:01 -07:00
Tejal Desai 18de5d6d2d
Merge pull request #1037 from antechrestos/feature/add_option_to_import_registry_certificate
Allow user to provide registry certificate
2020-03-12 12:56:40 -07:00
Vincent Latombe 4d26c1f51e
Set OS/ARCH when assembling image 2020-03-12 10:41:32 +01:00
Yuheng Zhang 8eef58939b fix: credential typo 2020-03-12 14:31:15 +08:00
Tejal Desai 9592f2640f more tests similar to docker cp 2020-03-09 15:54:35 -07:00
Ben Einaudi b73c2c10c8 Allow user to provide registry certificate
Fixes #1100
Fixes #1101
2020-03-08 18:17:50 +01:00
Tejal Desai 2181c5e6f5 create correct link across Multistage builds 2020-03-07 11:28:16 -08:00
Yoan Blanc 53326fa0bf
executor: add --label flag
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-07 18:17:26 +01:00
Tejal Desai c523c691eb revert back to old 0.17.1 behavior 2020-03-06 23:48:31 -08:00
Tejal Desai 8f87267002 fix lint error 2020-03-06 21:37:00 -08:00
Tejal Desai 6c14d202a3 better error wrapping and add more tests for copy 2020-03-06 17:18:36 -08:00
Tejal Desai b4b70d040a wip 2020-03-06 13:08:02 -08:00
Tejal Desai 3cc8aa2a9c fix resolve link for dirs with trailing / 2020-03-06 09:38:00 -08:00
Yoan Blanc fc668cd2b5
fixup! fixup! fixup! run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 17:26:04 +01:00
Yoan Blanc bd77f131fd
fixup! fixup! run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 17:00:14 +01:00
Yoan Blanc 67fbdef360
fixup! run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 16:49:13 +01:00
Yoan Blanc c7028c6d2f
run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 16:44:55 +01:00
Yoan Blanc c553184416
user_test: docker doesn't care about group really
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 16:34:06 +01:00
Tejal Desai 579ec52a30 add more tests for stageBuilder_build 2020-03-04 17:04:42 -08:00
Tejal Desai b9f4c4bab5 add more tests for stageBuilder_build 2020-03-04 16:57:26 -08:00
Tejal Desai 9e83210b07 Rebase over 1105 2020-03-04 14:16:24 -08:00
Yoan Blanc 3d9bb21f75 user: keep existing HOME env
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-04 14:14:25 -08:00
Yoan Blanc fec7c126f9 run: set userStr
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-04 14:14:10 -08:00
Yoan Blanc 52acd798f8 run: unset any HOME variable after USER ran
Running USER didn't reset HOME which had to be explicitly set.

Closes #1082

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-04 14:12:42 -08:00
Tejal Desai c1e3f42625 fix user metadata string 2020-03-04 13:08:38 -08:00
Tejal Desai dd5d2e6661 ignore errors while deleting 2020-02-28 04:48:22 -08:00
Tejal Desai 17f808da8a check for filepath.Walk error everywhere 2020-02-28 04:23:27 -08:00
Tejal Desai ae5a7bf158 fix tests 2020-02-28 02:53:42 -08:00
Tejal Desai 58c0d47fa6 fix npe 2020-02-28 02:44:43 -08:00
Tejal Desai 2ba7d25090 remove build args from composite key and replace all build args before adding command to the cache 2020-02-28 01:51:31 -08:00
Tejal Desai a1af057f99
Merge pull request #1069 from cvgw/u/cgwippern/implement-filepath-resolver
Resolve filepaths before scanning for changes
2020-02-25 00:19:38 -08:00
cvgw 60e2187d85 Fix #1020 os.Chtimes invalid arg
The zero value of time.Time is not a valid argument to os.Chtimes
because of the syscall that os.Chtimes calls. Instead we update the zero
value of time.Time to the zero value of Unix Epoch
2020-02-24 08:29:19 -08:00
cvgw 965b606720 remove cruft and unneeded loop 2020-02-23 13:38:08 -08:00
cvgw 01f6aba517 update resolveSymlinkAncestor and add tests 2020-02-22 11:17:39 -08:00
cvgw a675ad998a Resolve filepaths before scanning for changes 2020-02-20 09:45:44 -08:00
Yoan Blanc 474fdcf68d
image: fix registry mirror setup
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-02-16 11:41:51 +01:00
Yoan Blanc edc4eb33f4
Merge remote-tracking branch 'origin/master' into registry-mirror 2020-02-16 09:28:54 +01:00
Tejal Desai 2716557379
Merge pull request #1032 from tinkerborg/consistent-sha
Ensure image SHA stays consistent when layer contents haven't changed
2020-02-13 11:22:27 -08:00
cvgw 7e2009b5e2 More idiomatic logging config
Update logging configuration to be more idiomatic for a Cobra
application
2020-02-07 16:13:03 -08:00
Tejal Desai 8d9e6b8ea5
Merge pull request #1027 from tejal29/small_refacto
refactor copy.chown code and add more tests
2020-02-07 13:48:05 -08:00
Tejal Desai c9dc474031 fix linter 2020-02-07 13:47:17 -08:00
Tejal Desai a17ad8e8e8
Merge pull request #882 from cvgw/u/cvgw/reuse-cached-layer
Do not recompute layers retrieved from cache
2020-02-07 13:37:39 -08:00
Tejal Desai 8b69a13641 fix commut 2020-02-06 15:37:47 -08:00
Tejal Desai 674f5bda5d fix tests 2020-02-06 15:17:09 -08:00
Or Sela 9dd050b892 set log format using a flag 2020-02-07 00:05:59 +02:00
Tejal Desai d94a1ed53b fix linter 2020-02-06 11:40:41 -08:00
Tejal Desai 9dba097a7e refactor and add more tests 2020-02-06 11:40:41 -08:00
tinkerborg 82bce229bd omit uname/gname in tar headers
When using cache, the rootfs may not have been extracted. This prevents uname/gname from resolving
as there is no /etc/password or /etc/group. This makes this layer unnecessarily differ from
a cached layer which does contain this information. Omitting these should be consistent with Docker's
behavior.
2020-02-06 13:36:26 -05:00
tinkerborg cc2c9a0663 sort filesToAdd in TakeSnapshot
filesToAdd is sorted in TakeSnapshotFS, but not here. This makes ordering unpredictable within the layer's tarball,
causing the SHA to differ even if layer contents haven't changed
2020-02-06 13:36:21 -05:00
Tejal Desai 9e17ffd6cb fix unit tests 2020-02-05 15:59:07 -08:00
Tejal Desai 2cd68d2e2f fix flake in copy symlink 2020-02-05 14:40:52 -08:00
Tejal Desai 2e95c3040c update initialWhitelist instead of whitelist 2020-02-04 11:06:26 -08:00
Tejal Desai 31f626cb22 initilize group to 0 2020-02-03 22:15:56 -08:00
Tejal Desai 56125ff464 Rebase from master with some changes 2020-02-03 14:57:10 -08:00
xanonid ac4c80d241 Refactor code, introduce constants for uid/gid = -1 2020-02-03 14:56:23 -08:00
xanonid 517cb50278 Add Unit test for GetUIDAndGIDFromString 2020-02-03 14:56:23 -08:00
xanonid e3b5a7b85d Support COPY --chown flag (Closes: #9) 2020-02-03 14:56:23 -08:00
Tejal Desai 18fec3ba38 fix lint 2020-02-03 11:03:58 -08:00
Tejal Desai db6f6c5ad3 more tests and do not error out if user not found 2020-02-03 10:11:40 -08:00
Tejal Desai 6894be442f remove duplicate code 2020-02-01 15:19:36 -08:00
Tejal Desai 956495784e fix group string being always set to uid in case a user has a gid set 2020-02-01 15:19:36 -08:00
Tejal Desai 0e833e4643 fix test 2020-02-01 09:29:18 -08:00
Tejal Desai 5a4a6c4a17 fix test flake 2020-01-31 15:23:38 -08:00
Tejal Desai 3f73230884
Merge pull request #1000 from tejal29/769
whitelist  /tmp/apt-key-gpghome.* directory
2020-01-31 10:50:26 -08:00
Thomas Bonfort 6b6742fd9d Changle loglevel for whiteouts to debug 2020-01-31 12:02:32 +01:00
Tejal Desai 611ba51c88 fix web editor conflicts resolutions 2020-01-30 11:18:53 -08:00
Tejal Desai 47ab3fe997
Merge branch 'master' into registry-mirror 2020-01-29 11:42:06 -08:00
Tejal Desai 8bdcb4f544
Merge pull request #854 from WanzenBug/ignored-files-invalidate-cache
Fix caching to respect .dockerignore
2020-01-29 11:32:43 -08:00
Cole Wippern 65cd9124d1
Merge pull request #957 from antechrestos/fix/scopes_asked_to_remote_registry
Upgrade go-containerregistry third-party library
2020-01-28 16:00:21 -08:00
Tejal Desai 8b991f6baf Add falg to --whitelist-var-run set to true to preserver default kaniko behaviour of /var/run ignored. Set it to false to add /var/run in destination directory 2020-01-28 10:52:36 -08:00
Tejal Desai d49c198c90 Revert "add option additonal-whitelist"
This reverts commit 72bfed1850.
2020-01-28 08:52:36 -08:00
Tejal Desai c8b19894bb Revert "additional-whitelist"
This reverts commit ac8e123815.
2020-01-28 08:52:29 -08:00
Tejal Desai 0529579db7
Merge pull request #971 from tejal29/915
Fix Symlinks not being copied across stages
2020-01-27 10:58:49 -08:00
Ben Einaudi 03b8922ede Upgrade dependency to go-containerregistry third-party library
By doing so it will fix issues met when mixing source/remote registry

Close #808
2020-01-26 18:55:52 +01:00
Tejal Desai e8fffda7cb
Merge pull request #995 from tejal29/507
Fix home and group set for user command
2020-01-24 22:18:09 -08:00
Tejal Desai e0322042c7 use filepath.Match instead of regex 2020-01-24 22:12:37 -08:00
Tejal Desai 68e8c6e902
Merge pull request #975 from cvgw/u/cgwippern/fix-issue-506
Fix #506 add option additional-whitelist
2020-01-24 16:11:32 -08:00
Tejal Desai 5951d9b0ee add /tmp/apt-key to whitelist for Dockerfiles which use command 2020-01-24 15:54:44 -08:00
Tejal Desai 308137f46d
Merge pull request #994 from wmuizelaar/fix_extracted_verbose_logging
Change loglevel for extractedFiles to debug. Fixes #901
2020-01-24 14:04:50 -08:00
Moritz Wanzenböck bab1c170e8 fix skipping ignored directories if they have whitelisted content 2020-01-24 22:27:56 +01:00
cvgw ac8e123815 additional-whitelist
* update readme
* improve test
* change default
2020-01-24 10:22:20 -08:00
Cole Wippern 72bfed1850 add option additonal-whitelist
Add a new option additonal-whitelist which defaults
to a single entry, "/var/run". This will allow users to
remove "/var/run" from the whitelist or retain the current
behavior with no change.
2020-01-23 16:50:33 -08:00
Cole Wippern cd9be5d513 reuse cached layer 2020-01-23 16:48:26 -08:00
Cole Wippern 782e4916b6 update build and composite cache tests 2020-01-23 15:37:38 -08:00
Moritz Wanzenböck ef692e55f3 composite_cache: add unit tests
Add unit tests that check the behaviour of CompositeCache
on adding filesytem resources. It checks that
* 2 identical directory trees produces the same hash
* an extra file produces a different hash
* an extra directry produces a different hash
* an extra file that is excluded does not alter the hash
* an extra directory that is excluded does not alter the hash
2020-01-23 15:37:37 -08:00
Moritz Wanzenböck 123dcaf83e Fix caching to respect .dockerignore
Previously kaniko would compute the cache key for any copy command by computing
the combined hash of all files in a directory, even if they were listed
as ignored.

With this change, the cache key creation was updated to be aware of ignored
files.

Related issues:
* https://github.com/GoogleContainerTools/kaniko/issues/594
2020-01-23 15:37:37 -08:00
Tejal Desai 048de00c33 Fix home and group set for user command 2020-01-23 15:07:15 -08:00
Cole Wippern a2aae6274d
Merge pull request #973 from cvgw/u/cgwippern/fix-issue-704
Fix issue #704
2020-01-23 13:00:34 -08:00
Wietse Muizelaar 05a316c26c Change loglevel for extractedFiles to debug. Fixes https://github.com/GoogleContainerTools/kaniko/issues/901 2020-01-23 21:37:06 +01:00
Tejal Desai 3e5d0a6334 add unit tests 2020-01-23 11:12:54 -08:00
Tejal Desai 79d226ade0 refactor 2020-01-22 16:32:46 -08:00
Tejal Desai bb129e9c88 code review comments 2020-01-22 16:27:06 -08:00
Tejal Desai b271eb8ab9 fix linter 2020-01-22 15:39:03 -08:00
Tejal Desai 478205e5ca fix adding symlinks to FS which do not exists 2020-01-22 15:27:01 -08:00
Tejal Desai f1f7297478 fix tests 2020-01-22 11:47:10 -08:00
Tejal Desai da7e9928e4 Fix Symlinks not being copies across stages 2020-01-22 11:47:10 -08:00
Tejal Desai 6a6c547811
Merge pull request #980 from PhoenixMage/PR646_rebase_fix
Prefer platform that is currently running for pulling remote images and kaniko binary Makefile target
2020-01-21 16:26:00 -08:00
Tejal Desai 76f0bf44e5 fix hack linter 2020-01-21 15:22:49 -08:00
Tejal Desai d362359314
Merge pull request #981 from cvgw/u/cgwippern/fix-issue-940
Fix #940 set modtime when extracting
2020-01-21 09:59:22 -08:00
Tejal Desai 649a0ed99b
Merge pull request #983 from cvgw/u/cvgw/fix-issue-519
Fix #519 capital letter in stage names
2020-01-21 09:56:06 -08:00
Tejal Desai b1970a3584
Merge pull request #792 from loganprice/FixAddTar
Fix Ability for ADD to unTar a file
2020-01-21 09:55:25 -08:00
Cole Wippern b9b61e2a12
Merge pull request #943 from shortstories/bugfix/copy-with-symlink
Fix #942 COPY or ADD to symlink destination breaks image
2020-01-20 20:55:35 -08:00
ohchang-kwon df767bb195 apply code review results 2020-01-21 12:07:13 +09:00
cvgw e652f815ef Fix #519 capital letter in stage names
update stage code so that when comparing the BaseName of
a stage against the recorded, lowercase version of a Stage name
the BaseName is also lowercased.
2020-01-20 15:57:05 -08:00
Cole Wippern 4e8bdb3947 Fix #940 set modtime when extracting
Certain utilities like Apt depend on modtime
for certain files. Kaniko was not setting modtime when
extracting files and so this broke things like apt.

Kaniko now sets the file mod time to the value from the tar
header.
2020-01-20 09:11:44 -08:00
Anthony Davies ed22afc19b Rebase fix for PR #646
I needed this for my arm64 k8s cluster. I have zero Go experience but
enough experience with other things to fix the rebase (I think!). This
patch is working fine on my cluster.
2020-01-20 13:43:45 +11:00
Yoan Blanc e26bd672f1
fixup! Merge remote-tracking branch 'origin/master' into registry-mirror
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-01-18 09:36:22 +01:00
Yoan Blanc 140879fadd
Merge remote-tracking branch 'origin/master' into registry-mirror 2020-01-18 09:35:56 +01:00
Cole Wippern 96093359c1 rename mock pkg for lint 2020-01-17 14:59:14 -08:00
Cole Wippern f96ea3bbfb hand edit mock license
for some reason hack/boilerplate.sh failed after using `addlicense`
so I copy/pasted the license header from another file
2020-01-17 14:56:12 -08:00
Cole Wippern a9896d5c43 update mocks with license 2020-01-17 14:32:36 -08:00
Cole Wippern 876f24b0c9 update caching run and copy extracted files
Update caching run and copy commands to use the new
GetFSFromLayers method and include the whiteout option so that
whiteout files are extracted and included in extractedFiles
2020-01-17 14:19:58 -08:00
Cole Wippern 71aed35094 GetFSFromLayers
* add util.GetFSFromLayers
* GetFSFromImage delegates to GetFSFromLayers
* add FSOpts and FSConfig for GetFSFromLayers
* add tests for GetFSFromLayers
* add gomock for test support
* add mock_v1 for layers
2020-01-17 13:36:23 -08:00
Cole Wippern e19cc228ba
Merge pull request #941 from stijndehaes/feature/cache-warmer-private-reg-support
Support for private registries in the cache warmer
2020-01-17 08:38:59 -08:00
Tejal Desai e702d75e9c
Merge pull request #964 from samos123/fix-argsescape
Fix bug with docker compatibility ArgsEscaped
2020-01-16 16:13:57 -08:00
Stijn De Haes 3fa8f686db Support for private registries in the cache warmer
Closes #503

Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2020-01-15 15:35:42 +01:00
lou-lan da0e7c7bdc Clean code (Condition is always 'false' because 'err' is always 'nil' ). 2020-01-14 15:11:22 +08:00
Sam Stoelinga 64bfc3eabb Fix bug with docker compatibility
ArgsEscaped according to Docker docs should only be set in Windows
environments: https://docs.docker.com/engine/api/v1.30/

It was causing integration test to fail with following message:
```
FAIL: TestRun/test_Dockerfile_test_metadata (8.48s)
           "Diff": {
             "Adds": [
               "ArgsEscaped: true"
             ],
             "Dels": [
               "ArgsEscaped: false"
             ]
```

However docker 18.xx returns ArgsEscaped: true
whereas docker 19.xx returns ArgsEscaped: false
Hence this patch also adds the docker version check to the integration
to ignore ArgsEscaped being different when 18.xx is used.
2020-01-13 21:18:54 -08:00
Cole Wippern 776fa43eb2 refactor cache.Warm and add tests 2020-01-10 12:29:17 -08:00
Cole Wippern b1b0513c05 Fix #926 cache warmer and method signature 2020-01-10 12:29:16 -08:00
ohchang-kwon fb4d1f9c8d add handling when source directory does not have slash suffix 2020-01-03 19:21:29 +09:00
ohchang-kwon 70270a65f0 apply goimports 2020-01-03 17:46:29 +09:00
ohchang-kwon d6fe98aa49 Fixed an issue where the image is broken if there are symlink in the destination path when ADD, COPY 2020-01-03 16:28:12 +09:00
Yoan Blanc 3678b70239
Merge remote-tracking branch 'origin/master' into registry-mirror 2019-12-22 11:10:15 +01:00
Cole Wippern 328847ffc2 Fix #925 broken insecure pull 2019-12-21 21:50:43 -08:00
Logan.Price 812c66ecc0 chore: add error check 2019-12-21 15:42:00 -08:00
Logan.Price c5a29962f2 chore: add unit test for unTar function 2019-12-21 15:42:00 -08:00
Logan.Price 6a35a84034 fix: update unTar function to actually extract and save the contents of an achieve 2019-12-21 15:42:00 -08:00
Nao YONASHIRO cbba7cec3f chore: fix typo 2019-12-21 14:44:12 -08:00
Nao YONASHIRO 2599c0c69b docs: fix wrong document 2019-12-21 14:42:58 -08:00
Cole Wippern 229820550e
Merge pull request #869 from prary/go_modules
changing to modules from dependencies
2019-12-21 12:56:15 -08:00
Cole Wippern bdabd774fc Update linter and fix errors
* disable linters that are now enabled by default in new version
* fix linter errors
2019-12-21 12:27:14 -08:00
Prashant Arya 976b4265fb changing to modules from dependencies 2019-12-21 12:18:09 -08:00
Prashant Arya 85f1a5db00 Merge branch 'master' of https://github.com/GoogleContainerTools/kaniko into log 2019-12-19 03:20:56 +00:00
Cole Wippern 9e9b8a6e71 Fix #899 cached copy results in inconsistent key
* Update cached copy command to return the same result for
files used from context so that cached and uncached copy
commands produce the same cache key
* Update tests for fix
* Add test for cached run command key consistency
2019-12-15 10:23:31 -08:00
Cole Wippern 2aa481c15e add unit tests for caching run and copy 2019-12-10 09:29:10 -08:00
Cole Wippern b19214ad1e Use cachekey not digest for COPY --from src
* use the cachekey of the src stage rather than the digest
for COPY --from commands as they are reproducible unlike digests
* track digest to cache keys and stage indexes to digest
* add extra debug logging for troubleshooting cachekey building issues
* convert Sha256 hashes to hex encoded strings rather than plain strings
for easier human reading
2019-12-10 08:45:13 -08:00
Cole Wippern 7b4b768edf Update copy command cache key logic
Include the digest of the stage specified in the --from argument for
COPY commands which use --from
2019-12-09 16:48:20 -08:00
Tejal Desai fbdb8f39c8
Merge branch 'master' into registry-mirror 2019-12-09 15:40:56 -08:00
Tejal Desai acb5b9f7c9
Merge pull request #839 from antechrestos/fix/upcase_for_from_stage_name
Fix failure when using capital letters in image alias in 'FROM ... AS…' instruction
2019-12-09 15:39:39 -08:00
Tejal Desai 03b1b1ce4c
Merge pull request #905 from poy/fixes/904
when copying, skip files with the same name
2019-12-09 13:07:13 -08:00
Tejal Desai 56f92e74af
Merge pull request #849 from wripley/master
Modified error message for writing image with digest file
2019-12-09 12:57:33 -08:00
poy 0a2f2957ec when copying, skip files with the same name
When using the COPY command, if the source and destination have the same
the file should be skipped rather than copied. This is to prevent the
file from being overwritten and therefore producing an empty file.

fixes #904
2019-12-08 00:57:27 -07:00
Yoan Blanc 6e331de346
fixup! Merge remote-tracking branch 'origin/master' into registry-mirror
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2019-12-06 20:10:35 +01:00
Ben Einaudi d22a7608c2 Fix failure when using capital letters in image alias in 'FROM ... AS' instruction
The third library moby/buildkit lowers the image alias used in 'FROM .. AS' instruction.
This change fixes this issue by making the resolve of dependencies agnostic to case.

Fixes #592
Fixes #770
2019-12-06 12:04:26 +01:00
Yoan Blanc 980c87ac17
Merge remote-tracking branch 'origin/master' into registry-mirror 2019-12-05 19:12:47 +01:00
Cole Wippern 7ba65daf7f cleanup executor/build_test.go 2019-11-28 09:36:39 -08:00
Cole Wippern 828e764b95 add boilerplate for composite_cache_test 2019-11-28 09:18:58 -08:00
Cole Wippern 6d0c8da90e more stagebuilder caching tests 2019-11-28 08:42:13 -08:00
Cole Wippern 33f3191b17 Don't hardcode hashes for stagebuilder tests 2019-11-27 21:47:00 -08:00
Cole Wippern 697037cbcf Add unit tests for compositecache and stagebuilder
* add mock types for testing
* enhance error messaging
* add tests
2019-11-27 21:47:00 -08:00
Cole Wippern 54635c3d39 don't exit optimize early so we record cache keys 2019-11-27 17:09:00 -08:00
Cole Wippern 2755ae4470 Final cachekey for stage
Store the last cachekey generated for each stage
If the base image for a stage is present in the map of digest
and cachekeys use the retrieved cachekey instead of the base image
digest in the compositecache
2019-11-27 14:40:05 -08:00
Prashant Arya 857715012f changing log level 2019-11-26 17:52:11 +00:00
Cole Wippern bfd8562855
Merge pull request #879 from cvgw/u/cvgw/update_error_handling_and_logging_for_cache_retrieve
Update error handling and logging for cache
2019-11-23 10:20:40 -08:00
Cole Wippern a6e458caf1 Update error handling and logging for cache
Previously we returned a low level file system error when checking for
a cached image. By adding a more human friendly log message and explicit
error handling we improve upon the user experience.
2019-11-22 14:54:38 -08:00
Cole Wippern b057776849
Merge pull request #857 from cvgw/u/cgwippern/symlink-bug
Resolve symlink targets to abs path before copying
2019-11-22 10:02:58 -08:00
Cole Wippern 2b26dfea61 Add unit tests for resolveIfSymlink 2019-11-15 11:23:15 -08:00
Cole Wippern 2c13842451 Resolve symlink paths 2019-11-15 11:23:15 -08:00
Cole Wippern 50f1373837 Update Add command RequiresUnpackedFS 2019-11-15 11:23:15 -08:00
Yoan Blanc 6e2287c969
readme: add --registry-mirror
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2019-11-10 12:55:31 +01:00
Cole Wippern 006b4996cb Fix quote strip behavior for ARG values
* fixes issue 847
* previous implementation did not properly parse blank
values which were enclosed in quotes
2019-11-08 14:28:31 -08:00
Will Ripley f6aa8f709b Modified error message for writing image with digest file 2019-11-08 12:59:25 -06:00
Tejal Desai 1d5e294f27
Merge pull request #841 from wripley/master
Added --image-name-with-digest flag
2019-11-08 08:24:26 -08:00
Yoan Blanc 73f7ea0d2c
fixup! integration: build using registry-mirror
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2019-11-08 16:17:00 +01:00
Tejal Desai b04d15db82 fix the issue now 2019-11-07 15:29:56 -08:00
Tejal Desai f36dd4013c only add tests 2019-11-07 15:17:25 -08:00
Tejal Desai 334f0c70d6 fix golint 2019-11-07 15:11:29 -08:00
Tejal Desai b29431227a fix tests for default home 2019-11-07 15:07:50 -08:00
Tejal Desai 5bbb40e4f0
Merge pull request #834 from cvgw/u/cgwippern/ISSUE_439_strip_arg_quotes
Issue #439 Strip out double quotes in ARG value
2019-11-07 10:54:17 -08:00
Don McCasland 5716c44ab8
Merge branch 'master' into bug-742-2 2019-11-04 09:25:25 -08:00
Will Ripley c8f089fdbb Fixed formatting errors with push.go and push_test.go 2019-10-29 13:09:27 -05:00
Will Ripley a0651436b5 Changed image name to be full image name minus tag. Also added test for new ImageNameDigestFile flag 2019-10-29 12:55:50 -05:00
Will Ripley c2faa4e5e6 Fixed spacing issue in config 2019-10-29 11:04:33 -05:00
Will Ripley 08d0948510 Merge remote-tracking branch 'upstream/master' 2019-10-29 11:02:16 -05:00
Will Ripley f38c47daa0 Made changes to --image-name-with-digest-file to support multiple destinations 2019-10-29 10:55:49 -05:00
Cole Wippern 0ce287d95c ISSUE #439 ci lint 2019-10-25 16:27:14 -07:00
Cole Wippern ec2e7705c8 Issue #439 add additional tests for quotes
Add additional tests to ensure that ARG values with quotes
are handled properly
2019-10-25 16:08:57 -07:00
Tejal Desai c5bc179c85
Merge pull request #816 from yangtaokm/azure-blobstorage-https
[WIP] Add support to download context file from Azure Blob Storage
2019-10-25 10:51:50 -07:00
Yoan Blanc 235fc4a61f
util: add test to RegistryMirror option
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2019-10-25 13:07:03 +02:00
Yoan Blanc 3873aa05e3
root: add --registry-mirror flag
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2019-10-25 11:34:49 +02:00
Cole Wippern 79649a1614 Issue #439 double quotes in ARG value
* Strip out double quotes enclosing ARG value after parsing
dockerfile
2019-10-24 16:30:25 -07:00
yangtaokm aa881d59cb fix gofmt 2019-10-22 17:10:08 +08:00
yangtaokm 45c43a2c89 refactory the code and add CreateTargetTarfile in fs_util.go 2019-10-22 16:44:51 +08:00
Will Ripley 4c9a800e12 Added image digest file path 2019-10-21 11:55:55 -05:00
Tao Yang a86ffc8c57
Apply suggestions from code review
fix formatting based on review

Co-Authored-By: Tejal Desai <tejal29@gmail.com>
2019-10-21 10:19:32 +08:00
Prashant fa162e3005 changing debug to trace 2019-10-18 19:03:11 +05:30
yangtaokm 78e9974694 Add more test case for azureblob_util.go 2019-10-14 13:55:03 +08:00
yangtaokm 3e6a24b152 Initial commit for Azure Blob Storage SupportX 2019-10-13 20:52:19 +08:00
Don McCasland 27020f23b9
Resolving comments 2019-10-08 12:19:20 -07:00
Don McCasland 2b6965b294
addressing lint issues 2019-10-07 13:11:09 -07:00
Don McCasland 2d73c2d996
addressing lint issues 2019-10-07 13:06:40 -07:00
Don McCasland 2eace1d511
Refactoring to add use of a tempdir 2019-10-07 12:56:34 -07:00
Don McCasland 4e1639c030
Addressing review comments 2019-10-07 10:46:28 -07:00
Tejal Desai 529bc4f20e
Merge pull request #805 from tejal29/fix_720
add debug level info what is the layer type
2019-10-04 16:19:49 -07:00
Tejal Desai 1a88d7ae9c better logging 2019-10-04 16:04:12 -07:00
Tejal Desai 3ad9255806 add debug level info what is the layer type 2019-10-04 15:51:05 -07:00
Tejal Desai 1c821d050c
Merge pull request #796 from prary/version
Kaniko version
2019-10-04 14:32:14 -07:00
Tejal Desai b57160198e fix bad merge 2019-10-04 11:20:41 -07:00
Tejal Desai f206e3f425
Merge branch 'master' into output 2019-10-04 08:43:02 -07:00
Tejal Desai 9eb4a1c1ff
Merge pull request #801 from victornoel/776-dockerignore-relative
Support Dockerfile.dockerignore
2019-10-04 01:56:40 -07:00
Victor Noel ff7abba47b Fix integration tests for #776 2019-10-04 10:31:40 +02:00
Tejal Desai 22188595f0
Merge pull request #768 from N4rm0/fix-envar-replacement-for-url
Fix #691 - ADD does not understand ENV variables
2019-10-04 01:17:22 -07:00
Victor Noel 5700de039d Add more tests for #776 2019-10-04 10:08:39 +02:00
Tejal Desai 865d49c631
Merge pull request #694 from orisano/feat-optimize-build
feat: optimize build
2019-10-04 00:51:23 -07:00
Prashant f9424d43e7 fixing lint and minor fix 2019-10-03 21:29:40 +05:30
Victor Noel db12a77e6c Fix #776 2019-10-03 17:53:14 +02:00
Don McCasland dbabcb1f5f
Adding CachingCopy command 2019-09-26 15:32:40 -07:00
Don McCasland e58ee0967a
Fixing logic for Copy command. The problem was not that tar files were being unpacked in wrong order. The problem was that the COPY command requires the FS to be unpacked before it does its work. 2019-09-26 11:29:53 -07:00
jonjohnsonjr 51734fc3a3
Merge pull request #765 from Onlinehead/fix/filesystem-delete-fix
Fix SIGSEGV on file system deletion while building
2019-09-25 11:47:35 -07:00
Monard Vong 019b26e9a0 Simplify logic 2019-09-22 01:09:36 +02:00
Don McCasland ccd99fbeec
Merge pull request #685 from tralexa/master
insecure flag not honored in cache
2019-09-20 13:32:19 -07:00
Антон Костенко a21129f889 Fix typo and run go fmt over fs_util 2019-09-18 10:43:26 +02:00
Nao YONASHIRO 38fa3608ce chore: fix megacheck 2019-09-18 17:19:15 +09:00
Nao YONASHIRO d9d865c74a test: add BenchmarkHasFilepathPrefix 2019-09-18 17:19:15 +09:00
Nao YONASHIRO 02d1ce5e1b feat: optimize HasFilepathPrefix 2019-09-18 17:19:15 +09:00
Nao YONASHIRO c87d2dba9c feat: replace md5 to highwayhash 2019-09-18 17:19:15 +09:00
Tejal Desai e048b87222
Merge pull request #671 from abergmeier/fix_comment
Correct CheckPushPermission comment.
This is a typo fix.
2019-09-17 10:44:54 -07:00
Don McCasland 1bb5a41d7d
Fixing lint issues. Adding tests for COPY command. Fixing issue with copying files out of snapshots 2019-09-16 10:48:31 -07:00
Don McCasland e003bae87d
first commit of copy_test 2019-09-16 10:46:57 -07:00
alexa 62acf3ac1d change schema to http when Insecure flag is enabled 2019-09-16 16:35:45 +03:00
tralexa 75083a0c55
Merge branch 'master' into master 2019-09-16 15:58:17 +03:00
Monard Vong d96cbfe0d3 Fix formatting 2019-09-16 00:03:10 +02:00
Monard Vong 13b8fb6345 remove trailing whitespace 2019-09-15 23:29:54 +02:00
Monard Vong 38474fad11 Fix #691 - ADD does not understand ENV variables 2019-09-15 23:08:44 +02:00
Tejal Desai 224ac8407c
Merge pull request #760 from prary/path_env
Setting PATH
2019-09-13 15:29:47 -07:00
Tejal Desai 08acf221c6
Merge pull request #635 from discordianfish/patch-1
Remove cruft
2019-09-13 14:27:58 -07:00
Tejal Desai 30f1a7dae9 fix lint 2019-09-13 12:16:40 -07:00
Tejal Desai 9b9fb815a7 fix format 2019-09-13 11:56:57 -07:00
Tejal Desai 469fdaa50d test 2019-09-13 11:49:30 -07:00
Tejal Desai f33c71b3c5 Merge branch 'master' into path_env 2019-09-13 11:48:01 -07:00
Tejal Desai f0e571839d add unit tests 2019-09-13 11:21:43 -07:00
Tejal Desai a014c4a1e8 added unit tests 2019-09-13 11:00:05 -07:00
Tejal Desai 873b577e1a
Merge pull request #729 from xanonid/Remove_leading_slash_in_layer_tarball_paths
Remove leading slash in layer tarball paths (Closes: #726)
2019-09-13 10:05:25 -07:00
priyawadhwa 8acab90dec
Merge pull request #698 from takmatsu/s3-endpoint
Add support for S3 custom endpoint
2019-09-13 09:32:25 -04:00
Антон Костенко a1cc42bdb7 Fix https://github.com/GoogleContainerTools/kaniko/issues/756 2019-09-10 00:44:27 +02:00
Prashant 0158cbf70c Setting PATH for empty image as well 2019-09-04 17:23:59 +05:30
Prashant 17d1059ec4 Setting PATH to default PATH if PATH is missing 2019-09-04 16:44:55 +05:30
chhsia0 ea1a92712c Avoid comparing uncompressed and compressed images in the unit test. 2019-09-03 23:17:57 -07:00
Prashant 2c44539151 Setting PATH 2019-08-30 13:33:36 +05:30
chhsia0 11f3b791cd Renamed to `--oci-layout-path` and added a unit test. 2019-08-24 01:28:58 -07:00
chhsia0 730b8b77c8 Added `--layout-path` flag to save image in OCI layout.
Fixed #296.

The output manifests may have `application/vnd.docker.distribution.manifest.v2+json`
as their media types instead of `application/vnd.oci.image.manifest.v1+json`.
2019-08-24 01:04:18 -07:00
Tejal Desai 96947b8ca4 fix lint 2019-08-23 13:13:14 -07:00
Tejal Desai 6daffd8dd7 add multiple user agents to kaniko if upstream_client_type value is set 2019-08-23 11:29:05 -07:00
priyawadhwa 7adf2fcb50
Merge pull request #714 from MJDSys/reproducible_add
Make container layers captured using FS snapshots reproducible
2019-08-19 14:39:18 -07:00
priyawadhwa 0694be3e85
Merge pull request #717 from vm86/fix
fix unpacking archives via ADD
2019-08-19 13:24:53 -07:00
Deniz Zoeteman c425f02866 Reverted not including build args in cache key 2019-08-16 15:09:52 +02:00
xanonid 30db2c07d3 Also remove leading / from paths to file which are marked as deleted 2019-08-09 17:30:45 +02:00
xanonid 1fa2527a7b Adapt tests such that expected paths do not contain a leading / 2019-08-09 14:24:03 +02:00
xanonid c75749b840 Do not use leading slashes for paths in layer tarballs to be more compatible with docker 2019-08-09 14:24:03 +02:00
Sharif Elgamal 56eeaf41e6
Merge pull request #452 from sharifelgamal/cache-fix
Create cache directory if it doesn't already exist
2019-08-02 13:41:38 -07:00
Sharif Elgamal 8a24115b6a
Prevent panic on nil image 2019-08-02 13:25:20 -07:00
v.rul 7750094ec1 Add checking image presence in cache prior to downloading it
This changes allow to use kaniko-warmer multiple times without unnecessary docker image downloads.
To check image presence in cache directory I'm using existing cache function that is used by kaniko-executor.
I've considered building separate function to only check image presence, but it will have pretty much the same code.
Questionable decision is to embed CacheOptions type to KanikoOptions and WarmerOptions. Probably this should be resolved by creating interface providing needed options and implement it both mentioned structs. But I've struggled to get a meaningfull name to it.
To replicate previous behaviour of downloading regardless of cache state I've added --force(-f) option.

This changes provides crucial speed-up when downloading images from remote registry is slow.

Closes #722
2019-07-24 18:09:21 +03:00
Taylor Barrella 3422d5572a Misc. small changes/refactoring (#712) 2019-07-23 15:10:22 -07:00
Tejal Desai 19fb253e9c
Merge pull request #680 from tbarrella/go-containerregistry-upgrade
Update go-containerregistry
2019-07-23 14:35:19 -07:00
Роман Небалуев be0dac2823 fix unpacking archives via ADD 2019-07-19 18:47:18 +05:00
Matthew Dawson 619fc5e59b Make container layers captured using FS snapshots reproducible
When a Dockerfile command requires using the TakeSnapshotFS function,
the resulting layer has a random ordering of files.  This causes the
layer to have a non-deterministic hash defeating the reproducible flag.
Issue #710 appears to document this issue as well.

To fix, always sort the list of files to be added in scanFullFilesystem.
This avoids trying to sort the file list during execution, and takes
almost no time to complete.
2019-07-11 21:58:42 -04:00
Carlos Alexandro Becker f578b09846
fix: remove single branch option
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2019-06-22 12:17:46 -03:00
Carlos Alexandro Becker c45e05f668
clean: remove debug msg
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2019-06-22 11:45:42 -03:00
Carlos Alexandro Becker f0b9ad3a57
feat: support specifying branch for cloning
Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2019-06-22 11:45:07 -03:00
Andreas Bergmeier 7cc899b09e Add SkipVerify support to CheckPushPermissions. (#663)
Extract makeTransport, which allows using the current mechanism used for
pushing.

Fixes #628.
2019-06-14 12:34:55 -07:00
Takeaki Matsumoto 35bb350a49 Add support for S3 custom endpoint
For S3-compatible object storage (like minio),
this patch enable to use custom endpoint-url.

Fix #531
2019-06-13 12:42:52 +09:00
tralexa 66a052255f
Insecure is now honored 2019-06-02 12:24:22 +03:00
Taylor Barrella 5c0603a967 Update go-containerregistry
Resolves #607

* Deleted a duplicate Gopkg.lock block for github.com/otiai10/copy to
  prevent `dep ensure` from deleting it from vendor/

* Searched for breaking changes. Only found ones for
  remote.Delete/List/Write/WriteIndex. Searched for those and fixed

* Noticed that NewInsecureRegistry was deprecated and replaced it
2019-05-25 15:56:20 -07:00
priyawadhwa 2fc8a7f4bc
Merge pull request #665 from orisano/chore-fix-typo
chore: fix typo
2019-05-23 16:06:11 +02:00
Kartik Verma f137f81884
Resolved Merge Conflicts
Signed-off-by: Kartik Verma <vkartik97@gmail.com>
2019-05-21 01:52:14 +05:30
Andreas Bergmeier 0a3a280e8f Correct CheckPushPermission comment. 2019-05-20 09:36:25 +02:00
Nao YONASHIRO 75fdad7319 chore: fix typo 2019-05-17 03:17:08 +09:00
priyawadhwa 38c1735d92
Merge pull request #655 from GijsWeterings/master
Add `--digest-file` flag to output built digest to file.
2019-05-14 12:37:14 -07:00
Gijs 3686b65426 Process feedback of priyawadhwa regarding naming/nilcheck.
Changes the argument flag from `--digestfile` to `--digest-file`.
Skips an unneeded nil check.
2019-05-11 15:14:25 +02:00
Johannes 'fish' Ziemke 8c732f6f52 Fix kaniko caching (#639)
* Revert "Change cache key calculation to be more reproducible. (#525)"

This reverts commit 1ffae47fdd.

* Add logging of composition key back

* Do not include build args in cache key

This should be save, given that the commands will have the args included
when the cache key gets built.
2019-05-10 09:57:03 -05:00
Gijs a6e3ddfc79 Add `--digestfile` flag to output built digest to file.
This flag, when set, takes a file in the container and writes the image digest to it. This can be used to extract the exact digest of the built image by surrounding tooling without having to parse the logs from Kaniko, for example by pointing the file to a mounted volume or to a file used durint exit status, such as with Kubernetes' [Termination message policy](https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/)]

When the flag is not set, the digest is not written to file and the executor behaves as before. The digest is also written to file in case of a tarball or a `--no-push`.

Closes #654
2019-05-02 14:57:44 +02:00
Dirk Gustke dd9d081447 this is quite spammy in my multistage build (#640)
.. and as i am surely not the only one, move it down to debug.
2019-04-15 13:22:46 -07:00
Kartik Verma fa2a2c803b
Updated PR Branch with master 2019-04-09 02:18:28 +05:30
Johannes 'fish' Ziemke 8f863213b8
Remove cruft 2019-04-01 11:13:10 +02:00
dlorenc c8fabdf6e4
Fix arg handling for multi-stage images in COPY instructions. (#621) 2019-03-22 12:24:43 -05:00
Daisuke Taniwaki 1bf4421047 Fix parent directory permissions (#619)
* Add parent directories of adding files

* Add integration Dockerfile to test parent directory permissions

* Remove unnecessary helper method

* Use a file on the internet for integration Dockerfile
2019-03-19 12:40:15 -05:00
Jason Hall 3fa411ceb9 Check push permissions before building images (#622)
* Check push permissions before building images

* Fix doc comment

* improve error messages
2019-03-19 12:39:59 -05:00
Daisuke Taniwaki 28bfb75a31 Fix file mode bug (#618)
* Fix file mode

* Add Dockerfile for special file permission test
2019-03-18 13:28:24 -05:00
dlorenc 246cc92a33
Optimize file copying and stage saving between stages. (#605)
This change calculates the exact files and directories needed between
stages used in the COPY command. Instead of saving the entire
stage as a tarball, we now save only the necessary files.
2019-03-13 07:47:28 -07:00
Jason Hall a916b72169
Add license header 2019-03-07 10:33:23 -05:00
dlorenc 9912ccbf8d
Fix USER handling. There were two issues: (#600)
- We were validating usernames/groupnames existed in etc/passwd. Docker does not do this
- We were incorrectly caching USER commands. This was fixed automatically by fixing the first part.
2019-03-07 07:05:24 -08:00
Jason Hall 295dd49487 Write data about pushed images if env var is set 2019-03-06 14:37:56 -05:00
dlorenc 2abe109eb2
Environment variables should be replaced in URLs in ADD commands. (#580)
We were previously explicitly skipping this for some reason, but Docker
seems to expand these in URLs so we should too.
2019-02-25 10:42:34 -06:00
dlorenc 4b7e2b3a06
Update the cache warmer to also save manifests. (#576) 2019-02-21 14:09:22 -06:00
Priya Wadhwa 3da2fdf2d0 Merge branch 'issue#269' of https://github.com/vkartik97/kaniko into vkartik97-issue#269 2019-02-21 11:55:51 -08:00
dlorenc 378a3f9573
Look for manifests in the local cache next to the full images. (#570)
Calculating a manifest from a v1.tarball is very expensive. We can
store those locally as well, and use them if they exist.

This should eventually be replaced with oci layout support once that exists
in ggcr.
2019-02-19 13:54:41 -06:00
Jason Hall faadb2af86 Log "Skipping paths under..." to debug (#571)
This reduces noise in the log output, since it isn't terribly useful to most end users.
2019-02-19 13:54:26 -06:00
Fredrik Lönnegren 1b54bda32e Fix typo 2019-02-19 16:43:20 +01:00
Shuhei Kitagawa e8564f0d28 Add missing tests for bucket util (#565)
* Add CheckDeepEqual test util

* Fix for srcContext with slash suffix

* Add tests for bucket_util.go
2019-02-19 09:05:23 -06:00
dlorenc 4feed0ff35
Stop storing a separate cache hash. (#560)
This is unrequired, mtimes should be taken into account during caching.
2019-02-14 12:24:43 -06:00
dlorenc 8d78db4842
Refactor snapshotting (#561) 2019-02-14 12:14:28 -06:00
dlorenc 15e70d4142
Speed up workdir by always returning an empty filelist (rather than a nil one). (#557)
Nil indicates that a full snapshot is required. Empty indicates that nothing should be snapshotted.
2019-02-13 11:55:08 -06:00
dlorenc 877abd30ed
Refactor whitelist handling. (#559)
Also speed up stage deletion.
2019-02-13 11:17:56 -06:00
dlorenc 8179c47f0d
Refactor the build loop to fetch stagebuilders earlier. (#558)
This will help with optimizations.
2019-02-12 20:43:22 -06:00
James Rawlings af50bcec69 let's return errors when warming a cache (#553)
* let's return errors when warming a cache

* Fix lint error in warm.go
2019-02-08 15:14:53 -06:00
dlorenc 9047ccf7cc
This fixes a bug in the interaction between volumes in base images (#555)
and our snapshot optimizations.

If a previous base image has a volume, the directory is added to the
list of files to snapshot. That directory may not actually exist in the image.
2019-02-08 14:40:37 -06:00
dlorenc e14b660947
Fix a bug in volume handling for multi-stage images. (#554)
We were previously not resetting the whitelist between stages, this caused issues with volumes.
2019-02-07 09:01:49 -06:00
Anurag Goel 82fe355f14 Fix issues with layer caching, noPush and tarPath (#549)
* Set TarPath to empty when pushing a layer

* Fix issues with layer caching, noPush and tarPath.

- Layer caching should work even when tarPath is specified, so this
commit changes the value of tarPath to empty when caching layers.

- When an image is built with just the tarPath and noPush
is true, we should still create the tarBall (which wasn't happening
before this commit).

* Set no-push to false for cache layers

* Remove extra log

* go-imports fix
2019-02-06 10:16:09 -06:00
Kartik Verma 2f8bdd99b7
Resolved Merge Conflicts 2019-02-02 11:28:40 +05:30
dlorenc c2514305ef
Fix a bug in snapshotting with multi-stage images. (#546)
We previously had an optimization that would skip snapshotting mutli-stage images
when in an intermediate stage, until the very end.

This conflicted with another optimization to avoid snapshotting when no files had changed.
2019-01-30 13:57:02 -06:00
Daisuke Taniwaki f8f59ea4c6 Add insecure-registry and tls-skip-verify-registry flags (#537) 2019-01-29 13:29:47 -06:00
dlorenc 1ffae47fdd
Change cache key calculation to be more reproducible. (#525)
Before we were using the full image digest, but that contains a timestamp. Now
we only use the layers themselves and the image config (env vars, etc.).

Also fix a bug in unpacking the layers themselves. mtimes can change during unpacking,
so set them all once at the end.
2019-01-23 13:46:12 -06:00
dlorenc fcd1976d3b
Make the Digest calculation faster for locally-cached images. (#534)
Right now when we find a v1.Tarball in the local disk cache, we
recompute the digest. This is very expensive and redundant, because
we store tarballs by their digest and use that as a key to look them up.
2019-01-22 13:28:21 -06:00
dlorenc 996bec885f
Refactor our credential fetching to only get k8s credentials if we are on k8s. (#522) 2019-01-11 11:36:43 -07:00
dlorenc 170e0a2d94
Add a lot more timing data. (#518) 2019-01-10 13:27:55 -07:00
dlorenc 9ab66560db
Simplify snapshotting. (#517) 2019-01-09 15:31:02 -07:00
dlorenc 2255837142
Tighten up the timing around Dockerfile commands. (#514)
Right now this timing also includes the snapshot time.
2019-01-09 10:34:23 -08:00
Myers Carpenter 1591f41f22 USER command needs /etc/passwd, thus require the fs to be unpacked (#508) 2019-01-08 17:26:54 -08:00
dlorenc 5f6fbfe74f
Add support for timing data in JSON format. (#510) 2019-01-08 17:24:47 -08:00
dlorenc a044e2b6e4
Even faster snapshotting with godirwalk. (#504)
This switches from filepath.Walk to godirwalk.Walk for even faster snapshotting.
A quick test shows a 40% improvement on the dockerfile_mv_add build.
2019-01-03 13:10:18 -06:00
linuxshokunin c3afcc0c7d Adding capability to get Dockerfile from URL. (#500)
* Changed to set http or https in Dockerfile path.

* Fixed return value when error.

* Fixed ineffectual error
2019-01-03 10:42:15 -06:00
MMeent e3bb8bc71a Adds COPY --from=previous stage name/number validation (#491)
* Adds COPY --from=previous stage name/number validation

This fixes an issue in which COPY --from=previous-stage-name would try to download docker image previous-stage-name instead of checking that previous-stage-name could be a named stage.

* Fix linting issues

goimports is implemented as 'gofmt + extras', so this should fix import warnings as well.

* Fix linting issues

Fixes linting issues introduced in the merge

* Fix linting issues.
2019-01-02 11:42:36 -06:00
dlorenc 8ced0930f4
Add more benchmarks (#487) 2018-12-17 10:23:40 -06:00
Sharif Elgamal 29b7c3e879
Adding TTL to caching (#488)
* Adding TTL to caching

* uncomment added lines

* cache TTL works for layers now

* remove debugging

* parse booleans correctly

* parse booleans correctly everywhere

* fix boolean parsing condition

* refactor benchmarking calls

* defer file properly
2018-12-13 14:20:25 -08:00
Priya Wadhwa a34ba5c233 Fixed merge conflict 2018-12-11 13:53:19 -08:00
Priya Wadhwa 7fd164deab Only parse .dockerignore once 2018-12-11 13:31:51 -08:00
Priya Wadhwa 9b01772cde Add logging 2018-12-11 12:44:23 -08:00
Priya Wadhwa b0b36ed85a Re-add support for .dockerignore file
This PR adds support for the dockerignore file. Previously when kaniko
had support for the dockerignore file, kaniko first went through the
build context and deleted files that were meant to be ignored. This
resulted in a really bad bug where files in user mounted volumes would
be deleted (my bad).

This time around, instead of modifying the build context at all, kaniko
will check if a file should be excluded when executing ADD/COPY
commands. If a file should be excluded (based on the .dockerignore) it
won't be copied over from the buildcontext and shouldn't end up in the
final image.

I also added a .dockerignore file and Dockerfile as an integration test,
which should fail if the dockerignore is not being processed correctly or if files aren't being excluded correctly.
Also, I removed all the integration testing from the previous version of the
dockerignore support.
2018-12-10 15:20:25 -08:00