Commit Graph

947 Commits

Author SHA1 Message Date
Tejal Desai cbf4452fe9 fix snapshot 2020-06-06 10:03:37 -07:00
Tejal Desai b7462742c1 add deletion logic 2020-06-06 02:05:45 -07:00
Yoan Blanc ce8298bb14
image: add test
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-06-06 10:31:42 +02:00
Tejal Desai ede69f8507 check file changed in loop 2020-06-06 01:24:46 -07:00
Tejal Desai 2a7d33526b lint 2020-06-06 00:30:07 -07:00
Tejal Desai 0f5387960e minimal changes 2020-06-06 00:16:14 -07:00
Tejal Desai 58276274e8 better improvements 2020-06-05 23:44:17 -07:00
Yoan Blanc fcb055e581
Merge remote-tracking branch 'origin/master' into fix/1172 2020-06-06 08:42:30 +02:00
Tejal Desai 43338d4b2f add a new run command along with a new flag 2020-06-05 19:53:16 -07:00
Tejal Desai 6ab97e2b5f ADD GCB benchmark code 2020-06-05 12:46:47 -07:00
Tejal Desai bea020f34f add another snapshotter 2020-06-05 02:54:32 -07:00
Tejal Desai ec3ca84ad9 add another redo snapshotter 2020-06-04 22:02:13 -07:00
Tejal Desai 86d77d8800
Merge pull request #1283 from tejal29/fix_looping
[Perf] Reduce loops over files when taking FS snapshot.
2020-06-04 14:30:53 -07:00
Tejal Desai 2303e47aad
Merge pull request #1260 from ljakimczuk/master
Resolving nested meta ARGs
2020-06-04 13:24:34 -07:00
Tejal Desai 9c68c9a36c Merge branch 'master' into fix_looping 2020-06-04 13:18:30 -07:00
Tejal Desai 8a585829b8
Update pkg/dockerfile/dockerfile_test.go 2020-06-04 12:28:51 -07:00
Tejal Desai 994a412d0b some more renames 2020-06-02 16:08:46 -07:00
Tejal Desai cbf3073fda rename whitelist to ignorelist 2020-06-02 15:56:27 -07:00
Yoan Blanc af14ef2c8d
Merge remote-tracking branch 'origin/master' into fix/1172 2020-05-25 13:54:37 +02:00
Tejal Desai e0f93578b6
Merge pull request #1284 from tejal29/add_timings
add timings for resolving paths
2020-05-23 15:37:58 -07:00
Tejal Desai dcea89c317 rm file 2020-05-23 15:27:37 -07:00
Tejal Desai 41bc04dc12 add timings for resolving pahts 2020-05-23 14:29:46 -07:00
Tejal Desai 503aad17b7 Fix looping through files twice 2020-05-23 14:24:07 -07:00
Ben Einaudi 2f6090dcd7 Remove use of DefaultTransport
Using DefaultTransport and manipulating its tls configuration may lead to unexpected behaviour
2020-05-20 11:43:13 +02:00
Tejal Desai 31feaa4c12
Merge pull request #1263 from yorinasub17/git-auth-token-env
Add ability to set git auth token using environment variables
2020-05-20 00:59:14 -07:00
Tejal Desai 3f3c19a545
Merge pull request #1234 from tejal29/fix_copy_from
Apply dockefile exclude only for first stage
2020-05-19 15:40:51 -07:00
Tejal Desai 1d11b40052
Merge pull request #1253 from tejal29/fix_non
fix switching to non existent workdir
2020-05-19 15:39:08 -07:00
Tejal Desai da5c420ee9 use a mock 2020-05-19 15:05:46 -07:00
Yoan Blanc fef4bb55b7
image: prepend library/ when using registry
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-05-12 14:46:25 +02:00
Yoriyasu Yano 6e615a8ff9 Add ability to set git auth token using environment variables
Currently the only way to set the git auth is by including it in the context
git URL. This can be problematic for certain environments such as ECS where the
command args for are stored in logs. Introduces updates to allow setting the
authentication info using the environment variables `GIT_USERNAME` and
`GIT_PASSWORD`.
2020-05-11 10:59:48 -05:00
Lukasz Jakimczuk 3dcac1b906 Resolving nested meta ARGs against themselves and build ARGs 2020-05-09 01:23:02 +02:00
Mitchell Friedman 6978fab45c Add retries to image push.
This uses the default provided retry transport by
go-containerregistry as this originally had no retries
built in.

This is useful to avoid intermittent failures of image
registries when returning a retryable status code.
2020-05-08 14:18:43 +01:00
Tejal Desai c271f989f5 fix switching to non existent workdir 2020-05-07 18:59:01 -07:00
Tejal Desai cb11a9982c
Merge pull request #1250 from gliptak/patch-2
Correct typo
2020-05-07 18:27:38 -07:00
Tejal Desai c71deae38c remove dedupes 2020-05-07 16:37:17 -07:00
Gábor Lipták bec0e6a8e3
Correct typo 2020-05-07 16:24:07 -04:00
Tejal Desai d9a6bbe956 add tests 2020-05-06 12:11:03 -07:00
Tejal Desai 23ac909b2d Revert "small perf optimizing. Only remove whiteout path if it needs to be included in the tar"
This reverts commit 67db51810b.
2020-05-06 12:08:44 -07:00
David Dooling 8bfd370ef9
Provide full path to Docker config file
Add tests.  The tests assume a POSIX file system, but it seems many
other tests assume Linux, so perhaps this is not a problem, or at
least does not add a new problem.

Fixes #1235
2020-05-06 10:02:47 -05:00
Tejal Desai 113c2397ea
Update comment. 2020-05-05 22:27:28 -07:00
Tejal Desai 5e275d9824 Apply dockefile exclude only for first stage 2020-05-05 21:45:36 -07:00
David Dooling 0871dfd6ab
Use DOCKER_CONFIG for docker config location
If the DOCKER_CONFIG environment variable is set, use it when
determining if the Docker config file exists.  Fall back to kaniko
default if it the DOCKER_CONFIG environment variable is not set.

Fixes #1228
2020-05-04 16:46:50 -05:00
Tejal Desai b62b3779fe
Merge pull request #1223 from sileht/fix-workdir-relative
feat: ensure relative WorkingDir work
2020-05-04 10:29:51 -07:00
Jordan Goasdoue a9c7839491 fix: stages are now resolved correctly if skip-unused 2020-05-04 18:38:18 +02:00
Mehdi Abaakouk cfc9f39176 feat: ensure relative WorkingDir work
Currently the default WorkingDir in test is "/", while in reallife it's
empty.

This change the tests to reflect reel life and fix the case where
First WorkingDir is relative.
2020-05-04 15:48:39 +02:00
Tejal Desai ae9c9b2813
Merge pull request #1211 from jonhenrik13/log-timestamp-v0.19.0
Add timestamp to logs
2020-05-03 22:41:17 -07:00
Tejal Desai e1c7862911
Merge pull request #1214 from tejal29/experiment
Snapshot FS on first cache miss.
2020-05-03 22:39:25 -07:00
Tejal Desai ee097f9b70 fix unit tests 2020-05-03 22:01:50 -07:00
Tejal Desai e32715ef55
Merge pull request #1164 from tstromberg/add-groups
Add secondary group impersonation w/ !cgo support
2020-05-03 22:00:36 -07:00
Tejal Desai cb14b65e08 fix boilerplate 2020-05-03 21:10:55 -07:00
Tejal Desai 5cbf34b67e
Update pkg/util/groupids_fallback.go 2020-05-03 21:04:22 -07:00
Tejal Desai d37896b94f
Merge branch 'master' into experiment 2020-05-03 21:02:41 -07:00
Tejal Desai 94ee809e1c
Merge branch 'master' into log-timestamp-v0.19.0 2020-05-03 20:58:34 -07:00
Tejal Desai fe0500f8e6
Merge pull request #1196 from yw-liu/master
Add http support for git repository context
2020-05-03 20:53:43 -07:00
Tejal Desai ae11db43d7
Merge pull request #1193 from tp-tc/dangling-symlink
Ignore the target of dangling symlinks.
2020-05-03 20:47:12 -07:00
Tejal Desai a2af3272cf
Merge pull request #1192 from tp-tc/snapshot-maintainers
Handle `MAINTAINERS` when passing `--single-snapshot`.
2020-05-03 20:45:48 -07:00
Tejal Desai 095ea2991a
remove unused command 2020-05-03 20:40:41 -07:00
Tejal Desai 5090baafda fix single snapshot 2020-05-03 20:23:07 -07:00
Tejal Desai db7687dbce wip 2020-05-03 18:37:45 -07:00
Tejal Desai 75864d5c85 flip the flag 2020-05-01 23:41:06 -07:00
Tejal Desai 7d32139a13 fix a bug where file check nil is insufficient 2020-05-01 22:38:14 -07:00
Tejal Desai 32e3336d4c add a new method to indicate if the command provides files to snapshot 2020-05-01 15:50:46 -07:00
Tejal Desai 36748facc5
Merge pull request #1203 from dani29/cache
Set very large logs to Trace level
2020-05-01 10:35:15 -07:00
Tejal Desai 7ccf05fae3
Merge pull request #1165 from JordanGoasdoue/multistage-now-respects-dependencies
feat: multistages now respect dependencies without building unnecessary stages
2020-05-01 10:34:47 -07:00
Tejal Desai 7eb2458ba4
Merge pull request #1166 from gilbsgilbs/snapshot-directories
Always add parent directories of files to snapshots.
2020-05-01 10:25:25 -07:00
Tejal Desai 8a780be748
Merge pull request #1139 from JordanGoasdoue/allow-injecting-context-tar-gz
feat: allow injecting through stdin tar.gz on kaniko
2020-05-01 10:24:05 -07:00
Tejal Desai 6c627645ad
Merge pull request #1194 from tp-tc/greedy-cache-substitution
Don't generate cache key, if not caching builds.
2020-05-01 10:21:44 -07:00
Tejal Desai 2e1ca5f19d
remove log added for debugging 2020-05-01 09:40:39 -07:00
Ben Einaudi d763b7e961 fix previous name checking in 'executor.build.fetchExtraStages' 2020-05-01 16:45:01 +02:00
Tejal Desai 8b3ff95b09 fmt 2020-04-30 17:07:18 -07:00
Tejal Desai 44b35b28eb remove read succesS 2020-04-30 15:18:39 -07:00
Tejal Desai 8fb17f60d9 Defer initial snapshot. Remove ReadSuccess() 2020-04-30 13:25:22 -07:00
Tejal Desai 691acd094a try different root dir 2020-04-29 22:42:33 -07:00
Jon Henrik Bjørnstad 41a95fe4bd Add timestamp to logs 2020-04-23 13:40:47 +02:00
Dani Raznikov 04888f2690 Set loud logs to trace level 2020-04-18 20:27:25 +03:00
Jordan Goasdoué 8cbc7a8ca2 feat: multistages is now built without unusued stages 2020-04-17 09:59:56 +02:00
yw-liu 73eb47c752 replace string literal with constant 2020-04-17 09:07:38 +08:00
yw-liu 2e90173238 fix boilerplate 2020-04-17 00:37:12 +08:00
yw-liu c9fc6b5bcf fix golint problem 2020-04-17 00:21:48 +08:00
yw-liu 0fc311a8b7 make string var as constant 2020-04-17 00:00:24 +08:00
yw-liu 7912e4c87b modify unit-test 2020-04-16 23:48:43 +08:00
yw-liu d8b8e811dd modify code format and unit-test 2020-04-16 23:44:37 +08:00
yw-liu 4f8d074e00
add unit-test 2020-04-16 22:21:09 +08:00
Dani Raznikov 8d6a75a745 merge conflict fixes 2020-04-15 16:10:45 +03:00
Dani Raznikov e8323441ab resolve conflict with PR #1160 2020-04-15 15:58:43 +03:00
Tejal Desai 2270ac03d2
Merge pull request #1160 from JordanGoasdoue/can-now-resolve-args-from-stage
feat: can now resolves args from all stages
2020-04-14 14:18:09 -07:00
yw-liu 34a6ec250f
add http support for git pull
usage: set the GIT_PULL_METHOD env var to http or https for starting the container
2020-04-14 20:25:20 +08:00
Tom Prince c520218cec Don't generate cache key, if not caching builds.
The cache key generation does environment subsitution in places that running
the commands doesn't. This causes issues if a command uses complex shell
substitutions. The cache key is generated even if caching isn't enabled.

This disables the cache key generation if caching is not enabled. This doesn't
fix the underlying issue, but limits it to when the cache is being used.
2020-04-13 16:50:13 -06:00
Tom Prince 5061b72e42 Ignore the target of dangling symlinks. 2020-04-13 16:26:19 -06:00
Tom Prince 484d03550c Handle `MAINTAINERS` when passing `--single-snapshot`. 2020-04-13 15:54:27 -06:00
Dani Raznikov f720c817c7 add unit tests 2020-04-12 20:17:40 +03:00
Dani Raznikov 961e634366 calculate dependency for onbuild 2020-04-10 18:49:34 +03:00
Dani Raznikov 70eb7ebcfa suport multistage onbuild 2020-04-10 18:30:25 +03:00
Dani Raznikov 6b44ed4477 calculateDepdencies on a copy of envvars 2020-04-04 23:27:15 +03:00
Dani Raznikov 3ab6524fe5 optimize: don't parse Dockerfile twice and just reuse stages 2020-04-03 18:53:22 +03:00
Dani Raznikov a3ce1cebf3 optimize: don't parse Dockerfile twice and just reuse stages 2020-04-03 17:39:59 +03:00
Gilbert Gilb's 14170aa455 Fix sorting of parent directories.
This refactoring reversed the order of the "ParentDirectories" function: ffc372a63b (diff-d36eb675aa49a7b471e3a2be77005b18R465)

As a side-effect, parent directories weren't added in lexicographical
order, which broke some tests. We now ensure in unit test that the order
of the ParentDirectories function is stable.
2020-03-31 20:18:24 +02:00
Gilbert Gilb's fd8a2d6dd8 Merge branch 'master' into snapshot-directories 2020-03-31 14:25:04 +02:00
Tejal Desai 0fe0a64113
Merge pull request #1168 from PhoenixMage/32bit-overflow-fix
32bit overflow fix
2020-03-30 10:23:31 -07:00
Tejal Desai 8dc64549a1
Merge pull request #1155 from tejal29/fix_multi_stage_symlinks
Refactor Kaniko to test across multistages
2020-03-30 10:21:51 -07:00
PhoenixMage aa6a3de316
Fix 32bit overflow
math.UaxUint32 will actually overflow on a 32 bit platform (tested on armv7)

This will resolve that issue.
2020-03-30 23:37:34 +11:00
Gilbert Gilb's e5585fded8 Always add parent directories of files to snapshots.
During a snapshot, when a file changed and not its parent directories,
the parent directories weren't added to the layer. This is inconsistent
with Docker's behavior which always add parent directories to the layer.
In some edge-cases, it could lead to problems with docker considering
that parent directories where owned by root in forthcoming layers
although they shouldn't (see #1163).

Also, Docker seems to be POSIX compliant regarding the name of
directories in the archive, which always have a slash appended. This
commit also fixes this.

Fixes #1163
2020-03-29 18:25:37 +02:00
Thomas Stromberg a6504769e3 Add secondary group impersonation w/ !cgo support 2020-03-28 22:01:20 -07:00
Thomas Stromberg 3426a24929 Add support for impersonating secondary groups 2020-03-28 12:10:37 -07:00
Jordan Goasdoué ed883b9015 feat: can now resolves args from stage 2020-03-27 18:42:46 +01:00
Tejal Desai 340ca79fbb lint 2020-03-25 11:01:12 -07:00
Tejal Desai 1978f1e08e
Merge pull request #1130 from Vlatombe/set_image_platform
Set image platform for any build
2020-03-25 10:10:41 -07:00
Tejal Desai 9567b755dd fix unit tests 2020-03-25 10:09:33 -07:00
Tejal Desai 2ea28fc7f5 revert back to previous file to save logic 2020-03-25 09:43:59 -07:00
Tejal Desai caaf6c8adf fix tests 2020-03-24 10:50:59 -07:00
Sam Stoelinga 1b662e7834
Merge pull request #1140 from samos123/fix-1122-public-gcr
Add ability to use public GCR repos without being authenticated
2020-03-24 10:31:58 -07:00
Vincent Latombe 336bcd3a9a
Merge branch 'master' into set_image_platform 2020-03-24 09:51:11 +01:00
Tejal Desai 5da32d3f18
Merge pull request #1147 from tejal29/small_perf
small perf optimizing. Only remove whiteout path if it needs to be included in base image
2020-03-23 18:12:39 -07:00
Tejal Desai ffc372a63b refactor to add unit tests 2020-03-23 17:48:49 -07:00
Sam Stoelinga 1fb1eddd8c Test: Remove DockerConf after CheckPushPermission 2020-03-23 10:51:09 -07:00
Shihab Hasan 99b08059c4 Checkout a specific git commit 2020-03-23 12:12:52 +06:00
Sam Stoelinga c2393df7c3 Cleanup push_test CheckPushPermission unit tests 2020-03-22 17:11:08 -07:00
Sam Stoelinga c56f16b163 Create constant for Docker conf location 2020-03-21 16:45:04 -07:00
Sam Stoelinga 8518b37ca1 Add unit tests for CheckPushPermissions 2020-03-21 16:15:34 -07:00
Jordan Goasdoué d08a767454 feat: allow injecting through stdin tar.gz on kaniko 2020-03-18 12:29:49 +01:00
Yoan Blanc 03fcc537c3
Merge remote-tracking branch 'origin/master' into fix-chown 2020-03-18 09:01:15 +01:00
Tejal Desai 67db51810b small perf optimizing. Only remove whiteout path if it needs to be included in the tar 2020-03-18 00:43:55 -07:00
Tejal Desai 0302e5150c
Merge pull request #1085 from tejal29/rm_buildargs_from_cache_key
remove build args from composite key and replace all build args
2020-03-17 16:02:59 -07:00
Tejal Desai a173bf020b
Merge pull request #1115 from JordanGoasdoue/allow-build-context-from-local-targz
feat: add support of local '.tar.gz' file inside the kaniko container
2020-03-17 15:24:10 -07:00
Tejal Desai 0185818bd8
Merge pull request #1134 from dani29/master
Add support to --chown flag to ADD command (Issue #57)
2020-03-17 15:07:02 -07:00
Sam Stoelinga e1e6c55386
Merge pull request #1128 from zhangyuheng/fix/typo
fix: credential typo
2020-03-17 11:26:36 -07:00
Sam Stoelinga 66268690b3 Add comments 2020-03-17 09:11:50 -07:00
Sam Stoelinga 8a020010b7 Add ability to use public GCR image
Kaniko by default used to configure the GCR credential helper however
this caused Kaniko to fail when trying to use a base image from a public
GCR image. This patch makes it possible to use public GCR images as base
image when using docker even when you're not authenticated to GCR.

Co-authored-by: Nate Williams <nate.williams@files.com>
2020-03-17 09:09:41 -07:00
Yoan Blanc 3ff2ee3ba5
copy: unit test for chown
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-17 15:19:00 +01:00
Dani Raznikov c136f886a9 Add support to --chown flag to ADD command 2020-03-14 16:24:40 +02:00
Jordan GOASDOUE fda7ed4f74 feat: add support of local '.tar.gz' file inside the kaniko container 2020-03-12 22:51:30 +01:00
Tejal Desai 32decf5798
Merge pull request #1075 from greut/label-flag
executor: add --label flag
2020-03-12 13:47:01 -07:00
Tejal Desai 18de5d6d2d
Merge pull request #1037 from antechrestos/feature/add_option_to_import_registry_certificate
Allow user to provide registry certificate
2020-03-12 12:56:40 -07:00
Vincent Latombe 4d26c1f51e
Set OS/ARCH when assembling image 2020-03-12 10:41:32 +01:00
Yuheng Zhang 8eef58939b fix: credential typo 2020-03-12 14:31:15 +08:00
Tejal Desai 9592f2640f more tests similar to docker cp 2020-03-09 15:54:35 -07:00
Ben Einaudi b73c2c10c8 Allow user to provide registry certificate
Fixes #1100
Fixes #1101
2020-03-08 18:17:50 +01:00
Tejal Desai 2181c5e6f5 create correct link across Multistage builds 2020-03-07 11:28:16 -08:00
Yoan Blanc 53326fa0bf
executor: add --label flag
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-07 18:17:26 +01:00
Tejal Desai c523c691eb revert back to old 0.17.1 behavior 2020-03-06 23:48:31 -08:00
Tejal Desai 8f87267002 fix lint error 2020-03-06 21:37:00 -08:00
Tejal Desai 6c14d202a3 better error wrapping and add more tests for copy 2020-03-06 17:18:36 -08:00
Tejal Desai b4b70d040a wip 2020-03-06 13:08:02 -08:00
Tejal Desai 3cc8aa2a9c fix resolve link for dirs with trailing / 2020-03-06 09:38:00 -08:00
Yoan Blanc fc668cd2b5
fixup! fixup! fixup! run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 17:26:04 +01:00
Yoan Blanc bd77f131fd
fixup! fixup! run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 17:00:14 +01:00
Yoan Blanc 67fbdef360
fixup! run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 16:49:13 +01:00
Yoan Blanc c7028c6d2f
run: user LookupId
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 16:44:55 +01:00
Yoan Blanc c553184416
user_test: docker doesn't care about group really
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-05 16:34:06 +01:00
Tejal Desai 579ec52a30 add more tests for stageBuilder_build 2020-03-04 17:04:42 -08:00
Tejal Desai b9f4c4bab5 add more tests for stageBuilder_build 2020-03-04 16:57:26 -08:00
Tejal Desai 9e83210b07 Rebase over 1105 2020-03-04 14:16:24 -08:00
Yoan Blanc 3d9bb21f75 user: keep existing HOME env
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-04 14:14:25 -08:00
Yoan Blanc fec7c126f9 run: set userStr
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-04 14:14:10 -08:00
Yoan Blanc 52acd798f8 run: unset any HOME variable after USER ran
Running USER didn't reset HOME which had to be explicitly set.

Closes #1082

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-04 14:12:42 -08:00
Tejal Desai c1e3f42625 fix user metadata string 2020-03-04 13:08:38 -08:00
Tejal Desai dd5d2e6661 ignore errors while deleting 2020-02-28 04:48:22 -08:00
Tejal Desai 17f808da8a check for filepath.Walk error everywhere 2020-02-28 04:23:27 -08:00
Tejal Desai ae5a7bf158 fix tests 2020-02-28 02:53:42 -08:00
Tejal Desai 58c0d47fa6 fix npe 2020-02-28 02:44:43 -08:00
Tejal Desai 2ba7d25090 remove build args from composite key and replace all build args before adding command to the cache 2020-02-28 01:51:31 -08:00
Tejal Desai a1af057f99
Merge pull request #1069 from cvgw/u/cgwippern/implement-filepath-resolver
Resolve filepaths before scanning for changes
2020-02-25 00:19:38 -08:00
cvgw 60e2187d85 Fix #1020 os.Chtimes invalid arg
The zero value of time.Time is not a valid argument to os.Chtimes
because of the syscall that os.Chtimes calls. Instead we update the zero
value of time.Time to the zero value of Unix Epoch
2020-02-24 08:29:19 -08:00
cvgw 965b606720 remove cruft and unneeded loop 2020-02-23 13:38:08 -08:00
cvgw 01f6aba517 update resolveSymlinkAncestor and add tests 2020-02-22 11:17:39 -08:00
cvgw a675ad998a Resolve filepaths before scanning for changes 2020-02-20 09:45:44 -08:00
Yoan Blanc 474fdcf68d
image: fix registry mirror setup
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-02-16 11:41:51 +01:00
Yoan Blanc edc4eb33f4
Merge remote-tracking branch 'origin/master' into registry-mirror 2020-02-16 09:28:54 +01:00
Tejal Desai 2716557379
Merge pull request #1032 from tinkerborg/consistent-sha
Ensure image SHA stays consistent when layer contents haven't changed
2020-02-13 11:22:27 -08:00
cvgw 7e2009b5e2 More idiomatic logging config
Update logging configuration to be more idiomatic for a Cobra
application
2020-02-07 16:13:03 -08:00
Tejal Desai 8d9e6b8ea5
Merge pull request #1027 from tejal29/small_refacto
refactor copy.chown code and add more tests
2020-02-07 13:48:05 -08:00
Tejal Desai c9dc474031 fix linter 2020-02-07 13:47:17 -08:00
Tejal Desai a17ad8e8e8
Merge pull request #882 from cvgw/u/cvgw/reuse-cached-layer
Do not recompute layers retrieved from cache
2020-02-07 13:37:39 -08:00
Tejal Desai 8b69a13641 fix commut 2020-02-06 15:37:47 -08:00
Tejal Desai 674f5bda5d fix tests 2020-02-06 15:17:09 -08:00
Or Sela 9dd050b892 set log format using a flag 2020-02-07 00:05:59 +02:00
Tejal Desai d94a1ed53b fix linter 2020-02-06 11:40:41 -08:00
Tejal Desai 9dba097a7e refactor and add more tests 2020-02-06 11:40:41 -08:00
tinkerborg 82bce229bd omit uname/gname in tar headers
When using cache, the rootfs may not have been extracted. This prevents uname/gname from resolving
as there is no /etc/password or /etc/group. This makes this layer unnecessarily differ from
a cached layer which does contain this information. Omitting these should be consistent with Docker's
behavior.
2020-02-06 13:36:26 -05:00
tinkerborg cc2c9a0663 sort filesToAdd in TakeSnapshot
filesToAdd is sorted in TakeSnapshotFS, but not here. This makes ordering unpredictable within the layer's tarball,
causing the SHA to differ even if layer contents haven't changed
2020-02-06 13:36:21 -05:00
Tejal Desai 9e17ffd6cb fix unit tests 2020-02-05 15:59:07 -08:00
Tejal Desai 2cd68d2e2f fix flake in copy symlink 2020-02-05 14:40:52 -08:00
Tejal Desai 2e95c3040c update initialWhitelist instead of whitelist 2020-02-04 11:06:26 -08:00
Tejal Desai 31f626cb22 initilize group to 0 2020-02-03 22:15:56 -08:00
Tejal Desai 56125ff464 Rebase from master with some changes 2020-02-03 14:57:10 -08:00
xanonid ac4c80d241 Refactor code, introduce constants for uid/gid = -1 2020-02-03 14:56:23 -08:00
xanonid 517cb50278 Add Unit test for GetUIDAndGIDFromString 2020-02-03 14:56:23 -08:00
xanonid e3b5a7b85d Support COPY --chown flag (Closes: #9) 2020-02-03 14:56:23 -08:00
Tejal Desai 18fec3ba38 fix lint 2020-02-03 11:03:58 -08:00
Tejal Desai db6f6c5ad3 more tests and do not error out if user not found 2020-02-03 10:11:40 -08:00
Tejal Desai 6894be442f remove duplicate code 2020-02-01 15:19:36 -08:00
Tejal Desai 956495784e fix group string being always set to uid in case a user has a gid set 2020-02-01 15:19:36 -08:00
Tejal Desai 0e833e4643 fix test 2020-02-01 09:29:18 -08:00
Tejal Desai 5a4a6c4a17 fix test flake 2020-01-31 15:23:38 -08:00
Tejal Desai 3f73230884
Merge pull request #1000 from tejal29/769
whitelist  /tmp/apt-key-gpghome.* directory
2020-01-31 10:50:26 -08:00
Thomas Bonfort 6b6742fd9d Changle loglevel for whiteouts to debug 2020-01-31 12:02:32 +01:00
Tejal Desai 611ba51c88 fix web editor conflicts resolutions 2020-01-30 11:18:53 -08:00
Tejal Desai 47ab3fe997
Merge branch 'master' into registry-mirror 2020-01-29 11:42:06 -08:00
Tejal Desai 8bdcb4f544
Merge pull request #854 from WanzenBug/ignored-files-invalidate-cache
Fix caching to respect .dockerignore
2020-01-29 11:32:43 -08:00
Cole Wippern 65cd9124d1
Merge pull request #957 from antechrestos/fix/scopes_asked_to_remote_registry
Upgrade go-containerregistry third-party library
2020-01-28 16:00:21 -08:00
Tejal Desai 8b991f6baf Add falg to --whitelist-var-run set to true to preserver default kaniko behaviour of /var/run ignored. Set it to false to add /var/run in destination directory 2020-01-28 10:52:36 -08:00
Tejal Desai d49c198c90 Revert "add option additonal-whitelist"
This reverts commit 72bfed1850.
2020-01-28 08:52:36 -08:00
Tejal Desai c8b19894bb Revert "additional-whitelist"
This reverts commit ac8e123815.
2020-01-28 08:52:29 -08:00
Tejal Desai 0529579db7
Merge pull request #971 from tejal29/915
Fix Symlinks not being copied across stages
2020-01-27 10:58:49 -08:00
Ben Einaudi 03b8922ede Upgrade dependency to go-containerregistry third-party library
By doing so it will fix issues met when mixing source/remote registry

Close #808
2020-01-26 18:55:52 +01:00
Tejal Desai e8fffda7cb
Merge pull request #995 from tejal29/507
Fix home and group set for user command
2020-01-24 22:18:09 -08:00
Tejal Desai e0322042c7 use filepath.Match instead of regex 2020-01-24 22:12:37 -08:00
Tejal Desai 68e8c6e902
Merge pull request #975 from cvgw/u/cgwippern/fix-issue-506
Fix #506 add option additional-whitelist
2020-01-24 16:11:32 -08:00
Tejal Desai 5951d9b0ee add /tmp/apt-key to whitelist for Dockerfiles which use command 2020-01-24 15:54:44 -08:00
Tejal Desai 308137f46d
Merge pull request #994 from wmuizelaar/fix_extracted_verbose_logging
Change loglevel for extractedFiles to debug. Fixes #901
2020-01-24 14:04:50 -08:00
Moritz Wanzenböck bab1c170e8 fix skipping ignored directories if they have whitelisted content 2020-01-24 22:27:56 +01:00
cvgw ac8e123815 additional-whitelist
* update readme
* improve test
* change default
2020-01-24 10:22:20 -08:00
Cole Wippern 72bfed1850 add option additonal-whitelist
Add a new option additonal-whitelist which defaults
to a single entry, "/var/run". This will allow users to
remove "/var/run" from the whitelist or retain the current
behavior with no change.
2020-01-23 16:50:33 -08:00
Cole Wippern cd9be5d513 reuse cached layer 2020-01-23 16:48:26 -08:00
Cole Wippern 782e4916b6 update build and composite cache tests 2020-01-23 15:37:38 -08:00
Moritz Wanzenböck ef692e55f3 composite_cache: add unit tests
Add unit tests that check the behaviour of CompositeCache
on adding filesytem resources. It checks that
* 2 identical directory trees produces the same hash
* an extra file produces a different hash
* an extra directry produces a different hash
* an extra file that is excluded does not alter the hash
* an extra directory that is excluded does not alter the hash
2020-01-23 15:37:37 -08:00
Moritz Wanzenböck 123dcaf83e Fix caching to respect .dockerignore
Previously kaniko would compute the cache key for any copy command by computing
the combined hash of all files in a directory, even if they were listed
as ignored.

With this change, the cache key creation was updated to be aware of ignored
files.

Related issues:
* https://github.com/GoogleContainerTools/kaniko/issues/594
2020-01-23 15:37:37 -08:00
Tejal Desai 048de00c33 Fix home and group set for user command 2020-01-23 15:07:15 -08:00
Cole Wippern a2aae6274d
Merge pull request #973 from cvgw/u/cgwippern/fix-issue-704
Fix issue #704
2020-01-23 13:00:34 -08:00
Wietse Muizelaar 05a316c26c Change loglevel for extractedFiles to debug. Fixes https://github.com/GoogleContainerTools/kaniko/issues/901 2020-01-23 21:37:06 +01:00
Tejal Desai 3e5d0a6334 add unit tests 2020-01-23 11:12:54 -08:00
Tejal Desai 79d226ade0 refactor 2020-01-22 16:32:46 -08:00
Tejal Desai bb129e9c88 code review comments 2020-01-22 16:27:06 -08:00
Tejal Desai b271eb8ab9 fix linter 2020-01-22 15:39:03 -08:00
Tejal Desai 478205e5ca fix adding symlinks to FS which do not exists 2020-01-22 15:27:01 -08:00
Tejal Desai f1f7297478 fix tests 2020-01-22 11:47:10 -08:00
Tejal Desai da7e9928e4 Fix Symlinks not being copies across stages 2020-01-22 11:47:10 -08:00
Tejal Desai 6a6c547811
Merge pull request #980 from PhoenixMage/PR646_rebase_fix
Prefer platform that is currently running for pulling remote images and kaniko binary Makefile target
2020-01-21 16:26:00 -08:00
Tejal Desai 76f0bf44e5 fix hack linter 2020-01-21 15:22:49 -08:00
Tejal Desai d362359314
Merge pull request #981 from cvgw/u/cgwippern/fix-issue-940
Fix #940 set modtime when extracting
2020-01-21 09:59:22 -08:00
Tejal Desai 649a0ed99b
Merge pull request #983 from cvgw/u/cvgw/fix-issue-519
Fix #519 capital letter in stage names
2020-01-21 09:56:06 -08:00
Tejal Desai b1970a3584
Merge pull request #792 from loganprice/FixAddTar
Fix Ability for ADD to unTar a file
2020-01-21 09:55:25 -08:00
Cole Wippern b9b61e2a12
Merge pull request #943 from shortstories/bugfix/copy-with-symlink
Fix #942 COPY or ADD to symlink destination breaks image
2020-01-20 20:55:35 -08:00
ohchang-kwon df767bb195 apply code review results 2020-01-21 12:07:13 +09:00
cvgw e652f815ef Fix #519 capital letter in stage names
update stage code so that when comparing the BaseName of
a stage against the recorded, lowercase version of a Stage name
the BaseName is also lowercased.
2020-01-20 15:57:05 -08:00
Cole Wippern 4e8bdb3947 Fix #940 set modtime when extracting
Certain utilities like Apt depend on modtime
for certain files. Kaniko was not setting modtime when
extracting files and so this broke things like apt.

Kaniko now sets the file mod time to the value from the tar
header.
2020-01-20 09:11:44 -08:00
Anthony Davies ed22afc19b Rebase fix for PR #646
I needed this for my arm64 k8s cluster. I have zero Go experience but
enough experience with other things to fix the rebase (I think!). This
patch is working fine on my cluster.
2020-01-20 13:43:45 +11:00
Yoan Blanc e26bd672f1
fixup! Merge remote-tracking branch 'origin/master' into registry-mirror
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-01-18 09:36:22 +01:00
Yoan Blanc 140879fadd
Merge remote-tracking branch 'origin/master' into registry-mirror 2020-01-18 09:35:56 +01:00
Cole Wippern 96093359c1 rename mock pkg for lint 2020-01-17 14:59:14 -08:00
Cole Wippern f96ea3bbfb hand edit mock license
for some reason hack/boilerplate.sh failed after using `addlicense`
so I copy/pasted the license header from another file
2020-01-17 14:56:12 -08:00
Cole Wippern a9896d5c43 update mocks with license 2020-01-17 14:32:36 -08:00
Cole Wippern 876f24b0c9 update caching run and copy extracted files
Update caching run and copy commands to use the new
GetFSFromLayers method and include the whiteout option so that
whiteout files are extracted and included in extractedFiles
2020-01-17 14:19:58 -08:00
Cole Wippern 71aed35094 GetFSFromLayers
* add util.GetFSFromLayers
* GetFSFromImage delegates to GetFSFromLayers
* add FSOpts and FSConfig for GetFSFromLayers
* add tests for GetFSFromLayers
* add gomock for test support
* add mock_v1 for layers
2020-01-17 13:36:23 -08:00
Cole Wippern e19cc228ba
Merge pull request #941 from stijndehaes/feature/cache-warmer-private-reg-support
Support for private registries in the cache warmer
2020-01-17 08:38:59 -08:00
Tejal Desai e702d75e9c
Merge pull request #964 from samos123/fix-argsescape
Fix bug with docker compatibility ArgsEscaped
2020-01-16 16:13:57 -08:00
Stijn De Haes 3fa8f686db Support for private registries in the cache warmer
Closes #503

Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2020-01-15 15:35:42 +01:00
lou-lan da0e7c7bdc Clean code (Condition is always 'false' because 'err' is always 'nil' ). 2020-01-14 15:11:22 +08:00
Sam Stoelinga 64bfc3eabb Fix bug with docker compatibility
ArgsEscaped according to Docker docs should only be set in Windows
environments: https://docs.docker.com/engine/api/v1.30/

It was causing integration test to fail with following message:
```
FAIL: TestRun/test_Dockerfile_test_metadata (8.48s)
           "Diff": {
             "Adds": [
               "ArgsEscaped: true"
             ],
             "Dels": [
               "ArgsEscaped: false"
             ]
```

However docker 18.xx returns ArgsEscaped: true
whereas docker 19.xx returns ArgsEscaped: false
Hence this patch also adds the docker version check to the integration
to ignore ArgsEscaped being different when 18.xx is used.
2020-01-13 21:18:54 -08:00
Cole Wippern 776fa43eb2 refactor cache.Warm and add tests 2020-01-10 12:29:17 -08:00