Switch from containers/image to go-containerregistry (#140)

* Vendor changes for go-containerregistry switch.

* Manual changes for go-containerregistry switch.

The biggest change is refactoring the tarball unpacking.

* Pull more of container-diff out.

* More vendor removals.

* More unit tests.
This commit is contained in:
dlorenc
2018-04-25 19:21:05 -07:00
committed by GitHub
parent 806555538f
commit cd5b744904
994 changed files with 5363 additions and 141520 deletions
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+26
View File
@@ -0,0 +1,26 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
// anonymous implements Authenticator for anonymous authentication.
type anonymous struct{}
// Authorization implements Authenticator.
func (a *anonymous) Authorization() (string, error) {
return "", nil
}
// Anonymous is a singleton Authenticator for providing anonymous auth.
var Anonymous Authenticator = &anonymous{}
+29
View File
@@ -0,0 +1,29 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import (
"fmt"
)
// auth implements Authenticator for an "auth" entry of the docker config.
type auth struct {
token string
}
// Authorization implements Authenticator.
func (a *auth) Authorization() (string, error) {
return fmt.Sprintf("Basic %s", a.token), nil
}
+21
View File
@@ -0,0 +1,21 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
// Authenticator is used to authenticate Docker transports.
type Authenticator interface {
// Authorization returns the value to use in an http transport's Authorization header.
Authorization() (string, error)
}
+33
View File
@@ -0,0 +1,33 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import (
"encoding/base64"
"fmt"
)
// Basic implements Authenticator for basic authentication.
type Basic struct {
Username string
Password string
}
// Authorization implements Authenticator.
func (b *Basic) Authorization() (string, error) {
delimited := fmt.Sprintf("%s:%s", b.Username, b.Password)
encoded := base64.StdEncoding.EncodeToString([]byte(delimited))
return fmt.Sprintf("Basic %s", encoded), nil
}
+29
View File
@@ -0,0 +1,29 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import (
"fmt"
)
// Bearer implements Authenticator for bearer authentication.
type Bearer struct {
Token string `json:"token"`
}
// Authorization implements Authenticator.
func (b *Bearer) Authorization() (string, error) {
return fmt.Sprintf("Bearer %s", b.Token), nil
}
+17
View File
@@ -0,0 +1,17 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package authn defines different methods of authentication for
// talking to a container registry.
package authn
+94
View File
@@ -0,0 +1,94 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import (
"bytes"
"encoding/json"
"fmt"
"os/exec"
"strings"
"github.com/google/go-containerregistry/name"
)
// magicNotFoundMessage is the string that the CLI special cases to mean
// that a given registry domain wasn't found.
const (
magicNotFoundMessage = "credentials not found in native keychain"
)
// runner allows us to swap out how we "Run" os/exec commands.
type runner interface {
Run(*exec.Cmd) error
}
// defaultRunner implements runner by just calling Run().
type defaultRunner struct{}
// Run implements runner.
func (dr *defaultRunner) Run(cmd *exec.Cmd) error {
return cmd.Run()
}
// helper executes the named credential helper against the given domain.
type helper struct {
name string
domain name.Registry
// We add this layer of indirection to facilitate unit testing.
r runner
}
// helperOutput is the expected JSON output form of a credential helper
// (or at least these are the fields that we care about).
type helperOutput struct {
Username string
Secret string
}
// Authorization implements Authenticator.
func (h *helper) Authorization() (string, error) {
helperName := fmt.Sprintf("docker-credential-%s", h.name)
// We want to execute:
// echo -n {domain} | docker-credential-{name} get
cmd := exec.Command(helperName, "get")
// Some keychains expect a scheme:
// https://github.com/bazelbuild/rules_docker/issues/111
cmd.Stdin = strings.NewReader(fmt.Sprintf("https://%v", h.domain))
var out bytes.Buffer
cmd.Stdout = &out
if err := h.r.Run(cmd); err != nil {
return "", err
}
output := out.String()
// If we see this specific message, it means the domain wasn't found
// and we should fall back on anonymous auth.
if output == magicNotFoundMessage {
return Anonymous.Authorization()
}
// Any other output should be parsed as JSON and the Username / Secret
// fields used for Basic authentication.
ho := helperOutput{}
if err := json.Unmarshal([]byte(output), &ho); err != nil {
return "", err
}
b := Basic{Username: ho.Username, Password: ho.Secret}
return b.Authorization()
}
+140
View File
@@ -0,0 +1,140 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"os"
"os/user"
"path"
"github.com/google/go-containerregistry/name"
)
// Keychain is an interface for resolving an image reference to a credential.
type Keychain interface {
// Resolve looks up the most appropriate credential for the specified registry.
Resolve(name.Registry) (Authenticator, error)
}
// defaultKeychain implements Keychain with the semantics of the standard Docker
// credential keychain.
type defaultKeychain struct{}
// configDir returns the directory containing Docker's config.json
func configDir() (string, error) {
if dc := os.Getenv("DOCKER_CONFIG"); dc != "" {
return dc, nil
}
usr, err := user.Current()
if err != nil {
return "", err
}
return path.Join(usr.HomeDir, ".docker"), nil
}
// authEntry is a helper for JSON parsing an "auth" entry of config.json
// This is not meant for direct consumption.
type authEntry struct {
Auth string `json:"auth"`
Username string `json:"username"`
Password string `json:"password"`
}
// cfg is a helper for JSON parsing Docker's config.json
// This is not meant for direct consumption.
type cfg struct {
CredHelper map[string]string `json:"credHelpers,omitempty"`
CredStore string `json:"credsStore,omitempty"`
Auths map[string]authEntry `json:"auths,omitempty"`
}
// There are a variety of ways a domain may get qualified within the Docker credential file.
// We enumerate them here as format strings.
var (
domainForms = []string{
// Allow naked domains
"%s",
// Allow scheme-prefixed.
"https://%s",
"http://%s",
// Allow scheme-prefixes with version in url path.
"https://%s/v1/",
"http://%s/v1/",
"https://%s/v2/",
"http://%s/v2/",
}
// Export an instance of the default keychain.
DefaultKeychain Keychain = &defaultKeychain{}
)
// Resolve implements Keychain.
func (dk *defaultKeychain) Resolve(reg name.Registry) (Authenticator, error) {
dir, err := configDir()
if err != nil {
log.Printf("Unable to determine config dir, falling back on anonymous: %v", err)
return Anonymous, nil
}
file := path.Join(dir, "config.json")
content, err := ioutil.ReadFile(file)
if err != nil {
log.Printf("Unable to read %q, falling back on anonymous: %v", file, err)
return Anonymous, nil
}
var cf cfg
if err := json.Unmarshal(content, &cf); err != nil {
log.Printf("Unable to parse %q, falling back on anonymous: %v", file, err)
return Anonymous, nil
}
// Per-registry credential helpers take precedence.
if cf.CredHelper != nil {
for _, form := range domainForms {
if entry, ok := cf.CredHelper[fmt.Sprintf(form, reg.Name())]; ok {
return &helper{name: entry, domain: reg, r: &defaultRunner{}}, nil
}
}
}
// A global credential helper is next in precedence.
if cf.CredStore != "" {
return &helper{name: cf.CredStore, domain: reg, r: &defaultRunner{}}, nil
}
// Lastly, the 'auths' section directly contains basic auth entries.
if cf.Auths != nil {
for _, form := range domainForms {
if entry, ok := cf.Auths[fmt.Sprintf(form, reg.Name())]; ok {
if entry.Auth != "" {
return &auth{entry.Auth}, nil
} else if entry.Username != "" {
return &Basic{Username: entry.Username, Password: entry.Password}, nil
} else {
// TODO(mattmoor): Support identitytoken
// TODO(mattmoor): Support registrytoken
return nil, fmt.Errorf("Unsupported entry in \"auths\" section of %q", file)
}
}
}
}
log.Printf("No matching credentials found for %v, falling back on anoynmous", reg)
return Anonymous, nil
}
+52
View File
@@ -0,0 +1,52 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"strings"
"unicode/utf8"
)
// Strictness defines the level of strictness for name validation.
type Strictness int
// Enums for CRUD operations.
const (
StrictValidation Strictness = iota
WeakValidation
)
// stripRunesFn returns a function which returns -1 (i.e. a value which
// signals deletion in strings.Map) for runes in 'runes', and the rune otherwise.
func stripRunesFn(runes string) func(rune) rune {
return func(r rune) rune {
if strings.ContainsRune(runes, r) {
return -1
}
return r
}
}
// checkElement checks a given named element matches character and length restrictions.
// Returns true if the given element adheres to the given restrictions, false otherwise.
func checkElement(name, element, allowedRunes string, minRunes, maxRunes int) error {
numRunes := utf8.RuneCountInString(element)
if (numRunes < minRunes) || (maxRunes < numRunes) {
return NewErrBadName("%s must be between %d and %d runes in length: %s", name, minRunes, maxRunes, element)
} else if len(strings.Map(stripRunesFn(allowedRunes), element)) != 0 {
return NewErrBadName("%s can only contain the runes `%s`: %s", name, allowedRunes, element)
}
return nil
}
+91
View File
@@ -0,0 +1,91 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package name defines structured types for representing image references.
package name
import (
"strings"
)
const (
// These have the form: sha256:<hex string>
// TODO(dekkagaijin): replace with opencontainers/go-digest or docker/distribution's validation.
digestChars = "sh:0123456789abcdef"
digestDelim = "@"
)
// Digest stores a digest name in a structured form.
type Digest struct {
Repository
digest string
}
// Ensure Digest implements Reference
var _ Reference = (*Digest)(nil)
// Context implements Reference.
func (d Digest) Context() Repository {
return d.Repository
}
// Identifier implements Reference.
func (d Digest) Identifier() string {
return d.DigestStr()
}
// DigestStr returns the digest component of the Digest.
func (d Digest) DigestStr() string {
return d.digest
}
// Name returns the name from which the Digest was derived.
func (d Digest) Name() string {
return d.Repository.Name() + digestDelim + d.DigestStr()
}
func (d Digest) String() string {
return d.Name()
}
func checkDigest(name string) error {
return checkElement("digest", name, digestChars, 7+64, 7+64)
}
// NewDigest returns a new Digest representing the given name, according to the given strictness.
func NewDigest(name string, strict Strictness) (Digest, error) {
// Split on "@"
parts := strings.Split(name, digestDelim)
if len(parts) != 2 {
return Digest{}, NewErrBadName("a digest must contain exactly one '@' separator (e.g. registry/repository@digest) saw: %s", name)
}
base := parts[0]
digest := parts[1]
// We don't require a digest, but if we get one check it's valid,
// even when not being strict.
// If we are being strict, we want to validate the digest regardless in case
// it's empty.
if digest != "" || strict == StrictValidation {
if err := checkDigest(digest); err != nil {
return Digest{}, err
}
}
repo, err := NewRepository(base, strict)
if err != nil {
return Digest{}, err
}
return Digest{repo, digest}, nil
}
+37
View File
@@ -0,0 +1,37 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import "fmt"
// ErrBadName is an error for when a bad docker name is supplied.
type ErrBadName struct {
info string
}
func (e *ErrBadName) Error() string {
return e.info
}
// NewErrBadName returns a ErrBadName which returns the given formatted string from Error().
func NewErrBadName(fmtStr string, args ...interface{}) *ErrBadName {
return &ErrBadName{fmt.Sprintf(fmtStr, args...)}
}
// IsErrBadName returns true if the given error is an ErrBadName.
func IsErrBadName(err error) bool {
_, ok := err.(*ErrBadName)
return ok
}
+49
View File
@@ -0,0 +1,49 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"errors"
"fmt"
)
// Reference defines the interface that consumers use when they can
// take either a tag or a digest.
type Reference interface {
fmt.Stringer
// Context accesses the Repository context of the reference.
Context() Repository
// Identifier accesses the type-specific portion of the reference.
Identifier() string
// Name is the fully-qualified reference name.
Name() string
// Scope is the scope needed to access this reference.
Scope(string) string
}
// ParseReference parses the string as a reference, either by tag or digest.
func ParseReference(s string, strict Strictness) (Reference, error) {
if t, err := NewTag(s, strict); err == nil {
return t, nil
}
if d, err := NewDigest(s, strict); err == nil {
return d, nil
}
return nil, errors.New("could not parse reference")
}
+79
View File
@@ -0,0 +1,79 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import "net/url"
const (
DefaultRegistry = "index.docker.io"
defaultRegistryAlias = "docker.io"
)
// Registry stores a docker registry name in a structured form.
type Registry struct {
registry string
}
// RegistryStr returns the registry component of the Registry.
func (r Registry) RegistryStr() string {
if r.registry != "" {
return r.registry
}
return DefaultRegistry
}
// Name returns the name from which the Registry was derived.
func (r Registry) Name() string {
return r.RegistryStr()
}
func (r Registry) String() string {
return r.Name()
}
// Scope returns the scope required to access the registry.
func (r Registry) Scope(string) string {
// The only resource under 'registry' is 'catalog'. http://goo.gl/N9cN9Z
return "registry:catalog:*"
}
func checkRegistry(name string) error {
// Per RFC 3986, registries (authorities) are required to be prefixed with "//"
// url.Host == hostname[:port] == authority
if url, err := url.Parse("//" + name); err != nil || url.Host != name {
return NewErrBadName("registries must be valid RFC 3986 URI authorities: %s", name)
}
return nil
}
// NewRegistry returns a Registry based on the given name.
// Strict validation requires explicit, valid RFC 3986 URI authorities to be given.
func NewRegistry(name string, strict Strictness) (Registry, error) {
if strict == StrictValidation && len(name) == 0 {
return Registry{}, NewErrBadName("strict validation requires the registry to be explicitly defined")
}
if err := checkRegistry(name); err != nil {
return Registry{}, err
}
// Rewrite "docker.io" to "index.docker.io".
// See: https://github.com/google/go-containerregistry/issues/68
if name == defaultRegistryAlias {
name = DefaultRegistry
}
return Registry{registry: name}, nil
}
+99
View File
@@ -0,0 +1,99 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"fmt"
"strings"
)
const (
defaultNamespace = "library"
repositoryChars = "abcdefghijklmnopqrstuvwxyz0123456789_-./"
regRepoDelimiter = "/"
)
// Repository stores a docker repository name in a structured form.
type Repository struct {
Registry
repository string
}
// See https://docs.docker.com/docker-hub/official_repos
func hasImplicitNamespace(repo string, reg Registry) bool {
return !strings.ContainsRune(repo, '/') && reg.RegistryStr() == DefaultRegistry
}
// RepositoryStr returns the repository component of the Repository.
func (r Repository) RepositoryStr() string {
if hasImplicitNamespace(r.repository, r.Registry) {
return fmt.Sprintf("%s/%s", defaultNamespace, r.repository)
}
return r.repository
}
// Name returns the name from which the Repository was derived.
func (r Repository) Name() string {
regName := r.Registry.Name()
if regName != "" {
return regName + regRepoDelimiter + r.RepositoryStr()
}
return r.RepositoryStr()
}
func (r Repository) String() string {
return r.Name()
}
// Scope returns the scope required to perform the given action on the registry.
// TODO(jonjohnsonjr): consider moving scopes to a separate package.
func (r Repository) Scope(action string) string {
return fmt.Sprintf("repository:%s:%s", r.RepositoryStr(), action)
}
func checkRepository(repository string) error {
return checkElement("repository", repository, repositoryChars, 2, 255)
}
// NewRepository returns a new Repository representing the given name, according to the given strictness.
func NewRepository(name string, strict Strictness) (Repository, error) {
if len(name) == 0 {
return Repository{}, NewErrBadName("a repository name must be specified")
}
var registry string
repo := name
parts := strings.SplitN(name, regRepoDelimiter, 2)
if len(parts) == 2 && (strings.ContainsRune(parts[0], '.') || strings.ContainsRune(parts[0], ':')) {
// The first part of the repository is treated as the registry domain
// iff it contains a '.' or ':' character, otherwise it is all repository
// and the domain defaults to DockerHub.
registry = parts[0]
repo = parts[1]
}
if err := checkRepository(repo); err != nil {
return Repository{}, err
}
reg, err := NewRegistry(registry, strict)
if err != nil {
return Repository{}, err
}
if hasImplicitNamespace(repo, reg) && strict == StrictValidation {
return Repository{}, NewErrBadName("strict validation requires the full repository path (missing 'library')")
}
return Repository{reg, repo}, nil
}
+99
View File
@@ -0,0 +1,99 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"strings"
)
const (
defaultTag = "latest"
// TODO(dekkagaijin): use the docker/distribution regexes for validation.
tagChars = "abcdefghijklmnopqrstuvwxyz0123456789_-.ABCDEFGHIJKLMNOPQRSTUVWXYZ"
tagDelim = ":"
)
// Tag stores a docker tag name in a structured form.
type Tag struct {
Repository
tag string
}
// Ensure Tag implements Reference
var _ Reference = (*Tag)(nil)
// Context implements Reference.
func (t Tag) Context() Repository {
return t.Repository
}
// Identifier implements Reference.
func (t Tag) Identifier() string {
return t.TagStr()
}
// TagStr returns the tag component of the Tag.
func (t Tag) TagStr() string {
if t.tag != "" {
return t.tag
}
return defaultTag
}
// Name returns the name from which the Tag was derived.
func (t Tag) Name() string {
return t.Repository.Name() + tagDelim + t.TagStr()
}
func (t Tag) String() string {
return t.Name()
}
// Scope returns the scope required to perform the given action on the tag.
func (t Tag) Scope(action string) string {
return t.Repository.Scope(action)
}
func checkTag(name string) error {
return checkElement("tag", name, tagChars, 1, 127)
}
// NewTag returns a new Tag representing the given name, according to the given strictness.
func NewTag(name string, strict Strictness) (Tag, error) {
base := name
tag := ""
// Split on ":"
if parts := strings.Split(name, tagDelim); len(parts) >= 2 {
base = strings.Join(parts[:len(parts)-1], tagDelim)
tag = parts[len(parts)-1]
}
// We don't require a tag, but if we get one check it's valid,
// even when not being strict.
// If we are being strict, we want to validate the tag regardless in case
// it's empty.
if tag != "" || strict == StrictValidation {
if err := checkTag(tag); err != nil {
return Tag{}, err
}
}
repo, err := NewRepository(base, strict)
if err != nil {
return Tag{}, err
}
return Tag{repo, tag}, nil
}
+108
View File
@@ -0,0 +1,108 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"encoding/json"
"io"
"time"
)
// ConfigFile is the configuration file that holds the metadata describing
// how to launch a container. The names of the fields are chosen to reflect
// the JSON payload of the ConfigFile as defined here: https://git.io/vrAEY
type ConfigFile struct {
Architecture string `json:"architecture"`
Container string `json:"container"`
Created Time `json:"created"`
DockerVersion string `json:"docker_version"`
History []History `json:"history"`
OS string `json:"os"`
RootFS RootFS `json:"rootfs"`
Config Config `json:"config"`
ContainerConfig Config `json:"container_config"`
OSVersion string `json:"osversion"`
}
// History is one entry of a list recording how this container image was built.
type History struct {
Author string `json:"author"`
Created Time `json:"created"`
CreatedBy string `json:"created_by"`
Comment string `json:"comment"`
EmptyLayer bool `json:"empty_layer,omitempty"`
}
// Time is a wrapper around time.Time to help with deep copying
type Time struct {
time.Time
}
// DeepCopyInto creates a deep-copy of the Time value. The underlying time.Time
// type is effectively immutable in the time API, so it is safe to
// copy-by-assign, despite the presence of (unexported) Pointer fields.
func (t *Time) DeepCopyInto(out *Time) {
*out = *t
}
// RootFS holds the ordered list of file system deltas that comprise the
// container image's root filesystem.
type RootFS struct {
Type string `json:"type"`
DiffIDs []Hash `json:"diff_ids"`
}
// Config is a submessage of the config file described as:
// The execution parameters which SHOULD be used as a base when running
// a container using the image.
// The names of the fields in this message are chosen to reflect the JSON
// payload of the Config as defined here:
// https://git.io/vrAET
// and
// https://github.com/opencontainers/image-spec/blob/master/config.md
type Config struct {
AttachStderr bool
AttachStdin bool
AttachStdout bool
Cmd []string
Domainname string
Entrypoint []string
Env []string
Hostname string
Image string
Labels map[string]string
OnBuild []string
OpenStdin bool
StdinOnce bool
Tty bool
User string
Volumes map[string]struct{}
WorkingDir string
ExposedPorts map[string]struct{}
ArgsEscaped bool
NetworkDisabled bool
MacAddress string
StopSignal string
Shell []string
}
// ParseConfigFile parses the io.Reader's contents into a ConfigFile.
func ParseConfigFile(r io.Reader) (*ConfigFile, error) {
cf := ConfigFile{}
if err := json.NewDecoder(r).Decode(&cf); err != nil {
return nil, err
}
return &cf, nil
}
+19
View File
@@ -0,0 +1,19 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package v1 defines structured types for OCI v1 images
// +k8s:deepcopy-gen=package
//go:generate deepcopy-gen -O zz_deepcopy_generated --go-header-file $BOILER_PLATE_FILE -i .
package v1
+16
View File
@@ -0,0 +1,16 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package empty provides an implementation of v1.Image equivalent to "FROM scratch".
package empty
+22
View File
@@ -0,0 +1,22 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package empty
import (
"github.com/google/go-containerregistry/v1/random"
)
// Image is a singleton empty image, think: FROM scratch.
var Image, _ = random.Image(0, 0)
+111
View File
@@ -0,0 +1,111 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"hash"
"io"
"strconv"
"strings"
)
// Hash is an unqualified digest of some content, e.g. sha256:deadbeef
type Hash struct {
// Algorithm holds the algorithm used to compute the hash.
Algorithm string
// Hex holds the hex portion of the content hash.
Hex string
}
// String reverses NewHash returning the string-form of the hash.
func (h Hash) String() string {
return fmt.Sprintf("%s:%s", h.Algorithm, h.Hex)
}
// NewHash validates the input string is a hash and returns a strongly type Hash object.
func NewHash(s string) (Hash, error) {
h := Hash{}
if err := h.parse(s); err != nil {
return Hash{}, err
}
return h, nil
}
// MarshalJSON implements json.Marshaler
func (h *Hash) MarshalJSON() ([]byte, error) {
return json.Marshal(h.String())
}
// UnmarshalJSON implements json.Unmarshaler
func (h *Hash) UnmarshalJSON(data []byte) error {
s, err := strconv.Unquote(string(data))
if err != nil {
return err
}
return h.parse(s)
}
// Hasher returns a hash.Hash for the named algorithm (e.g. "sha256")
func Hasher(name string) (hash.Hash, error) {
switch name {
case "sha256":
return sha256.New(), nil
default:
return nil, fmt.Errorf("unsupported hash: %q", name)
}
}
func (h *Hash) parse(unquoted string) error {
parts := strings.Split(unquoted, ":")
if len(parts) != 2 {
return fmt.Errorf("too many parts in hash: %s", unquoted)
}
rest := strings.TrimLeft(parts[1], "0123456789abcdef")
if len(rest) != 0 {
return fmt.Errorf("found non-hex character in hash: %c", rest[0])
}
hasher, err := Hasher(parts[0])
if err != nil {
return err
}
// Compare the hex to the expected size (2 hex characters per byte)
if len(parts[1]) != hasher.Size()*2 {
return fmt.Errorf("wrong number of hex digits for %s: %s", parts[0], parts[1])
}
h.Algorithm = parts[0]
h.Hex = parts[1]
return nil
}
// SHA256 computes the Hash of the provided io.Reader's content.
func SHA256(r io.Reader) (Hash, int64, error) {
hasher := sha256.New()
n, err := io.Copy(hasher, r)
if err != nil {
return Hash{}, 0, err
}
return Hash{
Algorithm: "sha256",
Hex: hex.EncodeToString(hasher.Sum(make([]byte, 0, hasher.Size()))),
}, n, nil
}
+58
View File
@@ -0,0 +1,58 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"github.com/google/go-containerregistry/v1/types"
)
// Image defines the interface for interacting with an OCI v1 image.
type Image interface {
// Layers returns the ordered collection of filesystem layers that comprise this image.
// The order of the list is oldest/base layer first, and most-recent/top layer last.
Layers() ([]Layer, error)
// BlobSet returns an unordered collection of all the blobs in the image.
BlobSet() (map[Hash]struct{}, error)
// MediaType of this image's manifest.
MediaType() (types.MediaType, error)
// ConfigName returns the hash of the image's config file.
ConfigName() (Hash, error)
// ConfigFile returns this image's config file.
ConfigFile() (*ConfigFile, error)
// RawConfigFile returns the serialized bytes of ConfigFile()
RawConfigFile() ([]byte, error)
// Digest returns the sha256 of this image's manifest.
Digest() (Hash, error)
// Manifest returns this image's Manifest object.
Manifest() (*Manifest, error)
// RawManifest returns the serialized bytes of Manifest()
RawManifest() ([]byte, error)
// LayerByDigest returns a Layer for interacting with a particular layer of
// the image, looking it up by "digest" (the compressed hash).
LayerByDigest(Hash) (Layer, error)
// LayerByDiffID is an analog to LayerByDigest, looking up by "diff id"
// (the uncompressed hash).
LayerByDiffID(Hash) (Layer, error)
}
+37
View File
@@ -0,0 +1,37 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"io"
)
// Layer is an interface for accessing the properties of a particular layer of a v1.Image
type Layer interface {
// Digest returns the Hash of the compressed layer.
Digest() (Hash, error)
// DiffID returns the Hash of the uncompressed layer.
DiffID() (Hash, error)
// Compressed returns an io.ReadCloser for the compressed layer contents.
Compressed() (io.ReadCloser, error)
// Uncompressed returns an io.ReadCloser for the uncompressed layer contents.
Uncompressed() (io.ReadCloser, error)
// Size returns the compressed size of the Layer.
Size() (int64, error)
}
+49
View File
@@ -0,0 +1,49 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"encoding/json"
"io"
"github.com/google/go-containerregistry/v1/types"
)
// Manifest represents the OCI image manifest in a structured way.
type Manifest struct {
SchemaVersion int64 `json:"schemaVersion"`
MediaType types.MediaType `json:"mediaType"`
Config Descriptor `json:"config"`
Layers []Descriptor `json:"layers"`
Annotations map[string]string `json:"annotations,omitempty"`
}
// Descriptor holds a reference from the manifest to one of its constituent elements.
type Descriptor struct {
MediaType types.MediaType `json:"mediaType"`
Size int64 `json:"size"`
Digest Hash `json:"digest"`
URLs []string `json:"urls,omitempty"`
Annotations map[string]string `json:"annotations,omitempty"`
}
// ParseManifest parses the io.Reader's contents into a Manifest.
func ParseManifest(r io.Reader) (*Manifest, error) {
m := Manifest{}
if err := json.NewDecoder(r).Decode(&m); err != nil {
return nil, err
}
return &m, nil
}
+233
View File
@@ -0,0 +1,233 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package mutate
import (
"encoding/json"
"errors"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/partial"
"github.com/google/go-containerregistry/v1/types"
)
// Addendum contains layers and history to be appended
// to a base image
type Addendum struct {
Layer v1.Layer
History v1.History
}
// AppendLayers applies layers to a base image
func AppendLayers(base v1.Image, layers ...v1.Layer) (v1.Image, error) {
additions := make([]Addendum, 0, len(layers))
for _, layer := range layers {
additions = append(additions, Addendum{Layer: layer})
}
return Append(base, additions...)
}
// Append will apply the list of addendums to the base image
func Append(base v1.Image, adds ...Addendum) (v1.Image, error) {
if len(adds) == 0 {
return base, nil
}
if err := validate(adds); err != nil {
return nil, err
}
m, err := base.Manifest()
if err != nil {
return nil, err
}
cf, err := base.ConfigFile()
if err != nil {
return nil, err
}
image := &image{
Image: base,
manifest: m.DeepCopy(),
configFile: cf.DeepCopy(),
diffIDMap: make(map[v1.Hash]v1.Layer),
digestMap: make(map[v1.Hash]v1.Layer),
}
diffIDs := image.configFile.RootFS.DiffIDs
history := image.configFile.History
for _, add := range adds {
diffID, err := add.Layer.DiffID()
if err != nil {
return nil, err
}
diffIDs = append(diffIDs, diffID)
history = append(history, add.History)
image.diffIDMap[diffID] = add.Layer
}
manifestLayers := image.manifest.Layers
for _, add := range adds {
d := v1.Descriptor{
MediaType: types.DockerLayer,
}
if d.Size, err = add.Layer.Size(); err != nil {
return nil, err
}
if d.Digest, err = add.Layer.Digest(); err != nil {
return nil, err
}
manifestLayers = append(manifestLayers, d)
image.digestMap[d.Digest] = add.Layer
}
image.configFile.RootFS.DiffIDs = diffIDs
image.configFile.History = history
image.manifest.Layers = manifestLayers
image.manifest.Config.Digest, err = image.ConfigName()
if err != nil {
return nil, err
}
return image, nil
}
// Config mutates the provided v1.Image to have the provided v1.Config
func Config(base v1.Image, cfg v1.Config) (v1.Image, error) {
m, err := base.Manifest()
if err != nil {
return nil, err
}
cf, err := base.ConfigFile()
if err != nil {
return nil, err
}
cf.Config = cfg
image := &image{
Image: base,
manifest: m.DeepCopy(),
configFile: cf.DeepCopy(),
diffIDMap: make(map[v1.Hash]v1.Layer),
digestMap: make(map[v1.Hash]v1.Layer),
}
image.manifest.Config.Digest, err = image.ConfigName()
if err != nil {
return nil, err
}
return image, nil
}
type image struct {
v1.Image
configFile *v1.ConfigFile
manifest *v1.Manifest
diffIDMap map[v1.Hash]v1.Layer
digestMap map[v1.Hash]v1.Layer
}
// Layers returns the ordered collection of filesystem layers that comprise this image.
// The order of the list is most-recent first, and oldest base layer last.
func (i *image) Layers() ([]v1.Layer, error) {
diffIDs, err := partial.DiffIDs(i)
if err != nil {
return nil, err
}
ls := make([]v1.Layer, 0, len(diffIDs))
for _, h := range diffIDs {
l, err := i.LayerByDiffID(h)
if err != nil {
return nil, err
}
ls = append(ls, l)
}
return ls, nil
}
// BlobSet returns an unordered collection of all the blobs in the image.
func (i *image) BlobSet() (map[v1.Hash]struct{}, error) {
return partial.BlobSet(i)
}
// ConfigName returns the hash of the image's config file.
func (i *image) ConfigName() (v1.Hash, error) {
return partial.ConfigName(i)
}
// ConfigFile returns this image's config file.
func (i *image) ConfigFile() (*v1.ConfigFile, error) {
return i.configFile, nil
}
// RawConfigFile returns the serialized bytes of ConfigFile()
func (i *image) RawConfigFile() ([]byte, error) {
return json.Marshal(i.configFile)
}
// Digest returns the sha256 of this image's manifest.
func (i *image) Digest() (v1.Hash, error) {
return partial.Digest(i)
}
// Manifest returns this image's Manifest object.
func (i *image) Manifest() (*v1.Manifest, error) {
return i.manifest, nil
}
// RawManifest returns the serialized bytes of Manifest()
func (i *image) RawManifest() ([]byte, error) {
return json.Marshal(i.manifest)
}
// LayerByDigest returns a Layer for interacting with a particular layer of
// the image, looking it up by "digest" (the compressed hash).
func (i *image) LayerByDigest(h v1.Hash) (v1.Layer, error) {
if cn, err := i.ConfigName(); err != nil {
return nil, err
} else if h == cn {
return partial.ConfigLayer(i)
}
if layer, ok := i.digestMap[h]; ok {
return layer, nil
}
return i.Image.LayerByDigest(h)
}
// LayerByDiffID is an analog to LayerByDigest, looking up by "diff id"
// (the uncompressed hash).
func (i *image) LayerByDiffID(h v1.Hash) (v1.Layer, error) {
if layer, ok := i.diffIDMap[h]; ok {
return layer, nil
}
return i.Image.LayerByDiffID(h)
}
func validate(adds []Addendum) error {
for _, add := range adds {
if add.Layer == nil {
return errors.New("Unable to add a nil layer to the image")
}
}
return nil
}
+154
View File
@@ -0,0 +1,154 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"io"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/v1util"
)
// CompressedLayer represents the bare minimum interface a natively
// compressed layer must implement for us to produce a v1.Layer
type CompressedLayer interface {
// Digest returns the Hash of the compressed layer.
Digest() (v1.Hash, error)
// Compressed returns an io.ReadCloser for the compressed layer contents.
Compressed() (io.ReadCloser, error)
// Size returns the compressed size of the Layer.
Size() (int64, error)
}
// compressedLayerExtender implements v1.Image using the compressed base properties.
type compressedLayerExtender struct {
CompressedLayer
}
// Uncompressed implements v1.Layer
func (ule *compressedLayerExtender) Uncompressed() (io.ReadCloser, error) {
u, err := ule.Compressed()
if err != nil {
return nil, err
}
return v1util.GunzipReadCloser(u)
}
// DiffID implements v1.Layer
func (ule *compressedLayerExtender) DiffID() (v1.Hash, error) {
r, err := ule.Uncompressed()
if err != nil {
return v1.Hash{}, err
}
defer r.Close()
h, _, err := v1.SHA256(r)
return h, err
}
// CompressedToLayer fills in the missing methods from a CompressedLayer so that it implements v1.Layer
func CompressedToLayer(ul CompressedLayer) (v1.Layer, error) {
return &compressedLayerExtender{ul}, nil
}
// CompressedImageCore represents the base minimum interface a natively
// compressed image must implement for us to produce a v1.Image.
type CompressedImageCore interface {
imageCore
// RawManifest returns the serialized bytes of the manifest.
RawManifest() ([]byte, error)
// LayerByDigest is a variation on the v1.Image method, which returns
// a CompressedLayer instead.
LayerByDigest(v1.Hash) (CompressedLayer, error)
}
// compressedImageExtender implements v1.Image by extending CompressedImageCore with the
// appropriate methods computed from the minimal core.
type compressedImageExtender struct {
CompressedImageCore
}
// Assert that our extender type completes the v1.Image interface
var _ v1.Image = (*compressedImageExtender)(nil)
// BlobSet implements v1.Image
func (i *compressedImageExtender) BlobSet() (map[v1.Hash]struct{}, error) {
return BlobSet(i)
}
// Digest implements v1.Image
func (i *compressedImageExtender) Digest() (v1.Hash, error) {
return Digest(i)
}
// ConfigName implements v1.Image
func (i *compressedImageExtender) ConfigName() (v1.Hash, error) {
return ConfigName(i)
}
// Layers implements v1.Image
func (i *compressedImageExtender) Layers() ([]v1.Layer, error) {
hs, err := FSLayers(i)
if err != nil {
return nil, err
}
ls := make([]v1.Layer, 0, len(hs))
for _, h := range hs {
l, err := i.LayerByDigest(h)
if err != nil {
return nil, err
}
ls = append(ls, l)
}
return ls, nil
}
// LayerByDigest implements v1.Image
func (i *compressedImageExtender) LayerByDigest(h v1.Hash) (v1.Layer, error) {
cl, err := i.CompressedImageCore.LayerByDigest(h)
if err != nil {
return nil, err
}
return CompressedToLayer(cl)
}
// LayerByDiffID implements v1.Image
func (i *compressedImageExtender) LayerByDiffID(h v1.Hash) (v1.Layer, error) {
h, err := DiffIDToBlob(i, h)
if err != nil {
return nil, err
}
return i.LayerByDigest(h)
}
// ConfigFile implements v1.Image
func (i *compressedImageExtender) ConfigFile() (*v1.ConfigFile, error) {
return ConfigFile(i)
}
// Manifest implements v1.Image
func (i *compressedImageExtender) Manifest() (*v1.Manifest, error) {
return Manifest(i)
}
// CompressedToImage fills in the missing methods from a CompressedImageCore so that it implements v1.Image
func CompressedToImage(cic CompressedImageCore) (v1.Image, error) {
return &compressedImageExtender{
CompressedImageCore: cic,
}, nil
}
+17
View File
@@ -0,0 +1,17 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package partial defines methods for building up a v1.Image from
// minimal subsets that are sufficient for defining a v1.Image.
package partial
+28
View File
@@ -0,0 +1,28 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"github.com/google/go-containerregistry/v1/types"
)
// imageCore is the core set of properties without which we cannot build a v1.Image
type imageCore interface {
// RawConfigFile returns the serialized bytes of this image's config file.
RawConfigFile() ([]byte, error)
// MediaType of this image's manifest.
MediaType() (types.MediaType, error)
}
@@ -0,0 +1,229 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"bytes"
"io"
"sync"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/types"
"github.com/google/go-containerregistry/v1/v1util"
)
// UncompressedLayer represents the bare minimum interface a natively
// uncompressed layer must implement for us to produce a v1.Layer
type UncompressedLayer interface {
// DiffID returns the Hash of the uncompressed layer.
DiffID() (v1.Hash, error)
// Uncompressed returns an io.ReadCloser for the uncompressed layer contents.
Uncompressed() (io.ReadCloser, error)
}
// uncompressedLayerExtender implements v1.Image using the uncompressed base properties.
type uncompressedLayerExtender struct {
UncompressedLayer
// TODO(mattmoor): Memoize size/hash so that the methods aren't twice as
// expensive as doing this manually.
}
// Compressed implements v1.Layer
func (ule *uncompressedLayerExtender) Compressed() (io.ReadCloser, error) {
u, err := ule.Uncompressed()
if err != nil {
return nil, err
}
return v1util.GzipReadCloser(u)
}
// Digest implements v1.Layer
func (ule *uncompressedLayerExtender) Digest() (v1.Hash, error) {
r, err := ule.Compressed()
if err != nil {
return v1.Hash{}, err
}
defer r.Close()
h, _, err := v1.SHA256(r)
return h, err
}
// Size implements v1.Layer
func (ule *uncompressedLayerExtender) Size() (int64, error) {
r, err := ule.Compressed()
if err != nil {
return -1, err
}
defer r.Close()
_, i, err := v1.SHA256(r)
return i, err
}
// UncompressedToLayer fills in the missing methods from an UncompressedLayer so that it implements v1.Layer
func UncompressedToLayer(ul UncompressedLayer) (v1.Layer, error) {
return &uncompressedLayerExtender{ul}, nil
}
// UncompressedImageCore represents the bare minimum interface a natively
// uncompressed image must implement for us to produce a v1.Image
type UncompressedImageCore interface {
imageCore
// LayerByDiffID is a variation on the v1.Image method, which returns
// an UncompressedLayer instead.
LayerByDiffID(v1.Hash) (UncompressedLayer, error)
}
// UncompressedToImage fills in the missing methods from an UncompressedImageCore so that it implements v1.Image.
func UncompressedToImage(uic UncompressedImageCore) (v1.Image, error) {
return &uncompressedImageExtender{
UncompressedImageCore: uic,
}, nil
}
// uncompressedImageExtender implements v1.Image by extending UncompressedImageCore with the
// appropriate methods computed from the minimal core.
type uncompressedImageExtender struct {
UncompressedImageCore
lock sync.Mutex
manifest *v1.Manifest
}
// Assert that our extender type completes the v1.Image interface
var _ v1.Image = (*uncompressedImageExtender)(nil)
// BlobSet implements v1.Image
func (i *uncompressedImageExtender) BlobSet() (map[v1.Hash]struct{}, error) {
return BlobSet(i)
}
// Digest implements v1.Image
func (i *uncompressedImageExtender) Digest() (v1.Hash, error) {
return Digest(i)
}
// Manifest implements v1.Image
func (i *uncompressedImageExtender) Manifest() (*v1.Manifest, error) {
i.lock.Lock()
defer i.lock.Unlock()
if i.manifest != nil {
return i.manifest, nil
}
b, err := i.RawConfigFile()
if err != nil {
return nil, err
}
cfgHash, cfgSize, err := v1.SHA256(bytes.NewReader(b))
if err != nil {
return nil, err
}
m := &v1.Manifest{
SchemaVersion: 2,
MediaType: types.DockerManifestSchema2,
Config: v1.Descriptor{
MediaType: types.DockerConfigJSON,
Size: cfgSize,
Digest: cfgHash,
},
}
ls, err := i.Layers()
if err != nil {
return nil, err
}
m.Layers = make([]v1.Descriptor, len(ls))
for i, l := range ls {
sz, err := l.Size()
if err != nil {
return nil, err
}
h, err := l.Digest()
if err != nil {
return nil, err
}
m.Layers[i] = v1.Descriptor{
MediaType: types.DockerLayer,
Size: sz,
Digest: h,
}
}
i.manifest = m
return i.manifest, nil
}
// RawManifest implements v1.Image
func (i *uncompressedImageExtender) RawManifest() ([]byte, error) {
return RawManifest(i)
}
// ConfigName implements v1.Image
func (i *uncompressedImageExtender) ConfigName() (v1.Hash, error) {
return ConfigName(i)
}
// ConfigFile implements v1.Image
func (i *uncompressedImageExtender) ConfigFile() (*v1.ConfigFile, error) {
return ConfigFile(i)
}
// Layers implements v1.Image
func (i *uncompressedImageExtender) Layers() ([]v1.Layer, error) {
diffIDs, err := DiffIDs(i)
if err != nil {
return nil, err
}
ls := make([]v1.Layer, 0, len(diffIDs))
for _, h := range diffIDs {
l, err := i.LayerByDiffID(h)
if err != nil {
return nil, err
}
ls = append(ls, l)
}
return ls, nil
}
// LayerByDiffID implements v1.Image
func (i *uncompressedImageExtender) LayerByDiffID(diffID v1.Hash) (v1.Layer, error) {
ul, err := i.UncompressedImageCore.LayerByDiffID(diffID)
if err != nil {
return nil, err
}
return UncompressedToLayer(ul)
}
// LayerByDigest implements v1.Image
func (i *uncompressedImageExtender) LayerByDigest(h v1.Hash) (v1.Layer, error) {
// Support returning the ConfigFile when asked for its hash.
if cfgName, err := i.ConfigName(); err != nil {
return nil, err
} else if cfgName == h {
return ConfigLayer(i)
}
diffID, err := BlobToDiffID(i, h)
if err != nil {
return nil, err
}
return i.LayerByDiffID(diffID)
}
+291
View File
@@ -0,0 +1,291 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"bytes"
"encoding/json"
"fmt"
"io"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/v1util"
)
// WithRawConfigFile defines the subset of v1.Image used by these helper methods
type WithRawConfigFile interface {
// RawConfigFile returns the serialized bytes of this image's config file.
RawConfigFile() ([]byte, error)
}
// ConfigFile is a helper for implementing v1.Image
func ConfigFile(i WithRawConfigFile) (*v1.ConfigFile, error) {
b, err := i.RawConfigFile()
if err != nil {
return nil, err
}
return v1.ParseConfigFile(bytes.NewReader(b))
}
// ConfigName is a helper for implementing v1.Image
func ConfigName(i WithRawConfigFile) (v1.Hash, error) {
b, err := i.RawConfigFile()
if err != nil {
return v1.Hash{}, err
}
h, _, err := v1.SHA256(bytes.NewReader(b))
return h, err
}
// configLayer implements v1.Layer from the raw config bytes.
// This is so that clients (e.g. remote) can access the config as a blob.
type configLayer struct {
hash v1.Hash
content []byte
}
// Digest implements v1.Layer
func (cl *configLayer) Digest() (v1.Hash, error) {
return cl.hash, nil
}
// DiffID implements v1.Layer
func (cl *configLayer) DiffID() (v1.Hash, error) {
return cl.hash, nil
}
// Uncompressed implements v1.Layer
func (cl *configLayer) Uncompressed() (io.ReadCloser, error) {
return v1util.NopReadCloser(bytes.NewBuffer(cl.content)), nil
}
// Compressed implements v1.Layer
func (cl *configLayer) Compressed() (io.ReadCloser, error) {
return v1util.NopReadCloser(bytes.NewBuffer(cl.content)), nil
}
// Size implements v1.Layer
func (cl *configLayer) Size() (int64, error) {
return int64(len(cl.content)), nil
}
var _ v1.Layer = (*configLayer)(nil)
func ConfigLayer(i WithRawConfigFile) (v1.Layer, error) {
h, err := ConfigName(i)
if err != nil {
return nil, err
}
rcfg, err := i.RawConfigFile()
if err != nil {
return nil, err
}
return &configLayer{
hash: h,
content: rcfg,
}, nil
}
// WithConfigFile defines the subset of v1.Image used by these helper methods
type WithConfigFile interface {
// ConfigFile returns this image's config file.
ConfigFile() (*v1.ConfigFile, error)
}
// DiffIDs is a helper for implementing v1.Image
func DiffIDs(i WithConfigFile) ([]v1.Hash, error) {
cfg, err := i.ConfigFile()
if err != nil {
return nil, err
}
dids := make([]v1.Hash, len(cfg.RootFS.DiffIDs))
for i, did := range cfg.RootFS.DiffIDs {
dids[len(dids)-i-1] = did
}
return dids, nil
}
// RawConfigFile is a helper for implementing v1.Image
func RawConfigFile(i WithConfigFile) ([]byte, error) {
cfg, err := i.ConfigFile()
if err != nil {
return nil, err
}
return json.Marshal(cfg)
}
// WithUncompressedLayer defines the subset of v1.Image used by these helper methods
type WithUncompressedLayer interface {
// UncompressedLayer is like UncompressedBlob, but takes the "diff id".
UncompressedLayer(v1.Hash) (io.ReadCloser, error)
}
// Layer is the same as Blob, but takes the "diff id".
func Layer(wul WithUncompressedLayer, h v1.Hash) (io.ReadCloser, error) {
rc, err := wul.UncompressedLayer(h)
if err != nil {
return nil, err
}
return v1util.GzipReadCloser(rc)
}
// WithRawManifest defines the subset of v1.Image used by these helper methods
type WithRawManifest interface {
// RawManifest returns the serialized bytes of this image's config file.
RawManifest() ([]byte, error)
}
// Digest is a helper for implementing v1.Image
func Digest(i WithRawManifest) (v1.Hash, error) {
mb, err := i.RawManifest()
if err != nil {
return v1.Hash{}, err
}
digest, _, err := v1.SHA256(bytes.NewReader(mb))
return digest, err
}
// Manifest is a helper for implementing v1.Image
func Manifest(i WithRawManifest) (*v1.Manifest, error) {
b, err := i.RawManifest()
if err != nil {
return nil, err
}
return v1.ParseManifest(bytes.NewReader(b))
}
// WithManifest defines the subset of v1.Image used by these helper methods
type WithManifest interface {
// Manifest returns this image's Manifest object.
Manifest() (*v1.Manifest, error)
}
// RawManifest is a helper for implementing v1.Image
func RawManifest(i WithManifest) ([]byte, error) {
m, err := i.Manifest()
if err != nil {
return nil, err
}
return json.Marshal(m)
}
// FSLayers is a helper for implementing v1.Image
func FSLayers(i WithManifest) ([]v1.Hash, error) {
m, err := i.Manifest()
if err != nil {
return nil, err
}
fsl := make([]v1.Hash, len(m.Layers))
for i, l := range m.Layers {
fsl[i] = l.Digest
}
return fsl, nil
}
// BlobSet is a helper for implementing v1.Image
func BlobSet(i WithManifest) (map[v1.Hash]struct{}, error) {
m, err := i.Manifest()
if err != nil {
return nil, err
}
bs := make(map[v1.Hash]struct{})
for _, l := range m.Layers {
bs[l.Digest] = struct{}{}
}
bs[m.Config.Digest] = struct{}{}
return bs, nil
}
// BlobSize is a helper for implementing v1.Image
func BlobSize(i WithManifest, h v1.Hash) (int64, error) {
m, err := i.Manifest()
if err != nil {
return -1, err
}
for _, l := range m.Layers {
if l.Digest == h {
return l.Size, nil
}
}
return -1, fmt.Errorf("blob %v not found", h)
}
// WithManifestAndConfigFile defines the subset of v1.Image used by these helper methods
type WithManifestAndConfigFile interface {
WithConfigFile
// Manifest returns this image's Manifest object.
Manifest() (*v1.Manifest, error)
}
// BlobToDiffID is a helper for mapping between compressed
// and uncompressed blob hashes.
func BlobToDiffID(i WithManifestAndConfigFile, h v1.Hash) (v1.Hash, error) {
blobs, err := FSLayers(i)
if err != nil {
return v1.Hash{}, err
}
diffIDs, err := DiffIDs(i)
if err != nil {
return v1.Hash{}, err
}
if len(blobs) != len(diffIDs) {
return v1.Hash{}, fmt.Errorf("mismatched fs layers (%d) and diff ids (%d)", len(blobs), len(diffIDs))
}
for i, blob := range blobs {
if blob == h {
return diffIDs[i], nil
}
}
return v1.Hash{}, fmt.Errorf("unknown blob %v", h)
}
// DiffIDToBlob is a helper for mapping between uncompressed
// and compressed blob hashes.
func DiffIDToBlob(wm WithManifestAndConfigFile, h v1.Hash) (v1.Hash, error) {
blobs, err := FSLayers(wm)
if err != nil {
return v1.Hash{}, err
}
diffIDs, err := DiffIDs(wm)
if err != nil {
return v1.Hash{}, err
}
if len(blobs) != len(diffIDs) {
return v1.Hash{}, fmt.Errorf("mismatched fs layers (%d) and diff ids (%d)", len(blobs), len(diffIDs))
}
for i, diffID := range diffIDs {
if diffID == h {
return blobs[i], nil
}
}
return v1.Hash{}, fmt.Errorf("unknown diffID %v", h)
}
// WithBlob defines the subset of v1.Image used by these helper methods
type WithBlob interface {
// Blob returns a ReadCloser for streaming the blob's content.
Blob(v1.Hash) (io.ReadCloser, error)
}
// UncompressedBlob returns a ReadCloser for streaming the blob's content uncompressed.
func UncompressedBlob(b WithBlob, h v1.Hash) (io.ReadCloser, error) {
rc, err := b.Blob(h)
if err != nil {
return nil, err
}
return v1util.GunzipReadCloser(rc)
}
+16
View File
@@ -0,0 +1,16 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package random provides a facility for synthesizing pseudo-random images.
package random
+110
View File
@@ -0,0 +1,110 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package random
import (
"bytes"
"crypto/rand"
"fmt"
"io"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/partial"
"github.com/google/go-containerregistry/v1/types"
"github.com/google/go-containerregistry/v1/v1util"
)
// uncompressedLayer implements partial.UncompressedLayer from raw bytes.
// TODO(mattmoor): Consider moving this into a library.
type uncompressedLayer struct {
diffID v1.Hash
content []byte
}
// DiffID implements partial.UncompressedLayer
func (ul *uncompressedLayer) DiffID() (v1.Hash, error) {
return ul.diffID, nil
}
// Uncompressed implements partial.UncompressedLayer
func (ul *uncompressedLayer) Uncompressed() (io.ReadCloser, error) {
return v1util.NopReadCloser(bytes.NewBuffer(ul.content)), nil
}
var _ partial.UncompressedLayer = (*uncompressedLayer)(nil)
// Image returns a pseudo-randomly generated Image.
func Image(byteSize, layers int64) (v1.Image, error) {
layerz := make(map[v1.Hash]partial.UncompressedLayer)
for i := int64(0); i < layers; i++ {
b := bytes.NewBuffer(nil)
if _, err := io.CopyN(b, rand.Reader, byteSize); err != nil {
return nil, err
}
bts := b.Bytes()
h, _, err := v1.SHA256(bytes.NewReader(bts))
if err != nil {
return nil, err
}
layerz[h] = &uncompressedLayer{
diffID: h,
content: bts,
}
}
cfg := &v1.ConfigFile{}
// It is ok that iteration order is random in Go, because this is the random image anyways.
for k := range layerz {
cfg.RootFS.DiffIDs = append(cfg.RootFS.DiffIDs, k)
}
return partial.UncompressedToImage(&image{
config: cfg,
layers: layerz,
})
}
// image is pseudo-randomly generated.
type image struct {
config *v1.ConfigFile
layers map[v1.Hash]partial.UncompressedLayer
}
var _ partial.UncompressedImageCore = (*image)(nil)
// RawConfigFile implements partial.UncompressedImageCore
func (i *image) RawConfigFile() ([]byte, error) {
return partial.RawConfigFile(i)
}
// ConfigFile implements v1.Image
func (i *image) ConfigFile() (*v1.ConfigFile, error) {
return i.config, nil
}
// MediaType implements partial.UncompressedImageCore
func (i *image) MediaType() (types.MediaType, error) {
return types.DockerManifestSchema2, nil
}
// LayerByDiffID implements partial.UncompressedImageCore
func (i *image) LayerByDiffID(diffID v1.Hash) (partial.UncompressedLayer, error) {
l, ok := i.layers[diffID]
if !ok {
return nil, fmt.Errorf("unknown diff_id: %v", diffID)
}
return l, nil
}
+70
View File
@@ -0,0 +1,70 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"fmt"
"io/ioutil"
"net/http"
"net/url"
"github.com/google/go-containerregistry/authn"
"github.com/google/go-containerregistry/name"
"github.com/google/go-containerregistry/v1/remote/transport"
)
// DeleteOptions are used to expose optional information to guide or
// control the image deletion.
type DeleteOptions struct {
// TODO(mattmoor): Fail on not found?
// TODO(mattmoor): Delete tag and manifest?
}
// Delete removes the specified image reference from the remote registry.
func Delete(ref name.Reference, auth authn.Authenticator, t http.RoundTripper, do DeleteOptions) error {
tr, err := transport.New(ref, auth, t, transport.DeleteScope)
if err != nil {
return err
}
c := &http.Client{Transport: tr}
u := url.URL{
Scheme: transport.Scheme(ref.Context().Registry),
Host: ref.Context().RegistryStr(),
Path: fmt.Sprintf("/v2/%s/manifests/%s", ref.Context().RepositoryStr(), ref.Identifier()),
}
req, err := http.NewRequest(http.MethodDelete, u.String(), nil)
if err != nil {
return err
}
resp, err := c.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
switch resp.StatusCode {
case http.StatusOK, http.StatusAccepted:
return nil
default:
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
return fmt.Errorf("unrecognized status code during DELETE: %v; %v", resp.Status, string(b))
}
}
+17
View File
@@ -0,0 +1,17 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package remote provides facilities for reading/writing v1.Images from/to
// a remote image registry.
package remote
+106
View File
@@ -0,0 +1,106 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strings"
)
// Error implements error to support the following error specification:
// https://github.com/docker/distribution/blob/master/docs/spec/api.md#errors
type Error struct {
Errors []Diagnostic `json:"errors,omitempty"`
}
// Check that Error implements error
var _ error = (*Error)(nil)
// Error implements error
func (e *Error) Error() string {
switch len(e.Errors) {
case 0:
return "<empty remote.Error response>"
case 1:
return e.Errors[0].String()
default:
var errors []string
for _, d := range e.Errors {
errors = append(errors, d.String())
}
return fmt.Sprintf("multiple errors returned: %s",
strings.Join(errors, ";"))
}
}
// Diagnostic represents a single error returned by a Docker registry interaction.
type Diagnostic struct {
Code ErrorCode `json:"code"`
Message string `json:"message,omitempty"`
Detail interface{} `json:"detail,omitempty"`
}
// String stringifies the Diagnostic
func (d Diagnostic) String() string {
return fmt.Sprintf("%s: %q", d.Code, d.Message)
}
// ErrorCode is an enumeration of supported error codes.
type ErrorCode string
// The set of error conditions a registry may return:
// https://github.com/docker/distribution/blob/master/docs/spec/api.md#errors-2
const (
BlobUnknownErrorCode ErrorCode = "BLOB_UNKNOWN"
BlobUploadInvalidErrorCode ErrorCode = "BLOB_UPLOAD_INVALID"
BlobUploadUnknownErrorCode ErrorCode = "BLOB_UPLOAD_UNKNOWN"
DigestInvalidErrorCode ErrorCode = "DIGEST_INVALID"
ManifestBlobUnknownErrorCode ErrorCode = "MANIFEST_BLOB_UNKNOWN"
ManifestInvalidErrorCode ErrorCode = "MANIFEST_INVALID"
ManifestUnknownErrorCode ErrorCode = "MANIFEST_UNKNOWN"
ManifestUnverifiedErrorCode ErrorCode = "MANIFEST_UNVERIFIED"
NameInvalidErrorCode ErrorCode = "NAME_INVALID"
NameUnknownErrorCode ErrorCode = "NAME_UNKNOWN"
SizeInvalidErrorCode ErrorCode = "SIZE_INVALID"
TagInvalidErrorCode ErrorCode = "TAG_INVALID"
UnauthorizedErrorCode ErrorCode = "UNAUTHORIZED"
DeniedErrorCode ErrorCode = "DENIED"
UnsupportedErrorCode ErrorCode = "UNSUPPORTED"
)
func checkError(resp *http.Response, codes ...int) error {
for _, code := range codes {
if resp.StatusCode == code {
// This is one of the supported status codes.
return nil
}
}
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
// https://github.com/docker/distribution/blob/master/docs/spec/api.md#errors
var structuredError Error
if err := json.Unmarshal(b, &structuredError); err != nil {
// If the response isn't an unstructured error, then return some
// reasonable error response containing the response body.
return fmt.Errorf("unsupported status code %d; body: %s", resp.StatusCode, string(b))
}
return &structuredError
}
+201
View File
@@ -0,0 +1,201 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"net/url"
"sync"
"github.com/google/go-containerregistry/authn"
"github.com/google/go-containerregistry/name"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/partial"
"github.com/google/go-containerregistry/v1/remote/transport"
"github.com/google/go-containerregistry/v1/types"
"github.com/google/go-containerregistry/v1/v1util"
)
// remoteImage accesses an image from a remote registry
type remoteImage struct {
ref name.Reference
client *http.Client
manifestLock sync.Mutex // Protects manifest
manifest []byte
configLock sync.Mutex // Protects config
config []byte
}
var _ partial.CompressedImageCore = (*remoteImage)(nil)
// Image accesses a given image reference over the provided transport, with the provided authentication.
func Image(ref name.Reference, auth authn.Authenticator, t http.RoundTripper) (v1.Image, error) {
tr, err := transport.New(ref, auth, t, transport.PullScope)
if err != nil {
return nil, err
}
return partial.CompressedToImage(&remoteImage{
ref: ref,
client: &http.Client{Transport: tr},
})
}
func (r *remoteImage) url(resource, identifier string) url.URL {
return url.URL{
Scheme: transport.Scheme(r.ref.Context().Registry),
Host: r.ref.Context().RegistryStr(),
Path: fmt.Sprintf("/v2/%s/%s/%s", r.ref.Context().RepositoryStr(), resource, identifier),
}
}
func (r *remoteImage) MediaType() (types.MediaType, error) {
// TODO(jonjohnsonjr): Determine this based on response.
return types.DockerManifestSchema2, nil
}
// TODO(jonjohnsonjr): Handle manifest lists.
func (r *remoteImage) RawManifest() ([]byte, error) {
r.manifestLock.Lock()
defer r.manifestLock.Unlock()
if r.manifest != nil {
return r.manifest, nil
}
u := r.url("manifests", r.ref.Identifier())
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
// TODO(jonjohnsonjr): Accept OCI manifest, manifest list, and image index.
req.Header.Set("Accept", string(types.DockerManifestSchema2))
resp, err := r.client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if err := checkError(resp, http.StatusOK); err != nil {
return nil, err
}
manifest, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
digest, _, err := v1.SHA256(bytes.NewReader(manifest))
if err != nil {
return nil, err
}
// Validate the digest matches what we asked for, if pulling by digest.
if dgst, ok := r.ref.(name.Digest); ok {
if digest.String() != dgst.DigestStr() {
return nil, fmt.Errorf("manifest digest: %q does not match requested digest: %q for %q", digest, dgst.DigestStr(), r.ref)
}
} else if checksum := resp.Header.Get("Docker-Content-Digest"); checksum != "" && checksum != digest.String() {
err := fmt.Errorf("manifest digest: %q does not match Docker-Content-Digest: %q for %q", digest, checksum, r.ref)
if r.ref.Context().RegistryStr() == name.DefaultRegistry {
// TODO(docker/distribution#2395): Remove this check.
log.Println(err)
} else {
// When pulling by tag, we can only validate that the digest matches what the registry told us it should be.
return nil, err
}
}
r.manifest = manifest
return r.manifest, nil
}
func (r *remoteImage) RawConfigFile() ([]byte, error) {
r.configLock.Lock()
defer r.configLock.Unlock()
if r.config != nil {
return r.config, nil
}
m, err := partial.Manifest(r)
if err != nil {
return nil, err
}
cl, err := r.LayerByDigest(m.Config.Digest)
if err != nil {
return nil, err
}
body, err := cl.Compressed()
if err != nil {
return nil, err
}
defer body.Close()
r.config, err = ioutil.ReadAll(body)
if err != nil {
return nil, err
}
return r.config, nil
}
// remoteLayer implements partial.CompressedLayer
type remoteLayer struct {
ri *remoteImage
digest v1.Hash
}
// Digest implements partial.CompressedLayer
func (rl *remoteLayer) Digest() (v1.Hash, error) {
return rl.digest, nil
}
// Compressed implements partial.CompressedLayer
func (rl *remoteLayer) Compressed() (io.ReadCloser, error) {
u := rl.ri.url("blobs", rl.digest.String())
resp, err := rl.ri.client.Get(u.String())
if err != nil {
return nil, err
}
if err := checkError(resp, http.StatusOK); err != nil {
resp.Body.Close()
return nil, err
}
return v1util.VerifyReadCloser(resp.Body, rl.digest)
}
// Manifest implements partial.WithManifest so that we can use partial.BlobSize below.
func (rl *remoteLayer) Manifest() (*v1.Manifest, error) {
return partial.Manifest(rl.ri)
}
// Size implements partial.CompressedLayer
func (rl *remoteLayer) Size() (int64, error) {
// Look up the size of this digest in the manifest to avoid a request.
return partial.BlobSize(rl, rl.digest)
}
// LayerByDigest implements partial.CompressedLayer
func (r *remoteImage) LayerByDigest(h v1.Hash) (partial.CompressedLayer, error) {
return &remoteLayer{
ri: r,
digest: h,
}, nil
}
@@ -0,0 +1,47 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
import (
"net/http"
"github.com/google/go-containerregistry/authn"
)
type basicTransport struct {
inner http.RoundTripper
auth authn.Authenticator
target string
}
var _ http.RoundTripper = (*basicTransport)(nil)
// RoundTrip implements http.RoundTripper
func (bt *basicTransport) RoundTrip(in *http.Request) (*http.Response, error) {
hdr, err := bt.auth.Authorization()
if err != nil {
return nil, err
}
// http.Client handles redirects at a layer above the http.RoundTripper
// abstraction, so to avoid forwarding Authorization headers to places
// we are redirected, only set it when the authorization header matches
// the host with which we are interacting.
if in.Host == bt.target {
in.Header.Set("Authorization", hdr)
}
in.Header.Set("User-Agent", transportName)
return bt.inner.RoundTrip(in)
}
@@ -0,0 +1,119 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
import (
"fmt"
"encoding/json"
"io/ioutil"
"net/http"
"net/url"
"github.com/google/go-containerregistry/authn"
"github.com/google/go-containerregistry/name"
)
type bearerTransport struct {
// Wrapped by bearerTransport.
inner http.RoundTripper
// Basic credentials that we exchange for bearer tokens.
basic authn.Authenticator
// Holds the bearer response from the token service.
bearer *authn.Bearer
// Registry to which we send bearer tokens.
registry name.Registry
// See https://tools.ietf.org/html/rfc6750#section-3
realm string
// See https://docs.docker.com/registry/spec/auth/token/
service string
scope string
}
var _ http.RoundTripper = (*bearerTransport)(nil)
// RoundTrip implements http.RoundTripper
func (bt *bearerTransport) RoundTrip(in *http.Request) (*http.Response, error) {
hdr, err := bt.bearer.Authorization()
if err != nil {
return nil, err
}
// http.Client handles redirects at a layer above the http.RoundTripper
// abstraction, so to avoid forwarding Authorization headers to places
// we are redirected, only set it when the authorization header matches
// the registry with which we are interacting.
if in.Host == bt.registry.RegistryStr() {
in.Header.Set("Authorization", hdr)
}
in.Header.Set("User-Agent", transportName)
// TODO(mattmoor): On 401s perform a single refresh() and retry.
return bt.inner.RoundTrip(in)
}
func (bt *bearerTransport) refresh() error {
u, err := url.Parse(bt.realm)
if err != nil {
return err
}
b := &basicTransport{
inner: bt.inner,
auth: bt.basic,
target: u.Host,
}
client := http.Client{Transport: b}
u.RawQuery = url.Values{
"scope": []string{bt.scope},
"service": []string{bt.service},
}.Encode()
resp, err := client.Get(u.String())
if err != nil {
return err
}
defer resp.Body.Close()
content, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
// Some registries don't have "token" in the response. See #54.
type tokenResponse struct {
Token string `json:"token"`
AccessToken string `json:"access_token"`
}
var response tokenResponse
if err := json.Unmarshal(content, &response); err != nil {
return err
}
// Find a token to turn into a Bearer authenticator
var bearer authn.Bearer
if response.Token != "" {
bearer = authn.Bearer{Token: response.Token}
} else if response.AccessToken != "" {
bearer = authn.Bearer{Token: response.AccessToken}
} else {
return fmt.Errorf("no token in bearer response:\n%s", content)
}
// Replace our old bearer authenticator (if we had one) with our newly refreshed authenticator.
bt.bearer = &bearer
return nil
}
@@ -0,0 +1,18 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package transport provides facilities for setting up an authenticated
// http.RoundTripper given an Authenticator and base RoundTripper. See
// transport.New for more information.
package transport
@@ -0,0 +1,89 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
import (
"fmt"
"net/http"
"strings"
"github.com/google/go-containerregistry/name"
)
type challenge string
const (
anonymous challenge = "Anonymous"
basic challenge = "Basic"
bearer challenge = "Bearer"
)
type pingResp struct {
challenge challenge
// Following the challenge there are often key/value pairs
// e.g. Bearer service="gcr.io",realm="https://auth.gcr.io/v36/tokenz"
parameters map[string]string
}
func parseChallenge(suffix string) map[string]string {
kv := make(map[string]string)
for _, token := range strings.Split(suffix, ",") {
// Trim any whitespace around each token.
token = strings.Trim(token, " ")
// Break the token into a key/value pair
if parts := strings.SplitN(token, "=", 2); len(parts) == 2 {
// Unquote the value, if it is quoted.
kv[parts[0]] = strings.Trim(parts[1], `"`)
} else {
// If there was only one part, treat is as a key with an empty value
kv[token] = ""
}
}
return kv
}
func ping(reg name.Registry, t http.RoundTripper) (*pingResp, error) {
client := http.Client{Transport: t}
url := fmt.Sprintf("%s://%s/v2/", Scheme(reg), reg.Name())
resp, err := client.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
switch resp.StatusCode {
case http.StatusOK:
// If we get a 200, then no authentication is needed.
return &pingResp{challenge: anonymous}, nil
case http.StatusUnauthorized:
wac := resp.Header.Get(http.CanonicalHeaderKey("WWW-Authenticate"))
if parts := strings.SplitN(wac, " ", 2); len(parts) == 2 {
// If there are two parts, then parse the challenge parameters.
return &pingResp{
challenge: challenge(strings.Title(parts[0])),
parameters: parseChallenge(parts[1]),
}, nil
}
// Otherwise, just return the challenge without parameters.
return &pingResp{
challenge: challenge(strings.Title(wac)),
}, nil
default:
return nil, fmt.Errorf("unrecognized HTTP status: %v", resp.Status)
}
}
@@ -0,0 +1,42 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
import (
"regexp"
"strings"
"github.com/google/go-containerregistry/name"
)
// Detect more complex forms of local references.
var reLocal = regexp.MustCompile(`.*\.local(?:host)?(?::\d{1,5})?$`)
// Detect the loopback IP (127.0.0.1)
var reLoopback = regexp.MustCompile(regexp.QuoteMeta("127.0.0.1"))
// Scheme returns https scheme for all the endpoints except localhost.
func Scheme(reg name.Registry) string {
if strings.HasPrefix(reg.Name(), "localhost:") {
return "http"
}
if reLocal.MatchString(reg.Name()) {
return "http"
}
if reLoopback.MatchString(reg.Name()) {
return "http"
}
return "https"
}
@@ -0,0 +1,27 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
// Scope is an enumeration of the supported scopes to pass a transport.
type Scope string
// Scopes suitable to pass to New()
const (
PullScope Scope = "pull"
PushScope Scope = "push,pull"
// For now DELETE is PUSH, which is the read/write ACL.
DeleteScope Scope = PushScope
CatalogScope Scope = "catalog"
)
@@ -0,0 +1,84 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
import (
"fmt"
"net/http"
"github.com/google/go-containerregistry/authn"
"github.com/google/go-containerregistry/name"
)
const (
transportName = "go-containerregistry"
)
// New returns a new RoundTripper based on the provided RoundTripper that has been
// setup to authenticate with the remote registry hosting "ref", in the capacity
// laid out by the specified Scope.
func New(ref name.Reference, auth authn.Authenticator, t http.RoundTripper, a Scope) (http.RoundTripper, error) {
// The handshake:
// 1. Use "t" to ping() the registry for the authentication challenge.
//
// 2a. If we get back a 200, then simply use "t".
//
// 2b. If we get back a 401 with a Basic challenge, then use a transport
// that just attachs auth each roundtrip.
//
// 2c. If we get back a 401 with a Bearer challenge, then use a transport
// that attaches a bearer token to each request, and refreshes is on 401s.
// Perform an initial refresh to seed the bearer token.
// First we ping the registry to determine the parameters of the authentication handshake
// (if one is even necessary).
pr, err := ping(ref.Context().Registry, t)
if err != nil {
return nil, err
}
switch pr.challenge {
case anonymous:
return t, nil
case basic:
return &basicTransport{inner: t, auth: auth, target: ref.Context().RegistryStr()}, nil
case bearer:
// We require the realm, which tells us where to send our Basic auth to turn it into Bearer auth.
realm, ok := pr.parameters["realm"]
if !ok {
return nil, fmt.Errorf("malformed www-authenticate, missing realm: %v", pr.parameters)
}
service, ok := pr.parameters["service"]
if !ok {
// If the service parameter is not specified, then default it to the registry
// with which we are talking.
service = ref.Context().Registry.String()
}
bt := &bearerTransport{
inner: t,
basic: auth,
realm: realm,
registry: ref.Context().Registry,
service: service,
scope: ref.Scope(string(a)),
}
if err := bt.refresh(); err != nil {
return nil, err
}
return bt, nil
default:
return nil, fmt.Errorf("Unrecognized challenge: %s", pr.challenge)
}
}
+281
View File
@@ -0,0 +1,281 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"errors"
"fmt"
"log"
"net/http"
"net/url"
"github.com/google/go-containerregistry/authn"
"github.com/google/go-containerregistry/name"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/remote/transport"
)
// WriteOptions are used to expose optional information to guide or
// control the image write.
type WriteOptions struct {
// The set of paths from which to attempt to mount blobs.
MountPaths []name.Repository
// TODO(mattmoor): Expose "threads" to limit parallelism?
}
// Write pushes the provided img to the specified image reference.
func Write(ref name.Reference, img v1.Image, auth authn.Authenticator, t http.RoundTripper,
wo WriteOptions) error {
tr, err := transport.New(ref, auth, t, transport.PushScope)
if err != nil {
return err
}
w := writer{
ref: ref,
client: &http.Client{Transport: tr},
img: img,
options: wo,
}
bs, err := img.BlobSet()
if err != nil {
return err
}
// Spin up go routines to publish each of the members of BlobSet(),
// and use an error channel to collect their results.
errCh := make(chan error)
defer close(errCh)
for h := range bs {
go func(h v1.Hash) {
errCh <- w.uploadOne(h)
}(h)
}
// Now wait for all of the blob uploads to complete.
var errors []error
for _ = range bs {
if err := <-errCh; err != nil {
errors = append(errors, err)
}
}
if len(errors) > 0 {
// Return the first error we encountered.
return errors[0]
}
// With all of the constituent elements uploaded, upload the manifest
// to commit the image.
return w.commitImage()
}
// writer writes the elements of an image to a remote image reference.
type writer struct {
ref name.Reference
client *http.Client
img v1.Image
options WriteOptions
}
// url returns a url.Url for the specified path in the context of this remote image reference.
func (w *writer) url(path string) url.URL {
return url.URL{
Scheme: transport.Scheme(w.ref.Context().Registry),
Host: w.ref.Context().RegistryStr(),
Path: path,
}
}
// nextLocation extracts the fully-qualified URL to which we should send the next request in an upload sequence.
func (w *writer) nextLocation(resp *http.Response) (string, error) {
loc := resp.Header.Get("Location")
if len(loc) == 0 {
return "", errors.New("missing Location header")
}
u, err := url.Parse(loc)
if err != nil {
return "", err
}
// If the location header returned is just a url path, then fully qualify it.
// We cannot simply call w.url, since there might be an embedded query string.
return resp.Request.URL.ResolveReference(u).String(), nil
}
// initiateUpload initiates the blob upload, which starts with a POST that can
// optionally include the hash of the layer and a list of repositories from
// which that layer might be read. On failure, an error is returned.
// On success, the layer was either mounted (nothing more to do) or a blob
// upload was initiated and the body of that blob should be sent to the returned
// location.
func (w *writer) initiateUpload(h v1.Hash) (location string, mounted bool, err error) {
u := w.url(fmt.Sprintf("/v2/%s/blobs/uploads/", w.ref.Context().RepositoryStr()))
uv := url.Values{
"mount": []string{h.String()},
}
var from []string
for _, m := range w.options.MountPaths {
from = append(from, m.RepositoryStr())
}
// We currently avoid HEAD because it's semi-redundant with the mount that is part
// of initiating the blob upload. GCR will perform an existence check on the initiation
// if "mount" is specified, even if no "from" sources are specified. If this turns out
// to not be broadly applicable then we should replace mounts without "from"s with a HEAD.
if len(from) > 0 {
uv["from"] = from
}
u.RawQuery = uv.Encode()
// Make the request to initiate the blob upload.
resp, err := w.client.Post(u.String(), "application/json", nil)
if err != nil {
return "", false, err
}
defer resp.Body.Close()
if err := checkError(resp, http.StatusCreated, http.StatusAccepted); err != nil {
return "", false, err
}
// Check the response code to determine the result.
switch resp.StatusCode {
case http.StatusCreated:
// We're done, we were able to fast-path.
return "", true, nil
case http.StatusAccepted:
// Proceed to PATCH, upload has begun.
loc, err := w.nextLocation(resp)
return loc, false, err
default:
panic("Unreachable: initiateUpload")
}
}
// streamBlob streams the contents of the blob to the specified location.
// On failure, this will return an error. On success, this will return the location
// header indicating how to commit the streamed blob.
func (w *writer) streamBlob(h v1.Hash, streamLocation string) (commitLocation string, err error) {
l, err := w.img.LayerByDigest(h)
if err != nil {
return "", err
}
blob, err := l.Compressed()
if err != nil {
return "", err
}
defer blob.Close()
req, err := http.NewRequest(http.MethodPatch, streamLocation, blob)
if err != nil {
return "", err
}
resp, err := w.client.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if err := checkError(resp, http.StatusNoContent, http.StatusAccepted, http.StatusCreated); err != nil {
return "", err
}
// The blob has been uploaded, return the location header indicating
// how to commit this layer.
return w.nextLocation(resp)
}
// commitBlob commits this blob by sending a PUT to the location returned from streaming the blob.
func (w *writer) commitBlob(h v1.Hash, location string) (err error) {
u, err := url.Parse(location)
if err != nil {
return err
}
v := u.Query()
v.Set("digest", h.String())
u.RawQuery = v.Encode()
req, err := http.NewRequest(http.MethodPut, u.String(), nil)
if err != nil {
return err
}
resp, err := w.client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
return checkError(resp, http.StatusCreated)
}
// uploadOne performs a complete upload of a single layer.
func (w *writer) uploadOne(h v1.Hash) error {
location, mounted, err := w.initiateUpload(h)
if err != nil {
return err
} else if mounted {
log.Printf("mounted %v", h)
return nil
}
location, err = w.streamBlob(h, location)
if err != nil {
return err
}
if err := w.commitBlob(h, location); err != nil {
return err
}
log.Printf("pushed %v", h)
return nil
}
// commitImage does a PUT of the image's manifest.
func (w *writer) commitImage() error {
raw, err := w.img.RawManifest()
if err != nil {
return err
}
mt, err := w.img.MediaType()
if err != nil {
return err
}
u := w.url(fmt.Sprintf("/v2/%s/manifests/%s", w.ref.Context().RepositoryStr(), w.ref.Identifier()))
// Make the request to PUT the serialized manifest
req, err := http.NewRequest(http.MethodPut, u.String(), bytes.NewBuffer(raw))
if err != nil {
return err
}
req.Header.Set("Content-Type", string(mt))
resp, err := w.client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if err := checkError(resp, http.StatusOK, http.StatusCreated, http.StatusAccepted); err != nil {
return err
}
// The image was successfully pushed!
log.Printf("pushed %v", w.ref)
return nil
}
// TODO(mattmoor): WriteIndex
+17
View File
@@ -0,0 +1,17 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package tarball provides facilities for reading/writing v1.Images from/to
// a tarball on-disk.
package tarball
+338
View File
@@ -0,0 +1,338 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package tarball
import (
"archive/tar"
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"sync"
"github.com/google/go-containerregistry/name"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/partial"
"github.com/google/go-containerregistry/v1/types"
"github.com/google/go-containerregistry/v1/v1util"
)
type image struct {
opener Opener
td *tarDescriptor
config []byte
imgDescriptor *singleImageTarDescriptor
tag *name.Tag
}
type uncompressedImage struct {
*image
}
type compressedImage struct {
*image
manifestLock sync.Mutex // Protects manifest
manifest *v1.Manifest
}
var _ partial.UncompressedImageCore = (*uncompressedImage)(nil)
var _ partial.CompressedImageCore = (*compressedImage)(nil)
type Opener func() (io.ReadCloser, error)
func pathOpener(path string) Opener {
return func() (io.ReadCloser, error) {
return os.Open(path)
}
}
func ImageFromPath(path string, tag *name.Tag) (v1.Image, error) {
return Image(pathOpener(path), tag)
}
// Image exposes an image from the tarball at the provided path.
func Image(opener Opener, tag *name.Tag) (v1.Image, error) {
img := &image{
opener: opener,
tag: tag,
}
if err := img.loadTarDescriptorAndConfig(); err != nil {
return nil, err
}
// Peek at the first layer and see if it's compressed.
compressed, err := img.areLayersCompressed()
if err != nil {
return nil, err
}
if compressed {
c := compressedImage{
image: img,
}
return partial.CompressedToImage(&c)
}
uc := uncompressedImage{
image: img,
}
return partial.UncompressedToImage(&uc)
}
func (i *image) MediaType() (types.MediaType, error) {
return types.DockerManifestSchema2, nil
}
// singleImageTarDescriptor is the struct used to represent a single image inside a `docker save` tarball.
type singleImageTarDescriptor struct {
Config string
RepoTags []string
Layers []string
}
// tarDescriptor is the struct used inside the `manifest.json` file of a `docker save` tarball.
type tarDescriptor []singleImageTarDescriptor
func (td tarDescriptor) findSpecifiedImageDescriptor(tag *name.Tag) (*singleImageTarDescriptor, error) {
if tag == nil {
if len(td) != 1 {
return nil, errors.New("tarball must contain only a single image to be used with tarball.Image")
}
return &(td)[0], nil
}
for _, img := range td {
for _, tagStr := range img.RepoTags {
repoTag, err := name.NewTag(tagStr, name.WeakValidation)
if err != nil {
return nil, err
}
// Compare the resolved names, since there are several ways to specify the same tag.
if repoTag.Name() == tag.Name() {
return &img, nil
}
}
}
return nil, fmt.Errorf("tag %s not found in tarball", tag)
}
func (i *image) areLayersCompressed() (bool, error) {
if len(i.imgDescriptor.Layers) == 0 {
return false, errors.New("0 layers found in image")
}
layer := i.imgDescriptor.Layers[0]
blob, err := extractFileFromTar(i.opener, layer)
if err != nil {
return false, err
}
defer blob.Close()
return v1util.IsGzipped(blob)
}
func (i *image) loadTarDescriptorAndConfig() error {
td, err := extractFileFromTar(i.opener, "manifest.json")
if err != nil {
return err
}
defer td.Close()
if err := json.NewDecoder(td).Decode(&i.td); err != nil {
return err
}
i.imgDescriptor, err = i.td.findSpecifiedImageDescriptor(i.tag)
if err != nil {
return err
}
cfg, err := extractFileFromTar(i.opener, i.imgDescriptor.Config)
if err != nil {
return err
}
defer cfg.Close()
i.config, err = ioutil.ReadAll(cfg)
if err != nil {
return err
}
return nil
}
func (i *image) RawConfigFile() ([]byte, error) {
return i.config, nil
}
// tarFile represents a single file inside a tar. Closing it closes the tar itself.
type tarFile struct {
io.Reader
io.Closer
}
func extractFileFromTar(opener Opener, filePath string) (io.ReadCloser, error) {
f, err := opener()
if err != nil {
return nil, err
}
tf := tar.NewReader(f)
for {
hdr, err := tf.Next()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
if hdr.Name == filePath {
return tarFile{
Reader: tf,
Closer: f,
}, nil
}
}
return nil, fmt.Errorf("file %s not found in tar", filePath)
}
// uncompressedLayerFromTarball implements partial.UncompressedLayer
type uncompressedLayerFromTarball struct {
diffID v1.Hash
opener Opener
filePath string
}
// DiffID implements partial.UncompressedLayer
func (ulft *uncompressedLayerFromTarball) DiffID() (v1.Hash, error) {
return ulft.diffID, nil
}
// Uncompressed implements partial.UncompressedLayer
func (ulft *uncompressedLayerFromTarball) Uncompressed() (io.ReadCloser, error) {
return extractFileFromTar(ulft.opener, ulft.filePath)
}
func (i *uncompressedImage) LayerByDiffID(h v1.Hash) (partial.UncompressedLayer, error) {
cfg, err := partial.ConfigFile(i)
if err != nil {
return nil, err
}
for idx, diffID := range cfg.RootFS.DiffIDs {
if diffID == h {
return &uncompressedLayerFromTarball{
diffID: diffID,
opener: i.opener,
filePath: i.imgDescriptor.Layers[idx],
}, nil
}
}
return nil, fmt.Errorf("diff id %q not found", h)
}
func (c *compressedImage) Manifest() (*v1.Manifest, error) {
c.manifestLock.Lock()
defer c.manifestLock.Unlock()
if c.manifest != nil {
return c.manifest, nil
}
b, err := c.RawConfigFile()
if err != nil {
return nil, err
}
cfgHash, cfgSize, err := v1.SHA256(bytes.NewReader(b))
if err != nil {
return nil, err
}
c.manifest = &v1.Manifest{
SchemaVersion: 2,
MediaType: types.DockerManifestSchema2,
Config: v1.Descriptor{
MediaType: types.DockerConfigJSON,
Size: cfgSize,
Digest: cfgHash,
},
}
for _, p := range c.imgDescriptor.Layers {
l, err := extractFileFromTar(c.opener, p)
if err != nil {
return nil, err
}
defer l.Close()
sha, size, err := v1.SHA256(l)
if err != nil {
return nil, err
}
c.manifest.Layers = append(c.manifest.Layers, v1.Descriptor{
MediaType: types.DockerLayer,
Size: size,
Digest: sha,
})
}
return c.manifest, nil
}
func (c *compressedImage) RawManifest() ([]byte, error) {
return partial.RawManifest(c)
}
// compressedLayerFromTarball implements partial.CompressedLayer
type compressedLayerFromTarball struct {
digest v1.Hash
opener Opener
filePath string
}
// DiffID implements partial.CompressedLayer
func (clft *compressedLayerFromTarball) Digest() (v1.Hash, error) {
return clft.digest, nil
}
// Compressed implements partial.CompressedLayer
func (clft *compressedLayerFromTarball) Compressed() (io.ReadCloser, error) {
return extractFileFromTar(clft.opener, clft.filePath)
}
// Size implements partial.CompressedLayer
func (clft *compressedLayerFromTarball) Size() (int64, error) {
r, err := clft.Compressed()
if err != nil {
return -1, err
}
defer r.Close()
_, i, err := v1.SHA256(r)
return i, err
}
func (c *compressedImage) LayerByDigest(h v1.Hash) (partial.CompressedLayer, error) {
m, err := c.Manifest()
if err != nil {
return nil, err
}
for i, l := range m.Layers {
if l.Digest == h {
fp := c.imgDescriptor.Layers[i]
return &compressedLayerFromTarball{
digest: h,
opener: c.opener,
filePath: fp,
}, nil
}
}
return nil, fmt.Errorf("blob %v not found", h)
}
+144
View File
@@ -0,0 +1,144 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package tarball
import (
"compress/gzip"
"io"
"io/ioutil"
"os"
"github.com/google/go-containerregistry/v1"
"github.com/google/go-containerregistry/v1/v1util"
)
type layer struct {
digest v1.Hash
diffID v1.Hash
size int64
opener Opener
compressed bool
}
func (l *layer) Digest() (v1.Hash, error) {
return l.digest, nil
}
func (l *layer) DiffID() (v1.Hash, error) {
return l.diffID, nil
}
func (l *layer) Compressed() (io.ReadCloser, error) {
rc, err := l.opener()
if err == nil && !l.compressed {
return v1util.GzipReadCloser(rc)
}
return rc, err
}
func (l *layer) Uncompressed() (io.ReadCloser, error) {
rc, err := l.opener()
if err == nil && l.compressed {
return v1util.GunzipReadCloser(rc)
}
return rc, err
}
func (l *layer) Size() (int64, error) {
return l.size, nil
}
// LayerFromFile returns a v1.Layer given a tarball
func LayerFromFile(path string) (v1.Layer, error) {
opener := func() (io.ReadCloser, error) {
return os.Open(path)
}
return LayerFromOpener(opener)
}
// LayerFromOpener returns a v1.Layer given an Opener function
func LayerFromOpener(opener Opener) (v1.Layer, error) {
rc, err := opener()
if err != nil {
return nil, err
}
defer rc.Close()
compressed, err := v1util.IsGzipped(rc)
if err != nil {
return nil, err
}
var digest v1.Hash
var size int64
if digest, size, err = computeDigest(opener, compressed); err != nil {
return nil, err
}
diffID, err := computeDiffID(opener, compressed)
if err != nil {
return nil, err
}
return &layer{
digest: digest,
diffID: diffID,
size: size,
compressed: compressed,
opener: opener,
}, nil
}
func computeDigest(opener Opener, compressed bool) (v1.Hash, int64, error) {
rc, err := opener()
if err != nil {
return v1.Hash{}, 0, err
}
defer rc.Close()
if compressed {
return v1.SHA256(rc)
}
reader, err := v1util.GzipReadCloser(ioutil.NopCloser(rc))
if err != nil {
return v1.Hash{}, 0, err
}
return v1.SHA256(reader)
}
func computeDiffID(opener Opener, compressed bool) (v1.Hash, error) {
rc, err := opener()
if err != nil {
return v1.Hash{}, err
}
defer rc.Close()
if !compressed {
digest, _, err := v1.SHA256(rc)
return digest, err
}
reader, err := gzip.NewReader(rc)
if err != nil {
return v1.Hash{}, err
}
diffID, _, err := v1.SHA256(reader)
return diffID, err
}
+122
View File
@@ -0,0 +1,122 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package tarball
import (
"archive/tar"
"bytes"
"encoding/json"
"io"
"os"
"github.com/google/go-containerregistry/name"
"github.com/google/go-containerregistry/v1"
)
// WriteOptions are used to expose optional information to guide or
// control the image write.
type WriteOptions struct {
// TODO(mattmoor): Whether to store things compressed?
}
// Write saves the image as the given tag in a tarball at the given path.
func Write(p string, tag name.Tag, img v1.Image, wo *WriteOptions) error {
// Write in the compressed format.
// This is a tarball, on-disk, with:
// One manifest.json file at the top level containing information about several images.
// One file for each layer, named after the layer's SHA.
// One file for the config blob, named after its SHA.
w, err := os.Create(p)
if err != nil {
return err
}
defer w.Close()
tf := tar.NewWriter(w)
defer tf.Close()
// Write the config.
cfgName, err := img.ConfigName()
if err != nil {
return err
}
cfg, err := img.ConfigFile()
if err != nil {
return err
}
cfgBlob, err := json.Marshal(cfg)
if err != nil {
return err
}
if err := writeFile(tf, cfgName.String(), bytes.NewReader(cfgBlob), int64(len(cfgBlob))); err != nil {
return err
}
// Write the layers.
layers, err := img.Layers()
if err != nil {
return err
}
digests := make([]string, len(layers))
for i, l := range layers {
d, err := l.Digest()
if err != nil {
return err
}
digests[i] = d.String()
r, err := l.Compressed()
if err != nil {
return err
}
blobSize, err := l.Size()
if err != nil {
return err
}
if err := writeFile(tf, d.String(), r, blobSize); err != nil {
return err
}
}
// Generate the tar descriptor and write it.
td := tarDescriptor{
singleImageTarDescriptor{
Config: cfgName.String(),
RepoTags: []string{tag.String()},
Layers: digests,
},
}
tdBytes, err := json.Marshal(td)
if err != nil {
return err
}
return writeFile(tf, "manifest.json", bytes.NewReader(tdBytes), int64(len(tdBytes)))
}
func writeFile(tf *tar.Writer, path string, r io.Reader, size int64) error {
hdr := &tar.Header{
Mode: 0644,
Typeflag: tar.TypeReg,
Size: size,
Name: path,
}
if err := tf.WriteHeader(hdr); err != nil {
return err
}
_, err := io.Copy(tf, r)
return err
}
+40
View File
@@ -0,0 +1,40 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
// MediaType is an enumeration of the supported mime types that an element of an image might have.
type MediaType string
// The collection of known MediaType values.
const (
OCIContentDescriptor MediaType = "application/vnd.oci.descriptor.v1+json"
OCIImageIndex MediaType = "application/vnd.oci.image.index.v1+json"
OCIManifestSchema1 MediaType = "application/vnd.oci.image.manifest.v1+json"
OCIConfigJSON MediaType = "application/vnd.oci.image.config.v1+json"
OCILayer MediaType = "application/vnd.oci.image.layer.v1.tar+gzip"
OCIRestrictedLayer MediaType = "application/vnd.oci.image.layer.nondistributable.v1.tar+gzip"
OCIUncompressedLayer MediaType = "application/vnd.oci.image.layer.v1.tar"
OCIUncompressedRestrictedLayer MediaType = "application/vnd.oci.image.layer.nondistributable.v1.tar"
DockerManifestSchema1 MediaType = "application/vnd.docker.distribution.manifest.v1+json"
DockerManifestSchema1Signed MediaType = "application/vnd.docker.distribution.manifest.v1+prettyjws"
DockerManifestSchema2 MediaType = "application/vnd.docker.distribution.manifest.v2+json"
DockerManifestList MediaType = "application/vnd.docker.distribution.manifest.list.v2+json"
DockerLayer MediaType = "application/vnd.docker.image.rootfs.diff.tar.gzip"
DockerConfigJSON MediaType = "application/vnd.docker.container.image.v1+json"
DockerPluginConfig MediaType = "application/vnd.docker.plugin.v1+json"
DockerForeignLayer MediaType = "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip"
DockerUncompressedLayer MediaType = "application/vnd.docker.image.rootfs.diff.tar"
)
+47
View File
@@ -0,0 +1,47 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1util
import (
"io"
)
// readAndCloser implements io.ReadCloser by reading from a particular io.Reader
// and then calling the provided "Close()" method.
type readAndCloser struct {
io.Reader
CloseFunc func() error
}
var _ io.ReadCloser = (*readAndCloser)(nil)
// Close implements io.ReadCloser
func (rac *readAndCloser) Close() error {
return rac.CloseFunc()
}
// writeAndCloser implements io.WriteCloser by reading from a particular io.Writer
// and then calling the provided "Close()" method.
type writeAndCloser struct {
io.Writer
CloseFunc func() error
}
var _ io.WriteCloser = (*writeAndCloser)(nil)
// Close implements io.WriteCloser
func (wac *writeAndCloser) Close() error {
return wac.CloseFunc()
}
+40
View File
@@ -0,0 +1,40 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1util
import (
"io"
)
func nop() error {
return nil
}
// NopWriteCloser wraps the io.Writer as an io.WriteCloser with a Close() method that does nothing.
func NopWriteCloser(w io.Writer) io.WriteCloser {
return &writeAndCloser{
Writer: w,
CloseFunc: nop,
}
}
// NopReadCloser wraps the io.Reader as an io.ReadCloser with a Close() method that does nothing.
// This is technically redundant with ioutil.NopCloser, but provided for symmetry and clarity.
func NopReadCloser(r io.Reader) io.ReadCloser {
return &readAndCloser{
Reader: r,
CloseFunc: nop,
}
}
+61
View File
@@ -0,0 +1,61 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1util
import (
"encoding/hex"
"fmt"
"hash"
"io"
"github.com/google/go-containerregistry/v1"
)
type verifyReader struct {
inner io.Reader
hasher hash.Hash
expected v1.Hash
}
// Read implements io.Reader
func (vc *verifyReader) Read(b []byte) (int, error) {
n, err := vc.inner.Read(b)
if err == io.EOF {
got := hex.EncodeToString(vc.hasher.Sum(make([]byte, 0, vc.hasher.Size())))
if want := vc.expected.Hex; got != want {
return n, fmt.Errorf("error verifying %s checksum; got %q, want %q",
vc.expected.Algorithm, got, want)
}
}
return n, err
}
// VerifyReadCloser wraps the given io.ReadCloser to verify that its contents match
// the provided v1.Hash before io.EOF is returned.
func VerifyReadCloser(r io.ReadCloser, h v1.Hash) (io.ReadCloser, error) {
w, err := v1.Hasher(h.Algorithm)
if err != nil {
return nil, err
}
r2 := io.TeeReader(r, w)
return &readAndCloser{
Reader: &verifyReader{
inner: r2,
hasher: w,
expected: h,
},
CloseFunc: r.Close,
}, nil
}
+117
View File
@@ -0,0 +1,117 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1util
import (
"bytes"
"compress/gzip"
"io"
)
var gzipMagicHeader = []byte{'\x1f', '\x8b'}
// GzipReadCloser reads uncompressed input data from the io.ReadCloser and
// returns an io.ReadCloser from which compressed data may be read.
func GzipReadCloser(r io.ReadCloser) (io.ReadCloser, error) {
pr, pw := io.Pipe()
go func() {
defer pw.Close()
defer r.Close()
gw, _ := gzip.NewWriterLevel(pw, gzip.BestCompression)
defer gw.Close()
_, err := io.Copy(gw, r)
if err != nil {
pr.CloseWithError(err)
}
}()
return pr, nil
}
// GunzipReadCloser reads compressed input data from the io.ReadCloser and
// returns an io.ReadCloser from which uncompessed data may be read.
func GunzipReadCloser(r io.ReadCloser) (io.ReadCloser, error) {
gr, err := gzip.NewReader(r)
if err != nil {
return nil, err
}
return &readAndCloser{
Reader: gr,
CloseFunc: func() error {
if err := gr.Close(); err != nil {
return err
}
return r.Close()
},
}, nil
}
// GzipWriteCloser returns an io.WriteCloser to which uncompressed data may be
// written, and the compressed data is then written to the provided
// io.WriteCloser.
func GzipWriteCloser(w io.WriteCloser) io.WriteCloser {
gw := gzip.NewWriter(w)
return &writeAndCloser{
Writer: gw,
CloseFunc: func() error {
if err := gw.Close(); err != nil {
return err
}
return w.Close()
},
}
}
// gunzipWriteCloser implements io.WriteCloser
// It is used to implement GunzipWriteClose.
type gunzipWriteCloser struct {
*bytes.Buffer
writer io.WriteCloser
}
// Close implements io.WriteCloser
func (gwc *gunzipWriteCloser) Close() error {
// TODO(mattmoor): How to avoid buffering this whole thing into memory?
gr, err := gzip.NewReader(gwc.Buffer)
if err != nil {
return err
}
if _, err := io.Copy(gwc.writer, gr); err != nil {
return err
}
return gwc.writer.Close()
}
// GunzipWriteCloser returns an io.WriteCloser to which compressed data may be
// written, and the uncompressed data is then written to the provided
// io.WriteCloser.
func GunzipWriteCloser(w io.WriteCloser) (io.WriteCloser, error) {
return &gunzipWriteCloser{
Buffer: bytes.NewBuffer(nil),
writer: w,
}, nil
}
// IsGzipped detects whether the input stream is compressed.
func IsGzipped(r io.Reader) (bool, error) {
magicHeader := make([]byte, 2)
if _, err := r.Read(magicHeader); err != nil {
return false, err
}
return bytes.Equal(magicHeader, gzipMagicHeader), nil
}
@@ -0,0 +1,232 @@
// +build !ignore_autogenerated
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Code generated by deepcopy-gen. DO NOT EDIT.
package v1
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Config) DeepCopyInto(out *Config) {
*out = *in
if in.Cmd != nil {
in, out := &in.Cmd, &out.Cmd
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.Entrypoint != nil {
in, out := &in.Entrypoint, &out.Entrypoint
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.Env != nil {
in, out := &in.Env, &out.Env
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.Labels != nil {
in, out := &in.Labels, &out.Labels
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.OnBuild != nil {
in, out := &in.OnBuild, &out.OnBuild
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.Volumes != nil {
in, out := &in.Volumes, &out.Volumes
*out = make(map[string]struct{}, len(*in))
for key := range *in {
(*out)[key] = struct{}{}
}
}
if in.ExposedPorts != nil {
in, out := &in.ExposedPorts, &out.ExposedPorts
*out = make(map[string]struct{}, len(*in))
for key := range *in {
(*out)[key] = struct{}{}
}
}
if in.Shell != nil {
in, out := &in.Shell, &out.Shell
*out = make([]string, len(*in))
copy(*out, *in)
}
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Config.
func (in *Config) DeepCopy() *Config {
if in == nil {
return nil
}
out := new(Config)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *ConfigFile) DeepCopyInto(out *ConfigFile) {
*out = *in
in.Created.DeepCopyInto(&out.Created)
if in.History != nil {
in, out := &in.History, &out.History
*out = make([]History, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
in.RootFS.DeepCopyInto(&out.RootFS)
in.Config.DeepCopyInto(&out.Config)
in.ContainerConfig.DeepCopyInto(&out.ContainerConfig)
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigFile.
func (in *ConfigFile) DeepCopy() *ConfigFile {
if in == nil {
return nil
}
out := new(ConfigFile)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Descriptor) DeepCopyInto(out *Descriptor) {
*out = *in
out.Digest = in.Digest
if in.URLs != nil {
in, out := &in.URLs, &out.URLs
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.Annotations != nil {
in, out := &in.Annotations, &out.Annotations
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Descriptor.
func (in *Descriptor) DeepCopy() *Descriptor {
if in == nil {
return nil
}
out := new(Descriptor)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Hash) DeepCopyInto(out *Hash) {
*out = *in
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Hash.
func (in *Hash) DeepCopy() *Hash {
if in == nil {
return nil
}
out := new(Hash)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *History) DeepCopyInto(out *History) {
*out = *in
in.Created.DeepCopyInto(&out.Created)
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new History.
func (in *History) DeepCopy() *History {
if in == nil {
return nil
}
out := new(History)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Manifest) DeepCopyInto(out *Manifest) {
*out = *in
in.Config.DeepCopyInto(&out.Config)
if in.Layers != nil {
in, out := &in.Layers, &out.Layers
*out = make([]Descriptor, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.Annotations != nil {
in, out := &in.Annotations, &out.Annotations
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Manifest.
func (in *Manifest) DeepCopy() *Manifest {
if in == nil {
return nil
}
out := new(Manifest)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *RootFS) DeepCopyInto(out *RootFS) {
*out = *in
if in.DiffIDs != nil {
in, out := &in.DiffIDs, &out.DiffIDs
*out = make([]Hash, len(*in))
copy(*out, *in)
}
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RootFS.
func (in *RootFS) DeepCopy() *RootFS {
if in == nil {
return nil
}
out := new(RootFS)
in.DeepCopyInto(out)
return out
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Time.
func (in *Time) DeepCopy() *Time {
if in == nil {
return nil
}
out := new(Time)
in.DeepCopyInto(out)
return out
}