public_git/kaniko
1
0
Fork 0
mirror of https://github.com/GoogleContainerTools/kaniko synced 2026-10-06 22:28:07 +02:00
Code Issues Releases Wiki Activity

Revert "Update go-containerregistry to get https://github.com/google/go-containerregistry/pull/693"

Browse Source 
This reverts commit f55c34ab92.
This commit is contained in:
Vincent Latombe
2020-03-24 09:50:59 +01:00
parent 4f7816f417
commit af42f14468
1068 changed files with 89953 additions and 179391 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vendor/github.com/google/btree/.travis.yml
Generated Vendored
+1
View File
@@ -0,0 +1 @@
language: go
vendor/github.com/google/btree/LICENSE
Generated Vendored
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
vendor/github.com/google/btree/README.md
Generated Vendored
+12
View File
@@ -0,0 +1,12 @@
# BTree implementation for Go
![Travis CI Build Status](https://api.travis-ci.org/google/btree.svg?branch=master)
This package provides an in-memory B-Tree implementation for Go, useful as
an ordered, mutable data structure.
The API is based off of the wonderful
http://godoc.org/github.com/petar/GoLLRB/llrb, and is meant to allow btree to
act as a drop-in replacement for gollrb trees.
See http://godoc.org/github.com/google/btree for documentation.
vendor/github.com/google/btree/btree.go
Generated Vendored
+881
View File
@@ -0,0 +1,881 @@
// Copyright 2014 Google Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package btree implements in-memory B-Trees of arbitrary degree.
//
// btree implements an in-memory B-Tree for use as an ordered data structure.
// It is not meant for persistent storage solutions.
//
// It has a flatter structure than an equivalent red-black or other binary tree,
// which in some cases yields better memory usage and/or performance.
// See some discussion on the matter here:
// http://google-opensource.blogspot.com/2013/01/c-containers-that-save-memory-and-time.html
// Note, though, that this project is in no way related to the C++ B-Tree
// implementation written about there.
//
// Within this tree, each node contains a slice of items and a (possibly nil)
// slice of children. For basic numeric values or raw structs, this can cause
// efficiency differences when compared to equivalent C++ template code that
// stores values in arrays within the node:
// * Due to the overhead of storing values as interfaces (each
// value needs to be stored as the value itself, then 2 words for the
// interface pointing to that value and its type), resulting in higher
// memory use.
// * Since interfaces can point to values anywhere in memory, values are
// most likely not stored in contiguous blocks, resulting in a higher
// number of cache misses.
// These issues don't tend to matter, though, when working with strings or other
// heap-allocated structures, since C++-equivalent structures also must store
// pointers and also distribute their values across the heap.
//
// This implementation is designed to be a drop-in replacement to gollrb.LLRB
// trees, (http://github.com/petar/gollrb), an excellent and probably the most
// widely used ordered tree implementation in the Go ecosystem currently.
// Its functions, therefore, exactly mirror those of
// llrb.LLRB where possible. Unlike gollrb, though, we currently don't
// support storing multiple equivalent values.
package btree
import (
"fmt"
"io"
"sort"
"strings"
"sync"
)
// Item represents a single object in the tree.
type Item interface {
// Less tests whether the current item is less than the given argument.
//
// This must provide a strict weak ordering.
// If !a.Less(b) && !b.Less(a), we treat this to mean a == b (i.e. we can only
// hold one of either a or b in the tree).
Less(than Item) bool
}
const (
DefaultFreeListSize = 32
)
var (
nilItems = make(items, 16)
nilChildren = make(children, 16)
)
// FreeList represents a free list of btree nodes. By default each
// BTree has its own FreeList, but multiple BTrees can share the same
// FreeList.
// Two Btrees using the same freelist are safe for concurrent write access.
type FreeList struct {
mu sync.Mutex
freelist []*node
}
// NewFreeList creates a new free list.
// size is the maximum size of the returned free list.
func NewFreeList(size int) *FreeList {
return &FreeList{freelist: make([]*node, 0, size)}
}
func (f *FreeList) newNode() (n *node) {
f.mu.Lock()
index := len(f.freelist) - 1
if index < 0 {
f.mu.Unlock()
return new(node)
}
n = f.freelist[index]
f.freelist[index] = nil
f.freelist = f.freelist[:index]
f.mu.Unlock()
return
}
// freeNode adds the given node to the list, returning true if it was added
// and false if it was discarded.
func (f *FreeList) freeNode(n *node) (out bool) {
f.mu.Lock()
if len(f.freelist) < cap(f.freelist) {
f.freelist = append(f.freelist, n)
out = true
}
f.mu.Unlock()
return
}
// ItemIterator allows callers of Ascend* to iterate in-order over portions of
// the tree. When this function returns false, iteration will stop and the
// associated Ascend* function will immediately return.
type ItemIterator func(i Item) bool
// New creates a new B-Tree with the given degree.
//
// New(2), for example, will create a 2-3-4 tree (each node contains 1-3 items
// and 2-4 children).
func New(degree int) *BTree {
return NewWithFreeList(degree, NewFreeList(DefaultFreeListSize))
}
// NewWithFreeList creates a new B-Tree that uses the given node free list.
func NewWithFreeList(degree int, f *FreeList) *BTree {
if degree <= 1 {
panic("bad degree")
}
return &BTree{
degree: degree,
cow: &copyOnWriteContext{freelist: f},
}
}
// items stores items in a node.
type items []Item
// insertAt inserts a value into the given index, pushing all subsequent values
// forward.
func (s *items) insertAt(index int, item Item) {
*s = append(*s, nil)
if index < len(*s) {
copy((*s)[index+1:], (*s)[index:])
}
(*s)[index] = item
}
// removeAt removes a value at a given index, pulling all subsequent values
// back.
func (s *items) removeAt(index int) Item {
item := (*s)[index]
copy((*s)[index:], (*s)[index+1:])
(*s)[len(*s)-1] = nil
*s = (*s)[:len(*s)-1]
return item
}
// pop removes and returns the last element in the list.
func (s *items) pop() (out Item) {
index := len(*s) - 1
out = (*s)[index]
(*s)[index] = nil
*s = (*s)[:index]
return
}
// truncate truncates this instance at index so that it contains only the
// first index items. index must be less than or equal to length.
func (s *items) truncate(index int) {
var toClear items
*s, toClear = (*s)[:index], (*s)[index:]
for len(toClear) > 0 {
toClear = toClear[copy(toClear, nilItems):]
}
}
// find returns the index where the given item should be inserted into this
// list. 'found' is true if the item already exists in the list at the given
// index.
func (s items) find(item Item) (index int, found bool) {
i := sort.Search(len(s), func(i int) bool {
return item.Less(s[i])
})
if i > 0 && !s[i-1].Less(item) {
return i - 1, true
}
return i, false
}
// children stores child nodes in a node.
type children []*node
// insertAt inserts a value into the given index, pushing all subsequent values
// forward.
func (s *children) insertAt(index int, n *node) {
*s = append(*s, nil)
if index < len(*s) {
copy((*s)[index+1:], (*s)[index:])
}
(*s)[index] = n
}
// removeAt removes a value at a given index, pulling all subsequent values
// back.
func (s *children) removeAt(index int) *node {
n := (*s)[index]
copy((*s)[index:], (*s)[index+1:])
(*s)[len(*s)-1] = nil
*s = (*s)[:len(*s)-1]
return n
}
// pop removes and returns the last element in the list.
func (s *children) pop() (out *node) {
index := len(*s) - 1
out = (*s)[index]
(*s)[index] = nil
*s = (*s)[:index]
return
}
// truncate truncates this instance at index so that it contains only the
// first index children. index must be less than or equal to length.
func (s *children) truncate(index int) {
var toClear children
*s, toClear = (*s)[:index], (*s)[index:]
for len(toClear) > 0 {
toClear = toClear[copy(toClear, nilChildren):]
}
}
// node is an internal node in a tree.
//
// It must at all times maintain the invariant that either
// * len(children) == 0, len(items) unconstrained
// * len(children) == len(items) + 1
type node struct {
items items
children children
cow *copyOnWriteContext
}
func (n *node) mutableFor(cow *copyOnWriteContext) *node {
if n.cow == cow {
return n
}
out := cow.newNode()
if cap(out.items) >= len(n.items) {
out.items = out.items[:len(n.items)]
} else {
out.items = make(items, len(n.items), cap(n.items))
}
copy(out.items, n.items)
// Copy children
if cap(out.children) >= len(n.children) {
out.children = out.children[:len(n.children)]
} else {
out.children = make(children, len(n.children), cap(n.children))
}
copy(out.children, n.children)
return out
}
func (n *node) mutableChild(i int) *node {
c := n.children[i].mutableFor(n.cow)
n.children[i] = c
return c
}
// split splits the given node at the given index. The current node shrinks,
// and this function returns the item that existed at that index and a new node
// containing all items/children after it.
func (n *node) split(i int) (Item, *node) {
item := n.items[i]
next := n.cow.newNode()
next.items = append(next.items, n.items[i+1:]...)
n.items.truncate(i)
if len(n.children) > 0 {
next.children = append(next.children, n.children[i+1:]...)
n.children.truncate(i + 1)
}
return item, next
}
// maybeSplitChild checks if a child should be split, and if so splits it.
// Returns whether or not a split occurred.
func (n *node) maybeSplitChild(i, maxItems int) bool {
if len(n.children[i].items) < maxItems {
return false
}
first := n.mutableChild(i)
item, second := first.split(maxItems / 2)
n.items.insertAt(i, item)
n.children.insertAt(i+1, second)
return true
}
// insert inserts an item into the subtree rooted at this node, making sure
// no nodes in the subtree exceed maxItems items. Should an equivalent item be
// be found/replaced by insert, it will be returned.
func (n *node) insert(item Item, maxItems int) Item {
i, found := n.items.find(item)
if found {
out := n.items[i]
n.items[i] = item
return out
}
if len(n.children) == 0 {
n.items.insertAt(i, item)
return nil
}
if n.maybeSplitChild(i, maxItems) {
inTree := n.items[i]
switch {
case item.Less(inTree):
// no change, we want first split node
case inTree.Less(item):
i++ // we want second split node
default:
out := n.items[i]
n.items[i] = item
return out
}
}
return n.mutableChild(i).insert(item, maxItems)
}
// get finds the given key in the subtree and returns it.
func (n *node) get(key Item) Item {
i, found := n.items.find(key)
if found {
return n.items[i]
} else if len(n.children) > 0 {
return n.children[i].get(key)
}
return nil
}
// min returns the first item in the subtree.
func min(n *node) Item {
if n == nil {
return nil
}
for len(n.children) > 0 {
n = n.children[0]
}
if len(n.items) == 0 {
return nil
}
return n.items[0]
}
// max returns the last item in the subtree.
func max(n *node) Item {
if n == nil {
return nil
}
for len(n.children) > 0 {
n = n.children[len(n.children)-1]
}
if len(n.items) == 0 {
return nil
}
return n.items[len(n.items)-1]
}
// toRemove details what item to remove in a node.remove call.
type toRemove int
const (
removeItem toRemove = iota // removes the given item
removeMin // removes smallest item in the subtree
removeMax // removes largest item in the subtree
)
// remove removes an item from the subtree rooted at this node.
func (n *node) remove(item Item, minItems int, typ toRemove) Item {
var i int
var found bool
switch typ {
case removeMax:
if len(n.children) == 0 {
return n.items.pop()
}
i = len(n.items)
case removeMin:
if len(n.children) == 0 {
return n.items.removeAt(0)
}
i = 0
case removeItem:
i, found = n.items.find(item)
if len(n.children) == 0 {
if found {
return n.items.removeAt(i)
}
return nil
}
default:
panic("invalid type")
}
// If we get to here, we have children.
if len(n.children[i].items) <= minItems {
return n.growChildAndRemove(i, item, minItems, typ)
}
child := n.mutableChild(i)
// Either we had enough items to begin with, or we've done some
// merging/stealing, because we've got enough now and we're ready to return
// stuff.
if found {
// The item exists at index 'i', and the child we've selected can give us a
// predecessor, since if we've gotten here it's got > minItems items in it.
out := n.items[i]
// We use our special-case 'remove' call with typ=maxItem to pull the
// predecessor of item i (the rightmost leaf of our immediate left child)
// and set it into where we pulled the item from.
n.items[i] = child.remove(nil, minItems, removeMax)
return out
}
// Final recursive call. Once we're here, we know that the item isn't in this
// node and that the child is big enough to remove from.
return child.remove(item, minItems, typ)
}
// growChildAndRemove grows child 'i' to make sure it's possible to remove an
// item from it while keeping it at minItems, then calls remove to actually
// remove it.
//
// Most documentation says we have to do two sets of special casing:
// 1) item is in this node
// 2) item is in child
// In both cases, we need to handle the two subcases:
// A) node has enough values that it can spare one
// B) node doesn't have enough values
// For the latter, we have to check:
// a) left sibling has node to spare
// b) right sibling has node to spare
// c) we must merge
// To simplify our code here, we handle cases #1 and #2 the same:
// If a node doesn't have enough items, we make sure it does (using a,b,c).
// We then simply redo our remove call, and the second time (regardless of
// whether we're in case 1 or 2), we'll have enough items and can guarantee
// that we hit case A.
func (n *node) growChildAndRemove(i int, item Item, minItems int, typ toRemove) Item {
if i > 0 && len(n.children[i-1].items) > minItems {
// Steal from left child
child := n.mutableChild(i)
stealFrom := n.mutableChild(i - 1)
stolenItem := stealFrom.items.pop()
child.items.insertAt(0, n.items[i-1])
n.items[i-1] = stolenItem
if len(stealFrom.children) > 0 {
child.children.insertAt(0, stealFrom.children.pop())
}
} else if i < len(n.items) && len(n.children[i+1].items) > minItems {
// steal from right child
child := n.mutableChild(i)
stealFrom := n.mutableChild(i + 1)
stolenItem := stealFrom.items.removeAt(0)
child.items = append(child.items, n.items[i])
n.items[i] = stolenItem
if len(stealFrom.children) > 0 {
child.children = append(child.children, stealFrom.children.removeAt(0))
}
} else {
if i >= len(n.items) {
i--
}
child := n.mutableChild(i)
// merge with right child
mergeItem := n.items.removeAt(i)
mergeChild := n.children.removeAt(i + 1)
child.items = append(child.items, mergeItem)
child.items = append(child.items, mergeChild.items...)
child.children = append(child.children, mergeChild.children...)
n.cow.freeNode(mergeChild)
}
return n.remove(item, minItems, typ)
}
type direction int
const (
descend = direction(-1)
ascend = direction(+1)
)
// iterate provides a simple method for iterating over elements in the tree.
//
// When ascending, the 'start' should be less than 'stop' and when descending,
// the 'start' should be greater than 'stop'. Setting 'includeStart' to true
// will force the iterator to include the first item when it equals 'start',
// thus creating a "greaterOrEqual" or "lessThanEqual" rather than just a
// "greaterThan" or "lessThan" queries.
func (n *node) iterate(dir direction, start, stop Item, includeStart bool, hit bool, iter ItemIterator) (bool, bool) {
var ok bool
switch dir {
case ascend:
for i := 0; i < len(n.items); i++ {
if start != nil && n.items[i].Less(start) {
continue
}
if len(n.children) > 0 {
if hit, ok = n.children[i].iterate(dir, start, stop, includeStart, hit, iter); !ok {
return hit, false
}
}
if !includeStart && !hit && start != nil && !start.Less(n.items[i]) {
hit = true
continue
}
hit = true
if stop != nil && !n.items[i].Less(stop) {
return hit, false
}
if !iter(n.items[i]) {
return hit, false
}
}
if len(n.children) > 0 {
if hit, ok = n.children[len(n.children)-1].iterate(dir, start, stop, includeStart, hit, iter); !ok {
return hit, false
}
}
case descend:
for i := len(n.items) - 1; i >= 0; i-- {
if start != nil && !n.items[i].Less(start) {
if !includeStart || hit || start.Less(n.items[i]) {
continue
}
}
if len(n.children) > 0 {
if hit, ok = n.children[i+1].iterate(dir, start, stop, includeStart, hit, iter); !ok {
return hit, false
}
}
if stop != nil && !stop.Less(n.items[i]) {
return hit, false // continue
}
hit = true
if !iter(n.items[i]) {
return hit, false
}
}
if len(n.children) > 0 {
if hit, ok = n.children[0].iterate(dir, start, stop, includeStart, hit, iter); !ok {
return hit, false
}
}
}
return hit, true
}
// Used for testing/debugging purposes.
func (n *node) print(w io.Writer, level int) {
fmt.Fprintf(w, "%sNODE:%v\n", strings.Repeat(" ", level), n.items)
for _, c := range n.children {
c.print(w, level+1)
}
}
// BTree is an implementation of a B-Tree.
//
// BTree stores Item instances in an ordered structure, allowing easy insertion,
// removal, and iteration.
//
// Write operations are not safe for concurrent mutation by multiple
// goroutines, but Read operations are.
type BTree struct {
degree int
length int
root *node
cow *copyOnWriteContext
}
// copyOnWriteContext pointers determine node ownership... a tree with a write
// context equivalent to a node's write context is allowed to modify that node.
// A tree whose write context does not match a node's is not allowed to modify
// it, and must create a new, writable copy (IE: it's a Clone).
//
// When doing any write operation, we maintain the invariant that the current
// node's context is equal to the context of the tree that requested the write.
// We do this by, before we descend into any node, creating a copy with the
// correct context if the contexts don't match.
//
// Since the node we're currently visiting on any write has the requesting
// tree's context, that node is modifiable in place. Children of that node may
// not share context, but before we descend into them, we'll make a mutable
// copy.
type copyOnWriteContext struct {
freelist *FreeList
}
// Clone clones the btree, lazily. Clone should not be called concurrently,
// but the original tree (t) and the new tree (t2) can be used concurrently
// once the Clone call completes.
//
// The internal tree structure of b is marked read-only and shared between t and
// t2. Writes to both t and t2 use copy-on-write logic, creating new nodes
// whenever one of b's original nodes would have been modified. Read operations
// should have no performance degredation. Write operations for both t and t2
// will initially experience minor slow-downs caused by additional allocs and
// copies due to the aforementioned copy-on-write logic, but should converge to
// the original performance characteristics of the original tree.
func (t *BTree) Clone() (t2 *BTree) {
// Create two entirely new copy-on-write contexts.
// This operation effectively creates three trees:
// the original, shared nodes (old b.cow)
// the new b.cow nodes
// the new out.cow nodes
cow1, cow2 := *t.cow, *t.cow
out := *t
t.cow = &cow1
out.cow = &cow2
return &out
}
// maxItems returns the max number of items to allow per node.
func (t *BTree) maxItems() int {
return t.degree*2 - 1
}
// minItems returns the min number of items to allow per node (ignored for the
// root node).
func (t *BTree) minItems() int {
return t.degree - 1
}
func (c *copyOnWriteContext) newNode() (n *node) {
n = c.freelist.newNode()
n.cow = c
return
}
type freeType int
const (
ftFreelistFull freeType = iota // node was freed (available for GC, not stored in freelist)
ftStored // node was stored in the freelist for later use
ftNotOwned // node was ignored by COW, since it's owned by another one
)
// freeNode frees a node within a given COW context, if it's owned by that
// context. It returns what happened to the node (see freeType const
// documentation).
func (c *copyOnWriteContext) freeNode(n *node) freeType {
if n.cow == c {
// clear to allow GC
n.items.truncate(0)
n.children.truncate(0)
n.cow = nil
if c.freelist.freeNode(n) {
return ftStored
} else {
return ftFreelistFull
}
} else {
return ftNotOwned
}
}
// ReplaceOrInsert adds the given item to the tree. If an item in the tree
// already equals the given one, it is removed from the tree and returned.
// Otherwise, nil is returned.
//
// nil cannot be added to the tree (will panic).
func (t *BTree) ReplaceOrInsert(item Item) Item {
if item == nil {
panic("nil item being added to BTree")
}
if t.root == nil {
t.root = t.cow.newNode()
t.root.items = append(t.root.items, item)
t.length++
return nil
} else {
t.root = t.root.mutableFor(t.cow)
if len(t.root.items) >= t.maxItems() {
item2, second := t.root.split(t.maxItems() / 2)
oldroot := t.root
t.root = t.cow.newNode()
t.root.items = append(t.root.items, item2)
t.root.children = append(t.root.children, oldroot, second)
}
}
out := t.root.insert(item, t.maxItems())
if out == nil {
t.length++
}
return out
}
// Delete removes an item equal to the passed in item from the tree, returning
// it. If no such item exists, returns nil.
func (t *BTree) Delete(item Item) Item {
return t.deleteItem(item, removeItem)
}
// DeleteMin removes the smallest item in the tree and returns it.
// If no such item exists, returns nil.
func (t *BTree) DeleteMin() Item {
return t.deleteItem(nil, removeMin)
}
// DeleteMax removes the largest item in the tree and returns it.
// If no such item exists, returns nil.
func (t *BTree) DeleteMax() Item {
return t.deleteItem(nil, removeMax)
}
func (t *BTree) deleteItem(item Item, typ toRemove) Item {
if t.root == nil || len(t.root.items) == 0 {
return nil
}
t.root = t.root.mutableFor(t.cow)
out := t.root.remove(item, t.minItems(), typ)
if len(t.root.items) == 0 && len(t.root.children) > 0 {
oldroot := t.root
t.root = t.root.children[0]
t.cow.freeNode(oldroot)
}
if out != nil {
t.length--
}
return out
}
// AscendRange calls the iterator for every value in the tree within the range
// [greaterOrEqual, lessThan), until iterator returns false.
func (t *BTree) AscendRange(greaterOrEqual, lessThan Item, iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(ascend, greaterOrEqual, lessThan, true, false, iterator)
}
// AscendLessThan calls the iterator for every value in the tree within the range
// [first, pivot), until iterator returns false.
func (t *BTree) AscendLessThan(pivot Item, iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(ascend, nil, pivot, false, false, iterator)
}
// AscendGreaterOrEqual calls the iterator for every value in the tree within
// the range [pivot, last], until iterator returns false.
func (t *BTree) AscendGreaterOrEqual(pivot Item, iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(ascend, pivot, nil, true, false, iterator)
}
// Ascend calls the iterator for every value in the tree within the range
// [first, last], until iterator returns false.
func (t *BTree) Ascend(iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(ascend, nil, nil, false, false, iterator)
}
// DescendRange calls the iterator for every value in the tree within the range
// [lessOrEqual, greaterThan), until iterator returns false.
func (t *BTree) DescendRange(lessOrEqual, greaterThan Item, iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(descend, lessOrEqual, greaterThan, true, false, iterator)
}
// DescendLessOrEqual calls the iterator for every value in the tree within the range
// [pivot, first], until iterator returns false.
func (t *BTree) DescendLessOrEqual(pivot Item, iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(descend, pivot, nil, true, false, iterator)
}
// DescendGreaterThan calls the iterator for every value in the tree within
// the range (pivot, last], until iterator returns false.
func (t *BTree) DescendGreaterThan(pivot Item, iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(descend, nil, pivot, false, false, iterator)
}
// Descend calls the iterator for every value in the tree within the range
// [last, first], until iterator returns false.
func (t *BTree) Descend(iterator ItemIterator) {
if t.root == nil {
return
}
t.root.iterate(descend, nil, nil, false, false, iterator)
}
// Get looks for the key item in the tree, returning it. It returns nil if
// unable to find that item.
func (t *BTree) Get(key Item) Item {
if t.root == nil {
return nil
}
return t.root.get(key)
}
// Min returns the smallest item in the tree, or nil if the tree is empty.
func (t *BTree) Min() Item {
return min(t.root)
}
// Max returns the largest item in the tree, or nil if the tree is empty.
func (t *BTree) Max() Item {
return max(t.root)
}
// Has returns true if the given key is in the tree.
func (t *BTree) Has(key Item) bool {
return t.Get(key) != nil
}
// Len returns the number of items currently in the tree.
func (t *BTree) Len() int {
return t.length
}
// Clear removes all items from the btree. If addNodesToFreelist is true,
// t's nodes are added to its freelist as part of this call, until the freelist
// is full. Otherwise, the root node is simply dereferenced and the subtree
// left to Go's normal GC processes.
//
// This can be much faster
// than calling Delete on all elements, because that requires finding/removing
// each element in the tree and updating the tree accordingly. It also is
// somewhat faster than creating a new tree to replace the old one, because
// nodes from the old tree are reclaimed into the freelist for use by the new
// one, instead of being lost to the garbage collector.
//
// This call takes:
// O(1): when addNodesToFreelist is false, this is a single operation.
// O(1): when the freelist is already full, it breaks out immediately
// O(freelist size): when the freelist is empty and the nodes are all owned
// by this tree, nodes are added to the freelist until full.
// O(tree size): when all nodes are owned by another tree, all nodes are
// iterated over looking for nodes to add to the freelist, and due to
// ownership, none are.
func (t *BTree) Clear(addNodesToFreelist bool) {
if t.root != nil && addNodesToFreelist {
t.root.reset(t.cow)
}
t.root, t.length = nil, 0
}
// reset returns a subtree to the freelist. It breaks out immediately if the
// freelist is full, since the only benefit of iterating is to fill that
// freelist up. Returns true if parent reset call should continue.
func (n *node) reset(c *copyOnWriteContext) bool {
for _, child := range n.children {
if !child.reset(c) {
return false
}
}
return c.freeNode(n) != ftFreelistFull
}
// Int implements the Item interface for integers.
type Int int
// Less returns true if int(a) < int(b).
func (a Int) Less(b Item) bool {
return a < b.(Int)
}
vendor/github.com/google/go-containerregistry/pkg/authn/README.md
Generated Vendored
-242
View File
@@ -1,242 +0,0 @@
# `authn`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/authn?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/authn)
This README outlines how we acquire and use credentials when interacting with a registry.
As much as possible, we attempt to emulate docker's authentication behavior and configuration so that this library "just works" if you've already configured credentials that work with docker; however, when things don't work, a basic understanding of what's going on can help with debugging.
The official documentation for how docker authentication works is (reasonably) scattered across several different sites and GitHub repositories, so we've tried to summarize the relevant bits here.
## tl;dr for consumers of this package
By default, [`pkg/v1/remote`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote) uses [`Anonymous`](https://godoc.org/github.com/google/go-containerregistry/pkg/authn#Anonymous) credentials (i.e. _none_), which for most registries will only allow read access to public images.
To use the credentials found in your docker config file, you can use the [`DefaultKeychain`](https://godoc.org/github.com/google/go-containerregistry/pkg/authn#DefaultKeychain), e.g.:
```go
package main
import (
"fmt"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func main() {
ref, err := name.ParseReference("registry.example.com/private/repo")
if err != nil {
panic(err)
}
// Fetch the manifest using default credentials.
img, err := remote.Get(ref, remote.WithAuthFromKeychain(authn.DefaultKeychain))
if err != nil {
panic(err)
}
// Prints the digest of registry.example.com/private/repo
fmt.Println(img.Digest)
}
```
(If you're only using [gcr.io](https://gcr.io), see the [`pkg/v1/google.Keychain`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/google#Keychain), which emulates [`docker-credential-gcr`](https://github.com/GoogleCloudPlatform/docker-credential-gcr).)
## The Config File
This file contains various configuration options for docker and is (by default) located at:
* `$HOME/.docker/config.json` (on linux and darwin), or
* `%USERPROFILE%\.docker\config.json` (on windows).
You can override this location with the `DOCKER_CONFIG` environment variable.
### Plaintext
The config file is where your credentials are stored when you invoke `docker login`, e.g. the contents may look something like this:
```json
{
"auths": {
"registry.example.com": {
"auth": "QXp1cmVEaWFtb25kOmh1bnRlcjI="
}
}
}
```
The `auths` map has an entry per registry, and the `auth` field contains your username and password encoded as [HTTP 'Basic' Auth](https://tools.ietf.org/html/rfc7617).
**NOTE**: This means that your credentials are stored _in plaintext_:
```bash
$ echo "QXp1cmVEaWFtb25kOmh1bnRlcjI=" | base64 -d
AzureDiamond:hunter2
```
For what it's worth, this config file is equivalent to:
```json
{
"auths": {
"registry.example.com": {
"username": "AzureDiamond",
"password": "hunter2"
}
}
}
```
... which is useful to know if e.g. your CI system provides you a registry username and password via environment variables and you want to populate this file manually without invoking `docker login`.
### Helpers
If you log in like this, docker will warn you that you should use a [credential helper](https://docs.docker.com/engine/reference/commandline/login/#credentials-store), and you should!
To configure a global credential helper:
```json
{
"credsStore": "osxkeychain"
}
```
To configure a per-registry credential helper:
```json
{
"credHelpers": {
"gcr.io": "gcr"
}
}
```
We use [`github.com/docker/cli/cli/config.Load`](https://godoc.org/github.com/docker/cli/cli/config#Load) to parse the config file and invoke any necessary credential helpers. This handles the logic of taking a [`ConfigFile`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/configfile/file.go#L25-L54) + registry domain and producing an [`AuthConfig`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L3-L22), which determines how we authenticate to the registry.
## Credential Helpers
The [credential helper protocol](https://github.com/docker/docker-credential-helpers) allows you to configure a binary that supplies credentials for the registry, rather than hard-coding them in the config file.
The protocol has several verbs, but the one we most care about is `get`.
For example, using the following config file:
```json
{
"credHelpers": {
"gcr.io": "gcr",
"eu.gcr.io": "gcr"
}
}
```
To acquire credentials for `gcr.io`, we look in the `credHelpers` map to find
the credential helper for `gcr.io` is `gcr`. By appending that value to
`docker-credential-`, we can get the name of the binary we need to use.
For this example, that's `docker-credential-gcr`, which must be on our `$PATH`.
We'll then invoke that binary to get credentials:
```bash
$ echo "gcr.io" | docker-credential-gcr get
{"Username":"_token","Secret":"<long access token>"}
```
You can configure the same credential helper for multiple registries, which is
why we need to pass the domain in via STDIN, e.g. if we were trying to access
`eu.gcr.io`, we'd do this instead:
```bash
$ echo "eu.gcr.io" | docker-credential-gcr get
{"Username":"_token","Secret":"<long access token>"}
```
### Debugging credential helpers
If a credential helper is configured but doesn't seem to be working, it can be
challenging to debug. Implementing a fake credential helper lets you poke around
to make it easier to see where the failure is happening.
This "implements" a credential helper with hard-coded values:
```
#!/usr/bin/env bash
echo '{"Username":"<token>","Secret":"hunter2"}'
```
This implements a credential helper that prints the output of
`docker-credential-gcr` to both stderr and whatever called it, which allows you
to snoop on another credential helper:
```
#!/usr/bin/env bash
docker-credential-gcr $@ | tee >(cat 1>&2)
```
Put those files somewhere on your path, naming them e.g.
`docker-credential-hardcoded` and `docker-credential-tee`, then modify the
config file to use them:
```json
{
"credHelpers": {
"gcr.io": "tee",
"eu.gcr.io": "hardcoded"
}
}
```
The `docker-credential-tee` trick works with both `crane` and `docker`:
```bash
$ crane manifest gcr.io/google-containers/pause > /dev/null
{"ServerURL":"","Username":"_dcgcr_1_5_0_token","Secret":"<redacted>"}
$ docker pull gcr.io/google-containers/pause
Using default tag: latest
{"ServerURL":"","Username":"_dcgcr_1_5_0_token","Secret":"<redacted>"}
latest: Pulling from google-containers/pause
a3ed95caeb02: Pull complete
4964c72cd024: Pull complete
Digest: sha256:a78c2d6208eff9b672de43f880093100050983047b7b0afe0217d3656e1b0d5f
Status: Downloaded newer image for gcr.io/google-containers/pause:latest
gcr.io/google-containers/pause:latest
```
## The Registry
There are two methods for authenticating against a registry:
[token](https://docs.docker.com/registry/spec/auth/token/) and
[oauth2](https://docs.docker.com/registry/spec/auth/oauth/).
Both methods are used to acquire an opaque `Bearer` token (or
[RegistryToken](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L21))
to use in the `Authorization` header. The registry will return a `401
Unauthorized` during the [version
check](https://github.com/opencontainers/distribution-spec/blob/2c3975d1f03b67c9a0203199038adea0413f0573/spec.md#api-version-check)
(or during normal operations) with
[Www-Authenticate](https://tools.ietf.org/html/rfc7235#section-4.1) challenge
indicating how to proceed.
### Token
If we get back an `AuthConfig` containing a [`Username/Password`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L5-L6)
or
[`Auth`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L7),
we'll use the token method for authentication:
![basic](../../images/credhelper-basic.svg)
### OAuth 2
If we get back an `AuthConfig` containing an [`IdentityToken`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L18)
we'll use the oauth2 method for authentication:
![oauth](../../images/credhelper-oauth.svg)
This happens when a credential helper returns a response with the
[`Username`](https://github.com/docker/docker-credential-helpers/blob/f78081d1f7fef6ad74ad6b79368de6348386e591/credentials/credentials.go#L16)
set to `<token>` (no, that's not a placeholder, the literal string `"<token>"`).
It is unclear why: [moby/moby#36926](https://github.com/moby/moby/issues/36926).
We only support the oauth2 `grant_type` for `refresh_token` ([#629](https://github.com/google/go-containerregistry/issues/629)),
since it's impossible to determine from the registry response whether we should
use oauth, and the token method for authentication is widely implemented by
registries.
vendor/github.com/google/go-containerregistry/pkg/authn/k8schain/k8schain.go
Generated Vendored
+6 -3
View File
@@ -17,12 +17,12 @@ package k8schain
import (
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
credentialprovider "github.com/vdemeester/k8s-pkg-credentialprovider"
credentialprovidersecrets "github.com/vdemeester/k8s-pkg-credentialprovider/secrets"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
"k8s.io/kubernetes/pkg/credentialprovider"
credentialprovidersecrets "k8s.io/kubernetes/pkg/credentialprovider/secrets"
)
// Options holds configuration data for guiding credential resolution.
@@ -38,6 +38,9 @@ type Options struct {
ImagePullSecrets []string
}
// origKeyRing is a variable so that testing can adjust it.
var origKeyRing = credentialprovider.NewDockerKeyring()
// New returns a new authn.Keychain suitable for resolving image references as
// scoped by the provided Options. It speaks to Kubernetes through the provided
// client interface.
@@ -82,7 +85,7 @@ func New(client kubernetes.Interface, opt Options) (authn.Keychain, error) {
}
// Third, extend the default keyring with the pull secrets.
kr, err := credentialprovidersecrets.MakeDockerKeyring(pullSecrets, credentialprovider.NewDockerKeyring())
kr, err := credentialprovidersecrets.MakeDockerKeyring(pullSecrets, origKeyRing)
if err != nil {
return nil, err
}
vendor/github.com/google/go-containerregistry/pkg/authn/k8schain/k8schain_aws.go
Generated Vendored
+1 -1
View File
@@ -17,5 +17,5 @@ package k8schain
import (
// aws credential provider
_ "github.com/vdemeester/k8s-pkg-credentialprovider/aws"
_ "k8s.io/kubernetes/pkg/credentialprovider/aws"
)
vendor/github.com/google/go-containerregistry/pkg/authn/k8schain/k8schain_azure.go
Generated Vendored
+1 -1
View File
@@ -17,5 +17,5 @@ package k8schain
import (
// azure credential provider
_ "github.com/vdemeester/k8s-pkg-credentialprovider/azure"
_ "k8s.io/kubernetes/pkg/credentialprovider/azure"
)
vendor/github.com/google/go-containerregistry/pkg/authn/k8schain/k8schain_gcp.go
Generated Vendored
+1 -1
View File
@@ -17,5 +17,5 @@ package k8schain
import (
// gcp credential provider
_ "github.com/vdemeester/k8s-pkg-credentialprovider/gcp"
_ "k8s.io/kubernetes/pkg/credentialprovider/gcp"
)
vendor/github.com/google/go-containerregistry/pkg/authn/keychain.go
Generated Vendored
+2 -4
View File
@@ -52,9 +52,7 @@ var (
)
const (
// DefaultAuthKey is the key used for dockerhub in config files, which
// is hardcoded for historical reasons.
DefaultAuthKey = "https://" + name.DefaultRegistry + "/v1/"
defaultAuthKey = "https://" + name.DefaultRegistry + "/v1/"
)
// Resolve implements Keychain.
@@ -69,7 +67,7 @@ func (dk *defaultKeychain) Resolve(target Resource) (Authenticator, error) {
// https://github.com/moby/moby/blob/fc01c2b481097a6057bec3cd1ab2d7b4488c50c4/registry/config.go#L397-L404
key := target.RegistryStr()
if key == name.DefaultRegistry {
key = DefaultAuthKey
key = defaultAuthKey
}
cfg, err := cf.GetAuthConfig(key)
vendor/github.com/google/go-containerregistry/pkg/name/README.md
Generated Vendored
-3
View File
@@ -1,3 +0,0 @@
# `name`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/name?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/name)
vendor/github.com/google/go-containerregistry/pkg/v1/config.go
Generated Vendored
+1 -4
View File
@@ -23,9 +23,6 @@ import (
// ConfigFile is the configuration file that holds the metadata describing
// how to launch a container. See:
// https://github.com/opencontainers/image-spec/blob/master/config.md
//
// docker_version and os.version are not part of the spec but included
// for backwards compatibility.
type ConfigFile struct {
Architecture string `json:"architecture"`
Author string `json:"author,omitempty"`
@@ -36,7 +33,7 @@ type ConfigFile struct {
OS string `json:"os"`
RootFS RootFS `json:"rootfs"`
Config Config `json:"config"`
OSVersion string `json:"os.version,omitempty"`
OSVersion string `json:"osversion,omitempty"`
}
// History is one entry of a list recording how this container image was built.
vendor/github.com/google/go-containerregistry/pkg/v1/daemon/README.md
Generated Vendored
-11
View File
@@ -1,11 +0,0 @@
# `daemon`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/daemon?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/daemon)
The `daemon` package enables reading/writing images from/to the docker daemon.
It is not fully fleshed out, but is useful for interoperability, see various issues:
* https://github.com/google/go-containerregistry/issues/205
* https://github.com/google/go-containerregistry/issues/552
* https://github.com/google/go-containerregistry/issues/627
vendor/github.com/google/go-containerregistry/pkg/v1/daemon/image.go
Generated Vendored
+28 -20
View File
@@ -36,7 +36,6 @@ var _ v1.Image = (*image)(nil)
type imageOpener struct {
ref name.Reference
buffered bool
client Client
}
// ImageOption is a functional option for Image.
@@ -46,16 +45,14 @@ func (i *imageOpener) Open() (v1.Image, error) {
var opener tarball.Opener
var err error
if i.buffered {
opener, err = i.bufferedOpener(i.ref)
opener, err = bufferedOpener(i.ref)
} else {
opener, err = i.unbufferedOpener(i.ref)
opener, err = unbufferedOpener(i.ref)
}
if err != nil {
return nil, err
}
i.client.NegotiateAPIVersion(context.Background())
tb, err := tarball.Image(opener, nil)
if err != nil {
return nil, err
@@ -66,13 +63,33 @@ func (i *imageOpener) Open() (v1.Image, error) {
return img, nil
}
func (i *imageOpener) saveImage(ref name.Reference) (io.ReadCloser, error) {
return i.client.ImageSave(context.Background(), []string{ref.Name()})
// ImageSaver is an interface for testing.
type ImageSaver interface {
ImageSave(context.Context, []string) (io.ReadCloser, error)
}
func (i *imageOpener) bufferedOpener(ref name.Reference) (tarball.Opener, error) {
// This is a variable so we can override in tests.
var getImageSaver = func() (ImageSaver, error) {
cli, err := client.NewClientWithOpts(client.FromEnv)
if err != nil {
return nil, err
}
cli.NegotiateAPIVersion(context.Background())
return cli, nil
}
func saveImage(ref name.Reference) (io.ReadCloser, error) {
cli, err := getImageSaver()
if err != nil {
return nil, err
}
return cli.ImageSave(context.Background(), []string{ref.Name()})
}
func bufferedOpener(ref name.Reference) (tarball.Opener, error) {
// Store the tarball in memory and return a new reader into the bytes each time we need to access something.
rc, err := i.saveImage(ref)
rc, err := saveImage(ref)
if err != nil {
return nil, err
}
@@ -89,10 +106,10 @@ func (i *imageOpener) bufferedOpener(ref name.Reference) (tarball.Opener, error)
}, nil
}
func (i *imageOpener) unbufferedOpener(ref name.Reference) (tarball.Opener, error) {
func unbufferedOpener(ref name.Reference) (tarball.Opener, error) {
// To avoid storing the tarball in memory, do a save every time we need to access something.
return func() (io.ReadCloser, error) {
return i.saveImage(ref)
return saveImage(ref)
}, nil
}
@@ -109,14 +126,5 @@ func Image(ref name.Reference, options ...ImageOption) (v1.Image, error) {
return nil, err
}
}
if i.client == nil {
var err error
i.client, err = client.NewClientWithOpts(client.FromEnv)
if err != nil {
return nil, err
}
}
return i.Open()
}
vendor/github.com/google/go-containerregistry/pkg/v1/daemon/options.go
Generated Vendored
-26
View File
@@ -14,13 +14,6 @@
package daemon
import (
"context"
"io"
"github.com/docker/docker/api/types"
)
// WithBufferedOpener buffers the image.
func WithBufferedOpener() ImageOption {
return func(i *imageOpener) error {
@@ -39,22 +32,3 @@ func (i *imageOpener) setBuffered(buffer bool) error {
i.buffered = buffer
return nil
}
// WithClient is a functional option to allow injecting a docker client.
//
// By default, github.com/docker/docker/client.FromEnv is used.
func WithClient(client Client) ImageOption {
return func(i *imageOpener) error {
i.client = client
return nil
}
}
// Client represents the subset of a docker client that the daemon
// package uses.
type Client interface {
NegotiateAPIVersion(ctx context.Context)
ImageSave(context.Context, []string) (io.ReadCloser, error)
ImageLoad(context.Context, io.Reader, bool) (types.ImageLoadResponse, error)
ImageTag(context.Context, string, string) error
}
vendor/github.com/google/go-containerregistry/pkg/v1/empty/README.md
Generated Vendored
-8
View File
@@ -1,8 +0,0 @@
# `empty`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/empty?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/empty)
The empty packages provides an empty base for constructing a `v1.Image` or `v1.ImageIndex`.
This is especially useful when paired with the [`mutate`](/pkg/v1/mutate) package,
see [`mutate.Append`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate#Append)
and [`mutate.AppendManifests`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate#AppendManifests).
vendor/github.com/google/go-containerregistry/pkg/v1/image.go
Generated Vendored
+2 -3
View File
@@ -31,14 +31,13 @@ type Image interface {
// Size returns the size of the manifest.
Size() (int64, error)
// ConfigName returns the hash of the image's config file, also known as
// the Image ID.
// ConfigName returns the hash of the image's config file.
ConfigName() (Hash, error)
// ConfigFile returns this image's config file.
ConfigFile() (*ConfigFile, error)
// RawConfigFile returns the serialized bytes of ConfigFile().
// RawConfigFile returns the serialized bytes of ConfigFile()
RawConfigFile() ([]byte, error)
// Digest returns the sha256 of this image's manifest.
vendor/github.com/google/go-containerregistry/pkg/v1/layout/README.md
Generated Vendored
-5
View File
@@ -1,5 +0,0 @@
# `layout`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/layout?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/layout)
The `layout` package implements support for interacting with an [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/master/image-layout.md).
vendor/github.com/google/go-containerregistry/pkg/v1/layout/write.go
Generated Vendored
+6 -10
View File
@@ -124,19 +124,15 @@ func (l Path) AppendDescriptor(desc v1.Descriptor) error {
return err
}
return l.WriteFile("index.json", rawIndex, os.ModePerm)
return l.writeFile("index.json", rawIndex)
}
// WriteFile write a file with arbitrary data at an arbitrary location in a v1
// layout. Used mostly internally to write files like "oci-layout" and
// "index.json", also can be used to write other arbitrary files. Do *not* use
// this to write blobs. Use only WriteBlob() for that.
func (l Path) WriteFile(name string, data []byte, perm os.FileMode) error {
func (l Path) writeFile(name string, data []byte) error {
if err := os.MkdirAll(l.path(), os.ModePerm); err != nil && !os.IsExist(err) {
return err
}
return ioutil.WriteFile(l.path(name), data, perm)
return ioutil.WriteFile(l.path(name), data, os.ModePerm)
}
@@ -263,12 +259,12 @@ func (l Path) writeIndexToFile(indexFile string, ii v1.ImageIndex) error {
return err
}
return l.WriteFile(indexFile, rawIndex, os.ModePerm)
return l.writeFile(indexFile, rawIndex)
}
func (l Path) writeIndex(ii v1.ImageIndex) error {
// Always just write oci-layout file, since it's small.
if err := l.WriteFile("oci-layout", []byte(layoutFile), os.ModePerm); err != nil {
if err := l.writeFile("oci-layout", []byte(layoutFile)); err != nil {
return err
}
@@ -295,7 +291,7 @@ func (l Path) writeIndex(ii v1.ImageIndex) error {
func Write(path string, ii v1.ImageIndex) (Path, error) {
lp := Path(path)
// Always just write oci-layout file, since it's small.
if err := lp.WriteFile("oci-layout", []byte(layoutFile), os.ModePerm); err != nil {
if err := lp.writeFile("oci-layout", []byte(layoutFile)); err != nil {
return "", err
}
vendor/github.com/google/go-containerregistry/pkg/v1/mutate/README.md
Generated Vendored
-56
View File
@@ -1,56 +0,0 @@
# `mutate`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate)
The `v1.Image`, `v1.ImageIndex`, and `v1.Layer` interfaces provide only
accessor methods, so they are essentially immutable. If you want to change
something about them, you need to produce a new instance of that interface.
A common use case for this library is to read an image from somewhere (a source),
change something about it, and write the image somewhere else (a sink).
Graphically, this looks something like:
<p align="center">
<img src="/images/mutate.dot.svg" />
</p>
## Mutations
This is obviously not a comprehensive set of useful transformations (PRs welcome!),
but a rough summary of what the `mutate` package currently does:
### `Config` and `ConfigFile`
These allow you to change the [image configuration](https://github.com/opencontainers/image-spec/blob/master/config.md#properties),
e.g. to change the entrypoint, environment, author, etc.
### `Time`, `Canonical`, and `CreatedAt`
These are useful in the context of [reproducible builds](https://reproducible-builds.org/),
where you may want to strip timestamps and other non-reproducible information.
### `Append`, `AppendLayers`, and `AppendManifests`
These functions allow the extension of a `v1.Image` or `v1.ImageIndex` with
new layers or manifests.
For constructing an image `FROM scratch`, see the [`empty`](/pkg/v1/empty) package.
### `MediaType` and `IndexMediaType`
Sometimes, it is necessary to change the media type of an image or index,
e.g. to appease a registry with strict validation of images (_looking at you, GCR_).
### `Rebase`
Rebase has [its own README](/cmd/crane/rebase.md).
This is the underlying implementation of [`crane rebase`](https://github.com/google/go-containerregistry/blob/master/cmd/crane/doc/crane_rebase.md).
### `Extract`
Extract will flatten an image filesystem into a single tar stream,
respecting whiteout files.
This is the underlying implementation of [`crane export`](https://github.com/google/go-containerregistry/blob/master/cmd/crane/doc/crane_export.md).
vendor/github.com/google/go-containerregistry/pkg/v1/mutate/image.go
Generated Vendored
+17 -16
View File
@@ -85,21 +85,26 @@ func (i *image) compute() error {
manifest := m.DeepCopy()
manifestLayers := manifest.Layers
for _, add := range i.adds {
desc, err := partial.Descriptor(add.Layer)
if err != nil {
d := v1.Descriptor{}
var err error
if d.Size, err = add.Layer.Size(); err != nil {
return err
}
// Fields in the addendum override the original descriptor.
if len(add.Annotations) != 0 {
desc.Annotations = add.Annotations
}
if len(add.URLs) != 0 {
desc.URLs = add.URLs
if d.Digest, err = add.Layer.Digest(); err != nil {
return err
}
manifestLayers = append(manifestLayers, *desc)
digestMap[desc.Digest] = add.Layer
if d.MediaType, err = add.Layer.MediaType(); err != nil {
return err
}
d.Annotations = add.Annotations
d.URLs = add.URLs
manifestLayers = append(manifestLayers, d)
digestMap[d.Digest] = add.Layer
}
configFile.RootFS.DiffIDs = diffIDs
@@ -120,12 +125,8 @@ func (i *image) compute() error {
// With OCI media types, this should not be set, see discussion:
// https://github.com/opencontainers/image-spec/pull/795
if i.mediaType != nil {
if strings.Contains(string(*i.mediaType), types.OCIVendorPrefix) {
manifest.MediaType = ""
} else if strings.Contains(string(*i.mediaType), types.DockerVendorPrefix) {
manifest.MediaType = *i.mediaType
}
if i.mediaType != nil && strings.Contains(string(*i.mediaType), types.OCIVendorPrefix) {
manifest.MediaType = ""
}
i.configFile = configFile
vendor/github.com/google/go-containerregistry/pkg/v1/mutate/index.go
Generated Vendored
+21 -27
View File
@@ -24,33 +24,31 @@ import (
"github.com/google/go-containerregistry/pkg/v1/types"
)
func computeDescriptor(ia IndexAddendum) (*v1.Descriptor, error) {
desc, err := partial.Descriptor(ia.Add)
func computeDescriptor(desc v1.Descriptor, add Appendable) (*v1.Descriptor, error) {
d, err := add.Digest()
if err != nil {
return nil, err
}
mt, err := add.MediaType()
if err != nil {
return nil, err
}
sz, err := add.Size()
if err != nil {
return nil, err
}
// The IndexAddendum allows overriding Descriptor values.
if ia.Descriptor.Size != 0 {
desc.Size = ia.Descriptor.Size
// The IndexAddendum allows overriding These values.
if desc.Size == 0 {
desc.Size = sz
}
if string(ia.Descriptor.MediaType) != "" {
desc.MediaType = ia.Descriptor.MediaType
if string(desc.MediaType) == "" {
desc.MediaType = mt
}
if ia.Descriptor.Digest != (v1.Hash{}) {
desc.Digest = ia.Descriptor.Digest
if desc.Digest == (v1.Hash{}) {
desc.Digest = d
}
if ia.Descriptor.Platform != nil {
desc.Platform = ia.Descriptor.Platform
}
if len(ia.Descriptor.URLs) != 0 {
desc.URLs = ia.Descriptor.URLs
}
if len(ia.Descriptor.Annotations) != 0 {
desc.Annotations = ia.Descriptor.Annotations
}
return desc, nil
return &desc, nil
}
type index struct {
@@ -91,7 +89,7 @@ func (i *index) compute() error {
manifest := m.DeepCopy()
manifests := manifest.Manifests
for _, add := range i.adds {
desc, err := computeDescriptor(add)
desc, err := computeDescriptor(add.Descriptor, add.Add)
if err != nil {
return err
}
@@ -109,12 +107,8 @@ func (i *index) compute() error {
// With OCI media types, this should not be set, see discussion:
// https://github.com/opencontainers/image-spec/pull/795
if i.mediaType != nil {
if strings.Contains(string(*i.mediaType), types.OCIVendorPrefix) {
manifest.MediaType = ""
} else if strings.Contains(string(*i.mediaType), types.DockerVendorPrefix) {
manifest.MediaType = *i.mediaType
}
if i.mediaType != nil && strings.Contains(string(*i.mediaType), types.OCIVendorPrefix) {
manifest.MediaType = ""
}
i.manifest = manifest
vendor/github.com/google/go-containerregistry/pkg/v1/mutate/mutate.go
Generated Vendored
+2 -4
View File
@@ -140,6 +140,8 @@ func CreatedAt(base v1.Image, created v1.Time) (v1.Image, error) {
//
// If a caller doesn't read the full contents, they should Close it to free up
// resources used during extraction.
//
// Adapted from https://github.com/google/containerregistry/blob/master/client/v2_2/docker_image_.py#L731
func Extract(img v1.Image) io.ReadCloser {
pr, pw := io.Pipe()
@@ -154,7 +156,6 @@ func Extract(img v1.Image) io.ReadCloser {
return pr
}
// Adapted from https://github.com/google/containerregistry/blob/da03b395ccdc4e149e34fbb540483efce962dc64/client/v2_2/docker_image_.py#L816
func extract(img v1.Image, w io.Writer) error {
tarWriter := tar.NewWriter(w)
defer tarWriter.Close()
@@ -279,9 +280,6 @@ func Time(img v1.Image, t time.Time) (v1.Image, error) {
cfg := cf.DeepCopy()
// Copy basic config over
cfg.Architecture = ocf.Architecture
cfg.Author = ocf.Author
cfg.OS = ocf.OS
cfg.Config = ocf.Config
// Strip away timestamps from the config file
vendor/github.com/google/go-containerregistry/pkg/v1/partial/README.md
Generated Vendored
-66
View File
@@ -1,66 +0,0 @@
# `partial`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial)
## Partial Implementations
There are roughly two kinds of image representations: compressed and uncompressed.
The implementations for these kinds of images are almost identical, with the only
major difference being how blobs (config and layers) are fetched. This common
code lives in this package, where you provide a _partial_ implementation of a
compressed or uncompressed image, and you get back a full `v1.Image` implementation.
### Examples
In a registry, blobs are compressed, so it's easiest to implement a `v1.Image` in terms
of compressed layers. `remote.remoteImage` does this by implementing `CompressedImageCore`:
```go
type CompressedImageCore interface {
RawConfigFile() ([]byte, error)
MediaType() (types.MediaType, error)
RawManifest() ([]byte, error)
LayerByDigest(v1.Hash) (CompressedLayer, error)
}
```
In a tarball, blobs are (often) uncompressed, so it's easiest to implement a `v1.Image` in terms
of uncompressed layers. `tarball.uncompressedImage` does this by implementing `UncompressedImageCore`:
```go
type CompressedImageCore interface {
RawConfigFile() ([]byte, error)
MediaType() (types.MediaType, error)
LayerByDiffID(v1.Hash) (UncompressedLayer, error)
}
```
## Optional Methods
Where possible, we access some information via optional methods as an optimization.
### [`partial.Descriptor`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial#Descriptor)
There are some properties of a [`Descriptor`](https://github.com/opencontainers/image-spec/blob/master/descriptor.md#properties) that aren't derivable from just image data:
* `MediaType`
* `Platform`
* `URLs`
* `Annotations`
For example, in a `tarball.Image`, there is a `LayerSources` field that contains
an entire layer descriptor with `URLs` information for foreign layers. This
information can be passed through to callers by implementing this optional
`Descriptor` method.
See [`#654`](https://github.com/google/go-containerregistry/pull/654).
### [`partial.UncompressedSize`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial#UncompressedSize)
Usually, you don't need to know the uncompressed size of a layer, since that
information isn't stored in a config file (just he sha256 is needed); however,
there are cases where it is very helpful to know the layer size, e.g. when
writing the uncompressed layer into a tarball.
See [`#655`](https://github.com/google/go-containerregistry/pull/655).
vendor/github.com/google/go-containerregistry/pkg/v1/partial/compressed.go
Generated Vendored
+7 -7
View File
@@ -44,22 +44,22 @@ type compressedLayerExtender struct {
}
// Uncompressed implements v1.Layer
func (cle *compressedLayerExtender) Uncompressed() (io.ReadCloser, error) {
r, err := cle.Compressed()
func (ule *compressedLayerExtender) Uncompressed() (io.ReadCloser, error) {
u, err := ule.Compressed()
if err != nil {
return nil, err
}
return v1util.GunzipReadCloser(r)
return v1util.GunzipReadCloser(u)
}
// DiffID implements v1.Layer
func (cle *compressedLayerExtender) DiffID() (v1.Hash, error) {
func (ule *compressedLayerExtender) DiffID() (v1.Hash, error) {
// If our nested CompressedLayer implements DiffID,
// then delegate to it instead.
if wdi, ok := cle.CompressedLayer.(WithDiffID); ok {
if wdi, ok := ule.CompressedLayer.(WithDiffID); ok {
return wdi.DiffID()
}
r, err := cle.Uncompressed()
r, err := ule.Uncompressed()
if err != nil {
return v1.Hash{}, err
}
@@ -76,7 +76,7 @@ func CompressedToLayer(ul CompressedLayer) (v1.Layer, error) {
// CompressedImageCore represents the base minimum interface a natively
// compressed image must implement for us to produce a v1.Image.
type CompressedImageCore interface {
ImageCore
imageCore
// RawManifest returns the serialized bytes of the manifest.
RawManifest() ([]byte, error)
vendor/github.com/google/go-containerregistry/pkg/v1/partial/image.go
Generated Vendored
+2 -2
View File
@@ -18,8 +18,8 @@ import (
"github.com/google/go-containerregistry/pkg/v1/types"
)
// ImageCore is the core set of properties without which we cannot build a v1.Image
type ImageCore interface {
// imageCore is the core set of properties without which we cannot build a v1.Image
type imageCore interface {
// RawConfigFile returns the serialized bytes of this image's config file.
RawConfigFile() ([]byte, error)
vendor/github.com/google/go-containerregistry/pkg/v1/partial/uncompressed.go
Generated Vendored
+44 -3
View File
@@ -81,6 +81,30 @@ func (ule *uncompressedLayerExtender) calcSizeHash() {
})
}
// Descriptor returns the layer manifest descriptor for this uncompressed layer.
// The embedded UncompressedLayer is checked to see if it has a Descriptor
// function which is returned if it exists. The underlying descriptor provides
// foreign layer data like URLs. Otherwise, a new descriptor is
// generated.
func (ule *uncompressedLayerExtender) Descriptor() (*v1.Descriptor, error) {
if withDesc, ok := ule.UncompressedLayer.(withDescriptor); ok {
return withDesc.Descriptor()
}
ule.calcSizeHash()
if ule.hashSizeError != nil {
return nil, ule.hashSizeError
}
mt, err := ule.MediaType()
if err != nil {
return nil, err
}
return &v1.Descriptor{
MediaType: mt,
Size: ule.size,
Digest: ule.hash,
}, nil
}
// UncompressedToLayer fills in the missing methods from an UncompressedLayer so that it implements v1.Layer
func UncompressedToLayer(ul UncompressedLayer) (v1.Layer, error) {
return &uncompressedLayerExtender{UncompressedLayer: ul}, nil
@@ -89,7 +113,7 @@ func UncompressedToLayer(ul UncompressedLayer) (v1.Layer, error) {
// UncompressedImageCore represents the bare minimum interface a natively
// uncompressed image must implement for us to produce a v1.Image
type UncompressedImageCore interface {
ImageCore
imageCore
// LayerByDiffID is a variation on the v1.Image method, which returns
// an UncompressedLayer instead.
@@ -155,12 +179,29 @@ func (i *uncompressedImageExtender) Manifest() (*v1.Manifest, error) {
m.Layers = make([]v1.Descriptor, len(ls))
for i, l := range ls {
desc, err := Descriptor(l)
// Check if the layer implementation provides the descriptor directly.
if withDesc, ok := l.(withDescriptor); ok {
desc, err := withDesc.Descriptor()
if err != nil {
return nil, err
}
m.Layers[i] = *desc
continue
}
sz, err := l.Size()
if err != nil {
return nil, err
}
h, err := l.Digest()
if err != nil {
return nil, err
}
m.Layers[i] = *desc
m.Layers[i] = v1.Descriptor{
MediaType: types.DockerLayer,
Size: sz,
Digest: h,
}
}
i.manifest = m
vendor/github.com/google/go-containerregistry/pkg/v1/partial/with.go
Generated Vendored
+11 -102
View File
@@ -129,6 +129,12 @@ func RawConfigFile(i WithConfigFile) ([]byte, error) {
return json.Marshal(cfg)
}
// WithUncompressedLayer defines the subset of v1.Image used by these helper methods
type WithUncompressedLayer interface {
// UncompressedLayer is like UncompressedBlob, but takes the "diff id".
UncompressedLayer(v1.Hash) (io.ReadCloser, error)
}
// WithRawManifest defines the subset of v1.Image used by these helper methods
type WithRawManifest interface {
// RawManifest returns the serialized bytes of this image's config file.
@@ -201,22 +207,22 @@ func BlobSize(i WithManifest, h v1.Hash) (int64, error) {
}
// BlobDescriptor is a helper for implementing v1.Image
func BlobDescriptor(i WithManifest, h v1.Hash) (*v1.Descriptor, error) {
func BlobDescriptor(i WithManifest, h v1.Hash) (v1.Descriptor, error) {
m, err := i.Manifest()
if err != nil {
return nil, err
return v1.Descriptor{}, err
}
if m.Config.Digest == h {
return &m.Config, nil
return m.Config, nil
}
for _, l := range m.Layers {
if l.Digest == h {
return &l, nil
return l, nil
}
}
return nil, fmt.Errorf("blob %v not found", h)
return v1.Descriptor{}, fmt.Errorf("blob %v not found", h)
}
// WithManifestAndConfigFile defines the subset of v1.Image used by these helper methods
@@ -283,100 +289,3 @@ type WithDiffID interface {
type withDescriptor interface {
Descriptor() (*v1.Descriptor, error)
}
// Describable represents something for which we can produce a v1.Descriptor.
type Describable interface {
Digest() (v1.Hash, error)
MediaType() (types.MediaType, error)
Size() (int64, error)
}
// Descriptor returns a v1.Descriptor given a Describable. It also encodes
// some logic for unwrapping things that have been wrapped by
// CompressedToLayer, UncompressedToLayer, CompressedToImage, or
// UncompressedToImage.
func Descriptor(d Describable) (*v1.Descriptor, error) {
// If Describable implements Descriptor itself, return that.
if wd, ok := d.(withDescriptor); ok {
return wd.Descriptor()
}
// Otherwise, try to unwrap any partial implementations to see
// if the wrapped struct implements Descriptor.
if ule, ok := d.(*uncompressedLayerExtender); ok {
if wd, ok := ule.UncompressedLayer.(withDescriptor); ok {
return wd.Descriptor()
}
}
if cle, ok := d.(*compressedLayerExtender); ok {
if wd, ok := cle.CompressedLayer.(withDescriptor); ok {
return wd.Descriptor()
}
}
if uie, ok := d.(*uncompressedImageExtender); ok {
if wd, ok := uie.UncompressedImageCore.(withDescriptor); ok {
return wd.Descriptor()
}
}
if cie, ok := d.(*compressedImageExtender); ok {
if wd, ok := cie.CompressedImageCore.(withDescriptor); ok {
return wd.Descriptor()
}
}
// If all else fails, compute the descriptor from the individual methods.
var (
desc v1.Descriptor
err error
)
if desc.Size, err = d.Size(); err != nil {
return nil, err
}
if desc.Digest, err = d.Digest(); err != nil {
return nil, err
}
if desc.MediaType, err = d.MediaType(); err != nil {
return nil, err
}
return &desc, nil
}
type withUncompressedSize interface {
UncompressedSize() (int64, error)
}
// UncompressedSize returns the size of the Uncompressed layer. If the
// underlying implementation doesn't implement UncompressedSize directly,
// this will compute the uncompressedSize by reading everything returned
// by Compressed(). This is potentially expensive and may consume the contents
// for streaming layers.
func UncompressedSize(l v1.Layer) (int64, error) {
// If the layer implements UncompressedSize itself, return that.
if wus, ok := l.(withUncompressedSize); ok {
return wus.UncompressedSize()
}
// Otherwise, try to unwrap any partial implementations to see
// if the wrapped struct implements UncompressedSize.
if ule, ok := l.(*uncompressedLayerExtender); ok {
if wus, ok := ule.UncompressedLayer.(withUncompressedSize); ok {
return wus.UncompressedSize()
}
}
if cle, ok := l.(*compressedLayerExtender); ok {
if wus, ok := cle.CompressedLayer.(withUncompressedSize); ok {
return wus.UncompressedSize()
}
}
// The layer doesn't implement UncompressedSize, we need to compute it.
rc, err := l.Uncompressed()
if err != nil {
return -1, err
}
defer rc.Close()
return io.Copy(ioutil.Discard, rc)
}
vendor/github.com/google/go-containerregistry/pkg/v1/remote/README.md
Generated Vendored
-117
View File
@@ -1,117 +0,0 @@
# `remote`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote)
The `remote` package implements a client for accessing a registry,
per the [OCI distribution spec](https://github.com/opencontainers/distribution-spec/blob/master/spec.md).
It leans heavily on the lower level [`transport`](/pkg/v1/remote/transport) package, which handles the
authentication handshake and structured errors.
## Usage
```go
package main
import (
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func main() {
ref, err := name.ParseReference("gcr.io/google-containers/pause")
if err != nil {
panic(err)
}
img, err := remote.Image(ref, remote.WithAuthFromKeychain(authn.DefaultKeychain))
if err != nil {
panic(err)
}
// do stuff with img
}
```
## Structure
<p align="center">
<img src="/images/remote.dot.svg" />
</p>
## Background
There are a lot of confusingly similar terms that come up when talking about images in registries.
### Anatomy of an image
In general...
* A tag refers to an image manifest.
* An image manifest references a config file and an orderered list of _compressed_ layers by sha256 digest.
* A config file references an ordered list of _uncompressed_ layers by sha256 digest and contains runtime configuration.
* The sha256 digest of the config file is the [image id](https://github.com/opencontainers/image-spec/blob/master/config.md#imageid) for the image.
For example, an image with two layers would look something like this:
![image anatomy](/images/image-anatomy.dot.svg)
### Anatomy of an index
In the normal case, an [index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) is used to represent a multi-platform image.
This was the original use case for a [manifest
list](https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list).
![image index anatomy](/images/index-anatomy.dot.svg)
It is possible for an index to reference another index, per the OCI
[image-spec](https://github.com/opencontainers/image-spec/blob/master/media-types.md#compatibility-matrix).
In theory, both an image and image index can reference arbitrary things via
[descriptors](https://github.com/opencontainers/image-spec/blob/master/descriptor.md),
e.g. see the [image layout
example](https://github.com/opencontainers/image-spec/blob/master/image-layout.md#index-example),
which references an application/xml file from an image index.
That could look something like this:
![exotic image index anatomy](/images/index-anatomy-exotic.dot.svg)
Using a recursive index like this might not be possible with all registries,
but this flexibility allows for some interesting applications, e.g. the
[OCI Artifacts](https://github.com/opencontainers/artifacts) effort.
### Anatomy of an image upload
The structure of an image requires a delicate ordering when uploading an image to a registry.
Below is a (slightly simplified) figure that describes how an image is prepared for upload
to a registry and how the data flows between various artifacts:
![upload](/images/upload.dot.svg)
Note that:
* A config file references the uncompressed layer contents by sha256.
* A manifest references the compressed layer contents by sha256 and the size of the layer.
* A manifest references the config file contents by sha256 and the size of the file.
It follows that during an upload, we need to upload layers before the config file,
and we need to upload the config file before the manifest.
Sometimes, we know all of this information ahead of time, (e.g. when copying from remote.Image),
so the ordering is less important.
In other cases, e.g. when using a [`stream.Layer`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/stream#Layer),
we can't compute anything until we have already uploaded the layer, so we need to be careful about ordering.
## Caveats
### schema 1
This package does not support schema 1 images, see [`#377`](https://github.com/google/go-containerregistry/issues/377),
however, it's possible to do _something_ useful with them via [`remote.Get`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote#Get),
which doesn't try to interpret what is returned by the registry.
[`crane.Copy`](https://godoc.org/github.com/google/go-containerregistry/pkg/crane#Copy) takes advantage of this to implement support for copying schema 1 images,
see [here](https://github.com/google/go-containerregistry/blob/master/pkg/internal/legacy/copy.go).
vendor/github.com/google/go-containerregistry/pkg/v1/remote/check.go
Generated Vendored
+1 -1
View File
@@ -34,7 +34,7 @@ func CheckPushPermission(ref name.Reference, kc authn.Keychain, t http.RoundTrip
// authorize a push. Figure out how to return early here when we can,
// to avoid a roundtrip for spec-compliant registries.
w := writer{
repo: ref.Context(),
ref: ref,
client: &http.Client{Transport: tr},
}
loc, _, err := w.initiateUpload("", "")
vendor/github.com/google/go-containerregistry/pkg/v1/remote/descriptor.go
Generated Vendored
+8 -8
View File
@@ -77,12 +77,14 @@ func (d *Descriptor) RawManifest() ([]byte, error) {
// querying what kind of artifact a reference represents.
func Get(ref name.Reference, options ...Option) (*Descriptor, error) {
acceptable := []types.MediaType{
types.DockerManifestSchema2,
types.OCIManifestSchema1,
types.DockerManifestList,
types.OCIImageIndex,
// Just to look at them.
types.DockerManifestSchema1,
types.DockerManifestSchema1Signed,
}
acceptable = append(acceptable, acceptableImageMediaTypes...)
acceptable = append(acceptable, acceptableIndexMediaTypes...)
return get(ref, acceptable, options...)
}
@@ -172,9 +174,8 @@ func (d *Descriptor) remoteImage() *remoteImage {
Ref: d.Ref,
Client: d.Client,
},
manifest: d.Manifest,
mediaType: d.MediaType,
descriptor: &d.Descriptor,
manifest: d.Manifest,
mediaType: d.MediaType,
}
}
@@ -184,9 +185,8 @@ func (d *Descriptor) remoteIndex() *remoteIndex {
Ref: d.Ref,
Client: d.Client,
},
manifest: d.Manifest,
mediaType: d.MediaType,
descriptor: &d.Descriptor,
manifest: d.Manifest,
mediaType: d.MediaType,
}
}
vendor/github.com/google/go-containerregistry/pkg/v1/remote/image.go
Generated Vendored
+5 -25
View File
@@ -29,11 +29,6 @@ import (
"github.com/google/go-containerregistry/pkg/v1/v1util"
)
var acceptableImageMediaTypes = []types.MediaType{
types.DockerManifestSchema2,
types.OCIManifestSchema1,
}
// remoteImage accesses an image from a remote registry
type remoteImage struct {
fetcher
@@ -42,7 +37,6 @@ type remoteImage struct {
configLock sync.Mutex // Protects config
config []byte
mediaType types.MediaType
descriptor *v1.Descriptor
}
var _ partial.CompressedImageCore = (*remoteImage)(nil)
@@ -74,14 +68,15 @@ func (r *remoteImage) RawManifest() ([]byte, error) {
// NOTE(jonjohnsonjr): We should never get here because the public entrypoints
// do type-checking via remote.Descriptor. I've left this here for tests that
// directly instantiate a remoteImage.
manifest, desc, err := r.fetchManifest(r.Ref, acceptableImageMediaTypes)
acceptable := []types.MediaType{
types.DockerManifestSchema2,
types.OCIManifestSchema1,
}
manifest, desc, err := r.fetchManifest(r.Ref, acceptable)
if err != nil {
return nil, err
}
if r.descriptor == nil {
r.descriptor = desc
}
r.mediaType = desc.MediaType
r.manifest = manifest
return r.manifest, nil
@@ -112,15 +107,6 @@ func (r *remoteImage) RawConfigFile() ([]byte, error) {
return r.config, nil
}
// Descriptor retains the original descriptor from an index manifest.
// See partial.Descriptor.
func (r *remoteImage) Descriptor() (*v1.Descriptor, error) {
// kind of a hack, but RawManifest does appropriate locking/memoization
// and makes sure r.descriptor is populated.
_, err := r.RawManifest()
return r.descriptor, err
}
// remoteImageLayer implements partial.CompressedLayer
type remoteImageLayer struct {
ri *remoteImage
@@ -206,12 +192,6 @@ func (rl *remoteImageLayer) DiffID() (v1.Hash, error) {
return partial.BlobToDiffID(rl, rl.digest)
}
// Descriptor retains the original descriptor from an image manifest.
// See partial.Descriptor.
func (rl *remoteImageLayer) Descriptor() (*v1.Descriptor, error) {
return partial.BlobDescriptor(rl, rl.digest)
}
// LayerByDigest implements partial.CompressedLayer
func (r *remoteImage) LayerByDigest(h v1.Hash) (partial.CompressedLayer, error) {
return &remoteImageLayer{
vendor/github.com/google/go-containerregistry/pkg/v1/remote/index.go
Generated Vendored
+13 -22
View File
@@ -25,23 +25,22 @@ import (
"github.com/google/go-containerregistry/pkg/v1/types"
)
var acceptableIndexMediaTypes = []types.MediaType{
types.DockerManifestList,
types.OCIImageIndex,
}
// remoteIndex accesses an index from a remote registry
type remoteIndex struct {
fetcher
manifestLock sync.Mutex // Protects manifest
manifest []byte
mediaType types.MediaType
descriptor *v1.Descriptor
}
// Index provides access to a remote index reference.
func Index(ref name.Reference, options ...Option) (v1.ImageIndex, error) {
desc, err := get(ref, acceptableIndexMediaTypes, options...)
acceptable := []types.MediaType{
types.DockerManifestList,
types.OCIImageIndex,
}
desc, err := get(ref, acceptable, options...)
if err != nil {
return nil, err
}
@@ -74,14 +73,15 @@ func (r *remoteIndex) RawManifest() ([]byte, error) {
// NOTE(jonjohnsonjr): We should never get here because the public entrypoints
// do type-checking via remote.Descriptor. I've left this here for tests that
// directly instantiate a remoteIndex.
manifest, desc, err := r.fetchManifest(r.Ref, acceptableIndexMediaTypes)
acceptable := []types.MediaType{
types.DockerManifestList,
types.OCIImageIndex,
}
manifest, desc, err := r.fetchManifest(r.Ref, acceptable)
if err != nil {
return nil, err
}
if r.descriptor == nil {
r.descriptor = desc
}
r.mediaType = desc.MediaType
r.manifest = manifest
return r.manifest, nil
@@ -105,15 +105,6 @@ func (r *remoteIndex) Image(h v1.Hash) (v1.Image, error) {
return desc.Image()
}
// Descriptor retains the original descriptor from an index manifest.
// See partial.Descriptor.
func (r *remoteIndex) Descriptor() (*v1.Descriptor, error) {
// kind of a hack, but RawManifest does appropriate locking/memoization
// and makes sure r.descriptor is populated.
_, err := r.RawManifest()
return r.descriptor, err
}
func (r *remoteIndex) ImageIndex(h v1.Hash) (v1.ImageIndex, error) {
desc, err := r.childByHash(h)
if err != nil {
@@ -174,7 +165,7 @@ func (r *remoteIndex) childByHash(h v1.Hash) (*Descriptor, error) {
// Convert one of this index's child's v1.Descriptor into a remote.Descriptor, with the given platform option.
func (r *remoteIndex) childDescriptor(child v1.Descriptor, platform v1.Platform) (*Descriptor, error) {
ref := r.Ref.Context().Digest(child.Digest.String())
manifest, _, err := r.fetchManifest(ref, []types.MediaType{child.MediaType})
manifest, desc, err := r.fetchManifest(ref, []types.MediaType{child.MediaType})
if err != nil {
return nil, err
}
@@ -184,7 +175,7 @@ func (r *remoteIndex) childDescriptor(child v1.Descriptor, platform v1.Platform)
Client: r.Client,
},
Manifest: manifest,
Descriptor: child,
Descriptor: *desc,
platform: platform,
}, nil
}
vendor/github.com/google/go-containerregistry/pkg/v1/remote/list.go
Generated Vendored
+4 -6
View File
@@ -50,12 +50,10 @@ func ListWithContext(ctx context.Context, repo name.Repository, options ...Optio
}
uri := &url.URL{
Scheme: repo.Registry.Scheme(),
Host: repo.Registry.RegistryStr(),
Path: fmt.Sprintf("/v2/%s/tags/list", repo.RepositoryStr()),
// ECR returns an error if n > 1000:
// https://github.com/google/go-containerregistry/issues/681
RawQuery: "n=1000",
Scheme: repo.Registry.Scheme(),
Host: repo.Registry.RegistryStr(),
Path: fmt.Sprintf("/v2/%s/tags/list", repo.RepositoryStr()),
RawQuery: "n=10000",
}
client := http.Client{Transport: tr}
vendor/github.com/google/go-containerregistry/pkg/v1/remote/mount.go
Generated Vendored
-13
View File
@@ -17,7 +17,6 @@ package remote
import (
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
)
// MountableLayer wraps a v1.Layer in a shim that enables the layer to be
@@ -28,12 +27,6 @@ type MountableLayer struct {
Reference name.Reference
}
// Descriptor retains the original descriptor from an image manifest.
// See partial.Descriptor.
func (ml *MountableLayer) Descriptor() (*v1.Descriptor, error) {
return partial.Descriptor(ml.Layer)
}
// mountableImage wraps the v1.Layer references returned by the embedded v1.Image
// in MountableLayer's so that remote.Write might attempt to mount them from their
// source repository.
@@ -82,9 +75,3 @@ func (mi *mountableImage) LayerByDiffID(d v1.Hash) (v1.Layer, error) {
Reference: mi.Reference,
}, nil
}
// Descriptor retains the original descriptor from an index manifest.
// See partial.Descriptor.
func (mi *mountableImage) Descriptor() (*v1.Descriptor, error) {
return partial.Descriptor(mi.Image)
}
vendor/github.com/google/go-containerregistry/pkg/v1/remote/transport/README.md
Generated Vendored
+1 -3
View File
@@ -1,8 +1,6 @@
# `transport`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/transport?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/transport)
The [distribution protocol](https://github.com/opencontainers/distribution-spec) is fairly simple, but correctly [implementing authentication](../../../authn/README.md) is **hard**.
The [distribution protocol](https://github.com/opencontainers/distribution-spec) is fairly simple, but correctly implementing authentication is **hard**.
This package [implements](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote/transport#New) an [`http.RoundTripper`](https://godoc.org/net/http#RoundTripper)
that transparently performs:
vendor/github.com/google/go-containerregistry/pkg/v1/remote/transport/bearer.go
Generated Vendored
+10 -3
View File
@@ -33,7 +33,7 @@ type bearerTransport struct {
// Basic credentials that we exchange for bearer tokens.
basic authn.Authenticator
// Holds the bearer response from the token service.
bearer authn.AuthConfig
bearer *authn.Bearer
// Registry to which we send bearer tokens.
registry name.Registry
// See https://tools.ietf.org/html/rfc6750#section-3
@@ -61,7 +61,11 @@ func (bt *bearerTransport) RoundTrip(in *http.Request) (*http.Response, error) {
// the registry with which we are interacting.
// In case of redirect http.Client can use an empty Host, check URL too.
if matchesHost(bt.registry, in, bt.scheme) {
hdr := fmt.Sprintf("Bearer %s", bt.bearer.RegistryToken)
auth, err := bt.bearer.Authorization()
if err != nil {
return nil, err
}
hdr := fmt.Sprintf("Bearer %s", auth.RegistryToken)
in.Header.Set("Authorization", hdr)
}
return bt.inner.RoundTrip(in)
@@ -130,8 +134,9 @@ func (bt *bearerTransport) refresh() error {
}
// Find a token to turn into a Bearer authenticator
var bearer authn.Bearer
if response.Token != "" {
bt.bearer.RegistryToken = response.Token
bearer = authn.Bearer{Token: response.Token}
} else {
return fmt.Errorf("no token in bearer response:\n%s", content)
}
@@ -143,6 +148,8 @@ func (bt *bearerTransport) refresh() error {
})
}
// Replace our old bearer authenticator (if we had one) with our newly refreshed authenticator.
bt.bearer = &bearer
return nil
}
vendor/github.com/google/go-containerregistry/pkg/v1/remote/transport/error.go
Generated Vendored
+1 -45
View File
@@ -19,27 +19,9 @@ import (
"fmt"
"io/ioutil"
"net/http"
"net/url"
"strings"
)
// The set of query string keys that we expect to send as part of the registry
// protocol. Anything else is potentially dangerous to leak, as it's probably
// from a redirect. These redirects often included tokens or signed URLs.
var paramWhitelist = map[string]struct{}{
// Token exchange
"scope": struct{}{},
"service": struct{}{},
// Cross-repo mounting
"mount": struct{}{},
"from": struct{}{},
// Layer PUT
"digest": struct{}{},
// Listing tags and catalog
"n": struct{}{},
"last": struct{}{},
}
// Error implements error to support the following error specification:
// https://github.com/docker/distribution/blob/master/docs/spec/api.md#errors
type Error struct {
@@ -48,8 +30,6 @@ type Error struct {
StatusCode int
// The raw body if we couldn't understand it.
rawBody string
// The request that failed.
request *http.Request
}
// Check that Error implements error
@@ -57,14 +37,6 @@ var _ error = (*Error)(nil)
// Error implements error
func (e *Error) Error() string {
prefix := ""
if e.request != nil {
prefix = fmt.Sprintf("%s %s: ", e.request.Method, redact(e.request.URL))
}
return prefix + e.responseErr()
}
func (e *Error) responseErr() string {
switch len(e.Errors) {
case 0:
if len(e.rawBody) == 0 {
@@ -79,7 +51,7 @@ func (e *Error) responseErr() string {
errors = append(errors, d.String())
}
return fmt.Sprintf("multiple errors returned: %s",
strings.Join(errors, "; "))
strings.Join(errors, ";"))
}
}
@@ -97,21 +69,6 @@ func (e *Error) Temporary() bool {
return true
}
func redact(original *url.URL) *url.URL {
qs := original.Query()
for k, v := range qs {
for i := range v {
if _, ok := paramWhitelist[k]; !ok {
// key is not in the whitelist
v[i] = "REDACTED"
}
}
}
redacted := *original
redacted.RawQuery = qs.Encode()
return &redacted
}
// Diagnostic represents a single error returned by a Docker registry interaction.
type Diagnostic struct {
Code ErrorCode `json:"code"`
@@ -170,6 +127,5 @@ func CheckError(resp *http.Response, codes ...int) error {
structuredError.rawBody = string(b)
}
structuredError.StatusCode = resp.StatusCode
structuredError.request = resp.Request
return structuredError
}
vendor/github.com/google/go-containerregistry/pkg/v1/remote/write.go
Generated Vendored
+26 -26
View File
@@ -50,13 +50,13 @@ func Write(ref name.Reference, img v1.Image, options ...Option) error {
return err
}
scopes := scopesForUploadingImage(ref.Context(), ls)
scopes := scopesForUploadingImage(ref, ls)
tr, err := transport.New(ref.Context().Registry, o.auth, o.transport, scopes)
if err != nil {
return err
}
w := writer{
repo: ref.Context(),
ref: ref,
client: &http.Client{Transport: tr},
}
@@ -125,20 +125,20 @@ func Write(ref name.Reference, img v1.Image, options ...Option) error {
// With all of the constituent elements uploaded, upload the manifest
// to commit the image.
return w.commitImage(img, ref)
return w.commitImage(img)
}
// writer writes the elements of an image to a remote image reference.
type writer struct {
repo name.Repository
ref name.Reference
client *http.Client
}
// url returns a url.Url for the specified path in the context of this remote image reference.
func (w *writer) url(path string) url.URL {
return url.URL{
Scheme: w.repo.Registry.Scheme(),
Host: w.repo.RegistryStr(),
Scheme: w.ref.Context().Registry.Scheme(),
Host: w.ref.Context().RegistryStr(),
Path: path,
}
}
@@ -164,7 +164,7 @@ func (w *writer) nextLocation(resp *http.Response) (string, error) {
// initiation if "mount" is specified, even if no "from" sources are specified.
// However, this is not broadly applicable to all registries, e.g. ECR.
func (w *writer) checkExistingBlob(h v1.Hash) (bool, error) {
u := w.url(fmt.Sprintf("/v2/%s/blobs/%s", w.repo.RepositoryStr(), h.String()))
u := w.url(fmt.Sprintf("/v2/%s/blobs/%s", w.ref.Context().RepositoryStr(), h.String()))
resp, err := w.client.Head(u.String())
if err != nil {
@@ -182,7 +182,7 @@ func (w *writer) checkExistingBlob(h v1.Hash) (bool, error) {
// checkExistingManifest checks if a manifest exists already in the repository
// by making a HEAD request to the manifest API.
func (w *writer) checkExistingManifest(h v1.Hash, mt types.MediaType) (bool, error) {
u := w.url(fmt.Sprintf("/v2/%s/manifests/%s", w.repo.RepositoryStr(), h.String()))
u := w.url(fmt.Sprintf("/v2/%s/manifests/%s", w.ref.Context().RepositoryStr(), h.String()))
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
if err != nil {
@@ -210,7 +210,7 @@ func (w *writer) checkExistingManifest(h v1.Hash, mt types.MediaType) (bool, err
// upload was initiated and the body of that blob should be sent to the returned
// location.
func (w *writer) initiateUpload(from, mount string) (location string, mounted bool, err error) {
u := w.url(fmt.Sprintf("/v2/%s/blobs/uploads/", w.repo.RepositoryStr()))
u := w.url(fmt.Sprintf("/v2/%s/blobs/uploads/", w.ref.Context().RepositoryStr()))
uv := url.Values{}
if mount != "" && from != "" {
// Quay will fail if we specify a "mount" without a "from".
@@ -313,7 +313,7 @@ func (w *writer) uploadOne(l v1.Layer) error {
mount = h.String()
}
if ml, ok := l.(*MountableLayer); ok {
if w.repo.RegistryStr() == ml.Reference.Context().RegistryStr() {
if w.ref.Context().RegistryStr() == ml.Reference.Context().RegistryStr() {
from = ml.Reference.Context().RepositoryStr()
}
}
@@ -407,7 +407,7 @@ func unpackTaggable(t Taggable) (*v1.Descriptor, error) {
}
// commitImage does a PUT of the image's manifest.
func (w *writer) commitImage(t Taggable, ref name.Reference) error {
func (w *writer) commitImage(t Taggable) error {
raw, err := t.RawManifest()
if err != nil {
return err
@@ -417,7 +417,7 @@ func (w *writer) commitImage(t Taggable, ref name.Reference) error {
return err
}
u := w.url(fmt.Sprintf("/v2/%s/manifests/%s", w.repo.RepositoryStr(), ref.Identifier()))
u := w.url(fmt.Sprintf("/v2/%s/manifests/%s", w.ref.Context().RepositoryStr(), w.ref.Identifier()))
// Make the request to PUT the serialized manifest
req, err := http.NewRequest(http.MethodPut, u.String(), bytes.NewBuffer(raw))
@@ -437,11 +437,11 @@ func (w *writer) commitImage(t Taggable, ref name.Reference) error {
}
// The image was successfully pushed!
logs.Progress.Printf("%v: digest: %v size: %d", ref, desc.Digest, len(raw))
logs.Progress.Printf("%v: digest: %v size: %d", w.ref, desc.Digest, len(raw))
return nil
}
func scopesForUploadingImage(repo name.Repository, layers []v1.Layer) []string {
func scopesForUploadingImage(ref name.Reference, layers []v1.Layer) []string {
// use a map as set to remove duplicates scope strings
scopeSet := map[string]struct{}{}
@@ -449,7 +449,7 @@ func scopesForUploadingImage(repo name.Repository, layers []v1.Layer) []string {
if ml, ok := l.(*MountableLayer); ok {
// we will add push scope for ref.Context() after the loop.
// for now we ask pull scope for references of the same registry
if ml.Reference.Context().String() != repo.String() && ml.Reference.Context().Registry.String() == repo.Registry.String() {
if ml.Reference.Context() != ref.Context() && ml.Reference.Context().Registry == ref.Context().Registry {
scopeSet[ml.Reference.Scope(transport.PullScope)] = struct{}{}
}
}
@@ -457,7 +457,7 @@ func scopesForUploadingImage(repo name.Repository, layers []v1.Layer) []string {
scopes := make([]string, 0)
// Push scope should be the first element because a few registries just look at the first scope to determine access.
scopes = append(scopes, repo.Scope(transport.PushScope))
scopes = append(scopes, ref.Scope(transport.PushScope))
for scope := range scopeSet {
scopes = append(scopes, scope)
@@ -485,7 +485,7 @@ func WriteIndex(ref name.Reference, ii v1.ImageIndex, options ...Option) error {
return err
}
w := writer{
repo: ref.Context(),
ref: ref,
client: &http.Client{Transport: tr},
}
@@ -523,22 +523,22 @@ func WriteIndex(ref name.Reference, ii v1.ImageIndex, options ...Option) error {
// With all of the constituent elements uploaded, upload the manifest
// to commit the image.
return w.commitImage(ii, ref)
return w.commitImage(ii)
}
// WriteLayer uploads the provided Layer to the specified repo.
func WriteLayer(repo name.Repository, layer v1.Layer, options ...Option) error {
o, err := makeOptions(repo, options...)
// WriteLayer uploads the provided Layer to the specified name.Digest.
func WriteLayer(ref name.Digest, layer v1.Layer, options ...Option) error {
o, err := makeOptions(ref.Context(), options...)
if err != nil {
return err
}
scopes := scopesForUploadingImage(repo, []v1.Layer{layer})
tr, err := transport.New(repo.Registry, o.auth, o.transport, scopes)
scopes := scopesForUploadingImage(ref, []v1.Layer{layer})
tr, err := transport.New(ref.Context().Registry, o.auth, o.transport, scopes)
if err != nil {
return err
}
w := writer{
repo: repo,
ref: ref,
client: &http.Client{Transport: tr},
}
@@ -564,9 +564,9 @@ func Tag(tag name.Tag, t Taggable, options ...Option) error {
return err
}
w := writer{
repo: tag.Context(),
ref: tag,
client: &http.Client{Transport: tr},
}
return w.commitImage(t, tag)
return w.commitImage(t)
}
vendor/github.com/google/go-containerregistry/pkg/v1/stream/README.md
Generated Vendored
-68
View File
@@ -1,68 +0,0 @@
# `stream`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/stream?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/stream)
The `stream` package contains an implementation of
[`v1.Layer`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1#Layer)
that supports _streaming_ access, i.e. the layer contents are read once and not
buffered.
## Usage
```go
package main
import (
"os"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/stream"
)
// upload the contents of stdin as a layer to a local registry
func main() {
repo, err := name.NewRepository("localhost:5000/stream")
if err != nil {
panic(err)
}
layer := stream.NewLayer(os.Stdin)
if err := remote.WriteLayer(repo, layer); err != nil {
panic(err)
}
}
```
## Structure
This implements the layer portion of an [image
upload](/pkg/v1/remote#anatomy-of-an-image-upload). We launch a goroutine that
is responsible for hashing the uncompressed contents to compute the `DiffID`,
gzipping them to produce the `Compressed` contents, and hashing/counting the
bytes to produce the `Digest`/`Size`. This goroutine writes to an
`io.PipeWriter`, which blocks until `Compressed` reads the gzipped contents from
the corresponding `io.PipeReader`.
<p align="center">
<img src="/images/stream.dot.svg" />
</p>
## Caveats
This assumes that you have an uncompressed layer (i.e. a tarball) and would like
to compress it. Calling `Uncompressed` is always an error. Likewise, other
methods are invalid until the contents of `Compressed` have been completely
consumed and `Close`d.
Using a `stream.Layer` will likely not work without careful consideration. For
example, in the `mutate` package, we defer computing the manifest and config
file until they are actually called. This allows you to `mutate.Append` a
streaming layer to an image without accidentally consuming it. Similarly, in
`remote.Write`, if calling `Digest` on a layer fails, we attempt to upload the
layer anyway, understanding that we may be dealing with a `stream.Layer` whose
contents need to be uploaded before we can upload the config file.
Given the [structure](#structure) of how this is implemented, forgetting to
`Close` a `stream.Layer` will leak a goroutine.
vendor/github.com/google/go-containerregistry/pkg/v1/tarball/README.md
Generated Vendored
-280
View File
@@ -1,280 +0,0 @@
# `tarball`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/tarball?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/tarball)
This package produces tarballs that can consumed via `docker load`. Note
that this is a _different_ format from the [`legacy`](/pkg/legacy/tarball)
tarballs that are produced by `docker save`, but this package is still able to
read the legacy tarballs produced by `docker save`.
## Usage
```go
package main
import (
"os"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/tarball"
)
func main() {
// Read a tarball from os.Args[1] that contains ubuntu.
tag, err := name.NewTag("ubuntu")
if err != nil {
panic(err)
}
img, err := tarball.ImageFromPath(os.Args[1], &tag)
if err != nil {
panic(err)
}
// Write that tarball to os.Args[2] with a different tag.
newTag, err := name.NewTag("ubuntu:newest")
if err != nil {
panic(err)
}
f, err := os.Create(os.Args[2])
if err != nil {
panic(err)
}
defer f.Close()
if err := tarball.Write(newTag, img, f); err != nil {
panic(err)
}
}
```
## Structure
<p align="center">
<img src="/images/tarball.dot.svg" />
</p>
Let's look at what happens when we write out a tarball:
### `ubuntu:latest`
```
$ crane pull ubuntu ubuntu.tar && mkdir ubuntu && tar xf ubuntu.tar -C ubuntu && rm ubuntu.tar
$ tree ubuntu/
ubuntu/
├── 423ae2b273f4c17ceee9e8482fa8d071d90c7d052ae208e1fe4963fceb3d6954.tar.gz
├── b6b53be908de2c0c78070fff0a9f04835211b3156c4e73785747af365e71a0d7.tar.gz
├── de83a2304fa1f7c4a13708a0d15b9704f5945c2be5cbb2b3ed9b2ccb718d0b3d.tar.gz
├── f9a83bce3af0648efaa60b9bb28225b09136d2d35d0bed25ac764297076dec1b.tar.gz
├── manifest.json
└── sha256:72300a873c2ca11c70d0c8642177ce76ff69ae04d61a5813ef58d40ff66e3e7c
0 directories, 6 files
```
There are a couple interesting files here.
`manifest.json` is the entrypoint: a list of [`tarball.Descriptor`s](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/tarball#Descriptor)
that describe the images containd in this tarball.
For each image, this has the `RepoTags` (how it was pulled), a `Config` file
that points to the image's config file, a list of `Layers`, and (optionally)
`LayerSources`.
```
$ jq < ubuntu/manifest.json
[
{
"Config": "sha256:72300a873c2ca11c70d0c8642177ce76ff69ae04d61a5813ef58d40ff66e3e7c",
"RepoTags": [
"ubuntu"
],
"Layers": [
"423ae2b273f4c17ceee9e8482fa8d071d90c7d052ae208e1fe4963fceb3d6954.tar.gz",
"de83a2304fa1f7c4a13708a0d15b9704f5945c2be5cbb2b3ed9b2ccb718d0b3d.tar.gz",
"f9a83bce3af0648efaa60b9bb28225b09136d2d35d0bed25ac764297076dec1b.tar.gz",
"b6b53be908de2c0c78070fff0a9f04835211b3156c4e73785747af365e71a0d7.tar.gz"
]
}
]
```
The config file and layers are exactly what you would expect, and match the
registry representations of the same artifacts. You'll notice that the
`manifest.json` contains similar information as the registry manifest, but isn't
quite the same:
```
$ crane manifest ubuntu@sha256:0925d086715714114c1988f7c947db94064fd385e171a63c07730f1fa014e6f9
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 3408,
"digest": "sha256:72300a873c2ca11c70d0c8642177ce76ff69ae04d61a5813ef58d40ff66e3e7c"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 26692096,
"digest": "sha256:423ae2b273f4c17ceee9e8482fa8d071d90c7d052ae208e1fe4963fceb3d6954"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 35365,
"digest": "sha256:de83a2304fa1f7c4a13708a0d15b9704f5945c2be5cbb2b3ed9b2ccb718d0b3d"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 852,
"digest": "sha256:f9a83bce3af0648efaa60b9bb28225b09136d2d35d0bed25ac764297076dec1b"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 163,
"digest": "sha256:b6b53be908de2c0c78070fff0a9f04835211b3156c4e73785747af365e71a0d7"
}
]
}
```
This makes it difficult to maintain image digests when roundtripping images
through the tarball format, so it's not a great format if you care about
provenance.
The ubuntu example didn't have any `LayerSources` -- let's look at another image
that does.
### `hello-world:nanoserver`
```
$ crane pull hello-world:nanoserver@sha256:63c287625c2b0b72900e562de73c0e381472a83b1b39217aef3856cd398eca0b nanoserver.tar
$ mkdir nanoserver && tar xf nanoserver.tar -C nanoserver && rm nanoserver.tar
$ tree nanoserver/
nanoserver/
├── 10d1439be4eb8819987ec2e9c140d44d74d6b42a823d57fe1953bd99948e1bc0.tar.gz
├── a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053.tar.gz
├── be21f08f670160cbae227e3053205b91d6bfa3de750b90c7e00bd2c511ccb63a.tar.gz
├── manifest.json
└── sha256:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6
0 directories, 5 files
$ jq < nanoserver/manifest.json
[
{
"Config": "sha256:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6",
"RepoTags": [
"index.docker.io/library/hello-world:i-was-a-digest"
],
"Layers": [
"a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053.tar.gz",
"be21f08f670160cbae227e3053205b91d6bfa3de750b90c7e00bd2c511ccb63a.tar.gz",
"10d1439be4eb8819987ec2e9c140d44d74d6b42a823d57fe1953bd99948e1bc0.tar.gz"
],
"LayerSources": {
"sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e": {
"mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
"size": 101145811,
"digest": "sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053",
"urls": [
"https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053"
]
}
}
}
]
```
A couple things to note about this `manifest.json` versus the other:
* The `RepoTags` field is a bit weird here. `hello-world` is a multi-platform
image, so We had to pull this image by digest, since we're (I'm) on
amd64/linux and wanted to grab a windows image. Since the tarball format
expects a tag under `RepoTags`, and we didn't pull by tag, we replace the
digest with a sentinel `i-was-a-digest` "tag" to appease docker.
* The `LayerSources` has enough information to reconstruct the foreign layers
pointer when pushing/pulling from the registry. For legal reasons, microsoft
doesn't want anyone but them to serve windows base images, so the mediaType
here indicates a "foreign" or "non-distributable" layer with an URL for where
you can download it from microsoft (see the [OCI
image-spec](https://github.com/opencontainers/image-spec/blob/master/layer.md#non-distributable-layers)).
We can look at what's in the registry to explain both of these things:
```
$ crane manifest hello-world:nanoserver | jq .
{
"manifests": [
{
"digest": "sha256:63c287625c2b0b72900e562de73c0e381472a83b1b39217aef3856cd398eca0b",
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "amd64",
"os": "windows",
"os.version": "10.0.17763.1040"
},
"size": 1124
}
],
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"schemaVersion": 2
}
# Note the media type and "urls" field.
$ crane manifest hello-world:nanoserver@sha256:63c287625c2b0b72900e562de73c0e381472a83b1b39217aef3856cd398eca0b | jq .
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 1721,
"digest": "sha256:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
"size": 101145811,
"digest": "sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053",
"urls": [
"https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053"
]
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 1669,
"digest": "sha256:be21f08f670160cbae227e3053205b91d6bfa3de750b90c7e00bd2c511ccb63a"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 949,
"digest": "sha256:10d1439be4eb8819987ec2e9c140d44d74d6b42a823d57fe1953bd99948e1bc0"
}
]
}
```
The `LayerSources` map is keyed by the diffid. Note that `sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e` matches the first layer in the config file:
```
$ jq '.[0].LayerSources' < nanoserver/manifest.json
{
"sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e": {
"mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
"size": 101145811,
"digest": "sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053",
"urls": [
"https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053"
]
}
}
$ jq < nanoserver/sha256\:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6 | jq .rootfs
{
"type": "layers",
"diff_ids": [
"sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e",
"sha256:601cf7d78c62e4b4d32a7bbf96a17606a9cea5bd9d22ffa6f34aa431d056b0e8",
"sha256:a1e1a3bf6529adcce4d91dce2cad86c2604a66b507ccbc4d2239f3da0ec5aab9"
]
}
```
vendor/github.com/google/go-containerregistry/pkg/v1/tarball/image.go
Generated Vendored
+8 -10
View File
@@ -79,17 +79,15 @@ func Image(opener Opener, tag *name.Tag) (v1.Image, error) {
}
// Peek at the first layer and see if it's compressed.
if len(img.imgDescriptor.Layers) > 0 {
compressed, err := img.areLayersCompressed()
if err != nil {
return nil, err
}
if compressed {
c := compressedImage{
image: img,
}
return partial.CompressedToImage(&c)
compressed, err := img.areLayersCompressed()
if err != nil {
return nil, err
}
if compressed {
c := compressedImage{
image: img,
}
return partial.CompressedToImage(&c)
}
uc := uncompressedImage{
vendor/github.com/google/go-containerregistry/pkg/v1/tarball/layer.go
Generated Vendored
+2 -2
View File
@@ -122,7 +122,7 @@ func LayerFromOpener(opener Opener, opts ...LayerOption) (v1.Layer, error) {
}
// LayerFromReader returns a v1.Layer given a io.Reader.
func LayerFromReader(reader io.Reader, opts ...LayerOption) (v1.Layer, error) {
func LayerFromReader(reader io.Reader) (v1.Layer, error) {
// Buffering due to Opener requiring multiple calls.
a, err := ioutil.ReadAll(reader)
if err != nil {
@@ -130,7 +130,7 @@ func LayerFromReader(reader io.Reader, opts ...LayerOption) (v1.Layer, error) {
}
return LayerFromOpener(func() (io.ReadCloser, error) {
return ioutil.NopCloser(bytes.NewReader(a)), nil
}, opts...)
})
}
func computeDigest(opener Opener, compressed bool, compression int) (v1.Hash, int64, error) {
vendor/github.com/google/go-containerregistry/pkg/v1/tarball/write.go
Generated Vendored
+1 -1
View File
@@ -146,7 +146,7 @@ func MultiRefWrite(refToImage map[name.Reference]v1.Image, w io.Writer) error {
if err != nil {
return err
}
layerSources[diffid] = *desc
layerSources[diffid] = desc
}
r, err := l.Compressed()