Compare commits

...
99 Commits
Author SHA1 Message Date
Fedor Korotkov c54b140750 Support Docker Helpers (#205)
Fixes #167
2022-08-29 20:26:53 +04:00
Fedor Korotkov 048a5506df Support trackpad on macOS and clipboard sharing on Linux (#202)
Fixes #66
Relates to #14 since fixes on Linux
2022-08-27 20:05:45 +04:00
Nikolay Edigaryev 553b36349b README.md: clarify "Pulling a Remote Image" section (#196) 2022-08-25 15:43:30 +04:00
Fedor Korotkov d0bf286aa1 Document Linux VM image creation (#193) 2022-08-24 23:21:50 +04:00
Nikolay Edigaryev 195a4b3a72 Make UnsupportedOSError more user-friendly (#192)
* Make UnsupportedOSError more user-friendly

* tart prune: allow only specifying a --gc flag
2022-08-24 15:03:54 -04:00
Nikolay Edigaryev 59393df2e1 Linux support (#190)
* Linux support

* Support macOS 12

* Improve readability a bit

* Remove useless variable
2022-08-23 22:38:50 +04:00
Nikolay Edigaryev 732c8244a4 Run garbage collection after each symlink and expose --gc in tart prune (#191) 2022-08-23 17:31:02 +04:00
Nikolay Edigaryev 9e69c8c161 testGetAndSetAccessTime: fix flakiness (#187) 2022-08-18 14:42:54 -04:00
Nikolay Edigaryev e4ac2275b9 tart ip: show a warning when DHCP lease and ARP cache entries mismatch (#186) 2022-08-18 22:25:02 +04:00
Nikolay Edigaryev 1a1f19e169 Implement garbage collection when deleting cached OCI VMs (#185)
* Properly calculate remotely-retrieved manifest digests

* Implement garbage collection when deleting cached OCI VMs
2022-08-18 17:13:06 +04:00
Nikolay Edigaryev fea916dacc OCI blob compression: fix Data memory leak when using InputFilter (#183)
* OCI blob compression: fix Data memory leak when using InputFilter

* Rename mappedDiskOffset to mappedDiskReadOffset

* Update progress.completedUnitCount differently
2022-08-17 12:57:39 -04:00
Fedor Korotkov b1be9730c7 Build and release from Ventrura (#182)
Might be related to #180
2022-08-17 17:42:24 +04:00
Nikolay EdigaryevandNikolay Edigaryev 14059a1d6f tart {pull,clone}: add missing --insecure support (#181)
Co-authored-by: Nikolay Edigaryev <edi@microsun.local>
2022-08-17 01:31:19 +04:00
Nikolay Edigaryev 440681320a Introduce "tart prune" command (#164) 2022-08-12 16:43:04 +03:00
Fedor KorotkovandNikolay Edigaryev a80954c888 Do not include Content-Range for monolithic uploads (#179)
* Do not include Content-Range for monolithic uploads

Some registries still assumes it's a chunked upload and verifies the "chunk" size which is too big.

* Update Sources/tart/OCI/Registry.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2022-08-12 16:33:15 +03:00
Nikolay Edigaryev cc8201dee6 OCI: support insecure registries and custom ports (#174) 2022-08-12 16:24:14 +03:00
Fedor Korotkov 2cab49b3f1 Use Ventura APIs for recovery mode (#175) 2022-08-12 08:02:44 +03:00
Fedor Korotkov 131827802a Friendly error message on Intel (#177)
In case someone tries to install Tart. Fixes #176
2022-08-10 11:13:21 -04:00
Fedor Korotkov e2b7f12388 Support chunked uploads (#159)
* Support chunked uploads

* Rebase fixes

* Bump swift http client
2022-08-08 10:16:47 -04:00
Fedor Korotkov 617a5d02dc Fixed potential infinity auth (#173)
Follow up to an attempt in #169
2022-08-08 16:50:13 +03:00
Nikolay Edigaryev b83bce4544 tart login: read whole stdin contents instead of just a line (#170) 2022-08-02 16:52:35 +03:00
Nikolay Edigaryev 7d16516b6a OCI: gcr.io fixes (#169)
* Make sure we don't craft an URLComponents with an empty query items

* Prevent auth() infinite loop

* OCI: ignore invalid RFC 3339 formatted dates
2022-08-01 19:24:11 -04:00
Nikolay Edigaryev 56ddd8df75 OCI's HTTP client: prevent the usage of NIO Transport Services (#168) 2022-08-01 18:17:50 -04:00
Fedor Korotkov b1534a05d5 Configure audio devices to pass through the signal from/to the host (#163)
Fixes #161
2022-08-01 17:54:55 +03:00
Nikolay Edigaryev f54e7c2cab tart login: only store one credential per registry (#162) 2022-08-01 09:55:37 +03:00
Nikolay Edigaryev 85dfa961c9 tart run: introduce --disk option (#156)
* tart run: introduce --disk option

* Document how to create disks using Disk Utility
2022-07-26 09:25:23 -04:00
Nikolay Edigaryev 3a74fc35e6 Introduce TART_HOME (#157) 2022-07-25 17:01:46 +03:00
Nikolay Edigaryev 7bf7f890c4 tart run: return exit status 2 when VM is already running (#155) 2022-07-21 07:53:47 -04:00
Nikolay Edigaryev 48cd4b47e4 Move full-fledged VNC support to --experimental-vnc (#154) 2022-07-21 12:37:43 +03:00
Nikolay Edigaryev c1dee4f9b2 Credentials: update Keychain entry if it already exists (#149) 2022-07-13 12:35:44 -04:00
Fedor Korotkov 116dc01f55 Document how to retrieve artifacts (#146) 2022-07-08 18:45:14 +03:00
Nikolay Edigaryev 52abb7589c OCI: support Basic authentication scheme (#145)
* OCI: support Basic authentication scheme

* .isValid → .isValid()

* tart login: make --username optional
2022-07-08 16:36:00 +03:00
Nikolay Edigaryev 4386192161 tart login: introduce --username and --password-stdin flags (#143) 2022-07-05 16:32:20 +03:00
Nikolay Edigaryev 85429cea0a Retrieve IP from DHCPD leases file instead of ARP cache (#141)
* Retrieve IP from DHCPD leases file instead of ARP cache

* Reference PLCache_read() from the retrieveRawLeases() parsing function
2022-06-30 17:58:52 +03:00
Nikolay Edigaryev 384abcd0bd OCI: make sure annotations are sorted (#138) 2022-06-27 16:25:52 +03:00
Fedor Korotkov 92529afa23 Handle tart run --no-graphics --vnc properly (#137)
Let's start a VNC server but not force open Screen Sharing if `--no-graphics` is also passed.
2022-06-24 22:54:41 +03:00
Nikolay Edigaryev c25364b2d4 Homebrew: depend on Softnet package (#136) 2022-06-21 14:55:51 -04:00
Nikolay Edigaryev e12f95878e Softnet: an alternative to built-in NAT with better isolation (#48)
* Softnet: an alternative to built-in NAT with better isolation

* Softnet: increase socketpair(2) socket buffer sizes to 1 MiB

* Pass VM's FD and MAC address to the Softnet

* Softnet: implement graceful shutdown

* Bring back the dispatchMain() and task cancellation

* tart pull: check for cancellation when pulling response body

* Don't dispatchMain() in withTaskCancellationHandler()

* Move VNC URL opening logic into VNCWrapper.open()
2022-06-21 19:17:02 +03:00
Fedor Korotkov 7efef28250 Allow VNC for Recovery mode (#134) 2022-06-20 20:08:31 +03:00
Nikolay Edigaryev 1e90752ded Full-fledged VNC support (#126) 2022-06-20 18:53:17 +03:00
Fedor Korotkov 2020324ef5 Fixed --no-graphics (#132)
Followup to #129 which broke CLI integration because `--no-graphics` option didn't wait for anything.
2022-06-20 10:28:10 -04:00
Nikolay Edigaryev b581db5be4 OCI: make annotations optional (#130)
* OCI: make annotations optional

* Don't initialize annotations by default
2022-06-17 10:53:52 -04:00
Fedor Korotkov 8ed2ce159f Install Go 2022-06-17 10:33:32 -04:00
Fedor Korotkov 89217c23a2 Fixes for Ventura (#129)
* Fixes for Ventura

Still a noob in SwiftUI and Swift concurrency, but it seems on Ventura a task group is not actually running on main or something. Either way I think this change simplifies things but launching a VM in a task and then just continuing with either VNC or built-in graphics.

* Check VM's state
2022-06-17 10:04:57 -04:00
Fedor Korotkov 0d633de04a Build Tart inside a Tart VM (#127) 2022-06-17 11:18:26 +03:00
Nikolay Edigaryev f59ef2722d OCI: store uncompressed disk size in manifest (#128) 2022-06-17 11:14:28 +03:00
Fedor Korotkov 7759c72a76 Don't use dispatchMain (#123) 2022-06-14 11:31:31 -04:00
Raymond 4cc8a9925a accept larger disk size (#122) 2022-06-10 09:05:57 -04:00
Nikolay Edigaryev b16bbf587a Registry: log upload location when encountering blob pushing error (#121) 2022-06-08 17:23:16 +03:00
Fedor Korotkov 022317bc17 Build Debug Binary (#117)
* Build Debug Binary

Xcode 14 Beta breaks something when building with a production configuration and Tart can't run a VM.

* Fixed path
2022-06-07 18:14:52 +03:00
Fedor Korotkov 87733290e7 Smaller icon (#116)
Initially it wasn't fully following the [guidelines](https://developer.apple.com/design/human-interface-guidelines/macos/icons-and-images/app-icon/).

Fixes #112
2022-06-06 18:38:30 +03:00
Fedor Korotkov c14b46adc3 Make sure VM directory exists before moving (#111) 2022-06-03 23:22:11 +03:00
Nikolay Edigaryev 63329ef363 Atomic tart {create,clone,pull} operations using rename(2) (#109)
* tart clone: always re-generate MAC-address

This is not really an issue for non-enterprise users[1].

[1]: https://github.com/cirruslabs/tart/issues/20#issuecomment-1136944455

* Atomic tart {create,clone,pull} operations using rename(2)

* Print a nicer error message when attempting to double-run a VM

* tart clone: bring back the old MAC-address generation logic

* Ensure VMDirectory.temporary() will be deleted on failure
2022-06-01 11:02:31 -04:00
Nikolay EdigaryevandFedor Korotkov 5446164a36 tart pull: introduce --populate-cache flag (#103)
* tart pull: introduce --populate-cache flag

* VMStorageOCI: introduce cache() method

* Review comments (#107)

* Rename SetCommand back to Set

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2022-05-28 23:20:24 -04:00
Nikolay Edigaryev f3068b9055 tart login: verify credentials (#102)
* tart login: verify credentials

* Make DictionaryCredentialsProvider fileprivate to Login.swift
2022-05-26 09:31:46 -04:00
Fedor Korotkov afa6b7b46c Auto-detect screen DPI (#106)
* Auto-detect screen DPI

Fixes #104

* Added a comment
2022-05-26 12:23:13 +03:00
Fedor Korotkov d277fb2941 Added helpfull messages to run command (#100) 2022-05-23 11:29:35 -04:00
Fedor Korotkov 088cdc51a3 Option to run with VNC (#97)
* Option to run with VNC

So copy/paste is working. For example, `tart run --vnc latest`.

Related to 14

* Add VNC task first

* Indicate the wait
2022-05-23 16:49:15 +03:00
Fedor Korotkov afb23a3e3a Init HTTP client only if Registry is used (#99) 2022-05-21 00:26:43 +03:00
Fedor Korotkov 35904dc637 Async http client for pull/push (#95)
* Use async http client for pull/push

* Don't update progress too frequently

* Removed unused variable

* Rebased after added tests
2022-05-20 11:04:21 -04:00
Nikolay Edigaryev fec803277d Registry functional/integration tests (#96)
* Registry functional/integration tests

* Remove DockerClientSwift import

* Encodable, Decodable → Codable
2022-05-20 09:52:37 -04:00
Nikolay EdigaryevandFedor Korotkov 13b05d75c5 .ci/set-version.sh: use temporary file (#93)
* .ci/set-version.sh: use temporary file

* Update .ci/set-version.sh

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2022-05-19 15:51:22 -04:00
Nikolay Edigaryev 54a321df7f Sort "tart list" by default (#94) 2022-05-19 15:05:16 -04:00
Fedor Korotkov b3695c8406 Log output of hooks (#92) 2022-05-19 19:37:04 +03:00
Fedor Korotkov 0a257a1547 Option to run in recovery mode (#91)
* Option to run in recovery mode

* Fixed typo

* Use Dynamic package to call private APIs

* Improved comment

* Move options closer to invocation
2022-05-19 08:17:05 -04:00
Nikolay Edigaryev 60c15e3e49 tart list: fix heading order (#90) 2022-05-18 14:20:52 -04:00
Fedor Korotkov a1bcbdbf0b Bump default CPU cores (#86)
To prevent frustrations like #68
2022-05-18 18:12:45 +03:00
Fedor Korotkov 7fec41b2cb Document VM location on disk (#87)
Fixes #68
2022-05-18 18:01:13 +03:00
Fedor Korotkov ea4fb9a2d5 Add Geekbench report link (#83)
Fixes #82
2022-05-17 15:20:04 -04:00
Nikolay Edigaryev c2da3fd919 Support "tart --version" (#81)
* Support "tart --version"

* Move CI.swift patcher into a separate script

Otherwise it doesn't work in GoReleaser.
2022-05-17 10:40:21 -04:00
Fedor Korotkov 63a2793c32 Support upper case in remote names (#79) 2022-05-16 18:14:42 +03:00
Fedor Korotkov 0fc3d3d1f4 Add virtual sound devices (#78) 2022-05-16 17:21:21 +03:00
Nikolay Edigaryev 60b705478b Handle Ctrl + C properly (#73)
* Terminate when Ctrl+C is encountered when entering credentials

* Handle Ctrl+C by catching SIGINT and converting it to task cancellation

* maxCharacters instead of (buf.count - 2)

* readStdinCredential: fix maxCharacters to be 255

* "user" variable should be named "credential"
2022-05-16 09:56:30 -04:00
Nikolay Edigaryev fa9e3146c1 tart list: filter out uninitialized VM directories (#76) 2022-05-16 08:30:58 -04:00
Nikolay Edigaryev 3dfe8f870c Provide more details when VM's network attachment disconnects (#74) 2022-05-16 08:23:34 -04:00
Fedor Korotkov 7afa446a73 Change default disk size upon creation (#71)
To help with frustrations like https://github.com/cirruslabs/tart/issues/44#issuecomment-1126831261

Experts will still use `--disk-size` flag to tailor disk size for their needs.
2022-05-16 09:44:31 +03:00
Nikolay EdigaryevandFedor Korotkov d1bed25023 tart pull: be more liberal when accepting local image as argument (#70)
* tart pull: be more liberal when accepting local image as argument

* Update Sources/tart/Commands/Pull.swift

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

* Single sentence and consistent capitalization

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2022-05-15 18:03:15 -04:00
Fedor Korotkov b39c3c9b52 Reworked CI integration documentation (#69)
It was a bit confusing to folks not familiar with Cirrus CLI. Reworked the documentation to better introduce Cirrus CLI.
2022-05-15 13:25:25 +03:00
Nikolay Edigaryev 8554e56e4b Registry: use issued_at field in the token response message (#65)
* Registry: use issued_at field in the token response message

* Configure JSONDecoder instead of the Decodable itself
2022-05-12 22:33:02 +03:00
Nikolay Edigaryev 3fdcc5c33c .goreleaser(brews): depend on macOS Monterey (#64)
* .goreleaser(brews): depend on macOS Monterey

* README.md: mention that macOS Monterey or later is required
2022-05-12 00:42:12 +03:00
Austin Culter 182ddf0268 Minor typo: 'reposiotry' > 'repository' (#61)
On the tin.
2022-05-10 17:16:03 -04:00
Fedor Korotkov 24375c24f3 Print total size of compressed layers (#60)
Before pulling an image.
2022-05-10 16:20:52 -04:00
Fedor Korotkov 4ebec53c77 Two more FAQs (#58)
* Two more FAQs

* Better summary

* Another clarification
2022-05-10 23:13:17 +03:00
Fedor KorotkovandNikolay Edigaryev 2f5a6790d8 Refactored registry token handling (#57)
* Refactored registry token handling

* Update Sources/tart/OCI/Registry.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Target our dedicated M1 Mini

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2022-05-09 16:13:22 +03:00
Fedor Korotkov cc697b4f6e More images in docs (#56)
* More images in docs

* More images in docs

* Updated GIF
2022-05-09 12:59:51 +03:00
Nikolay Edigaryev f6acbe8fe5 Registry: invalidate expired tokens (#55) 2022-05-08 19:46:11 -04:00
Fedor Korotkov 341fd168b3 Fixed release build entitlements (#54) 2022-05-08 13:05:10 +03:00
Fedor Korotkov e8a6efa60a Return title to docs (#53)
* Return title to docs

* smaller
2022-05-07 11:24:25 +03:00
Fedor Korotkov efc9c74b6c Report if an image already pulled (#52) 2022-05-06 17:06:28 -04:00
Fedor KorotkovandNikolay Edigaryev f712ba8ce3 Proper documentation (#49)
* WIP proper docs

* Update README.md

* wip

* More updates

* Apply suggestions from code review

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Use GitHub Packages links

* Promote Cirrus CLI first and then mentioned Cirrus CI

* Apply suggestions from code review

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2022-05-06 23:59:33 +03:00
Nikolay Edigaryev 4a60c41dd3 Registry: provide response body on auth token request failure (#51)
* Registry: provide response body on auth token request failure

* Clone.swift: fix typo
2022-05-06 14:19:27 -04:00
Fedor Korotkov f20a98bf01 Smart generation of new MAC addresses (#45)
* Introduce --new-mac-address flag for Clone command

Related to https://github.com/cirruslabs/tart/issues/20#issuecomment-1116732505

* Smart generation of new MAC addresses
2022-05-06 20:38:56 +03:00
Nikolay Edigaryev d5e3a7718e Registry: suport pulling public images (#50) 2022-05-06 20:30:04 +03:00
Nikolay Edigaryev a4db60d656 Registry: don't loose query parameters on URL absolutization (#47)
* Registry: don't loose query parameters on URL absolutization

* Registry: throw a proper exception when Location header parsing fails
2022-05-06 09:03:58 -04:00
Fedor Korotkov 29e08220e4 Get OCI credentials only when needed (#46)
So we don't even request them for a public image
2022-05-06 02:12:38 +03:00
Fedor Korotkov cdf4932aa3 Improved logging a bit (#43)
* Improved logging a bit

* Progress for all bytes
2022-05-05 10:43:09 -04:00
Nikolay Edigaryev 95316c0d67 Support pulling/pushing VMs to OCI-compatible registries (#32)
* Support pulling/pushing VMs to OCI-compatible registries

* Registry: rename request() to endpointRequest() for clarity

* Registry: include JSON details on HTTP status code mismatch errors

* .cirrus.yml: run tests

* Fix testDigest

* Registry: set Content-{Length,Type} headers when pushing blob

* Refactor Registry.auth() and enrich RegistryError.AuthFailed

* Remove useless comment

* Fix WWWAuthenticate a bit and add tests

* WWWAuthenticate: expect a Bearer scheme

* Registry.auth(): document the passing of ["scope", "service"] parameters

* Clarify unexpected HTTP code error when retrieving auth token

* Make RemoteName parser more relaxed for now

* tart clone: pull the VM if it's OCI-based first

* VMStorageOCI: ensure the old symbolic link is overwritten

* tart push: support multiple remote VM names

* Logging: push/pull progress

* Use 500 MB chunks (instead of 500 MiB) to evenly cut disk

...which also uses powers of 10.

* Credentials: only read credentials labeled "Tart Credentials"
2022-05-03 14:30:26 -04:00
Fedor Korotkov 4a5efefbc9 Embed AppIcon.png within binary (#30)
* Build product explicitly

Without it resources are not embedded in the executable

* Embed icon in code
2022-04-22 10:29:52 +03:00
Fedor Korotkov f07ffed6c8 --wait option for IP command (#29) 2022-04-19 19:56:59 +03:00
80 changed files with 5807 additions and 375 deletions
+5
View File
@@ -0,0 +1,5 @@
#!/bin/sh
TMPFILE=$(mktemp)
envsubst < Sources/tart/CI/CI.swift > $TMPFILE
mv $TMPFILE Sources/tart/CI/CI.swift
+18 -10
View File
@@ -1,23 +1,31 @@
task:
name: Test on Ventura
persistent_worker:
labels:
name: Mac-Mini-M1
build_script: swift test
test_script: swift test
task:
name: Build
only_if: $CIRRUS_TAG == ''
persistent_worker:
labels:
os: darwin
arch: arm64
build_script: swift build
sign_script: codesign --sign - --entitlements Sources/tart/tart.entitlements --force .build/debug/tart
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-xcode:latest
build_script: swift build --product tart
sign_script: codesign --sign - --entitlements Resources/tart.entitlements --force .build/debug/tart
binary_artifacts:
path: .build/debug/tart
task:
name: Release
only_if: $CIRRUS_TAG != ''
persistent_worker:
labels:
os: darwin
arch: arm64
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-xcode:latest
env:
GITHUB_TOKEN: ENCRYPTED[!98ace8259c6024da912c14d5a3c5c6aac186890a8d4819fad78f3e0c41a4e0cd3a2537dd6e91493952fb056fa434be7c!]
GORELEASER_KEY: ENCRYPTED[!9b80b6ef684ceaf40edd4c7af93014ee156c8aba7e6e5795f41c482729887b5c31f36b651491d790f1f668670888d9fd!]
install_script: brew install go goreleaser/tap/goreleaser-pro
info_script:
- xcodebuild -version
- swift -version
release_script: goreleaser
+1
View File
@@ -1 +1,2 @@
*.png filter=lfs diff=lfs merge=lfs -text
*.gif filter=lfs diff=lfs merge=lfs -text
+15 -7
View File
@@ -7,15 +7,13 @@ builds:
goarch:
- arm64
prebuilt:
path: .build/{{ .Arch }}-apple-macosx/release/tart
path: .build/{{ .Arch }}-apple-macosx/debug/tart
before:
hooks:
- swift build -c release --product tart
after:
hooks:
- codesign --sign - --entitlements Sources/tart/tart.entitlements --force .build/arm64-apple-macosx/release/tart
- .ci/set-version.sh
- swift build -c debug --product tart
- codesign --sign - --entitlements Resources/tart.entitlements --force .build/arm64-apple-macosx/debug/tart
archives:
- id: binary
@@ -34,7 +32,17 @@ brews:
tap:
owner: cirruslabs
name: homebrew-cli
caveats: See the Github repository for more information
caveats: See the GitHub repository for more information
homepage: https://github.com/cirruslabs/tart
description: Run macOS VMs on Apple Silicon
skip_upload: auto
dependencies:
- "cirruslabs/cli/softnet"
custom_block: |
depends_on :macos => :monterey
on_macos do
unless Hardware::CPU.arm?
odie "Tart only works on Apple Silicon!"
end
end
+2 -2
View File
@@ -1,6 +1,6 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="sign debug" type="ShConfigurationType">
<option name="SCRIPT_TEXT" value="codesign --sign - --entitlements Sources/tart/tart.entitlements --force .build/debug/tart" />
<option name="SCRIPT_TEXT" value="codesign --sign - --entitlements Resources/tart.entitlements --force .build/debug/tart" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="$PROJECT_DIR$/scripts/sign.sh" />
<option name="SCRIPT_OPTIONS" value="" />
@@ -14,4 +14,4 @@
<envs />
<method v="2" />
</configuration>
</component>
</component>
+135
View File
@@ -1,5 +1,32 @@
{
"pins" : [
{
"identity" : "async-http-client",
"kind" : "remoteSourceControl",
"location" : "https://github.com/swift-server/async-http-client",
"state" : {
"revision" : "df87a860fdc41a595d5ca67f74cde9adbccc099a",
"version" : "1.11.4"
}
},
{
"identity" : "dynamic",
"kind" : "remoteSourceControl",
"location" : "https://github.com/mhdhejazi/Dynamic",
"state" : {
"branch" : "master",
"revision" : "772883073d044bc754d401cabb6574624eb3778f"
}
},
{
"identity" : "swift-algorithms",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-algorithms",
"state" : {
"revision" : "b14b7f4c528c942f121c8b860b9410b2bf57825e",
"version" : "1.0.0"
}
},
{
"identity" : "swift-argument-parser",
"kind" : "remoteSourceControl",
@@ -8,6 +35,114 @@
"revision" : "f3c9084a71ef4376f2fabbdf1d3d90a49f1fabdb",
"version" : "1.1.2"
}
},
{
"identity" : "swift-atomics",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-atomics.git",
"state" : {
"revision" : "919eb1d83e02121cdb434c7bfc1f0c66ef17febe",
"version" : "1.0.2"
}
},
{
"identity" : "swift-case-paths",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-case-paths",
"state" : {
"revision" : "ce9c0d897db8a840c39de64caaa9b60119cf4be8",
"version" : "0.8.1"
}
},
{
"identity" : "swift-log",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-log.git",
"state" : {
"revision" : "5d66f7ba25daf4f94100e7022febf3c75e37a6c7",
"version" : "1.4.2"
}
},
{
"identity" : "swift-nio",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio.git",
"state" : {
"revision" : "124119f0bb12384cef35aa041d7c3a686108722d",
"version" : "2.40.0"
}
},
{
"identity" : "swift-nio-extras",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-extras.git",
"state" : {
"revision" : "8eea84ec6144167354387ef9244b0939f5852dc8",
"version" : "1.11.0"
}
},
{
"identity" : "swift-nio-http2",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-http2.git",
"state" : {
"revision" : "108ac15087ea9b79abb6f6742699cf31de0e8772",
"version" : "1.22.0"
}
},
{
"identity" : "swift-nio-ssl",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-ssl.git",
"state" : {
"revision" : "1750873bce84b4129b5303655cce2c3d35b9ed3a",
"version" : "2.19.0"
}
},
{
"identity" : "swift-nio-transport-services",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-transport-services.git",
"state" : {
"revision" : "1a4692acb88156e3da1b0c6732a8a38b2a744166",
"version" : "1.12.0"
}
},
{
"identity" : "swift-numerics",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-numerics",
"state" : {
"revision" : "0a5bc04095a675662cf24757cc0640aa2204253b",
"version" : "1.0.2"
}
},
{
"identity" : "swift-parsing",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/swift-parsing",
"state" : {
"revision" : "28d32e9ace1c4c43f5e5a177be837a202494c2d5",
"version" : "0.9.2"
}
},
{
"identity" : "swiftdate",
"kind" : "remoteSourceControl",
"location" : "https://github.com/malcommac/SwiftDate",
"state" : {
"revision" : "6190d0cefff3013e77ed567e6b074f324e5c5bf5",
"version" : "6.3.1"
}
},
{
"identity" : "xctest-dynamic-overlay",
"kind" : "remoteSourceControl",
"location" : "https://github.com/pointfreeco/xctest-dynamic-overlay",
"state" : {
"revision" : "50a70a9d3583fe228ce672e8923010c8df2deddd",
"version" : "0.2.1"
}
}
],
"version" : 2
+16 -9
View File
@@ -1,7 +1,6 @@
// swift-tools-version:5.6
// swift-tools-version:5.7
import PackageDescription
let package = Package(
name: "Tart",
platforms: [
@@ -12,14 +11,22 @@ let package = Package(
],
dependencies: [
.package(url: "https://github.com/apple/swift-argument-parser", from: "1.1.2"),
.package(url: "https://github.com/mhdhejazi/Dynamic", branch: "master"),
.package(url: "https://github.com/pointfreeco/swift-parsing", from: "0.9.2"),
.package(url: "https://github.com/swift-server/async-http-client", from: "1.11.4"),
.package(url: "https://github.com/apple/swift-algorithms", from: "1.0.0"),
.package(url: "https://github.com/malcommac/SwiftDate", from: "6.3.1")
],
targets: [
.executableTarget(name: "tart",
dependencies: [
.product(name: "ArgumentParser", package: "swift-argument-parser"),
],
resources: [
.process("Resources/AppIcon.png")
]),
.executableTarget(name: "tart", dependencies: [
.product(name: "Algorithms", package: "swift-algorithms"),
.product(name: "ArgumentParser", package: "swift-argument-parser"),
.product(name: "AsyncHTTPClient", package: "async-http-client"),
.product(name: "Dynamic", package: "Dynamic"),
.product(name: "Parsing", package: "swift-parsing"),
.product(name: "SwiftDate", package: "SwiftDate"),
]),
.testTarget(name: "TartTests", dependencies: ["tart"])
]
)
+253 -2
View File
@@ -1,3 +1,254 @@
# Tart
![Tart – open source virtualization for your automation needs](Resources/TartSocial.png)
macOS VMs on Apple Silicon to use in CI and other automations
*Tart* is a virtualization toolset to build, run and manage macOS and Linux virtual machines on Apple Silicon.
Built by CI engineers for your automation needs. Here are some highlights of Tart:
* Tart uses Apple's own `Virtualization.Framework` for [near-native performance](https://browser.geekbench.com/v5/cpu/compare/14966395?baseline=14966339).
* Push/Pull virtual machines from any OCI-compatible container registry.
* Use Tart Packer Plugin to automate VM creation.
* Built-in CI integration.
Try running a Tart VM on your Apple Silicon device running macOS Monterey or later (will download a 25 GB image):
```shell
brew install cirruslabs/cli/tart
tart clone ghcr.io/cirruslabs/macos-monterey-base:latest monterey-base
tart run monterey-base
```
![tart VM view app](Resources/TartScreenshot.png)
## CI Integration
Tart itself is only responsible for managing virtual machines, but we've built Tart support into a tool called Cirrus CLI
also developed by Cirrus Labs. [Cirrus CLI](https://github.com/cirruslabs/cirrus-cli) is a command line tool with
one configuration format to execute common CI steps (run a script, cache a folder, etc.) locally or in any CI system.
We built Cirrus CLI to solve "But it works on my machine!" problem.
Here is an example of a `.cirrus.yml` configuration file which will start a Tart VM, will copy over working directory and
will run scripts and [other instructions](https://cirrus-ci.org/guide/writing-tasks/#supported-instructions) inside the virtual machine:
```yaml
task:
name: hello
macos_instance:
# can be a remote or a local virtual machine
image: ghcr.io/cirruslabs/macos-monterey-base:latest
hello_script:
- echo "Hello from within a Tart VM!"
- echo "Here is my CPU info:"
- sysctl -n machdep.cpu.brand_string
- sleep 15
```
Put the above `.cirrus.yml` file in the root of your repository and run it with the following command:
```shell
brew install cirruslabs/cli/cirrus
cirrus run
```
![Cirrus CLI Run](Resources/TartCirrusCLI.gif)
[Cirrus CI](https://cirrus-ci.org/) already leverages Tart to power its macOS cloud infrastructure. The `.cirrus.yml`
config from above will just work in Cirrus CI and your tasks will be executed inside Tart VMs in our cloud.
**Note:** Cirrus CI only allows [images managed and regularly updated by us](https://github.com/orgs/cirruslabs/packages?tab=packages&q=macos).
### Retrieving artifacts from within Tart VMs
In many cases there is a need to retrieve particular files or a folder from within a Tart virtual machine.
For example, the below `.cirrus.yml` configuration defines a single task that builds a `tart` binary and
exposes it via [`artifacts` instruction](https://cirrus-ci.org/guide/writing-tasks/#artifacts-instruction):
```yaml
task:
name: Build
macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest
build_script: swift build --product tart
binary_artifacts:
path: .build/debug/tart
```
Running Cirrus CLI with `--artifacts-dir` will write defined `artifacts` to the provided local directory on the host:
```bash
cirrus run --artifacts-dir artifacts
```
Note that all retrieved artifacts will be prefixed with the associated task name and `artifacts` instruction name.
For the example above, `tart` binary will be saved to `$PWD/artifacts/Build/binary/.build/debug/tart`.
## Virtual Machine Management
### Creating from scratch
Tart supports macOS and Linux virtual machines. All commands like `run` and `pull` work the same way regarding of the underlying OS a particular VM image has.
The only difference is how such VM images are created. Please check sections below for [macOS](#creating-a-macos-vm-image-from-scratch) and [Linux](#creating-a-linux-vm-image-from-scratch) instructions.
#### Creating a macOS VM image from scratch
Tart can create VMs from `*.ipsw` files. You can download a specific `*.ipsw` file [here](https://ipsw.me/) or you can
use `latest` instead of a path to `*.ipsw` to download the latest available version:
```shell
tart create --from-ipsw=latest monterey-vanilla
tart run monterey-vanilla
```
After the initial booting of the VM you'll need to manually go through the macOS installation process. As a convention we recommend creating an `admin` user with an `admin` password. After the regular installation please do some additional modifications in the VM:
1. Enable Auto-Login. Users & Groups -> Login Options -> Automatic login -> admin.
2. Allow SSH. Sharing -> Remote Login
3. Disable Lock Screen. Preferences -> Lock Screen -> disable "Require Password" after 5.
4. Disable Screen Saver.
5. Run `sudo visudo` in Terminal, find `%admin ALL=(ALL) ALL` add `admin ALL=(ALL) NOPASSWD: ALL` to allow sudo without a password.
#### Creating a Linux VM image from scratch
```bash
# Create a bare VM
tart create --linux ubuntu
# Install Ubuntu
tart run --disk focal-desktop-arm64.iso ubuntu
# Run VM
tart run ubuntu
```
After the initial setup please make sure your VM can be SSH-ed into by running the following commands inside your VM:
```shell
sudo apt update
sudo apt install -y openssh-server
sudo ufw allow ssh
```
### Configuring a VM
By default, a tart VM uses 2 CPUs and 4 GB of memory with a `1024x768` display. This can be changed with `tart set` command.
Please refer to `tart set --help` for additional details.
### Building with Packer
Please refer to [Tart Packer Plugin repository](https://github.com/cirruslabs/packer-plugin-tart) for setup instructions.
Here is an example of a template to build `monterey-base` local image based of a remote image:
```json
{
"builders": [
{
"name": "tart",
"type": "tart-cli",
"vm_base_name": "tartvm/vanilla:latest",
"vm_name": "monterey-base",
"cpu_count": 4,
"memory_gb": 8,
"disk_size_gb": 32,
"ssh_username": "admin",
"ssh_password": "admin",
"ssh_timeout": "120s"
}
],
"provisioners": [
{
"inline": [
"echo 'Disabling spotlight indexing...'",
"sudo mdutil -a -i off"
],
"type": "shell"
},
# more provisioners
]
}
```
Here is a [repository with Packer templates](https://github.com/cirruslabs/macos-image-templates) used to build [all the images managed by us](https://github.com/orgs/cirruslabs/packages?tab=packages&q=macos).
### Working with a Remote OCI Container Registry
For example, let's say you want to push/pull images to a registry hosted at https://acme.io/.
#### Registry Authorization
First, you need to log in and save credential for `acme.io` host via `tart login` command:
```shell
tart login acme.io
```
Credentials are securely stored in Keychain.
#### Pushing a Local Image
Once credentials are saved for `acme.io`, run the following command to push a local images remotely with two tags:
```shell
tart push my-local-vm-name acme.io/remoteorg/name:latest acme.io/remoteorg/name:v1.0.0
```
#### Pulling a Remote Image
You can either pull an image:
```shell
tart pull acme.io/remoteorg/name:latest
```
...or instantiate a VM from a remote image:
```shell
tart clone acme.io/remoteorg/name:latest my-local-vm-name
```
This invocation calls the `tart pull` implicitly (if the image is not being present) before doing the actual cloning.
## FAQ
<details>
<summary>How Tart is different from Anka</summary>
Under the hood Tart is using the same technology as Anka 3.0 so there should be no real difference in performance
or features supported. If there is some feature missing please don't hesitate to [create a feature request](https://github.com/cirruslabs/tart/issues).
Instead of Anka Registry, Tart can work with any OCI-compatible container registry.
Tart doesn't yet have an analogue of Anka Controller for managing long living VMs. Please take a look at [CI integration](#ci-integration)
section for an option to run ephemeral VMs for your needs.
</details>
<details>
<summary>Why Tart is free and open sourced?</summary>
Tart is a relatively small project, and it didn't feel right to try to monetize it.
Apple did all the heavy lifting with their `Virtualization.Framework`.
</details>
<details>
<summary>How to change VM's disk size?</summary>
You can choose disk size upon creation of a virtual machine:
```shell
tart create --from-ipsw=latest --disk-size=25 monterey-vanilla
```
For an existing VM please use [Packer Plugin](https://github.com/cirruslabs/packer-plugin-tart) which can increase
disk size for new virtual machines. Here is an example of [how to change disk size in a Packer template](https://github.com/cirruslabs/macos-image-templates/blob/fb0bcf68e0b093129136875c050205a66729b596/templates/base.pkr.hcl#L15).
</details>
<details>
<summary>VM location on disk</summary>
Tart stores all it's files in `~/.tart/` directory. Local images that you can run are stored in `~/.tart/vms/`.
Remote images are pulled into `~/.tart/vms/cache/OCIs/`.
</details>
<details>
<summary>Nested virtualization support?</summary>
Tart is limited by functionality of Apple's `Virtualization.Framework`. At the moment `Virtualization.Framework`
doesn't support nested virtualization.
</details>
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1fe96aed7a965b075300f092a3ca76e09053eb7cf2f3125c3a819098a8bc4b31
size 123360
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a3a324193c4bd7797102765ab16f44adf58e49ca615bac3963cefd0d3a10594
size 339678
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a43f541b1ab0b57ae2060d371cba5dbb1f5c80b89c76434b7154d8144f66e61
size 205325
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:769dcd5411f44071cb46f63459118fef728c866a581cf94a28811f407ca9827d
size 606936
-119
View File
@@ -1,119 +0,0 @@
import Foundation
import Network
import Virtualization
struct ARPCommandFailedError: Error, CustomStringConvertible {
var terminationReason: Process.TerminationReason
var terminationStatus: Int32
var description: String {
var reason: String
switch terminationReason {
case .exit:
reason = "exit code \(terminationStatus)"
case .uncaughtSignal:
reason = "uncaught signal"
default:
reason = "unknown reason"
}
return "arp command failed: \(reason)"
}
}
struct ARPCommandYieldedInvalidOutputError: Error, CustomStringConvertible {
var explanation: String
var description: String {
"arp command yielded invalid output: \(explanation)"
}
}
struct ARPCacheInternalError: Error, CustomStringConvertible {
var explanation: String
var description: String {
"ARPCache internal error: \(explanation)"
}
}
struct ARPCache {
static func ResolveMACAddress(macAddress: MACAddress, bridgeOnly: Bool = true) throws -> IPv4Address? {
let process = Process.init()
process.executableURL = URL.init(fileURLWithPath: "/usr/sbin/arp")
process.arguments = ["-an"]
let pipe = Pipe()
process.standardOutput = pipe
process.standardError = pipe
process.standardInput = FileHandle.nullDevice
try process.run()
process.waitUntilExit()
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
throw ARPCommandFailedError(
terminationReason: process.terminationReason,
terminationStatus: process.terminationStatus)
}
guard let rawLines = try pipe.fileHandleForReading.readToEnd() else {
throw ARPCommandYieldedInvalidOutputError(explanation: "empty output")
}
let lines = String(decoding: rawLines, as: UTF8.self)
.trimmingCharacters(in: .whitespacesAndNewlines)
.components(separatedBy: "\n")
// Based on https://opensource.apple.com/source/network_cmds/network_cmds-606.40.2/arp.tproj/arp.c.auto.html
let regex = try NSRegularExpression(pattern: #"^.* \((?<ip>.*)\) at (?<mac>.*) on (?<interface>.*) .*$"#)
for line in lines {
let nsLineRange = NSRange(line.startIndex..<line.endIndex, in: line)
guard let match = regex.firstMatch(in: line, range: nsLineRange) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "unparseable entry \"\(line)\"")
}
let rawIP = try match.getCaptureGroup(name: "ip", for: line)
guard let ip = IPv4Address(rawIP) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "failed to parse IPv4 address \(rawIP)")
}
let rawMAC = try match.getCaptureGroup(name: "mac", for: line)
if rawMAC == "(incomplete)" {
continue
}
guard let mac = MACAddress(fromString: rawMAC) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "failed to parse MAC address \(rawMAC)")
}
let interface = try match.getCaptureGroup(name: "interface", for: line)
if bridgeOnly && !interface.starts(with: "bridge") {
continue
}
if macAddress == mac {
return ip
}
}
return nil
}
}
extension NSTextCheckingResult {
func getCaptureGroup(name: String, for string: String) throws -> String {
let nsRange = self.range(withName: name)
if nsRange.location == NSNotFound {
throw ARPCacheInternalError(explanation: "attempted to retrieve non-existent named capture group \(name)")
}
guard let range = Range.init(nsRange, in: string) else {
throw ARPCacheInternalError(explanation: "failed to convert NSRange to Range")
}
return String(string[range])
}
}
+13
View File
@@ -0,0 +1,13 @@
struct CI {
private static let rawVersion = "${CIRRUS_TAG}"
static var version: String {
rawVersion.expanded() ? rawVersion : "SNAPSHOT"
}
}
private extension String {
func expanded() -> Bool {
!isEmpty && !starts(with: "$")
}
}
+38 -10
View File
@@ -1,7 +1,6 @@
import ArgumentParser
import Foundation
import SystemConfiguration
import Virtualization
struct Clone: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Clone a VM")
@@ -12,19 +11,36 @@ struct Clone: AsyncParsableCommand {
@Argument(help: "new VM name")
var newName: String
@Flag(help: "connect to the OCI registry via insecure HTTP protocol")
var insecure: Bool = false
func validate() throws {
if newName.contains("/") {
throw ValidationError("<new-name> should be a local name")
}
}
func run() async throws {
do {
let vmStorage = VMStorage()
let sourceVMDir = try vmStorage.read(sourceName)
let newVMDir = try vmStorage.create(newName)
let ociStorage = VMStorageOCI()
let localStorage = VMStorageLocal()
try FileManager.default.copyItem(at: sourceVMDir.configURL, to: newVMDir.configURL)
try FileManager.default.copyItem(at: sourceVMDir.nvramURL, to: newVMDir.nvramURL)
try FileManager.default.copyItem(at: sourceVMDir.diskURL, to: newVMDir.diskURL)
if let remoteName = try? RemoteName(sourceName), !ociStorage.exists(remoteName) {
// Pull the VM in case it's OCI-based and doesn't exist locally yet
let registry = try Registry(host: remoteName.host, namespace: remoteName.namespace, insecure: insecure)
try await ociStorage.pull(remoteName, registry: registry)
}
var newVMConfig = try VMConfig(fromURL: newVMDir.configURL)
newVMConfig.macAddress = VZMACAddress.randomLocallyAdministered()
try newVMConfig.save(toURL: newVMDir.configURL)
let sourceVM = try VMStorageHelper.open(sourceName)
let generateMAC = try localStorage.hasVMsWithMACAddress(macAddress: sourceVM.macAddress())
let tmpVMDir = try VMDirectory.temporary()
try await withTaskCancellationHandler(operation: {
try sourceVM.clone(to: tmpVMDir, generateMAC: generateMAC)
try localStorage.move(newName, from: tmpVMDir)
}, onCancel: {
try? FileManager.default.removeItem(at: tmpVMDir.baseURL)
})
Foundation.exit(0)
} catch {
@@ -34,3 +50,15 @@ struct Clone: AsyncParsableCommand {
}
}
}
fileprivate extension VMDirectory {
func macAddress() throws -> String {
try VMConfig(fromURL: configURL).macAddress.string
}
}
fileprivate extension VMStorageLocal {
func hasVMsWithMACAddress(macAddress: String) throws -> Bool {
try list().contains { try $1.macAddress() == macAddress }
}
}
+28 -10
View File
@@ -9,27 +9,45 @@ struct Create: AsyncParsableCommand {
@Argument(help: "VM name")
var name: String
@Option(help: ArgumentHelp("Path to the IPSW file (or \"latest\") to fetch the latest appropriate IPSW", valueName: "path"))
@Option(help: ArgumentHelp("create a macOS VM using path to the IPSW file (or \"latest\") to fetch the latest appropriate IPSW", valueName: "path"))
var fromIPSW: String?
@Flag(help: "create a Linux VM")
var linux: Bool = false
@Option(help: ArgumentHelp("Disk size in Gb"))
var diskSize: UInt8 = 32
var diskSize: UInt16 = 50
func validate() throws {
if fromIPSW == nil {
throw ValidationError("Please specify a --from-ipsw option!")
if fromIPSW == nil && !linux {
throw ValidationError("Please specify either a --from-ipsw or --linux option!")
}
}
func run() async throws {
do {
let vmDir = try VMStorage().create(name)
let tmpVMDir = try VMDirectory.temporary()
try await withTaskCancellationHandler(operation: {
if let fromIPSW = fromIPSW {
if fromIPSW == "latest" {
_ = try await VM(vmDir: tmpVMDir, ipswURL: nil, diskSizeGB: diskSize)
} else {
_ = try await VM(vmDir: tmpVMDir, ipswURL: URL(fileURLWithPath: fromIPSW), diskSizeGB: diskSize)
}
}
if fromIPSW! == "latest" {
_ = try await VM(vmDir: vmDir, ipswURL: nil, diskSizeGB: diskSize)
} else {
_ = try await VM(vmDir: vmDir, ipswURL: URL(fileURLWithPath: fromIPSW!), diskSizeGB: diskSize)
}
if linux {
if #available(macOS 13, *) {
_ = try await VM.linux(vmDir: tmpVMDir, diskSizeGB: diskSize)
} else {
throw UnsupportedOSError()
}
}
try VMStorageLocal().move(name, from: tmpVMDir)
}, onCancel: {
try? FileManager.default.removeItem(at: tmpVMDir.baseURL)
})
Foundation.exit(0)
} catch {
+1 -1
View File
@@ -10,7 +10,7 @@ struct Delete: AsyncParsableCommand {
func run() async throws {
do {
try VMStorage().delete(name)
try VMStorageHelper.delete(name)
Foundation.exit(0)
} catch {
+28 -4
View File
@@ -1,5 +1,6 @@
import ArgumentParser
import Foundation
import Network
import SystemConfiguration
struct IP: AsyncParsableCommand {
@@ -8,19 +9,28 @@ struct IP: AsyncParsableCommand {
@Argument(help: "VM name")
var name: String
@Option(help: "Number of seconds to wait for a potential VM booting")
var wait: UInt16 = 0
func run() async throws {
do {
let vmDir = try VMStorage().read(name)
let vmDir = try VMStorageLocal().open(name)
let vmConfig = try VMConfig.init(fromURL: vmDir.configURL)
let vmMacAddress = MACAddress(fromString: vmConfig.macAddress.string)!
let vmMACAddress = MACAddress(fromString: vmConfig.macAddress.string)!
guard let ip = try ARPCache.ResolveMACAddress(macAddress: vmMacAddress) else {
guard let ipViaDHCP = try await IP.resolveIP(vmMACAddress, secondsToWait: wait) else {
print("no IP address found, is your VM running?")
Foundation.exit(1)
}
print(ip)
if let ipViaARP = try ARPCache.ResolveMACAddress(macAddress: vmMACAddress), ipViaARP != ipViaDHCP {
fputs("WARNING: DHCP lease and ARP cache entries for MAC address \(vmMACAddress) differ: "
+ "got \(ipViaDHCP) and \(ipViaARP) respectively, consider reporting this case to"
+ " https://github.com/cirruslabs/tart/issues/172\n", stderr)
}
print(ipViaDHCP)
Foundation.exit(0)
} catch {
@@ -29,4 +39,18 @@ struct IP: AsyncParsableCommand {
Foundation.exit(1)
}
}
static public func resolveIP(_ vmMACAddress: MACAddress, secondsToWait: UInt16) async throws -> IPv4Address? {
let waitUntil = Calendar.current.date(byAdding: .second, value: Int(secondsToWait), to: Date.now)!
repeat {
if let ip = try Leases().resolveMACAddress(macAddress: vmMACAddress) {
return ip
}
try await Task.sleep(nanoseconds: 1_000_000)
} while Date.now < waitUntil
return nil
}
}
+10 -3
View File
@@ -7,9 +7,10 @@ struct List: AsyncParsableCommand {
func run() async throws {
do {
for vmURL in try VMStorage().list() {
print(vmURL)
}
print("Source\tName")
displayTable("local", try VMStorageLocal().list())
displayTable("oci", try VMStorageOCI().list().map { (name, vmDir, _) in (name, vmDir) })
Foundation.exit(0)
} catch {
@@ -18,4 +19,10 @@ struct List: AsyncParsableCommand {
Foundation.exit(1)
}
}
private func displayTable(_ source: String, _ vms: [(String, VMDirectory)]) {
for (name, _) in vms.sorted(by: { left, right in left.0 < right.0 }) {
print("\(source)\t\(name)")
}
}
}
+81
View File
@@ -0,0 +1,81 @@
import ArgumentParser
import Dispatch
import SwiftUI
struct Login: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Login to a registry")
@Argument(help: "host")
var host: String
@Option(help: "username")
var username: String?
@Flag(help: "password-stdin")
var passwordStdin: Bool = false
@Flag(help: "connect to the OCI registry via insecure HTTP protocol")
var insecure: Bool = false
func validate() throws {
let usernameProvided = username != nil
let passwordProvided = passwordStdin
if usernameProvided != passwordProvided {
throw ValidationError("both --username and --password-stdin are required")
}
}
func run() async throws {
do {
var user: String
var password: String
if let username = username {
user = username
let passwordData = FileHandle.standardInput.readDataToEndOfFile()
password = String(decoding: passwordData, as: UTF8.self)
} else {
(user, password) = try StdinCredentials.retrieve()
}
let credentialsProvider = DictionaryCredentialsProvider([
host: (user, password)
])
do {
let registry = try Registry(host: host, namespace: "", insecure: insecure,
credentialsProviders: [credentialsProvider])
try await registry.ping()
} catch {
print("invalid credentials: \(error)")
Foundation.exit(1)
}
try KeychainCredentialsProvider().store(host: host, user: user, password: password)
Foundation.exit(0)
} catch {
print(error)
Foundation.exit(1)
}
}
}
fileprivate class DictionaryCredentialsProvider: CredentialsProvider {
var credentials: Dictionary<String, (String, String)>
init(_ credentials: Dictionary<String, (String, String)>) {
self.credentials = credentials
}
func retrieve(host: String) throws -> (String, String)? {
credentials[host]
}
func store(host: String, user: String, password: String) throws {
credentials[host] = (user, password)
}
}
+108
View File
@@ -0,0 +1,108 @@
import ArgumentParser
import Dispatch
import SwiftUI
import SwiftDate
struct Prune: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Prune OCI and IPSW caches")
@Option(help: ArgumentHelp("Remove cache entries last accessed more than n days ago",
discussion: "For example, --older-than=7 will remove entries that weren't accessed by Tart in the last 7 days.",
valueName: "n"))
var olderThan: UInt?
@Option(help: ArgumentHelp("Remove least recently used cache entries that do not fit the specified cache size budget n, expressed in gigabytes",
discussion: "For example, --cache-budget=50 will effectively shrink all caches to a total size of 50 gigabytes.",
valueName: "n"))
var cacheBudget: UInt?
@Flag(help: .hidden)
var gc: Bool = false
func validate() throws {
if olderThan == nil && cacheBudget == nil && !gc {
throw ValidationError("at least one pruning criteria must be specified")
}
}
func run() async throws {
do {
if gc {
try VMStorageOCI().gc()
}
// Clean up cache entries based on last accessed date
if let olderThan = olderThan {
let olderThanInterval = Int(exactly: olderThan)!.days.timeInterval
let olderThanDate = Date().addingTimeInterval(olderThanInterval)
try Prune.pruneOlderThan(olderThanDate: olderThanDate)
}
// Clean up cache entries based on imposed cache size limit and entry's last accessed date
if let cacheBudget = cacheBudget {
try Prune.pruneCacheBudget(cacheBudgetBytes: UInt64(cacheBudget) * 1024 * 1024 * 1024)
}
Foundation.exit(0)
} catch {
print(error)
Foundation.exit(1)
}
}
static func pruneOlderThan(olderThanDate: Date) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages.flatMap { try $0.prunables() }
try prunables.filter { try $0.accessDate() <= olderThanDate }.forEach { try $0.delete() }
}
static func pruneCacheBudget(cacheBudgetBytes: UInt64) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages
.flatMap { try $0.prunables() }
.sorted { try $0.accessDate() < $1.accessDate() }
let cacheUsedBytes = try prunables.map { try $0.sizeBytes() }.reduce(0, +)
var cacheReclaimedBytes: Int = 0
var it = prunables.makeIterator()
while (cacheUsedBytes - cacheReclaimedBytes) > cacheBudgetBytes {
guard let prunable = it.next() else {
break
}
cacheReclaimedBytes -= try prunable.sizeBytes()
try prunable.delete()
}
}
static func pruneReclaim(reclaimBytes: UInt64) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages
.flatMap { try $0.prunables() }
.sorted { try $0.accessDate() < $1.accessDate() }
// Does it even make sense to start?
let cacheUsedBytes = try prunables.map { try $0.sizeBytes() }.reduce(0, +)
if cacheUsedBytes < reclaimBytes {
return
}
var cacheReclaimedBytes: Int = 0
var it = prunables.makeIterator()
while cacheReclaimedBytes <= reclaimBytes {
guard let prunable = it.next() else {
break
}
cacheReclaimedBytes -= try prunable.sizeBytes()
try prunable.delete()
}
}
}
+38
View File
@@ -0,0 +1,38 @@
import ArgumentParser
import Dispatch
import SwiftUI
struct Pull: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Pull a VM from a registry")
@Argument(help: "remote VM name")
var remoteName: String
@Flag(help: "connect to the OCI registry via insecure HTTP protocol")
var insecure: Bool = false
func run() async throws {
do {
// Be more liberal when accepting local image as argument,
// see https://github.com/cirruslabs/tart/issues/36
if VMStorageLocal().exists(remoteName) {
print("\"\(remoteName)\" is a local image, nothing to pull here!")
Foundation.exit(0)
}
let remoteName = try RemoteName(remoteName)
let registry = try Registry(host: remoteName.host, namespace: remoteName.namespace, insecure: insecure)
defaultLogger.appendNewLine("pulling \(remoteName)...")
try await VMStorageOCI().pull(remoteName, registry: registry)
Foundation.exit(0)
} catch {
print(error)
Foundation.exit(1)
}
}
}
+93
View File
@@ -0,0 +1,93 @@
import ArgumentParser
import Dispatch
import Foundation
import Compression
struct Push: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Push a VM to a registry")
@Argument(help: "local VM name")
var localName: String
@Argument(help: "remote VM name(s)")
var remoteNames: [String]
@Flag(help: "connect to the OCI registry via insecure HTTP protocol")
var insecure: Bool = false
@Option(help: ArgumentHelp("chunk size in MB if registry supports chunked uploads",
discussion: """
By default monolithic method is used for uploading blobs to the registry but some registries support a more efficient chunked method.
For example, AWS Elastic Container Registry supports only chunks larger than 5MB but GitHub Container Registry supports only chunks smaller than 4MB. Google Container Registry on the other hand doesn't support chunked uploads at all.
Please refer to the documentation of your particular registry in order to see if this option is suitable for you and what's the recommended chunk size.
"""))
var chunkSize: Int = 0
@Flag(help: ArgumentHelp("cache pushed images locally",
discussion: "Increases disk usage, but saves time if you're going to pull the pushed images later."))
var populateCache: Bool = false
func run() async throws {
do {
let localVMDir = try VMStorageLocal().open(localName)
// Parse remote names supplied by the user
let remoteNames = try remoteNames.map{
try RemoteName($0)
}
// Group remote names by registry
struct RegistryIdentifier: Hashable, Equatable {
var host: String
var namespace: String
}
let registryGroups = Dictionary(grouping: remoteNames, by: {
RegistryIdentifier(host: $0.host, namespace: $0.namespace)
})
// Push VM
for (registryIdentifier, remoteNamesForRegistry) in registryGroups {
let registry = try Registry(host: registryIdentifier.host, namespace: registryIdentifier.namespace,
insecure: insecure)
defaultLogger.appendNewLine("pushing \(localName) to "
+ "\(registryIdentifier.host)/\(registryIdentifier.namespace)\(remoteNamesForRegistry.referenceNames())...")
let pushedRemoteName = try await localVMDir.pushToRegistry(
registry: registry,
references: remoteNamesForRegistry.map{ $0.reference.value },
chunkSizeMb: chunkSize
)
// Populate the local cache (if requested)
if populateCache {
let ociStorage = VMStorageOCI()
let expectedPushedVMDir = try ociStorage.create(pushedRemoteName)
try localVMDir.clone(to: expectedPushedVMDir, generateMAC: false)
for remoteName in remoteNamesForRegistry {
try ociStorage.link(from: remoteName, to: pushedRemoteName)
}
}
}
Foundation.exit(0)
} catch {
print(error)
Foundation.exit(1)
}
}
}
extension Collection where Element == RemoteName {
func referenceNames() -> String {
let references = self.map{ $0.reference.fullyQualified }
switch count {
case 0: return "∅"
case 1: return references.first!
default: return "{" + references.joined(separator: ",") + "}"
}
}
}
+147 -32
View File
@@ -5,63 +5,178 @@ import Virtualization
var vm: VM?
struct IPNotFound: Error {
}
struct Run: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Run a VM")
@Argument(help: "VM name")
var name: String
@Flag var noGraphics: Bool = false
@Flag(help: ArgumentHelp(
"Don't open a UI window.",
discussion: "Useful for integrating Tart VMs into other tools.\nUse `tart ip` in order to get an IP for SSHing or VNCing into the VM."))
var noGraphics: Bool = false
@Flag(help: "Boot into recovery mode")
var recovery: Bool = false
@Flag(help: ArgumentHelp(
"Use screen sharing instead of the built-in UI.",
discussion: "Useful since Screen Sharing supports copy/paste, drag and drop, etc.\n"
+ "Note that Remote Login option should be enabled inside the VM."))
var vnc: Bool = false
@Flag(help: ArgumentHelp(
"Use Virtualization.Framework's VNC server instead of the build-in UI.",
discussion: "Useful since this type of VNC is available in recovery mode and in macOS installation.\n"
+ "Note that this feature is experimental and there may be bugs present when using VNC."))
var vncExperimental: Bool = false
@Flag var withSoftnet: Bool = false
@Option(help: ArgumentHelp("""
Additional disk attachments with an optional read-only specifier\n(e.g. --disk=\"disk.bin\" --disk=\"disk.bin:ro\")
""", discussion: """
Learn how to create a disk image using Disk Utility here:
https://support.apple.com/en-gb/guide/disk-utility/dskutl11888/mac
"""))
var disk: [String] = []
func validate() throws {
if vnc && vncExperimental {
throw ValidationError("--vnc and --vnc-experimental are mutually exclusive")
}
}
@MainActor
func run() async throws {
let vmDir = try VMStorage().read(name)
vm = try VM(vmDir: vmDir)
let vmDir = try VMStorageLocal().open(name)
vm = try VM(
vmDir: vmDir,
withSoftnet: withSoftnet,
additionalDiskAttachments: additionalDiskAttachments()
)
Task {
let vncImpl: VNC? = try {
if vnc {
let vmConfig = try VMConfig.init(fromURL: vmDir.configURL)
return ScreenSharingVNC(vmConfig: vmConfig)
} else if vncExperimental {
return FullFledgedVNC(virtualMachine: vm!.virtualMachine)
} else {
return nil
}
}()
let task = Task {
do {
try await vm!.run()
if let vncImpl = vncImpl {
let vncURL = try await vncImpl.waitForURL()
if noGraphics || ProcessInfo.processInfo.environment["CI"] != nil {
print("VNC server is running at \(vncURL)")
} else {
print("Opening \(vncURL)...")
NSWorkspace.shared.open(vncURL)
}
}
try await vm!.run(recovery)
if let vncImpl = vncImpl {
try vncImpl.stop()
}
Foundation.exit(0)
} catch {
print(error)
if error.localizedDescription.contains("Failed to lock auxiliary storage.") {
print("Virtual machine \"\(name)\" is already running!")
Foundation.exit(2)
}
print(error)
Foundation.exit(1)
}
}
if noGraphics {
let sigintSrc = DispatchSource.makeSignalSource(signal: SIGINT)
sigintSrc.setEventHandler {
task.cancel()
}
sigintSrc.activate()
if noGraphics || vnc || vncExperimental {
dispatchMain()
} else {
// UI mumbo-jumbo
let nsApp = NSApplication.shared
nsApp.setActivationPolicy(.regular)
nsApp.activate(ignoringOtherApps: true)
runUI()
}
}
let icon = Bundle.module.image(forResource: "AppIcon.png")
nsApp.applicationIconImage = icon
func additionalDiskAttachments() throws -> [VZDiskImageStorageDeviceAttachment] {
var result: [VZDiskImageStorageDeviceAttachment] = []
let readOnlySuffix = ":ro"
struct MainApp: App {
var body: some Scene {
WindowGroup(vm!.name) {
Group {
VMView(vm: vm!).onAppear {
NSWindow.allowsAutomaticWindowTabbing = false
}
}.frame(width: CGFloat(vm!.config.display.width), height: CGFloat(vm!.config.display.height))
}.commands {
// Remove some standard menu options
CommandGroup(replacing: .help, addition: {})
CommandGroup(replacing: .newItem, addition: {})
CommandGroup(replacing: .pasteboard, addition: {})
CommandGroup(replacing: .textEditing, addition: {})
CommandGroup(replacing: .undoRedo, addition: {})
CommandGroup(replacing: .windowSize, addition: {})
}
}
for rawDisk in disk {
if rawDisk.hasSuffix(readOnlySuffix) {
result.append(try VZDiskImageStorageDeviceAttachment(
url: URL(fileURLWithPath: String(rawDisk.prefix(rawDisk.count - readOnlySuffix.count))),
readOnly: true
))
} else {
result.append(try VZDiskImageStorageDeviceAttachment(
url: URL(fileURLWithPath: rawDisk),
readOnly: false
))
}
}
MainApp.main()
return result
}
private func runUI() {
let nsApp = NSApplication.shared
nsApp.setActivationPolicy(.regular)
nsApp.activate(ignoringOtherApps: true)
nsApp.applicationIconImage = NSImage(data: AppIconData)
struct MainApp: App {
var body: some Scene {
WindowGroup(vm!.name) {
Group {
VMView(vm: vm!).onAppear {
NSWindow.allowsAutomaticWindowTabbing = false
}
}.frame(width: CGFloat(vm!.config.display.width), height: CGFloat(vm!.config.display.height))
}.commands {
// Remove some standard menu options
CommandGroup(replacing: .help, addition: {})
CommandGroup(replacing: .newItem, addition: {})
CommandGroup(replacing: .pasteboard, addition: {})
CommandGroup(replacing: .textEditing, addition: {})
CommandGroup(replacing: .undoRedo, addition: {})
CommandGroup(replacing: .windowSize, addition: {})
// Replace some standard menu options
CommandGroup(replacing: .appInfo) { AboutTart() }
}
}
}
MainApp.main()
}
}
struct AboutTart: View {
var body: some View {
Button("About Tart") {
NSApplication.shared.orderFrontStandardAboutPanel(options: [
NSApplication.AboutPanelOptionKey.applicationIcon: NSApplication.shared.applicationIconImage as Any,
NSApplication.AboutPanelOptionKey.applicationName: "Tart",
NSApplication.AboutPanelOptionKey.applicationVersion: CI.version,
NSApplication.AboutPanelOptionKey.credits: try! NSAttributedString(markdown: "https://github.com/cirruslabs/tart"),
])
}
}
}
+6 -11
View File
@@ -2,7 +2,7 @@ import ArgumentParser
import Foundation
struct Set: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Modify VM's configuration")
static var configuration = CommandConfiguration(commandName: "set", abstract: "Modify VM's configuration")
@Argument(help: "VM name")
var name: String
@@ -13,16 +13,15 @@ struct Set: AsyncParsableCommand {
@Option(help: "VM memory size in megabytes")
var memory: UInt16?
@Option(help: "VM display settings in a format of <width>x<height>(x<dpi>)?. For example, 1200x800 or 1200x800x72")
@Option(help: "VM display resolution in a format of <width>x<height>. For example, 1200x800")
var display: VMDisplayConfig?
@Option(help: .hidden)
var diskSize: UInt8?
var diskSize: UInt16?
func run() async throws {
do {
let vmStorage = VMStorage()
let vmDir = try vmStorage.read(name)
let vmDir = try VMStorageLocal().open(name)
var vmConfig = try VMConfig(fromURL: vmDir.configURL)
if let cpu = cpu {
@@ -40,13 +39,10 @@ struct Set: AsyncParsableCommand {
if (display.height > 0) {
vmConfig.display.height = display.height
}
if (display.dpi > 0) {
vmConfig.display.dpi = display.dpi
}
}
try vmConfig.save(toURL: vmDir.configURL)
if diskSize != nil {
try vmDir.resizeDisk(diskSize!)
}
@@ -67,8 +63,7 @@ extension VMDisplayConfig: ExpressibleByArgument {
}
self = VMDisplayConfig(
width: parts[safe: 0] ?? 0,
height: parts[safe: 1] ?? 0,
dpi: parts[safe: 2] ?? 0
height: parts[safe: 1] ?? 0
)
}
}
+33
View File
@@ -0,0 +1,33 @@
import Foundation
struct Config {
let tartHomeDir: URL
let tartCacheDir: URL
init() {
var tartHomeDir: URL
if let customTartHome = ProcessInfo.processInfo.environment["TART_HOME"] {
tartHomeDir = URL(fileURLWithPath: customTartHome)
} else {
tartHomeDir = FileManager.default
.homeDirectoryForCurrentUser
.appendingPathComponent(".tart", isDirectory: true)
}
self.tartHomeDir = tartHomeDir
tartCacheDir = tartHomeDir.appendingPathComponent("cache", isDirectory: true)
}
static func jsonEncoder() -> JSONEncoder {
let encoder = JSONEncoder()
encoder.outputFormatting = [.sortedKeys]
return encoder
}
static func jsonDecoder() -> JSONDecoder {
JSONDecoder()
}
}
@@ -0,0 +1,10 @@
import Foundation
enum CredentialsProviderError: Error {
case Failed(message: String)
}
protocol CredentialsProvider {
func retrieve(host: String) throws -> (String, String)?
func store(host: String, user: String, password: String) throws
}
@@ -0,0 +1,63 @@
import Foundation
class HelperProgramCredentialsProvider: CredentialsProvider {
func retrieve(host: String) throws -> (String, String)? {
let dockerConfigURL = FileManager.default.homeDirectoryForCurrentUser.appendingPathComponent(".docker").appendingPathComponent("config.json")
if !FileManager.default.fileExists(atPath: dockerConfigURL.path) {
return nil
}
let config = try JSONDecoder().decode(DockerConfig.self, from: Data(contentsOf: dockerConfigURL))
if let helperProgram = config.credHelpers[host] {
return try executeHelper(binaryName: "docker-credential-\(helperProgram)", host: host)
}
return nil
}
private func executeHelper(binaryName: String, host: String) throws -> (String, String)? {
guard let executableURL = resolveBinaryPath(binaryName) else {
throw CredentialsProviderError.Failed(message: "\(binaryName) not found in PATH")
}
let process = Process.init()
process.executableURL = executableURL
process.arguments = ["get"]
let outPipe = Pipe()
let inPipe = Pipe()
process.standardOutput = outPipe
process.standardError = outPipe
process.standardInput = inPipe
process.launch()
inPipe.fileHandleForWriting.write("\(host)\n".data(using: .utf8)!)
inPipe.fileHandleForWriting.closeFile()
process.waitUntilExit()
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
throw CredentialsProviderError.Failed(message: "Docker helper failed!")
}
let getOutput = try JSONDecoder().decode(
DockerGetOutput.self, from: outPipe.fileHandleForReading.readDataToEndOfFile()
)
return (getOutput.Username, getOutput.Secret)
}
func store(host: String, user: String, password: String) throws {
throw CredentialsProviderError.Failed(message: "Docker helpers don't support storing!")
}
}
struct DockerConfig: Codable {
var credHelpers: Dictionary<String, String> = Dictionary()
}
struct DockerGetOutput: Codable {
var Username: String
var Secret: String
}
@@ -0,0 +1,70 @@
import Foundation
class KeychainCredentialsProvider: CredentialsProvider {
func retrieve(host: String) throws -> (String, String)? {
let query: [String: Any] = [kSecClass as String: kSecClassInternetPassword,
kSecAttrProtocol as String: kSecAttrProtocolHTTPS,
kSecAttrServer as String: host,
kSecMatchLimit as String: kSecMatchLimitOne,
kSecReturnAttributes as String: true,
kSecReturnData as String: true,
kSecAttrLabel as String: "Tart Credentials",
]
var item: CFTypeRef?
let status = SecItemCopyMatching(query as CFDictionary, &item)
if status != errSecSuccess {
if status == errSecItemNotFound {
return nil
}
throw CredentialsProviderError.Failed(message: "Keychain returned unsuccessful status \(status)")
}
guard let item = item as? [String: Any],
let user = item[kSecAttrAccount as String] as? String,
let passwordData = item[kSecValueData as String] as? Data,
let password = String(data: passwordData, encoding: .utf8)
else {
throw CredentialsProviderError.Failed(message: "Keychain item has unexpected format")
}
return (user, password)
}
func store(host: String, user: String, password: String) throws {
let passwordData = password.data(using: .utf8)
let key: [String: Any] = [kSecClass as String: kSecClassInternetPassword,
kSecAttrProtocol as String: kSecAttrProtocolHTTPS,
kSecAttrServer as String: host,
kSecAttrLabel as String: "Tart Credentials",
]
let value: [String: Any] = [kSecAttrAccount as String: user,
kSecValueData as String: passwordData,
]
let status = SecItemCopyMatching(key as CFDictionary, nil)
switch status {
case errSecItemNotFound:
let status = SecItemAdd(key.merging(value) { (current, _) in current } as CFDictionary, nil)
if status != errSecSuccess {
throw CredentialsProviderError.Failed(message: "Keychain failed to add item: \(status.explanation())")
}
case errSecSuccess:
let status = SecItemUpdate(key as CFDictionary, value as CFDictionary)
if status != errSecSuccess {
throw CredentialsProviderError.Failed(message: "Keychain failed to update item: \(status.explanation())")
}
default:
throw CredentialsProviderError.Failed(message: "Keychain failed to find item: \(status.explanation())")
}
}
}
extension OSStatus {
func explanation() -> CFString {
SecCopyErrorMessageString(self, nil) ?? "Unknown status code \(self)." as CFString
}
}
@@ -0,0 +1,31 @@
import Foundation
enum StdinCredentialsError: Error {
case CredentialRequired(which: String)
case CredentialTooLong(message: String)
}
class StdinCredentials {
static func retrieve() throws -> (String, String) {
let user = try readStdinCredential(name: "username", prompt: "User: ", isSensitive: false)
let password = try readStdinCredential(name: "password", prompt: "Password: ", isSensitive: true)
return (user, password)
}
private static func readStdinCredential(name: String, prompt: String, maxCharacters: Int = 255, isSensitive: Bool) throws -> String {
var buf = [CChar](repeating: 0, count: maxCharacters + 1 /* sentinel */ + 1 /* NUL */)
guard let rawCredential = readpassphrase(prompt, &buf, buf.count, isSensitive ? RPP_ECHO_OFF : RPP_ECHO_ON) else {
throw StdinCredentialsError.CredentialRequired(which: name)
}
let credential = String(cString: rawCredential).trimmingCharacters(in: .newlines)
if credential.count > maxCharacters {
throw StdinCredentialsError.CredentialTooLong(
message: "\(name) should contain no more than \(maxCharacters) characters")
}
return credential
}
}
File diff suppressed because one or more lines are too long
+20
View File
@@ -0,0 +1,20 @@
import Foundation
import Virtualization
class IPSWCache: PrunableStorage {
let baseURL: URL
init() throws {
baseURL = Config().tartCacheDir.appendingPathComponent("IPSWs", isDirectory: true)
try FileManager.default.createDirectory(at: baseURL, withIntermediateDirectories: true)
}
func locationFor(image: VZMacOSRestoreImage) -> URL {
baseURL.appendingPathComponent("\(image.buildVersion).ipsw", isDirectory: false)
}
func prunables() throws -> [Prunable] {
try FileManager.default.contentsOfDirectory(at: baseURL, includingPropertiesForKeys: nil)
.filter { $0.lastPathComponent.hasSuffix(".ipsw")}
}
}
+6 -1
View File
@@ -3,6 +3,7 @@ import Foundation
public class ProgressObserver: NSObject {
@objc var progressToObserve: Progress
var observation: NSKeyValueObservation?
var lastTimeUpdated = Date.now
public init(_ progress: Progress) {
progressToObserve = progress
@@ -11,7 +12,11 @@ public class ProgressObserver: NSObject {
func log(_ renderer: Logger) {
renderer.appendNewLine(ProgressObserver.lineToRender(progressToObserve))
observation = observe(\.progressToObserve.fractionCompleted) { progress, _ in
renderer.updateLastLine(ProgressObserver.lineToRender(self.progressToObserve))
let currentTime = Date.now
if self.progressToObserve.isFinished || currentTime.timeIntervalSince(self.lastTimeUpdated) >= 1.0 {
self.lastTimeUpdated = currentTime
renderer.updateLastLine(ProgressObserver.lineToRender(self.progressToObserve))
}
}
}
@@ -0,0 +1,119 @@
import Foundation
import Network
import Virtualization
struct ARPCommandFailedError: Error, CustomStringConvertible {
var terminationReason: Process.TerminationReason
var terminationStatus: Int32
var description: String {
var reason: String
switch terminationReason {
case .exit:
reason = "exit code \(terminationStatus)"
case .uncaughtSignal:
reason = "uncaught signal"
default:
reason = "unknown reason"
}
return "arp command failed: \(reason)"
}
}
struct ARPCommandYieldedInvalidOutputError: Error, CustomStringConvertible {
var explanation: String
var description: String {
"arp command yielded invalid output: \(explanation)"
}
}
struct ARPCacheInternalError: Error, CustomStringConvertible {
var explanation: String
var description: String {
"ARPCache internal error: \(explanation)"
}
}
struct ARPCache {
static func ResolveMACAddress(macAddress: MACAddress, bridgeOnly: Bool = true) throws -> IPv4Address? {
let process = Process.init()
process.executableURL = URL.init(fileURLWithPath: "/usr/sbin/arp")
process.arguments = ["-an"]
let pipe = Pipe()
process.standardOutput = pipe
process.standardError = pipe
process.standardInput = FileHandle.nullDevice
try process.run()
process.waitUntilExit()
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
throw ARPCommandFailedError(
terminationReason: process.terminationReason,
terminationStatus: process.terminationStatus)
}
guard let rawLines = try pipe.fileHandleForReading.readToEnd() else {
throw ARPCommandYieldedInvalidOutputError(explanation: "empty output")
}
let lines = String(decoding: rawLines, as: UTF8.self)
.trimmingCharacters(in: .whitespacesAndNewlines)
.components(separatedBy: "\n")
// Based on https://opensource.apple.com/source/network_cmds/network_cmds-606.40.2/arp.tproj/arp.c.auto.html
let regex = try NSRegularExpression(pattern: #"^.* \((?<ip>.*)\) at (?<mac>.*) on (?<interface>.*) .*$"#)
for line in lines {
let nsLineRange = NSRange(line.startIndex..<line.endIndex, in: line)
guard let match = regex.firstMatch(in: line, range: nsLineRange) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "unparseable entry \"\(line)\"")
}
let rawIP = try match.getCaptureGroup(name: "ip", for: line)
guard let ip = IPv4Address(rawIP) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "failed to parse IPv4 address \(rawIP)")
}
let rawMAC = try match.getCaptureGroup(name: "mac", for: line)
if rawMAC == "(incomplete)" {
continue
}
guard let mac = MACAddress(fromString: rawMAC) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "failed to parse MAC address \(rawMAC)")
}
let interface = try match.getCaptureGroup(name: "interface", for: line)
if bridgeOnly && !interface.starts(with: "bridge") {
continue
}
if macAddress == mac {
return ip
}
}
return nil
}
}
extension NSTextCheckingResult {
func getCaptureGroup(name: String, for string: String) throws -> String {
let nsRange = self.range(withName: name)
if nsRange.location == NSNotFound {
throw ARPCacheInternalError(explanation: "attempted to retrieve non-existent named capture group \(name)")
}
guard let range = Range.init(nsRange, in: string) else {
throw ARPCacheInternalError(explanation: "failed to convert NSRange to Range")
}
return String(string[range])
}
}
@@ -0,0 +1,32 @@
import Network
struct Lease {
var mac: MACAddress
var ip: IPv4Address
init?(fromRawLease: [String : String]) {
// Retrieve the required fields
guard let hwAddress = fromRawLease["hw_address"] else { return nil }
guard let ipAddress = fromRawLease["ip_address"] else { return nil }
// Parse MAC address
let hwAddressSplits = hwAddress.split(separator: ",")
if hwAddressSplits.count != 2 {
return nil
}
if let hwAddressProto = Int(hwAddressSplits[0]), hwAddressProto != ARPHRD_ETHER {
return nil
}
guard let mac = MACAddress(fromString: String(hwAddressSplits[1])) else {
return nil
}
// Parse IP address
guard let ip = IPv4Address(ipAddress) else {
return nil
}
self.ip = ip
self.mac = mac
}
}
@@ -0,0 +1,106 @@
import Foundation
import Network
enum LeasesError: Error {
case UnexpectedFormat(name: String = "unexpected DHCPD leases file format", message: String, line: Int)
case Truncated(name: String = "truncated DHCPD leases file")
var description: String {
switch self {
case .UnexpectedFormat(name: let name, message: let message, line: let line):
return "\(name) on line \(line): \(message)"
case .Truncated(name: let name):
return "\(name)"
}
}
}
class Leases {
private let leases: [MACAddress : Lease]
convenience init() throws {
try self.init(URL(fileURLWithPath: "/var/db/dhcpd_leases"))
}
convenience init(_ fromURL: URL) throws {
let fileContents = try String(contentsOf: fromURL, encoding: .utf8)
try self.init(fileContents)
}
init(_ fromString: String) throws {
var leases: [MACAddress : Lease] = Dictionary()
for lease in try Self.retrieveRawLeases(fromString).compactMap({ Lease(fromRawLease: $0) }) {
leases[lease.mac] = lease
}
self.leases = leases
}
/// Parse leases from the host cache similarly to the PLCache_read() function found in Apple's Open Source releases.
///
/// [1]: https://github.com/apple-opensource/bootp/blob/master/bootplib/NICache.c#L285-L391
private static func retrieveRawLeases(_ dhcpdLeasesContents: String) throws -> [[String : String]] {
var rawLeases: [[String : String]] = Array()
enum State {
case Nowhere
case Start
case Body
case End
}
var state = State.Nowhere
var currentRawLease: [String : String] = Dictionary()
for (lineNumber, line) in dhcpdLeasesContents.split(separator: "\n").enumerated().map({ ($0 + 1, $1) }) {
if line == "{" {
// Handle lease block start
if state != .Nowhere && state != .End {
throw LeasesError.UnexpectedFormat(message: "unexpected lease block start ({)", line: lineNumber)
}
state = .Start
} else if line == "}" {
// Handle lease block end
if state != .Body {
throw LeasesError.UnexpectedFormat(message: "unexpected lease block end (})", line: lineNumber)
}
rawLeases.append(currentRawLease)
currentRawLease = Dictionary()
state = .End
} else {
// Handle lease block contents
let lineWithoutTabs = String(line.drop { $0 == " " || $0 == "\t"})
if lineWithoutTabs.isEmpty {
continue
}
let splits = lineWithoutTabs.split(separator: "=", maxSplits: 1)
if splits.count != 2 {
throw LeasesError.UnexpectedFormat(message: "key-value pair with only a key", line: lineNumber)
}
let (key, value) = (String(splits[0]), String(splits[1]))
currentRawLease[key] = value
state = .Body
}
}
if state == .Start || state == .Body {
throw LeasesError.Truncated()
}
return rawLeases
}
func resolveMACAddress(macAddress: MACAddress) throws -> IPv4Address? {
leases[macAddress]?.ip
}
}
@@ -1,6 +1,6 @@
import Foundation
struct MACAddress: Equatable, CustomStringConvertible {
struct MACAddress: Equatable, Hashable, CustomStringConvertible {
var mac: [UInt8] = Array(repeating: 0, count: 6)
init?(fromString: String) {
@@ -16,6 +16,6 @@ struct MACAddress: Equatable, CustomStringConvertible {
}
var description: String {
return String(format: "%02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5])
String(format: "%02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5])
}
}
+21
View File
@@ -0,0 +1,21 @@
import Foundation
protocol Authentication {
func header() -> (String, String)
func isValid() -> Bool
}
struct BasicAuthentication: Authentication {
let user: String
let password: String
func header() -> (String, String) {
let creds = Data("\(user):\(password)".utf8).base64EncodedString()
return ("Authorization", "Basic \(creds)")
}
func isValid() -> Bool {
true
}
}
+27
View File
@@ -0,0 +1,27 @@
import Foundation
import CryptoKit
class Digest {
var hash: SHA256 = SHA256()
func update(_ data: Data) {
hash.update(data: data)
}
func finalize() -> String {
hash.finalize().hexdigest()
}
static func hash(_ data: Data) -> String {
SHA256.hash(data: data).hexdigest()
}
}
extension SHA256.Digest {
func hexdigest() -> String {
"sha256:" + self.map {
String(format: "%02x", $0)
}
.joined()
}
}
+72
View File
@@ -0,0 +1,72 @@
import Foundation
let ociManifestMediaType = "application/vnd.oci.image.manifest.v1+json"
let ociConfigMediaType = "application/vnd.oci.image.config.v1+json"
// Annotations
let uncompressedDiskSizeAnnotation = "org.cirruslabs.tart.uncompressed-disk-size"
struct OCIManifest: Codable, Equatable {
var schemaVersion: Int = 2
var mediaType: String = ociManifestMediaType
var config: OCIManifestConfig
var layers: [OCIManifestLayer] = Array()
var annotations: Dictionary<String, String>?
init(config: OCIManifestConfig, layers: [OCIManifestLayer], uncompressedDiskSize: UInt64? = nil) {
self.config = config
self.layers = layers
if let uncompressedDiskSize = uncompressedDiskSize {
annotations = [
uncompressedDiskSizeAnnotation: String(uncompressedDiskSize)
]
}
}
init(fromJSON: Data) throws {
self = try Config.jsonDecoder().decode(Self.self, from: fromJSON)
}
func toJSON() throws -> Data {
try Config.jsonEncoder().encode(self)
}
func digest() throws -> String {
try Digest.hash(toJSON())
}
func uncompressedDiskSize() -> UInt64? {
guard let value = annotations?[uncompressedDiskSizeAnnotation] else {
return nil
}
return UInt64(value)
}
}
struct OCIConfig: Codable {
var architecture: Architecture = .arm64
var os: OS = .darwin
func toJSON() throws -> Data {
try Config.jsonEncoder().encode(self)
}
}
struct OCIManifestConfig: Codable, Equatable {
var mediaType: String = ociConfigMediaType
var size: Int
var digest: String
}
struct OCIManifestLayer: Codable, Equatable {
var mediaType: String
var size: Int
var digest: String
}
struct Descriptor: Equatable {
var size: Int
var digest: String
}
+378
View File
@@ -0,0 +1,378 @@
import Foundation
import NIOCore
import NIOHTTP1
import AsyncHTTPClient
import Algorithms
import NIOPosix
enum RegistryError: Error {
case UnexpectedHTTPStatusCode(when: String, code: UInt, details: String = "")
case MissingLocationHeader
case AuthFailed(why: String, details: String = "")
case MalformedHeader(why: String)
}
extension HTTPClientResponse.Body {
func readTextResponse() async throws -> String? {
let data = try await readResponse()
return String(decoding: data, as: UTF8.self)
}
func readResponse() async throws -> Data {
var result = Data()
for try await part in self {
result.append(Data(buffer: part))
}
return result
}
}
struct TokenResponse: Decodable, Authentication {
let defaultIssuedAt = Date()
let defaultExpiresIn = 60
var token: String
var expiresIn: Int?
var issuedAt: Date?
static func parse(fromData: Data) throws -> Self {
let decoder = Config.jsonDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
let dateFormatter = ISO8601DateFormatter()
dateFormatter.formatOptions = [.withInternetDateTime]
dateFormatter.timeZone = TimeZone(secondsFromGMT: 0)
decoder.dateDecodingStrategy = .custom { decoder in
let container = try decoder.singleValueContainer()
let dateString = try container.decode(String.self)
return dateFormatter.date(from: dateString) ?? Date()
}
return try decoder.decode(TokenResponse.self, from: fromData)
}
var tokenExpiresAt: Date {
get {
// Tokens can expire and expire_in field is used to determine when:
//
// >The duration in seconds since the token was issued that it will remain valid.
// >When omitted, this defaults to 60 seconds. For compatibility with older clients,
// >a token should never be returned with less than 60 seconds to live.
//
// [1]: https://docs.docker.com/registry/spec/auth/token/#requesting-a-token
(issuedAt ?? defaultIssuedAt) + TimeInterval(expiresIn ?? defaultExpiresIn)
}
}
func header() -> (String, String) {
("Authorization", "Bearer \(token)")
}
func isValid() -> Bool {
Date() < tokenExpiresAt
}
}
class Registry {
private let httpClient = HTTPClient(
eventLoopGroupProvider: .shared(MultiThreadedEventLoopGroup(numberOfThreads: 1))
)
deinit {
try! httpClient.syncShutdown()
}
let baseURL: URL
let namespace: String
let credentialsProviders: [CredentialsProvider]
var currentAuthToken: Authentication? = nil
init(urlComponents: URLComponents,
namespace: String,
credentialsProviders: [CredentialsProvider] = [HelperProgramCredentialsProvider(), KeychainCredentialsProvider()]
) throws {
baseURL = urlComponents.url!
self.namespace = namespace
self.credentialsProviders = credentialsProviders
}
convenience init(
host: String,
namespace: String,
insecure: Bool = false,
credentialsProviders: [CredentialsProvider] = [HelperProgramCredentialsProvider(), KeychainCredentialsProvider()]
) throws {
let proto = insecure ? "http" : "https"
let baseURLComponents = URLComponents(string: proto + "://" + host + "/v2/")!
try self.init(urlComponents: baseURLComponents, namespace: namespace, credentialsProviders: credentialsProviders)
}
func ping() async throws {
let response = try await endpointRequest(.GET, "/v2/")
if response.status != .ok {
throw RegistryError.UnexpectedHTTPStatusCode(when: "doing ping", code: response.status.code)
}
}
func pushManifest(reference: String, manifest: OCIManifest) async throws -> String {
let manifestJSON = try manifest.toJSON()
let response = try await endpointRequest(.PUT, "\(namespace)/manifests/\(reference)",
headers: ["Content-Type": manifest.mediaType],
body: manifestJSON)
if response.status != .created {
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing manifest", code: response.status.code,
details: try await response.body.readTextResponse() ?? "")
}
return Digest.hash(manifestJSON)
}
public func pullManifest(reference: String) async throws -> (OCIManifest, Data) {
let response = try await endpointRequest(.GET, "\(namespace)/manifests/\(reference)",
headers: ["Accept": ociManifestMediaType])
if response.status != .ok {
let body = try await response.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling manifest", code: response.status.code,
details: body ?? "")
}
let manifestData = try await response.body.readResponse()
let manifest = try OCIManifest(fromJSON: manifestData)
return (manifest, manifestData)
}
private func uploadLocationFromResponse(_ response: HTTPClientResponse) throws -> URLComponents {
guard let uploadLocationRaw = response.headers.first(name: "Location") else {
throw RegistryError.MissingLocationHeader
}
guard let uploadLocation = URL(string: uploadLocationRaw) else {
throw RegistryError.MalformedHeader(why: "Location header contains invalid URL: \"\(uploadLocationRaw)\"")
}
return URLComponents(url: uploadLocation.absolutize(baseURL), resolvingAgainstBaseURL: true)!
}
public func pushBlob(fromData: Data, chunkSizeMb: Int = 0) async throws -> String {
// Initiate a blob upload
let postResponse = try await endpointRequest(.POST, "\(namespace)/blobs/uploads/",
headers: ["Content-Length": "0"])
if postResponse.status != .accepted {
let body = try await postResponse.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing blob (POST)", code: postResponse.status.code,
details: body ?? "")
}
// Figure out where to upload the blob
var uploadLocation = try uploadLocationFromResponse(postResponse)
let digest = Digest.hash(fromData)
if chunkSizeMb == 0 {
// monolithic upload
let response = try await rawRequest(
.PUT,
uploadLocation,
headers: [
"Content-Type": "application/octet-stream",
],
parameters: ["digest": digest],
body: fromData
)
if response.status != .created {
let body = try await response.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing blob (PUT) to \(uploadLocation)",
code: response.status.code, details: body ?? "")
}
return digest
}
// chunked upload
var uploadedBytes = 0
let chunks = fromData.chunks(ofCount: chunkSizeMb == 0 ? fromData.count : chunkSizeMb * 1_000_000)
for (index, chunk) in chunks.enumerated() {
let lastChunk = index == (chunks.count - 1)
let response = try await rawRequest(
lastChunk ? .PUT : .PATCH,
uploadLocation,
headers: [
"Content-Type": "application/octet-stream",
"Content-Range": "\(uploadedBytes)-\(uploadedBytes + chunk.count - 1)",
],
parameters: lastChunk ? ["digest": digest] : [:],
body: chunk
)
let expectedStatus: HTTPResponseStatus = lastChunk ? .created : .accepted
if response.status != expectedStatus {
let body = try await response.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "streaming blob to \(uploadLocation)",
code: response.status.code, details: body ?? "")
}
uploadedBytes += chunk.count
// Update location for the next chunk
uploadLocation = try uploadLocationFromResponse(response)
}
return digest
}
public func pullBlob(_ digest: String, handler: (ByteBuffer) throws -> Void) async throws {
let response = try await endpointRequest(.GET, "\(namespace)/blobs/\(digest)")
if response.status != .ok {
let body = try await response.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling blob", code: response.status.code,
details: body ?? "")
}
for try await part in response.body {
try Task.checkCancellation()
try handler(part)
}
}
private func endpointRequest(
_ method: HTTPMethod,
_ endpoint: String,
headers: Dictionary<String, String> = Dictionary(),
parameters: Dictionary<String, String> = Dictionary(),
body: Data? = nil
) async throws -> HTTPClientResponse {
let url = URL(string: endpoint, relativeTo: baseURL)!
let urlComponents = URLComponents(url: url, resolvingAgainstBaseURL: true)!
return try await rawRequest(method, urlComponents, headers: headers, parameters: parameters, body: body)
}
private func rawRequest(
_ method: HTTPMethod,
_ urlComponents: URLComponents,
headers: Dictionary<String, String> = Dictionary(),
parameters: Dictionary<String, String> = Dictionary(),
body: Data? = nil,
doAuth: Bool = true
) async throws -> HTTPClientResponse {
var urlComponents = urlComponents
if urlComponents.queryItems == nil && !parameters.isEmpty {
urlComponents.queryItems = []
}
urlComponents.queryItems?.append(contentsOf: parameters.map { key, value -> URLQueryItem in
URLQueryItem(name: key, value: value)
})
var request = HTTPClientRequest(url: urlComponents.string!)
request.method = method
for (key, value) in headers {
request.headers.add(name: key, value: value)
}
if body != nil {
request.headers.add(name: "Content-Length", value: "\(body!.count)")
request.body = HTTPClientRequest.Body.bytes(body!)
}
// Invalidate token if it has expired
if currentAuthToken?.isValid() == false {
currentAuthToken = nil
}
var response = try await authAwareRequest(request: request)
if doAuth && response.status == .unauthorized {
try await auth(response: response)
response = try await authAwareRequest(request: request)
}
return response
}
private func auth(response: HTTPClientResponse) async throws {
// Process WWW-Authenticate header
guard let wwwAuthenticateRaw = response.headers.first(name: "WWW-Authenticate") else {
throw RegistryError.AuthFailed(why: "got HTTP 401, but WWW-Authenticate header is missing")
}
let wwwAuthenticate = try WWWAuthenticate(rawHeaderValue: wwwAuthenticateRaw)
if wwwAuthenticate.scheme == "Basic" {
if let (user, password) = try lookupCredentials(host: baseURL.host!) {
currentAuthToken = BasicAuthentication(user: user, password: password)
}
return
}
if wwwAuthenticate.scheme != "Bearer" {
throw RegistryError.AuthFailed(why: "WWW-Authenticate header's authentication scheme "
+ "\"\(wwwAuthenticate.scheme)\" is unsupported, expected \"Bearer\" scheme")
}
guard let realm = wwwAuthenticate.kvs["realm"] else {
throw RegistryError.AuthFailed(why: "WWW-Authenticate header is missing a \"realm\" directive")
}
// Request a token
guard var authenticateURL = URLComponents(string: realm) else {
throw RegistryError.AuthFailed(why: "WWW-Authenticate header's realm directive "
+ "\"\(realm)\" doesn't look like URL")
}
// Token Authentication Specification[1]:
//
// >To respond to this challenge, the client will need to make a GET request
// >[...] using the service and scope values from the WWW-Authenticate header.
//
// [1]: https://docs.docker.com/registry/spec/auth/token/
authenticateURL.queryItems = ["scope", "service"].compactMap { key in
if let value = wwwAuthenticate.kvs[key] {
return URLQueryItem(name: key, value: value)
} else {
return nil
}
}
var headers: Dictionary<String, String> = Dictionary()
if let (user, password) = try lookupCredentials(host: baseURL.host!) {
let encodedCredentials = "\(user):\(password)".data(using: .utf8)?.base64EncodedString()
headers["Authorization"] = "Basic \(encodedCredentials!)"
}
let response = try await rawRequest(.GET, authenticateURL, headers: headers, doAuth: false)
if response.status != .ok {
let body = try await response.body.readTextResponse() ?? ""
throw RegistryError.AuthFailed(why: "received unexpected HTTP status code \(response.status.code) "
+ "while retrieving an authentication token", details: body)
}
let bodyData = try await response.body.readResponse()
currentAuthToken = try TokenResponse.parse(fromData: bodyData)
}
private func lookupCredentials(host: String) throws -> (String, String)? {
for provider in credentialsProviders {
if let (user, password) = try provider.retrieve(host: host) {
return (user, password)
}
}
return nil
}
private func authAwareRequest(request: HTTPClientRequest) async throws -> HTTPClientResponse {
var request = request
if let token = currentAuthToken {
let (name, value) = token.header()
request.headers.add(name: name, value: value)
}
return try await httpClient.execute(request, deadline: .distantFuture)
}
}
+128
View File
@@ -0,0 +1,128 @@
import Foundation
import Parsing
struct Reference: Comparable, Hashable, CustomStringConvertible {
enum ReferenceType: Comparable {
case Tag
case Digest
}
let type: ReferenceType
let value: String
var fullyQualified: String {
get {
switch type {
case .Tag:
return ":" + value
case .Digest:
return "@" + value
}
}
}
init(tag: String) {
type = .Tag
value = tag
}
init(digest: String) {
type = .Digest
value = digest
}
static func <(lhs: Reference, rhs: Reference) -> Bool {
if lhs.type != rhs.type {
return lhs.type < rhs.type
} else {
return lhs.value < rhs.value
}
}
var description: String {
get {
fullyQualified
}
}
}
struct RemoteName: Comparable, Hashable, CustomStringConvertible {
var host: String
var namespace: String
var reference: Reference
init(host: String, namespace: String, reference: Reference) {
self.host = host
self.namespace = namespace
self.reference = reference
}
init(_ name: String) throws {
let csNormal = [
UInt8(ascii: "a")...UInt8(ascii: "z"),
UInt8(ascii: "A")...UInt8(ascii: "Z"),
UInt8(ascii: "0")...UInt8(ascii: "9"),
].asCharacterSet().union(CharacterSet(charactersIn: "_-."))
let csHex = [
UInt8(ascii: "a")...UInt8(ascii: "f"),
UInt8(ascii: "0")...UInt8(ascii: "9"),
].asCharacterSet()
let parser = Parse {
Consumed {
csNormal
Optionally {
":"
Digits()
}
}
"/"
csNormal.union(CharacterSet(charactersIn: "/"))
Optionally {
OneOf {
Parse {
":"
csNormal.map {
Reference(tag: String($0))
}
}
Parse {
"@sha256:"
csHex.map {
Reference(digest: "sha256:" + String($0))
}
}
}
}
End()
}
let result = try parser.parse(name)
host = String(result.0)
namespace = String(result.1)
reference = result.2 ?? Reference(tag: "latest")
}
static func <(lhs: RemoteName, rhs: RemoteName) -> Bool {
if lhs.host != rhs.host {
return lhs.host < rhs.host
} else if lhs.namespace != rhs.namespace {
return lhs.namespace < rhs.namespace
} else {
return lhs.reference < rhs.reference
}
}
var description: String {
"\(host)/\(namespace)\(reference.fullyQualified)"
}
}
extension Array where Self.Element == ClosedRange<UInt8> {
func asCharacterSet() -> CharacterSet {
let characters = self.joined().map { String(UnicodeScalar($0)) }.joined()
return CharacterSet(charactersIn: characters)
}
}
+7
View File
@@ -0,0 +1,7 @@
import Foundation
extension URL {
func absolutize(_ baseURL: URL) -> Self {
URL(string: absoluteString, relativeTo: baseURL)!
}
}
+63
View File
@@ -0,0 +1,63 @@
import Foundation
// WWW-Authenticate header parser based on details from RFCs[1][2]
///
// [1]: https://www.rfc-editor.org/rfc/rfc2617#section-3.2.1
// [2]: https://www.rfc-editor.org/rfc/rfc6750#section-3
class WWWAuthenticate {
var scheme: String
var kvs: Dictionary<String, String> = Dictionary()
init(rawHeaderValue: String) throws {
let splits = rawHeaderValue.split(separator: " ", maxSplits: 1)
if splits.count == 2 {
scheme = String(splits[0])
} else {
throw RegistryError.MalformedHeader(why: "WWW-Authenticate header should consist of two parts: "
+ "scheme and directives")
}
let rawDirectives = contextAwareCommaSplit(rawDirectives: String(splits[1]))
try rawDirectives.forEach { sequence in
let parts = sequence.split(separator: "=", maxSplits: 1)
if parts.count != 2 {
throw RegistryError.MalformedHeader(why: "Each WWW-Authenticate header directive should be in the form of "
+ "key=value or key=\"value\"")
}
let key = String(parts[0])
var value = String(parts[1])
value = value.trimmingCharacters(in: CharacterSet(charactersIn: "\""))
kvs[key] = value
}
}
private func contextAwareCommaSplit(rawDirectives: String) -> Array<String> {
var result: Array<String> = Array()
var inQuotation: Bool = false
var accumulator: Array<Character> = Array()
for ch in rawDirectives {
if ch == "," && !inQuotation {
result.append(String(accumulator))
accumulator.removeAll()
continue
}
accumulator.append(ch)
if ch == "\"" {
inQuotation.toggle()
}
}
if !accumulator.isEmpty {
result.append(String(accumulator))
}
return result
}
}
@@ -0,0 +1,13 @@
import Foundation
struct PassphraseGenerator: Sequence {
func makeIterator() -> PassphraseIterator {
PassphraseIterator()
}
}
struct PassphraseIterator: IteratorProtocol {
mutating func next() -> String? {
passphrases[Int(arc4random_uniform(UInt32(passphrases.count)))]
}
}
File diff suppressed because it is too large Load Diff
+14
View File
@@ -0,0 +1,14 @@
import Foundation
enum Architecture: String, Codable {
case arm64
case amd64
}
func CurrentArchitecture() -> Architecture {
#if arch(arm64)
return .arm64
#elseif arch(x86_64)
return .amd64
#endif
}
+97
View File
@@ -0,0 +1,97 @@
import Virtualization
struct Darwin: Platform {
var ecid: VZMacMachineIdentifier
var hardwareModel: VZMacHardwareModel
init(ecid: VZMacMachineIdentifier, hardwareModel: VZMacHardwareModel) {
self.ecid = ecid
self.hardwareModel = hardwareModel
}
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
let encodedECID = try container.decode(String.self, forKey: .ecid)
guard let data = Data.init(base64Encoded: encodedECID) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize Data using the provided value")
}
guard let ecid = VZMacMachineIdentifier.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize VZMacMachineIdentifier using the provided value")
}
self.ecid = ecid
let encodedHardwareModel = try container.decode(String.self, forKey: .hardwareModel)
guard let data = Data.init(base64Encoded: encodedHardwareModel) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
guard let hardwareModel = VZMacHardwareModel.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
self.hardwareModel = hardwareModel
}
func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: CodingKeys.self)
try container.encode(ecid.dataRepresentation.base64EncodedString(), forKey: .ecid)
try container.encode(hardwareModel.dataRepresentation.base64EncodedString(), forKey: .hardwareModel)
}
func os() -> OS {
.darwin
}
func bootLoader(nvramURL: URL) throws -> VZBootLoader {
VZMacOSBootLoader()
}
func platform(nvramURL: URL) -> VZPlatformConfiguration {
let result = VZMacPlatformConfiguration()
result.machineIdentifier = ecid
result.auxiliaryStorage = VZMacAuxiliaryStorage(contentsOf: nvramURL)
result.hardwareModel = hardwareModel
return result
}
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration {
let result = VZMacGraphicsDeviceConfiguration()
if let hostMainScreen = NSScreen.main {
let vmScreenSize = NSSize(width: vmConfig.display.width, height: vmConfig.display.height)
result.displays = [
VZMacGraphicsDisplayConfiguration(for: hostMainScreen, sizeInPoints: vmScreenSize)
]
return result
}
result.displays = [
VZMacGraphicsDisplayConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height,
// A reasonable guess according to Apple's documentation[1]
// [1]: https://developer.apple.com/documentation/coregraphics/1456599-cgdisplayscreensize
pixelsPerInch: 72
)
]
return result
}
func pointingDevices() -> [VZPointingDeviceConfiguration] {
if #available(macOS 13, *) {
// Trackpad is only supported starting with macOS Ventura
// macOS Monterey will continue using a USB device == .darwin
return [VZMacTrackpadConfiguration(), VZUSBScreenCoordinatePointingDeviceConfiguration()]
} else {
return [VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
}
}
+37
View File
@@ -0,0 +1,37 @@
import Virtualization
@available(macOS 13, *)
struct Linux: Platform {
func os() -> OS {
.linux
}
func bootLoader(nvramURL: URL) throws -> VZBootLoader {
let result = VZEFIBootLoader()
result.variableStore = VZEFIVariableStore(url: nvramURL)
return result
}
func platform(nvramURL: URL) -> VZPlatformConfiguration {
VZGenericPlatformConfiguration()
}
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration {
let result = VZVirtioGraphicsDeviceConfiguration()
result.scanouts = [
VZVirtioGraphicsScanoutConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height
)
]
return result
}
func pointingDevices() -> [VZPointingDeviceConfiguration] {
[VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
}
+6
View File
@@ -0,0 +1,6 @@
import Virtualization
enum OS: String, Codable {
case darwin
case linux
}
+9
View File
@@ -0,0 +1,9 @@
import Virtualization
protocol Platform: Codable {
func os() -> OS
func bootLoader(nvramURL: URL) throws -> VZBootLoader
func platform(nvramURL: URL) -> VZPlatformConfiguration
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration
func pointingDevices() -> [VZPointingDeviceConfiguration]
}
+11
View File
@@ -0,0 +1,11 @@
import Foundation
protocol PrunableStorage {
func prunables() throws -> [Prunable]
}
protocol Prunable {
func delete() throws
func accessDate() throws -> Date
func sizeBytes() throws -> Int
}
-3
View File
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8dd6af1a08bbcdc4faf0ff53601b38136c90231e11bd81bc8cd477d6f1c7d3f2
size 209404
+41 -1
View File
@@ -1,8 +1,48 @@
import ArgumentParser
import Foundation
@main
struct Root: AsyncParsableCommand {
static var configuration = CommandConfiguration(
commandName: "tart",
subcommands: [Create.self, Clone.self, Run.self, Set.self, List.self, IP.self, Delete.self])
version: CI.version,
subcommands: [
Create.self,
Clone.self,
Run.self,
Set.self,
List.self,
Login.self,
IP.self,
Pull.self,
Push.self,
Prune.self,
Delete.self,
])
public static func main() async throws {
// Ensure the default SIGINT handled is disabled,
// otherwise there's a race between two handlers
signal(SIGINT, SIG_IGN);
// Handle cancellation by Ctrl+C ourselves
let task = withUnsafeCurrentTask { $0 }!
let sigintSrc = DispatchSource.makeSignalSource(signal: SIGINT)
sigintSrc.setEventHandler {
task.cancel()
}
sigintSrc.activate()
// Parse and run command
do {
var command = try parseAsRoot()
if var asyncCommand = command as? AsyncParsableCommand {
try await asyncCommand.run()
} else {
try command.run()
}
} catch {
exit(withError: error)
}
}
}
+60
View File
@@ -0,0 +1,60 @@
import Foundation
enum SoftnetError: Error {
case InitializationFailed(why: String)
}
class Softnet {
private let process = Process()
let vmFD: Int32
init(vmMACAddress: String) throws {
let binaryName = "softnet"
guard let executableURL = resolveBinaryPath(binaryName) else {
throw SoftnetError.InitializationFailed(why: "\(binaryName) not found in PATH")
}
let fds = UnsafeMutablePointer<Int32>.allocate(capacity: MemoryLayout<Int>.stride * 2)
let ret = socketpair(AF_UNIX, SOCK_DGRAM, 0, fds)
if ret != 0 {
throw SoftnetError.InitializationFailed(why: "socketpair() failed with exit code \(ret)")
}
vmFD = fds[0]
let softnetFD = fds[1]
try setSocketBuffers(vmFD, 1 * 1024 * 1024);
try setSocketBuffers(softnetFD, 1 * 1024 * 1024);
process.executableURL = executableURL
process.arguments = ["--vm-fd", String(STDIN_FILENO), "--vm-mac-address", vmMACAddress]
process.standardInput = FileHandle(fileDescriptor: softnetFD, closeOnDealloc: false)
}
func run() throws {
try process.run()
}
func stop() throws {
process.interrupt()
process.waitUntilExit()
}
private func setSocketBuffers(_ fd: Int32, _ sizeBytes: Int) throws {
var option_value = sizeBytes
let option_len = socklen_t(MemoryLayout<Int>.size)
var ret = setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &option_value, option_len)
if ret != 0 {
throw SoftnetError.InitializationFailed(why: "setsockopt(SO_RCVBUF) returned \(ret)")
}
ret = setsockopt(fd, SOL_SOCKET, SO_SNDBUF, &option_value, option_len)
if ret != 0 {
throw SoftnetError.InitializationFailed(why: "setsockopt(SO_SNDBUF) returned \(ret)")
}
}
}
+25
View File
@@ -0,0 +1,25 @@
import Foundation
extension URL {
func accessDate() throws -> Date {
let attrs = try resourceValues(forKeys: [.contentAccessDateKey])
return attrs.contentAccessDate!
}
func updateAccessDate(_ accessDate: Date = Date()) throws {
let attrs = try resourceValues(forKeys: [.contentAccessDateKey])
let modificationDate = attrs.contentAccessDate!
let times = [accessDate.asTimeval(), modificationDate.asTimeval()]
let ret = utimes(path, times)
if ret != 0 {
throw RuntimeError("utimes(2) failed: \(ret.explanation())")
}
}
}
extension Date {
func asTimeval() -> timeval {
timeval(tv_sec: Int(timeIntervalSince1970), tv_usec: 0)
}
}
+11
View File
@@ -0,0 +1,11 @@
import Foundation
extension URL: Prunable {
func delete() throws {
try FileManager.default.removeItem(at: self)
}
func sizeBytes() throws -> Int {
try resourceValues(forKeys: [.totalFileAllocatedSizeKey]).totalFileAllocatedSize!
}
}
+17
View File
@@ -5,3 +5,20 @@ extension Collection {
indices.contains(index) ? self[index] : nil
}
}
func resolveBinaryPath(_ name: String) -> URL? {
guard let path = ProcessInfo.processInfo.environment["PATH"] else {
return nil
}
for pathComponent in path.split(separator: ":") {
let url = URL(fileURLWithPath: String(pathComponent))
.appendingPathComponent(name, isDirectory: false)
if FileManager.default.fileExists(atPath: url.path) {
return url
}
}
return nil
}
+38
View File
@@ -0,0 +1,38 @@
import Foundation
import Virtualization
import Dynamic
// Kudos to @saagarjha's VirtualApple for finding about _VZVirtualMachineStartOptions
extension VZVirtualMachine {
@available(macOS 12, *)
func start(_ recovery: Bool) async throws {
if !recovery {
// just use the regular API
return try await withCheckedThrowingContinuation { continuation in
DispatchQueue.main.async {
self.start(completionHandler: { result in
continuation.resume(with: result)
})
}
}
}
// use some private stuff only for recovery
return try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<Void, Error>) in
DispatchQueue.main.async {
let handler: @convention(block) (_ result: Any?) -> Void = { result in
if let error = result as? Error {
continuation.resume(throwing: error)
} else {
continuation.resume(returning: ())
}
}
// dynamic magic
let options = Dynamic._VZVirtualMachineStartOptions()
options.bootMacOSRecovery = recovery
Dynamic(self)._start(withOptions: options, completionHandler: handler)
}
}
}
}
+132 -45
View File
@@ -10,6 +10,13 @@ struct NoMainScreenFoundError: Error {
struct DownloadFailed: Error {
}
struct UnsupportedOSError: Error, CustomStringConvertible {
private(set) var description: String = "error: Linux VMs are only supported on macOS 13.0 (Ventura) or newer"
}
struct UnsupportedArchitectureError: Error {
}
class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Virtualization.Framework's virtual machine
@Published var virtualMachine: VZVirtualMachine
@@ -23,17 +30,30 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// VM's config
var config: VMConfig
init(vmDir: VMDirectory) throws {
let auxStorage = VZMacAuxiliaryStorage(contentsOf: vmDir.nvramURL)
var softnet: Softnet? = nil
init(vmDir: VMDirectory,
withSoftnet: Bool = false,
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment] = []
) throws {
name = vmDir.name
config = try VMConfig.init(fromURL: vmDir.configURL)
let configuration = try VM.craftConfiguration(diskURL: vmDir.diskURL, auxStorage: auxStorage, vmConfig: config)
if config.arch != CurrentArchitecture() {
throw UnsupportedArchitectureError()
}
// Initialize the virtual machine and its configuration
if withSoftnet {
softnet = try Softnet(vmMACAddress: config.macAddress.string)
}
let configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL,
nvramURL: vmDir.nvramURL, vmConfig: config,
softnet: softnet, additionalDiskAttachments: additionalDiskAttachments)
virtualMachine = VZVirtualMachine(configuration: configuration)
super.init()
virtualMachine.delegate = self
}
@@ -45,14 +65,11 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
}
let ipswCacheFolder = VMStorage.tartCacheDir.appendingPathComponent("IPSWs", isDirectory: true)
try FileManager.default.createDirectory(at: ipswCacheFolder, withIntermediateDirectories: true)
let expectedIPSWLocation = ipswCacheFolder.appendingPathComponent("\(image.buildVersion).ipsw", isDirectory: false)
let expectedIPSWLocation = try IPSWCache().locationFor(image: image)
if FileManager.default.fileExists(atPath: expectedIPSWLocation.path) {
defaultLogger.appendNewLine("Using cached *.ipsw file...")
try expectedIPSWLocation.updateAccessDate()
return expectedIPSWLocation
}
@@ -77,8 +94,23 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
try data.write(to: expectedIPSWLocation, options: [.atomic])
return expectedIPSWLocation
}
var inFinalState: Bool {
get {
virtualMachine.state == VZVirtualMachine.State.stopped ||
virtualMachine.state == VZVirtualMachine.State.paused ||
virtualMachine.state == VZVirtualMachine.State.error
}
}
init(vmDir: VMDirectory, ipswURL: URL?, diskSizeGB: UInt8) async throws {
init(
vmDir: VMDirectory,
ipswURL: URL?,
diskSizeGB: UInt16,
withSoftnet: Bool = false,
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment] = []
) async throws {
let ipswURL = ipswURL != nil ? ipswURL! : try await VM.retrieveLatestIPSW();
// Load the restore image and try to get the requirements
@@ -94,7 +126,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
// Create NVRAM
let auxStorage = try VZMacAuxiliaryStorage(creatingStorageAt: vmDir.nvramURL, hardwareModel: requirements.hardwareModel)
_ = try VZMacAuxiliaryStorage(creatingStorageAt: vmDir.nvramURL, hardwareModel: requirements.hardwareModel)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
@@ -102,18 +134,25 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
name = vmDir.name
// Create config
config = VMConfig(
hardwareModel: requirements.hardwareModel,
platform: Darwin(ecid: VZMacMachineIdentifier(), hardwareModel: requirements.hardwareModel),
cpuCountMin: requirements.minimumSupportedCPUCount,
memorySizeMin: requirements.minimumSupportedMemorySize
)
// allocate at least 4 CPUs because otherwise VMs are frequently freezing
try config.setCPU(cpuCount: max(4, requirements.minimumSupportedCPUCount))
try config.save(toURL: vmDir.configURL)
// Initialize the virtual machine and its configuration
let configuration = try VM.craftConfiguration(diskURL: vmDir.diskURL, auxStorage: auxStorage, vmConfig: config)
if withSoftnet {
softnet = try Softnet(vmMACAddress: config.macAddress.string)
}
let configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL, nvramURL: vmDir.nvramURL,
vmConfig: config, softnet: softnet,
additionalDiskAttachments: additionalDiskAttachments)
virtualMachine = VZVirtualMachine(configuration: configuration)
super.init()
virtualMachine.delegate = self
// Run automated installation
@@ -131,62 +170,110 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
}
func run() async throws {
try await withCheckedThrowingContinuation { continuation in
DispatchQueue.main.async {
self.virtualMachine.start(completionHandler: { result in
continuation.resume(with: result)
})
@available(macOS 13, *)
static func linux(vmDir: VMDirectory, diskSizeGB: UInt16) async throws -> VM {
// Create NVRAM
_ = try VZEFIVariableStore(creatingVariableStoreAt: vmDir.nvramURL)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
// Create config
let config = VMConfig(platform: Linux(), cpuCountMin: 4, memorySizeMin: 4096 * 1024 * 1024)
try config.save(toURL: vmDir.configURL)
return try VM(vmDir: vmDir)
}
func run(_ recovery: Bool) async throws {
if let softnet = softnet {
try softnet.run()
}
DispatchQueue.main.sync {
Task {
if #available(macOS 13, *) {
// new API introduced in Ventura
let startOptions = VZMacOSVirtualMachineStartOptions()
startOptions.startUpFromMacOSRecovery = recovery
try await virtualMachine.start(options: startOptions)
} else {
// use method that also available on Monterey
try await virtualMachine.start(recovery)
}
}
}
sema.wait()
await withTaskCancellationHandler(operation: {
sema.wait()
}, onCancel: {
sema.signal()
})
if Task.isCancelled {
DispatchQueue.main.sync {
Task {
try await self.virtualMachine.stop()
}
}
}
if let softnet = softnet {
try softnet.stop();
}
}
static func craftConfiguration(diskURL: URL, auxStorage: VZMacAuxiliaryStorage, vmConfig: VMConfig) throws -> VZVirtualMachineConfiguration {
static func craftConfiguration(
diskURL: URL,
nvramURL: URL,
vmConfig: VMConfig,
softnet: Softnet? = nil,
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment]
) throws -> VZVirtualMachineConfiguration {
let configuration = VZVirtualMachineConfiguration()
// Boot loader
configuration.bootLoader = VZMacOSBootLoader()
configuration.bootLoader = try vmConfig.platform.bootLoader(nvramURL: nvramURL)
// CPU and memory
configuration.cpuCount = vmConfig.cpuCount
configuration.memorySize = vmConfig.memorySize
// Platform
let platform = VZMacPlatformConfiguration()
platform.machineIdentifier = vmConfig.ecid
platform.auxiliaryStorage = auxStorage
platform.hardwareModel = vmConfig.hardwareModel
configuration.platform = platform
configuration.platform = vmConfig.platform.platform(nvramURL: nvramURL)
// Display
let graphicsDeviceConfiguration = VZMacGraphicsDeviceConfiguration()
graphicsDeviceConfiguration.displays = [
VZMacGraphicsDisplayConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height,
pixelsPerInch: vmConfig.display.dpi
)
]
configuration.graphicsDevices = [graphicsDeviceConfiguration]
configuration.graphicsDevices = [vmConfig.platform.graphicsDevice(vmConfig: vmConfig)]
// Audio
let soundDeviceConfiguration = VZVirtioSoundDeviceConfiguration()
let inputAudioStreamConfiguration = VZVirtioSoundDeviceInputStreamConfiguration()
inputAudioStreamConfiguration.source = VZHostAudioInputStreamSource()
let outputAudioStreamConfiguration = VZVirtioSoundDeviceOutputStreamConfiguration()
outputAudioStreamConfiguration.sink = VZHostAudioOutputStreamSink()
soundDeviceConfiguration.streams = [inputAudioStreamConfiguration, outputAudioStreamConfiguration]
configuration.audioDevices = [soundDeviceConfiguration]
// Keyboard and mouse
configuration.keyboards = [VZUSBKeyboardConfiguration()]
configuration.pointingDevices = [VZUSBScreenCoordinatePointingDeviceConfiguration()]
configuration.pointingDevices = vmConfig.platform.pointingDevices()
// Networking
let vio = VZVirtioNetworkDeviceConfiguration()
vio.attachment = VZNATNetworkDeviceAttachment()
if let softnet = softnet {
let fh = FileHandle.init(fileDescriptor: softnet.vmFD)
vio.attachment = VZFileHandleNetworkDeviceAttachment(fileHandle: fh)
} else {
vio.attachment = VZNATNetworkDeviceAttachment()
}
vio.macAddress = vmConfig.macAddress
configuration.networkDevices = [vio]
// Storage
let attachment = try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false)
let storage = VZVirtioBlockDeviceConfiguration(attachment: attachment)
configuration.storageDevices = [storage]
var attachments = [try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false)]
attachments.append(contentsOf: additionalDiskAttachments)
configuration.storageDevices = attachments.map { VZVirtioBlockDeviceConfiguration(attachment: $0) }
// Entropy
configuration.entropyDevices = [VZVirtioEntropyDeviceConfiguration()]
@@ -207,7 +294,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
func virtualMachine(_ virtualMachine: VZVirtualMachine, networkDevice: VZNetworkDevice, attachmentWasDisconnectedWithError error: Error) {
print("virtual machine's network attachment has been disconnected")
print("virtual machine's network attachment \(networkDevice) has been disconnected with error: \(error)")
sema.signal()
}
}
+39 -39
View File
@@ -16,43 +16,46 @@ class LessThanMinimalResourcesError: NSObject, LocalizedError {
enum CodingKeys: String, CodingKey {
case version
case ecid
case hardwareModel
case os
case arch
case cpuCountMin
case cpuCount
case memorySizeMin
case memorySize
case macAddress
case display
// macOS-specific keys
case ecid
case hardwareModel
}
struct VMDisplayConfig: Codable {
var width: Int = 1024
var height: Int = 768
var dpi: Int = 72
}
struct VMConfig: Codable {
var version: Int = 1
var ecid: VZMacMachineIdentifier
var hardwareModel: VZMacHardwareModel
var os: OS
var arch: Architecture
var platform: Platform
var cpuCountMin: Int
private(set) var cpuCount: Int
var memorySizeMin: UInt64
private(set) var memorySize: UInt64
var macAddress: VZMACAddress
var display: VMDisplayConfig = VMDisplayConfig()
init(
ecid: VZMacMachineIdentifier = VZMacMachineIdentifier(),
hardwareModel: VZMacHardwareModel,
cpuCountMin: Int,
memorySizeMin: UInt64,
macAddress: VZMACAddress = VZMACAddress.randomLocallyAdministered()
platform: Platform,
cpuCountMin: Int,
memorySizeMin: UInt64,
macAddress: VZMACAddress = VZMACAddress.randomLocallyAdministered()
) {
self.ecid = ecid
self.hardwareModel = hardwareModel
self.os = platform.os()
self.arch = CurrentArchitecture()
self.platform = platform
self.macAddress = macAddress
self.cpuCountMin = cpuCountMin
self.memorySizeMin = memorySizeMin
@@ -60,9 +63,16 @@ struct VMConfig: Codable {
memorySize = memorySizeMin
}
init(fromJSON: Data) throws {
self = try Config.jsonDecoder().decode(Self.self, from: fromJSON)
}
init(fromURL: URL) throws {
let jsonConfigData = try FileHandle.init(forReadingFrom: fromURL).readToEnd()!
self = try JSONDecoder().decode(VMConfig.self, from: jsonConfigData)
self = try Self(fromJSON: try Data(contentsOf: fromURL))
}
func toJSON() throws -> Data {
try Config.jsonEncoder().encode(self)
}
func save(toURL: URL) throws {
@@ -75,29 +85,18 @@ struct VMConfig: Codable {
let container = try decoder.container(keyedBy: CodingKeys.self)
version = try container.decode(Int.self, forKey: .version)
let encodedECID = try container.decode(String.self, forKey: .ecid)
guard let data = Data.init(base64Encoded: encodedECID) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize Data using the provided value")
os = try container.decodeIfPresent(OS.self, forKey: .os) ?? .darwin
arch = try container.decodeIfPresent(Architecture.self, forKey: .arch) ?? .arm64
switch os {
case .darwin:
platform = try Darwin(from: decoder)
case .linux:
if #available(macOS 13, *) {
platform = try Linux(from: decoder)
} else {
throw UnsupportedOSError()
}
}
guard let ecid = VZMacMachineIdentifier.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize VZMacMachineIdentifier using the provided value")
}
self.ecid = ecid
let encodedHardwareModel = try container.decode(String.self, forKey: .hardwareModel)
guard let data = Data.init(base64Encoded: encodedHardwareModel) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
guard let hardwareModel = VZMacHardwareModel.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
self.hardwareModel = hardwareModel
cpuCountMin = try container.decode(Int.self, forKey: .cpuCountMin)
cpuCount = try container.decode(Int.self, forKey: .cpuCount)
memorySizeMin = try container.decode(UInt64.self, forKey: .memorySizeMin)
@@ -119,8 +118,9 @@ struct VMConfig: Codable {
var container = encoder.container(keyedBy: CodingKeys.self)
try container.encode(version, forKey: .version)
try container.encode(ecid.dataRepresentation.base64EncodedString(), forKey: .ecid)
try container.encode(hardwareModel.dataRepresentation.base64EncodedString(), forKey: .hardwareModel)
try container.encode(os, forKey: .os)
try container.encode(arch, forKey: .arch)
try platform.encode(to: encoder)
try container.encode(cpuCountMin, forKey: .cpuCountMin)
try container.encode(cpuCount, forKey: .cpuCount)
try container.encode(memorySizeMin, forKey: .memorySizeMin)
+169
View File
@@ -0,0 +1,169 @@
import Foundation
import Compression
enum OCIError: Error {
case ShouldBeExactlyOneLayer
case ShouldBeAtLeastOneLayer
case FailedToCreateVmFile
}
extension VMDirectory {
private static let bufferSizeBytes = 64 * 1024 * 1024
private static let layerLimitBytes = 500 * 1000 * 1000
private static let configMediaType = "application/vnd.cirruslabs.tart.config.v1"
private static let diskMediaType = "application/vnd.cirruslabs.tart.disk.v1"
private static let nvramMediaType = "application/vnd.cirruslabs.tart.nvram.v1"
func pullFromRegistry(registry: Registry, reference: String) async throws {
defaultLogger.appendNewLine("pulling manifest...")
let (manifest, _) = try await registry.pullManifest(reference: reference)
return try await pullFromRegistry(registry: registry, manifest: manifest)
}
func pullFromRegistry(registry: Registry, manifest: OCIManifest) async throws {
// Pull VM's config file layer and re-serialize it into a config file
let configLayers = manifest.layers.filter {
$0.mediaType == Self.configMediaType
}
if configLayers.count != 1 {
throw OCIError.ShouldBeExactlyOneLayer
}
if !FileManager.default.createFile(atPath: configURL.path, contents: nil) {
throw OCIError.FailedToCreateVmFile
}
let configFile = try FileHandle(forWritingTo: configURL)
try await registry.pullBlob(configLayers.first!.digest) { buffer in
configFile.write(Data(buffer: buffer))
}
try configFile.close()
// Pull VM's disk layers and decompress them sequentially into a disk file
let diskLayers = manifest.layers.filter {
$0.mediaType == Self.diskMediaType
}
if diskLayers.isEmpty {
throw OCIError.ShouldBeAtLeastOneLayer
}
if !FileManager.default.createFile(atPath: diskURL.path, contents: nil) {
throw OCIError.FailedToCreateVmFile
}
let disk = try FileHandle(forWritingTo: diskURL)
let filter = try OutputFilter(.decompress, using: .lz4, bufferCapacity: Self.bufferSizeBytes) { data in
if let data = data {
disk.write(data)
}
}
// Progress
let diskCompressedSize: Int64 = Int64(diskLayers.map {
$0.size
}
.reduce(0) {
$0 + $1
})
let prettyDiskSize = String(format: "%.1f", Double(diskCompressedSize) / 1_000_000_000.0)
defaultLogger.appendNewLine("pulling disk (\(prettyDiskSize) GB compressed)...")
let progress = Progress(totalUnitCount: diskCompressedSize)
ProgressObserver(progress).log(defaultLogger)
for diskLayer in diskLayers {
try await registry.pullBlob(diskLayer.digest) { buffer in
let data = Data(buffer: buffer)
try filter.write(data)
progress.completedUnitCount += Int64(data.count)
}
}
try filter.finalize()
try disk.close()
// Pull VM's NVRAM file layer and store it in an NVRAM file
defaultLogger.appendNewLine("pulling NVRAM...")
let nvramLayers = manifest.layers.filter {
$0.mediaType == Self.nvramMediaType
}
if nvramLayers.count != 1 {
throw OCIError.ShouldBeExactlyOneLayer
}
if !FileManager.default.createFile(atPath: nvramURL.path, contents: nil) {
throw OCIError.FailedToCreateVmFile
}
let nvram = try FileHandle(forWritingTo: nvramURL)
try await registry.pullBlob(nvramLayers.first!.digest) { buffer in
nvram.write(Data(buffer: buffer))
}
try nvram.close()
}
func pushToRegistry(registry: Registry, references: [String], chunkSizeMb: Int) async throws -> RemoteName {
var layers = Array<OCIManifestLayer>()
// Read VM's config and push it as blob
let config = try VMConfig(fromURL: configURL)
let configJSON = try JSONEncoder().encode(config)
defaultLogger.appendNewLine("pushing config...")
let configDigest = try await registry.pushBlob(fromData: configJSON, chunkSizeMb: chunkSizeMb)
layers.append(OCIManifestLayer(mediaType: Self.configMediaType, size: configJSON.count, digest: configDigest))
// Progress
let diskSize = try FileManager.default.attributesOfItem(atPath: diskURL.path)[.size] as! Int64
defaultLogger.appendNewLine("pushing disk... this will take a while...")
let progress = Progress(totalUnitCount: diskSize)
ProgressObserver(progress).log(defaultLogger)
// Read VM's compressed disk as chunks
// and sequentially upload them as blobs
let mappedDisk = try Data(contentsOf: diskURL, options: [.alwaysMapped])
let mappedDiskSize = mappedDisk.count
var mappedDiskReadOffset = 0
let compressingFilter = try InputFilter(.compress, using: .lz4, bufferCapacity: Self.bufferSizeBytes) { (length: Int) -> Data? in
let bytesRead = min(length, mappedDiskSize - mappedDiskReadOffset)
let data = mappedDisk.subdata(in: mappedDiskReadOffset ..< mappedDiskReadOffset + bytesRead)
mappedDiskReadOffset += bytesRead
progress.completedUnitCount = Int64(mappedDiskReadOffset)
return data
}
while let compressedLayerData = try compressingFilter.readData(ofLength: Self.layerLimitBytes) {
let layerDigest = try await registry.pushBlob(fromData: compressedLayerData, chunkSizeMb: chunkSizeMb)
layers.append(OCIManifestLayer(mediaType: Self.diskMediaType, size: compressedLayerData.count, digest: layerDigest))
}
// Read VM's NVRAM and push it as blob
defaultLogger.appendNewLine("pushing NVRAM...")
let nvram = try FileHandle(forReadingFrom: nvramURL).readToEnd()!
let nvramDigest = try await registry.pushBlob(fromData: nvram, chunkSizeMb: chunkSizeMb)
layers.append(OCIManifestLayer(mediaType: Self.nvramMediaType, size: nvram.count, digest: nvramDigest))
// Craft a stub OCI config for Docker Hub compatibility
let ociConfigJSON = try OCIConfig(architecture: config.arch, os: config.os).toJSON()
let ociConfigDigest = try await registry.pushBlob(fromData: ociConfigJSON, chunkSizeMb: chunkSizeMb)
let manifest = OCIManifest(
config: OCIManifestConfig(size: ociConfigJSON.count, digest: ociConfigDigest),
layers: layers,
uncompressedDiskSize: UInt64(mappedDiskReadOffset)
)
// Manifest
for reference in references {
defaultLogger.appendNewLine("pushing manifest for \(reference)...")
_ = try await registry.pushManifest(reference: reference, manifest: manifest)
}
let pushedReference = Reference(digest: try manifest.digest())
return RemoteName(host: registry.baseURL.host!, namespace: registry.namespace, reference: pushedReference)
}
}
extension Progress {
func percentage() -> String {
String(Int(100 * fractionCompleted)) + "%"
}
}
+58 -6
View File
@@ -1,4 +1,5 @@
import Foundation
import Virtualization
struct UninitializedVMDirectoryError: Error {
}
@@ -6,8 +7,7 @@ struct UninitializedVMDirectoryError: Error {
struct AlreadyInitializedVMDirectoryError: Error {
}
struct VMDirectory {
var name: String
struct VMDirectory: Prunable {
var baseURL: URL
var configURL: URL {
@@ -20,18 +20,37 @@ struct VMDirectory {
baseURL.appendingPathComponent("nvram.bin")
}
var explicitlyPulledMark: URL {
baseURL.appendingPathComponent(".explicitly-pulled")
}
var name: String {
baseURL.lastPathComponent
}
static func temporary() throws -> VMDirectory {
let tmpDir = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString)
try FileManager.default.createDirectory(at: tmpDir, withIntermediateDirectories: false)
return VMDirectory(baseURL: tmpDir)
}
var initialized: Bool {
FileManager.default.fileExists(atPath: configURL.path) &&
FileManager.default.fileExists(atPath: diskURL.path) &&
FileManager.default.fileExists(atPath: nvramURL.path)
}
func initialize() throws {
if initialized {
func initialize(overwrite: Bool = false) throws {
if !overwrite && initialized {
throw AlreadyInitializedVMDirectoryError()
}
try FileManager.default.createDirectory(at: baseURL, withIntermediateDirectories: true, attributes: nil)
try? FileManager.default.removeItem(at: configURL)
try? FileManager.default.removeItem(at: diskURL)
try? FileManager.default.removeItem(at: nvramURL)
}
func validate() throws {
@@ -39,8 +58,21 @@ struct VMDirectory {
throw UninitializedVMDirectoryError()
}
}
func resizeDisk(_ sizeGB: UInt8) throws {
func clone(to: VMDirectory, generateMAC: Bool) throws {
try FileManager.default.copyItem(at: configURL, to: to.configURL)
try FileManager.default.copyItem(at: nvramURL, to: to.nvramURL)
try FileManager.default.copyItem(at: diskURL, to: to.diskURL)
// Re-generate MAC address
var newVMConfig = try VMConfig(fromURL: to.configURL)
if generateMAC {
newVMConfig.macAddress = VZMACAddress.randomLocallyAdministered()
}
try newVMConfig.save(toURL: to.configURL)
}
func resizeDisk(_ sizeGB: UInt16) throws {
if !FileManager.default.fileExists(atPath: diskURL.path) {
FileManager.default.createFile(atPath: diskURL.path, contents: nil, attributes: nil)
}
@@ -49,4 +81,24 @@ struct VMDirectory {
try diskFileHandle.truncate(atOffset: UInt64(sizeGB) * 1000 * 1000 * 1000)
try diskFileHandle.close()
}
func delete() throws {
try FileManager.default.removeItem(at: baseURL)
}
func accessDate() throws -> Date {
try baseURL.accessDate()
}
func sizeBytes() throws -> Int {
try configURL.sizeBytes() + diskURL.sizeBytes() + nvramURL.sizeBytes()
}
func markExplicitlyPulled() {
FileManager.default.createFile(atPath: explicitlyPulledMark.path, contents: nil)
}
func isExplicitlyPulled() -> Bool {
FileManager.default.fileExists(atPath: explicitlyPulledMark.path)
}
}
-58
View File
@@ -1,58 +0,0 @@
import Foundation
struct VMStorage {
public static let tartHomeDir: URL = FileManager.default
.homeDirectoryForCurrentUser
.appendingPathComponent(".tart", isDirectory: true)
public static let tartVMsDir: URL = tartHomeDir.appendingPathComponent("vms", isDirectory: true)
public static let tartCacheDir: URL = tartHomeDir.appendingPathComponent("cache", isDirectory: true)
func create(_ name: String) throws -> VMDirectory {
let vmDir = VMDirectory(name: name, baseURL: vmURL(name))
try vmDir.initialize()
return vmDir
}
func read(_ name: String) throws -> VMDirectory {
let vmDir = VMDirectory(name: name, baseURL: vmURL(name))
try vmDir.validate()
return vmDir
}
func delete(_ name: String) throws {
try FileManager.default.removeItem(at: vmURL(name))
}
func list() throws -> [URL] {
do {
return try FileManager.default.contentsOfDirectory(
at: VMStorage.tartVMsDir,
includingPropertiesForKeys: [.isDirectoryKey],
options: .skipsSubdirectoryDescendants)
} catch {
if error.isFileNotFound() {
return []
}
throw error
}
}
private func vmURL(_ name: String) -> URL {
return URL.init(
fileURLWithPath: name,
isDirectory: true,
relativeTo: VMStorage.tartVMsDir)
}
}
extension Error {
func isFileNotFound() -> Bool {
return (self as NSError).code == NSFileReadNoSuchFileError
}
}
+53
View File
@@ -0,0 +1,53 @@
import Foundation
class VMStorageHelper {
static func open(_ name: String) throws -> VMDirectory {
try missingVMWrap(name) {
if let remoteName = try? RemoteName(name) {
return try VMStorageOCI().open(remoteName)
} else {
return try VMStorageLocal().open(name)
}
}
}
static func delete(_ name: String) throws {
try missingVMWrap(name) {
if let remoteName = try? RemoteName(name) {
try VMStorageOCI().delete(remoteName)
} else {
try VMStorageLocal().delete(name)
}
}
}
private static func missingVMWrap<R: Any>(_ name: String, closure: () throws -> R) throws -> R {
do {
return try closure()
} catch {
if error.isFileNotFound() {
throw RuntimeError("source VM \"\(name)\" not found, is it listed in \"tart list\"?")
}
throw error
}
}
}
extension Error {
func isFileNotFound() -> Bool {
(self as NSError).code == NSFileReadNoSuchFileError
}
}
class RuntimeError: Error, CustomStringConvertible {
let message: String
init(_ message: String) {
self.message = message
}
var description: String {
message
}
}
+61
View File
@@ -0,0 +1,61 @@
import Foundation
class VMStorageLocal {
let baseURL: URL = Config().tartHomeDir.appendingPathComponent("vms", isDirectory: true)
private func vmURL(_ name: String) -> URL {
baseURL.appendingPathComponent(name, isDirectory: true)
}
func exists(_ name: String) -> Bool {
VMDirectory(baseURL: vmURL(name)).initialized
}
func open(_ name: String) throws -> VMDirectory {
let vmDir = VMDirectory(baseURL: vmURL(name))
try vmDir.validate()
return vmDir
}
func create(_ name: String, overwrite: Bool = false) throws -> VMDirectory {
let vmDir = VMDirectory(baseURL: vmURL(name))
try vmDir.initialize(overwrite: overwrite)
return vmDir
}
func move(_ name: String, from: VMDirectory) throws {
_ = try FileManager.default.createDirectory(at: baseURL, withIntermediateDirectories: true)
_ = try FileManager.default.replaceItemAt(vmURL(name), withItemAt: from.baseURL)
}
func delete(_ name: String) throws {
try FileManager.default.removeItem(at: vmURL(name))
}
func list() throws -> [(String, VMDirectory)] {
do {
return try FileManager.default.contentsOfDirectory(
at: baseURL,
includingPropertiesForKeys: [.isDirectoryKey],
options: .skipsSubdirectoryDescendants).compactMap { url in
let vmDir = VMDirectory(baseURL: url)
if !vmDir.initialized {
return nil
}
return (vmDir.name, vmDir)
}
} catch {
if error.isFileNotFound() {
return []
}
throw error
}
}
}
+184
View File
@@ -0,0 +1,184 @@
import Foundation
class VMStorageOCI: PrunableStorage {
let baseURL = Config().tartCacheDir.appendingPathComponent("OCIs", isDirectory: true)
private func vmURL(_ name: RemoteName) -> URL {
baseURL.appendingRemoteName(name)
}
func exists(_ name: RemoteName) -> Bool {
VMDirectory(baseURL: vmURL(name)).initialized
}
func open(_ name: RemoteName) throws -> VMDirectory {
let vmDir = VMDirectory(baseURL: vmURL(name))
try vmDir.validate()
try vmDir.baseURL.updateAccessDate()
return vmDir
}
func create(_ name: RemoteName, overwrite: Bool = false) throws -> VMDirectory {
let vmDir = VMDirectory(baseURL: vmURL(name))
try vmDir.initialize(overwrite: overwrite)
return vmDir
}
func move(_ name: RemoteName, from: VMDirectory) throws{
let targetURL = vmURL(name)
// Pre-create intermediate directories (e.g. creates ~/.tart/cache/OCIs/github.com/org/repo/
// for github.com/org/repo:latest)
try FileManager.default.createDirectory(at: targetURL.deletingLastPathComponent(),
withIntermediateDirectories: true)
_ = try FileManager.default.replaceItemAt(targetURL, withItemAt: from.baseURL)
}
func delete(_ name: RemoteName) throws {
try FileManager.default.removeItem(at: vmURL(name))
try gc()
}
func gc() throws {
var refCounts = Dictionary<URL, UInt>()
guard let enumerator = FileManager.default.enumerator(at: baseURL,
includingPropertiesForKeys: [.isSymbolicLinkKey]) else {
return
}
for case let foundURL as URL in enumerator {
let isSymlink = try foundURL.resourceValues(forKeys: [.isSymbolicLinkKey]).isSymbolicLink!
// Perform garbage collection for tag-based images
// with broken outgoing references
if isSymlink && foundURL == foundURL.resolvingSymlinksInPath() {
try FileManager.default.removeItem(at: foundURL)
continue
}
let vmDir = VMDirectory(baseURL: foundURL.resolvingSymlinksInPath())
if !vmDir.initialized {
continue
}
refCounts[vmDir.baseURL] = (refCounts[vmDir.baseURL] ?? 0) + (isSymlink ? 1 : 0)
}
// Perform garbage collection for digest-based images
// with no incoming references
for (baseURL, incRefCount) in refCounts {
let vmDir = VMDirectory(baseURL: baseURL)
if !vmDir.isExplicitlyPulled() && incRefCount == 0 {
try FileManager.default.removeItem(at: baseURL)
}
}
}
func list() throws -> [(String, VMDirectory, Bool)] {
var result: [(String, VMDirectory, Bool)] = Array()
guard let enumerator = FileManager.default.enumerator(at: baseURL,
includingPropertiesForKeys: [.isSymbolicLinkKey], options: [.producesRelativePathURLs]) else {
return []
}
for case let foundURL as URL in enumerator {
let vmDir = VMDirectory(baseURL: foundURL)
if !vmDir.initialized {
continue
}
let parts = [foundURL.deletingLastPathComponent().relativePath, foundURL.lastPathComponent]
var name: String
let isSymlink = try foundURL.resourceValues(forKeys: [.isSymbolicLinkKey]).isSymbolicLink!
if isSymlink {
name = parts.joined(separator: ":")
} else {
name = parts.joined(separator: "@")
}
result.append((name, vmDir, isSymlink))
}
return result
}
func prunables() throws -> [Prunable] {
try list().filter { (_, _, isSymlink) in !isSymlink }.map { (_, vmDir, _) in vmDir }
}
func pull(_ name: RemoteName, registry: Registry) async throws {
defaultLogger.appendNewLine("pulling manifest...")
let (manifest, manifestData) = try await registry.pullManifest(reference: name.reference.value)
let digestName = RemoteName(host: name.host, namespace: name.namespace,
reference: Reference(digest: Digest.hash(manifestData)))
if !exists(digestName) {
let tmpVMDir = try VMDirectory.temporary()
// Try to reclaim some cache space if we know the VM size in advance
if let uncompressedDiskSize = manifest.uncompressedDiskSize() {
let requiredCapacityBytes = UInt64(uncompressedDiskSize + 128 * 1024 * 1024)
let attrs = try tmpVMDir.baseURL.resourceValues(forKeys: [.volumeAvailableCapacityForImportantUsageKey])
let availableCapacityBytes = UInt64(attrs.volumeAvailableCapacityForImportantUsage!)
if availableCapacityBytes < requiredCapacityBytes {
try Prune.pruneReclaim(reclaimBytes: requiredCapacityBytes - availableCapacityBytes)
}
}
try await withTaskCancellationHandler(operation: {
try await tmpVMDir.pullFromRegistry(registry: registry, manifest: manifest)
try move(digestName, from: tmpVMDir)
}, onCancel: {
try? FileManager.default.removeItem(at: tmpVMDir.baseURL)
})
} else {
defaultLogger.appendNewLine("\(digestName) image is already cached! creating a symlink...")
}
if name != digestName {
// Create new or overwrite the old symbolic link
try link(from: digestName, to: name)
} else {
// Ensure that images pulled by content digest
// are excluded from garbage collection
VMDirectory(baseURL: vmURL(name)).markExplicitlyPulled()
}
}
func link(from: RemoteName, to: RemoteName) throws {
if FileManager.default.fileExists(atPath: vmURL(to).path) {
try FileManager.default.removeItem(at: vmURL(to))
}
try FileManager.default.createSymbolicLink(at: vmURL(to), withDestinationURL: vmURL(from))
try gc()
}
}
extension URL {
func appendingRemoteName(_ name: RemoteName) -> URL {
var result: URL = self
for pathComponent in (name.host + "/" + name.namespace + "/" + name.reference.value).split(separator: "/") {
result = result.appendingPathComponent(String(pathComponent))
}
return result
}
}
+38
View File
@@ -0,0 +1,38 @@
import Foundation
import Dynamic
import Virtualization
class FullFledgedVNC: VNC {
let password: String
private let vnc: Dynamic
init(virtualMachine: VZVirtualMachine) {
password = Array(PassphraseGenerator().prefix(4)).joined(separator: "-")
let securityConfiguration = Dynamic._VZVNCAuthenticationSecurityConfiguration(password: password)
vnc = Dynamic._VZVNCServer(port: 0, queue: DispatchQueue.global(),
securityConfiguration: securityConfiguration)
vnc.virtualMachine = virtualMachine
vnc.start()
}
func waitForURL() async throws -> URL {
while true {
// Port is 0 shortly after start(),
// but will be initialized later
if let port = vnc.port.asUInt16, port != 0 {
return URL(string: "vnc://:\(password)@127.0.0.1:\(port)")!
}
// Wait 50 ms.
try await Task.sleep(nanoseconds: 50_000_000)
}
}
func stop() throws {
vnc.stop()
}
deinit {
try? stop()
}
}
+26
View File
@@ -0,0 +1,26 @@
import Foundation
import Dynamic
import Virtualization
class ScreenSharingVNC: VNC {
let vmConfig: VMConfig
init(vmConfig: VMConfig) {
self.vmConfig = vmConfig
}
func waitForURL() async throws -> URL {
let vmMACAddress = MACAddress(fromString: vmConfig.macAddress.string)!
let ip = try await IP.resolveIP(vmMACAddress, secondsToWait: 60)
if let ip = ip {
return URL(string: "vnc://\(ip)")!
}
throw IPNotFound()
}
func stop() throws {
// nothing to do
}
}
+6
View File
@@ -0,0 +1,6 @@
import Foundation
protocol VNC {
func waitForURL() async throws -> URL
func stop() throws
}
+24
View File
@@ -0,0 +1,24 @@
import XCTest
@testable import tart
final class DigestTests: XCTestCase {
func testEmptyData() throws {
let data = Data("".utf8)
let digest = Digest()
digest.update(data)
XCTAssertEqual(digest.finalize(), "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")
XCTAssertEqual(Digest.hash(data), "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")
}
func testNonEmptyData() throws {
let data = Data("The quick brown fox jumps over the lazy dog".utf8)
let digest = Digest()
digest.update(data)
XCTAssertEqual(digest.finalize(), "sha256:d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592")
XCTAssertEqual(Digest.hash(data), "sha256:d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592")
}
}
@@ -0,0 +1,35 @@
import XCTest
import Network
@testable import tart
final class MACAddressResolverTests: XCTestCase {
func testSingleEntry() throws {
let leases = try Leases("""
{
ip_address=1.2.3.4
hw_address=1,00:11:22:33:44:55
}
""")
XCTAssertEqual(IPv4Address("1.2.3.4"),
try leases.resolveMACAddress(macAddress: MACAddress(fromString: "00:11:22:33:44:55")!))
}
func testMultipleEntries() throws {
let leases = try Leases("""
{
ip_address=1.2.3.4
hw_address=1,00:11:22:33:44:55
}
{
ip_address=5.6.7.8
hw_address=1,AA:BB:CC:DD:EE:FF
}
""")
XCTAssertEqual(IPv4Address("1.2.3.4"),
try leases.resolveMACAddress(macAddress: MACAddress(fromString: "00:11:22:33:44:55")!))
XCTAssertEqual(IPv4Address("5.6.7.8"),
try leases.resolveMACAddress(macAddress: MACAddress(fromString: "AA:BB:CC:DD:EE:FF")!))
}
}
+89
View File
@@ -0,0 +1,89 @@
import XCTest
@testable import tart
final class RegistryTests: XCTestCase {
var registryRunner: RegistryRunner?
override func setUp() async throws {
try await super.setUp()
do {
registryRunner = try await RegistryRunner()
} catch {
try XCTSkipIf(ProcessInfo.processInfo.environment["CI"] == nil)
}
}
override func tearDown() async throws {
try await super.tearDown()
registryRunner = nil
}
var registry: Registry {
registryRunner!.registry
}
func testPushPullBlobSmall() async throws {
// Generate a simple blob
let pushedBlob = Data("The quick brown fox jumps over the lazy dog".utf8)
// Push it
let pushedBlobDigest = try await registry.pushBlob(fromData: pushedBlob)
XCTAssertEqual("sha256:d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592", pushedBlobDigest)
// Pull it
var pulledBlob = Data()
try await registry.pullBlob(pushedBlobDigest) { buffer in
pulledBlob.append(Data(buffer: buffer))
}
// Ensure that both blobs are identical
XCTAssertEqual(pushedBlob, pulledBlob)
}
func testPushPullBlobHugeInChunks() async throws {
// Generate a large enough blob
let fh = FileHandle(forReadingAtPath: "/dev/urandom")!
let largeBlobToPush = try fh.read(upToCount: 768 * 1024 * 1024)!
// Push it
let largeBlobDigest = try await registry.pushBlob(fromData: largeBlobToPush, chunkSizeMb: 10)
// Pull it
var pulledLargeBlob = Data()
try await registry.pullBlob(largeBlobDigest) { buffer in
pulledLargeBlob.append(Data(buffer: buffer))
}
// Ensure that both blobs are identical
XCTAssertEqual(largeBlobToPush, pulledLargeBlob)
}
func testPushPullManifest() async throws {
// Craft a basic config
let configData = try OCIConfig().toJSON()
let configDigest = try await registry.pushBlob(fromData: configData)
// Craft a basic layer
let layerData = Data("doesn't matter".utf8)
let layerDigest = try await registry.pushBlob(fromData: layerData)
// Craft a basic manifest and push it
let manifest = OCIManifest(
config: OCIManifestConfig(size: configData.count, digest: configDigest),
layers: [
OCIManifestLayer(mediaType: "application/octet-stream", size: layerData.count, digest: layerDigest)
]
)
let pushedManifestDigest = try await registry.pushManifest(reference: "latest", manifest: manifest)
// Ensure that the manifest pulled by tag matches with the one pushed above
let (pulledByTagManifest, _) = try await registry.pullManifest(reference: "latest")
XCTAssertEqual(manifest, pulledByTagManifest)
// Ensure that the manifest pulled by digest matches with the one pushed above
let (pulledByDigestManifest, _) = try await registry.pullManifest(reference: "\(pushedManifestDigest)")
XCTAssertEqual(manifest, pulledByDigestManifest)
}
}
+47
View File
@@ -0,0 +1,47 @@
import XCTest
@testable import tart
final class RemoteNameTests: XCTestCase {
func testTag() throws {
let expectedRemoteName = RemoteName(host: "ghcr.io", namespace: "a/b", reference: Reference(tag: "latest"))
XCTAssertEqual(expectedRemoteName, try RemoteName("ghcr.io/a/b:latest"))
}
func testComplexTag() throws {
let expectedRemoteName = RemoteName(host: "ghcr.io", namespace: "a/b", reference: Reference(tag: "1.2.3-RC-1"))
XCTAssertEqual(expectedRemoteName, try RemoteName("ghcr.io/a/b:1.2.3-RC-1"))
}
func testDigest() throws {
let expectedRemoteName = RemoteName(
host: "ghcr.io",
namespace: "a/b",
reference: Reference(digest: "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")
)
XCTAssertEqual(expectedRemoteName,
try RemoteName("ghcr.io/a/b@sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"))
}
func testASCIIOnly() throws {
// Only ASCII letters are supported
XCTAssertEqual(try? RemoteName("touché.fr/a/b:latest"), nil)
XCTAssertEqual(try? RemoteName("ghcr.io/tou/ché:latest"), nil)
XCTAssertEqual(try? RemoteName("ghcr.io/a/b:touché"), nil)
}
func testLocal() throws {
// Local image names (those that don't include a registry) are not supported
XCTAssertEqual(try? RemoteName("debian:latest"), nil)
}
func testPort() throws {
// Port is included in host
XCTAssertEqual(try RemoteName("127.0.0.1:8080/a/b").host, "127.0.0.1:8080")
// Port must be specified when ":" is used
XCTAssertEqual(try? RemoteName("127.0.0.1:/a/b").host, nil)
}
}
+38
View File
@@ -0,0 +1,38 @@
import XCTest
@testable import tart
final class TokenResponseTests: XCTestCase {
func testBasic() throws {
let tokenResponseRaw = Data("{\"token\":\"some token\"}".utf8)
let tokenResponse = try TokenResponse.parse(fromData: tokenResponseRaw)
XCTAssertEqual(tokenResponse.token, "some token")
let expectedTokenExpiresAtRange = Date()...Date().addingTimeInterval(60)
XCTAssertTrue(expectedTokenExpiresAtRange.contains(tokenResponse.tokenExpiresAt))
XCTAssertTrue(tokenResponse.isValid())
}
func testExpirationBasic() throws {
let tokenResponseRaw = Data("{\"token\":\"some token\",\"expires_in\":2}".utf8)
let tokenResponse = try TokenResponse.parse(fromData: tokenResponseRaw)
XCTAssertEqual(tokenResponse.expiresIn, 2)
let expectedTokenExpiresAtRange = Date()...Date().addingTimeInterval(2)
XCTAssertTrue(expectedTokenExpiresAtRange.contains(tokenResponse.tokenExpiresAt))
XCTAssertTrue(tokenResponse.isValid())
_ = XCTWaiter.wait(for: [expectation(description: "Wait 3 seconds for the token to become invalid")], timeout: 2)
XCTAssertFalse(tokenResponse.isValid())
}
func testExpirationWithIssuedAt() throws {
let tokenResponseRaw = Data("{\"token\":\"some token\",\"expires_in\":3600,\"issued_at\":\"1970-01-01T00:00:00Z\"}".utf8)
let tokenResponse = try TokenResponse.parse(fromData: tokenResponseRaw)
XCTAssertEqual(Date(timeIntervalSince1970: 3600), tokenResponse.tokenExpiresAt)
XCTAssertFalse(tokenResponse.isValid())
}
}
@@ -0,0 +1,18 @@
import XCTest
@testable import tart
final class URLAbsolutizationTets: XCTestCase {
func testNeedsAbsolutization() throws {
let url = URL(string: "/v2/some/path?some=query")!
.absolutize(URL(string: "https://example.com/v2/")!)
XCTAssertEqual(url.absoluteString, "https://example.com/v2/some/path?some=query")
}
func testDoesntNeedAbsolutization() throws {
let url = URL(string: "https://example.org/v2/some/path?some=query")!
.absolutize(URL(string: "https://example.com/v2/")!)
XCTAssertEqual(url.absoluteString, "https://example.org/v2/some/path?some=query")
}
}
+22
View File
@@ -0,0 +1,22 @@
import XCTest
@testable import tart
final class URLAccessDateTests: XCTestCase {
func testGetAndSetAccessTime() throws {
// Create a temporary file
let tmpDir = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true)
var tmpFile = tmpDir.appendingPathComponent(UUID().uuidString)
FileManager.default.createFile(atPath: tmpFile.path, contents: nil)
// Ensure it's access date is different than our desired access date
let arbitraryDate = Date.init(year: 2008, month: 09, day: 28, hour: 23, minute: 15)
XCTAssertNotEqual(arbitraryDate, try tmpFile.accessDate())
// Set our desired access date for a file
try tmpFile.updateAccessDate(arbitraryDate)
// Ensure the access date has changed to our value
tmpFile.removeCachedResourceValue(forKey: .contentAccessDateKey)
XCTAssertEqual(arbitraryDate, try tmpFile.accessDate())
}
}
+54
View File
@@ -0,0 +1,54 @@
import Foundation
@testable import tart
enum RegistryRunnerError: Error {
case DockerFailed(exitCode: Int32)
}
class RegistryRunner {
let containerID: String
let registry: Registry
static func dockerCmd(_ arguments: String...) throws -> String {
let stdoutPipe = Pipe()
let proc = Process()
proc.executableURL = URL(fileURLWithPath: "/usr/local/bin/docker")
proc.arguments = arguments
proc.standardOutput = stdoutPipe
try proc.run()
let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
proc.waitUntilExit()
if proc.terminationStatus != 0 {
throw RegistryRunnerError.DockerFailed(exitCode: proc.terminationStatus)
}
return String(data: stdoutData, encoding: .utf8) ?? ""
}
init() async throws {
// Start container
let container = try Self.dockerCmd("run", "-d", "--rm", "-p", "5000", "registry:2")
.trimmingCharacters(in: CharacterSet.newlines)
containerID = container
// Get forwarded port
let port = try Self.dockerCmd("inspect", containerID, "--format", "{{(index (index .NetworkSettings.Ports \"5000/tcp\") 0).HostPort}}")
.trimmingCharacters(in: CharacterSet.newlines)
registry = try Registry(urlComponents: URLComponents(string: "http://127.0.0.1:\(port)/v2/")!,
namespace: "vm-image")
// Wait for the Docker Registry to start
while ((try? await registry.ping()) == nil) {
try await Task.sleep(nanoseconds: 100_000_000)
}
}
deinit {
_ = try! Self.dockerCmd("kill", containerID)
}
}
@@ -0,0 +1,41 @@
import XCTest
@testable import tart
final class WWWAuthenticateTests: XCTestCase {
func testExample() throws {
// Test example from Token Authentication Specification[1]
//
// [1]: https://docs.docker.com/registry/spec/auth/token/
let wwwAuthenticate = try WWWAuthenticate(rawHeaderValue: "Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:samalba/my-app:pull,push\"")
XCTAssertEqual("Bearer", wwwAuthenticate.scheme)
XCTAssertEqual([
"realm": "https://auth.docker.io/token",
"service": "registry.docker.io",
"scope": "repository:samalba/my-app:pull,push",
], wwwAuthenticate.kvs)
}
func testBasic() throws {
let wwwAuthenticate = try WWWAuthenticate(rawHeaderValue: "Bearer a=b,c=\"d\"")
XCTAssertEqual("Bearer", wwwAuthenticate.scheme)
XCTAssertEqual(["a": "b", "c": "d"], wwwAuthenticate.kvs)
}
func testIncompleteHeader() throws {
XCTAssertThrowsError(try WWWAuthenticate(rawHeaderValue: "Whatever")) {
XCTAssertTrue($0 is RegistryError)
}
XCTAssertThrowsError(try WWWAuthenticate(rawHeaderValue: "Bearer ")) {
XCTAssertTrue($0 is RegistryError)
}
}
func testIncompleteDirective() throws {
XCTAssertThrowsError(try WWWAuthenticate(rawHeaderValue: "Bearer whatever")) {
XCTAssertTrue($0 is RegistryError)
}
}
}