Compare commits

..
46 Commits
Author SHA1 Message Date
Nikolay Edigaryev 166e3e570f Ditch AsyncHTTPClient in favor of URLSession (#260) 2022-10-05 00:39:36 +04:00
Nikolay Edigaryev 7a2c20ba30 tart create: support fetching URLs specified in the --from-ipsw option (#256)
* tart create: support fetching URLs specified in the --from-ipsw option

* Use x-amz-meta-digest-sha256 header to cache IPSWs
2022-09-27 23:36:18 +04:00
Nikolay Edigaryev e90d53eceb tart delete: allow removing multiple VMs at once (#257) 2022-09-27 10:30:44 -04:00
Nikolay Edigaryev afbc2e0764 tart ip: keep waiting for the /var/db/dhcpd_leases file to appear (#254) 2022-09-22 10:24:14 -04:00
Fedor Korotkov 0229138bd5 Fixed wait delay for IP command (#250)
I initially wanted a one-second wait but made a human mistake.

Fixes #249
2022-09-19 21:44:14 +04:00
Nikolay EdigaryevandPete Goldsmith 4d08e6365e Set line-buffered output for stdout and introduce --graphics for tart run (#248)
* Set line-buffered output for stdout

* tart run: introduce --graphics

* Update Sources/tart/Commands/Run.swift

Co-authored-by: Pete Goldsmith <peter.n.goldsmith@gmail.com>

* Update Sources/tart/Commands/Run.swift

Co-authored-by: Pete Goldsmith <peter.n.goldsmith@gmail.com>

Co-authored-by: Pete Goldsmith <peter.n.goldsmith@gmail.com>
2022-09-19 21:43:37 +04:00
Nikolay Edigaryev 8cd68ea8ef Revert com.apple.vm.networking entitlement (#247) 2022-09-14 11:01:16 -04:00
Fedor Korotkov f9001304c8 Fixed packaging 2022-09-14 10:06:22 -04:00
Nikolay Edigaryev 4e20ea8f72 tart run: introduce --net-bridged (#245)
* tart run: introduce --net-bridged

* tart.entitlements: add com.apple.vm.networking
2022-09-14 17:53:04 +04:00
Nikolay EdigaryevandPete Goldsmith 678ce0a55a Introduce "tart rename" command to rename VMs (#246)
* Introduce "tart rename" command to rename VMs

* Remove unused SystemConfiguration import

* Update Sources/tart/Commands/Rename.swift

Co-authored-by: Pete Goldsmith <peter.n.goldsmith@gmail.com>

Co-authored-by: Pete Goldsmith <peter.n.goldsmith@gmail.com>
2022-09-14 17:44:26 +04:00
Fedor Korotkov 8273ae66e1 Update Developer Certificates (#242)
* Update Developer Certificates

Now the signature will state `Cirrus Labs, Inc.` and not `Fedor Korotkov`.

* Updated identity
2022-09-13 09:36:35 -04:00
Fedor Korotkov 4a9316a377 [skip ci] Add sponsorship option (#241) 2022-09-13 16:30:26 +04:00
Fedor Korotkov 6321d547cf Improve available capacity checking (#240)
* Improve available capacity checking

* Fixed expression
2022-09-12 21:55:25 +04:00
Nikolay Edigaryev f241e21614 Self-hosted temporary directory (#238) 2022-09-12 20:56:52 +04:00
Nikolay Edigaryev 0eca923604 Document NAT subnet change procedure (#236) 2022-09-08 16:26:40 +04:00
Pete Goldsmith 87f29cc11f Clarify requirements for dir argument (#231) 2022-09-07 09:02:46 -04:00
Pete Goldsmith 90d1393137 Handle tilde in path for directory share (#233)
* Expand Tilde in path

* Expand tilde in disk paths
2022-09-07 08:15:29 -04:00
Nikolay Edigaryev 625d431d10 Revert "Improve RemoteName parser (#225)" (#230)
This reverts commit ae7018c31f.
2022-09-06 22:13:30 +04:00
Nikolay Edigaryev 4648e1aea1 tart clone: clone VM and generate MAC under a file lock (#215)
* tart clone: clone VM and generate MAC under a file lock

* Lock concurrent "tart pull"'s for the same host

* Config: ensure Tart's home and cache directories always exist
2022-09-06 21:33:51 +04:00
Fedor Korotkov 4b62b73015 Document Adopters (#229)
* Document early adopters

Related to #208

* Use HTML
2022-09-06 21:25:22 +04:00
Nikolay Edigaryev ae7018c31f Improve RemoteName parser (#225)
* Improve RemoteName parser

* Remove Parsing import

* Permit namespace components to contain separators, but no more than one

* Add testNoPathTraversal
2022-09-06 17:28:49 +04:00
Pete Goldsmith e54c89da0c Clarify if guest or host is unsupported (#228) 2022-09-06 09:23:41 -04:00
Nikolay Edigaryev 9a0ec3e6b0 Fix cache reclaimed bytes calculation (#223) 2022-09-01 16:12:48 -04:00
Fedor Korotkov 400f85a493 Check Tart Home for auto-pruning (#220)
Otherwise it's weird we check $TMP but prune $TART_HOME
2022-09-01 15:44:19 -04:00
Fedor Korotkov 0105280b5d Support plaintext auths from Docker config (#219) 2022-09-01 23:10:52 +04:00
Fedor Korotkov c063c5bdc2 Include version in installer (#218) 2022-09-01 19:42:50 +04:00
Fedor Korotkov 8a2b0151e0 Create Apple Installer in dist folder (#217)
Seems goreleaser reset git state before submitting and therefore removes Tart.pkg
2022-09-01 18:04:43 +04:00
Fedor Korotkov 2eea51d6ec Create Apple Installer (#216)
The installer will install tart binary to `/usr/local/bin/tart`

Fixes #153
2022-08-31 18:36:28 -04:00
Fedor Korotkov 1890909d28 Sign release binaries (#207)
* Sign release binaries

Should fix #184

* Use VMs for building

* Test release

* Fixed password

* Revert testing
2022-08-30 16:26:06 -04:00
Nikolay Edigaryev 0cad2e454c Directory sharing support (#211) 2022-08-30 10:26:16 +04:00
Nikolay Edigaryev 17dcf942c1 Terminate VM on GUI window close (#210) 2022-08-30 02:41:58 +04:00
Nikolay Edigaryev 6296df7c0c Credential helpers: "credHelpers" map is optional in Docker's config (#209) 2022-08-29 16:38:51 -04:00
Fedor Korotkov c54b140750 Support Docker Helpers (#205)
Fixes #167
2022-08-29 20:26:53 +04:00
Fedor Korotkov 048a5506df Support trackpad on macOS and clipboard sharing on Linux (#202)
Fixes #66
Relates to #14 since fixes on Linux
2022-08-27 20:05:45 +04:00
Nikolay Edigaryev 553b36349b README.md: clarify "Pulling a Remote Image" section (#196) 2022-08-25 15:43:30 +04:00
Fedor Korotkov d0bf286aa1 Document Linux VM image creation (#193) 2022-08-24 23:21:50 +04:00
Nikolay Edigaryev 195a4b3a72 Make UnsupportedOSError more user-friendly (#192)
* Make UnsupportedOSError more user-friendly

* tart prune: allow only specifying a --gc flag
2022-08-24 15:03:54 -04:00
Nikolay Edigaryev 59393df2e1 Linux support (#190)
* Linux support

* Support macOS 12

* Improve readability a bit

* Remove useless variable
2022-08-23 22:38:50 +04:00
Nikolay Edigaryev 732c8244a4 Run garbage collection after each symlink and expose --gc in tart prune (#191) 2022-08-23 17:31:02 +04:00
Nikolay Edigaryev 9e69c8c161 testGetAndSetAccessTime: fix flakiness (#187) 2022-08-18 14:42:54 -04:00
Nikolay Edigaryev e4ac2275b9 tart ip: show a warning when DHCP lease and ARP cache entries mismatch (#186) 2022-08-18 22:25:02 +04:00
Nikolay Edigaryev 1a1f19e169 Implement garbage collection when deleting cached OCI VMs (#185)
* Properly calculate remotely-retrieved manifest digests

* Implement garbage collection when deleting cached OCI VMs
2022-08-18 17:13:06 +04:00
Nikolay Edigaryev fea916dacc OCI blob compression: fix Data memory leak when using InputFilter (#183)
* OCI blob compression: fix Data memory leak when using InputFilter

* Rename mappedDiskOffset to mappedDiskReadOffset

* Update progress.completedUnitCount differently
2022-08-17 12:57:39 -04:00
Fedor Korotkov b1be9730c7 Build and release from Ventrura (#182)
Might be related to #180
2022-08-17 17:42:24 +04:00
Nikolay EdigaryevandNikolay Edigaryev 14059a1d6f tart {pull,clone}: add missing --insecure support (#181)
Co-authored-by: Nikolay Edigaryev <edi@microsun.local>
2022-08-17 01:31:19 +04:00
Nikolay Edigaryev 440681320a Introduce "tart prune" command (#164) 2022-08-12 16:43:04 +03:00
55 changed files with 1542 additions and 368 deletions
+11
View File
@@ -0,0 +1,11 @@
#!/bin/sh
set -e
export VERSION="${CIRRUS_TAG:-0}"
mkdir -p .ci/pkg/
cp .build/arm64-apple-macosx/debug/tart .ci/pkg/
pkgbuild --root .ci/pkg --version $VERSION --install-location /usr/local/bin/ --identifier com.github.cirruslabs.tart --sign "Developer ID Installer: Cirrus Labs, Inc. (9M2P8L4D89)" "./dist/Tart-$VERSION.pkg"
xcrun notarytool submit "./dist/Tart-$VERSION.pkg" --keychain-profile "notarytool" --wait
xcrun stapler staple "./dist/Tart-$VERSION.pkg"
+17 -3
View File
@@ -10,7 +10,7 @@ task:
name: Build
only_if: $CIRRUS_TAG == ''
macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode:14
image: ghcr.io/cirruslabs/macos-ventura-xcode:latest
build_script: swift build --product tart
sign_script: codesign --sign - --entitlements Resources/tart.entitlements --force .build/debug/tart
binary_artifacts:
@@ -20,12 +20,26 @@ task:
name: Release
only_if: $CIRRUS_TAG != ''
macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode:14
image: ghcr.io/cirruslabs/macos-ventura-xcode:latest
env:
MACOS_CERTIFICATE: ENCRYPTED[552b9d275d1c2bdbc1bff778b104a8f9a53cbd0d59344d4b7f6d0ca3c811a5cefb97bef9ba0ef31c219cb07bdacdd2c2]
AC_PASSWORD: ENCRYPTED[4a761023e7e06fe2eb350c8b6e8e7ca961af193cb9ba47605f25f1d353abc3142606f412e405be48fd897a78787ea8c2]
GITHUB_TOKEN: ENCRYPTED[!98ace8259c6024da912c14d5a3c5c6aac186890a8d4819fad78f3e0c41a4e0cd3a2537dd6e91493952fb056fa434be7c!]
GORELEASER_KEY: ENCRYPTED[!9b80b6ef684ceaf40edd4c7af93014ee156c8aba7e6e5795f41c482729887b5c31f36b651491d790f1f668670888d9fd!]
install_script: brew install go goreleaser/tap/goreleaser-pro
setup_script:
- cd $HOME
- echo $MACOS_CERTIFICATE | base64 --decode > certificate.p12
- security create-keychain -p password101 build.keychain
- security default-keychain -s build.keychain
- security unlock-keychain -p password101 build.keychain
- security import certificate.p12 -k build.keychain -P password101 -T /usr/bin/codesign -T /usr/bin/pkgbuild
- security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k password101 build.keychain
- xcrun notarytool store-credentials "notarytool" --apple-id "hello@cirruslabs.org" --team-id "9M2P8L4D89" --password $AC_PASSWORD
install_script:
- brew install go goreleaser/tap/goreleaser-pro
- brew install mitchellh/gon/gon
info_script:
- security find-identity -v
- xcodebuild -version
- swift -version
release_script: goreleaser
+1
View File
@@ -0,0 +1 @@
github: [cirruslabs]
+7 -2
View File
@@ -7,13 +7,16 @@ builds:
goarch:
- arm64
prebuilt:
path: .build/{{ .Arch }}-apple-macosx/debug/tart
path: .build/arm64-apple-macosx/debug/tart
hooks:
post:
- gon gon.hcl
- .ci/create-pkg.sh
before:
hooks:
- .ci/set-version.sh
- swift build -c debug --product tart
- codesign --sign - --entitlements Resources/tart.entitlements --force .build/arm64-apple-macosx/debug/tart
archives:
- id: binary
@@ -24,6 +27,8 @@ archives:
release:
prerelease: auto
extra_files:
- glob: ./dist/Tart-{{ .Tag }}.pkg
brews:
- name: tart
+17 -62
View File
@@ -1,14 +1,5 @@
{
"pins" : [
{
"identity" : "async-http-client",
"kind" : "remoteSourceControl",
"location" : "https://github.com/swift-server/async-http-client",
"state" : {
"revision" : "df87a860fdc41a595d5ca67f74cde9adbccc099a",
"version" : "1.11.4"
}
},
{
"identity" : "dynamic",
"kind" : "remoteSourceControl",
@@ -37,12 +28,12 @@
}
},
{
"identity" : "swift-atomics",
"identity" : "swift-async-algorithms",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-atomics.git",
"location" : "https://github.com/apple/swift-async-algorithms",
"state" : {
"revision" : "919eb1d83e02121cdb434c7bfc1f0c66ef17febe",
"version" : "1.0.2"
"branch" : "main",
"revision" : "f05e450f0b909c0e80670a47516c4b9700b9e5da"
}
},
{
@@ -55,57 +46,12 @@
}
},
{
"identity" : "swift-log",
"identity" : "swift-collections",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-log.git",
"location" : "https://github.com/apple/swift-collections.git",
"state" : {
"revision" : "5d66f7ba25daf4f94100e7022febf3c75e37a6c7",
"version" : "1.4.2"
}
},
{
"identity" : "swift-nio",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio.git",
"state" : {
"revision" : "124119f0bb12384cef35aa041d7c3a686108722d",
"version" : "2.40.0"
}
},
{
"identity" : "swift-nio-extras",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-extras.git",
"state" : {
"revision" : "8eea84ec6144167354387ef9244b0939f5852dc8",
"version" : "1.11.0"
}
},
{
"identity" : "swift-nio-http2",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-http2.git",
"state" : {
"revision" : "108ac15087ea9b79abb6f6742699cf31de0e8772",
"version" : "1.22.0"
}
},
{
"identity" : "swift-nio-ssl",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-ssl.git",
"state" : {
"revision" : "1750873bce84b4129b5303655cce2c3d35b9ed3a",
"version" : "2.19.0"
}
},
{
"identity" : "swift-nio-transport-services",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-nio-transport-services.git",
"state" : {
"revision" : "1a4692acb88156e3da1b0c6732a8a38b2a744166",
"version" : "1.12.0"
"revision" : "f504716c27d2e5d4144fa4794b12129301d17729",
"version" : "1.0.3"
}
},
{
@@ -126,6 +72,15 @@
"version" : "0.9.2"
}
},
{
"identity" : "swiftdate",
"kind" : "remoteSourceControl",
"location" : "https://github.com/malcommac/SwiftDate",
"state" : {
"revision" : "6190d0cefff3013e77ed567e6b074f324e5c5bf5",
"version" : "6.3.1"
}
},
{
"identity" : "xctest-dynamic-overlay",
"kind" : "remoteSourceControl",
+4 -2
View File
@@ -13,16 +13,18 @@ let package = Package(
.package(url: "https://github.com/apple/swift-argument-parser", from: "1.1.2"),
.package(url: "https://github.com/mhdhejazi/Dynamic", branch: "master"),
.package(url: "https://github.com/pointfreeco/swift-parsing", from: "0.9.2"),
.package(url: "https://github.com/swift-server/async-http-client", from: "1.11.4"),
.package(url: "https://github.com/apple/swift-algorithms", from: "1.0.0"),
.package(url: "https://github.com/apple/swift-async-algorithms", branch: "main"),
.package(url: "https://github.com/malcommac/SwiftDate", from: "6.3.1")
],
targets: [
.executableTarget(name: "tart", dependencies: [
.product(name: "Algorithms", package: "swift-algorithms"),
.product(name: "AsyncAlgorithms", package: "swift-async-algorithms"),
.product(name: "ArgumentParser", package: "swift-argument-parser"),
.product(name: "AsyncHTTPClient", package: "async-http-client"),
.product(name: "Dynamic", package: "Dynamic"),
.product(name: "Parsing", package: "swift-parsing"),
.product(name: "SwiftDate", package: "SwiftDate"),
]),
.testTarget(name: "TartTests", dependencies: ["tart"])
]
+74 -4
View File
@@ -1,6 +1,6 @@
![Tart – open source virtualization for your automation needs](Resources/TartSocial.png)
*Tart* is a virtualization toolset to build, run and manage virtual machines on Apple Silicon.
*Tart* is a virtualization toolset to build, run and manage macOS and Linux virtual machines on Apple Silicon.
Built by CI engineers for your automation needs. Here are some highlights of Tart:
* Tart uses Apple's own `Virtualization.Framework` for [near-native performance](https://browser.geekbench.com/v5/cpu/compare/14966395?baseline=14966339).
@@ -8,6 +8,22 @@ Built by CI engineers for your automation needs. Here are some highlights of Tar
* Use Tart Packer Plugin to automate VM creation.
* Built-in CI integration.
*Tart* is already adopted by several automation services:
<p align="center">
<a href="https://cirrus-ci.org/guide/macOS/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/CirrusCI.png" height="65"/>
</a>
<a href="https://codemagic.io/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/Codemagic.png" height="65"/>
</a>
<a href="https://testingbot.com/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/TestingBot.png" height="65"/>
</a>
</p>
## Usage
Try running a Tart VM on your Apple Silicon device running macOS Monterey or later (will download a 25 GB image):
```shell
@@ -84,6 +100,11 @@ For the example above, `tart` binary will be saved to `$PWD/artifacts/Build/bina
### Creating from scratch
Tart supports macOS and Linux virtual machines. All commands like `run` and `pull` work the same way regarding of the underlying OS a particular VM image has.
The only difference is how such VM images are created. Please check sections below for [macOS](#creating-a-macos-vm-image-from-scratch) and [Linux](#creating-a-linux-vm-image-from-scratch) instructions.
#### Creating a macOS VM image from scratch
Tart can create VMs from `*.ipsw` files. You can download a specific `*.ipsw` file [here](https://ipsw.me/) or you can
use `latest` instead of a path to `*.ipsw` to download the latest available version:
@@ -100,6 +121,27 @@ After the initial booting of the VM you'll need to manually go through the macOS
4. Disable Screen Saver.
5. Run `sudo visudo` in Terminal, find `%admin ALL=(ALL) ALL` add `admin ALL=(ALL) NOPASSWD: ALL` to allow sudo without a password.
#### Creating a Linux VM image from scratch
```bash
# Create a bare VM
tart create --linux ubuntu
# Install Ubuntu
tart run --disk focal-desktop-arm64.iso ubuntu
# Run VM
tart run ubuntu
```
After the initial setup please make sure your VM can be SSH-ed into by running the following commands inside your VM:
```shell
sudo apt update
sudo apt install -y openssh-server
sudo ufw allow ssh
```
### Configuring a VM
By default, a tart VM uses 2 CPUs and 4 GB of memory with a `1024x768` display. This can be changed with `tart set` command.
@@ -165,10 +207,20 @@ tart push my-local-vm-name acme.io/remoteorg/name:latest acme.io/remoteorg/name:
#### Pulling a Remote Image
You can either pull an image:
```shell
tart pull acme.io/remoteorg/name:latest my-local-vm-name
tart pull acme.io/remoteorg/name:latest
```
...or instantiate a VM from a remote image:
```shell
tart clone acme.io/remoteorg/name:latest my-local-vm-name
```
This invocation calls the `tart pull` implicitly (if the image is not being present) before doing the actual cloning.
## FAQ
<details>
@@ -186,8 +238,8 @@ tart pull acme.io/remoteorg/name:latest my-local-vm-name
<details>
<summary>Why Tart is free and open sourced?</summary>
Tart is a relatively small project, and it didn't feel right to try to monetize it.
Apple did all the heavy lifting with their `Virtualization.Framework`.
Apple did all the heavy lifting with their `Virtualization.Framework` and it just felt right to develop Tart in the open.
Please consider [becoming a sponsor](https://github.com/sponsors/cirruslabs) if you find Tart saving a substantial amount of money on licensing and engineering hours for your company.
</details>
<details>
@@ -216,3 +268,21 @@ tart pull acme.io/remoteorg/name:latest my-local-vm-name
Tart is limited by functionality of Apple's `Virtualization.Framework`. At the moment `Virtualization.Framework`
doesn't support nested virtualization.
</details>
<details>
<summary>Changing the default NAT subnet</summary>
To change the default network to `192.168.77.1`:
```
sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist Shared_Net_Address -string 192.168.77.1
```
Note that even through a network would normally be specified as `192.168.77.0`, the [vmnet framework](https://developer.apple.com/documentation/vmnet) seems to treat this as a starting address too and refuses to pick up such network-like values.
The default subnet mask `255.255.255.0` should suffice for most use-cases, however, you can also change it to `255.255.0.0`, for example:
```
sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist Shared_Net_Mask -string 255.255.0.0
```
</details>
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dbcf6f65f26c0e17b2fe55a9712fc783315cd64905a0febce758b31d7e4e923b
size 6787
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4239996f30145992936cfc8faa0232fa8904aedf4de61e30ef4c4fc86cad587f
size 14588
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:227192fcd215c9cc05b1ebcd2616bd1f8c95be184726f64d91fc4507a88c66e1
size 18393
+1 -1
View File
@@ -5,4 +5,4 @@
<key>com.apple.security.virtualization</key>
<true/>
</dict>
</plist>
</plist>
+15 -2
View File
@@ -11,6 +11,9 @@ struct Clone: AsyncParsableCommand {
@Argument(help: "new VM name")
var newName: String
@Flag(help: "connect to the OCI registry via insecure HTTP protocol")
var insecure: Bool = false
func validate() throws {
if newName.contains("/") {
throw ValidationError("<new-name> should be a local name")
@@ -24,17 +27,27 @@ struct Clone: AsyncParsableCommand {
if let remoteName = try? RemoteName(sourceName), !ociStorage.exists(remoteName) {
// Pull the VM in case it's OCI-based and doesn't exist locally yet
let registry = try Registry(host: remoteName.host, namespace: remoteName.namespace)
let registry = try Registry(host: remoteName.host, namespace: remoteName.namespace, insecure: insecure)
try await ociStorage.pull(remoteName, registry: registry)
}
let sourceVM = try VMStorageHelper.open(sourceName)
let generateMAC = try localStorage.hasVMsWithMACAddress(macAddress: sourceVM.macAddress())
let tmpVMDir = try VMDirectory.temporary()
// Lock the temporary VM directory to prevent it's garbage collection
let tmpVMDirLock = try FileLock(lockURL: tmpVMDir.baseURL)
try tmpVMDirLock.lock()
try await withTaskCancellationHandler(operation: {
let lock = try FileLock(lockURL: Config().tartHomeDir)
try lock.lock()
let generateMAC = try localStorage.hasVMsWithMACAddress(macAddress: sourceVM.macAddress())
try sourceVM.clone(to: tmpVMDir, generateMAC: generateMAC)
try localStorage.move(newName, from: tmpVMDir)
try lock.unlock()
}, onCancel: {
try? FileManager.default.removeItem(at: tmpVMDir.baseURL)
})
+31 -7
View File
@@ -9,26 +9,50 @@ struct Create: AsyncParsableCommand {
@Argument(help: "VM name")
var name: String
@Option(help: ArgumentHelp("Path to the IPSW file (or \"latest\") to fetch the latest appropriate IPSW", valueName: "path"))
@Option(help: ArgumentHelp("create a macOS VM using path to the IPSW file or URL (or \"latest\", to fetch the latest supported IPSW automatically)", valueName: "path"))
var fromIPSW: String?
@Flag(help: "create a Linux VM")
var linux: Bool = false
@Option(help: ArgumentHelp("Disk size in Gb"))
var diskSize: UInt16 = 50
func validate() throws {
if fromIPSW == nil {
throw ValidationError("Please specify a --from-ipsw option!")
if fromIPSW == nil && !linux {
throw ValidationError("Please specify either a --from-ipsw or --linux option!")
}
}
func run() async throws {
do {
let tmpVMDir = try VMDirectory.temporary()
// Lock the temporary VM directory to prevent it's garbage collection
let tmpVMDirLock = try FileLock(lockURL: tmpVMDir.baseURL)
try tmpVMDirLock.lock()
try await withTaskCancellationHandler(operation: {
if fromIPSW! == "latest" {
_ = try await VM(vmDir: tmpVMDir, ipswURL: nil, diskSizeGB: diskSize)
} else {
_ = try await VM(vmDir: tmpVMDir, ipswURL: URL(fileURLWithPath: fromIPSW!), diskSizeGB: diskSize)
if let fromIPSW = fromIPSW {
let ipswURL: URL
if fromIPSW == "latest" {
ipswURL = try await VM.latestIPSWURL()
} else if fromIPSW.starts(with: "http://") || fromIPSW.starts(with: "https://") {
ipswURL = URL(string: fromIPSW)!
} else {
ipswURL = URL(fileURLWithPath: fromIPSW)
}
_ = try await VM(vmDir: tmpVMDir, ipswURL: ipswURL, diskSizeGB: diskSize)
}
if linux {
if #available(macOS 13, *) {
_ = try await VM.linux(vmDir: tmpVMDir, diskSizeGB: diskSize)
} else {
throw UnsupportedOSError("Linux VMs", "are")
}
}
try VMStorageLocal().move(name, from: tmpVMDir)
+4 -2
View File
@@ -6,11 +6,13 @@ struct Delete: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Delete a VM")
@Argument(help: "VM name")
var name: String
var name: [String]
func run() async throws {
do {
try VMStorageHelper.delete(name)
for it in name {
try VMStorageHelper.delete(it)
}
Foundation.exit(0)
} catch {
+13 -6
View File
@@ -16,14 +16,21 @@ struct IP: AsyncParsableCommand {
do {
let vmDir = try VMStorageLocal().open(name)
let vmConfig = try VMConfig.init(fromURL: vmDir.configURL)
let vmMACAddress = MACAddress(fromString: vmConfig.macAddress.string)!
guard let ip = try await IP.resolveIP(vmConfig, secondsToWait: wait) else {
guard let ipViaDHCP = try await IP.resolveIP(vmMACAddress, secondsToWait: wait) else {
print("no IP address found, is your VM running?")
Foundation.exit(1)
}
print(ip)
if let ipViaARP = try ARPCache.ResolveMACAddress(macAddress: vmMACAddress), ipViaARP != ipViaDHCP {
fputs("WARNING: DHCP lease and ARP cache entries for MAC address \(vmMACAddress) differ: "
+ "got \(ipViaDHCP) and \(ipViaARP) respectively, consider reporting this case to"
+ " https://github.com/cirruslabs/tart/issues/172\n", stderr)
}
print(ipViaDHCP)
Foundation.exit(0)
} catch {
@@ -33,16 +40,16 @@ struct IP: AsyncParsableCommand {
}
}
static public func resolveIP(_ config: VMConfig, secondsToWait: UInt16) async throws -> IPv4Address? {
static public func resolveIP(_ vmMACAddress: MACAddress, secondsToWait: UInt16) async throws -> IPv4Address? {
let waitUntil = Calendar.current.date(byAdding: .second, value: Int(secondsToWait), to: Date.now)!
let vmMacAddress = MACAddress(fromString: config.macAddress.string)!
repeat {
if let ip = try Leases().resolveMACAddress(macAddress: vmMacAddress) {
if let leases = try Leases(), let ip = try leases.resolveMACAddress(macAddress: vmMACAddress) {
return ip
}
try await Task.sleep(nanoseconds: 1_000_000)
// wait a second
try await Task.sleep(nanoseconds: 1_000_000_000)
} while Date.now < waitUntil
return nil
+1 -1
View File
@@ -10,7 +10,7 @@ struct List: AsyncParsableCommand {
print("Source\tName")
displayTable("local", try VMStorageLocal().list())
displayTable("oci", try VMStorageOCI().list())
displayTable("oci", try VMStorageOCI().list().map { (name, vmDir, _) in (name, vmDir) })
Foundation.exit(0)
} catch {
+1 -1
View File
@@ -45,7 +45,7 @@ struct Login: AsyncParsableCommand {
do {
let registry = try Registry(host: host, namespace: "", insecure: insecure,
credentialsProvider: credentialsProvider)
credentialsProviders: [credentialsProvider])
try await registry.ping()
} catch {
print("invalid credentials: \(error)")
+108
View File
@@ -0,0 +1,108 @@
import ArgumentParser
import Dispatch
import SwiftUI
import SwiftDate
struct Prune: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Prune OCI and IPSW caches")
@Option(help: ArgumentHelp("Remove cache entries last accessed more than n days ago",
discussion: "For example, --older-than=7 will remove entries that weren't accessed by Tart in the last 7 days.",
valueName: "n"))
var olderThan: UInt?
@Option(help: ArgumentHelp("Remove least recently used cache entries that do not fit the specified cache size budget n, expressed in gigabytes",
discussion: "For example, --cache-budget=50 will effectively shrink all caches to a total size of 50 gigabytes.",
valueName: "n"))
var cacheBudget: UInt?
@Flag(help: .hidden)
var gc: Bool = false
func validate() throws {
if olderThan == nil && cacheBudget == nil && !gc {
throw ValidationError("at least one pruning criteria must be specified")
}
}
func run() async throws {
do {
if gc {
try VMStorageOCI().gc()
}
// Clean up cache entries based on last accessed date
if let olderThan = olderThan {
let olderThanInterval = Int(exactly: olderThan)!.days.timeInterval
let olderThanDate = Date().addingTimeInterval(olderThanInterval)
try Prune.pruneOlderThan(olderThanDate: olderThanDate)
}
// Clean up cache entries based on imposed cache size limit and entry's last accessed date
if let cacheBudget = cacheBudget {
try Prune.pruneCacheBudget(cacheBudgetBytes: UInt64(cacheBudget) * 1024 * 1024 * 1024)
}
Foundation.exit(0)
} catch {
print(error)
Foundation.exit(1)
}
}
static func pruneOlderThan(olderThanDate: Date) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages.flatMap { try $0.prunables() }
try prunables.filter { try $0.accessDate() <= olderThanDate }.forEach { try $0.delete() }
}
static func pruneCacheBudget(cacheBudgetBytes: UInt64) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages
.flatMap { try $0.prunables() }
.sorted { try $0.accessDate() < $1.accessDate() }
let cacheUsedBytes = try prunables.map { try $0.sizeBytes() }.reduce(0, +)
var cacheReclaimedBytes: Int = 0
var it = prunables.makeIterator()
while (cacheUsedBytes - cacheReclaimedBytes) > cacheBudgetBytes {
guard let prunable = it.next() else {
break
}
cacheReclaimedBytes -= try prunable.sizeBytes()
try prunable.delete()
}
}
static func pruneReclaim(reclaimBytes: UInt64) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages
.flatMap { try $0.prunables() }
.sorted { try $0.accessDate() < $1.accessDate() }
// Does it even make sense to start?
let cacheUsedBytes = try prunables.map { try $0.sizeBytes() }.reduce(0, +)
if cacheUsedBytes < reclaimBytes {
return
}
var cacheReclaimedBytes: Int = 0
var it = prunables.makeIterator()
while cacheReclaimedBytes <= reclaimBytes {
guard let prunable = it.next() else {
break
}
cacheReclaimedBytes += try prunable.sizeBytes()
try prunable.delete()
}
}
}
+4 -1
View File
@@ -8,6 +8,9 @@ struct Pull: AsyncParsableCommand {
@Argument(help: "remote VM name")
var remoteName: String
@Flag(help: "connect to the OCI registry via insecure HTTP protocol")
var insecure: Bool = false
func run() async throws {
do {
// Be more liberal when accepting local image as argument,
@@ -19,7 +22,7 @@ struct Pull: AsyncParsableCommand {
}
let remoteName = try RemoteName(remoteName)
let registry = try Registry(host: remoteName.host, namespace: remoteName.namespace)
let registry = try Registry(host: remoteName.host, namespace: remoteName.namespace, insecure: insecure)
defaultLogger.appendNewLine("pulling \(remoteName)...")
+40
View File
@@ -0,0 +1,40 @@
import ArgumentParser
import Foundation
struct Rename: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Rename a VM")
@Argument(help: "VM name")
var name: String
@Argument(help: "new VM name")
var newName: String
func validate() throws {
if newName.contains("/") {
throw ValidationError("<new-name> should be a local name")
}
}
func run() async throws {
do {
let localStorage = VMStorageLocal()
if !localStorage.exists(name) {
throw ValidationError("failed to rename a non-existent VM: \(name)")
}
if localStorage.exists(newName) {
throw ValidationError("failed to rename VM \(name), target VM \(name) already exists, delete it first!")
}
try localStorage.rename(name, newName)
Foundation.exit(0)
} catch {
print(error)
Foundation.exit(1)
}
}
}
+108 -6
View File
@@ -19,6 +19,9 @@ struct Run: AsyncParsableCommand {
discussion: "Useful for integrating Tart VMs into other tools.\nUse `tart ip` in order to get an IP for SSHing or VNCing into the VM."))
var noGraphics: Bool = false
@Flag(help: "Force open a UI window, even when VNC is enabled.")
var graphics: Bool = false
@Flag(help: "Boot into recovery mode")
var recovery: Bool = false
@@ -37,17 +40,42 @@ struct Run: AsyncParsableCommand {
@Flag var withSoftnet: Bool = false
@Option(help: ArgumentHelp("""
Additional disk attachments with an optional read-only specifier\n(e.g. --disk=\"disk.bin\" --disk=\"disk.bin:ro\")
Additional disk attachments with an optional read-only specifier\n(e.g. --disk=\"disk.bin\" --disk=\"ubuntu.iso:ro\")
""", discussion: """
Learn how to create a disk image using Disk Utility here:
https://support.apple.com/en-gb/guide/disk-utility/dskutl11888/mac
"""))
""", valueName: "path[:ro]"))
var disk: [String] = []
@Option(help: ArgumentHelp("""
Additional directory shares with an optional read-only specifier\n(e.g. --dir=\"build:~/src/build\" --dir=\"sources:~/src/sources:ro\")
""", discussion: """
Requires host to be macOS 13.0 (Ventura) or newer.
All shared directories are automatically mounted to "/Volumes/My Shared Files" directory on macOS,
while on Linux you have to do it manually: "mount -t virtiofs com.apple.virtio-fs.automount /mount/point".
For macOS guests, they must be running macOS 13.0 (Ventura) or newer.
""", valueName: "name:path[:ro]"))
var dir: [String] = []
@Option(help: ArgumentHelp("""
Use bridged networking instead of the default shared (NAT) networking \n(e.g. --net-bridged=en0 or --net-bridged=\"Wi-Fi\")
""", discussion: """
Specify "list" as an interface name (--net-bridged=list) to list the available bridged interfaces.
""", valueName: "interface name"))
var netBridged: String?
func validate() throws {
if vnc && vncExperimental {
throw ValidationError("--vnc and --vnc-experimental are mutually exclusive")
}
if withSoftnet && netBridged != nil {
throw ValidationError("--with-softnet and --net-bridged are mutually exclusive")
}
if graphics && noGraphics {
throw ValidationError("--graphics and --no-graphics are mutually exclusive")
}
}
@MainActor
@@ -55,8 +83,9 @@ struct Run: AsyncParsableCommand {
let vmDir = try VMStorageLocal().open(name)
vm = try VM(
vmDir: vmDir,
withSoftnet: withSoftnet,
additionalDiskAttachments: additionalDiskAttachments()
network: userSpecifiedNetwork(vmDir: vmDir) ?? NetworkShared(),
additionalDiskAttachments: additionalDiskAttachments(),
directoryShares: directoryShares()
)
let vncImpl: VNC? = try {
@@ -107,18 +136,61 @@ struct Run: AsyncParsableCommand {
}
sigintSrc.activate()
if noGraphics || vnc || vncExperimental {
let useVNCWithoutGraphics = (vnc || vncExperimental) && !graphics
if noGraphics || useVNCWithoutGraphics {
dispatchMain()
} else {
runUI()
}
}
func userSpecifiedNetwork(vmDir: VMDirectory) throws -> Network? {
if withSoftnet {
let config = try VMConfig.init(fromURL: vmDir.configURL)
return try Softnet(vmMACAddress: config.macAddress.string)
}
if let netBridged = netBridged {
let matchingInterfaces = VZBridgedNetworkInterface.networkInterfaces.filter { interface in
interface.identifier == netBridged || interface.localizedDisplayName == netBridged
}
if matchingInterfaces.isEmpty {
let available = bridgeInterfaces().joined(separator: ", ")
throw ValidationError("no bridge interfaces matched \"\(netBridged)\", "
+ "available interfaces: \(available)")
}
if matchingInterfaces.count > 1 {
throw ValidationError("more than one bridge interface matched \"\(netBridged)\", "
+ "consider refining the search criteria")
}
return NetworkBridged(interface: matchingInterfaces.first!)
}
return nil
}
func bridgeInterfaces() -> [String] {
VZBridgedNetworkInterface.networkInterfaces.map { interface in
var bridgeDescription = interface.identifier
if let localizedDisplayName = interface.localizedDisplayName {
bridgeDescription += " (or \"\(localizedDisplayName)\")"
}
return bridgeDescription
}
}
func additionalDiskAttachments() throws -> [VZDiskImageStorageDeviceAttachment] {
var result: [VZDiskImageStorageDeviceAttachment] = []
let readOnlySuffix = ":ro"
let expandedDiskPaths = disk.map { NSString(string:$0).expandingTildeInPath }
for rawDisk in disk {
for rawDisk in expandedDiskPaths {
if rawDisk.hasSuffix(readOnlySuffix) {
result.append(try VZDiskImageStorageDeviceAttachment(
url: URL(fileURLWithPath: String(rawDisk.prefix(rawDisk.count - readOnlySuffix.count))),
@@ -135,6 +207,34 @@ struct Run: AsyncParsableCommand {
return result
}
func directoryShares() throws -> [DirectoryShare] {
var result: [DirectoryShare] = []
for rawDir in dir {
let splits = rawDir.split(maxSplits: 2) { $0 == ":" }
if splits.count < 2 {
throw ValidationError("invalid --dir syntax: should at least include name and path, colon-separated")
}
var readOnly: Bool = false
if splits.count == 3 {
if splits[2] == "ro" {
readOnly = true
} else {
throw ValidationError("invalid --dir syntax: optional read-only specifier can only be \"ro\"")
}
}
let (name, path) = (String(splits[0]), String(splits[1]))
result.append(DirectoryShare(name: name, path: URL(fileURLWithPath: NSString(string: path).expandingTildeInPath), readOnly: readOnly))
}
return result
}
private func runUI() {
let nsApp = NSApplication.shared
nsApp.setActivationPolicy(.regular)
@@ -148,6 +248,8 @@ struct Run: AsyncParsableCommand {
Group {
VMView(vm: vm!).onAppear {
NSWindow.allowsAutomaticWindowTabbing = false
}.onDisappear {
NSApplication.shared.terminate(self)
}
}.frame(width: CGFloat(vm!.config.display.width), height: CGFloat(vm!.config.display.height))
}.commands {
+21 -2
View File
@@ -3,8 +3,9 @@ import Foundation
struct Config {
let tartHomeDir: URL
let tartCacheDir: URL
let tartTmpDir: URL
init() {
init() throws {
var tartHomeDir: URL
if let customTartHome = ProcessInfo.processInfo.environment["TART_HOME"] {
@@ -14,9 +15,27 @@ struct Config {
.homeDirectoryForCurrentUser
.appendingPathComponent(".tart", isDirectory: true)
}
self.tartHomeDir = tartHomeDir
tartCacheDir = tartHomeDir.appendingPathComponent("cache", isDirectory: true)
try FileManager.default.createDirectory(at: tartCacheDir, withIntermediateDirectories: true)
tartTmpDir = tartHomeDir.appendingPathComponent("tmp", isDirectory: true)
try FileManager.default.createDirectory(at: tartTmpDir, withIntermediateDirectories: true)
}
func gc() throws {
for entry in try FileManager.default.contentsOfDirectory(at: tartTmpDir,
includingPropertiesForKeys: [], options: []) {
let lock = try FileLock(lockURL: entry)
if try !lock.trylock() {
continue
}
try FileManager.default.removeItem(at: entry)
try lock.unlock()
}
}
static func jsonEncoder() -> JSONEncoder {
@@ -0,0 +1,88 @@
import Foundation
class DockerConfigCredentialsProvider: CredentialsProvider {
func retrieve(host: String) throws -> (String, String)? {
let dockerConfigURL = FileManager.default.homeDirectoryForCurrentUser.appendingPathComponent(".docker").appendingPathComponent("config.json")
if !FileManager.default.fileExists(atPath: dockerConfigURL.path) {
return nil
}
let config = try JSONDecoder().decode(DockerConfig.self, from: Data(contentsOf: dockerConfigURL))
if let credentialsFromAuth = config.auths?[host]?.decodeCredentials() {
return credentialsFromAuth
}
if let helperProgram = config.credHelpers?[host] {
return try executeHelper(binaryName: "docker-credential-\(helperProgram)", host: host)
}
return nil
}
private func executeHelper(binaryName: String, host: String) throws -> (String, String)? {
guard let executableURL = resolveBinaryPath(binaryName) else {
throw CredentialsProviderError.Failed(message: "\(binaryName) not found in PATH")
}
let process = Process.init()
process.executableURL = executableURL
process.arguments = ["get"]
let outPipe = Pipe()
let inPipe = Pipe()
process.standardOutput = outPipe
process.standardError = outPipe
process.standardInput = inPipe
process.launch()
inPipe.fileHandleForWriting.write("\(host)\n".data(using: .utf8)!)
inPipe.fileHandleForWriting.closeFile()
process.waitUntilExit()
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
throw CredentialsProviderError.Failed(message: "Docker helper failed!")
}
let getOutput = try JSONDecoder().decode(
DockerGetOutput.self, from: outPipe.fileHandleForReading.readDataToEndOfFile()
)
return (getOutput.Username, getOutput.Secret)
}
func store(host: String, user: String, password: String) throws {
throw CredentialsProviderError.Failed(message: "Docker helpers don't support storing!")
}
}
struct DockerConfig: Codable {
var auths: Dictionary<String, DockerAuthConfig>? = Dictionary()
var credHelpers: Dictionary<String, String>? = Dictionary()
}
struct DockerAuthConfig: Codable {
var auth: String? = nil
func decodeCredentials() -> (String, String)? {
// auth is a base64("username:password")
guard let authBase64 = auth else {
return nil
}
guard let data = Data(base64Encoded: authBase64) else {
return nil
}
guard let components = String(data: data, encoding: .utf8)?.components(separatedBy: ":") else {
return nil
}
if components.count != 2 {
return nil
}
return (components[0], components[1])
}
}
struct DockerGetOutput: Codable {
var Username: String
var Secret: String
}
+48
View File
@@ -0,0 +1,48 @@
import Foundation
import System
enum FileLockError: Error, Equatable {
case Failed(_ message: String)
case AlreadyLocked
}
class FileLock {
let url: URL
let fd: Int32
init(lockURL: URL) throws {
url = lockURL
fd = open(lockURL.path, 0)
}
deinit {
close(fd)
}
func trylock() throws -> Bool {
try flockWrapper(LOCK_EX | LOCK_NB)
}
func lock() throws {
_ = try flockWrapper(LOCK_EX)
}
func unlock() throws {
_ = try flockWrapper(LOCK_UN)
}
func flockWrapper(_ operation: Int32) throws -> Bool {
let ret = flock(fd, operation)
if ret != 0 {
let details = Errno(rawValue: CInt(errno))
if (operation & LOCK_NB) != 0 && details == .wouldBlock {
return false
}
throw FileLockError.Failed("failed to lock \(url): \(details)")
}
return true
}
}
+20
View File
@@ -0,0 +1,20 @@
import Foundation
import Virtualization
class IPSWCache: PrunableStorage {
let baseURL: URL
init() throws {
baseURL = try Config().tartCacheDir.appendingPathComponent("IPSWs", isDirectory: true)
try FileManager.default.createDirectory(at: baseURL, withIntermediateDirectories: true)
}
func locationFor(fileName: String) -> URL {
baseURL.appendingPathComponent(fileName, isDirectory: false)
}
func prunables() throws -> [Prunable] {
try FileManager.default.contentsOfDirectory(at: baseURL, includingPropertiesForKeys: nil)
.filter { $0.lastPathComponent.hasSuffix(".ipsw")}
}
}
@@ -0,0 +1,119 @@
import Foundation
import Network
import Virtualization
struct ARPCommandFailedError: Error, CustomStringConvertible {
var terminationReason: Process.TerminationReason
var terminationStatus: Int32
var description: String {
var reason: String
switch terminationReason {
case .exit:
reason = "exit code \(terminationStatus)"
case .uncaughtSignal:
reason = "uncaught signal"
default:
reason = "unknown reason"
}
return "arp command failed: \(reason)"
}
}
struct ARPCommandYieldedInvalidOutputError: Error, CustomStringConvertible {
var explanation: String
var description: String {
"arp command yielded invalid output: \(explanation)"
}
}
struct ARPCacheInternalError: Error, CustomStringConvertible {
var explanation: String
var description: String {
"ARPCache internal error: \(explanation)"
}
}
struct ARPCache {
static func ResolveMACAddress(macAddress: MACAddress, bridgeOnly: Bool = true) throws -> IPv4Address? {
let process = Process.init()
process.executableURL = URL.init(fileURLWithPath: "/usr/sbin/arp")
process.arguments = ["-an"]
let pipe = Pipe()
process.standardOutput = pipe
process.standardError = pipe
process.standardInput = FileHandle.nullDevice
try process.run()
process.waitUntilExit()
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
throw ARPCommandFailedError(
terminationReason: process.terminationReason,
terminationStatus: process.terminationStatus)
}
guard let rawLines = try pipe.fileHandleForReading.readToEnd() else {
throw ARPCommandYieldedInvalidOutputError(explanation: "empty output")
}
let lines = String(decoding: rawLines, as: UTF8.self)
.trimmingCharacters(in: .whitespacesAndNewlines)
.components(separatedBy: "\n")
// Based on https://opensource.apple.com/source/network_cmds/network_cmds-606.40.2/arp.tproj/arp.c.auto.html
let regex = try NSRegularExpression(pattern: #"^.* \((?<ip>.*)\) at (?<mac>.*) on (?<interface>.*) .*$"#)
for line in lines {
let nsLineRange = NSRange(line.startIndex..<line.endIndex, in: line)
guard let match = regex.firstMatch(in: line, range: nsLineRange) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "unparseable entry \"\(line)\"")
}
let rawIP = try match.getCaptureGroup(name: "ip", for: line)
guard let ip = IPv4Address(rawIP) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "failed to parse IPv4 address \(rawIP)")
}
let rawMAC = try match.getCaptureGroup(name: "mac", for: line)
if rawMAC == "(incomplete)" {
continue
}
guard let mac = MACAddress(fromString: rawMAC) else {
throw ARPCommandYieldedInvalidOutputError(explanation: "failed to parse MAC address \(rawMAC)")
}
let interface = try match.getCaptureGroup(name: "interface", for: line)
if bridgeOnly && !interface.starts(with: "bridge") {
continue
}
if macAddress == mac {
return ip
}
}
return nil
}
}
extension NSTextCheckingResult {
func getCaptureGroup(name: String, for string: String) throws -> String {
let nsRange = self.range(withName: name)
if nsRange.location == NSNotFound {
throw ARPCacheInternalError(explanation: "attempted to retrieve non-existent named capture group \(name)")
}
guard let range = Range.init(nsRange, in: string) else {
throw ARPCacheInternalError(explanation: "failed to convert NSRange to Range")
}
return String(string[range])
}
}
+11 -4
View File
@@ -19,14 +19,21 @@ enum LeasesError: Error {
class Leases {
private let leases: [MACAddress : Lease]
convenience init() throws {
convenience init?() throws {
try self.init(URL(fileURLWithPath: "/var/db/dhcpd_leases"))
}
convenience init(_ fromURL: URL) throws {
let fileContents = try String(contentsOf: fromURL, encoding: .utf8)
convenience init?(_ fromURL: URL) throws {
do {
let urlContents = try String(contentsOf: fromURL, encoding: .utf8)
try self.init(urlContents)
} catch {
if error.isFileNotFound() {
return nil
}
try self.init(fileContents)
throw error
}
}
init(_ fromString: String) throws {
@@ -16,6 +16,6 @@ struct MACAddress: Equatable, Hashable, CustomStringConvertible {
}
var description: String {
return String(format: "%02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5])
String(format: "%02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5])
}
}
+7
View File
@@ -0,0 +1,7 @@
import Virtualization
protocol Network {
func attachment() -> VZNetworkDeviceAttachment
func run() throws
func stop() throws
}
+22
View File
@@ -0,0 +1,22 @@
import Foundation
import Virtualization
class NetworkBridged: Network {
let interface: VZBridgedNetworkInterface
init(interface: VZBridgedNetworkInterface) {
self.interface = interface
}
func attachment() -> VZNetworkDeviceAttachment {
VZBridgedNetworkDeviceAttachment(interface: interface)
}
func run() throws {
// no-op, only used for Softnet
}
func stop() throws {
// no-op, only used for Softnet
}
}
+16
View File
@@ -0,0 +1,16 @@
import Foundation
import Virtualization
class NetworkShared: Network {
func attachment() -> VZNetworkDeviceAttachment {
VZNATNetworkDeviceAttachment()
}
func run() throws {
// no-op, only used for Softnet
}
func stop() throws {
// no-op, only used for Softnet
}
}
@@ -1,10 +1,11 @@
import Foundation
import Virtualization
enum SoftnetError: Error {
case InitializationFailed(why: String)
}
class Softnet {
class Softnet: Network {
private let process = Process()
let vmFD: Int32
@@ -12,7 +13,7 @@ class Softnet {
init(vmMACAddress: String) throws {
let binaryName = "softnet"
guard let executableURL = Self.resolveBinaryPath(binaryName) else {
guard let executableURL = resolveBinaryPath(binaryName) else {
throw SoftnetError.InitializationFailed(why: "\(binaryName) not found in PATH")
}
@@ -43,23 +44,6 @@ class Softnet {
process.waitUntilExit()
}
private static func resolveBinaryPath(_ name: String) -> URL? {
guard let path = ProcessInfo.processInfo.environment["PATH"] else {
return nil
}
for pathComponent in path.split(separator: ":") {
let url = URL(fileURLWithPath: String(pathComponent))
.appendingPathComponent(name, isDirectory: false)
if FileManager.default.fileExists(atPath: url.path) {
return url
}
}
return nil
}
private func setSocketBuffers(_ fd: Int32, _ sizeBytes: Int) throws {
var option_value = sizeBytes
let option_len = socklen_t(MemoryLayout<Int>.size)
@@ -74,4 +58,9 @@ class Softnet {
throw SoftnetError.InitializationFailed(why: "setsockopt(SO_SNDBUF) returned \(ret)")
}
}
func attachment() -> VZNetworkDeviceAttachment {
let fh = FileHandle.init(fileDescriptor: vmFD)
return VZFileHandleNetworkDeviceAttachment(fileHandle: fh)
}
}
+2 -2
View File
@@ -46,8 +46,8 @@ struct OCIManifest: Codable, Equatable {
}
struct OCIConfig: Codable {
var architecture: String = "arm64"
var os: String = "darwin"
var architecture: Architecture = .arm64
var os: OS = .darwin
func toJSON() throws -> Data {
try Config.jsonEncoder().encode(self)
+103 -84
View File
@@ -1,28 +1,44 @@
import Foundation
import NIOCore
import NIOHTTP1
import AsyncHTTPClient
import Algorithms
import NIOPosix
import AsyncAlgorithms
let chunkSizeBytes = 1 * 1024 * 1024
enum RegistryError: Error {
case UnexpectedHTTPStatusCode(when: String, code: UInt, details: String = "")
case UnexpectedHTTPStatusCode(when: String, code: Int, details: String = "")
case MissingLocationHeader
case AuthFailed(why: String, details: String = "")
case MalformedHeader(why: String)
}
extension HTTPClientResponse.Body {
func readTextResponse() async throws -> String? {
let data = try await readResponse()
return String(decoding: data, as: UTF8.self)
}
enum HTTPMethod: String {
case GET = "GET"
case POST = "POST"
case PUT = "PUT"
case PATCH = "PATCH"
}
func readResponse() async throws -> Data {
enum HTTPCode: Int {
case Ok = 200
case Created = 201
case Accepted = 202
case Unauthorized = 401
}
extension Data {
func asText() async throws -> String? {
String(decoding: self, as: UTF8.self)
}
}
extension URLSession.AsyncBytes {
func asData() async throws -> Data {
var result = Data()
for try await part in self {
result.append(Data(buffer: part))
for try await chunk in chunks(ofCount: chunkSizeBytes) {
result += chunk
}
return result
}
}
@@ -78,79 +94,71 @@ struct TokenResponse: Decodable, Authentication {
}
class Registry {
private let httpClient = HTTPClient(
eventLoopGroupProvider: .shared(MultiThreadedEventLoopGroup(numberOfThreads: 1))
)
deinit {
try! httpClient.syncShutdown()
}
let baseURL: URL
let namespace: String
let credentialsProvider: CredentialsProvider
let credentialsProviders: [CredentialsProvider]
var currentAuthToken: Authentication? = nil
init(urlComponents: URLComponents,
namespace: String,
credentialsProvider: CredentialsProvider = KeychainCredentialsProvider()
credentialsProviders: [CredentialsProvider] = [DockerConfigCredentialsProvider(), KeychainCredentialsProvider()]
) throws {
baseURL = urlComponents.url!
self.namespace = namespace
self.credentialsProvider = credentialsProvider
self.credentialsProviders = credentialsProviders
}
convenience init(
host: String,
namespace: String,
insecure: Bool = false,
credentialsProvider: CredentialsProvider = KeychainCredentialsProvider()
credentialsProviders: [CredentialsProvider] = [DockerConfigCredentialsProvider(), KeychainCredentialsProvider()]
) throws {
let proto = insecure ? "http" : "https"
let baseURLComponents = URLComponents(string: proto + "://" + host + "/v2/")!
try self.init(urlComponents: baseURLComponents, namespace: namespace, credentialsProvider: credentialsProvider)
try self.init(urlComponents: baseURLComponents, namespace: namespace, credentialsProviders: credentialsProviders)
}
func ping() async throws {
let response = try await endpointRequest(.GET, "/v2/")
if response.status != .ok {
throw RegistryError.UnexpectedHTTPStatusCode(when: "doing ping", code: response.status.code)
let (_, response) = try await endpointRequest(.GET, "/v2/")
if response.statusCode != HTTPCode.Ok.rawValue {
throw RegistryError.UnexpectedHTTPStatusCode(when: "doing ping", code: response.statusCode)
}
}
func pushManifest(reference: String, manifest: OCIManifest) async throws -> String {
let manifestJSON = try manifest.toJSON()
let response = try await endpointRequest(.PUT, "\(namespace)/manifests/\(reference)",
let (bytes, response) = try await endpointRequest(.PUT, "\(namespace)/manifests/\(reference)",
headers: ["Content-Type": manifest.mediaType],
body: manifestJSON)
if response.status != .created {
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing manifest", code: response.status.code,
details: try await response.body.readTextResponse() ?? "")
if response.statusCode != HTTPCode.Created.rawValue {
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing manifest", code: response.statusCode,
details: try await bytes.asData().asText() ?? "")
}
return Digest.hash(manifestJSON)
}
public func pullManifest(reference: String) async throws -> (OCIManifest, Data) {
let response = try await endpointRequest(.GET, "\(namespace)/manifests/\(reference)",
let (bytes, response) = try await endpointRequest(.GET, "\(namespace)/manifests/\(reference)",
headers: ["Accept": ociManifestMediaType])
if response.status != .ok {
let body = try await response.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling manifest", code: response.status.code,
if response.statusCode != HTTPCode.Ok.rawValue {
let body = try await bytes.asData().asText()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling manifest", code: response.statusCode,
details: body ?? "")
}
let manifestData = try await response.body.readResponse()
let manifestData = try await bytes.asData()
let manifest = try OCIManifest(fromJSON: manifestData)
return (manifest, manifestData)
}
private func uploadLocationFromResponse(_ response: HTTPClientResponse) throws -> URLComponents {
guard let uploadLocationRaw = response.headers.first(name: "Location") else {
private func uploadLocationFromResponse(_ response: HTTPURLResponse) throws -> URLComponents {
guard let uploadLocationRaw = response.value(forHTTPHeaderField: "Location") else {
throw RegistryError.MissingLocationHeader
}
@@ -163,11 +171,11 @@ class Registry {
public func pushBlob(fromData: Data, chunkSizeMb: Int = 0) async throws -> String {
// Initiate a blob upload
let postResponse = try await endpointRequest(.POST, "\(namespace)/blobs/uploads/",
let (bytes, postResponse) = try await endpointRequest(.POST, "\(namespace)/blobs/uploads/",
headers: ["Content-Length": "0"])
if postResponse.status != .accepted {
let body = try await postResponse.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing blob (POST)", code: postResponse.status.code,
if postResponse.statusCode != HTTPCode.Accepted.rawValue {
let body = try await bytes.asData().asText()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing blob (POST)", code: postResponse.statusCode,
details: body ?? "")
}
@@ -178,7 +186,7 @@ class Registry {
if chunkSizeMb == 0 {
// monolithic upload
let response = try await rawRequest(
let (bytes, response) = try await rawRequest(
.PUT,
uploadLocation,
headers: [
@@ -187,10 +195,10 @@ class Registry {
parameters: ["digest": digest],
body: fromData
)
if response.status != .created {
let body = try await response.body.readTextResponse()
if response.statusCode != HTTPCode.Created.rawValue {
let body = try await bytes.asData().asText()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pushing blob (PUT) to \(uploadLocation)",
code: response.status.code, details: body ?? "")
code: response.statusCode, details: body ?? "")
}
return digest
}
@@ -200,7 +208,7 @@ class Registry {
let chunks = fromData.chunks(ofCount: chunkSizeMb == 0 ? fromData.count : chunkSizeMb * 1_000_000)
for (index, chunk) in chunks.enumerated() {
let lastChunk = index == (chunks.count - 1)
let response = try await rawRequest(
let (bytes, response) = try await rawRequest(
lastChunk ? .PUT : .PATCH,
uploadLocation,
headers: [
@@ -210,11 +218,11 @@ class Registry {
parameters: lastChunk ? ["digest": digest] : [:],
body: chunk
)
let expectedStatus: HTTPResponseStatus = lastChunk ? .created : .accepted
if response.status != expectedStatus {
let body = try await response.body.readTextResponse()
let expectedStatus = lastChunk ? HTTPCode.Created.rawValue : HTTPCode.Accepted.rawValue
if response.statusCode != expectedStatus {
let body = try await bytes.asData().asText()
throw RegistryError.UnexpectedHTTPStatusCode(when: "streaming blob to \(uploadLocation)",
code: response.status.code, details: body ?? "")
code: response.statusCode, details: body ?? "")
}
uploadedBytes += chunk.count
// Update location for the next chunk
@@ -224,18 +232,18 @@ class Registry {
return digest
}
public func pullBlob(_ digest: String, handler: (ByteBuffer) throws -> Void) async throws {
let response = try await endpointRequest(.GET, "\(namespace)/blobs/\(digest)")
if response.status != .ok {
let body = try await response.body.readTextResponse()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling blob", code: response.status.code,
public func pullBlob(_ digest: String, handler: (Data) throws -> Void) async throws {
let (bytes, response) = try await endpointRequest(.GET, "\(namespace)/blobs/\(digest)")
if response.statusCode != HTTPCode.Ok.rawValue {
let body = try await bytes.asData().asText()
throw RegistryError.UnexpectedHTTPStatusCode(when: "pulling blob", code: response.statusCode,
details: body ?? "")
}
for try await part in response.body {
for try await part in bytes.chunks(ofCount: chunkSizeBytes) {
try Task.checkCancellation()
try handler(part)
try handler(Data(part))
}
}
@@ -245,7 +253,7 @@ class Registry {
headers: Dictionary<String, String> = Dictionary(),
parameters: Dictionary<String, String> = Dictionary(),
body: Data? = nil
) async throws -> HTTPClientResponse {
) async throws -> (URLSession.AsyncBytes, HTTPURLResponse) {
let url = URL(string: endpoint, relativeTo: baseURL)!
let urlComponents = URLComponents(url: url, resolvingAgainstBaseURL: true)!
@@ -259,7 +267,7 @@ class Registry {
parameters: Dictionary<String, String> = Dictionary(),
body: Data? = nil,
doAuth: Bool = true
) async throws -> HTTPClientResponse {
) async throws -> (URLSession.AsyncBytes, HTTPURLResponse) {
var urlComponents = urlComponents
if urlComponents.queryItems == nil && !parameters.isEmpty {
@@ -269,14 +277,14 @@ class Registry {
URLQueryItem(name: key, value: value)
})
var request = HTTPClientRequest(url: urlComponents.string!)
request.method = method
var request = URLRequest(url: urlComponents.url!)
request.httpMethod = method.rawValue
for (key, value) in headers {
request.headers.add(name: key, value: value)
request.addValue(value, forHTTPHeaderField: key)
}
if body != nil {
request.headers.add(name: "Content-Length", value: "\(body!.count)")
request.body = HTTPClientRequest.Body.bytes(body!)
if let body = body {
request.addValue("\(body.count)", forHTTPHeaderField: "Content-Length")
request.httpBody = body
}
// Invalidate token if it has expired
@@ -284,26 +292,26 @@ class Registry {
currentAuthToken = nil
}
var response = try await authAwareRequest(request: request)
var (bytes, response) = try await authAwareRequest(request: request)
if doAuth && response.status == .unauthorized {
if doAuth && response.statusCode == HTTPCode.Unauthorized.rawValue {
try await auth(response: response)
response = try await authAwareRequest(request: request)
(bytes, response) = try await authAwareRequest(request: request)
}
return response
return (bytes, response)
}
private func auth(response: HTTPClientResponse) async throws {
private func auth(response: HTTPURLResponse) async throws {
// Process WWW-Authenticate header
guard let wwwAuthenticateRaw = response.headers.first(name: "WWW-Authenticate") else {
guard let wwwAuthenticateRaw = response.value(forHTTPHeaderField: "WWW-Authenticate") else {
throw RegistryError.AuthFailed(why: "got HTTP 401, but WWW-Authenticate header is missing")
}
let wwwAuthenticate = try WWWAuthenticate(rawHeaderValue: wwwAuthenticateRaw)
if wwwAuthenticate.scheme == "Basic" {
if let (user, password) = try credentialsProvider.retrieve(host: baseURL.host!) {
if let (user, password) = try lookupCredentials(host: baseURL.host!) {
currentAuthToken = BasicAuthentication(user: user, password: password)
}
@@ -340,30 +348,41 @@ class Registry {
var headers: Dictionary<String, String> = Dictionary()
if let (user, password) = try credentialsProvider.retrieve(host: baseURL.host!) {
if let (user, password) = try lookupCredentials(host: baseURL.host!) {
let encodedCredentials = "\(user):\(password)".data(using: .utf8)?.base64EncodedString()
headers["Authorization"] = "Basic \(encodedCredentials!)"
}
let response = try await rawRequest(.GET, authenticateURL, headers: headers, doAuth: false)
if response.status != .ok {
let body = try await response.body.readTextResponse() ?? ""
throw RegistryError.AuthFailed(why: "received unexpected HTTP status code \(response.status.code) "
let (bytes, response) = try await rawRequest(.GET, authenticateURL, headers: headers, doAuth: false)
if response.statusCode != HTTPCode.Ok.rawValue {
let body = try await bytes.asData() .asText() ?? ""
throw RegistryError.AuthFailed(why: "received unexpected HTTP status code \(response.statusCode) "
+ "while retrieving an authentication token", details: body)
}
let bodyData = try await response.body.readResponse()
let bodyData = try await bytes.asData()
currentAuthToken = try TokenResponse.parse(fromData: bodyData)
}
private func authAwareRequest(request: HTTPClientRequest) async throws -> HTTPClientResponse {
private func lookupCredentials(host: String) throws -> (String, String)? {
for provider in credentialsProviders {
if let (user, password) = try provider.retrieve(host: host) {
return (user, password)
}
}
return nil
}
private func authAwareRequest(request: URLRequest) async throws -> (URLSession.AsyncBytes, HTTPURLResponse) {
var request = request
if let token = currentAuthToken {
let (name, value) = token.header()
request.headers.add(name: name, value: value)
request.addValue(value, forHTTPHeaderField: name)
}
return try await httpClient.execute(request, deadline: .distantFuture)
let (bytes, response) = try await URLSession.shared.bytes(for: request)
return (bytes, response as! HTTPURLResponse)
}
}
+14
View File
@@ -0,0 +1,14 @@
import Foundation
enum Architecture: String, Codable {
case arm64
case amd64
}
func CurrentArchitecture() -> Architecture {
#if arch(arm64)
return .arm64
#elseif arch(x86_64)
return .amd64
#endif
}
+97
View File
@@ -0,0 +1,97 @@
import Virtualization
struct Darwin: Platform {
var ecid: VZMacMachineIdentifier
var hardwareModel: VZMacHardwareModel
init(ecid: VZMacMachineIdentifier, hardwareModel: VZMacHardwareModel) {
self.ecid = ecid
self.hardwareModel = hardwareModel
}
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
let encodedECID = try container.decode(String.self, forKey: .ecid)
guard let data = Data.init(base64Encoded: encodedECID) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize Data using the provided value")
}
guard let ecid = VZMacMachineIdentifier.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize VZMacMachineIdentifier using the provided value")
}
self.ecid = ecid
let encodedHardwareModel = try container.decode(String.self, forKey: .hardwareModel)
guard let data = Data.init(base64Encoded: encodedHardwareModel) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
guard let hardwareModel = VZMacHardwareModel.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
self.hardwareModel = hardwareModel
}
func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: CodingKeys.self)
try container.encode(ecid.dataRepresentation.base64EncodedString(), forKey: .ecid)
try container.encode(hardwareModel.dataRepresentation.base64EncodedString(), forKey: .hardwareModel)
}
func os() -> OS {
.darwin
}
func bootLoader(nvramURL: URL) throws -> VZBootLoader {
VZMacOSBootLoader()
}
func platform(nvramURL: URL) -> VZPlatformConfiguration {
let result = VZMacPlatformConfiguration()
result.machineIdentifier = ecid
result.auxiliaryStorage = VZMacAuxiliaryStorage(contentsOf: nvramURL)
result.hardwareModel = hardwareModel
return result
}
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration {
let result = VZMacGraphicsDeviceConfiguration()
if let hostMainScreen = NSScreen.main {
let vmScreenSize = NSSize(width: vmConfig.display.width, height: vmConfig.display.height)
result.displays = [
VZMacGraphicsDisplayConfiguration(for: hostMainScreen, sizeInPoints: vmScreenSize)
]
return result
}
result.displays = [
VZMacGraphicsDisplayConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height,
// A reasonable guess according to Apple's documentation[1]
// [1]: https://developer.apple.com/documentation/coregraphics/1456599-cgdisplayscreensize
pixelsPerInch: 72
)
]
return result
}
func pointingDevices() -> [VZPointingDeviceConfiguration] {
if #available(macOS 13, *) {
// Trackpad is only supported starting with macOS Ventura
// macOS Monterey will continue using a USB device == .darwin
return [VZMacTrackpadConfiguration(), VZUSBScreenCoordinatePointingDeviceConfiguration()]
} else {
return [VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
}
}
+37
View File
@@ -0,0 +1,37 @@
import Virtualization
@available(macOS 13, *)
struct Linux: Platform {
func os() -> OS {
.linux
}
func bootLoader(nvramURL: URL) throws -> VZBootLoader {
let result = VZEFIBootLoader()
result.variableStore = VZEFIVariableStore(url: nvramURL)
return result
}
func platform(nvramURL: URL) -> VZPlatformConfiguration {
VZGenericPlatformConfiguration()
}
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration {
let result = VZVirtioGraphicsDeviceConfiguration()
result.scanouts = [
VZVirtioGraphicsScanoutConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height
)
]
return result
}
func pointingDevices() -> [VZPointingDeviceConfiguration] {
[VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
}
+6
View File
@@ -0,0 +1,6 @@
import Virtualization
enum OS: String, Codable {
case darwin
case linux
}
+9
View File
@@ -0,0 +1,9 @@
import Virtualization
protocol Platform: Codable {
func os() -> OS
func bootLoader(nvramURL: URL) throws -> VZBootLoader
func platform(nvramURL: URL) -> VZPlatformConfiguration
func graphicsDevice(vmConfig: VMConfig) -> VZGraphicsDeviceConfiguration
func pointingDevices() -> [VZPointingDeviceConfiguration]
}
+11
View File
@@ -0,0 +1,11 @@
import Foundation
protocol PrunableStorage {
func prunables() throws -> [Prunable]
}
protocol Prunable {
func delete() throws
func accessDate() throws -> Date
func sizeBytes() throws -> Int
}
+8
View File
@@ -16,6 +16,8 @@ struct Root: AsyncParsableCommand {
IP.self,
Pull.self,
Push.self,
Prune.self,
Rename.self,
Delete.self,
])
@@ -31,10 +33,16 @@ struct Root: AsyncParsableCommand {
}
sigintSrc.activate()
// Set line-buffered output for stdout
setlinebuf(stdout)
// Parse and run command
do {
var command = try parseAsRoot()
// Run garbage-collection before each command (shouldn't take too long)
try Config().gc()
if var asyncCommand = command as? AsyncParsableCommand {
try await asyncCommand.run()
} else {
+25
View File
@@ -0,0 +1,25 @@
import Foundation
extension URL {
func accessDate() throws -> Date {
let attrs = try resourceValues(forKeys: [.contentAccessDateKey])
return attrs.contentAccessDate!
}
func updateAccessDate(_ accessDate: Date = Date()) throws {
let attrs = try resourceValues(forKeys: [.contentAccessDateKey])
let modificationDate = attrs.contentAccessDate!
let times = [accessDate.asTimeval(), modificationDate.asTimeval()]
let ret = utimes(path, times)
if ret != 0 {
throw RuntimeError("utimes(2) failed: \(ret.explanation())")
}
}
}
extension Date {
func asTimeval() -> timeval {
timeval(tv_sec: Int(timeIntervalSince1970), tv_usec: 0)
}
}
+11
View File
@@ -0,0 +1,11 @@
import Foundation
extension URL: Prunable {
func delete() throws {
try FileManager.default.removeItem(at: self)
}
func sizeBytes() throws -> Int {
try resourceValues(forKeys: [.totalFileAllocatedSizeKey]).totalFileAllocatedSize!
}
}
+17
View File
@@ -5,3 +5,20 @@ extension Collection {
indices.contains(index) ? self[index] : nil
}
}
func resolveBinaryPath(_ name: String) -> URL? {
guard let path = ProcessInfo.processInfo.environment["PATH"] else {
return nil
}
for pathComponent in path.split(separator: ":") {
let url = URL(fileURLWithPath: String(pathComponent))
.appendingPathComponent(name, isDirectory: false)
if FileManager.default.fileExists(atPath: url.path) {
return url
}
}
return nil
}
+119 -88
View File
@@ -10,6 +10,17 @@ struct NoMainScreenFoundError: Error {
struct DownloadFailed: Error {
}
struct UnsupportedOSError: Error, CustomStringConvertible {
let description: String
init(_ what: String, _ plural: String) {
description = "error: \(what) \(plural) only supported on hosts running macOS 13.0 (Ventura) or newer"
}
}
struct UnsupportedArchitectureError: Error {
}
class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Virtualization.Framework's virtual machine
@Published var virtualMachine: VZVirtualMachine
@@ -23,53 +34,55 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// VM's config
var config: VMConfig
var softnet: Softnet? = nil
var network: Network
init(vmDir: VMDirectory,
withSoftnet: Bool = false,
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment] = []
network: Network = NetworkShared(),
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment] = [],
directoryShares: [DirectoryShare] = []
) throws {
let auxStorage = VZMacAuxiliaryStorage(contentsOf: vmDir.nvramURL)
name = vmDir.name
config = try VMConfig.init(fromURL: vmDir.configURL)
// Initialize the virtual machine and its configuration
if withSoftnet {
softnet = try Softnet(vmMACAddress: config.macAddress.string)
if config.arch != CurrentArchitecture() {
throw UnsupportedArchitectureError()
}
let configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL, auxStorage: auxStorage, vmConfig: config,
softnet: softnet, additionalDiskAttachments: additionalDiskAttachments)
// Initialize the virtual machine and its configuration
self.network = network
let configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL,
nvramURL: vmDir.nvramURL, vmConfig: config,
network: network, additionalDiskAttachments: additionalDiskAttachments,
directoryShares: directoryShares)
virtualMachine = VZVirtualMachine(configuration: configuration)
super.init()
virtualMachine.delegate = self
}
static func retrieveLatestIPSW() async throws -> URL {
defaultLogger.appendNewLine("Looking up the latest supported IPSW...")
let image = try await withCheckedThrowingContinuation { continuation in
VZMacOSRestoreImage.fetchLatestSupported() { result in
continuation.resume(with: result)
static func retrieveIPSW(remoteURL: URL) async throws -> URL {
// Check if we already have this IPSW in cache
var request = URLRequest(url: remoteURL)
request.httpMethod = "HEAD"
let (_, response) = try await URLSession.shared.data(for: request)
let httpURLResponse = response as! HTTPURLResponse
if let hash = httpURLResponse.value(forHTTPHeaderField: "x-amz-meta-digest-sha256") {
let ipswLocation = try IPSWCache().locationFor(fileName: "sha256:\(hash).ipsw")
if FileManager.default.fileExists(atPath: ipswLocation.path) {
defaultLogger.appendNewLine("Using cached *.ipsw file...")
try ipswLocation.updateAccessDate()
return ipswLocation
}
}
let ipswCacheFolder = Config().tartCacheDir.appendingPathComponent("IPSWs", isDirectory: true)
try FileManager.default.createDirectory(at: ipswCacheFolder, withIntermediateDirectories: true)
let expectedIPSWLocation = ipswCacheFolder.appendingPathComponent("\(image.buildVersion).ipsw", isDirectory: false)
if FileManager.default.fileExists(atPath: expectedIPSWLocation.path) {
defaultLogger.appendNewLine("Using cached *.ipsw file...")
return expectedIPSWLocation
}
defaultLogger.appendNewLine("Fetching \(expectedIPSWLocation.lastPathComponent)...")
// Download the IPSW
defaultLogger.appendNewLine("Fetching \(remoteURL.lastPathComponent)...")
let data: Data = try await withCheckedThrowingContinuation { continuation in
let downloadedTask = URLSession.shared.dataTask(with: image.url) { data, response, error in
let downloadedTask = URLSession.shared.dataTask(with: remoteURL) { data, response, error in
if error != nil {
continuation.resume(throwing: error!)
return
@@ -84,10 +97,24 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
downloadedTask.resume()
}
try data.write(to: expectedIPSWLocation, options: [.atomic])
return expectedIPSWLocation
let ipswLocation = try IPSWCache().locationFor(fileName: Digest.hash(data) + ".ipsw")
try data.write(to: ipswLocation, options: [.atomic])
return ipswLocation
}
static func latestIPSWURL() async throws -> URL {
defaultLogger.appendNewLine("Looking up the latest supported IPSW...")
let image = try await withCheckedThrowingContinuation { continuation in
VZMacOSRestoreImage.fetchLatestSupported() { result in
continuation.resume(with: result)
}
}
return image.url
}
var inFinalState: Bool {
get {
virtualMachine.state == VZVirtualMachine.State.stopped ||
@@ -99,12 +126,16 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
init(
vmDir: VMDirectory,
ipswURL: URL?,
ipswURL: URL,
diskSizeGB: UInt16,
withSoftnet: Bool = false,
network: Network = NetworkShared(),
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment] = []
) async throws {
let ipswURL = ipswURL != nil ? ipswURL! : try await VM.retrieveLatestIPSW();
var ipswURL = ipswURL
if !ipswURL.isFileURL {
ipswURL = try await VM.retrieveIPSW(remoteURL: ipswURL)
}
// Load the restore image and try to get the requirements
// that match both the image and our platform
@@ -119,7 +150,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
// Create NVRAM
let auxStorage = try VZMacAuxiliaryStorage(creatingStorageAt: vmDir.nvramURL, hardwareModel: requirements.hardwareModel)
_ = try VZMacAuxiliaryStorage(creatingStorageAt: vmDir.nvramURL, hardwareModel: requirements.hardwareModel)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
@@ -127,7 +158,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
name = vmDir.name
// Create config
config = VMConfig(
hardwareModel: requirements.hardwareModel,
platform: Darwin(ecid: VZMacMachineIdentifier(), hardwareModel: requirements.hardwareModel),
cpuCountMin: requirements.minimumSupportedCPUCount,
memorySizeMin: requirements.minimumSupportedMemorySize
)
@@ -136,12 +167,11 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
try config.save(toURL: vmDir.configURL)
// Initialize the virtual machine and its configuration
if withSoftnet {
softnet = try Softnet(vmMACAddress: config.macAddress.string)
}
let configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL, auxStorage: auxStorage, vmConfig: config,
softnet: softnet, additionalDiskAttachments: additionalDiskAttachments)
self.network = network
let configuration = try Self.craftConfiguration(diskURL: vmDir.diskURL, nvramURL: vmDir.nvramURL,
vmConfig: config, network: network,
additionalDiskAttachments: additionalDiskAttachments,
directoryShares: [])
virtualMachine = VZVirtualMachine(configuration: configuration)
super.init()
@@ -149,7 +179,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Run automated installation
try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<Void, Error>) in
DispatchQueue.main.async {
DispatchQueue.main.async { [ipswURL] in
let installer = VZMacOSInstaller(virtualMachine: self.virtualMachine, restoringFromImageAt: ipswURL)
defaultLogger.appendNewLine("Installing OS...")
@@ -162,10 +192,23 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
}
@available(macOS 13, *)
static func linux(vmDir: VMDirectory, diskSizeGB: UInt16) async throws -> VM {
// Create NVRAM
_ = try VZEFIVariableStore(creatingVariableStoreAt: vmDir.nvramURL)
// Create disk
try vmDir.resizeDisk(diskSizeGB)
// Create config
let config = VMConfig(platform: Linux(), cpuCountMin: 4, memorySizeMin: 4096 * 1024 * 1024)
try config.save(toURL: vmDir.configURL)
return try VM(vmDir: vmDir)
}
func run(_ recovery: Bool) async throws {
if let softnet = softnet {
try softnet.run()
}
try network.run()
DispatchQueue.main.sync {
Task {
@@ -195,57 +238,31 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
}
}
if let softnet = softnet {
try softnet.stop();
}
try network.stop()
}
static func craftConfiguration(
diskURL: URL,
auxStorage: VZMacAuxiliaryStorage,
nvramURL: URL,
vmConfig: VMConfig,
softnet: Softnet? = nil,
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment]
network: Network = NetworkShared(),
additionalDiskAttachments: [VZDiskImageStorageDeviceAttachment],
directoryShares: [DirectoryShare]
) throws -> VZVirtualMachineConfiguration {
let configuration = VZVirtualMachineConfiguration()
// Boot loader
configuration.bootLoader = VZMacOSBootLoader()
configuration.bootLoader = try vmConfig.platform.bootLoader(nvramURL: nvramURL)
// CPU and memory
configuration.cpuCount = vmConfig.cpuCount
configuration.memorySize = vmConfig.memorySize
// Platform
let platform = VZMacPlatformConfiguration()
platform.machineIdentifier = vmConfig.ecid
platform.auxiliaryStorage = auxStorage
platform.hardwareModel = vmConfig.hardwareModel
configuration.platform = platform
configuration.platform = vmConfig.platform.platform(nvramURL: nvramURL)
// Display
let graphicsDeviceConfiguration = VZMacGraphicsDeviceConfiguration()
if let hostMainScreen = NSScreen.main {
let vmScreenSize = NSSize(
width: vmConfig.display.width,
height: vmConfig.display.height
)
graphicsDeviceConfiguration.displays = [
VZMacGraphicsDisplayConfiguration(for: hostMainScreen, sizeInPoints: vmScreenSize)
]
} else {
graphicsDeviceConfiguration.displays = [
VZMacGraphicsDisplayConfiguration(
widthInPixels: vmConfig.display.width,
heightInPixels: vmConfig.display.height,
// Reasonable guess like https://developer.apple.com/documentation/coregraphics/1456599-cgdisplayscreensize
pixelsPerInch: 72
)
]
}
configuration.graphicsDevices = [graphicsDeviceConfiguration]
configuration.graphicsDevices = [vmConfig.platform.graphicsDevice(vmConfig: vmConfig)]
// Audio
let soundDeviceConfiguration = VZVirtioSoundDeviceConfiguration()
@@ -258,17 +275,11 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Keyboard and mouse
configuration.keyboards = [VZUSBKeyboardConfiguration()]
configuration.pointingDevices = [VZUSBScreenCoordinatePointingDeviceConfiguration()]
configuration.pointingDevices = vmConfig.platform.pointingDevices()
// Networking
let vio = VZVirtioNetworkDeviceConfiguration()
if let softnet = softnet {
let fh = FileHandle.init(fileDescriptor: softnet.vmFD)
vio.attachment = VZFileHandleNetworkDeviceAttachment(fileHandle: fh)
} else {
vio.attachment = VZNATNetworkDeviceAttachment()
}
vio.attachment = network.attachment()
vio.macAddress = vmConfig.macAddress
configuration.networkDevices = [vio]
@@ -280,6 +291,20 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Entropy
configuration.entropyDevices = [VZVirtioEntropyDeviceConfiguration()]
// Directory share
if #available(macOS 13, *) {
var directories: [String : VZSharedDirectory] = Dictionary()
directoryShares.forEach { directories[$0.name] = VZSharedDirectory(url: $0.path, readOnly: $0.readOnly) }
let automountTag = VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag
let sharingDevice = VZVirtioFileSystemDeviceConfiguration(tag: automountTag)
sharingDevice.share = VZMultipleDirectoryShare(directories: directories)
configuration.directorySharingDevices = [sharingDevice]
} else if !directoryShares.isEmpty {
throw UnsupportedOSError("directory sharing", "is")
}
try configuration.validate()
return configuration
@@ -300,3 +325,9 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
sema.signal()
}
}
struct DirectoryShare {
let name: String
let path: URL
let readOnly: Bool
}
+30 -36
View File
@@ -16,14 +16,18 @@ class LessThanMinimalResourcesError: NSObject, LocalizedError {
enum CodingKeys: String, CodingKey {
case version
case ecid
case hardwareModel
case os
case arch
case cpuCountMin
case cpuCount
case memorySizeMin
case memorySize
case macAddress
case display
// macOS-specific keys
case ecid
case hardwareModel
}
struct VMDisplayConfig: Codable {
@@ -33,25 +37,25 @@ struct VMDisplayConfig: Codable {
struct VMConfig: Codable {
var version: Int = 1
var ecid: VZMacMachineIdentifier
var hardwareModel: VZMacHardwareModel
var os: OS
var arch: Architecture
var platform: Platform
var cpuCountMin: Int
private(set) var cpuCount: Int
var memorySizeMin: UInt64
private(set) var memorySize: UInt64
var macAddress: VZMACAddress
var display: VMDisplayConfig = VMDisplayConfig()
init(
ecid: VZMacMachineIdentifier = VZMacMachineIdentifier(),
hardwareModel: VZMacHardwareModel,
cpuCountMin: Int,
memorySizeMin: UInt64,
macAddress: VZMACAddress = VZMACAddress.randomLocallyAdministered()
platform: Platform,
cpuCountMin: Int,
memorySizeMin: UInt64,
macAddress: VZMACAddress = VZMACAddress.randomLocallyAdministered()
) {
self.ecid = ecid
self.hardwareModel = hardwareModel
self.os = platform.os()
self.arch = CurrentArchitecture()
self.platform = platform
self.macAddress = macAddress
self.cpuCountMin = cpuCountMin
self.memorySizeMin = memorySizeMin
@@ -81,29 +85,18 @@ struct VMConfig: Codable {
let container = try decoder.container(keyedBy: CodingKeys.self)
version = try container.decode(Int.self, forKey: .version)
let encodedECID = try container.decode(String.self, forKey: .ecid)
guard let data = Data.init(base64Encoded: encodedECID) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize Data using the provided value")
os = try container.decodeIfPresent(OS.self, forKey: .os) ?? .darwin
arch = try container.decodeIfPresent(Architecture.self, forKey: .arch) ?? .arm64
switch os {
case .darwin:
platform = try Darwin(from: decoder)
case .linux:
if #available(macOS 13, *) {
platform = try Linux(from: decoder)
} else {
throw UnsupportedOSError("Linux VMs", "are")
}
}
guard let ecid = VZMacMachineIdentifier.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .ecid,
in: container,
debugDescription: "failed to initialize VZMacMachineIdentifier using the provided value")
}
self.ecid = ecid
let encodedHardwareModel = try container.decode(String.self, forKey: .hardwareModel)
guard let data = Data.init(base64Encoded: encodedHardwareModel) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
guard let hardwareModel = VZMacHardwareModel.init(dataRepresentation: data) else {
throw DecodingError.dataCorruptedError(forKey: .hardwareModel, in: container, debugDescription: "")
}
self.hardwareModel = hardwareModel
cpuCountMin = try container.decode(Int.self, forKey: .cpuCountMin)
cpuCount = try container.decode(Int.self, forKey: .cpuCount)
memorySizeMin = try container.decode(UInt64.self, forKey: .memorySizeMin)
@@ -125,8 +118,9 @@ struct VMConfig: Codable {
var container = encoder.container(keyedBy: CodingKeys.self)
try container.encode(version, forKey: .version)
try container.encode(ecid.dataRepresentation.base64EncodedString(), forKey: .ecid)
try container.encode(hardwareModel.dataRepresentation.base64EncodedString(), forKey: .hardwareModel)
try container.encode(os, forKey: .os)
try container.encode(arch, forKey: .arch)
try platform.encode(to: encoder)
try container.encode(cpuCountMin, forKey: .cpuCountMin)
try container.encode(cpuCount, forKey: .cpuCount)
try container.encode(memorySizeMin, forKey: .memorySizeMin)
+15 -14
View File
@@ -35,8 +35,8 @@ extension VMDirectory {
throw OCIError.FailedToCreateVmFile
}
let configFile = try FileHandle(forWritingTo: configURL)
try await registry.pullBlob(configLayers.first!.digest) { buffer in
configFile.write(Data(buffer: buffer))
try await registry.pullBlob(configLayers.first!.digest) { data in
configFile.write(data)
}
try configFile.close()
@@ -70,8 +70,7 @@ extension VMDirectory {
ProgressObserver(progress).log(defaultLogger)
for diskLayer in diskLayers {
try await registry.pullBlob(diskLayer.digest) { buffer in
let data = Data(buffer: buffer)
try await registry.pullBlob(diskLayer.digest) { data in
try filter.write(data)
progress.completedUnitCount += Int64(data.count)
}
@@ -92,8 +91,8 @@ extension VMDirectory {
throw OCIError.FailedToCreateVmFile
}
let nvram = try FileHandle(forWritingTo: nvramURL)
try await registry.pullBlob(nvramLayers.first!.digest) { buffer in
nvram.write(Data(buffer: buffer))
try await registry.pullBlob(nvramLayers.first!.digest) { data in
nvram.write(data)
}
try nvram.close()
}
@@ -117,13 +116,15 @@ extension VMDirectory {
// Read VM's compressed disk as chunks
// and sequentially upload them as blobs
let disk = try FileHandle(forReadingFrom: diskURL)
var diskReadBytes: UInt64 = 0
let compressingFilter = try InputFilter<Data>(.compress, using: .lz4, bufferCapacity: Self.bufferSizeBytes) { _ in
let data = try disk.read(upToCount: Self.bufferSizeBytes)
diskReadBytes += UInt64(data?.count ?? 0)
let mappedDisk = try Data(contentsOf: diskURL, options: [.alwaysMapped])
let mappedDiskSize = mappedDisk.count
var mappedDiskReadOffset = 0
let compressingFilter = try InputFilter(.compress, using: .lz4, bufferCapacity: Self.bufferSizeBytes) { (length: Int) -> Data? in
let bytesRead = min(length, mappedDiskSize - mappedDiskReadOffset)
let data = mappedDisk.subdata(in: mappedDiskReadOffset ..< mappedDiskReadOffset + bytesRead)
mappedDiskReadOffset += bytesRead
progress.completedUnitCount += Int64(data?.count ?? 0)
progress.completedUnitCount = Int64(mappedDiskReadOffset)
return data
}
@@ -140,12 +141,12 @@ extension VMDirectory {
layers.append(OCIManifestLayer(mediaType: Self.nvramMediaType, size: nvram.count, digest: nvramDigest))
// Craft a stub OCI config for Docker Hub compatibility
let ociConfigJSON = try OCIConfig().toJSON()
let ociConfigJSON = try OCIConfig(architecture: config.arch, os: config.os).toJSON()
let ociConfigDigest = try await registry.pushBlob(fromData: ociConfigJSON, chunkSizeMb: chunkSizeMb)
let manifest = OCIManifest(
config: OCIManifestConfig(size: ociConfigJSON.count, digest: ociConfigDigest),
layers: layers,
uncompressedDiskSize: diskReadBytes
uncompressedDiskSize: UInt64(mappedDiskReadOffset)
)
// Manifest
+26 -2
View File
@@ -7,7 +7,7 @@ struct UninitializedVMDirectoryError: Error {
struct AlreadyInitializedVMDirectoryError: Error {
}
struct VMDirectory {
struct VMDirectory: Prunable {
var baseURL: URL
var configURL: URL {
@@ -20,12 +20,16 @@ struct VMDirectory {
baseURL.appendingPathComponent("nvram.bin")
}
var explicitlyPulledMark: URL {
baseURL.appendingPathComponent(".explicitly-pulled")
}
var name: String {
baseURL.lastPathComponent
}
static func temporary() throws -> VMDirectory {
let tmpDir = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString)
let tmpDir = try Config().tartTmpDir.appendingPathComponent(UUID().uuidString)
try FileManager.default.createDirectory(at: tmpDir, withIntermediateDirectories: false)
return VMDirectory(baseURL: tmpDir)
@@ -77,4 +81,24 @@ struct VMDirectory {
try diskFileHandle.truncate(atOffset: UInt64(sizeGB) * 1000 * 1000 * 1000)
try diskFileHandle.close()
}
func delete() throws {
try FileManager.default.removeItem(at: baseURL)
}
func accessDate() throws -> Date {
try baseURL.accessDate()
}
func sizeBytes() throws -> Int {
try configURL.sizeBytes() + diskURL.sizeBytes() + nvramURL.sizeBytes()
}
func markExplicitlyPulled() {
FileManager.default.createFile(atPath: explicitlyPulledMark.path, contents: nil)
}
func isExplicitlyPulled() -> Bool {
FileManager.default.fileExists(atPath: explicitlyPulledMark.path)
}
}
+5 -1
View File
@@ -1,7 +1,7 @@
import Foundation
class VMStorageLocal {
let baseURL: URL = Config().tartHomeDir.appendingPathComponent("vms", isDirectory: true)
let baseURL: URL = try! Config().tartHomeDir.appendingPathComponent("vms", isDirectory: true)
private func vmURL(_ name: String) -> URL {
baseURL.appendingPathComponent(name, isDirectory: true)
@@ -32,6 +32,10 @@ class VMStorageLocal {
_ = try FileManager.default.replaceItemAt(vmURL(name), withItemAt: from.baseURL)
}
func rename(_ name: String, _ newName: String) throws {
_ = try FileManager.default.replaceItemAt(vmURL(newName), withItemAt: vmURL(name))
}
func delete(_ name: String) throws {
try FileManager.default.removeItem(at: vmURL(name))
}
+106 -10
View File
@@ -1,12 +1,16 @@
import Foundation
class VMStorageOCI {
let baseURL = Config().tartCacheDir.appendingPathComponent("OCIs", isDirectory: true)
class VMStorageOCI: PrunableStorage {
let baseURL = try! Config().tartCacheDir.appendingPathComponent("OCIs", isDirectory: true)
private func vmURL(_ name: RemoteName) -> URL {
baseURL.appendingRemoteName(name)
}
private func hostDirectoryURL(_ name: RemoteName) -> URL {
baseURL.appendingHost(name)
}
func exists(_ name: RemoteName) -> Bool {
VMDirectory(baseURL: vmURL(name)).initialized
}
@@ -16,6 +20,8 @@ class VMStorageOCI {
try vmDir.validate()
try vmDir.baseURL.updateAccessDate()
return vmDir
}
@@ -40,10 +46,48 @@ class VMStorageOCI {
func delete(_ name: RemoteName) throws {
try FileManager.default.removeItem(at: vmURL(name))
try gc()
}
func list() throws -> [(String, VMDirectory)] {
var result: [(String, VMDirectory)] = Array()
func gc() throws {
var refCounts = Dictionary<URL, UInt>()
guard let enumerator = FileManager.default.enumerator(at: baseURL,
includingPropertiesForKeys: [.isSymbolicLinkKey]) else {
return
}
for case let foundURL as URL in enumerator {
let isSymlink = try foundURL.resourceValues(forKeys: [.isSymbolicLinkKey]).isSymbolicLink!
// Perform garbage collection for tag-based images
// with broken outgoing references
if isSymlink && foundURL == foundURL.resolvingSymlinksInPath() {
try FileManager.default.removeItem(at: foundURL)
continue
}
let vmDir = VMDirectory(baseURL: foundURL.resolvingSymlinksInPath())
if !vmDir.initialized {
continue
}
refCounts[vmDir.baseURL] = (refCounts[vmDir.baseURL] ?? 0) + (isSymlink ? 1 : 0)
}
// Perform garbage collection for digest-based images
// with no incoming references
for (baseURL, incRefCount) in refCounts {
let vmDir = VMDirectory(baseURL: baseURL)
if !vmDir.isExplicitlyPulled() && incRefCount == 0 {
try FileManager.default.removeItem(at: baseURL)
}
}
}
func list() throws -> [(String, VMDirectory, Bool)] {
var result: [(String, VMDirectory, Bool)] = Array()
guard let enumerator = FileManager.default.enumerator(at: baseURL,
includingPropertiesForKeys: [.isSymbolicLinkKey], options: [.producesRelativePathURLs]) else {
@@ -60,28 +104,70 @@ class VMStorageOCI {
let parts = [foundURL.deletingLastPathComponent().relativePath, foundURL.lastPathComponent]
var name: String
if try foundURL.resourceValues(forKeys: [.isSymbolicLinkKey]).isSymbolicLink! {
let isSymlink = try foundURL.resourceValues(forKeys: [.isSymbolicLinkKey]).isSymbolicLink!
if isSymlink {
name = parts.joined(separator: ":")
} else {
name = parts.joined(separator: "@")
}
result.append((name, vmDir))
result.append((name, vmDir, isSymlink))
}
return result
}
func prunables() throws -> [Prunable] {
try list().filter { (_, _, isSymlink) in !isSymlink }.map { (_, vmDir, _) in vmDir }
}
func pull(_ name: RemoteName, registry: Registry) async throws {
defaultLogger.appendNewLine("pulling manifest...")
let (manifest, _) = try await registry.pullManifest(reference: name.reference.value)
let (manifest, manifestData) = try await registry.pullManifest(reference: name.reference.value)
var digestName = RemoteName(host: name.host, namespace: name.namespace,
reference: Reference(digest: try manifest.digest()))
let digestName = RemoteName(host: name.host, namespace: name.namespace,
reference: Reference(digest: Digest.hash(manifestData)))
// Ensure that host directory for given RemoteName exists in OCI storage
let hostDirectoryURL = hostDirectoryURL(digestName)
try FileManager.default.createDirectory(at: hostDirectoryURL, withIntermediateDirectories: true)
// Acquire a lock on it to prevent concurrent pulls for a single host
let lock = try FileLock(lockURL: hostDirectoryURL)
let sucessfullyLocked = try lock.trylock()
if !sucessfullyLocked {
print("waiting for lock...")
try lock.lock()
}
defer { try! lock.unlock() }
if Task.isCancelled {
throw CancellationError()
}
if !exists(digestName) {
let tmpVMDir = try VMDirectory.temporary()
// Lock the temporary VM directory to prevent it's garbage collection
let tmpVMDirLock = try FileLock(lockURL: tmpVMDir.baseURL)
try tmpVMDirLock.lock()
// Try to reclaim some cache space if we know the VM size in advance
if let uncompressedDiskSize = manifest.uncompressedDiskSize() {
let requiredCapacityBytes = UInt64(uncompressedDiskSize + 128 * 1024 * 1024)
let attrs = try Config().tartCacheDir.resourceValues(forKeys: [.volumeAvailableCapacityForImportantUsageKey, .volumeAvailableCapacityKey])
let availableCapacityBytes = max(UInt64(attrs.volumeAvailableCapacityForImportantUsage!), UInt64(attrs.volumeAvailableCapacity!))
// There is a suspicious that occasionally capacity is returned as zero which can't be true.
// Let's validate to avoid unnecessary pruning.
if 0 < availableCapacityBytes && availableCapacityBytes < requiredCapacityBytes {
try Prune.pruneReclaim(reclaimBytes: requiredCapacityBytes - availableCapacityBytes)
}
}
try await withTaskCancellationHandler(operation: {
try await tmpVMDir.pullFromRegistry(registry: registry, manifest: manifest)
try move(digestName, from: tmpVMDir)
@@ -93,8 +179,12 @@ class VMStorageOCI {
}
if name != digestName {
// Overwrite the old symbolic link
// Create new or overwrite the old symbolic link
try link(from: digestName, to: name)
} else {
// Ensure that images pulled by content digest
// are excluded from garbage collection
VMDirectory(baseURL: vmURL(name)).markExplicitlyPulled()
}
}
@@ -104,6 +194,8 @@ class VMStorageOCI {
}
try FileManager.default.createSymbolicLink(at: vmURL(to), withDestinationURL: vmURL(from))
try gc()
}
}
@@ -117,4 +209,8 @@ extension URL {
return result
}
func appendingHost(_ name: RemoteName) -> URL {
self.appendingPathComponent(name.host, isDirectory: true)
}
}
+2 -1
View File
@@ -10,7 +10,8 @@ class ScreenSharingVNC: VNC {
}
func waitForURL() async throws -> URL {
let ip = try await IP.resolveIP(vmConfig, secondsToWait: 60)
let vmMACAddress = MACAddress(fromString: vmConfig.macAddress.string)!
let ip = try await IP.resolveIP(vmMACAddress, secondsToWait: 60)
if let ip = ip {
return URL(string: "vnc://\(ip)")!
+34
View File
@@ -0,0 +1,34 @@
import XCTest
@testable import tart
final class FileLockTests: XCTestCase {
func testSimple() throws {
// Create a temporary file that will be used as a lock
let url = temporaryFile()
// Make sure this file can be locked and unlocked
let lock = try FileLock(lockURL: url)
try lock.lock()
try lock.unlock()
}
func testDoubleLockResultsInError() throws {
// Create a temporary file that will be used as a lock
let url = temporaryFile()
// Create two locks on a same file and ensure one of them fails
let firstLock = try FileLock(lockURL: url)
try firstLock.lock()
let secondLock = try! FileLock(lockURL: url)
XCTAssertFalse(try secondLock.trylock())
}
private func temporaryFile() -> URL {
let url = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent(UUID().uuidString)
FileManager.default.createFile(atPath: url.path, contents: nil)
return url
}
}
+4 -4
View File
@@ -34,8 +34,8 @@ final class RegistryTests: XCTestCase {
// Pull it
var pulledBlob = Data()
try await registry.pullBlob(pushedBlobDigest) { buffer in
pulledBlob.append(Data(buffer: buffer))
try await registry.pullBlob(pushedBlobDigest) { data in
pulledBlob.append(data)
}
// Ensure that both blobs are identical
@@ -52,8 +52,8 @@ final class RegistryTests: XCTestCase {
// Pull it
var pulledLargeBlob = Data()
try await registry.pullBlob(largeBlobDigest) { buffer in
pulledLargeBlob.append(Data(buffer: buffer))
try await registry.pullBlob(largeBlobDigest) { data in
pulledLargeBlob.append(data)
}
// Ensure that both blobs are identical
+22
View File
@@ -0,0 +1,22 @@
import XCTest
@testable import tart
final class URLAccessDateTests: XCTestCase {
func testGetAndSetAccessTime() throws {
// Create a temporary file
let tmpDir = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true)
var tmpFile = tmpDir.appendingPathComponent(UUID().uuidString)
FileManager.default.createFile(atPath: tmpFile.path, contents: nil)
// Ensure it's access date is different than our desired access date
let arbitraryDate = Date.init(year: 2008, month: 09, day: 28, hour: 23, minute: 15)
XCTAssertNotEqual(arbitraryDate, try tmpFile.accessDate())
// Set our desired access date for a file
try tmpFile.updateAccessDate(arbitraryDate)
// Ensure the access date has changed to our value
tmpFile.removeCachedResourceValue(forKey: .contentAccessDateKey)
XCTAssertEqual(arbitraryDate, try tmpFile.accessDate())
}
}
+12
View File
@@ -0,0 +1,12 @@
source = [".build/arm64-apple-macosx/debug/tart"]
bundle_id = "com.github.cirruslabs.tart"
apple_id {
username = "hello@cirruslabs.org"
password = "@env:AC_PASSWORD"
}
sign {
application_identity = "Developer ID Application: Cirrus Labs, Inc."
entitlements_file = "Resources/tart.entitlements"
}