Compare commits

..
27 Commits
Author SHA1 Message Date
Fedor Korotkov 4bb248e7b4 Support wildcards in credHelpers (#592)
* Support wildcards in `credHelpers`

With #591 `tart pull` fails when for example you have `ecr-login` set as the default `credsStore` but you try to pull our images from `ghcr.io`.

This change reverts #591 and instead supports regex in `credHelpers`. This is not supported by Docker itself but highly demanded in https://github.com/docker/cli/issues/2928

I think it's fine to support it for Tart.

Additionally this change bumps the minimum host macOS version to Ventura in order to bring `Regex`. Yes, `Regex` only supported in Swift for macOS 13+ 🤦‍♂️I think it's fine in the light of Sonoma release and Tart 2.0.0.

* Removed Monterey mentions from docs
2023-08-28 11:37:49 -04:00
Fedor Korotkov f45551cbf0 Support Docker's credsStore (#591)
This way for #581 we don't need to specify a fully quialified URL and can simply use the following `~/.docker/config.json`:

```json
{
	"credsStore": "ecr-login"
}
```

Related to https://github.com/docker/cli/issues/2928
2023-08-21 19:44:35 +00:00
Nikolay Edigaryev f68297097e Add a simple integration test for "tart run" (#587)
* Add a simple integration test for "tart run"

* Integration tests: only "tart list" local VM images
2023-08-16 04:04:52 -04:00
Fedor Korotkov 637a2387e7 No bridged network interfaces by default (#586)
Fixes #585
2023-08-15 19:47:25 +04:00
Fedor Korotkov 050d6a6ff1 Clarify host cpu core usage (#584)
* Clarify usage of cores of the host CPU

* Updated phrasing
2023-08-15 11:25:30 +00:00
Nikolay Edigaryev 5eddd1ce41 Introduce "tart logout" command (#583)
* Introduce "tart logout"

* tart login: introduce --no-validate
2023-08-15 15:16:33 +04:00
Fedor Korotkov 35f5b30bc4 Multiple bridged interfaces (#578)
* Support multiple Bridged Network interfaces

Fixes #572

* Allow duplicated bridged interfaces
2023-08-14 19:17:38 +04:00
Fedor Korotkov 8b27fea745 Document running scripts via ssh (#579) 2023-08-14 06:45:44 -04:00
fedor e00f62c95a Bumped dotlottie file 2023-08-10 13:52:27 -04:00
TommyandFedor Korotkov d90893e9a0 Add CONTRIBUTING.md? (#575)
* Create contribute.md

* Update CONTRIBUTING.md

Co-Authored-By: Fedor Korotkov <fedor.korotkov@gmail.com>

* Update CONTRIBUTING.md

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

* Update CONTRIBUTING.md

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

* missing '''

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2023-08-03 13:10:34 -04:00
TommyandFedor Korotkov feb733a7c0 GC avoidance and tmpDeterminstic (#570)
* GC avoidance and tmpDeterminstic

* change tmpDeterministic to use hashing

- temporaryDeterministic() now takes in a key and hashes it
- creates directory with the hash

* Update Sources/tart/VMStorageOCI.swift

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2023-07-31 18:21:42 +00:00
Fedor Korotkov 545b6fcd94 Provide license usage examples (#568) 2023-07-27 12:06:40 -04:00
Nikolay Edigaryev c750d63ac9 Clarify licensing (#566)
* Clarify licensing

* Refactor "License Tiers" section
2023-07-26 14:50:44 +00:00
Fedor Korotkov 704811e671 Introduce TART_NO_AUTO_PRUNE (#565)
* Introduce `TART_NO_AUTO_PRUNE`

Similar to `HOMEBREW_NO_AUTO_UPDATE`

* Self review
2023-07-25 16:01:53 +00:00
Fedor Korotkov d4fcecd47c [skip ci] Downgrade Lottie Player (#563)
Seems in 2.0.0 release they broke looping. Right now animation on https://tart.run/ is always looping even without `loop` property. I was able to disable it, so I just downgraded to the last known working version.
2023-07-20 19:46:58 +04:00
Nikolay Edigaryev 33ca96e1a0 Retrieve VM's IP for use in VNC after the VM is started #2 (#562) 2023-07-17 12:51:43 +04:00
Fedor Korotkov 4ce06279ff [skip ci] Clarify default runner group assigment (#559) 2023-07-14 07:50:31 -04:00
Fedor Korotkov fa97adfc9e Highlight Cirrus Runners in README.md (#558)
* Mention Cirrus Runners in the README.md

* Increase font size

* typo

* Fixed branch
2023-07-13 18:09:19 +00:00
Nikolay Edigaryev 6c377029d6 tart prune: allow pruning local VMs with --entries=vms (#557) 2023-07-13 22:05:28 +04:00
Fedor Korotkov 93a1b70ecb Don't delete an initiator of pruning (#556)
* Don't delete an initiator of pruning

Sometimes people have an image that is greater than half of the disk itself. In that case such image will be pulled and prunned right away.

This change makes sure that an image that is being cloned from is not pruned right away.

* Resolve symbolic links
2023-07-13 13:59:23 +04:00
Fedor Korotkov cf9a3a9221 Allow mounting a single directory without a name (#555)
* Allow mounting a single directory without a name

To utilize `VZSingleDirectoryShare` which seems more stable than `VZMultipleDirectoryShare`.

We've been having reports from users that mounted directories occasionally return "no such file" errors when building large projects. I took a stab at reproducing the issue by running https://github.com/devMEremenko/XcodeBenchmark in a mounted directory:

```bash
tart run --dir=workdir-test:~/workspace-temp/XcodeBenchmark ventura-xcode
```

And I was able to reproduce the "no such file" error on the first try! After looking into the issue I decided to try `VZSingleDirectoryShare` as this PR changes and to my pleasant surprise it all worked like a charm the next run. So it seems there is a bug in `VZMultipleDirectoryShare` integration with virtiofs. Since in most cases users only mount a single directory it makes sense to allow doing it wihtout providing a `name`.

So now it will be possible to run the following command:

```bash
tart run --dir=~/workspace-temp/XcodeBenchmark ventura-xcode
```

Which will make `~/workspace-temp/XcodeBenchmark` available under `/Volumes/My Shared Files/` without any intermediate directories.

* Reformat

* Updated description
2023-07-12 19:08:52 +00:00
Stanisław Chmiela ad566faa23 Add OCI annotation with upload time (#551) 2023-07-11 19:02:38 -04:00
Fedor Korotkov 1afaa7ec7a Reclaim disk space only in the very end of tart clone (#553)
* Reclaim disk space only in the very end of `tart clone`

Please check comments in code for reasoning.

* Phrasing

* Removed hidden argument
2023-07-11 15:49:43 +00:00
Fedor Korotkov 826e508646 Upload debug files to both Sentry projects (#552)
* Upload debug files to both Sentry projects

Apparently it's not possible to share debug files cross projects. So let's upload to both of them. To one we use in production and one we use for testing.

* Use environment variable
2023-07-11 17:49:04 +04:00
Stanisław ChmielaandFedor Korotkov 8653ca4115 Fix prune --older-than deleting all cache (#549)
* Fix `prune --older-than`

* Update Sources/tart/Commands/Prune.swift

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2023-07-11 16:10:49 +04:00
Nikolay Edigaryev 63b74f407b Version console device (#546) 2023-07-10 15:23:50 +00:00
Nikolay Edigaryev 3a2cba6929 Introduce TART_REGISTRY_HOSTNAME (#545)
* Introduce TART_REGISTRY_HOST

* TART_REGISTRY_HOST → TART_REGISTRY_HOSTNAME
2023-07-10 18:19:21 +04:00
44 changed files with 540 additions and 185 deletions
+3 -1
View File
@@ -1,7 +1,7 @@
use_compute_credits: true use_compute_credits: true
env: env:
XCODE_TAG: 15-beta-2 XCODE_TAG: 15-beta-5
task: task:
name: Test on Ventura name: Test on Ventura
@@ -128,9 +128,11 @@ task:
# Generate and upload symbols # Generate and upload symbols
- dsymutil tart - dsymutil tart
- sentry-cli debug-files upload tart.dSYM/ - sentry-cli debug-files upload tart.dSYM/
- SENTRY_PROJECT=tart sentry-cli debug-files upload tart.dSYM/
# Bundle and upload sources # Bundle and upload sources
- sentry-cli debug-files bundle-sources tart.dSYM - sentry-cli debug-files bundle-sources tart.dSYM
- sentry-cli debug-files upload tart.src.zip - sentry-cli debug-files upload tart.src.zip
- SENTRY_PROJECT=tart sentry-cli debug-files upload tart.src.zip
create_sentry_release_script: create_sentry_release_script:
- export SENTRY_RELEASE="tart@$CIRRUS_TAG" - export SENTRY_RELEASE="tart@$CIRRUS_TAG"
- sentry-cli releases new $SENTRY_RELEASE - sentry-cli releases new $SENTRY_RELEASE
+1 -1
View File
@@ -45,7 +45,7 @@ brews:
libexec.install Dir["*"] libexec.install Dir["*"]
bin.write_exec_script "#{libexec}/tart.app/Contents/MacOS/tart" bin.write_exec_script "#{libexec}/tart.app/Contents/MacOS/tart"
custom_block: | custom_block: |
depends_on :macos => :monterey depends_on :macos => :ventura
on_macos do on_macos do
unless Hardware::CPU.arm? unless Hardware::CPU.arm?
+40
View File
@@ -0,0 +1,40 @@
# Contributing to Tart
Table of Contents
-----------------
- [How to Build](#how-to-build)
- [How to Create an Issue/Enhancement](#how-to-create-an-issueenhancement)
- [Style Guidelines](#style-guidelines)
- [Pull Requests](#Pull-Requests)
## How to Build
1. Fork the repository to your own GitHub account
2. Clone the forked repository to your local machine
3. If using Xcode, use from Xcode 15 or newer
4. Run ./scripts/run-signed.sh from the root of your repository
```bash
./scripts/run-signed.sh list
```
## How to Create an Issue/Enhancement
1. Go to the [Issue page](https://github.com/cirruslabs/tart/issues) of the repository
2. Click on the "New Issue" button
3. Provide a descriptive title and detailed description of the issue or enhancement you're suggesting
4. Submit the issue
## Style Guidelines
1. Code should follow camel case
2. Code should follow [SwiftFormat](https://github.com/nicklockwood/SwiftFormat#swift-package-manager-plugin) guidelines. You can auto-format the code by running the following command:
```bash
swift package plugin --allow-writing-to-package-directory swiftformat --cache ignore .
```
## Pull Requests
1. Provide a detailed description of the changes you made in the pull request
2. Wait for pull request to be reviewed
3. Make adjustments if necessary
+1 -1
View File
@@ -4,7 +4,7 @@ import PackageDescription
let package = Package( let package = Package(
name: "Tart", name: "Tart",
platforms: [ platforms: [
.macOS(.v12) .macOS(.v13)
], ],
products: [ products: [
.executable(name: "tart", targets: ["tart"]) .executable(name: "tart", targets: ["tart"])
+12 -3
View File
@@ -6,9 +6,18 @@ Built by CI engineers for your automation needs. Here are some highlights of Tar
* Tart uses Apple's own `Virtualization.Framework` for [near-native performance](https://browser.geekbench.com/v5/cpu/compare/20382844?baseline=20382722). * Tart uses Apple's own `Virtualization.Framework` for [near-native performance](https://browser.geekbench.com/v5/cpu/compare/20382844?baseline=20382722).
* Push/Pull virtual machines from any OCI-compatible container registry. * Push/Pull virtual machines from any OCI-compatible container registry.
* Use Tart Packer Plugin to automate VM creation. * Use Tart Packer Plugin to automate VM creation.
* Built-in CI integration. * Easily integrates with any CI system.
*Tart* is already adopted by several automation services: Tart powers [Cirrus Runners](https://tart.run/integrations/github-actions/?utm_source=github&utm_medium=referral)
service — a drop-in replacement for the standard GitHub-hosted runners, offering 2-3 times better performance for a fraction of the price.
<p align="center">
<a href="https://tart.run/integrations/github-actions/?utm_source=github&utm_medium=referral" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/CirrusRunnersForGHA.png" height="65"/>
</a>
</p>
Tart is also adopted by several other automation services:
<p align="center"> <p align="center">
<a href="https://cirrus-ci.org/guide/macOS/" target=_blank> <a href="https://cirrus-ci.org/guide/macOS/" target=_blank>
@@ -52,7 +61,7 @@ Many more companies are using Tart in their internal setups. Here are a few of t
## Usage ## Usage
Try running a Tart VM on your Apple Silicon device running macOS 12.0 (Monterey) or later (will download a 25 GB image): Try running a Tart VM on your Apple Silicon device running macOS 13.0 (Ventura) or later (will download a 25 GB image):
```bash ```bash
brew install cirruslabs/cli/tart brew install cirruslabs/cli/tart
Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

+20 -4
View File
@@ -3,7 +3,20 @@ import Foundation
import SystemConfiguration import SystemConfiguration
struct Clone: AsyncParsableCommand { struct Clone: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Clone a VM") static var configuration = CommandConfiguration(
abstract: "Clone a VM",
discussion: """
Creates a local virtual machine by cloning either a remote or another local virtual machine.
Due to copy-on-write magic in Apple File System a cloned VM won't actually claim all the space right away.
Only changes to a cloned disk will be written and claim new space. By default, Tart checks available capacity
in Tart's home directory and checks if there is enough space for the worst possible scenario: when the whole disk
will be modified.
This behaviour can be disabled by setting TART_NO_AUTO_PRUNE environment variable. This might be helpful
for use cases when the original image is very big and a workload is known to only modify a fraction of the cloned disk.
"""
)
@Argument(help: "source VM name") @Argument(help: "source VM name")
var sourceName: String var sourceName: String
@@ -31,8 +44,6 @@ struct Clone: AsyncParsableCommand {
} }
let sourceVM = try VMStorageHelper.open(sourceName) let sourceVM = try VMStorageHelper.open(sourceName)
try Prune.reclaimIfNeeded(UInt64(sourceVM.sizeBytes()))
let tmpVMDir = try VMDirectory.temporary() let tmpVMDir = try VMDirectory.temporary()
// Lock the temporary VM directory to prevent it's garbage collection // Lock the temporary VM directory to prevent it's garbage collection
@@ -44,13 +55,18 @@ struct Clone: AsyncParsableCommand {
let lock = try FileLock(lockURL: Config().tartHomeDir) let lock = try FileLock(lockURL: Config().tartHomeDir)
try lock.lock() try lock.lock()
let generateMAC = try localStorage.hasVMsWithMACAddress(macAddress: sourceVM.macAddress()) let generateMAC = try localStorage.hasVMsWithMACAddress(macAddress: sourceVM.macAddress())
&& sourceVM.state() != "suspended" && sourceVM.state() != "suspended"
try sourceVM.clone(to: tmpVMDir, generateMAC: generateMAC) try sourceVM.clone(to: tmpVMDir, generateMAC: generateMAC)
try localStorage.move(newName, from: tmpVMDir) try localStorage.move(newName, from: tmpVMDir)
try lock.unlock() try lock.unlock()
// APFS is doing copy-on-write so the above cloning operation (just copying files on disk)
// is not actually claiming new space until the VM is started and it writes something to disk.
// So once we clone the VM let's try to claim a little bit of space for the VM to run.
try Prune.reclaimIfNeeded(UInt64(sourceVM.sizeBytes()), sourceVM)
}, onCancel: { }, onCancel: {
try? FileManager.default.removeItem(at: tmpVMDir.baseURL) try? FileManager.default.removeItem(at: tmpVMDir.baseURL)
}) })
+1 -5
View File
@@ -47,11 +47,7 @@ struct Create: AsyncParsableCommand {
} }
if linux { if linux {
if #available(macOS 13, *) { _ = try await VM.linux(vmDir: tmpVMDir, diskSizeGB: diskSize)
_ = try await VM.linux(vmDir: tmpVMDir, diskSizeGB: diskSize)
} else {
throw UnsupportedOSError("Linux VMs", "are")
}
} }
try VMStorageLocal().move(name, from: tmpVMDir) try VMStorageLocal().move(name, from: tmpVMDir)
+11 -6
View File
@@ -17,6 +17,9 @@ struct Login: AsyncParsableCommand {
@Flag(help: "connect to the OCI registry via insecure HTTP protocol") @Flag(help: "connect to the OCI registry via insecure HTTP protocol")
var insecure: Bool = false var insecure: Bool = false
@Flag(help: "skip validation of the registry's credentials before logging-in")
var noValidate: Bool = false
func validate() throws { func validate() throws {
let usernameProvided = username != nil let usernameProvided = username != nil
let passwordProvided = passwordStdin let passwordProvided = passwordStdin
@@ -45,12 +48,14 @@ struct Login: AsyncParsableCommand {
host: (user, password) host: (user, password)
]) ])
do { if !noValidate {
let registry = try Registry(host: host, namespace: "", insecure: insecure, do {
credentialsProviders: [credentialsProvider]) let registry = try Registry(host: host, namespace: "", insecure: insecure,
try await registry.ping() credentialsProviders: [credentialsProvider])
} catch { try await registry.ping()
throw RuntimeError.InvalidCredentials("invalid credentials: \(error)") } catch {
throw RuntimeError.InvalidCredentials("invalid credentials: \(error)")
}
} }
try KeychainCredentialsProvider().store(host: host, user: user, password: password) try KeychainCredentialsProvider().store(host: host, user: user, password: password)
+14
View File
@@ -0,0 +1,14 @@
import ArgumentParser
import Dispatch
import SwiftUI
struct Logout: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Logout from a registry")
@Argument(help: "host")
var host: String
func run() async throws {
try KeychainCredentialsProvider().remove(host: host)
}
}
+57 -21
View File
@@ -5,23 +5,40 @@ import SwiftUI
import SwiftDate import SwiftDate
struct Prune: AsyncParsableCommand { struct Prune: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Prune OCI and IPSW caches") static var configuration = CommandConfiguration(abstract: "Prune OCI and IPSW caches or local VMs")
@Option(help: ArgumentHelp("Remove cache entries last accessed more than n days ago", @Option(help: ArgumentHelp("Entries to remove: \"caches\" targets OCI and IPSW caches and \"vms\" targets local VMs."))
var entries: String = "caches"
@Option(help: ArgumentHelp("Remove entries that were last accessed more than n days ago",
discussion: "For example, --older-than=7 will remove entries that weren't accessed by Tart in the last 7 days.", discussion: "For example, --older-than=7 will remove entries that weren't accessed by Tart in the last 7 days.",
valueName: "n")) valueName: "n"))
var olderThan: UInt? var olderThan: UInt?
@Option(help: ArgumentHelp("Remove least recently used cache entries that do not fit the specified cache size budget n, expressed in gigabytes", @Option(help: .hidden)
discussion: "For example, --cache-budget=50 will effectively shrink all caches to a total size of 50 gigabytes.",
valueName: "n"))
var cacheBudget: UInt? var cacheBudget: UInt?
@Option(help: ArgumentHelp("Remove the least recently used entries that do not fit the specified space size budget n, expressed in gigabytes",
discussion: "For example, --space-budget=50 will effectively shrink all entries to a total size of 50 gigabytes.",
valueName: "n"))
var spaceBudget: UInt?
@Flag(help: .hidden) @Flag(help: .hidden)
var gc: Bool = false var gc: Bool = false
func validate() throws { mutating func validate() throws {
if olderThan == nil && cacheBudget == nil && !gc { // --cache-budget deprecation logic
if let cacheBudget = cacheBudget {
fputs("--cache-budget is deprecated, please use --space-budget\n", stderr)
if spaceBudget != nil {
throw ValidationError("--cache-budget is deprecated, please use --space-budget")
}
spaceBudget = cacheBudget
}
if olderThan == nil && spaceBudget == nil && !gc {
throw ValidationError("at least one pruning criteria must be specified") throw ValidationError("at least one pruning criteria must be specified")
} }
} }
@@ -31,43 +48,53 @@ struct Prune: AsyncParsableCommand {
try VMStorageOCI().gc() try VMStorageOCI().gc()
} }
// Build a list of prunable storages that we're going to prune based on user's request
let prunableStorages: [PrunableStorage]
switch entries {
case "caches":
prunableStorages = [VMStorageOCI(), try IPSWCache()]
case "vms":
prunableStorages = [VMStorageLocal()]
default:
throw ValidationError("unsupported --entries value, please specify either \"caches\" or \"vms\"")
}
// Clean up cache entries based on last accessed date // Clean up cache entries based on last accessed date
if let olderThan = olderThan { if let olderThan = olderThan {
let olderThanInterval = Int(exactly: olderThan)!.days.timeInterval let olderThanInterval = Int(exactly: olderThan)!.days.timeInterval
let olderThanDate = Date().addingTimeInterval(olderThanInterval) let olderThanDate = Date() - olderThanInterval
try Prune.pruneOlderThan(olderThanDate: olderThanDate) try Prune.pruneOlderThan(prunableStorages: prunableStorages, olderThanDate: olderThanDate)
} }
// Clean up cache entries based on imposed cache size limit and entry's last accessed date // Clean up cache entries based on imposed cache size limit and entry's last accessed date
if let cacheBudget = cacheBudget { if let spaceBudget = spaceBudget {
try Prune.pruneCacheBudget(cacheBudgetBytes: UInt64(cacheBudget) * 1024 * 1024 * 1024) try Prune.pruneSpaceBudget(prunableStorages: prunableStorages, spaceBudgetBytes: UInt64(spaceBudget) * 1024 * 1024 * 1024)
} }
} }
static func pruneOlderThan(olderThanDate: Date) throws { static func pruneOlderThan(prunableStorages: [PrunableStorage], olderThanDate: Date) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages.flatMap { try $0.prunables() } let prunables: [Prunable] = try prunableStorages.flatMap { try $0.prunables() }
try prunables.filter { try $0.accessDate() <= olderThanDate }.forEach { try $0.delete() } try prunables.filter { try $0.accessDate() <= olderThanDate }.forEach { try $0.delete() }
} }
static func pruneCacheBudget(cacheBudgetBytes: UInt64) throws { static func pruneSpaceBudget(prunableStorages: [PrunableStorage], spaceBudgetBytes: UInt64) throws {
let prunableStorages: [PrunableStorage] = [VMStorageOCI(), try IPSWCache()]
let prunables: [Prunable] = try prunableStorages let prunables: [Prunable] = try prunableStorages
.flatMap { try $0.prunables() } .flatMap { try $0.prunables() }
.sorted { try $0.accessDate() > $1.accessDate() } .sorted { try $0.accessDate() > $1.accessDate() }
var cacheBudgetBytes = cacheBudgetBytes var spaceBudgetBytes = spaceBudgetBytes
var prunablesToDelete: [Prunable] = [] var prunablesToDelete: [Prunable] = []
for prunable in prunables { for prunable in prunables {
let prunableSizeBytes = UInt64(try prunable.sizeBytes()) let prunableSizeBytes = UInt64(try prunable.sizeBytes())
if prunableSizeBytes <= cacheBudgetBytes { if prunableSizeBytes <= spaceBudgetBytes {
// Don't mark for deletion as // Don't mark for deletion as
// there's a budget available // there's a budget available
cacheBudgetBytes -= prunableSizeBytes spaceBudgetBytes -= prunableSizeBytes
} else { } else {
// Mark for deletion // Mark for deletion
prunablesToDelete.append(prunable) prunablesToDelete.append(prunable)
@@ -77,7 +104,11 @@ struct Prune: AsyncParsableCommand {
try prunablesToDelete.forEach { try $0.delete() } try prunablesToDelete.forEach { try $0.delete() }
} }
static func reclaimIfNeeded(_ requiredBytes: UInt64) throws { static func reclaimIfNeeded(_ requiredBytes: UInt64, _ initiator: Prunable? = nil) throws {
if ProcessInfo.processInfo.environment.keys.contains("TART_NO_AUTO_PRUNE") {
return
}
SentrySDK.configureScope { scope in SentrySDK.configureScope { scope in
scope.setContext(value: ["requiredBytes": requiredBytes], key: "Prune") scope.setContext(value: ["requiredBytes": requiredBytes], key: "Prune")
} }
@@ -114,10 +145,10 @@ struct Prune: AsyncParsableCommand {
return return
} }
try Prune.reclaimIfPossible(requiredBytes - volumeAvailableCapacityCalculated) try Prune.reclaimIfPossible(requiredBytes - volumeAvailableCapacityCalculated, initiator)
} }
private static func reclaimIfPossible(_ reclaimBytes: UInt64) throws { private static func reclaimIfPossible(_ reclaimBytes: UInt64, _ initiator: Prunable? = nil) throws {
let transaction = SentrySDK.startTransaction(name: "Pruning cache", operation: "prune", bindToScope: true) let transaction = SentrySDK.startTransaction(name: "Pruning cache", operation: "prune", bindToScope: true)
defer { transaction.finish() } defer { transaction.finish() }
@@ -141,6 +172,11 @@ struct Prune: AsyncParsableCommand {
break break
} }
if prunable.url == initiator?.url.resolvingSymlinksInPath() {
// do not prune the initiator
continue
}
try SentrySDK.span?.setData(value: prunable.sizeBytes(), key: prunable.url.path) try SentrySDK.span?.setData(value: prunable.sizeBytes(), key: prunable.url.path)
cacheReclaimedBytes += try prunable.sizeBytes() cacheReclaimedBytes += try prunable.sizeBytes()
+10 -1
View File
@@ -3,7 +3,16 @@ import Dispatch
import SwiftUI import SwiftUI
struct Pull: AsyncParsableCommand { struct Pull: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Pull a VM from a registry") static var configuration = CommandConfiguration(
abstract: "Pull a VM from a registry",
discussion: """
Pulls a virtual machine from a remote OCI-compatible registry. Supports authorization via Keychain (see "tart login --help"),
Docker credential helpers defined in ~/.docker/config.json or via TART_REGISTRY_USERNAME/TART_REGISTRY_PASSWORD environment variables.
By default, Tart checks available capacity in Tart's home directory and tries to reclaim minimum possible storage for the remote image to fit via "tart prune".
This behaviour can be disabled by setting TART_NO_AUTO_PRUNE environment variable.
"""
)
@Argument(help: "remote VM name") @Argument(help: "remote VM name")
var remoteName: String var remoteName: String
+90 -63
View File
@@ -73,13 +73,17 @@ struct Run: AsyncParsableCommand {
var rosettaTag: String? var rosettaTag: String?
@Option(help: ArgumentHelp(""" @Option(help: ArgumentHelp("""
Additional directory shares with an optional read-only specifier\n(e.g. --dir=\"build:~/src/build\" --dir=\"sources:~/src/sources:ro\") Additional directory shares with an optional read-only specifier\n(e.g. --dir=\"~/src/build\" or --dir=\"~/src/sources:ro\")
""", discussion: """ """, discussion: """
Requires host to be macOS 13.0 (Ventura) or newer. Requires host to be macOS 13.0 (Ventura) or newer.
All shared directories are automatically mounted to "/Volumes/My Shared Files" directory on macOS, A shared directory is automatically mounted to "/Volumes/My Shared Files" directory on macOS,
while on Linux you have to do it manually: "mount -t virtiofs com.apple.virtio-fs.automount /mount/point". while on Linux you have to do it manually: "mount -t virtiofs com.apple.virtio-fs.automount /mount/point".
For macOS guests, they must be running macOS 13.0 (Ventura) or newer. For macOS guests, they must be running macOS 13.0 (Ventura) or newer.
""", valueName: "name:path[:ro]"))
In case of passing multiple directories it is required to prefix them with names e.g. --dir=\"build:~/src/build\" --dir=\"sources:~/src/sources:ro\"
These names will be used as directory names under the mounting point inside guests. For the example above it will be
"/Volumes/My Shared Files/build" and "/Volumes/My Shared Files/sources" respectively.
""", valueName: "[name:]path[:ro]"))
var dir: [String] = [] var dir: [String] = []
@Option(help: ArgumentHelp(""" @Option(help: ArgumentHelp("""
@@ -87,7 +91,7 @@ struct Run: AsyncParsableCommand {
""", discussion: """ """, discussion: """
Specify "list" as an interface name (--net-bridged=list) to list the available bridged interfaces. Specify "list" as an interface name (--net-bridged=list) to list the available bridged interfaces.
""", valueName: "interface name")) """, valueName: "interface name"))
var netBridged: String? var netBridged: [String] = []
@Flag(help: ArgumentHelp("Use software networking instead of the default shared (NAT) networking", @Flag(help: ArgumentHelp("Use software networking instead of the default shared (NAT) networking",
discussion: "Learn how to configure Softnet for use with Tart here: https://github.com/cirruslabs/softnet")) discussion: "Learn how to configure Softnet for use with Tart here: https://github.com/cirruslabs/softnet"))
@@ -101,7 +105,7 @@ struct Run: AsyncParsableCommand {
throw ValidationError("--vnc and --vnc-experimental are mutually exclusive") throw ValidationError("--vnc and --vnc-experimental are mutually exclusive")
} }
if netBridged != nil && netSoftnet { if netBridged.count > 0 && netSoftnet {
throw ValidationError("--net-bridged and --net-softnet are mutually exclusive") throw ValidationError("--net-bridged and --net-softnet are mutually exclusive")
} }
@@ -216,17 +220,6 @@ struct Run: AsyncParsableCommand {
let task = Task { let task = Task {
do { do {
if let vncImpl = vncImpl {
let vncURL = try await vncImpl.waitForURL()
if noGraphics || ProcessInfo.processInfo.environment["CI"] != nil {
print("VNC server is running at \(vncURL)")
} else {
print("Opening \(vncURL)...")
NSWorkspace.shared.open(vncURL)
}
}
var resume = false var resume = false
if #available(macOS 14, *) { if #available(macOS 14, *) {
@@ -239,7 +232,20 @@ struct Run: AsyncParsableCommand {
} }
} }
try await vm!.run(recovery: recovery, resume: resume) try await vm!.start(recovery: recovery, resume: resume)
if let vncImpl = vncImpl {
let vncURL = try await vncImpl.waitForURL()
if noGraphics || ProcessInfo.processInfo.environment["CI"] != nil {
print("VNC server is running at \(vncURL)")
} else {
print("Opening \(vncURL)...")
NSWorkspace.shared.open(vncURL)
}
}
try await vm!.run()
if let vncImpl = vncImpl { if let vncImpl = vncImpl {
try vncImpl.stop() try vncImpl.stop()
@@ -325,23 +331,19 @@ struct Run: AsyncParsableCommand {
return try Softnet(vmMACAddress: config.macAddress.string) return try Softnet(vmMACAddress: config.macAddress.string)
} }
if let netBridged = netBridged { if netBridged.count > 0 {
let matchingInterfaces = VZBridgedNetworkInterface.networkInterfaces.filter { interface in func findBridgedInterface(_ name: String) throws -> VZBridgedNetworkInterface {
interface.identifier == netBridged || interface.localizedDisplayName == netBridged let interface = VZBridgedNetworkInterface.networkInterfaces.first { interface in
interface.identifier == name || interface.localizedDisplayName == name
}
if (interface == nil) {
throw ValidationError("no bridge interfaces matched \"\(netBridged)\", "
+ "available interfaces: \(bridgeInterfaces())")
}
return interface!
} }
if matchingInterfaces.isEmpty { return NetworkBridged(interfaces: try netBridged.map { try findBridgedInterface($0) })
let available = bridgeInterfaces().joined(separator: ", ")
throw ValidationError("no bridge interfaces matched \"\(netBridged)\", "
+ "available interfaces: \(available)")
}
if matchingInterfaces.count > 1 {
throw ValidationError("more than one bridge interface matched \"\(netBridged)\", "
+ "consider refining the search criteria")
}
return NetworkBridged(interface: matchingInterfaces.first!)
} }
return nil return nil
@@ -390,46 +392,31 @@ struct Run: AsyncParsableCommand {
throw UnsupportedOSError("directory sharing", "is") throw UnsupportedOSError("directory sharing", "is")
} }
struct DirectoryShare {
let name: String
let path: URL
let readOnly: Bool
}
var directoryShares: [DirectoryShare] = [] var directoryShares: [DirectoryShare] = []
var allNamedShares = true
for rawDir in dir { for rawDir in dir {
let splits = rawDir.split(maxSplits: 2) { $0 == ":" } let directoryShare = try DirectoryShare(parseFrom: rawDir)
if (directoryShare.name == nil) {
if splits.count < 2 { allNamedShares = false
throw ValidationError("invalid --dir syntax: should at least include name and path, colon-separated")
} }
directoryShares.append(directoryShare)
var readOnly: Bool = false
if splits.count == 3 {
if splits[2] == "ro" {
readOnly = true
} else {
throw ValidationError("invalid --dir syntax: optional read-only specifier can only be \"ro\"")
}
}
let (name, path) = (String(splits[0]), String(splits[1]))
directoryShares.append(DirectoryShare(
name: name,
path: URL(fileURLWithPath: NSString(string: path).expandingTildeInPath),
readOnly: readOnly)
)
} }
var directories: [String : VZSharedDirectory] = Dictionary()
directoryShares.forEach { directories[$0.name] = VZSharedDirectory(url: $0.path, readOnly: $0.readOnly) }
let automountTag = VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag let automountTag = VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag
let sharingDevice = VZVirtioFileSystemDeviceConfiguration(tag: automountTag) let sharingDevice = VZVirtioFileSystemDeviceConfiguration(tag: automountTag)
sharingDevice.share = VZMultipleDirectoryShare(directories: directories) if allNamedShares {
var directories: [String : VZSharedDirectory] = Dictionary()
directoryShares.forEach { directories[$0.name!] = VZSharedDirectory(url: $0.path, readOnly: $0.readOnly) }
sharingDevice.share = VZMultipleDirectoryShare(directories: directories)
} else if dir.count > 1 {
throw ValidationError("invalid --dir syntax: for multiple directory shares each one of them should be named")
} else if dir.count == 1 {
let directoryShare = directoryShares.first!
let singleDirectoryShare = VZSingleDirectoryShare(directory: VZSharedDirectory(url: directoryShare.path, readOnly: directoryShare.readOnly))
sharingDevice.share = singleDirectoryShare
}
return [sharingDevice] return [sharingDevice]
} }
@@ -595,3 +582,43 @@ struct VMView: NSViewRepresentable {
nsView.virtualMachine = vm.virtualMachine nsView.virtualMachine = vm.virtualMachine
} }
} }
struct DirectoryShare {
let name: String?
let path: URL
let readOnly: Bool
init(parseFrom: String) throws {
let splits = parseFrom.split(maxSplits: 2) { $0 == ":" }
if splits.count == 3 {
if splits[2] == "ro" {
readOnly = true
} else {
throw ValidationError("invalid --dir syntax: optional read-only specifier can only be \"ro\"")
}
name = String(splits[0])
path = String(splits[1]).toFilePathURL()
} else if splits.count == 2 {
if splits[1] == "ro" {
name = nil
path = String(splits[0]).toFilePathURL()
readOnly = true
} else {
name = String(splits[0])
path = String(splits[1]).toFilePathURL()
readOnly = false
}
} else {
name = nil
path = String(splits[0]).toFilePathURL()
readOnly = false
}
}
}
extension String {
func toFilePathURL() -> URL {
URL(fileURLWithPath: NSString(string: self).expandingTildeInPath)
}
}
@@ -11,7 +11,7 @@ class DockerConfigCredentialsProvider: CredentialsProvider {
if let credentialsFromAuth = config.auths?[host]?.decodeCredentials() { if let credentialsFromAuth = config.auths?[host]?.decodeCredentials() {
return credentialsFromAuth return credentialsFromAuth
} }
if let helperProgram = config.credHelpers?[host] { if let helperProgram = try config.findCredHelper(host: host) {
return try executeHelper(binaryName: "docker-credential-\(helperProgram)", host: host) return try executeHelper(binaryName: "docker-credential-\(helperProgram)", host: host)
} }
@@ -59,6 +59,26 @@ class DockerConfigCredentialsProvider: CredentialsProvider {
struct DockerConfig: Codable { struct DockerConfig: Codable {
var auths: Dictionary<String, DockerAuthConfig>? = Dictionary() var auths: Dictionary<String, DockerAuthConfig>? = Dictionary()
var credHelpers: Dictionary<String, String>? = Dictionary() var credHelpers: Dictionary<String, String>? = Dictionary()
func findCredHelper(host: String) throws -> String? {
// Tart supports wildcards in credHelpers
// Similar to what is requested from Docker: https://github.com/docker/cli/issues/2928
guard let credHelpers else {
return nil
}
for (hostPattern, helperProgram) in credHelpers {
if (hostPattern == host) {
return helperProgram
}
let compiledPattern = try? Regex(hostPattern)
if (try compiledPattern?.wholeMatch(in: host) != nil) {
return helperProgram
}
}
return nil
}
} }
struct DockerAuthConfig: Codable { struct DockerAuthConfig: Codable {
@@ -2,6 +2,11 @@ import Foundation
class EnvironmentCredentialsProvider: CredentialsProvider { class EnvironmentCredentialsProvider: CredentialsProvider {
func retrieve(host: String) throws -> (String, String)? { func retrieve(host: String) throws -> (String, String)? {
if let tartRegistryHostname = ProcessInfo.processInfo.environment["TART_REGISTRY_HOSTNAME"],
tartRegistryHostname != host {
return nil
}
let username = ProcessInfo.processInfo.environment["TART_REGISTRY_USERNAME"] let username = ProcessInfo.processInfo.environment["TART_REGISTRY_USERNAME"]
let password = ProcessInfo.processInfo.environment["TART_REGISTRY_PASSWORD"] let password = ProcessInfo.processInfo.environment["TART_REGISTRY_PASSWORD"]
if let username = username, let password = password { if let username = username, let password = password {
@@ -61,6 +61,24 @@ class KeychainCredentialsProvider: CredentialsProvider {
throw CredentialsProviderError.Failed(message: "Keychain failed to find item: \(status.explanation())") throw CredentialsProviderError.Failed(message: "Keychain failed to find item: \(status.explanation())")
} }
} }
func remove(host: String) throws {
let query: [String: Any] = [kSecClass as String: kSecClassInternetPassword,
kSecAttrServer as String: host,
kSecAttrLabel as String: "Tart Credentials",
]
let status = SecItemDelete(query as CFDictionary)
switch status {
case errSecSuccess:
return
case errSecItemNotFound:
return
default:
throw CredentialsProviderError.Failed(message: "Failed to remove Keychain item(s): \(status.explanation())")
}
}
} }
extension OSStatus { extension OSStatus {
+1 -1
View File
@@ -1,7 +1,7 @@
import Virtualization import Virtualization
protocol Network { protocol Network {
func attachment() -> VZNetworkDeviceAttachment func attachments() -> [VZNetworkDeviceAttachment]
func run(_ sema: DispatchSemaphore) throws func run(_ sema: DispatchSemaphore) throws
func stop() async throws func stop() async throws
} }
+5 -5
View File
@@ -2,14 +2,14 @@ import Foundation
import Virtualization import Virtualization
class NetworkBridged: Network { class NetworkBridged: Network {
let interface: VZBridgedNetworkInterface let interfaces: [VZBridgedNetworkInterface]
init(interface: VZBridgedNetworkInterface) { init(interfaces: [VZBridgedNetworkInterface]) {
self.interface = interface self.interfaces = interfaces
} }
func attachment() -> VZNetworkDeviceAttachment { func attachments() -> [VZNetworkDeviceAttachment] {
VZBridgedNetworkDeviceAttachment(interface: interface) interfaces.map { VZBridgedNetworkDeviceAttachment(interface: $0) }
} }
func run(_ sema: DispatchSemaphore) throws { func run(_ sema: DispatchSemaphore) throws {
+2 -2
View File
@@ -2,8 +2,8 @@ import Foundation
import Virtualization import Virtualization
class NetworkShared: Network { class NetworkShared: Network {
func attachment() -> VZNetworkDeviceAttachment { func attachments() -> [VZNetworkDeviceAttachment] {
VZNATNetworkDeviceAttachment() [VZNATNetworkDeviceAttachment()]
} }
func run(_ sema: DispatchSemaphore) throws { func run(_ sema: DispatchSemaphore) throws {
+2 -2
View File
@@ -92,9 +92,9 @@ class Softnet: Network {
} }
} }
func attachment() -> VZNetworkDeviceAttachment { func attachments() -> [VZNetworkDeviceAttachment] {
let fh = FileHandle.init(fileDescriptor: vmFD) let fh = FileHandle.init(fileDescriptor: vmFD)
return VZFileHandleNetworkDeviceAttachment(fileHandle: fh) return [VZFileHandleNetworkDeviceAttachment(fileHandle: fh)]
} }
static func configureSUIDBitIfNeeded() throws { static func configureSUIDBitIfNeeded() throws {
+11 -4
View File
@@ -5,6 +5,7 @@ let ociConfigMediaType = "application/vnd.oci.image.config.v1+json"
// Annotations // Annotations
let uncompressedDiskSizeAnnotation = "org.cirruslabs.tart.uncompressed-disk-size" let uncompressedDiskSizeAnnotation = "org.cirruslabs.tart.uncompressed-disk-size"
let uploadTimeAnnotation = "org.cirruslabs.tart.upload-time"
struct OCIManifest: Codable, Equatable { struct OCIManifest: Codable, Equatable {
var schemaVersion: Int = 2 var schemaVersion: Int = 2
@@ -13,15 +14,21 @@ struct OCIManifest: Codable, Equatable {
var layers: [OCIManifestLayer] = Array() var layers: [OCIManifestLayer] = Array()
var annotations: Dictionary<String, String>? var annotations: Dictionary<String, String>?
init(config: OCIManifestConfig, layers: [OCIManifestLayer], uncompressedDiskSize: UInt64? = nil) { init(config: OCIManifestConfig, layers: [OCIManifestLayer], uncompressedDiskSize: UInt64? = nil, uploadDate: Date? = nil) {
self.config = config self.config = config
self.layers = layers self.layers = layers
var annotations: [String: String] = [:]
if let uncompressedDiskSize = uncompressedDiskSize { if let uncompressedDiskSize = uncompressedDiskSize {
annotations = [ annotations[uncompressedDiskSizeAnnotation] = String(uncompressedDiskSize)
uncompressedDiskSizeAnnotation: String(uncompressedDiskSize)
]
} }
if let uploadDate = uploadDate {
annotations[uploadTimeAnnotation] = uploadDate.toISO()
}
self.annotations = annotations
} }
init(fromJSON: Data) throws { init(fromJSON: Data) throws {
+3 -11
View File
@@ -116,19 +116,11 @@ struct Darwin: PlatformSuspendable {
} }
func pointingDevices() -> [VZPointingDeviceConfiguration] { func pointingDevices() -> [VZPointingDeviceConfiguration] {
if #available(macOS 13, *) { // Trackpad is only supported by guests starting with macOS Ventura
// Trackpad is only supported by guests starting with macOS Ventura [VZMacTrackpadConfiguration(), VZUSBScreenCoordinatePointingDeviceConfiguration()]
return [VZMacTrackpadConfiguration(), VZUSBScreenCoordinatePointingDeviceConfiguration()]
} else {
return [VZUSBScreenCoordinatePointingDeviceConfiguration()]
}
} }
func pointingDevicesSuspendable() -> [VZPointingDeviceConfiguration] { func pointingDevicesSuspendable() -> [VZPointingDeviceConfiguration] {
if #available(macOS 13, *) { [VZMacTrackpadConfiguration()]
return [VZMacTrackpadConfiguration()]
} else {
return []
}
} }
} }
+7 -4
View File
@@ -16,6 +16,7 @@ struct Root: AsyncParsableCommand {
Get.self, Get.self,
List.self, List.self,
Login.self, Login.self,
Logout.self,
IP.self, IP.self,
Pull.self, Pull.self,
Push.self, Push.self,
@@ -81,10 +82,12 @@ struct Root: AsyncParsableCommand {
var command = try parseAsRoot() var command = try parseAsRoot()
// Run garbage-collection before each command (shouldn't take too long) // Run garbage-collection before each command (shouldn't take too long)
do { if type(of: command) != type(of: Pull()) && type(of: command) != type(of: Clone()){
try Config().gc() do {
} catch { try Config().gc()
fputs("Failed to perform garbage collection!\n\(error)\n", stderr) } catch {
fputs("Failed to perform garbage collection!\n\(error)\n", stderr)
}
} }
if var asyncCommand = command as? AsyncParsableCommand { if var asyncCommand = command as? AsyncParsableCommand {
+27 -15
View File
@@ -225,7 +225,7 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
return try VM(vmDir: vmDir) return try VM(vmDir: vmDir)
} }
func run(recovery: Bool, resume shouldResume: Bool) async throws { func start(recovery: Bool, resume shouldResume: Bool) async throws {
try network.run(sema) try network.run(sema)
if shouldResume { if shouldResume {
@@ -233,7 +233,9 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
} else { } else {
try await start(recovery) try await start(recovery)
} }
}
func run() async throws {
await withTaskCancellationHandler(operation: { await withTaskCancellationHandler(operation: {
// Wait for the VM to finish running // Wait for the VM to finish running
// or for the exit condition // or for the exit condition
@@ -251,15 +253,9 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
@MainActor @MainActor
private func start(_ recovery: Bool) async throws { private func start(_ recovery: Bool) async throws {
if #available(macOS 13, *) { let startOptions = VZMacOSVirtualMachineStartOptions()
// new API introduced in Ventura startOptions.startUpFromMacOSRecovery = recovery
let startOptions = VZMacOSVirtualMachineStartOptions() try await virtualMachine.start(options: startOptions)
startOptions.startUpFromMacOSRecovery = recovery
try await virtualMachine.start(options: startOptions)
} else {
// use method that also available on Monterey
try await virtualMachine.start(recovery)
}
} }
@MainActor @MainActor
@@ -318,15 +314,19 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
} }
// Networking // Networking
let vio = VZVirtioNetworkDeviceConfiguration() configuration.networkDevices = network.attachments().map {
vio.attachment = network.attachment() let vio = VZVirtioNetworkDeviceConfiguration()
vio.macAddress = vmConfig.macAddress vio.attachment = $0
configuration.networkDevices = [vio] vio.macAddress = vmConfig.macAddress
return vio
}
// Storage // Storage
var attachments = [try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false)] var attachments = [try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false)]
attachments.append(contentsOf: additionalDiskAttachments) attachments.append(contentsOf: additionalDiskAttachments)
configuration.storageDevices = attachments.map { VZVirtioBlockDeviceConfiguration(attachment: $0) } configuration.storageDevices = attachments.map {
VZVirtioBlockDeviceConfiguration(attachment: $0)
}
// Entropy // Entropy
if !suspendable { if !suspendable {
@@ -339,6 +339,18 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Serial Port // Serial Port
configuration.serialPorts = serialPorts configuration.serialPorts = serialPorts
// Version console device
//
// A dummy console device useful for implementing
// host feature checks in the guest agent software.
let consolePort = VZVirtioConsolePortConfiguration()
consolePort.name = "tart-version-\(CI.version)"
let consoleDevice = VZVirtioConsoleDeviceConfiguration()
consoleDevice.ports[0] = consolePort
configuration.consoleDevices.append(consoleDevice)
try configuration.validate() try configuration.validate()
return configuration return configuration
+1 -5
View File
@@ -97,11 +97,7 @@ struct VMConfig: Codable {
case .darwin: case .darwin:
platform = try Darwin(from: decoder) platform = try Darwin(from: decoder)
case .linux: case .linux:
if #available(macOS 13, *) { platform = try Linux(from: decoder)
platform = try Linux(from: decoder)
} else {
throw UnsupportedOSError("Linux VMs", "are")
}
} }
cpuCountMin = try container.decode(Int.self, forKey: .cpuCountMin) cpuCountMin = try container.decode(Int.self, forKey: .cpuCountMin)
cpuCount = try container.decode(Int.self, forKey: .cpuCount) cpuCount = try container.decode(Int.self, forKey: .cpuCount)
+2 -1
View File
@@ -148,7 +148,8 @@ extension VMDirectory {
let manifest = OCIManifest( let manifest = OCIManifest(
config: OCIManifestConfig(size: ociConfigJSON.count, digest: ociConfigDigest), config: OCIManifestConfig(size: ociConfigJSON.count, digest: ociConfigDigest),
layers: layers, layers: layers,
uncompressedDiskSize: UInt64(mappedDiskReadOffset) uncompressedDiskSize: UInt64(mappedDiskReadOffset),
uploadDate: Date()
) )
// Manifest // Manifest
+12
View File
@@ -1,5 +1,6 @@
import Foundation import Foundation
import Virtualization import Virtualization
import CryptoKit
struct VMDirectory: Prunable { struct VMDirectory: Prunable {
var baseURL: URL var baseURL: URL
@@ -59,6 +60,17 @@ struct VMDirectory: Prunable {
return VMDirectory(baseURL: tmpDir) return VMDirectory(baseURL: tmpDir)
} }
//Create tmp directory with hashing
static func temporaryDeterministic(key: String) throws -> VMDirectory {
let keyData = Data(key.utf8)
let hash = Insecure.MD5.hash(data: keyData)
// Convert hash to string
let hashString = hash.compactMap { String(format: "%02x", $0) }.joined()
let tmpDir = try Config().tartTmpDir.appendingPathComponent(hashString)
try FileManager.default.createDirectory(at: tmpDir, withIntermediateDirectories: true)
return VMDirectory(baseURL: tmpDir)
}
var initialized: Bool { var initialized: Bool {
FileManager.default.fileExists(atPath: configURL.path) && FileManager.default.fileExists(atPath: configURL.path) &&
FileManager.default.fileExists(atPath: diskURL.path) && FileManager.default.fileExists(atPath: diskURL.path) &&
+7 -1
View File
@@ -1,6 +1,6 @@
import Foundation import Foundation
class VMStorageLocal { class VMStorageLocal: PrunableStorage {
let baseURL: URL = try! Config().tartHomeDir.appendingPathComponent("vms", isDirectory: true) let baseURL: URL = try! Config().tartHomeDir.appendingPathComponent("vms", isDirectory: true)
private func vmURL(_ name: String) -> URL { private func vmURL(_ name: String) -> URL {
@@ -16,6 +16,8 @@ class VMStorageLocal {
try vmDir.validate(userFriendlyName: name) try vmDir.validate(userFriendlyName: name)
try vmDir.baseURL.updateAccessDate()
return vmDir return vmDir
} }
@@ -63,6 +65,10 @@ class VMStorageLocal {
} }
} }
func prunables() throws -> [Prunable] {
try list().map { (_, vmDir) in vmDir }
}
func hasVMsWithMACAddress(macAddress: String) throws -> Bool { func hasVMsWithMACAddress(macAddress: String) throws -> Bool {
try list().contains { try $1.macAddress() == macAddress } try list().contains { try $1.macAddress() == macAddress }
} }
+1 -1
View File
@@ -170,7 +170,7 @@ class VMStorageOCI: PrunableStorage {
if !exists(digestName) { if !exists(digestName) {
let transaction = SentrySDK.startTransaction(name: name.description, operation: "pull", bindToScope: true) let transaction = SentrySDK.startTransaction(name: name.description, operation: "pull", bindToScope: true)
let tmpVMDir = try VMDirectory.temporary() let tmpVMDir = try VMDirectory.temporaryDeterministic(key: name.description)
// Lock the temporary VM directory to prevent it's garbage collection // Lock the temporary VM directory to prevent it's garbage collection
let tmpVMDirLock = try FileLock(lockURL: tmpVMDir.baseURL) let tmpVMDirLock = try FileLock(lockURL: tmpVMDir.baseURL)
+32
View File
@@ -0,0 +1,32 @@
import XCTest
@testable import tart
final class DirectoryShareTests: XCTestCase {
func testNamedParsing() throws {
let share = try DirectoryShare(parseFrom: "build:/Users/admin/build")
XCTAssertEqual(share.name, "build")
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertFalse(share.readOnly)
}
func testNamedReadOnlyParsing() throws {
let share = try DirectoryShare(parseFrom: "build:/Users/admin/build:ro")
XCTAssertEqual(share.name, "build")
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertTrue(share.readOnly)
}
func testOptionalNameParsing() throws {
let share = try DirectoryShare(parseFrom: "/Users/admin/build")
XCTAssertNil(share.name)
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertFalse(share.readOnly)
}
func testOptionalNameReadOnlyParsing() throws {
let share = try DirectoryShare(parseFrom: "/Users/admin/build:ro")
XCTAssertNil(share.name)
XCTAssertEqual(share.path, URL(filePath: "/Users/admin/build"))
XCTAssertTrue(share.readOnly)
}
}
+16
View File
@@ -0,0 +1,16 @@
import XCTest
@testable import tart
final class DockerConfigTests: XCTestCase {
func testHelpers() throws {
let config = DockerConfig(credHelpers: [
"(.*).dkr.ecr.(.*).amazonaws.com": "ecr-login",
"gcr.io": "gcloud"
])
XCTAssertEqual(try config.findCredHelper(host: "gcr.io"), "gcloud")
XCTAssertEqual(try config.findCredHelper(host: "123.dkr.ecr.eu-west-1.amazonaws.com"), "ecr-login")
XCTAssertEqual(try config.findCredHelper(host: "456.dkr.ecr.us-east-1.amazonaws.com"), "ecr-login")
XCTAssertNil(try config.findCredHelper(host: "ghcr.io"))
}
}
+2 -2
View File
@@ -13,7 +13,7 @@ task:
name: hello name: hello
macos_instance: macos_instance:
# can be a remote or a local virtual machine # can be a remote or a local virtual machine
image: ghcr.io/cirruslabs/macos-monterey-base:latest image: ghcr.io/cirruslabs/macos-ventura-base:latest
hello_script: hello_script:
- echo "Hello from within a Tart VM!" - echo "Hello from within a Tart VM!"
- echo "Here is my CPU info:" - echo "Here is my CPU info:"
@@ -45,7 +45,7 @@ exposes it via [`artifacts` instruction](https://cirrus-ci.org/guide/writing-tas
task: task:
name: Build name: Build
macos_instance: macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest image: ghcr.io/cirruslabs/macos-ventura-xcode:latest
build_script: swift build --product tart build_script: swift build --product tart
binary_artifacts: binary_artifacts:
path: .build/debug/tart path: .build/debug/tart
+1
View File
@@ -28,5 +28,6 @@ jobs:
``` ```
When workflows are executing you'll see Cirrus on-demand runners on your organization's settings page at `https://github.com/organizations/<ORGANIZATION>/settings/actions/runners`. When workflows are executing you'll see Cirrus on-demand runners on your organization's settings page at `https://github.com/organizations/<ORGANIZATION>/settings/actions/runners`.
Note that Cirrus Runners will get added to the default runner group. By default, only private repositories can access runners in a default runner group, but you can override this in your organization's settings.
![](/assets/images/TartGHARunners.png) ![](/assets/images/TartGHARunners.png)
+3 -3
View File
@@ -11,8 +11,8 @@ Tart can create VMs from `*.ipsw` files. You can download a specific `*.ipsw` fi
use `latest` instead of a path to `*.ipsw` to download the latest available version: use `latest` instead of a path to `*.ipsw` to download the latest available version:
```bash ```bash
tart create --from-ipsw=latest monterey-vanilla tart create --from-ipsw=latest ventura-vanilla
tart run monterey-vanilla tart run ventura-vanilla
``` ```
After the initial booting of the VM you'll need to manually go through the macOS installation process. As a convention we recommend creating an `admin` user with an `admin` password. After the regular installation please do some additional modifications in the VM: After the initial booting of the VM you'll need to manually go through the macOS installation process. As a convention we recommend creating an `admin` user with an `admin` password. After the regular installation please do some additional modifications in the VM:
@@ -54,7 +54,7 @@ Please refer to `tart set --help` for additional details.
## Building with Packer ## Building with Packer
Please refer to [Tart Packer Plugin repository](https://github.com/cirruslabs/packer-plugin-tart) for setup instructions. Please refer to [Tart Packer Plugin repository](https://github.com/cirruslabs/packer-plugin-tart) for setup instructions.
Here is an example of a template to build `monterey-base` local image based of a remote image: Here is an example of a template to build a local image based of a remote image:
```hcl ```hcl
packer { packer {
+42 -8
View File
@@ -8,19 +8,53 @@ are licensed under [Fair Source License](https://fair.io/). Usage on personal co
but organizations that exceed a certain number of server installations (100 CPU cores for Tart and/or 4 hosts for Orchard) but organizations that exceed a certain number of server installations (100 CPU cores for Tart and/or 4 hosts for Orchard)
will be required to obtain a paid license. will be required to obtain a paid license.
??? note "Performance and Efficiency Cores" ??? note "Host CPU Core usage"
The virtual CPU cores in Tart VMs do not differentiate between the high-performance and high-efficient cores The virtual CPU cores of Tart VMs are not tied to specific physical cores of the host CPU. Instead, for optimal performance
of the host CPU. Instead, Tart VMs automatically alternate between these types of cores depending on the workload Tart VMs will automatically try to balance compute between all available cores of the host CPU. As a result,
being executed within the virtual machines. As a result, both performance and energy-efficient cores of the host CPU all performance and energy-efficient cores of the host CPU are always counted towards the license usage.
are treated equally in terms of licensing.
# License Tiers # License Tiers
When an organization surpasses the 100 CPU cores limit, it is required to obtain a Gold Tier License, which costs \$1000 per month. ## Free Tier
Upon reaching a limit of 500 CPU cores, a Platinum Tier License (\$5000 per month) will be required, and for organizations
that exceed 5000 CPU cores, a custom Diamond Tier License (\$1 per core per month) will be necessary. By default, when no [license is purchased](#get-the-license), it is assumed that an organization is using a Free Tier license.
You can find the Free Tier license text in [Tart](https://github.com/cirruslabs/tart/blob/main/LICENSE) and [Orchard](https://github.com/cirruslabs/orchard/blob/main/LICENSE) repositories.
Free Tier license has a 100 CPU core limit for Tart and 4 Orchard Workers limit for Orchard.
??? info "Usage Scenarios Examples"
Here are a few examples that fit into the free tier:
- Using Tart on 12 Mac Minis with 8 CPUs each running up to 24 VMs in parallel.
- Creating an Orchard cluster of 4 Mac Studio workers with 24 CPUs each.
Here are a few examples that do not fit into the free tier:
- Using Tart on 13 Mac Minis with 8 CPUs each.
- Creating an Orchard cluster of 5 Mac Minis workers with 8 CPUs each.
## Gold Tier
If an organization wishes to exceed the limits of the Free Tier license, a purchase of the [Gold Tier License](#get-the-license) is required, which costs \$1000 per month.
Gold Tier license has a 500 CPU core limit for Tart and 20 Orchard Workers limit for Orchard.
## Platinum Tier
If an organization wishes to exceed the limits of the Gold Tier license, a purchase of the [Platinum Tier License](#get-the-license) is required, which costs \$5000 per month.
Platinum Tier license has a 5,000 CPU core limit for Tart and 200 Orchard Workers limit for Orchard.
## Diamond Tier
For organizations that wish to exceed the limits of the Platinum Tier license, a purchase of a [custom Diamond Tier License](#get-the-license) is required, which costs \$1 per CPU core per month and gives the ability to run unlimited Orchard Workers.
# Get the license
If your organization is interested in purchasing one of the license tiers, please email [licensing@cirruslabs.org](mailto:licensing@cirruslabs.org). If your organization is interested in purchasing one of the license tiers, please email [licensing@cirruslabs.org](mailto:licensing@cirruslabs.org).
You can see a template of a license subscription agreement [here](assets/TartLicenseSubscription.pdf). You can see a template of a license subscription agreement [here](assets/TartLicenseSubscription.pdf).
# General Support # General Support
+12 -2
View File
@@ -3,7 +3,7 @@ hide:
- navigation - navigation
--- ---
Try running a Tart VM on your Apple Silicon device running macOS 12.0 (Monterey) or later (will download a 25 GB image): Try running a Tart VM on your Apple Silicon device running macOS 13.0 (Ventura) or later (will download a 25 GB image):
```bash ```bash
brew install cirruslabs/cli/tart brew install cirruslabs/cli/tart
@@ -33,9 +33,19 @@ tart run ventura-base
If the guest VM is running and configured to accept incoming SSH connections you can conveniently connect to it like so: If the guest VM is running and configured to accept incoming SSH connections you can conveniently connect to it like so:
```bash ```bash
ssh admin@$(tart ip macos-ventura-base) ssh admin@$(tart ip ventura-base)
``` ```
!!! tip "Running scripts inside Tart virtual machines"
We recommend using [Cirrus CLI](integrations/cirrus-cli.md) to run scripts and/or retrieve artifacts
from within Tart virtual machines. Alternatively, you can use plain ssh connection and `tart ip` command:
```bash
brew install sshpass
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip ventura-base) "uname -a"
sshpass -p admin ssh -o "StrictHostKeyChecking no" admin@$(tart ip ventura-base) < script.sh
```
## Mounting directories ## Mounting directories
To mount a directory, run the VM with the `--dir` argument: To mount a directory, run the VM with the `--dir` argument:
+1 -1
View File
@@ -81,7 +81,7 @@
} }
</style> </style>
<script src="https://unpkg.com/@dotlottie/player-component@latest/dist/dotlottie-player.js"></script> <script src="https://unpkg.com/@dotlottie/player-component@1.4.2/dist/dotlottie-player.js"></script>
<!-- landing page for landing page --> <!-- landing page for landing page -->
<!-- Hero --> <!-- Hero -->
+1
View File
@@ -3,3 +3,4 @@ testcontainers
requests requests
bitmath bitmath
pytest-dependency pytest-dependency
paramiko
+9 -4
View File
@@ -7,10 +7,9 @@ class Tart:
def __init__(self): def __init__(self):
self.tmp_dir = tempfile.TemporaryDirectory(dir=os.environ.get("CIRRUS_WORKING_DIR")) self.tmp_dir = tempfile.TemporaryDirectory(dir=os.environ.get("CIRRUS_WORKING_DIR"))
# Link to the users IPSW cache to make things faster # Link to the users cache to make things faster
src = os.path.join(os.path.expanduser("~"), ".tart", "cache", "IPSWs") src = os.path.join(os.path.expanduser("~"), ".tart", "cache")
dst = os.path.join(self.tmp_dir.name, "cache", "IPSWs") dst = os.path.join(self.tmp_dir.name, "cache")
os.makedirs(os.path.join(self.tmp_dir.name, "cache"))
os.symlink(src, dst) os.symlink(src, dst)
def __enter__(self): def __enter__(self):
@@ -31,3 +30,9 @@ class Tart:
completed_process.check_returncode() completed_process.check_returncode()
return completed_process.stdout.decode("utf-8"), completed_process.stderr.decode("utf-8") return completed_process.stdout.decode("utf-8"), completed_process.stderr.decode("utf-8")
def run_async(self, args) -> subprocess.Popen:
env = os.environ.copy()
env.update({"TART_HOME": self.tmp_dir.name})
return subprocess.Popen(["tart"] + args, env=env)
+1 -1
View File
@@ -6,5 +6,5 @@ def test_clone(tart):
tart.run(["clone", "debian", "ubuntu"]) tart.run(["clone", "debian", "ubuntu"])
# Ensure that we have now 2 VMs # Ensure that we have now 2 VMs
stdout, _, = tart.run(["list", "--quiet"]) stdout, _, = tart.run(["list", "--source", "local", "--quiet"])
assert stdout == "debian\nubuntu\n" assert stdout == "debian\nubuntu\n"
+2 -2
View File
@@ -3,7 +3,7 @@ def test_create_macos(tart):
tart.run(["create", "--from-ipsw", "latest", "macos-vm"]) tart.run(["create", "--from-ipsw", "latest", "macos-vm"])
# Ensure that the VM was created # Ensure that the VM was created
stdout, _ = tart.run(["list", "--quiet"]) stdout, _ = tart.run(["list", "--source", "local", "--quiet"])
assert stdout == "macos-vm\n" assert stdout == "macos-vm\n"
@@ -12,5 +12,5 @@ def test_create_linux(tart):
tart.run(["create", "--linux", "linux-vm"]) tart.run(["create", "--linux", "linux-vm"])
# Ensure that the VM was created # Ensure that the VM was created
stdout, _ = tart.run(["list", "--quiet"]) stdout, _ = tart.run(["list", "--source", "local", "--quiet"])
assert stdout == "linux-vm\n" assert stdout == "linux-vm\n"
+2 -2
View File
@@ -3,12 +3,12 @@ def test_delete(tart):
tart.run(["create", "--linux", "debian"]) tart.run(["create", "--linux", "debian"])
# Ensure that the VM exists # Ensure that the VM exists
stdout, _, = tart.run(["list", "--quiet"]) stdout, _, = tart.run(["list", "--source", "local", "--quiet"])
assert stdout == "debian\n" assert stdout == "debian\n"
# Delete the VM # Delete the VM
tart.run(["delete", "debian"]) tart.run(["delete", "debian"])
# Ensure that the VM was removed # Ensure that the VM was removed
stdout, _, = tart.run(["list", "--quiet"]) stdout, _, = tart.run(["list", "--source", "local", "--quiet"])
assert stdout == "" assert stdout == ""
+1 -1
View File
@@ -6,5 +6,5 @@ def test_rename(tart):
tart.run(["rename", "debian", "ubuntu"]) tart.run(["rename", "debian", "ubuntu"])
# Ensure that the VM is now named "ubuntu" # Ensure that the VM is now named "ubuntu"
stdout, _, = tart.run(["list", "--quiet"]) stdout, _, = tart.run(["list", "--source", "local", "--quiet"])
assert stdout == "ubuntu\n" assert stdout == "ubuntu\n"
+30
View File
@@ -0,0 +1,30 @@
import uuid
from paramiko.client import SSHClient, AutoAddPolicy
def test_run(tart):
vm_name = f"integration-test-run-{uuid.uuid4()}"
# Instantiate a VM with admin:admin SSH access
tart.run(["clone", "ghcr.io/cirruslabs/macos-ventura-base:latest", vm_name])
# Run the VM asynchronously
tart_run_process = tart.run_async(["run", vm_name])
# Obtain the VM's IP
stdout, _ = tart.run(["ip", vm_name, "--wait", "120"])
ip = stdout.strip()
# Connect to the VM over SSH and shutdown it
client = SSHClient()
client.set_missing_host_key_policy(AutoAddPolicy)
client.connect(ip, username="admin", password="admin")
client.exec_command("sudo shutdown -h now")
# Wait for the "tart run" to finish successfully
tart_run_process.wait()
assert tart_run_process.returncode == 0
# Delete the VM
_, _ = tart.run(["delete", vm_name])