Compare commits

...
8 Commits
Author SHA1 Message Date
Fedor Korotkov 227301436c Revert "Drop Monterey Support (#843)" (#893)
This reverts commit 017592075f.
2024-08-14 14:57:55 -04:00
Nikolay Edigaryev 106eb5a2c8 tart push: re-try when encountering errors when pushing disk layers (#888)
* tart push: re-try when encountering errors when pushing disk layers

* Only re-try on URLError
2024-08-10 13:06:39 -04:00
Nikolay Edigaryev 10bf706653 tart push: avoid uploading blobs if they are already present (#887)
By issuing HEAD requests to the registry before doing the actual upload.
2024-08-09 17:26:20 +04:00
Fedor Korotkov ff928ad77d Optimize DiskV2 Deduplication (#878)
* Revert "Lowercase `tart.app` (#751)"

This reverts commit a9e2a19015.

* Optimize DiskV2 deduplication logic

In case we cloned `disk.img` from a local image, check if data at offset has the expected contents already.

* Hole punch only if needed

* Calculate hash only if needed

* subdataChunks optimization

* Reapply "Lowercase `tart.app` (#751)"

This reverts commit e74e9c845a.

* format

* Save at least 1GB on deduplication logic

* Build separately

* Revert "subdataChunks optimization"

This reverts commit e59382aeba.

* Another optimization

* Removed debug log

* reformat

* Revert "Hole punch only if needed"

This reverts commit 8c569fc5
2024-08-05 12:24:31 -04:00
Nikolay Edigaryev 33b5cfe2ed tart run: delay tilde (~) expansion until we're dealing with local path (#880) 2024-08-05 15:42:10 +04:00
Nikolay Edigaryev 3892cdb00d tart run: replace --sync with --root-disk-opts (#879)
* VZDiskImageSynchronizationMode's "description" field is a dead code

* Re-use the VZDiskImageSynchronizationMode extension

* tart run: replace --sync with --root-disk-opts

* VM: support root disk synchronization mode on macOS
2024-08-05 15:17:58 +04:00
Nicholas FitzRoy-Dale 5f2199ef3e Support setting root disk synchronization mode (#875)
* Support setting root disk synchronization mode

Adds a new VMConfig parameter (tart get / tart set) called 'sync' which
can be set to 'full' (default), 'fsync', or 'none', corresponding with
the values of VZDiskImageSynchronizationMode and allowing a tradeoff
between data integrity and speed.

* Remove unused import

* Fix formatting

* Make root disk sync behaviour a commandline option
2024-08-05 13:12:43 +04:00
Fedor Korotkov 3f26baa341 Update testimonials to focus on Tart (#876)
Cirrus Runners have their own testimonials and a website now. No need to mix things together.
2024-08-01 17:09:08 +04:00
15 changed files with 296 additions and 200 deletions
+2 -2
View File
@@ -8,11 +8,11 @@ task:
name: dev-mini name: dev-mini
resources: resources:
tart-vms: 1 tart-vms: 1
build_script:
- swift build
test_script: test_script:
- swift test - swift test
integration_test_script: integration_test_script:
# Build Tart
- swift build
- codesign --sign - --entitlements Resources/tart-dev.entitlements --force .build/debug/tart - codesign --sign - --entitlements Resources/tart-dev.entitlements --force .build/debug/tart
- export PATH=$(pwd)/.build/arm64-apple-macosx/debug:$PATH - export PATH=$(pwd)/.build/arm64-apple-macosx/debug:$PATH
# Run integration tests # Run integration tests
+4 -1
View File
@@ -17,7 +17,7 @@ service — a drop-in replacement for the standard GitHub-hosted runners, offeri
</a> </a>
</p> </p>
Many companies are using Tart in their internal setups. Here are a few of them: Many companies are using Tart in their internal setups. Here are just a few of them:
<p align="center"> <p align="center">
<a href="https://atlassian.com/" target=_blank> <a href="https://atlassian.com/" target=_blank>
@@ -47,6 +47,9 @@ Many companies are using Tart in their internal setups. Here are a few of them:
<a href="https://www.pitsdatarecovery.net/" target=_blank> <a href="https://www.pitsdatarecovery.net/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/PITSGlobalDataRecoveryServices.png" height="65"/> <img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/PITSGlobalDataRecoveryServices.png" height="65"/>
</a> </a>
<a href="https://expo.dev/" target=_blank>
<img src="https://github.com/cirruslabs/tart/raw/main/Resources/Users/Expo.png" height="65"/>
</a>
</p> </p>
**Note:** If your company or project is using Tart please consider [sharing with the community](https://github.com/cirruslabs/tart/discussions/857). **Note:** If your company or project is using Tart please consider [sharing with the community](https://github.com/cirruslabs/tart/discussions/857).
Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

+82 -59
View File
@@ -12,6 +12,39 @@ var vm: VM?
struct IPNotFound: Error { struct IPNotFound: Error {
} }
@available(macOS 14, *)
extension VZDiskSynchronizationMode {
public init(_ description: String) throws {
switch description {
case "none":
self = .none
case "full":
self = .full
case "":
self = .full
default:
throw RuntimeError.VMConfigurationError("unsupported disk synchronization mode: \"\(description)\"")
}
}
}
extension VZDiskImageSynchronizationMode {
public init(_ description: String) throws {
switch description {
case "none":
self = .none
case "fsync":
self = .fsync
case "full":
self = .full
case "":
self = .full
default:
throw RuntimeError.VMConfigurationError("unsupported disk image synchronization mode: \"\(description)\"")
}
}
}
struct Run: AsyncParsableCommand { struct Run: AsyncParsableCommand {
static var configuration = CommandConfiguration(abstract: "Run a VM") static var configuration = CommandConfiguration(abstract: "Run a VM")
@@ -140,6 +173,20 @@ struct Run: AsyncParsableCommand {
@Flag(help: ArgumentHelp("Restrict network access to the host-only network")) @Flag(help: ArgumentHelp("Restrict network access to the host-only network"))
var netHost: Bool = false var netHost: Bool = false
@Option(help: ArgumentHelp("Set the root disk options (e.g. --root-disk-opts=\"ro\" or --root-disk-opts=\"sync=none\")",
discussion: """
Options are comma-separated and are as follows:
* ro — attach the root disk in read-only mode instead of the default read-write (e.g. --root-disk-opts="ro")
* sync=none — disable data synchronization with the permanent storage to increase performance at the cost of a higher chance of data loss (e.g. --root-disk-opts="sync=none")
* sync=fsync — enable data synchronization with the permanent storage, but don't ensure that it was actually written (e.g. --root-disk-opts="sync=fsync")
* sync=full — enable data synchronization with the permanent storage and ensure that it was actually written (e.g. --root-disk-opts="sync=full")
""", valueName: "options"))
var rootDiskOpts: String = ""
#if arch(arm64) #if arch(arm64)
@Flag(help: ArgumentHelp("Disables audio and entropy devices and switches to only Mac-specific input devices.", discussion: "Useful for running a VM that can be suspended via \"tart suspend\".")) @Flag(help: ArgumentHelp("Disables audio and entropy devices and switches to only Mac-specific input devices.", discussion: "Useful for running a VM that can be suspended via \"tart suspend\"."))
#endif #endif
@@ -182,7 +229,7 @@ struct Run: AsyncParsableCommand {
if suspendable { if suspendable {
let config = try VMConfig.init(fromURL: vmDir.configURL) let config = try VMConfig.init(fromURL: vmDir.configURL)
if (config.platform is Linux) { if !(config.platform is PlatformSuspendable) {
throw ValidationError("You can only suspend macOS VMs") throw ValidationError("You can only suspend macOS VMs")
} }
if dir.count > 0 { if dir.count > 0 {
@@ -219,8 +266,6 @@ struct Run: AsyncParsableCommand {
try Softnet.configureSUIDBitIfNeeded() try Softnet.configureSUIDBitIfNeeded()
} }
let additionalDiskAttachments = try additionalDiskAttachments()
var serialPorts: [VZSerialPortConfiguration] = [] var serialPorts: [VZSerialPortConfiguration] = []
if serial { if serial {
let tty_fd = createPTY() let tty_fd = createPTY()
@@ -239,15 +284,19 @@ struct Run: AsyncParsableCommand {
serialPorts.append(createSerialPortConfiguration(tty_read!, tty_write!)) serialPorts.append(createSerialPortConfiguration(tty_read!, tty_write!))
} }
// Parse root disk options
let diskOptions = DiskOptions(rootDiskOpts)
vm = try VM( vm = try VM(
vmDir: vmDir, vmDir: vmDir,
network: userSpecifiedNetwork(vmDir: vmDir) ?? NetworkShared(), network: userSpecifiedNetwork(vmDir: vmDir) ?? NetworkShared(),
additionalStorageDevices: additionalDiskAttachments, additionalStorageDevices: try additionalDiskAttachments(),
directorySharingDevices: directoryShares() + rosettaDirectoryShare(), directorySharingDevices: directoryShares() + rosettaDirectoryShare(),
serialPorts: serialPorts, serialPorts: serialPorts,
suspendable: suspendable, suspendable: suspendable,
audio: !noAudio, audio: !noAudio,
clipboard: !noClipboard clipboard: !noClipboard,
sync: VZDiskImageSynchronizationMode(diskOptions.syncModeRaw)
) )
let vncImpl: VNC? = try { let vncImpl: VNC? = try {
@@ -455,15 +504,9 @@ struct Run: AsyncParsableCommand {
} }
func additionalDiskAttachments() throws -> [VZStorageDeviceConfiguration] { func additionalDiskAttachments() throws -> [VZStorageDeviceConfiguration] {
var result: [VZStorageDeviceConfiguration] = [] try disk.map {
try AdditionalDisk(parseFrom: $0).configuration
let expandedDiskPaths = disk.map { NSString(string:$0).expandingTildeInPath }
for rawDisk in expandedDiskPaths {
result.append(try AdditionalDisk(parseFrom: rawDisk).configuration)
} }
return result
} }
func directoryShares() throws -> [VZDirectorySharingDeviceConfiguration] { func directoryShares() throws -> [VZDirectorySharingDeviceConfiguration] {
@@ -700,18 +743,20 @@ struct AdditionalDisk {
throw UnsupportedOSError("attaching Network Block Devices", "are") throw UnsupportedOSError("attaching Network Block Devices", "are")
} }
let syncMode = try parseSyncMode(syncModeRaw)
let nbdAttachment = try VZNetworkBlockDeviceStorageDeviceAttachment( let nbdAttachment = try VZNetworkBlockDeviceStorageDeviceAttachment(
url: diskURL!, url: diskURL!,
timeout: 30, timeout: 30,
isForcedReadOnly: diskReadOnly, isForcedReadOnly: diskReadOnly,
synchronizationMode: syncMode synchronizationMode: try VZDiskSynchronizationMode(syncModeRaw)
) )
return VZVirtioBlockDeviceConfiguration(attachment: nbdAttachment) return VZVirtioBlockDeviceConfiguration(attachment: nbdAttachment)
} }
// Expand the tilde (~) since at this point we're dealing with a local path,
// and "expandingTildeInPath" seems to corrupt the remote URLs like nbd://
let diskPath = NSString(string: diskPath).expandingTildeInPath
let diskFileURL = URL(fileURLWithPath: diskPath) let diskFileURL = URL(fileURLWithPath: diskPath)
if pathHasMode(diskPath, mode: S_IFBLK) { if pathHasMode(diskPath, mode: S_IFBLK) {
@@ -734,7 +779,7 @@ struct AdditionalDisk {
} }
let blockAttachment = try VZDiskBlockDeviceStorageDeviceAttachment(fileHandle: FileHandle(fileDescriptor: fd, closeOnDealloc: true), let blockAttachment = try VZDiskBlockDeviceStorageDeviceAttachment(fileHandle: FileHandle(fileDescriptor: fd, closeOnDealloc: true),
readOnly: diskReadOnly, synchronizationMode: try parseSyncMode(syncModeRaw)) readOnly: diskReadOnly, synchronizationMode: try VZDiskSynchronizationMode(syncModeRaw))
return VZVirtioBlockDeviceConfiguration(attachment: blockAttachment) return VZVirtioBlockDeviceConfiguration(attachment: blockAttachment)
} }
@@ -768,7 +813,7 @@ struct AdditionalDisk {
url: diskFileURL, url: diskFileURL,
readOnly: diskReadOnly, readOnly: diskReadOnly,
cachingMode: .automatic, cachingMode: .automatic,
synchronizationMode: try Self.parseImageSyncMode(syncModeRaw) synchronizationMode: try VZDiskImageSynchronizationMode(syncModeRaw)
) )
return VZVirtioBlockDeviceConfiguration(attachment: diskImageAttachment) return VZVirtioBlockDeviceConfiguration(attachment: diskImageAttachment)
@@ -776,58 +821,36 @@ struct AdditionalDisk {
static func parseOptions(_ parseFrom: String) -> (String, Bool, String) { static func parseOptions(_ parseFrom: String) -> (String, Bool, String) {
var arguments = parseFrom.split(separator: ":") var arguments = parseFrom.split(separator: ":")
let options = arguments.last!.split(separator: ",")
var readOnly: Bool = false let options = DiskOptions(String(arguments.last!))
var syncModeRaw: String = "" if options.foundAtLeastOneOption {
arguments.removeLast()
}
var foundAtLeastOneOption: Bool = false return (arguments.joined(separator: ":"), options.readOnly, options.syncModeRaw)
}
}
struct DiskOptions {
var readOnly: Bool = false
var syncModeRaw: String = ""
var foundAtLeastOneOption: Bool = false
init(_ parseFrom: String) {
let options = parseFrom.split(separator: ",")
for option in options { for option in options {
switch true { switch true {
case option == "ro": case option == "ro":
readOnly = true self.readOnly = true
foundAtLeastOneOption = true self.foundAtLeastOneOption = true
case option.hasPrefix("sync="): case option.hasPrefix("sync="):
syncModeRaw = String(option.dropFirst("sync=".count)) self.syncModeRaw = String(option.dropFirst("sync=".count))
foundAtLeastOneOption = true self.foundAtLeastOneOption = true
default: default:
continue continue
} }
} }
if foundAtLeastOneOption {
arguments.removeLast()
}
return (arguments.joined(separator: ":"), readOnly, syncModeRaw)
}
@available(macOS 14, *)
static func parseSyncMode(_ parseFrom: String) throws -> VZDiskSynchronizationMode {
switch parseFrom {
case "none":
return .none
case "full":
return .full
case "":
return .full
default:
throw RuntimeError.VMConfigurationError("unsupported disk synchronization mode: \"\(parseFrom)\"")
}
}
static func parseImageSyncMode(_ parseFrom: String) throws -> VZDiskImageSynchronizationMode {
switch parseFrom {
case "none":
return .none
case "full":
return .full
case "":
return .full
default:
throw RuntimeError.VMConfigurationError("unsupported disk image synchronization mode: \"\(parseFrom)\"")
}
} }
} }
+23 -6
View File
@@ -1,12 +1,19 @@
import Foundation import Foundation
struct LocalLayerCache { struct LocalLayerCache {
struct DigestInfo {
let range: Range<Data.Index>
let compressedDigest: String
let uncompressedContentDigest: String?
}
let name: String let name: String
let deduplicatedBytes: UInt64 let deduplicatedBytes: UInt64
let diskURL: URL let diskURL: URL
private let mappedDisk: Data private let mappedDisk: Data
private var digestToRange: [String : Range<Data.Index>] = [:] private var digestToRange: [String: DigestInfo] = [:]
private var offsetToRange: [UInt64: DigestInfo] = [:]
init?(_ name: String, _ deduplicatedBytes: UInt64, _ diskURL: URL, _ manifest: OCIManifest) throws { init?(_ name: String, _ deduplicatedBytes: UInt64, _ diskURL: URL, _ manifest: OCIManifest) throws {
self.name = name self.name = name
@@ -24,17 +31,27 @@ struct LocalLayerCache {
return nil return nil
} }
self.digestToRange[layer.digest] = Int(offset)..<Int(offset+uncompressedSize) let info = DigestInfo(
range: Int(offset)..<Int(offset + uncompressedSize),
compressedDigest: layer.digest,
uncompressedContentDigest: layer.uncompressedContentDigest()!
)
self.digestToRange[layer.digest] = info
self.offsetToRange[offset] = info
offset += uncompressedSize offset += uncompressedSize
} }
} }
func find(_ digest: String) -> Data? { func findInfo(digest: String, offsetHint: UInt64) -> DigestInfo? {
guard let foundRange = self.digestToRange[digest] else { // Layers can have the same digests, for example, empty ones. Let's use the offset hint to make a better guess.
return nil if let info = self.offsetToRange[offsetHint], info.compressedDigest == digest {
return info
} }
return self.digestToRange[digest]
}
return self.mappedDisk.subdata(in: foundRange) func subdata(_ range: Range<Data.Index>) -> Data {
return self.mappedDisk.subdata(in: range)
} }
} }
+35 -9
View File
@@ -1,6 +1,7 @@
import Foundation import Foundation
import Compression import Compression
import System import System
import Retry
class DiskV2: Disk { class DiskV2: Disk {
private static let bufferSizeBytes = 4 * 1024 * 1024 private static let bufferSizeBytes = 4 * 1024 * 1024
@@ -26,8 +27,22 @@ class DiskV2: Disk {
// Launch a disk layer pushing task // Launch a disk layer pushing task
group.addTask { group.addTask {
let compressedData = try (data as NSData).compressed(using: .lz4) as Data let compressedData = try (data as NSData).compressed(using: .lz4) as Data
let compressedDataDigest = Digest.hash(compressedData)
let layerDigest = try await registry.pushBlob(fromData: compressedData, chunkSizeMb: chunkSizeMb) try await retry(maxAttempts: 5, backoff: .exponentialWithFullJitter(baseDelay: .seconds(5), maxDelay: .seconds(60))) {
if try await !registry.blobExists(compressedDataDigest) {
_ = try await registry.pushBlob(fromData: compressedData, chunkSizeMb: chunkSizeMb, digest: compressedDataDigest)
}
} recoverFromFailure: { error in
if error is URLError {
print("Error: \(error.localizedDescription)")
print("Attempting to re-try...")
return .retry
}
return .throw
}
// Update progress using a relative value // Update progress using a relative value
progress.completedUnitCount += Int64(data.count) progress.completedUnitCount += Int64(data.count)
@@ -35,7 +50,7 @@ class DiskV2: Disk {
return (index, OCIManifestLayer( return (index, OCIManifestLayer(
mediaType: diskV2MediaType, mediaType: diskV2MediaType,
size: compressedData.count, size: compressedData.count,
digest: layerDigest, digest: compressedDataDigest,
uncompressedSize: UInt64(data.count), uncompressedSize: UInt64(data.count),
uncompressedContentDigest: Digest.hash(data) uncompressedContentDigest: Digest.hash(data)
)) ))
@@ -121,11 +136,14 @@ class DiskV2: Disk {
// Launch a fetching and decompression task // Launch a fetching and decompression task
group.addTask { group.addTask {
// No need to fetch and decompress anything if we've already done so // No need to fetch and decompress anything if we've already done so
if try pullResumed && Digest.hash(diskURL, offset: diskWritingOffset, size: uncompressedLayerSize) == uncompressedLayerContentDigest { if pullResumed {
// Update the progress // do not check hash in the condition above to make it lazy e.g. only do expensive calculations if needed
progress.completedUnitCount += Int64(diskLayer.size) if try Digest.hash(diskURL, offset: diskWritingOffset, size: uncompressedLayerSize) == uncompressedLayerContentDigest {
// Update the progress
progress.completedUnitCount += Int64(diskLayer.size)
return return
}
} }
// Open the disk file for writing // Open the disk file for writing
@@ -140,9 +158,17 @@ class DiskV2: Disk {
} }
// Check if we already have this layer contents in the local layer cache // Check if we already have this layer contents in the local layer cache
if let localLayerCache = localLayerCache, let data = localLayerCache.find(diskLayer.digest), Digest.hash(data) == uncompressedLayerContentDigest { if let localLayerCache = localLayerCache, let localLayerInfo = localLayerCache.findInfo(digest: diskLayer.digest, offsetHint: diskWritingOffset) {
// Fulfil the layer contents from the local blob cache // indicates that the locally cloned disk image has the same content at the given offset
_ = try zeroSkippingWrite(disk, rdisk, fsBlockSize, diskWritingOffset, data) let localHit = localLayerInfo.uncompressedContentDigest == uncompressedLayerContentDigest
&& localLayerInfo.range.lowerBound == diskWritingOffset
// doesn't seem that localHit can ever be false if the localLayerCache is not nil
// but let's just add extra safety here and check it
if !localHit {
// Fulfil the layer contents from the local blob cache
let data = localLayerCache.subdata(localLayerInfo.range)
_ = try zeroSkippingWrite(disk, rdisk, fsBlockSize, diskWritingOffset, data)
}
try disk.close() try disk.close()
// Update the progress // Update the progress
+17 -2
View File
@@ -10,6 +10,7 @@ enum RegistryError: Error {
} }
enum HTTPMethod: String { enum HTTPMethod: String {
case HEAD = "HEAD"
case GET = "GET" case GET = "GET"
case POST = "POST" case POST = "POST"
case PUT = "PUT" case PUT = "PUT"
@@ -21,6 +22,7 @@ enum HTTPCode: Int {
case Created = 201 case Created = 201
case Accepted = 202 case Accepted = 202
case Unauthorized = 401 case Unauthorized = 401
case NotFound = 404
} }
extension Data { extension Data {
@@ -189,7 +191,7 @@ class Registry {
return URLComponents(url: uploadLocation.absolutize(baseURL), resolvingAgainstBaseURL: true)! return URLComponents(url: uploadLocation.absolutize(baseURL), resolvingAgainstBaseURL: true)!
} }
public func pushBlob(fromData: Data, chunkSizeMb: Int = 0) async throws -> String { public func pushBlob(fromData: Data, chunkSizeMb: Int = 0, digest: String? = nil) async throws -> String {
// Initiate a blob upload // Initiate a blob upload
let (data, postResponse) = try await dataRequest(.POST, endpointURL("\(namespace)/blobs/uploads/"), let (data, postResponse) = try await dataRequest(.POST, endpointURL("\(namespace)/blobs/uploads/"),
headers: ["Content-Length": "0"]) headers: ["Content-Length": "0"])
@@ -201,7 +203,7 @@ class Registry {
// Figure out where to upload the blob // Figure out where to upload the blob
var uploadLocation = try uploadLocationFromResponse(postResponse) var uploadLocation = try uploadLocationFromResponse(postResponse)
let digest = Digest.hash(fromData) let digest = digest ?? Digest.hash(fromData)
if chunkSizeMb == 0 { if chunkSizeMb == 0 {
// monolithic upload // monolithic upload
@@ -249,6 +251,19 @@ class Registry {
return digest return digest
} }
public func blobExists(_ digest: String) async throws -> Bool {
let (data, response) = try await dataRequest(.HEAD, endpointURL("\(namespace)/blobs/\(digest)"))
switch response.statusCode {
case HTTPCode.Ok.rawValue:
return true
case HTTPCode.NotFound.rawValue:
return false
default:
throw RegistryError.UnexpectedHTTPStatusCode(when: "checking blob", code: response.statusCode, details: data.asText())
}
}
public func pullBlob(_ digest: String, handler: (Data) async throws -> Void) async throws { public func pullBlob(_ digest: String, handler: (Data) async throws -> Void) async throws {
let (channel, response) = try await channelRequest(.GET, endpointURL("\(namespace)/blobs/\(digest)"), viaFile: true) let (channel, response) = try await channelRequest(.GET, endpointURL("\(namespace)/blobs/\(digest)"), viaFile: true)
if response.statusCode != HTTPCode.Ok.rawValue { if response.statusCode != HTTPCode.Ok.rawValue {
+18 -4
View File
@@ -8,7 +8,7 @@ struct UnsupportedHostOSError: Error, CustomStringConvertible {
#if arch(arm64) #if arch(arm64)
struct Darwin: Platform { struct Darwin: PlatformSuspendable {
var ecid: VZMacMachineIdentifier var ecid: VZMacMachineIdentifier
var hardwareModel: VZMacHardwareModel var hardwareModel: VZMacHardwareModel
@@ -103,18 +103,32 @@ struct UnsupportedHostOSError: Error, CustomStringConvertible {
func keyboards() -> [VZKeyboardConfiguration] { func keyboards() -> [VZKeyboardConfiguration] {
if #available(macOS 14, *) { if #available(macOS 14, *) {
// Mac keyboard is only supported by guests starting with macOS Ventura // Mac keyboard is only supported by guests starting with macOS Ventura
return [VZMacKeyboardConfiguration()] return [VZUSBKeyboardConfiguration(), VZMacKeyboardConfiguration()]
} else { } else {
return [VZUSBKeyboardConfiguration()] return [VZUSBKeyboardConfiguration()]
} }
} }
func keyboardsSuspendable() -> [VZKeyboardConfiguration] {
if #available(macOS 14, *) {
return [VZMacKeyboardConfiguration()]
} else {
// fallback to the regular configuration
return keyboards()
}
}
func pointingDevices() -> [VZPointingDeviceConfiguration] { func pointingDevices() -> [VZPointingDeviceConfiguration] {
if #available(macOS 13, *) { // Trackpad is only supported by guests starting with macOS Ventura
[VZUSBScreenCoordinatePointingDeviceConfiguration(), VZMacTrackpadConfiguration()]
}
func pointingDevicesSuspendable() -> [VZPointingDeviceConfiguration] {
if #available(macOS 14, *) {
return [VZMacTrackpadConfiguration()] return [VZMacTrackpadConfiguration()]
} else { } else {
// fallback to the regular configuration // fallback to the regular configuration
return [VZUSBScreenCoordinatePointingDeviceConfiguration()] return pointingDevices()
} }
} }
} }
+5
View File
@@ -8,3 +8,8 @@ protocol Platform: Codable {
func keyboards() -> [VZKeyboardConfiguration] func keyboards() -> [VZKeyboardConfiguration]
func pointingDevices() -> [VZPointingDeviceConfiguration] func pointingDevices() -> [VZPointingDeviceConfiguration]
} }
protocol PlatformSuspendable: Platform {
func pointingDevicesSuspendable() -> [VZPointingDeviceConfiguration]
func keyboardsSuspendable() -> [VZKeyboardConfiguration]
}
-1
View File
@@ -17,7 +17,6 @@ extension URL: Prunable {
func deduplicatedSizeBytes() throws -> Int { func deduplicatedSizeBytes() throws -> Int {
let values = try resourceValues(forKeys: [.totalFileAllocatedSizeKey, .mayShareFileContentKey]) let values = try resourceValues(forKeys: [.totalFileAllocatedSizeKey, .mayShareFileContentKey])
// make sure the file's origin file is there and duplication works // make sure the file's origin file is there and duplication works
var dedublicatedSize = 0
if values.mayShareFileContent == true { if values.mayShareFileContent == true {
return Int(deduplicatedBytes()) return Int(deduplicatedBytes())
} }
+15 -7
View File
@@ -48,7 +48,8 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
serialPorts: [VZSerialPortConfiguration] = [], serialPorts: [VZSerialPortConfiguration] = [],
suspendable: Bool = false, suspendable: Bool = false,
audio: Bool = true, audio: Bool = true,
clipboard: Bool = true clipboard: Bool = true,
sync: VZDiskImageSynchronizationMode = .full
) throws { ) throws {
name = vmDir.name name = vmDir.name
config = try VMConfig.init(fromURL: vmDir.configURL) config = try VMConfig.init(fromURL: vmDir.configURL)
@@ -66,7 +67,8 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
serialPorts: serialPorts, serialPorts: serialPorts,
suspendable: suspendable, suspendable: suspendable,
audio: audio, audio: audio,
clipboard: clipboard clipboard: clipboard,
sync: sync
) )
virtualMachine = VZVirtualMachine(configuration: configuration) virtualMachine = VZVirtualMachine(configuration: configuration)
@@ -294,7 +296,8 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
serialPorts: [VZSerialPortConfiguration], serialPorts: [VZSerialPortConfiguration],
suspendable: Bool = false, suspendable: Bool = false,
audio: Bool = true, audio: Bool = true,
clipboard: Bool = true clipboard: Bool = true,
sync: VZDiskImageSynchronizationMode = .full
) throws -> VZVirtualMachineConfiguration { ) throws -> VZVirtualMachineConfiguration {
let configuration = VZVirtualMachineConfiguration() let configuration = VZVirtualMachineConfiguration()
@@ -326,8 +329,13 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
configuration.audioDevices = [soundDeviceConfiguration] configuration.audioDevices = [soundDeviceConfiguration]
// Keyboard and mouse // Keyboard and mouse
configuration.keyboards = vmConfig.platform.keyboards() if suspendable, let platformSuspendable = vmConfig.platform.self as? PlatformSuspendable {
configuration.pointingDevices = vmConfig.platform.pointingDevices() configuration.keyboards = platformSuspendable.keyboardsSuspendable()
configuration.pointingDevices = platformSuspendable.pointingDevicesSuspendable()
} else {
configuration.keyboards = vmConfig.platform.keyboards()
configuration.pointingDevices = vmConfig.platform.pointingDevices()
}
// Networking // Networking
configuration.networkDevices = network.attachments().map { configuration.networkDevices = network.attachments().map {
@@ -350,8 +358,8 @@ class VM: NSObject, VZVirtualMachineDelegate, ObservableObject {
// Storage // Storage
let attachment: VZDiskImageStorageDeviceAttachment = vmConfig.os == .linux ? let attachment: VZDiskImageStorageDeviceAttachment = vmConfig.os == .linux ?
// Use "cached" caching mode for virtio drive to prevent fs corruption on linux // Use "cached" caching mode for virtio drive to prevent fs corruption on linux
try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false, cachingMode: .cached, synchronizationMode: .full) : try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false, cachingMode: .cached, synchronizationMode: sync) :
try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false) try VZDiskImageStorageDeviceAttachment(url: diskURL, readOnly: false, cachingMode: .automatic, synchronizationMode: sync)
var device: VZStorageDeviceConfiguration var device: VZStorageDeviceConfiguration
if #available(macOS 14, *), vmConfig.os == .linux { if #available(macOS 14, *), vmConfig.os == .linux {
+7 -7
View File
@@ -208,14 +208,14 @@ class VMStorageOCI: PrunableStorage {
try await tmpVMDir.pullFromRegistry(registry: registry, manifest: manifest, concurrency: concurrency, localLayerCache: localLayerCache) try await tmpVMDir.pullFromRegistry(registry: registry, manifest: manifest, concurrency: concurrency, localLayerCache: localLayerCache)
} recoverFromFailure: { error in } recoverFromFailure: { error in
if error is RuntimeError { if error is Retryable {
return .throw print("Error: \(error.localizedDescription)")
print("Attempting to re-try...")
return .retry
} }
print("Error: \(error.localizedDescription)") return .throw
print("Attempting to re-try...")
return .retry
} }
try move(digestName, from: tmpVMDir) try move(digestName, from: tmpVMDir)
transaction.finish() transaction.finish()
@@ -297,7 +297,7 @@ class VMStorageOCI: PrunableStorage {
// Now, find the best match based on how many bytes we'll deduplicate // Now, find the best match based on how many bytes we'll deduplicate
let choosen = candidates.filter { let choosen = candidates.filter {
$0.deduplicatedBytes > 0 $0.deduplicatedBytes > 1024 * 1024 * 1024 // save at least 1GB
}.max { left, right in }.max { left, right in
return left.deduplicatedBytes < right.deduplicatedBytes return left.deduplicatedBytes < right.deduplicatedBytes
} }
Binary file not shown.

After

Width:  |  Height:  |  Size: 5.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.4 KiB

+88 -102
View File
@@ -85,10 +85,10 @@
<!-- landing image --> <!-- landing image -->
<div class="tx-landing__image"> <div class="tx-landing__image">
<dotlottie-player <dotlottie-player
src="/assets/animations/TartLogo.lottie" src="/assets/animations/TartLogo.lottie"
mode="normal" mode="normal"
style="width: 75%; margin: auto" style="width: 75%; margin: auto"
autoplay autoplay
/> />
</div> </div>
@@ -114,10 +114,9 @@
<header class="md-typeset"> <header class="md-typeset">
<h1 id="virtualization-and-beyond"> <h1 id="virtualization-and-beyond">
Virtualization and beyond Virtualization and beyond
<a <a href="#virtualization-and-beyond"
href="#virtualization-and-beyond" class="headerlink"
class="headerlink" title="Permanent link"
title="Permanent link"
> >
</a> </a>
@@ -125,12 +124,11 @@
</header> </header>
<div class="mdx-spotlight"> <div class="mdx-spotlight">
<figure class="mdx-spotlight__feature"> <figure class="mdx-spotlight__feature">
<img <img src="assets/images/spotlight/virtualization-framework.webp"
src="assets/images/spotlight/virtualization-framework.webp" alt="Apple’s native Virtualization.Framework"
alt="Apple’s native Virtualization.Framework" loading="lazy"
loading="lazy" width="500"
width="500" height="212"
height="212"
/> />
<figcaption class="md-typeset"> <figcaption class="md-typeset">
<h2>Native performance</h2> <h2>Native performance</h2>
@@ -144,12 +142,11 @@
</figcaption> </figcaption>
</figure> </figure>
<figure class="mdx-spotlight__feature"> <figure class="mdx-spotlight__feature">
<img <img src="assets/images/spotlight/supported-registries.webp"
src="assets/images/spotlight/supported-registries.webp" alt="OCI-compatible container registries"
alt="OCI-compatible container registries" loading="lazy"
loading="lazy" width="500"
width="500" height="160"
height="160"
/> />
<figcaption class="md-typeset"> <figcaption class="md-typeset">
<h2>Remote storage for Virtual Machines</h2> <h2>Remote storage for Virtual Machines</h2>
@@ -161,39 +158,39 @@
</figcaption> </figcaption>
</figure> </figure>
<figure class="mdx-spotlight__feature"> <figure class="mdx-spotlight__feature">
<img <img src="assets/images/spotlight/github-actions-runners.webp"
src="assets/images/spotlight/github-actions-runners.webp" alt="GitHub Actions Runners"
alt="GitHub Actions Runners" loading="lazy"
loading="lazy" width="500"
width="500" height="280"
height="280"
/> />
<figcaption class="md-typeset"> <figcaption class="md-typeset">
<h2>Seamless integration with your existing automations</h2> <h2>Seamless integration with your existing automations</h2>
<p> <p>
Tart integrates with many continuous integration systems, including a dedicated Tart integrates with many continuous integration systems, including a dedicated
service of on-demand GitHub Actions Runners. With a single line change, you can cut your service of on-demand GitHub Actions Runners. With a single line change, you can cut your
CI/CD costs by up to <b>30 times</b> by using <a href="https://cirrus-runners.app/">Cirrus Runners</a> CI/CD costs by up to <b>30 times</b> by using <a href="https://cirrus-runners.app/">Cirrus
Runners</a>
to run your workflows. to run your workflows.
</p> </p>
</figcaption> </figcaption>
</figure> </figure>
<figure class="mdx-spotlight__feature"> <figure class="mdx-spotlight__feature">
<div id="lottie-player"> <div id="lottie-player">
<dotlottie-player <dotlottie-player src="/assets/animations/Orchard.lottie"
src="/assets/animations/Orchard.lottie" mode="normal"
mode="normal" style="height: 280px; margin: auto"
style="height: 280px; margin: auto" autoplay
autoplay loop
loop
/> />
</div> </div>
<figcaption class="md-typeset"> <figcaption class="md-typeset">
<h2>Run at scale with <a href="https://github.com/cirruslabs/orchard">Orchard</a></h2> <h2>Run at scale with <a href="https://github.com/cirruslabs/orchard">Orchard</a></h2>
<p> <p>
Tart toolset includes Orchard Orchestration &mdash; tool to run and manage Tart virtual machines Tart toolset includes Orchard Orchestration &mdash; tool to run and manage Tart virtual
at scale on a cluster of Apple Silicon hosts. An Orchard Cluster exposes a simple REST API to manage machines at scale on a cluster of Apple Silicon hosts. An Orchard Cluster exposes a simple REST API to
thousands virtual machines. Orchard CLI allows accessing remote virtual machines like they run locally. manage thousands virtual machines. Orchard CLI allows accessing remote virtual machines like they run
locally.
</p> </p>
</figcaption> </figcaption>
</figure> </figure>
@@ -208,38 +205,38 @@
<header class="md-typeset"> <header class="md-typeset">
<h1 id="powerhouse"> <h1 id="powerhouse">
Automation Powerhouse Automation Powerhouse
<a <a href="#powerhouse"
href="#powerhouse" class="headerlink"
class="headerlink" title="Permanent link"
title="Permanent link"
> >
</a> </a>
</h1> </h1>
</header> </header>
<script> <script>
fetch("https://api.github.com/repos/cirruslabs/tart/releases?per_page=100") fetch("https://api.github.com/repos/cirruslabs/tart/releases?per_page=100")
.then((response) => response.json()) .then((response) => response.json())
.then((releases) => { .then((releases) => {
let allDownloads = 0; let allDownloads = 0;
for (let release of releases) { for (let release of releases) {
for (let asset of release.assets) { for (let asset of release.assets) {
if (asset && asset.content_type === "application/octet-stream") { if (asset && asset.content_type === "application/octet-stream") {
allDownloads += asset.download_count || 0 allDownloads += asset.download_count || 0
} }
} }
} }
let counterElement = document.getElementById('installation-counter'); let counterElement = document.getElementById('installation-counter');
if (counterElement) { if (counterElement) {
// Live installation count is available starting version 1.0.0 // Live installation count is available starting version 1.0.0
// Prior Tart was installed a little over 14,000 times, let's count them too // Prior Tart was installed a little over 14,000 times, let's count them too
let installationPriorV1 = 14 let installationPriorV1 = 14
counterElement.textContent = (installationPriorV1 + Math.round(allDownloads / 1000)) + ",000" counterElement.textContent = (installationPriorV1 + Math.round(allDownloads / 1000)) + ",000"
} }
}) })
</script> </script>
<h2> <h2>
With more than <strong id="installation-counter">25,000</strong> installations to date, Tart has been adopted for various scenarios. With more than <strong id="installation-counter">25,000</strong> installations to date, Tart has been
adopted for various scenarios.
Its applications range from powering CI/CD pipelines and reproducible local development environments, Its applications range from powering CI/CD pipelines and reproducible local development environments,
to helping in the testing of device management systems without actual physical devices. to helping in the testing of device management systems without actual physical devices.
</h2> </h2>
@@ -254,10 +251,9 @@
<header class="md-typeset"> <header class="md-typeset">
<h1 id="what-our-users-say"> <h1 id="what-our-users-say">
What our users say What our users say
<a <a href="#what-our-users-say"
href="#what-our-users-say" class="headerlink"
class="headerlink" title="Permanent link"
title="Permanent link"
> >
</a> </a>
@@ -265,70 +261,60 @@
</header> </header>
<div class="mdx-users"> <div class="mdx-users">
<figure class="mdx-users__testimonial"> <figure class="mdx-users__testimonial">
<img <img src="assets/images/users/mikhail-tokarev.webp"
src="assets/images/users/mitchell-hashimoto.webp" alt="Mikhail Tokarev"
alt="Mitchell Hashimoto" loading="lazy"
loading="lazy" width="200"
width="200" height="200"
height="200"
/> />
<figcaption class="md-typeset"> <figcaption class="md-typeset">
<h2>Mitchell Hashimoto</h2>
<h3> <h3>
<a href="https://www.hashicorp.com/" target="_blank">HashiCorp</a> co-founder Mikhail Tokarev, CTO at <a href="https://codemagic.io/start/" target="_blank">Codemagic</a>
</h3> </h3>
<hr/> <hr/>
<cite> <cite>
I've been using "Cirrus Runners" since <a href="https://x.com/mitchellh/status/1731071326201561194" target="_blank">that tweet</a> and Thanks to the minimal overhead of using the Apple Virtualization
it has been fantastic. Huge speed increase, huge cost decrease, zero maintenance, exactly what I wanted. API, we’ve seen some performance improvements in booting new
virtual machines compared with Anka.
</cite> </cite>
</figcaption> </figcaption>
</figure> </figure>
<figure class="mdx-users__testimonial"> <figure class="mdx-users__testimonial">
<img <img src="assets/images/users/expo.webp"
src="assets/images/users/seb-jachec.webp" alt="Expo"
alt="Sebastian Jachec" loading="lazy"
loading="lazy" width="200"
width="200" height="200"
height="200"
/> />
<figcaption class="md-typeset"> <figcaption class="md-typeset">
<h2>Sebastian Jachec</h2>
<h3> <h3>
Mobile Engineer at Infrastructure Team at <a href="https://expo.dev/" target="_blank">Expo</a>
<a href="https://daybridge.com/" target="_blank">Daybridge</a>
</h3> </h3>
<hr/> <hr/>
<cite> <cite>
It&rsquo;s been plain-sailing with the Tart was the practical way for us to use the Virtualization framework. Cirrus Labs’
<a href="/integrations/github-actions">Cirrus Runners</a>&nbsp;&mdash; continued maintenance and support gives us confidence, and it is also important for us
they&rsquo;ve been great! They&rsquo;re consistently&nbsp;60+% to be able to read the source code when we need to understand an abstraction layer below.
faster on&nbsp;workflows that we&nbsp;previously used Github
Actions&rsquo; macOS runners for.
</cite> </cite>
</figcaption> </figcaption>
</figure> </figure>
<figure class="mdx-users__testimonial"> <figure class="mdx-users__testimonial">
<img <img src="assets/images/users/snowflake.webp"
src="assets/images/users/max-lapides.webp" alt="Snowflake"
alt="Max Lapides" loading="lazy"
loading="lazy" width="200"
width="200" height="200"
height="200"
/> />
<figcaption class="md-typeset"> <figcaption class="md-typeset">
<h2>Max Lapides</h2>
<h3> <h3>
Senior Mobile Engineer at Red Team at <a href="https://www.snowflake.com/" target="_blank">Snowflake</a>
<a href="https://www.tonal.com/" target="_blank">Tonal</a>
</h3> </h3>
<hr/> <hr/>
<cite> <cite>
Previously, we were using the GitHub&#8209;hosted macOS runners The Snowflake Red Team had a need for macOS CI/CD and a segmented macOS development
and our iOS build took ~30&nbsp;minutes. Now with environment. We solved this problem and shared our implementation with macOS EC2 and Tart.
<a href="/integrations/github-actions">Cirrus Runners</a>, the iOS build only We also automated this process with Terraform/Packer to simplify the deployment of our
takes ~12&nbsp;minutes. That’s a huge boost to our productivity, infrastructure and machine images.
and for only $150/month per runner it is much less expensive too.
</cite> </cite>
</figcaption> </figcaption>
</figure> </figure>