Commit Graph

250 Commits

Author SHA1 Message Date
Fedor Korotkov 5c7743b7cd
Show both size and actual size of files (#742)
Right now we show only actual size of files on disk which excludes empty blocks of the recently introduced sparced format in #671. This makes impossible to get info about disk size that we just set via `tart set`.

Here is an example of `tart list` output before the change:

```
Source Name                                                                                                            Size State
local  sonoma-base                                                                                                     22   stopped
local  sonoma-vanilla                                                                                                  18   stopped
local  sonoma-xcode                                                                                                    67   stopped
local  ubuntu                                                                                                          1    stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base:latest                                                                     22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base@sha256:16c1593bbaf787b20b3c0bc094c5b6baf71c937d22c2e4596da85ac55c92e6cc    22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla:14.3                                                                    17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla@sha256:23c4e853d48d00a4333346d66a32b2b5aad900cc0dc10e7ecb9dbe67b6f587f4 17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode:latest                                                                    67   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode@sha256:d0cb8d01424a68b89e0f16f5371bf2152b2c115bd886341a6ba8da42121d1f41   67   stopped
oci    ghcr.io/cirruslabs/ubuntu:22.04                                                                                 1    stopped
oci    ghcr.io/cirruslabs/ubuntu@sha256:037763feb7a15d6077edeb7a097738c34313637d16036764b4c196d28d8b429c               1    stopped
```

And here is the output after the change:

```
Source Name                                                                                                            Disk Size State
local  sonoma-base                                                                                                     50   22   stopped
local  sonoma-vanilla                                                                                                  50   18   stopped
local  sonoma-xcode                                                                                                    90   67   stopped
local  ubuntu                                                                                                          20   1    stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base:latest                                                                     50   22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-base@sha256:16c1593bbaf787b20b3c0bc094c5b6baf71c937d22c2e4596da85ac55c92e6cc    50   22   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla:14.3                                                                    50   17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-vanilla@sha256:23c4e853d48d00a4333346d66a32b2b5aad900cc0dc10e7ecb9dbe67b6f587f4 50   17   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode:latest                                                                    90   67   stopped
oci    ghcr.io/cirruslabs/macos-sonoma-xcode@sha256:d0cb8d01424a68b89e0f16f5371bf2152b2c115bd886341a6ba8da42121d1f41   90   67   stopped
oci    ghcr.io/cirruslabs/ubuntu:22.04                                                                                 20   1    stopped
oci    ghcr.io/cirruslabs/ubuntu@sha256:037763feb7a15d6077edeb7a097738c34313637d16036764b4c196d28d8b429c               20   1    stopped
```

Additionally, `tart get` will print actual size with a 3 decimal point precision which will help to track growth in disk images for our templates.

`tart get` before:

```
CPU Memory Disk Display  State
4   8192   67   1024x768 stopped
```

`tart get` after:

```
CPU Memory Disk Size   Display  State
4   8192   90   67.333 1024x768 stopped
```
2024-02-27 20:13:38 +04:00
Nikolay Edigaryev e2d6c13ed0
DHCP MAC-address resolver: handle duplicate leases (#740) 2024-02-23 16:49:06 +04:00
Nikolay Edigaryev 3f17884ac2
Introduce "tart fqn" command (#735)
* tart pull: experimental --json-digest option

* Introduce "tart fqn" command

* Revert "tart pull: experimental --json-digest option"

This reverts commit 842052f5bd.
2024-02-19 21:13:45 +00:00
Fedor Korotkov 7dcebf9c04
Allow to override VirtioFS tag for shared directories (#733) 2024-02-19 20:51:25 +00:00
Tor Arne Vestbø f6c56ed8eb
Add option to disable audio pass-though (#728) 2024-02-19 12:26:31 -05:00
Fedor Korotkov a48f4d4ec9
Do not magically set `--no-graphics` when `--vnc` is passed (#732)
From a discussion in #728 it appeared that having both `--graphics` and `--no-graphics` is a bit confusing.

`--graphics` was introduced in #248 to support having both VNC and UI for debugging Packer plugin in cirruslabs/packer-plugin-tart#21. This is because `--vnc` flag has a side effect of hiding UI which I think was wrong in retrospective. One can run `tart run --vnc --no-graphics`. In most of the cases this is automated via Alfred or something like that.

Now we have so many arguments that IMO it's worth to remove `--graphics` for overall consistency in arguments: everything is enabled by default and can be disabled via `--no-*` flags.
2024-02-19 08:47:27 -05:00
Noah Martin 9bb71a4051
Improve rename error message (#723) 2024-01-30 10:01:42 -05:00
Fedor Korotkov 18d462dd3d
Build x86 binary (#716)
* Build x86 binary

To support Linux VMs on Intel aka x86_64

* Fixed paths and formatting

* Unique IDs

* Fixed Goreleaser

* Skip creation integration test for now

* import

* Reenable create test

* Revert "Reenable create test"

This reverts commit 4c947c1f0e.

* Reenable create test
2024-01-26 13:09:05 +00:00
Nikolay Edigaryev 96f6f94fa7
tart set: make --disk-size change idempotent (#698)
...when the size is not changed.
2024-01-08 14:34:13 +00:00
Nikolay Edigaryev fbc481250d
tart run: disable dynamic display reconfiguration for Linux (#697) 2024-01-08 14:25:56 +00:00
Nikolay Edigaryev 35538c2c5d
tart set: fix typo in --disk-size help (#695) 2024-01-04 01:09:25 +00:00
Nikolay Edigaryev 4c33064916
tart set: bring back the --disk-size command-line argument (#694)
* tart set: bring back the --disk-size command-line argument

* Add a --disk-size explainer
2024-01-04 00:34:55 +04:00
Nikolay Edigaryev 1a267d4a39
tart create --linux: allow scaling VM down to 1 CPU and 256 MiB (#693)
* tart create --linux: allow scaling VM down to 1 CPU and 256 MiB

* Revert "tart create --linux: allow scaling VM down to 1 CPU and 256 MiB"

This reverts commit 7a31443eea.

* Check minimum CPU and memory sizes in "tart set"
2024-01-03 19:48:13 +04:00
Nikolay Edigaryev 1d8bfafde5
tart create --from-ipsw: expand tilde (~) in path (#688) 2023-12-18 13:05:48 -05:00
Nikolay Edigaryev 537f0ae5db
OCI storage: unconditionally remove the old link when link()'ing (#686) 2023-12-08 17:20:49 +04:00
Fedor Korotkov 60f0eac7a8
Cache only non-empty archives (#685)
Fixes #684. But I'm not sure how it got into this state in the first place. `URLSession.shared.data` should've throw.
2023-12-08 10:43:43 +04:00
Andrew Malchuk 35377a3475
Fix the filesystem corruption on Linux VMs (#675)
* Use NVMe drive, cached mode and full synchronization mode on Linux

* Inline getting storage device attachment
2023-12-01 15:56:58 +00:00
Nikolay Edigaryev ac5f794e6d
tart delete: prevent the deletion of running VMs (#676)
And introduce a VMDirectory.lock() method to avoid duplication of
the PIDLock(lockURL: vmDir.configURL) snippet.
2023-12-01 09:33:01 -05:00
Fedor Korotkov bf03873c8d
Validate that a disk is not amd64 (#673)
To improve UX for cases like #672
2023-11-28 14:55:34 +00:00
Nikolay Edigaryev bad37b129c
DiskV2: write layers sparsely to avoid unnecessary disk usage (#671) 2023-11-27 23:27:07 +04:00
Nikolay Edigaryev 0f47cca746
MAC address resolver: skip expired leases (#669) 2023-11-27 10:12:36 -05:00
Fedor Korotkov aca768a838
Print put errors from Docker Helpers (#654)
* Print put errors from Docker Helpers

* Update Sources/tart/Credentials/DockerConfigCredentialsProvider.swift

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>

* Check output data is not empty

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-11-10 22:44:51 +04:00
Tor Arne Vestbø 68b3557747
Hide dock icon in no graphics mode (#653)
* Package tart binary into app bundle when running via run-signed.sh

This is what happens when installing the tart application package
as built by CI. We should stay as close as possible to the install
situation during development, so that we get bug/behavior parity.

For example, an app bundle behaves differently than a standalone
executable when it comes to bringing up a Dock icon for the app.

* Set activation policy to prohibited when starting in no graphics mode

This ensures that the Dock icon is hidden.
2023-11-10 09:05:20 -05:00
Fedor Korotkov b2c923f2fe
Properly enter main even loop in headless mode (#651)
Fixes #638
2023-11-09 00:05:00 +04:00
Fedor Korotkov c75009e46f
Introduce `--capture-system-keys` flag (#650)
To allow guest to capture things like Cmd+Tab.

Fixes #636
2023-11-08 20:21:55 +04:00
Nikolay Edigaryev 70040b633c
Introduce AuthenticationKeeper actor to serialize authn modification (#647) 2023-11-06 14:58:23 -05:00
Nikolay Edigaryev f4bc02d175
DiskV2.push(): map disk into memory to avoid large allocations (#645) 2023-11-03 17:13:10 +04:00
Nikolay Edigaryev d8b69de52d
Fetcher.fetchViaFile(): use an mmap(2)-ed file, similarly to DiskV1 (#641)
* Fetcher.fetchViaFile(): use an mmap(2)-ed file, similarly to DiskV1

* No need to convert Data to Data
2023-11-01 15:00:48 +04:00
Fedor Korotkov b95585b56b
Don't forget to finalize output stream (#640)
There is a suspicion that this might leak memory
2023-11-01 12:19:11 +04:00
Fedor Korotkov 8d5574ed3f
Removed usage of deprecated APIs (#628)
See https://developer.apple.com/documentation/virtualization/vzmacauxiliarystorage/3816043-init
2023-10-11 17:05:49 -04:00
Fedor Korotkov 8e9d61d5f5
Validate that a suspendable VM doesn't have shared directories (#623) 2023-10-04 20:21:45 +04:00
Fedor Korotkov 71d03226fe
Support mounting remote archives (#620)
* Support mounting remote archives

Allow to pass an HTTPS link instead of a local path to `tart run --dir` argument. HTTPS link should point to a gzipped Tar archive aka `*.tar.gz` file.

In this situation Tart will download an archive by the link if necessary, will cache it and will unarchive it into a temporary folder inside `$TART_HOME` to be mounted to the VM.

This use case is useful for mounting something external that updates faster than the VM itself. For example, GitHub Actions Runner installation.

* Don't use async/await APIs to prevent from deadlocks because of the MainActor thing

* Prefer cached data

* Moved comment

* Fix URLCache caching files in memory instead of on-disk (#622)

* Fix URLCache caching files in memory instead of on-disk

* Fix disk capacity typo

* Moved log

* Moved fetching logic to `DirectoryShare#createConfiguration` method

---------

Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-10-03 23:01:29 +04:00
Nikolay Edigaryev 36dab9878d
tart: bump max password characters from 256 to 1024 (#618) 2023-10-02 13:58:55 +00:00
Nikolay Edigaryev f634002813
tart run: disable console device when --suspendable is requested (#615) 2023-09-30 21:13:44 +04:00
Fedor Korotkov 50ce44c3eb
Support block devices on Sonoma (#611)
* Support block devices on Sonoma

* Updated docs

* Removed unused error
2023-09-22 22:30:18 +04:00
Fedor Korotkov 6df50e55d8
`--suspendable` devices fallback on Ventura host (#605)
Fixes #604
2023-09-22 07:38:13 -04:00
Nikolay Edigaryev 3ff4fc34c6
Improved format for fast and efficient pulls from remote OCI-registry (#589)
* Improved format for fast and efficient pulls from remote OCI-registry

* Tests: introduce fileWithRandomData() helper function

* Remove useless continuation

* --concurrency should be an option, not an argument

* --v2-disk-format → --old-disk-format and use the new V2 by default

* Reduce LZ4 buffer size from 64 to 4 MiB

* --old-disk-format → --disk-format=...
2023-09-20 10:14:05 -04:00
Nikolay Edigaryev 27cadc3f3b
tart create --from-ipsw: do a HEAD instead of a GET first (#599) 2023-09-14 17:16:08 +00:00
Nikolay Edigaryev d4d3852745
Return exit code 2 on RuntimeError.VMDoesNotExist (#597)
* Return exit code 2 on RuntimeError.VMDoesNotExist

* Upgrade isFileNotFound() do detect underlying errors
2023-09-12 13:56:49 +04:00
Fedor Korotkov 4bb248e7b4
Support wildcards in `credHelpers` (#592)
* Support wildcards in `credHelpers`

With #591 `tart pull` fails when for example you have `ecr-login` set as the default `credsStore` but you try to pull our images from `ghcr.io`.

This change reverts #591 and instead supports regex in `credHelpers`. This is not supported by Docker itself but highly demanded in https://github.com/docker/cli/issues/2928

I think it's fine to support it for Tart.

Additionally this change bumps the minimum host macOS version to Ventura in order to bring `Regex`. Yes, `Regex` only supported in Swift for macOS 13+ 🤦‍♂️I think it's fine in the light of Sonoma release and Tart 2.0.0.

* Removed Monterey mentions from docs
2023-08-28 11:37:49 -04:00
Fedor Korotkov f45551cbf0
Support Docker's `credsStore` (#591)
This way for #581 we don't need to specify a fully quialified URL and can simply use the following `~/.docker/config.json`:

```json
{
	"credsStore": "ecr-login"
}
```

Related to https://github.com/docker/cli/issues/2928
2023-08-21 19:44:35 +00:00
Fedor Korotkov 637a2387e7
No bridged network interfaces by default (#586)
Fixes #585
2023-08-15 19:47:25 +04:00
Nikolay Edigaryev 5eddd1ce41
Introduce "tart logout" command (#583)
* Introduce "tart logout"

* tart login: introduce --no-validate
2023-08-15 15:16:33 +04:00
Fedor Korotkov 35f5b30bc4
Multiple bridged interfaces (#578)
* Support multiple Bridged Network interfaces

Fixes #572

* Allow duplicated bridged interfaces
2023-08-14 19:17:38 +04:00
Tommy feb733a7c0
GC avoidance and tmpDeterminstic (#570)
* GC avoidance and tmpDeterminstic

* change tmpDeterministic to use hashing

- temporaryDeterministic() now takes in a key and hashes it
- creates directory with the hash

* Update Sources/tart/VMStorageOCI.swift

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>

---------

Co-authored-by: Fedor Korotkov <fedor.korotkov@gmail.com>
2023-07-31 18:21:42 +00:00
Fedor Korotkov 704811e671
Introduce `TART_NO_AUTO_PRUNE` (#565)
* Introduce `TART_NO_AUTO_PRUNE`

Similar to `HOMEBREW_NO_AUTO_UPDATE`

* Self review
2023-07-25 16:01:53 +00:00
Nikolay Edigaryev 33ca96e1a0
Retrieve VM's IP for use in VNC after the VM is started #2 (#562) 2023-07-17 12:51:43 +04:00
Nikolay Edigaryev 6c377029d6
tart prune: allow pruning local VMs with --entries=vms (#557) 2023-07-13 22:05:28 +04:00
Fedor Korotkov 93a1b70ecb
Don't delete an initiator of pruning (#556)
* Don't delete an initiator of pruning

Sometimes people have an image that is greater than half of the disk itself. In that case such image will be pulled and prunned right away.

This change makes sure that an image that is being cloned from is not pruned right away.

* Resolve symbolic links
2023-07-13 13:59:23 +04:00
Fedor Korotkov cf9a3a9221
Allow mounting a single directory without a name (#555)
* Allow mounting a single directory without a name

To utilize `VZSingleDirectoryShare` which seems more stable than `VZMultipleDirectoryShare`.

We've been having reports from users that mounted directories occasionally return "no such file" errors when building large projects. I took a stab at reproducing the issue by running https://github.com/devMEremenko/XcodeBenchmark in a mounted directory:

```bash
tart run --dir=workdir-test:~/workspace-temp/XcodeBenchmark ventura-xcode
```

And I was able to reproduce the "no such file" error on the first try! After looking into the issue I decided to try `VZSingleDirectoryShare` as this PR changes and to my pleasant surprise it all worked like a charm the next run. So it seems there is a bug in `VZMultipleDirectoryShare` integration with virtiofs. Since in most cases users only mount a single directory it makes sense to allow doing it wihtout providing a `name`.

So now it will be possible to run the following command:

```bash
tart run --dir=~/workspace-temp/XcodeBenchmark ventura-xcode
```

Which will make `~/workspace-temp/XcodeBenchmark` available under `/Volumes/My Shared Files/` without any intermediate directories.

* Reformat

* Updated description
2023-07-12 19:08:52 +00:00