docs(mac): improve documentation for mac build (#1873)

* docs(mac): Improve documentation for mac build

- added documentation to build from current master
- added troubleshooting information

Signed-off-by: Sebastian <tauven@gmail.com>

* docs(max): fix typo

Signed-off-by: Sebastian <tauven@gmail.com>

---------

Signed-off-by: Sebastian <tauven@gmail.com>
This commit is contained in:
Sebastian 2024-03-21 22:08:33 +01:00 committed by GitHub
parent 3cf64d1e7e
commit 743095b7d8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -45,6 +45,8 @@ To install the dependencies follow the instructions below:
{{< tabs tabTotal="3" >}} {{< tabs tabTotal="3" >}}
{{% tab tabName="Apple" %}} {{% tab tabName="Apple" %}}
Install `xcode` from the App Store
```bash ```bash
brew install abseil cmake go grpc protobuf wget brew install abseil cmake go grpc protobuf wget
``` ```
@ -111,10 +113,12 @@ docker run --rm -ti -p 8080:8080 -e DEBUG=true -e MODELS_PATH=/models -e THREADS
### Example: Build on mac ### Example: Build on mac
Building on Mac (M1 or M2) works, but you may need to install some prerequisites using `brew`. Building on Mac (M1, M2 or M3) works, but you may need to install some prerequisites using `brew`.
The below has been tested by one mac user and found to work. Note that this doesn't use Docker to run the server: The below has been tested by one mac user and found to work. Note that this doesn't use Docker to run the server:
Install `xcode` from the Apps Store (needed for metalkit)
``` ```
# install build dependencies # install build dependencies
brew install abseil cmake go grpc protobuf wget brew install abseil cmake go grpc protobuf wget
@ -146,8 +150,20 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso
}' }'
``` ```
### Build with Image generation support #### Troublshooting mac
If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store.
If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256).
If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again.
```
# reinstall build dependencies
brew reinstall abseil cmake go grpc protobuf wget
make clean
make build
```
**Requirements**: OpenCV, Gomp **Requirements**: OpenCV, Gomp
@ -239,13 +255,12 @@ make BUILD_TYPE=sycl_f32 build # for float32
#### Metal (Apple Silicon) #### Metal (Apple Silicon)
``` ```
make BUILD_TYPE=metal build make build
# Set `gpu_layers: 1` to your YAML model config file and `f16: true` # correct build type is automatically used on mac (BUILD_TYPE=metal)
# Note: only models quantized with q4_0 are supported! # Set `gpu_layers: 256` (or equal to the number of model layers) to your YAML model config file and `f16: true`
``` ```
### Windows compatibility ### Windows compatibility
Make sure to give enough resources to the running container. See https://github.com/go-skynet/LocalAI/issues/2 Make sure to give enough resources to the running container. See https://github.com/go-skynet/LocalAI/issues/2