From 743095b7d82b20998b4dde1e557292c7fcc6bd82 Mon Sep 17 00:00:00 2001 From: Sebastian Date: Thu, 21 Mar 2024 22:08:33 +0100 Subject: [PATCH] docs(mac): improve documentation for mac build (#1873) * docs(mac): Improve documentation for mac build - added documentation to build from current master - added troubleshooting information Signed-off-by: Sebastian * docs(max): fix typo Signed-off-by: Sebastian --------- Signed-off-by: Sebastian --- docs/content/docs/getting-started/build.md | 27 +++++++++++++++++----- 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/docs/content/docs/getting-started/build.md b/docs/content/docs/getting-started/build.md index b26a16d7..238bdbec 100644 --- a/docs/content/docs/getting-started/build.md +++ b/docs/content/docs/getting-started/build.md @@ -45,6 +45,8 @@ To install the dependencies follow the instructions below: {{< tabs tabTotal="3" >}} {{% tab tabName="Apple" %}} +Install `xcode` from the App Store + ```bash brew install abseil cmake go grpc protobuf wget ``` @@ -111,10 +113,12 @@ docker run --rm -ti -p 8080:8080 -e DEBUG=true -e MODELS_PATH=/models -e THREADS ### Example: Build on mac -Building on Mac (M1 or M2) works, but you may need to install some prerequisites using `brew`. +Building on Mac (M1, M2 or M3) works, but you may need to install some prerequisites using `brew`. The below has been tested by one mac user and found to work. Note that this doesn't use Docker to run the server: +Install `xcode` from the Apps Store (needed for metalkit) + ``` # install build dependencies brew install abseil cmake go grpc protobuf wget @@ -146,8 +150,20 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso }' ``` -### Build with Image generation support +#### Troublshooting mac +If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store. +If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256). +If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again. + +``` +# reinstall build dependencies +brew reinstall abseil cmake go grpc protobuf wget + +make clean + +make build +``` **Requirements**: OpenCV, Gomp @@ -239,13 +255,12 @@ make BUILD_TYPE=sycl_f32 build # for float32 #### Metal (Apple Silicon) ``` -make BUILD_TYPE=metal build +make build -# Set `gpu_layers: 1` to your YAML model config file and `f16: true` -# Note: only models quantized with q4_0 are supported! +# correct build type is automatically used on mac (BUILD_TYPE=metal) +# Set `gpu_layers: 256` (or equal to the number of model layers) to your YAML model config file and `f16: true` ``` - ### Windows compatibility Make sure to give enough resources to the running container. See https://github.com/go-skynet/LocalAI/issues/2