mirror of
https://github.com/mudler/LocalAI.git
synced 2024-12-18 20:27:57 +00:00
docs(mac): improve documentation for mac build (#1873)
* docs(mac): Improve documentation for mac build - added documentation to build from current master - added troubleshooting information Signed-off-by: Sebastian <tauven@gmail.com> * docs(max): fix typo Signed-off-by: Sebastian <tauven@gmail.com> --------- Signed-off-by: Sebastian <tauven@gmail.com>
This commit is contained in:
parent
3cf64d1e7e
commit
743095b7d8
@ -45,6 +45,8 @@ To install the dependencies follow the instructions below:
|
|||||||
{{< tabs tabTotal="3" >}}
|
{{< tabs tabTotal="3" >}}
|
||||||
{{% tab tabName="Apple" %}}
|
{{% tab tabName="Apple" %}}
|
||||||
|
|
||||||
|
Install `xcode` from the App Store
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
brew install abseil cmake go grpc protobuf wget
|
brew install abseil cmake go grpc protobuf wget
|
||||||
```
|
```
|
||||||
@ -111,10 +113,12 @@ docker run --rm -ti -p 8080:8080 -e DEBUG=true -e MODELS_PATH=/models -e THREADS
|
|||||||
|
|
||||||
### Example: Build on mac
|
### Example: Build on mac
|
||||||
|
|
||||||
Building on Mac (M1 or M2) works, but you may need to install some prerequisites using `brew`.
|
Building on Mac (M1, M2 or M3) works, but you may need to install some prerequisites using `brew`.
|
||||||
|
|
||||||
The below has been tested by one mac user and found to work. Note that this doesn't use Docker to run the server:
|
The below has been tested by one mac user and found to work. Note that this doesn't use Docker to run the server:
|
||||||
|
|
||||||
|
Install `xcode` from the Apps Store (needed for metalkit)
|
||||||
|
|
||||||
```
|
```
|
||||||
# install build dependencies
|
# install build dependencies
|
||||||
brew install abseil cmake go grpc protobuf wget
|
brew install abseil cmake go grpc protobuf wget
|
||||||
@ -146,8 +150,20 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso
|
|||||||
}'
|
}'
|
||||||
```
|
```
|
||||||
|
|
||||||
### Build with Image generation support
|
#### Troublshooting mac
|
||||||
|
|
||||||
|
If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store.
|
||||||
|
If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256).
|
||||||
|
If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again.
|
||||||
|
|
||||||
|
```
|
||||||
|
# reinstall build dependencies
|
||||||
|
brew reinstall abseil cmake go grpc protobuf wget
|
||||||
|
|
||||||
|
make clean
|
||||||
|
|
||||||
|
make build
|
||||||
|
```
|
||||||
|
|
||||||
**Requirements**: OpenCV, Gomp
|
**Requirements**: OpenCV, Gomp
|
||||||
|
|
||||||
@ -239,13 +255,12 @@ make BUILD_TYPE=sycl_f32 build # for float32
|
|||||||
#### Metal (Apple Silicon)
|
#### Metal (Apple Silicon)
|
||||||
|
|
||||||
```
|
```
|
||||||
make BUILD_TYPE=metal build
|
make build
|
||||||
|
|
||||||
# Set `gpu_layers: 1` to your YAML model config file and `f16: true`
|
# correct build type is automatically used on mac (BUILD_TYPE=metal)
|
||||||
# Note: only models quantized with q4_0 are supported!
|
# Set `gpu_layers: 256` (or equal to the number of model layers) to your YAML model config file and `f16: true`
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
### Windows compatibility
|
### Windows compatibility
|
||||||
|
|
||||||
Make sure to give enough resources to the running container. See https://github.com/go-skynet/LocalAI/issues/2
|
Make sure to give enough resources to the running container. See https://github.com/go-skynet/LocalAI/issues/2
|
||||||
|
Loading…
Reference in New Issue
Block a user