ExternalVendorCode/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2024-12-18 20:27:57 +00:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity

docs(mac): improve documentation for mac build (#1873)

Browse Source 
* docs(mac): Improve documentation for mac build

- added documentation to build from current master
- added troubleshooting information

Signed-off-by: Sebastian <tauven@gmail.com>

* docs(max): fix typo

Signed-off-by: Sebastian <tauven@gmail.com>

---------

Signed-off-by: Sebastian <tauven@gmail.com>
This commit is contained in:
Sebastian 2024-03-21 22:08:33 +01:00 committed by GitHub
parent 3cf64d1e7e
commit 743095b7d8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 21 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
docs/content/docs/getting-started/build.md
View File

@ -45,6 +45,8 @@ To install the dependencies follow the instructions below:
{{< tabs tabTotal="3" >}}
{{% tab tabName="Apple" %}}
Install `xcode` from the App Store
```bash
brew install abseil cmake go grpc protobuf wget
```
@ -111,10 +113,12 @@ docker run --rm -ti -p 8080:8080 -e DEBUG=true -e MODELS_PATH=/models -e THREADS
### Example: Build on mac
Building on Mac (M1 or M2) works, but you may need to install some prerequisites using `brew`.
Building on Mac (M1, M2 or M3) works, but you may need to install some prerequisites using `brew`.
The below has been tested by one mac user and found to work. Note that this doesn't use Docker to run the server:
Install `xcode` from the Apps Store (needed for metalkit)
```
# install build dependencies
brew install abseil cmake go grpc protobuf wget
@ -146,8 +150,20 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso
}'
```
### Build with Image generation support
#### Troublshooting mac
If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store.
If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256).
If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again.
```
# reinstall build dependencies
brew reinstall abseil cmake go grpc protobuf wget
make clean
make build
```
**Requirements**: OpenCV, Gomp
@ -239,13 +255,12 @@ make BUILD_TYPE=sycl_f32 build # for float32
#### Metal (Apple Silicon)
```
make BUILD_TYPE=metal build
make build
# Set `gpu_layers: 1` to your YAML model config file and `f16: true`
# Note: only models quantized with q4_0 are supported!
# correct build type is automatically used on mac (BUILD_TYPE=metal)
# Set `gpu_layers: 256` (or equal to the number of model layers) to your YAML model config file and `f16: true`
```
### Windows compatibility
Make sure to give enough resources to the running container. See https://github.com/go-skynet/LocalAI/issues/2