Connect with us

Code

LLMFarm – llama and other large language models on iOS and MacOS offline

LLMFarm is an iOS and MacOS app to work with large language models (LLM). It allows you to load different LLMs with certain parameters.With LLMFarm, you can test the performance of different LLMs on iOS and macOS and find the most suitable model for your project.

Based on ggml and llama.cpp by Georgi Gerganov.

LLMFarm Features

  •  MacOS (13+)
  •  iOS (16+)
  •  Various inferences
  •  Various sampling methods
  •  Metal (dont work on intel Mac)
  •  Model setting templates
  •  LoRA adapters support (read more)
  •  LoRA finetune support
  •  LoRA export as model support
  •  Restore context state (now only chat history)

Inferences

Note: For Falcon, Alpaca, GPT4All, Chinese LLaMA / Alpaca and Chinese LLaMA-2 / Alpaca-2, Vigogne (French), Vicuna, Koala, OpenBuddy (Multilingual), Pygmalion/Metharme, WizardLM, Baichuan 1 & 2 + derivations, Aquila 1 & 2, Mistral AI v0.1, Refact, Persimmon 8B, MPT, Bloom select llama inferece in model settings.

Sampling methods

Getting Started

You can find answers to some questions in the FAQ section.

Inference options

When creating a chat, a JSON file is generated in which you can specify additional inference options. The chat files are located in the “chats” directory. You can see all inference options here.

Models

You can download some of the supported models here.

LLMFarm on GitHub: https://github.com/guinmoon/LLMFarm
Platform: iOS
⭐️: 578
Advertisement

Trending