How do I connect it to a local llama.cpp instance? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

dur-randir 4 months ago | parent | context | favorite | on: Show HN: OSS Agent I built topped the TerminalBenc...

How do I connect it to a local llama.cpp instance?

GodelNumbering 4 months ago [–]

It supports LMStudio or you can start a local endpoint, then run

OPENAI_COMPATIBLE_CUSTOM_KEY="xxx" dirac -y --provider "https://localhost/v1" --model <model_name> "hi..."

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact