We value your privacy
This site uses cookies to improve your browsing experience, analyze site traffic, and show personalized content.
User and Developer Documentation, a Large-scale LLM inference engine. Breathing Life Into Language Aphrodite Engine is an advanced inference software developed by PygmalionAI for efficiently running large language models (LLMs) at scale. Built upon vLLM, it supports a wide range of Hugging Face models, including Llama, Mistral, and Mixtral, and accommodates various weight quantization methods to optimize performance across different deployment scenarios.