Run LLM from Linux Command Line
This revision is from 2024/07/16 20:28. You can Restore it.
python3 -m pip install --upgrade pip
pip install transformers
Python script:
import os
from transformers import pipeline
model_path = "/path/to/Lexi-Llama-3-8B-Uncensored_Q8_0.gguf"
prompt = "What is the meaning of life?"
# Load the modelllm = pipeline('text-generation', model=model_path)
# Generate a responseresponse = llm(prompt)
# Print the responseprint(response[0]['generated_text'])