This revision is from 2024/07/16 20:29. You can Restore it.

python3 -m pip install --upgrade pip
pip install transformers
Copy

Python script:

import os
from transformers import pipeline

model_path = "/path/to/Lexi-Llama-3-8B-Uncensored_Q8_0.gguf"
prompt = "What is the meaning of life?"

# Load the model
llm = pipeline('text-generation', model=model_path)

# Generate a response
response = llm(prompt)

# Print the response
print(response[0]['generated_text'])
Copy

Execute:

python llm_script.py
Copy