Promptify

Task-based NLP engine with Pydantic structured outputs, built-in evaluation, and LiteLLM as the universal LLM backend. Think "scikit-learn for LLM-powered NLP".

Installation

With pip

Requires Python 3.9+.

pip install promptify

or

pip install git+https://github.com/promptslab/Promptify.git

For evaluation metrics support:

pip install promptify[eval]

Quick Tour

3-Line NER

from promptify import NER

ner = NER(model="gpt-4o-mini", domain="medical")
result = ner("The patient is a 93-year-old female with a medical history of chronic right hip pain, osteoporosis, hypertension, depression, and chronic atrial fibrillation admitted for evaluation and management of severe nausea and vomiting and urinary tract infection")

Output:

NERResult(entities=[
    Entity(text="93-year-old", label="AGE"),
    Entity(text="chronic right hip pain", label="CONDITION"),
    Entity(text="osteoporosis", label="CONDITION"),
    Entity(text="hypertension", label="CONDITION"),
    Entity(text="depression", label="CONDITION"),
    Entity(text="chronic atrial fibrillation", label="CONDITION"),
    Entity(text="severe nausea and vomiting", label="SYMPTOM"),
    Entity(text="urinary tract infection", label="CONDITION"),
])

Classification

from promptify import Classify

clf = Classify(model="gpt-4o-mini", labels=["positive", "negative", "neutral"])
result = clf("Amazing product! Best purchase I've ever made.")
# Classification(label="positive", confidence=0.95)

Question Answering

from promptify import QA

qa = QA(model="gpt-4o-mini")
answer = qa("Einstein was born in Ulm in 1879.", question="Where was Einstein born?")
# Answer(answer="Ulm", evidence="Einstein was born in Ulm", confidence=0.98)

Custom Task with Any Pydantic Schema

from promptify import Task
from pydantic import BaseModel

class MovieReview(BaseModel):
    sentiment: str
    rating: float
    key_themes: list[str]

task = Task(model="gpt-4o", output_schema=MovieReview, instruction="Analyze this movie review.")
review = task("Nolan's best work. Stunning visuals but the plot drags.")
# MovieReview(sentiment="mostly positive", rating=7.5, key_themes=["visuals", "pacing"])

Any Provider - Just Change the Model String

ner_openai = NER(model="gpt-4o-mini")
ner_claude = NER(model="claude-sonnet-4-20250514")
ner_local  = NER(model="ollama/llama3")

Batch Processing

results = ner.batch(["text1", "text2", "text3"], max_concurrent=10)

Async Support

result = await ner.acall("Patient has diabetes")

Built-in Evaluation

from promptify.eval import evaluate

scores = evaluate(task=ner, dataset=labeled_data, metrics=["precision", "recall", "f1"])
# {"precision": 0.92, "recall": 0.88, "f1": 0.90}

Features

• **2-3