feat: add llm.prompt

[teamdandelion]

This is a first stab at implementing the llm.prompt api, as discussed in #884, particularly #884 (comment).

It is basically a sibling to llm.call that doesn't require providing the model and provider upfront, but rather assumes that they will be present via the llm.context context manager. Rather than re-implement the logic, under the hood it has some minimal wrapping and then calls llm.call. Once we implement performance benchmarking (as discussed in #896), we may find it's better not to wrap llm.call here, but for a proof of concept for the interface, this seemed fine.

I copied a lot of the (complex) type signature for llm.call, including making a PromptDecorator class, which may be cargo cult-y.

Initially, rather than taking a simple decorator approach, I wanted to return a callable Prompt class. This would have enabled nice APIs like chaining function calls on the prompt in order to change its structural parameters; e.g. if prompt.stream(True) would return a new Prompt that has streaming enabled, or prompt.response_model(Book) would make a new version of the same prompt with a response model configured. However, I found this to be a nightmare from a typing perspective, so I backed off to just implementing a decorator a-la call.

I think there's value in having a way to set up decorated llm calls while not providing the provider or model upfront, hence advocating for this API. Though, I wonder if having a separate llm.prompt api distinct from llm.call is worth it, versus just making the provider and model optional on llm.call, with an error thrown at runtime if no default model/provider are present, and it was called outside of a context.

I didn't update docs or examples in this PR, however here's a code snippet which both works and puts llm.prompt a bit through the paces:

from mirascope import llm
from pydantic import BaseModel


# Simple example using the prompt decorator
@llm.prompt()
def recommend_book(genre: str) -> str:
    """Recommend a book in the given genre."""
    return f"Recommend a {genre} book."


# Example with response model
class Book(BaseModel):
    """An extracted book recommendation."""

    title: str
    author: str
    year: int
    description: str


@llm.prompt(response_model=Book)
def extract_book(genre: str) -> str:
    """Extract a structured book recommendation."""
    return f"""
    Recommend a {genre} book. Respond with a JSON object with the following fields:
    - title: the title of the book
    - author: the author of the book
    - year: the year the book was published
    - description: a brief description of the book
    """


# Example with streaming
@llm.prompt(stream=True)
def stream_book_review(book_title: str) -> str:
    """Generate a streaming review for the given book."""
    return f"Write a detailed review of the book '{book_title}'."


def main():
    print("Provider-agnostic LLM prompts with mirascope.llm.prompt\n")

    # Example 1: Simple prompt with OpenAI
    print("Example 1: Simple book recommendation with OpenAI")
    with llm.context(provider="openai", model="gpt-4o-mini"):
        openai_response: llm.CallResponse = recommend_book("fantasy")
        print(f"OpenAI Response: {openai_response.content}")

    # Example 2: Same prompt with Anthropic
    print("\nExample 2: Same prompt with Anthropic")
    with llm.context(provider="anthropic", model="claude-3-haiku-20240307"):
        anthropic_response: llm.CallResponse = recommend_book("science fiction")
        print(f"Anthropic Response: {anthropic_response.content}")

    # Example 3: Using a structured response model
    print("\nExample 3: Structured response with OpenAI")
    with llm.context(provider="openai", model="gpt-4o-mini"):
        book: Book = extract_book("mystery")
        print(f"Extracted Book: {book}")
        print(f"Book Title: {book.title}")
        print(f"Book Author: {book.author}")

    # Example 4: Streaming response
    print("\nExample 4: Streaming response")
    with llm.context(provider="openai", model="gpt-4o-mini"):
        stream: llm.Stream = stream_book_review("The Lord of the Rings")
        print("Streaming review:")
        for chunk, _ in stream:
            # Print without newline and flush to show streaming effect
            print(chunk.content, end="", flush=True)
        print()  # Add a newline at the end


if __name__ == "__main__":
    main()

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (1ad5c05) to head (6f12ffe).

Additional details and impacted files

@@            Coverage Diff             @@
##              main      #930    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files          515       517     +2     
  Lines        21070     21382   +312     
==========================================
+ Hits         21070     21382   +312     

Flag	Coverage Δ
tests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[willbakst]

nit: refactor this into a private utility with the above duplicate (just so the messages will always match)

[teamdandelion]

Done

[willbakst]

I see your point here now about whether we need llm.prompt or if llm.call should just make provider/model optional and raise the error if not provided at runtime, especially since the return types aren't different so the only way to know you need context is to see it has the llm.prompt decorator (at which point you could see that it doesn't have model/provider). Could also potentially update to return something like RequiresContext[...] as a type wrapper passthrough or something if model or provider aren't provided.

[teamdandelion]

Yeah, and that puts us back into the issue of not having a way for the type system to check if a context is provided. Also, even RequiresContext is underspecified because model and provider are optional on the context. Really we require a context that has both model and provider... which hypothetically could be coming from two separate contexts.

[willbakst]

Hmm, I think this will actually be moot somewhat when we merge provider and model.

In the short term, we could provide the following type aliases:

RequiresContext[T]: TypeAlias = T
RequiresProvider[T]: TypeAlias = T
RequiresModel[T]: TypeAlias = T

We would then remove RequiresProvider and RequiresModel as part of the merge.

I imagine these do little but let someone know that it requires context right where they are in the code.

I think this would be helpful/useful even without explicit lint errors.