Contexts and Sessions documentations

[LouiFi]

Hi mates,

Is there any documentation about how to use contexts and sessions ? I did not find anything but would love to use this feature.

Thank you for your answers.

[LaBalaDeOro]

@LouiFi, Here's what I've found. This applies to using the OpenAI API. YMMV with other APIs.

Both options are used to maintain conversation continuity.

Context

Reference
https://community.openai.com/t/how-can-i-maintain-conversation-continuity-with-chat-gpt-api/574819/2

Context Window: ChatGPT has a limited context window, meaning it can only see a certain amount of the most recent text. Ensure that you include relevant parts of the previous conversation in each new API request to maintain context.

https://github.com/danielmiessler/fabric/blob/main/patterns/suggest_pattern/user.md

--context, -c: Uses a Context file to add context to your pattern. Enhances the relevance of AI responses by providing additional background information.

Format
https://community.openai.com/t/how-to-format-context-documents-to-allow-model-to-recognize-specific-fields-within-documents/263459

Session

Reference
https://community.openai.com/t/how-can-i-maintain-conversation-continuity-with-chat-gpt-api/574819/2

Session Management: Utilize the session feature provided by the API. Each session can maintain a context for a certain period, allowing the model to remember previous interactions within that session.

@eugeis, it looks like sessions are broken in main now. In session.go the current code just prints a message and does not create the session like it used to: 75ee3ac#diff-d1e13f591e0e9045391d8585399ae66f1441c26574e7ade30aa961b7d4b09367

Format
I couldn't find specifics about the format, but sessions used to be stored as a json file, but are no longer. 75ee3ac#diff-d1e13f591e0e9045391d8585399ae66f1441c26574e7ade30aa961b7d4b09367

FWIW, --session=foo doesn't seem to work with --dry-run

HTH.

[Melaron]

@LouiFi

I have been wondering this myself for a long time. I assumed it would be documented eventually, but I've finally decided to experiment, and I find that context does what I thought it would do: sends the contents of a file (that you add yourself) from the ~/.config/fabric/contexts directory along with your query and pattern. I do not think fabric context has anything to do with token length context in LLMs. I did not look at sessions as I typically use Open WebUI for that, but I would love to read documentation on how it is intended to work in fabric.

Here I echo my query and pipe it to fabric along with a context file. I believe the context file is sent first and unaltered, based on this small experiment (newl is just an alias to add two new lines before and after each line for readability):

user@host:~$ alias | grep newl
alias newl='printf "\n\n"'
user@host:~$ newl ; echo 'repeat back the question I am asking you right now, and the context, in the same order in which they are received, whether context come first, or this query' | fabric -s --context=context0.md -p raw_query ; newl

The context provided is: "You are new to using LLMs and could use some guidance. You are familiar with basic IT terminology. You work as a cybersecurity analyst."

The question you are asking me is: "repeat back the question I am asking you right now, and the context, in the same order in which they are received, whether context come first, or this query."

user@host:~$ newl ; echo 'repeat back the question I am asking you right now, and the context, in the same order in which they are received, whether context come first, or this query' | fabric -s -p raw_query --context=context0.md ; newl

The context provided is:

• The user is new to using LLMs and could use some guidance.
• The user is familiar with basic IT terminology.
• The user works as a cybersecurity analyst.

The question asked is: "repeat back the question I am asking you right now, and the context, in the same order in which they are received, whether context come first, or this query."

The contents of context0.md seems to be passed as-is; for reference, this is what I wrote in the file:

### Context
- The user is new to using LLMs and could use some guidance.
- The user is familiar with basic IT terminology.
- The user works as a cybersecurity analyst.

My experiment focused on user contextual information that could swapped out for another, but you could include a context file for anything such as:

• Homelab/prod: explain your environment and get back answers more relevant to a homelab or prod.
• Cross-categories: a prompt like assess_risk can be modified with context based on the type of risk or entity involved.
• Audience: a summarization pattern can be adjusted for a layman/professional/executive audience without modifying the pattern itself.

The world is your oyster, but the real value here is flexibility through modulating all existing and custom patterns on the fly by adding context.