Handling of messages

[jtpio]

Problem

Currently it looks like some providers recreate a list of messages for each completion request, for example:

ai/src/llm-models/anthropic-completer.ts

Lines 30 to 35 in ba99ad6

	const messages = [
	new SystemMessage(
	'You are a code-completion AI completing the following code from a Jupyter Notebook cell.'
	),
	new AIMessage(trimmedPrompt)
	];

Should be persisted per provider instead?

Proposed Solution

Should this list of messages be shared across providers? Or should it be reset when a user switches to another provider?

Maybe a reset would be fine to keep things tidy. Also users probably wouldn't change providers often in practice?

Additional context

This would also be relevant to other providers like #27.

[brichet]

The providers could also be improved by adding some context, like the existing code (in the cell, in the Notebook...).
But it would increase (a lot) the size of the completion requests.

[jtpio]

But it would increase (a lot) the size of the completion requests.

Indeed. Maybe that's something langchain can make easier to work with (would need to double check).

[jtpio]

Also, it looks like the current use of AIMessage could be replaced by HumanMessage (https://js.langchain.com/docs/concepts/messages/#humanmessage) since the related message is coming from a user?

[brichet]

Yes, the idea was to let the AI complete the sentence as if it had started it (I may have read that hack somewhere but can't find it again).

Indeed, with more context in an initial SystemMessage (like what is started here), we could switch to a HumanMessage.