Anthropic (Claude) provider #22

brichet · 2024-12-09T09:32:29Z

This PR adds the Anthropic provider as a chat model.
~~It does not include completion.~~

brichet · 2024-12-18T13:56:32Z

I added a first version of code completion.
The idea is to give a context to the AI, and to send the current prompt as a message to complete.
The 2 issues I have so far are:

it is really slow on my side
anthropic raises if there is a space at the end of the prompt. Currently this PR is removing it, but it can lead to wrong suggestions

jtpio · 2024-12-19T17:14:55Z

Thanks @brichet for looking into this 👍

I just tried this PR locally and there seems to be many requests made to the anthropic API, so maybe there is indeed something not expected happening:

jupyterlite-anthropic.webm

jtpio · 2024-12-19T17:22:20Z

Interestingly when testing with Jupyter AI, completions don't seem to work and trigger some warnings in the dev tools console:

jupyter-ai-lab-434-anthropic.webm

The Jupyter Server logs also show some long response time for the inline completer:

brichet · 2024-12-20T13:09:41Z

Thanks for testing it @jtpio

Using Anthropic completion with jupyter-ai do not trigger error on my side, but the suggestions are not really relevant (most of the time):

Indeed, you're right about the extra requests, I'll take a closer look.

jtpio · 2024-12-20T13:38:47Z

Using Anthropic completion with jupyter-ai do not trigger error on my side, but the suggestions are not really relevant (most of the time):

Indeed according to your screenshot there seems to be an issue with how Jupyter AI handles the inline completer suggestions. So likely an issue in Jupyter AI (we could open one if non exists already)

brichet · 2025-01-21T21:49:54Z

Looks like the default model is claude-2.1, which seems to be slower and less accurate than claude-3-5-sonnet-20241022. Setting it manually improve the performances of the code completion.

Maybe we should revive #14 to get all the settings for each providers.

jtpio · 2025-01-22T07:54:12Z

Maybe we should revive #14 to get all the settings for each providers.

Yeah the JupyterLab related change should now be available in 4.4.0a2.

jtpio · 2025-01-22T08:31:52Z

Looks like the default model is claude-2.1, which seems to be slower and less accurate than claude-3-5-sonnet-20241022. Setting it manually improve the performances of the code completion.

Indeed, I just tried it locally and it seems to be faster with Claude 3.5 Sonnet.

There seems to be some issues with the quality of completion being returned sometimes, but it's something that can likely be fixed by improving the prompt or the use of the langchain primitives:

brichet · 2025-01-23T14:47:44Z

Unfortunately #14 does not work out of the box for Anthropic provider.

jtpio · 2025-01-24T11:18:27Z

Unfortunately #14 does not work out of the box for Anthropic provider.

Looks like the settings generator script needs to be updated for each new provided being added?

brichet · 2025-01-24T12:24:51Z

Looks like the settings generator script needs to be updated for each new provided being added?

Yes, but it would be great if we could have a generic way to retrieve the available settings.

If @langchainjs is using a naming convention for the modules and the "input" interfaces we can make it more generic, but it seems to not be the case (eg. ChatMistralAIInput and AnthropicInput).

The other problem is about the use of ts-json-schema-generator, which raises an error when trying to build the settings for Anthropic without excluding clientOptions (this king of error seems to not be handled correctly):

/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/NodeParser/TypeReferenceNodeParser.js:59
        return this.childNodeParser.createType(typeSymbol.declarations.filter((n) => !invalidTypes[n.kind])[0], this.createSubContext(node, context));
                                                                       ^

TypeError: Cannot read properties of undefined (reading 'filter')
    at TypeReferenceNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/NodeParser/TypeReferenceNodeParser.js:59:72)
    at ChainNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/ChainNodeParser.js:30:45)
    at TypeReferenceNodeParser.createSubContext (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/NodeParser/TypeReferenceNodeParser.js:65:62)
    at TypeReferenceNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/NodeParser/TypeReferenceNodeParser.js:59:118)
    at ChainNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/ChainNodeParser.js:30:45)
    at TypeAliasNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/NodeParser/TypeAliasNodeParser.js:36:43)
    at AnnotatedNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/NodeParser/AnnotatedNodeParser.js:29:47)
    at ExposeNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/ExposeNodeParser.js:24:45)
    at CircularReferenceNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/CircularReferenceNodeParser.js:22:43)
    at ChainNodeParser.createType (/home/brichet/projects/jupyterlite-ai/node_modules/ts-json-schema-generator/dist/src/ChainNodeParser.js:30:45)

In conclusion, for now I think we'll have to update the generator, but hopefully it should be painless next time 😄

jtpio · 2025-01-24T20:24:44Z

scripts/settings-generator.js

+  let schema;
+
+  // Workaround to exclude some properties from a type or interface.
+  if (desc.excludedProps) {


It's probably fine to have exclusion clauses like this for now as we figure things out with several providers 👍

Also found that some props had to be excluded over in #27.

jtpio · 2025-01-24T20:26:20Z

src/llm-models/anthropic-completer.ts

+    // Anthropic does not allow whitespace at the end of the AIMessage
+    const trimmedPrompt = prompt.trim();
+
+    const messages = [


Wondering if this list of messages should be recreated each time, or it should be persisted per provider?

Some other open questions: should the list of messages be shared across providers? Or should it be reset when a user switches to another provider?

Maybe a reset would be fine to keep things tidy. Also users probably wouldn't change providers often in practice?

jtpio

Thanks!

Testing locally and it seems to be working fine 👍

By default the inline completion does seem to be quite slow:

jupyterlite-ai-anthropic-default.mov

But switching to claude-3-5-sonnet-latest as the model does seem to be making things faster:

jupyterlite-ai-anthropic-35.mov

Left a couple of inline comments, that we can also track separately.

brichet · 2025-01-24T21:18:58Z

Thanks for the review @jtpio

brichet added the enhancement New feature or request label Dec 9, 2024

brichet mentioned this pull request Dec 10, 2024

Fix anthropic sdk import langchain-ai/langchainjs#7334

Merged

brichet marked this pull request as ready for review December 16, 2024 12:09

brichet marked this pull request as draft December 17, 2024 12:36

brichet force-pushed the anthropic_provider branch from fbb3833 to 54f96ca Compare January 23, 2025 14:35

brichet marked this pull request as ready for review January 24, 2025 11:15

brichet added 7 commits January 24, 2025 17:09

Add Anthropic (Claude) chat provider

7ae0a5e

Bump package

ea31382

Add code completer based on Anthropic Chat

80c60e2

Update system message

7f3781a

Workaround to build settings for Anthropic

317fbbb

lint

dc8e602

Update README

b74d481

brichet force-pushed the anthropic_provider branch from 3096bc5 to b74d481 Compare January 24, 2025 16:10

jtpio reviewed Jan 24, 2025

View reviewed changes

jtpio approved these changes Jan 24, 2025

View reviewed changes

jtpio merged commit ba99ad6 into jupyterlite:main Jan 24, 2025
7 checks passed

brichet deleted the anthropic_provider branch January 24, 2025 21:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Anthropic (Claude) provider #22

Anthropic (Claude) provider #22

brichet commented Dec 9, 2024 •

edited

Loading

brichet commented Dec 18, 2024 •

edited

Loading

jtpio commented Dec 19, 2024

jtpio commented Dec 19, 2024

brichet commented Dec 20, 2024 •

edited

Loading

jtpio commented Dec 20, 2024

brichet commented Jan 21, 2025

jtpio commented Jan 22, 2025

jtpio commented Jan 22, 2025

brichet commented Jan 23, 2025

jtpio commented Jan 24, 2025

brichet commented Jan 24, 2025

jtpio Jan 24, 2025

jtpio Jan 24, 2025

jtpio Jan 24, 2025

jtpio Jan 24, 2025

jtpio left a comment

brichet commented Jan 24, 2025

Anthropic (Claude) provider #22

Anthropic (Claude) provider #22

Conversation

brichet commented Dec 9, 2024 • edited Loading

brichet commented Dec 18, 2024 • edited Loading

jtpio commented Dec 19, 2024

jtpio commented Dec 19, 2024

brichet commented Dec 20, 2024 • edited Loading

jtpio commented Dec 20, 2024

brichet commented Jan 21, 2025

jtpio commented Jan 22, 2025

jtpio commented Jan 22, 2025

brichet commented Jan 23, 2025

jtpio commented Jan 24, 2025

brichet commented Jan 24, 2025

jtpio Jan 24, 2025

Choose a reason for hiding this comment

jtpio Jan 24, 2025

Choose a reason for hiding this comment

jtpio Jan 24, 2025

Choose a reason for hiding this comment

jtpio Jan 24, 2025

Choose a reason for hiding this comment

jtpio left a comment

Choose a reason for hiding this comment

brichet commented Jan 24, 2025

brichet commented Dec 9, 2024 •

edited

Loading

brichet commented Dec 18, 2024 •

edited

Loading

brichet commented Dec 20, 2024 •

edited

Loading