LLM co-operative server + client package (Claude support via deno)

[bfollington]

Wrap an LLM behind a Deno webserver.

Create a thread and append messages to it, support for tool calling on the client and server in cooperation.

Client Input: what is 3+1? add some emojis to the answer using a tool
Client Tool call: calculator { expression: "3+1" }
Client Tool result: 4
Server Tool call: addEmojis { text: "The answer is 4" }
Server Tool result: Here's the text with relevant emojis added:\n\nThe answer is 4️⃣ ✅
Server Response: Here's the text with relevant emojis added:\n\nThe answer is 4️⃣ ✅

Example client usage:

import { LLMClient } from "@commontools/llm-client";
const client = new LLMClient({
  serverUrl: "http://localhost:8000",
  tools: [{
      name: "calculator",
      input_schema: {
        type: "object",
        properties: {
          expression: {
            type: "string",
            description: "A mathematical expression to evaluate",
          },
        },
        required: ["expression"],
      },
      implementation: async ({ expression }) => {
        return `${await eval(expression)}`;
      },
    }],
  system: "use your tools to answer the request"
});

client
  .handleConversation("calculate 2+2 and then add emojis")
  .then((response) => {
    console.log(response);
  });