generateData -> llm + lift methods

[anotherjesse]

generateData -> llm

• simplified generate code to just be about LLM conversations
• moves all the logic around prep / parsing / ... to recipe
• ergo - json / html mode is now handled by lift inside recipes
• example of using Zod / schemas to parse json results

llm improvements:

• streaming
• stop sequences
• support for multiple messages (user -> assistant -> user -> assistant)
• support for putting words into LLMs mouth (forcing it to generate json)
• support for (base64) images

common file input:

• added ability to load images / json data from file input

[anotherjesse]

todo:

• cleanup schema helpers / patterns (perhaps leverage zod?)
• @seefeldb had mentioned migrating generateData to being a recipe - I used lifts...
• is there a way to return multiple values from a lift?

        const { result: jsonResult } = generateText({
            messages: generateJSONMessages({ jsonprompt }),
            system: "You are a helpful assistant that generates JSON objects for testing.  Respond in JSON",
            stop: '```'
        });

replace that with something like:

        const { result: jsonResult } = generateText( prep({ jsonprompt }) )

perhaps this is the recipe idea @seefeldb had mentioned

[seefeldb]

Ah yes, when I meant convert to a recipe, I meant to a combination of recipe and lifted functions.

Lifted functions can return { multiple, things }, which you can then use in the recipe as const { multiple, things } = lifted_function(params).

[anotherjesse]

cool - it might be nice to do:

   const maybeHtml = parse(generateText(prep({ prompt })

[seefeldb]

Yes, this should work! From a caller's perspective there is no difference between a lifted function or a recipe, btw.

[seefeldb]

ship it! looks good overall and we can iterate over details later.