ChainForge/chainforge/oaievals/internal_representations.cforge

1 line
9.0 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-internal_representations", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "You will be prompted with a logical question. Respond only with a number, nothing else is required.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "You will be prompted with a logical question. Respond only with a number, nothing else is required.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-internal_representations", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideal = response.meta['Ideal'];\n\treturn response.text.startsWith(ideal);\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-internal_representations", "type": "vis", "data": {"input": "eval-internal_representations"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-internal_representations", "type": "inspect", "data": {"input": "prompt-internal_representations"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-internal_representations", "type": "table", "data": {"rows": [{"prompt": "Bob has three apples. Alice has one apple. Alice trades Bob 1 apple for 2 apples in return. Bob eats one apple. How many apples does Alice have?", "ideal": "2"}, {"prompt": "Bob has three apples. Bob eats 2 apples. Alice gives Bob 1 apple. How many apples does Bob have left?", "ideal": "2"}, {"prompt": "There are five trees. One tree dies, two more are cut down, but two more trees are planted. However, the trees will not grow for another year. How many trees are currently alive?", "ideal": "2"}, {"prompt": "Bob has three apples. Bob eats 2 apples. Alice gives Bob 1 apple. How many apples does Bob have left?", "ideal": "2"}, {"prompt": "There are five cats in a room. Two cats die, another leaves the room, and two more cats enter the room. How many living cats are there in the room?", "ideal": "4"}, {"prompt": "There are ten cats in a room. Three cats die, one cat eats food, and two more cats enter the room. How many living cats are there in the room?", "ideal": "9"}, {"prompt": "Roger and Emily are in a room. Roger puts a cat in an empty box while Emily is in the room. They both leave, and Emily returns and puts a turtle in the box as well. How many items does Roger think are in the box?", "ideal": "1"}, {"prompt": "There are four darts already on a dart board. Bob throws one dart at the dartboard, and it lands. Alice throws two darts, and one misses but one hits the dartboard. How many darts are on the dartboard?", "ideal": "6"}, {"prompt": "A bakery sold 25 cakes in the morning and 17 cakes in the afternoon. The day after, they sold 15 more cakes. How many cakes did they sell on the second day?", "ideal": "15"}, {"prompt": "There are six blue birds and three red birds sitting on a tree. Two blue birds fly away, and one red bird flies away. How many birds are left on the tree?", "ideal": "6"}, {"prompt": "John has double the amount of pencils than Sarah, and Sarah has as many pencils as Mark. Mark has 2 pencils. How many pencils does John have?", "ideal": "4"}, {"prompt": "Bob has two apples. Alice has three apples. Alice trades Bob 1 apple for 2 apples in return. Bob eats one apple. How many apples does Bob have?", "ideal": "0"}, {"prompt": "There are seven trees in the park