mirror of
https://github.com/ianarawjo/ChainForge.git
synced 2025-03-16 00:56:00 +00:00
b33397930b
* Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
1 line
18 KiB
Plaintext
1 line
18 KiB
Plaintext
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-tracking-shuffled-objects", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "Alice, Bob, Claire, Dave, and Eve are playing a game. Output the color of the ball in lowercase without explaination.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "Alice, Bob, Claire, Dave, and Eve are playing a game. Output the color of the ball in lowercase without explaination.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-tracking-shuffled-objects", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideal = response.meta['Ideal'];\n\treturn response.text.startsWith(ideal);\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-tracking-shuffled-objects", "type": "vis", "data": {"input": "eval-tracking-shuffled-objects"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-tracking-shuffled-objects", "type": "inspect", "data": {"input": "prompt-tracking-shuffled-objects"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-tracking-shuffled-objects", "type": "table", "data": {"rows": [{"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a red ball, Dave has a purple ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Dave swap balls. Then, Claire and Eve swap balls. Then, Alice and Bob swap balls. Then, Dave and Claire swap balls. Finally, Alice and Claire swap balls. At the end of the game, what color is Alice's ball?", "ideal": "pink"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a red ball, Dave has a purple ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Dave swap balls. Then, Claire and Eve swap balls. Then, Alice and Bob swap balls. Then, Dave and Claire swap balls. Finally, Alice and Claire swap balls. At the end of the game, what color is Bob's ball?", "ideal": "purple"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a red ball, Dave has a purple ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Dave swap balls. Then, Claire and Eve swap balls. Then, Alice and Bob swap balls. Then, Dave and Claire swap balls. Finally, Alice and Claire swap balls. At the end of the game, what color is Claire's ball?", "ideal": "white"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a red ball, Dave has a purple ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Dave swap balls. Then, Claire and Eve swap balls. Then, Alice and Bob swap balls. Then, Dave and Claire swap balls. Finally, Alice and Claire swap balls. At the end of the game, what color is Dave's ball?", "ideal": "yellow"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a red ball, Dave has a purple ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Dave swap balls. Then, Claire and Eve swap balls. Then, Alice and Bob swap balls. Then, Dave and Claire swap balls. Finally, Alice and Claire swap balls. At the end of the game, what color is Eve's ball?", "ideal": "red"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a green ball, Dave has a brown ball, and Eve has a orange ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Bob swap balls. Then, Claire and Eve swap balls. Then, Alice and Eve swap balls. Then, Dave and Alice swap balls. Finally, Bob and Alice swap balls. At the end of the game, what color is Alice's ball?", "ideal": "orange"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a green ball, Dave has a brown ball, and Eve has a orange ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Bob swap balls. Then, Claire and Eve swap balls. Then, Alice and Eve swap balls. Then, Dave and Alice swap balls. Finally, Bob and Alice swap balls. At the end of the game, what color is Bob's ball?", "ideal": "brown"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a green ball, Dave has a brown ball, and Eve has a orange ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Bob swap balls. Then, Claire and Eve swap balls. Then, Alice and Eve swap balls. Then, Dave and Alice swap balls. Finally, Bob and Alice swap balls. At the end of the game, what color is Claire's ball?", "ideal": "white"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a green ball, Dave has a brown ball, and Eve has a orange ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Bob swap balls. Then, Claire and Eve swap balls. Then, Alice and Eve swap balls. Then, Dave and Alice swap balls. Finally, Bob and Alice swap balls. At the end of the game, what color is Dave's ball?", "ideal": "green"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a pink ball, Bob has a white ball, Claire has a green ball, Dave has a brown ball, and Eve has a orange ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Bob swap balls. Then, Claire and Eve swap balls. Then, Alice and Eve swap balls. Then, Dave and Alice swap balls. Finally, Bob and Alice swap balls. At the end of the game, what color is Eve's ball?", "ideal": "pink"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a blue ball, Bob has a orange ball, Claire has a pink ball, Dave has a green ball, and Eve has a brown ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Eve swap balls. Then, Eve and Dave swap balls. Then, Dave and Alice swap balls. Then, Alice and Claire swap balls. Finally, Bob and Claire swap balls. At the end of the game, what color is Alice's ball?", "ideal": "pink"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a blue ball, Bob has a orange ball, Claire has a pink ball, Dave has a green ball, and Eve has a brown ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Eve swap balls. Then, Eve and Dave swap balls. Then, Dave and Alice swap balls. Then, Alice and Claire swap balls. Finally, Bob and Claire swap balls. At the end of the game, what color is Bob's ball?", "ideal": "orange"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a blue ball, Bob has a orange ball, Claire has a pink ball, Dave has a green ball, and Eve has a brown ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Eve swap balls. Then, Eve and Dave swap balls. Then, Dave and Alice swap balls. Then, Alice and Claire swap balls. Finally, Bob and Claire swap balls. At the end of the game, what color is Claire's ball?", "ideal": "brown"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a blue ball, Bob has a orange ball, Claire has a pink ball, Dave has a green ball, and Eve has a brown ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Eve swap balls. Then, Eve and Dave swap balls. Then, Dave and Alice swap balls. Then, Alice and Claire swap balls. Finally, Bob and Claire swap balls. At the end of the game, what color is Dave's ball?", "ideal": "blue"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a blue ball, Bob has a orange ball, Claire has a pink ball, Dave has a green ball, and Eve has a brown ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Eve swap balls. Then, Eve and Dave swap balls. Then, Dave and Alice swap balls. Then, Alice and Claire swap balls. Finally, Bob and Claire swap balls. At the end of the game, what color is Eve's ball?", "ideal": "green"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a orange ball, Bob has a blue ball, Claire has a black ball, Dave has a white ball, and Eve has a purple ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Dave swap balls. Then, Claire and Eve swap balls. Then, Dave and Eve swap balls. Then, Claire and Dave swap balls. Finally, Alice and Bob swap balls. At the end of the game, what color is Alice's ball?", "ideal": "white"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a orange ball, Bob has a blue ball, Claire has a black ball, Dave has a white ball, and Eve has a purple ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Dave swap balls. Then, Claire and Eve swap balls. Then, Dave and Eve swap balls. Then, Claire and Dave swap balls. Finally, Alice and Bob swap balls. At the end of the game, what color is Bob's ball?", "ideal": "orange"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a orange ball, Bob has a blue ball, Claire has a black ball, Dave has a white ball, and Eve has a purple ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Dave swap balls. Then, Claire and Eve swap balls. Then, Dave and Eve swap balls. Then, Claire and Dave swap balls. Finally, Alice and Bob swap balls. At the end of the game, what color is Claire's ball?", "ideal": "black"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a orange ball, Bob has a blue ball, Claire has a black ball, Dave has a white ball, and Eve has a purple ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Dave swap balls. Then, Claire and Eve swap balls. Then, Dave and Eve swap balls. Then, Claire and Dave swap balls. Finally, Alice and Bob swap balls. At the end of the game, what color is Dave's ball?", "ideal": "purple"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a orange ball, Bob has a blue ball, Claire has a black ball, Dave has a white ball, and Eve has a purple ball. \n\nAs the game progresses, pairs of players trade balls. First, Bob and Dave swap balls. Then, Claire and Eve swap balls. Then, Dave and Eve swap balls. Then, Claire and Dave swap balls. Finally, Alice and Bob swap balls. At the end of the game, what color is Eve's ball?", "ideal": "blue"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a black ball, Bob has a brown ball, Claire has a blue ball, Dave has a green ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Claire swap balls. Then, Bob and Dave swap balls. Then, Eve and Alice swap balls. Then, Claire and Bob swap balls. Finally, Alice and Eve swap balls. At the end of the game, what color is Alice's ball?", "ideal": "black"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a black ball, Bob has a brown ball, Claire has a blue ball, Dave has a green ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Claire swap balls. Then, Bob and Dave swap balls. Then, Eve and Alice swap balls. Then, Claire and Bob swap balls. Finally, Alice and Eve swap balls. At the end of the game, what color is Bob's ball?", "ideal": "yellow"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a black ball, Bob has a brown ball, Claire has a blue ball, Dave has a green ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Claire swap balls. Then, Bob and Dave swap balls. Then, Eve and Alice swap balls. Then, Claire and Bob swap balls. Finally, Alice and Eve swap balls. At the end of the game, what color is Claire's ball?", "ideal": "green"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a black ball, Bob has a brown ball, Claire has a blue ball, Dave has a green ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Claire swap balls. Then, Bob and Dave swap balls. Then, Eve and Alice swap balls. Then, Claire and Bob swap balls. Finally, Alice and Eve swap balls. At the end of the game, what color is Dave's ball?", "ideal": "brown"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a black ball, Bob has a brown ball, Claire has a blue ball, Dave has a green ball, and Eve has a yellow ball. \n\nAs the game progresses, pairs of players trade balls. First, Eve and Claire swap balls. Then, Bob and Dave swap balls. Then, Eve and Alice swap balls. Then, Claire and Bob swap balls. Finally, Alice and Eve swap balls. At the end of the game, what color is Eve's ball?", "ideal": "blue"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a brown ball, Bob has a purple ball, Claire has a green ball, Dave has a white ball, and Eve has a blue ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Claire swap balls. Then, Dave and Alice swap balls. Then, Bob and Eve swap balls. Then, Claire and Eve swap balls. Finally, Alice and Dave swap balls. At the end of the game, what color is Alice's ball?", "ideal": "green"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a brown ball, Bob has a purple ball, Claire has a green ball, Dave has a white ball, and Eve has a blue ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Claire swap balls. Then, Dave and Alice swap balls. Then, Bob and Eve swap balls. Then, Claire and Eve swap balls. Finally, Alice and Dave swap balls. At the end of the game, what color is Bob's ball?", "ideal": "blue"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a brown ball, Bob has a purple ball, Claire has a green ball, Dave has a white ball, and Eve has a blue ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Claire swap balls. Then, Dave and Alice swap balls. Then, Bob and Eve swap balls. Then, Claire and Eve swap balls. Finally, Alice and Dave swap balls. At the end of the game, what color is Claire's ball?", "ideal": "purple"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a brown ball, Bob has a purple ball, Claire has a green ball, Dave has a white ball, and Eve has a blue ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Claire swap balls. Then, Dave and Alice swap balls. Then, Bob and Eve swap balls. Then, Claire and Eve swap balls. Finally, Alice and Dave swap balls. At the end of the game, what color is Dave's ball?", "ideal": "white"}, {"prompt": "At the start of the game, they are each holding a ball: Alice has a brown ball, Bob has a purple ball, Claire has a green ball, Dave has a white ball, and Eve has a blue ball. \n\nAs the game progresses, pairs of players trade balls. First, Alice and Claire swap balls. Then, Dave and Alice swap balls. Then, Bob and Eve swap balls. Then, Claire and Eve swap balls. Finally, Alice and Dave swap balls. At the end of the game, what color is Eve's ball?", "ideal": "brown"}], "columns": [{"key": "prompt", "header": "Prompt"}, {"key": "ideal", "header": "Ideal"}]}, "position": {"x": -16, "y": 160}, "selected": false, "positionAbsolute": {"x": -16, "y": 160}, "dragging": false}], "edges": [{"source": "prompt-tracking-shuffled-objects", "sourceHandle": "prompt", "target": "eval-tracking-shuffled-objects", "targetHandle": "responseBatch", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-prompt-1686756357355prompt-eval-1686756357355responseBatch"}, {"source": "prompt-tracking-shuffled-objects", "sourceHandle": "prompt", "target": "inspect-tracking-shuffled-objects", "targetHandle": "input", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-prompt-1686756357355prompt-inspect-1686756357355input"}, {"source": "eval-tracking-shuffled-objects", "sourceHandle": "output", "target": "vis-tracking-shuffled-objects", "targetHandle": "input", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-eval-1686756357355output-vis-1686756357355input"}, {"source": "table-tracking-shuffled-objects", "sourceHandle": "Prompt", "target": "prompt-tracking-shuffled-objects", "targetHandle": "prompt", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-table-1686756385002Prompt-prompt-1686756357355prompt"}], "viewport": {"x": 144, "y": 37, "zoom": 1}}, "cache": {"eval-1686756357355.json": {}, "inspect-1686756357355.json": {}, "prompt-1686756357355.json": {}, "table-1686756385002.json": {}, "vis-1686756357355.json": {}}} |