ChainForge/chainforge/oaievals/arithmetical_puzzles.cforge

1 line
14 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-arithmetical_puzzles", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "Complete the phrase as concisely as possible.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "Complete the phrase as concisely as possible.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-arithmetical_puzzles", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideals = JSON.parse(response.meta['Ideal']);\n\treturn ideals.some(i => response.text.startsWith(i));\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-arithmetical_puzzles", "type": "vis", "data": {"input": "eval-arithmetical_puzzles"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-arithmetical_puzzles", "type": "inspect", "data": {"input": "prompt-arithmetical_puzzles"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-arithmetical_puzzles", "type": "table", "data": {"rows": [{"prompt": "Tommy: \"How old are you, mamma?\" Mamma: \"Let me think, Tommy. Well, our three ages add up to exactly seventy years.\" Tommy: \"That's a lot, isn't it? And how old are you, papa?\" Papa: \"Just six times as old as you, my son.\" Tommy: \"Shall I ever be half as old as you, papa?\" Papa: \"Yes, Tommy; and when that happens our three ages will add up to exactly twice as much as to-day.\" Tommy: \"And supposing I was born before you, papa; and supposing mamma had forgot all about it, and hadn't been at home when I came; and supposing\" Mamma: \"Supposing, Tommy, we talk about bed. Come along, darling. You'll have a headache.\" Now, if Tommy had been some years older he might have calculated the exact ages of his parents from the information they had given him. Can you find out the exact age of mamma in months? Please give only the numerical answer as a single integer indicating the number of months. Do not give any reasoning or logic for your answer.", "ideal": "[\"350\"]"}, {"prompt": "\"My husband's age,\" remarked a lady the other day, \"is represented by the figures of my own age reversed. He is my senior, and the difference between our ages is one-eleventh of their sum.\" What is the husband's age? Please give only the numerical answer as a single integer indicating the number of years. Do not give any reasoning or logic for your answer.", "ideal": "[\"54\"]"}, {"prompt": "When the Smileys recently received a visit from the favourite uncle, the fond parents had all the five children brought into his presence. First came Billie and little Gertrude, and the uncle was informed that the boy was exactly twice as old as the girl. Then Henrietta arrived, and it was pointed out that the combined ages of herself and Gertrude equalled twice the age of Billie. Then Charlie came running in, and somebody remarked that now the combined ages of the two boys were exactly twice the combined ages of the two girls. The uncle was expressing his astonishment at these coincidences when Janet came in. \"Ah! uncle,\" she exclaimed, \"you have actually arrived on my twenty-first birthday!\" To this Mr. Smiley added the final staggerer: \"Yes, and now the combined ages of the three girls are exactly equal to twice the combined ages of the two boys.\" Please give the age of Billie. Please give only the nu