ChainForge/chainforge/oaievals/multistep-word-problems.cforge

1 line
7.4 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-multistep-word-problems", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "Solve the following questions. Answer with either a single whole number or a simple fraction. If units are used in the question, make sure the answer is in the same units used in the question, though do not include the units in the answer. Do not include additional text or explanation in your answer. You are a helpful assistant.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "Solve the following questions. Answer with either a single whole number or a simple fraction. If units are used in the question, make sure the answer is in the same units used in the question, though do not include the units in the answer. Do not include additional text or explanation in your answer. You are a helpful assistant.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-multistep-word-problems", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideal = response.meta['Ideal'];\n\treturn response.text.startsWith(ideal);\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-multistep-word-problems", "type": "vis", "data": {"input": "eval-multistep-word-problems"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-multistep-word-problems", "type": "inspect", "data": {"input": "prompt-multistep-word-problems"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-multistep-word-problems", "type": "table", "data": {"rows": [{"prompt": "Sarah has 23 apples, and she wants to divide them among her 5 friends such that each will get an equal amount of apples and she gets to keep twice as many as she hands out to each friend. How many apples are left over?", "ideal": "2"}, {"prompt": "A store is offering a 25% discount on a jacket that originally costs $120. The store is also offering another 20% discount on pants that cost half of the jacket. Jane has $200. How much will she have after buying both the jacket and the pants?", "ideal": "62"}, {"prompt": "A recipe calls for 1/2 cup of sugar. John wants to make half of the recipe, but also another recipe which will take three times as much sugar as full first recipe. If John only has enough sugar to make the second recipe in full but still wants to make half of the first recipe, what fraction of the second recipe can John make?", "ideal": "5/6"}, {"prompt": "There are two circles. The second circle has twice the circumference of the first circle. The diameter of both circles add up to 20. What is the diameter of the first circle?", "ideal": "20/3"}, {"prompt": "If a bank account has an annual interest rate of 10% and a starting balance of $1,000, how much interest will be earned after 3 years?", "ideal": "331"}, {"prompt": "In a standard deck of 52 playing cards, what is the probability of drawing an ace, a spade, or a red face card?", "ideal": "11/26"}, {"prompt": "A train travels at a constant and max speed of 100 km/h until it reaches 5/10 of its journey, at which point it slows down to 50km/h. If the journey distance is as far as it can travel at max speed in 2 hour, how long will it take to complete the trip?", "ideal": "3"}, {"prompt": "There are 2.54 cm in an inch. How many inches are in three lengths added toget