ChainForge/chainforge/oaievals/probability_questions.cforge

1 line
38 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-probability_questions", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "You are a helpful statistician. Answer the questions with only the numerical answer rounded to 4 decimal places. Provide no explanation.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "You are a helpful statistician. Answer the questions with only the numerical answer rounded to 4 decimal places. Provide no explanation.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-probability_questions", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideals = JSON.parse(response.meta['Ideal']);\n\treturn ideals.some(i => response.text.startsWith(i));\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-probability_questions", "type": "vis", "data": {"input": "eval-probability_questions"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-probability_questions", "type": "inspect", "data": {"input": "prompt-probability_questions"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-probability_questions", "type": "table", "data": {"rows": [{"prompt": "A pair of fair, standard dice are rolled. What is the probability the sum of the dice is 5", "ideal": "[\"0.1111\"]"}, {"prompt": "An airplane is built to be able to fly on one engine. If the plane's two engines operate independently, and each has a 1% chance of failing in any given four-hour flight, what is the chance the plane will fail to complete a four-hour flight to Oklahoma due to engine failure?", "ideal": "[\"0.0001\"]"}, {"prompt": "A 1-inch-diameter coin is thrown on a table covered with a grid of lines two inches apart. What is the probability the coin lands in a square without touching any of the lines of the grid?", "ideal": "[\"0.2500\"]"}, {"prompt": "Of the 50 students in a certain class, 5 speak French. Two students of the class will be selected at random. Which of the following is closest to the probability that neither of the students selected will speak French?", "ideal": "[\"0.8100\"]"}, {"prompt": "Of the 10 marbles in a box, 2 are green. A person will select 2 marbles simultaneously and at random from the box. What is the probability that neither of the marbles selected will be green?", "ideal": "[\"0.6222\"]"}, {"prompt": "On a number line, there are 6 distinct points, of which 4 are positive and 2 are negative. If 2 different points are to be randomly selected, what is the probability that the 2 points selected will both be positive?", "ideal": "[\"0.4000\"]"}, {"prompt": "Of the 20 employees in a company, 5 have an MBA. If 3 employees are to be simultaneously selected at random, what is the probability that only 1 of the 3 employees selected will have an MBA?", "ideal": "[\"0.4605\"]"}, {"prompt": "Suppose that there is a coin that is weighted in such a way that each time the coin is tossed, the probability of tossing head is twice the probability of tossing a tail. What is the probability of tossing a head?", "ideal": "[\"0.6667\"]"}, {"prompt": "Suppose that there is a 6-sided die with faces numbered 1 through 6. This die is rolled twice. What is the probability that the first roll will be an odd number and the second roll will be an even number?", "ideal": "[\"0.2500\