ChainForge/chainforge/oaievals/test-comp-sci.cforge

1 line
10 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-test-comp-sci", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "Choose the best multiple choice answer to this question. Reply ONLY with the single letter of the answer you have chosen.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "Choose the best multiple choice answer to this question. Reply ONLY with the single letter of the answer you have chosen.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-test-comp-sci", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideal = response.meta['Ideal'];\n\treturn response.text.startsWith(ideal);\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-test-comp-sci", "type": "vis", "data": {"input": "eval-test-comp-sci"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-test-comp-sci", "type": "inspect", "data": {"input": "prompt-test-comp-sci"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-test-comp-sci", "type": "table", "data": {"rows": [{"prompt": "How many children does a binary tree have? a) 2 b) any number of children c) 0 or 1 or 2 d) 0 or 1", "ideal": "c"}, {"prompt": "What is/are the disadvantages of implementing tree using normal arrays? a) difficulty in knowing children nodes of a node b) difficult in finding the parent of a node c) have to know the maximum number of nodes possible before creation of trees d) difficult to implement", "ideal": "c"}, {"prompt": "What must be the ideal size of array if the height of tree is \u2018l\u2019? a) (2^l)-1 b) l-1 c) l d) 2l", "ideal": "a"}, {"prompt": "What are the children for node \u2018w\u2019 of a complete-binary tree in an array representation? a) 2w and 2w+1 b) 2+w and 2-w c) w+1/2 and w/2 d) w-1/2 and w+1/2", "ideal": "a"}, {"prompt": "What is the parent for a node \u2018w\u2019 of a complete binary tree in an array representation when w is not 0? a) floor(w-1/2) b) ceil(w-1/2) c) w-1/2 d) w/2", "ideal": "a"}, {"prompt": "If the tree is not a complete binary tree then what changes can be made for easy access of children of a node in the array? a) every node stores data saying which of its children exist in the array b) no need of any changes continue with 2w and 2w+1, if node is at i c) keep a seperate table telling children of a node d) use another array parallel to the array with tree", "ideal": "a"}, {"prompt": "Consider a situation of writing a binary tree into a file with memory storage efficiency in mind, is array representation of tree is good? a) yes because we are overcoming the need of pointers and so space efficiency b) yes because array values are indexable c) No it is not efficient in case of sparse trees and remaning cases it is fine d) No linked list representation of tree is only fine", "ideal": "c"}, {"prompt": "Can a tree stored in an array using either one of inorder or post order or pre order traversals be again reformed? a) Yes just traverse through the array and form the tree b) No we need one more traversal to form a tree c) No in case of sparse trees d) Yes by using both inorder and array elements", "ideal": "b"}, {"prompt": "Consider the original array 17 8 12 4 26, How many comparisons are needed to construct the BST on the original array?