ChainForge/chainforge/oaievals/swedish_sat.cforge

1 line
88 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-swedish_sat", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "You will be prompted with a question in swedish You will be provided with multiple choices for the question (A, B, C, D or E). Answer with exactly one of the following: A, B, C, D or E. Don't add anything else to the response.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "You will be prompted with a question in swedish You will be provided with multiple choices for the question (A, B, C, D or E). Answer with exactly one of the following: A, B, C, D or E. Don't add anything else to the response.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-swedish_sat", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideal = response.meta['Ideal'];\n\treturn response.text.startsWith(ideal);\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-swedish_sat", "type": "vis", "data": {"input": "eval-swedish_sat"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-swedish_sat", "type": "inspect", "data": {"input": "prompt-swedish_sat"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-swedish_sat", "type": "table", "data": {"rows": [{"prompt": "f\u00f6respr\u00e5ka\nA reflektera\nB reservera\nC rekommendera\nD resonera\nE regissera\n", "ideal": "C"}, {"prompt": "obefogad\nA grundl\u00f6s\nB maktl\u00f6s\nC ansvarsl\u00f6s\nD gr\u00e4nsl\u00f6s\nE meningsl\u00f6s\n", "ideal": "A"}, {"prompt": "emballage\nA symbol\nB f\u00f6rpackning\nC underh\u00e5ll\nD avsp\u00e4rrning\nE handelsf\u00f6rbud\n", "ideal": "B"}, {"prompt": "tidvis\nA ofta\nB f\u00f6rut\nC punktligt\nD ibland\nE numera\n", "ideal": "D"}, {"prompt": "framfusig\nA nyt\u00e4nkande\nB sj\u00e4lvs\u00e4ker\nC uppjagad\nD f\u00f6rv\u00e4ntansfull\nE p\u00e5tr\u00e4ngande\n", "ideal": "E"}, {"prompt": "extravagans\nA \u00f6verd\u00e5d\nB v\u00e4rdighet\nC merkostnad\nD passion\nE \u00f6vermod\n", "ideal": "A"}, {"prompt": "ge sig till t\u00e5ls\nA ta sitt ansvar\nB vara i underl\u00e4ge\nC fatta ett beslut\nD lugnt inv\u00e4nta n\u00e5got\nE visa sin styrka\n", "ideal": "D"}, {"prompt": "baryton\nA notst\u00e4ll\nB pianostycke\nC r\u00f6stl\u00e4ge\nD slagverkare\nE taktpinne\n", "ideal": "C"}, {"prompt": "r\u00e5gad\nA kraftig\nB bristf\u00e4llig\nC \u00e5tg\u00e4rdad\nD \u00f6verfull\nE avslutad\n", "ideal": "D"}, {"prompt": "maxim\nA reaktion\nB j\u00e4mvikt\nC slutresultat\nD f\u00f6rbindelse\nE levnadsregel \n", "ideal": "E"}, {"prompt": "Studentfabriken\nNu har vi l\u00e4rt alla grundskoleelever att fr\u00e5ga \u201dVad ska det\nvara bra f\u00f6r?\u201d och alla gymnasieelever att fr\u00e5ga \u201dKommer\ndet p\u00e5 betyget?\u201d Att fokusera yrkesanknytning p\u00e5 universitetet \u00e4r en f\u00f6rl\u00e4ngning av samma syns\u00e4tt; det viktiga \u00e4r inte fascinationen inf\u00f6r ny kunskap, utan \u201dKommer det p\u00e5 tentan?\u201d och \u201dVad kan jag f\u00e5 ut av det h\u00e4r?\u201d\nUniversiteten uppstod historiskt sett i Europa, och\nEuropas h\u00f6gre utbildning var l\u00e4nge beundrad i \u00f6vriga\ndelar av v\u00e4rlden. Under de senare decennierna har dock\nuniversiteten utsatts f\u00f6r h\u00e5rdh\u00e4nt statlig p\u00e5verkan i\n\u00adsamband med \u00adatt \u00adallt \u00ad