ChainForge/chainforge/oaievals/financial-derivatives.cforge

1 line
6.8 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-financial-derivatives", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "You are a derivatives analyst. Answer the question in one word.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "You are a derivatives analyst. Answer the question in one word.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-financial-derivatives", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet txt = response.text;\n\tlet ideals = JSON.parse(response.meta['Ideal']);\n\treturn ideals.some(i => (i.includes(txt) || txt.includes(i)));\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-financial-derivatives", "type": "vis", "data": {"input": "eval-financial-derivatives"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-financial-derivatives", "type": "inspect", "data": {"input": "prompt-financial-derivatives"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-financial-derivatives", "type": "table", "data": {"rows": [{"prompt": "Crude oil futures, the front month contract price for JUN 2023 is 69.07, while the price for FEB 2024 is 67.13. Is this market in contango or backwardation?", "ideal": "[\"backwardation\"]"}, {"prompt": "Palm oil futures, the front month contract price for MAY 2023 is 20.07, while the price for JUN 2024 is 30.20. Is this market in contango or backwardation?", "ideal": "[\"contango\"]"}, {"prompt": "Fed fund futures (ZQ), the front month contract price for MAY 2023 is 95, while the price for JUN 2024 is 96. Does the market expect the Federal Reserve to decrease or increase rates in the future?", "ideal": "[\"decrease\"]"}, {"prompt": "Fed fund futures (ZQ), the front month contract price for MAY 2023 is 96, while the price for JUN 2024 is 95. Does the market expect the Federal Reserve to decrease or increase rates in the future?", "ideal": "[\"increase\"]"}, {"prompt": "10-Year treasury futures, the front month contract price for MAY 2023 is 120.07 and the market is in contango. Is the MAY 2024 10-year treasury futures priced higher or lower than the front month contract?", "ideal": "[\"higher\"]"}, {"prompt": "10-Year treasury futures, the front month contract price for MAY 2023 is 120.07 and the market is in backwardation. Is the MAY 2024 10-year treasury futures priced higher or lower than the front month contract?", "ideal": "[\"lower\"]"}, {"prompt": "2-Year treasury futures, the front month contract price for MAY 2023 is 103.07 and the market is in backwardation. Is the yield on the MAY 2024 2-year treasury futures higher or lower than the front month contract?", "ideal": "[\"higher\"]"}, {"prompt": "2-Year treasury futures, the front month contract price for MAY 2023 is 103.07 and the market is in contango. Is the yield on the MAY 2024 2-year treasury futures higher or lower than the front month contract?", "ideal": "[\"lower\"]"}, {"prompt": "Gold futures, the front month contract price for JUL 2023 is 1800, while the price for AUG 2023 is 1850. Is this market in contango or backwardation?", "ideal": "[\"contango\"]"}, {"prompt": "Natural gas futures, the front month contract price for AUG 2023 is 3.50, while the price for JAN 2024 is 3.00. Is this market in contango or backwardation?", "ideal": "