ChainForge/chainforge/oaievals/fcc_amateur_extra.cforge

1 line
183 KiB
Plaintext
Raw Permalink Normal View History

TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81) * Beginning to convert Python backend to Typescript * Change all fetch() calls to fetch_from_backend switcher * wip converting query.py to query.ts * wip started utils.js conversion. Tested that OpenAI API call works * more progress on converting utils.py to Typescript * jest tests for query, utils, template.ts. Confirmed PromptPipeline works. * wip converting queryLLM in flask_app to TS * Tested queryLLM and StorageCache compressed saving/loading * wip execute() in backend.ts * Added execute() and tested w concrete func. Need to test eval() * Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd * Execute JS code on iframe sandbox * Tested and working JS Evaluator execution. * wip swapping backends * Tested TypeScript backendgit status! :) woot * Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally * Route Anthropic calls through Flask when running locally * Added info button to Eval nodes. Rebuilt react * Edits to info modal on Eval node * Remove/error out on Python eval nodes when not running locally. * Check browser compat and display error if not supported * Changed all example flows to use JS. Bug fix in query.ts * Refactored to LLMProvider to streamline model additions * Added HuggingFace models API * Added back Dalai call support, routing through Flask * Remove flask app calls and socketio server that are no longer used * Added Comment Nodes. Rebuilt react. * Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-fcc_amateur_extra", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "The following are multiple choice questions (with answers) from the FCC Amateur Radio License question pool. Select the correct answer for each question. Respond only with the letter of the correct anser. For example, if the correct answer is \"B\", repond with \"B\" (without quotes).", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "The following are multiple choice questions (with answers) from the FCC Amateur Radio License question pool. Select the correct answer for each question. Respond only with the letter of the correct anser. For example, if the correct answer is \"B\", repond with \"B\" (without quotes).", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-fcc_amateur_extra", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideal = response.meta['Ideal'];\n\treturn response.text.startsWith(ideal);\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-fcc_amateur_extra", "type": "vis", "data": {"input": "eval-fcc_amateur_extra"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-fcc_amateur_extra", "type": "inspect", "data": {"input": "prompt-fcc_amateur_extra"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-fcc_amateur_extra", "type": "table", "data": {"rows": [{"prompt": "Which of the following carrier frequencies is illegal for LSB AFSK emissions on the 17 meter band RTTY and data segment of 18.068 to 18.110 MHz?\nA. 18.068 MHz\nB. 18.100 MHz\nC. 18.107 MHz\nD. 18.110 MHz\n", "ideal": "A"}, {"prompt": "When using a transceiver that displays the carrier frequency of phone signals, which of the following displayed frequencies represents the lowest frequency at which a properly adjusted LSB emission will be totally within the band?\nA. The exact lower band edge\nB. 300 Hz above the lower band edge\nC. 1 kHz above the lower band edge\nD. 3 kHz above the lower band edge\n", "ideal": "D"}, {"prompt": "What is the maximum legal carrier frequency on the 20 meter band for transmitting USB AFSK digital signals having a 1 kHz bandwidth?\nA. 14.070 MHz\nB. 14.100 MHz\nC. 14.149 MHz\nD. 14.349 MHz\n", "ideal": "C"}, {"prompt": "With your transceiver displaying the carrier frequency of phone signals, you hear a DX station calling CQ on 3.601 MHz LSB. Is it legal to return the call using lower sideband on the same frequency?\nA. Yes, because the DX station initiated the contact\nB. Yes, because the displayed frequency is within the 75 meter phone band segment\nC. No, the sideband will extend beyond the edge of the phone band segment\nD. No, U.S. stations are not permitted to use phone emissions below 3.610 MHz\n", "ideal": "C"}, {"prompt": "What is the maximum power output permitted on the 60 meter band?\nA. 50 watts PEP effective radiated power relative to an isotropic radiator\nB. 50 watts PEP effective radiated power relative to a dipole\nC. 100 watts PEP effective radiated power relative to the gain of a half-wave dipole\nD. 100 watts PEP effective radiated power relative to an isotropic radiator\n", "ideal": "C"}, {"prompt": "Where must the carrier frequency of a CW signal be set to comply with FCC rules for 60 meter operation?\nA. At