ChainForge/chainforge/oaievals/atpl_exams.cforge
ianarawjo b33397930b
TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81)
* Beginning to convert Python backend to Typescript

* Change all fetch() calls to fetch_from_backend switcher

* wip converting query.py to query.ts

* wip started utils.js conversion. Tested that OpenAI API call works

* more progress on converting utils.py to Typescript

* jest tests for query, utils, template.ts. Confirmed PromptPipeline works.

* wip converting queryLLM in flask_app to TS

* Tested queryLLM and StorageCache compressed saving/loading

* wip execute() in backend.ts

* Added execute() and tested w concrete func. Need to test eval()

* Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd

* Execute JS code on iframe sandbox

* Tested and working JS Evaluator execution.

* wip swapping backends

* Tested TypeScript backendgit status! :) woot

* Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally

* Route Anthropic calls through Flask when running locally

* Added info button to Eval nodes. Rebuilt react

* Edits to info modal on Eval node

* Remove/error out on Python eval nodes when not running locally.

* Check browser compat and display error if not supported

* Changed all example flows to use JS. Bug fix in query.ts

* Refactored to LLMProvider to streamline model additions

* Added HuggingFace models API

* Added back Dalai call support, routing through Flask

* Remove flask app calls and socketio server that are no longer used

* Added Comment Nodes. Rebuilt react.

* Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00

1 line
8.6 KiB
Plaintext

{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-atpl_exams", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "Instructions: You will be presented with an aviation question and four options from A to D. Your role is to choose the option that is most likely correct. For each question, you should choose one answer option. There is no need to explain anything. Just answer with a single letter.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "Instructions: You will be presented with an aviation question and four options from A to D. Your role is to choose the option that is most likely correct. For each question, you should choose one answer option. There is no need to explain anything. Just answer with a single letter.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-atpl_exams", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideal = response.meta['Ideal'];\n\treturn response.text.startsWith(ideal);\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-atpl_exams", "type": "vis", "data": {"input": "eval-atpl_exams"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-atpl_exams", "type": "inspect", "data": {"input": "prompt-atpl_exams"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-atpl_exams", "type": "table", "data": {"rows": [{"prompt": "The Transition Level: A) shall be the lowest flight level available for use above the transition altitude B) shall be the highest available flight level below the transition altitude that has been established C) is calculated and declared for an approach by the Pilot-in command D) is published for the aerodrome in the Section ENR of the AIP. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "A"}, {"prompt": "LFCG 221100Z 1219 22010KT 4500 RA BKN010 OVC015 TEMPO 1500 + RA OVC005 FM1430 29020G35KT 6000 SHRA BKN008TCU OVC015 TEMPO 1619 25010 KT 9999 NSW BKN020 PROB30 TEMPO 1619 1500 TSGR BKN007CB What is the lowest cloud base you may encounter at 1345Z? A) 700 ft B) 1000 ft C) 500 ft D) 12.000 ft. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "C"}, {"prompt": "What approximate rate of descent is required in order to maintain a 3\u00b0 glide path at a groundspeed of 120 kt? A) 600 FT/MIN. B) 950 FT/MIN. C) 550 FT/MIN. D) 800 FT/MIN. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "A"}, {"prompt": "The colour used on the B737-400 EHSI weather display to show turbulence is: A) flashing red. B) white or magenta. C) high colour gradient. D) magenta. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "D"}, {"prompt": "According to ICAO Document 8168 Part V, when using Procedure A for noise abatement, when must you reduce power: A) 2000 ft. B) 3000 ft. C) 1500 ft. D) 1000 ft. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "C"}, {"prompt": "In the ATS flight plan Item 10 (equipment), the letter to indicate the carriage of a serviceable transponder - mode A (4 digits-4096 codes) and mode C, is: A) A B) P C) C D) B Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "C"}, {"prompt": "The words TAKE OFF are only to be used when: A) An aircraft is cleared for take-off or, when cancelling a take-off clearance. At all times the words DEPARTURE or AIRBORNE is used. Readback is mandatory. B) An aircraft is cleared to start. C) An aircraft is cleared to line-up only. D) When clearance delivery authorises its use. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "A"}, {"prompt": "Clearance limit is defined as: A) The point to which aircraft is granted ATC clearance. B) The time at which the ATC clearance expires. C) The height below which you will hit the first obstacle. D) The flight level to which an aircraft is granted ATC clearance. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "A"}, {"prompt": "In a coordinated and level turn: A) an aeroplane cannot stall. B) the stall speed increases. C) the stall speed decreases. D) the stall speed remains the same. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "B"}, {"prompt": "When establishing an instrument approach procedure, 5 aircraft categories according to their speed at the threshold (Vat) are established. This speed is equal to the stalling speed in the landing configuration at the maximum certified landing weight multiplied by a factor of: A) 1.3 B) 1.45 C) 1.5 D) 1.15 Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "A"}, {"prompt": "The term used to describe the visual phase of flight after completing an instrument approach, to bring an aircraft into position for landing on runway which is not suitably located for straight-in approach, is: A) Visual manoeuvring (circling) B) Visual approach C) Contact approach D) Aerodrome traffic pattern Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "A"}, {"prompt": "What is defined as the portion of a flight in which the aircraft descends below 1000 ft above the relevant DH or MDH? A) Go around / Missed Approach B) Glide path C) Final approach D) Approach to landing Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "D"}, {"prompt": "The Transition Level: A) shall be the lowest flight level available for use above the transition altitude B) shall be the highest available flight level below the transition altitude that has been established C) is calculated and declared for an approach by the Pilot-in command D) is published for the aerodrome in the Section ENR of the AIP Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "A"}, {"prompt": "A radial is: A) a magnetic bearing to a VOR station B) a magnetic bearing extending from a VOR station C) a magnetic bearing to or from a VOR station, depending on whether the aircraft is inbound or outbound to or from the VOR D) a QDM Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "B"}, {"prompt": "In an aeroplane utilising a constant frequency AC power supply, DC power is obtained from a: A) 3 phase current transformer unit. B) static inverter. C) rotary converter. D) Transformer Rectifier Unit. Reply by choosing one of the following answers: 'A', 'B', 'C', 'D'.", "ideal": "D"}], "columns": [{"key": "prompt", "header": "Prompt"}, {"key": "ideal", "header": "Ideal"}]}, "position": {"x": -16, "y": 160}, "selected": false, "positionAbsolute": {"x": -16, "y": 160}, "dragging": false}], "edges": [{"source": "prompt-atpl_exams", "sourceHandle": "prompt", "target": "eval-atpl_exams", "targetHandle": "responseBatch", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-prompt-1686756357355prompt-eval-1686756357355responseBatch"}, {"source": "prompt-atpl_exams", "sourceHandle": "prompt", "target": "inspect-atpl_exams", "targetHandle": "input", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-prompt-1686756357355prompt-inspect-1686756357355input"}, {"source": "eval-atpl_exams", "sourceHandle": "output", "target": "vis-atpl_exams", "targetHandle": "input", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-eval-1686756357355output-vis-1686756357355input"}, {"source": "table-atpl_exams", "sourceHandle": "Prompt", "target": "prompt-atpl_exams", "targetHandle": "prompt", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-table-1686756385002Prompt-prompt-1686756357355prompt"}], "viewport": {"x": 144, "y": 37, "zoom": 1}}, "cache": {"eval-1686756357355.json": {}, "inspect-1686756357355.json": {}, "prompt-1686756357355.json": {}, "table-1686756385002.json": {}, "vis-1686756357355.json": {}}}