ChainForge/chainforge/oaievals/arithmetical_puzzles.cforge
ianarawjo b33397930b
TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81)
* Beginning to convert Python backend to Typescript

* Change all fetch() calls to fetch_from_backend switcher

* wip converting query.py to query.ts

* wip started utils.js conversion. Tested that OpenAI API call works

* more progress on converting utils.py to Typescript

* jest tests for query, utils, template.ts. Confirmed PromptPipeline works.

* wip converting queryLLM in flask_app to TS

* Tested queryLLM and StorageCache compressed saving/loading

* wip execute() in backend.ts

* Added execute() and tested w concrete func. Need to test eval()

* Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd

* Execute JS code on iframe sandbox

* Tested and working JS Evaluator execution.

* wip swapping backends

* Tested TypeScript backendgit status! :) woot

* Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally

* Route Anthropic calls through Flask when running locally

* Added info button to Eval nodes. Rebuilt react

* Edits to info modal on Eval node

* Remove/error out on Python eval nodes when not running locally.

* Check browser compat and display error if not supported

* Changed all example flows to use JS. Bug fix in query.ts

* Refactored to LLMProvider to streamline model additions

* Added HuggingFace models API

* Added back Dalai call support, routing through Flask

* Remove flask app calls and socketio server that are no longer used

* Added Comment Nodes. Rebuilt react.

* Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00

1 line
14 KiB
Plaintext

{"flow": {"nodes": [{"width": 312, "height": 311, "id": "prompt-arithmetical_puzzles", "type": "prompt", "data": {"prompt": "{prompt}", "n": 1, "llms": [{"key": "aa3c0f03-22bd-416e-af4d-4bf5c4278c99", "settings": {"system_msg": "Complete the phrase as concisely as possible.", "temperature": 1, "functions": [], "function_call": "", "top_p": 1, "stop": [], "presence_penalty": 0, "frequency_penalty": 0}, "name": "GPT3.5", "emoji": "\ud83d\ude42", "model": "gpt-3.5-turbo", "base_model": "gpt-3.5-turbo", "temp": 1, "formData": {"shortname": "GPT3.5", "model": "gpt-3.5-turbo", "system_msg": "Complete the phrase as concisely as possible.", "temperature": 1, "functions": "", "function_call": "", "top_p": 1, "stop": "", "presence_penalty": 0, "frequency_penalty": 0}}]}, "position": {"x": 448, "y": 224}, "selected": false, "positionAbsolute": {"x": 448, "y": 224}, "dragging": false}, {"width": 333, "height": 182, "id": "eval-arithmetical_puzzles", "type": "evaluator", "data": {"code": "function evaluate(response) {\n\tlet ideals = JSON.parse(response.meta['Ideal']);\n\treturn ideals.some(i => response.text.startsWith(i));\n}", "language": "javascript"}, "position": {"x": 820, "y": 150}, "positionAbsolute": {"x": 820, "y": 150}}, {"width": 228, "height": 196, "id": "vis-arithmetical_puzzles", "type": "vis", "data": {"input": "eval-arithmetical_puzzles"}, "position": {"x": 1200, "y": 250}, "positionAbsolute": {"x": 1200, "y": 250}}, {"width": 302, "height": 260, "id": "inspect-arithmetical_puzzles", "type": "inspect", "data": {"input": "prompt-arithmetical_puzzles"}, "position": {"x": 820, "y": 400}, "positionAbsolute": {"x": 820, "y": 400}}, {"width": 423, "height": 417, "id": "table-arithmetical_puzzles", "type": "table", "data": {"rows": [{"prompt": "Tommy: \"How old are you, mamma?\" Mamma: \"Let me think, Tommy. Well, our three ages add up to exactly seventy years.\" Tommy: \"That's a lot, isn't it? And how old are you, papa?\" Papa: \"Just six times as old as you, my son.\" Tommy: \"Shall I ever be half as old as you, papa?\" Papa: \"Yes, Tommy; and when that happens our three ages will add up to exactly twice as much as to-day.\" Tommy: \"And supposing I was born before you, papa; and supposing mamma had forgot all about it, and hadn't been at home when I came; and supposing\" Mamma: \"Supposing, Tommy, we talk about bed. Come along, darling. You'll have a headache.\" Now, if Tommy had been some years older he might have calculated the exact ages of his parents from the information they had given him. Can you find out the exact age of mamma in months? Please give only the numerical answer as a single integer indicating the number of months. Do not give any reasoning or logic for your answer.", "ideal": "[\"350\"]"}, {"prompt": "\"My husband's age,\" remarked a lady the other day, \"is represented by the figures of my own age reversed. He is my senior, and the difference between our ages is one-eleventh of their sum.\" What is the husband's age? Please give only the numerical answer as a single integer indicating the number of years. Do not give any reasoning or logic for your answer.", "ideal": "[\"54\"]"}, {"prompt": "When the Smileys recently received a visit from the favourite uncle, the fond parents had all the five children brought into his presence. First came Billie and little Gertrude, and the uncle was informed that the boy was exactly twice as old as the girl. Then Henrietta arrived, and it was pointed out that the combined ages of herself and Gertrude equalled twice the age of Billie. Then Charlie came running in, and somebody remarked that now the combined ages of the two boys were exactly twice the combined ages of the two girls. The uncle was expressing his astonishment at these coincidences when Janet came in. \"Ah! uncle,\" she exclaimed, \"you have actually arrived on my twenty-first birthday!\" To this Mr. Smiley added the final staggerer: \"Yes, and now the combined ages of the three girls are exactly equal to twice the combined ages of the two boys.\" Please give the age of Billie. Please give only the numerical answer as a single integer indicating the number of months. Do not give any reasoning or logic for your answer.", "ideal": "[\"42\"]"}, {"prompt": "Edwin: \"Do you know, when the Timpkinses married eighteen years ago Timpkins was three times as old as his wife, and to-day he is just twice as old as she?\" Angelina: \"Then how old was Mrs. Timpkins on the wedding day?\" Can you answer Angelina's question? Please give the age of Mrs. Timpkins. Please give only the numerical answer as a single integer indicating the number of years. Do not give any reasoning or logic for your answer.", "ideal": "[\"18\"]"}, {"prompt": "\"Mother, I wish you would give me a bicycle,\" said a girl of twelve the other day. \"I do not think you are old enough yet, my dear,\" was the reply. \"When I am only three times as old as you are you shall have one.\" Now, the mother's age is forty-five years. When may the young lady expect to receive her present? Please give the length of time as a single integer indicating the number of months. Do not give any reasoning or logic for your answer.", "ideal": "[\"54\"]"}, {"prompt": "There were two families living next door to one another at Tooting Becthe Jupps and the Simkins. The united ages of the four Jupps amounted to one hundred years, and the united ages of the Simkins also amounted to the same. It was found in the case of each family that the sum obtained by adding the squares of each of the children's ages to the square of the mother's age equalled the square of the father's age. In the case of the Jupps, however, Julia was one year older than her brother Joe, whereas Sophy Simkin was two years older than her brother Sammy. What was the age of Mr. Jupp? Please give the age as a single integer indicating the number of years. Do not give any reasoning or logic for your answer.", "ideal": "[\"39\"]"}, {"prompt": "Write eleven thousand eleven hundred and eleven. Please give the answer as a single integer. Do not give any reasoning or logic for your answer.", "ideal": "[\"12111\"]"}, {"prompt": "In a recent motor ride it was found that we had gone at the rate of ten miles an hour, but we did the return journey over the same route, owing to the roads being more clear of traffic, at fifteen miles an hour. What was our average speed? Please give the answer as an integer. Do not give any reasoning or logic for your answer. \n", "ideal": "[\"12\"]"}, {"prompt": "Two trains start at the same time, one from London to Liverpool, the other from Liverpool to London. If they arrive at their destinations one hour and four hours respectively after passing one another, how much faster is one train running than the other? Please give the answer as an integer, indicating the ratio. Do not give any reasoning or logic for your answer.", "ideal": "[\"2\"]"}, {"prompt": "I set out the other day to ride in a motor-car from Acrefield to Butterford, but by mistake I took the road going via Cheesebury, which is nearer Acrefield than Butterford, and is twelve miles to the left of the direct road I should have travelled. After arriving at Butterford I found that I had gone thirty-five miles. What is the distance from Acrefield to Butterford? Please give the answer as an integer. Do not give any reasoning or logic for your answer. \n", "ideal": "[\"25\"]"}, {"prompt": "Sir Edwyn de Tudor was going to rescue his lady-love, the fair Isabella, who was held a captive by a neighbouring wicked baron. Sir Edwyn calculated that if he rode fifteen miles an hour he would arrive at the castle an hour too soon, while if he rode ten miles an hour he would get there just an hour too late. Now, it was of the first importance that he should arrive at the exact time appointed, in order that the rescue that he had planned should be a success, and the time of the tryst was five o'clock, when the captive lady would be taking her afternoon tea. How many miles did Sir Edwyn de Tudor have to ride. Please give the answer as an integer. Do not give any reasoning or logic for your answer. \n", "ideal": "[\"60\"]"}, {"prompt": "A man had a basket containing fifty potatoes. He proposed to his son, as a little recreation, that he should place these potatoes on the ground in a straight line. The distance between the first and second potatoes was to be one yard, between the second and third three yards, between the third and fourth five yards, between the fourth and fifth seven yards, and so onan increase of two yards for every successive potato laid down. Then the boy was to pick them up and put them in the basket one at a time, the basket being placed beside the first potato. How far would the boy have to travel to accomplish the feat of picking them all up? We will not consider the journey involved in placing the potatoes, so that he starts from the basket with them all laid out. Please give the answer as an integer indicating the number of yards. Do not give any reasoning or logic for your answer.", "ideal": "[\"80850\"]"}, {"prompt": "Using each figure once, and only once, we can form two multiplication sums that have the same product, and this may be done in many ways. For example, 7x658 and 14x329 contain all the digits once, and the product in each case is the same4,606. Now, it will be seen that the sum of the digits in the product is 16, which is neither the highest nor the lowest sum so obtainable. Can you find the solution of the problem that gives the lowest possible sum of digits in the common product? Please give the answer as an integer. Do not give any reasoning or logic for your answer. ", "ideal": "[\"4002\"]"}, {"prompt": "If I multiply 51,249,876 by 3 (thus using all the nine digits once, and once only), I get 153,749,628 (which again contains all the nine digits once). Similarly, if I multiply 16,583,742 by 9 the result is 149,253,678, where in each case all the nine digits are used. Now, take 6 as your multiplier and try to arrange the remaining eight digits so as to produce by multiplication a number containing all nine once, and once only. What is the result? Please give the answer as an integer. Do not give any reasoning or logic for your answer.", "ideal": "[\"195287346\", \"195,287,346\"]"}, {"prompt": "In a certain mixed school, where a special feature was made of the inculcation of good manners, they had a curious rule on assembling every morning. There were twice as many girls as boys. Every girl made a bow to every other girl, to every boy, and to the teacher. Every boy made a bow to every other boy, to every girl, and to the teacher. In all there were nine hundred bows made in that model academy every morning. Now, can you say exactly how many boys there were in the school? Please give the answer as an integer. Do not give any reasoning or logic for your answer.", "ideal": "[\"10\"]"}, {"prompt": "\"A man I know,\" said Teddy Nicholson at a certain family party, \"possesses a string of thirty-three pearls. The middle pearl is the largest and best of all, and the others are so selected and arranged that, starting from one end, each successive pearl is worth 100 more than the preceding one, right up to the big pearl. From the other end the pearls increase in value by 150 up to the large pearl. The whole string is worth 65,000. What is the value of that large pearl?\" Please give the answer as an integer. Do not give any reasoning or logic for your answer.", "ideal": "[\"3000\"]"}, {"prompt": "Tim Murphy and Pat Donovan were engaged by the local authorities to paint the lamp-posts in a certain street. Tim, who was an early riser, arrived first on the job, and had painted three on the south side when Pat turned up and pointed out that Tim's contract was for the north side. So Tim started afresh on the north side and Pat continued on the south. When Pat had finished his side he went across the street and painted six posts for Tim, and then the job was finished. As there was an equal number of lamp-posts on each side of the street, the simple question is: How many more did the man who painted the most lamp-posts paint than the other? Please give the answer as an integer. Do not give any reasoning or logic for your answer. \n", "ideal": "[\"6\"]"}], "columns": [{"key": "prompt", "header": "Prompt"}, {"key": "ideal", "header": "Ideal"}]}, "position": {"x": -16, "y": 160}, "selected": false, "positionAbsolute": {"x": -16, "y": 160}, "dragging": false}], "edges": [{"source": "prompt-arithmetical_puzzles", "sourceHandle": "prompt", "target": "eval-arithmetical_puzzles", "targetHandle": "responseBatch", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-prompt-1686756357355prompt-eval-1686756357355responseBatch"}, {"source": "prompt-arithmetical_puzzles", "sourceHandle": "prompt", "target": "inspect-arithmetical_puzzles", "targetHandle": "input", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-prompt-1686756357355prompt-inspect-1686756357355input"}, {"source": "eval-arithmetical_puzzles", "sourceHandle": "output", "target": "vis-arithmetical_puzzles", "targetHandle": "input", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-eval-1686756357355output-vis-1686756357355input"}, {"source": "table-arithmetical_puzzles", "sourceHandle": "Prompt", "target": "prompt-arithmetical_puzzles", "targetHandle": "prompt", "interactionWidth": 100, "markerEnd": {"type": "arrow", "width": "22px", "height": "22px"}, "id": "reactflow__edge-table-1686756385002Prompt-prompt-1686756357355prompt"}], "viewport": {"x": 144, "y": 37, "zoom": 1}}, "cache": {"eval-1686756357355.json": {}, "inspect-1686756357355.json": {}, "prompt-1686756357355.json": {}, "table-1686756385002.json": {}, "vis-1686756357355.json": {}}}