73 Commits

Author SHA1 Message Date
Ian Arawjo
ce897b55f2 Add Edge and Brave support 2023-08-03 02:25:27 -04:00
Ian Arawjo
3fbc88abb8 Make special #vars work on chat nodes 2023-08-01 14:56:19 -04:00
Ian Arawjo
4730009a1c Add # special fill variables to prompt templating. 2023-08-01 11:34:51 -04:00
ianarawjo
f7fab1d76d
Update 0.2.5: Add Chat Turns, LLM Scorers (#109)
* Add LLM scorer node (#107)

* Modularize the LLM list container, extracting it from prompt node

* Working LLM scorer node

* Bug and minor fixes

* Change modals to use perc left. 

* Add inspect response footer to LLMEvalNode.

* Make Play buttons light green

* Fix React errors w keys in JSX arrays

* Add Chat Turn node and support for chat history (#108)

* Adds chat_history across backend's cache and querying mechanisms.

* Adds Chat Turn nodes, which allow for continuing a conversation.

* Adds automatic conversions of ChatHistory (in OpenAI format) to Anthropic and Google PaLM's chat formats. Converts chat history to appropriate format and passes it as context in the API call.

* Bug fix and error popup when missing past convo in Chat Turn

* Bug squashing to progress in chat turn node

* bug squashing

* Color false scores bright red in eval inspector

* fix tooltip when cont chat present

* Rebuild react

* bug fix llm eval node

* Add HF chat model support.

* Show multiple response objs in table inspector view

* Fix LLM item deletion bug

* Rebuild react and update package version

* Fix obscure bug when LLM outputs have no 'llm' property (due to prior CF version)

* Fix isLooselyEqual bug

* Update examples so that their cached 'fields' include llm nicknames

* rebuild react

* Add Chelse to readme
2023-07-26 12:06:09 -04:00
ianarawjo
a27d092ccc
Choose plot column in table view of response inspector (#106)
* Make vars in inspect table view multiline

* Choose var to use for columns in table view of resp inspector

* Only default to prompt var as column if num vars > 1

* Rebuild react and update package version
2023-07-23 14:22:13 -04:00
Ian Arawjo
16258e5588 Force height of wide handles 2023-07-21 12:47:21 -04:00
ianarawjo
666d5900b5
'Dirty' downstream nodes whenever upstream changes are made. Minor styling improvements. (#105)
* Invalidate eval node upon upstream changes.

* Chain update pings across nodes. Autoresize textfields when typing.

* Wide output handles when entire node is output

* update package version
2023-07-21 12:39:08 -04:00
Ian Arawjo
69f52e7df9 Fix getUniqueLLMMetavarKey bug + remove unnecessary deps 2023-07-20 15:40:15 -04:00
Ian Arawjo
5826c822ee Rebuild react and update package version 2023-07-20 11:23:52 -04:00
Ian Arawjo
07150fc6ae Let users choose hostname and port 2023-07-20 11:17:58 -04:00
Ian Arawjo
31340bedc5 Add 0.0.0.0 to localhost hostnames check 2023-07-20 10:30:20 -04:00
ianarawjo
038832910f
Table view and persistent response inspector (#103)
* WIP Tabular data view in Inspect modal

* Keep response inspector mounted

* Rebuild react app and update package version
2023-07-19 17:15:19 -04:00
Ian Arawjo
973970b8ab Escape braces in outputs of prompt nodes by default 2023-07-14 09:13:24 -04:00
ianarawjo
3657609c32
Add prompt preview tooltip, add ability to disable textfields selectively, bug fixes (#97)
* Add tooltip to prompt preview button

* Focus scrollwheel on textfields textareas

* Replace escaped { and } with their bare versions

* Escape braces in tabular data by default. Ignore empty rows.

* Add ability to disable fields on textfields

* Make sure deleting a field deletes its fields_visibility

* Add withinPortal to Tooltips on side-buttons in text fields

* Add Anthropic model Claude-2.
2023-07-12 17:12:16 -04:00
ianarawjo
318f81e1df
Add prompts previews on hover and click of PromptNode (#91)
* Added Prompt Preview panes

* Update package version and react app
2023-07-07 20:35:50 -04:00
Ian Arawjo
e62be7eaf1 Rebuild react app, update package version 2023-07-07 15:36:26 -04:00
ianarawjo
f094fc937b
Collapseable response supergroups; nicer colors for headers of response groups; resizeable Prompt Node text areas (#89)
* Resizeable textfield in prompt nodes

* Popup when user clicks X to delete node

* Collapseable supergroups in response inspector

* Nicer hierarchical color scheme for response group headers in response inspectors
2023-07-07 15:30:56 -04:00
Ian Arawjo
9cf9673b27 Add allow_empty_str prop so react-json-form-schema treats empty text area as empty string, not undefined 2023-07-07 14:57:13 -04:00
Ian Arawjo
c735008d88 Fix overflow issue in root prompt box in response inspector 2023-07-07 14:56:01 -04:00
Ian Arawjo
e57cfebfc2 Fix duplicate handles bug in Prompt Node 2023-07-07 14:01:04 -04:00
Ian Arawjo
6ad88f0ad9 Update package and rebuild react 2023-07-07 11:39:20 -04:00
Ian Arawjo
0c3ea969dc Added HuggingFace key to load from env variable 2023-07-07 11:12:47 -04:00
Ian Arawjo
9d8dfdb43f Change manifest description 2023-07-07 10:49:49 -04:00
Ian Arawjo
5691f0d65b Remove unused js in build 2023-07-04 10:22:04 -04:00
Ian Arawjo
8f88061dc8 Merge branch 'main' of https://github.com/ianarawjo/ChainForge 2023-07-04 10:19:35 -04:00
ianarawjo
e46bf5c7cc
Add Share Button, add hosted version link to README.md (#83)
* Share button implementation and testing

* Bug fix for Azure OpenAI (missing `bind` call)

* Change `call_anthropic` to go through server proxy when not running locally

* Cleanup of unused imports

* Fixed bug in Azure OpenAI call

* Update README.md with Share, Play link, etc

* Update package version to 0.2.0.2
2023-07-04 10:18:55 -04:00
Ian Arawjo
90a3862669 Stringify LLM error message jic its not a string 2023-07-03 17:14:23 -04:00
Ian Arawjo
b06be34c94 Patched bug: unbound client in Azure OpenAI func 2023-07-03 08:43:59 -04:00
Ian Arawjo
7e60c50338 Add better example flow upon load 2023-07-02 10:05:20 -04:00
ianarawjo
b33397930b
TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81)
* Beginning to convert Python backend to Typescript

* Change all fetch() calls to fetch_from_backend switcher

* wip converting query.py to query.ts

* wip started utils.js conversion. Tested that OpenAI API call works

* more progress on converting utils.py to Typescript

* jest tests for query, utils, template.ts. Confirmed PromptPipeline works.

* wip converting queryLLM in flask_app to TS

* Tested queryLLM and StorageCache compressed saving/loading

* wip execute() in backend.ts

* Added execute() and tested w concrete func. Need to test eval()

* Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd

* Execute JS code on iframe sandbox

* Tested and working JS Evaluator execution.

* wip swapping backends

* Tested TypeScript backendgit status! :) woot

* Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally

* Route Anthropic calls through Flask when running locally

* Added info button to Eval nodes. Rebuilt react

* Edits to info modal on Eval node

* Remove/error out on Python eval nodes when not running locally.

* Check browser compat and display error if not supported

* Changed all example flows to use JS. Bug fix in query.ts

* Refactored to LLMProvider to streamline model additions

* Added HuggingFace models API

* Added back Dalai call support, routing through Flask

* Remove flask app calls and socketio server that are no longer used

* Added Comment Nodes. Rebuilt react.

* Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
Ian Arawjo
9b0d3a7779 group by different 'LLM sets' in Vis Nodes 2023-06-23 18:07:23 -04:00
Ian Arawjo
ef51e9e9cf Minor bug fix with empty response str in Vis Node 2023-06-23 09:28:40 -04:00
ianarawjo
be9c176f7c
Autosaving, on the way to better plots (#78)
* Flow autosaving every 60 seconds

* Set viewport upon resetFlow

* Added x-axis, y-axis etc header to Vis node. Ensured left padding sizes to shortnames.

* When num gen per prompt = 1, now plots single bar chart w solid LLM color

* Rebuilt react and update package version
2023-06-23 09:16:48 -04:00
ianarawjo
4bd5c15e82
Immediate bug fix when extending ChainForge examples that were created before the new OpenAI 'functions' feature (#77)
* Fix bug in call_chatgpt

* Bug fix and add New Flow button
2023-06-21 13:55:44 -04:00
ianarawjo
ea3d730bba
Merge 0.1.7 (#76)
* Added asMarkdownAST to ResponseInfo

* Increased margin between LLM responses in response inspector

* Added Inspect Results footer to eval node after run

* Fixed bug when cacheing responses after deleting a model and adding a new one

* Fixed bug in catching error in PromptTemplate's `is_concrete()` method

* Improvements to response inspector UI

* Ensured eval results (scores) are sorted alongside their responses the response inspector UI

* Removed response previews footer, replaced with Inspect responses button

* Prompt and eval nodes now load cached responses upon initialization

* Rebuilt React and raised package version
2023-06-21 10:05:46 -04:00
Ian Arawjo
d283cfdaf6 Bug fixes to queryLLM and merge responses 2023-06-17 09:34:23 -04:00
ianarawjo
8400b56f94
Change default boolean vis to be accuracy bar chart (#75) 2023-06-16 12:31:06 -04:00
Ian Arawjo
08bd734e96 Update xy pos of tabular nodes in oaievals 2023-06-15 15:48:49 -04:00
ianarawjo
1d08507c93
Load OpenAI evals as example flows (#74)
* Add OpenAI Evals tab to Example Flows pane.

* Add OpenAI evals examples (preconverted).

* Set unique IDs for each oaievals cforge file

* Use contenteditable divs in tables to improve performance.

* Update eval code to use json.loads instead of eval()

* Fix bug with $s in templates

* Update package info and point oaievals to main branch

* Made column headers use contenteditable p tags

* Add requests to dependency list

* Rebuilt react and updated package version
2023-06-15 15:41:58 -04:00
ianarawjo
655e1e6312
Add Azure OpenAI support (#72)
* Add OpenAI function calls to settings screen for Azure models, and backend support in the Azure call for functions and function_calls
2023-06-13 18:01:24 -04:00
ianarawjo
e5b940b96b
Add basic support for OpenAI function calls (#73)
* Also includes start of categorical variables support in vis node

* Collapses same model responses (within `n` responses requested) and provides number of times they appeared

* Adds basic support for OpenAI function calls.

* Adds example flow illustrating OpenAI func calls
2023-06-13 17:33:15 -04:00
ianarawjo
caf8a3642b
Bug fix when inputing multiple vars from different prompt nodes into vis nodes (#70) 2023-06-12 12:06:38 -04:00
ianarawjo
915f30988d
v0.1.5: Tabular Data Node, Evaluator Print Output (#66)
* Use Mantine Textarea controlled for Textfields node, instead of textarea

* Added Tabular data node

* TabularData context menus

* Make TabularData template hooks responsive to column name changes.

* Reduced spacing between template hooks

* Better table styling and rename column popup

* Add 'carry with' feature to Prompt Permutation recursive generation using `associate_id`

* Much nicer var tags on inspect window

* Nicer styling for LLM group headers in inspect screen

* Pass metavars around in the backend to surface them to front-end

* Set min-height on inspect node to be larger

* Added in-line print in Eval nodes

* Append error message to print output

* Minor inspect node CSS tweaks and fixes

* Removed mix-blend-mode due to performance issues scrolling large text

* Added ground truth eval example for math problems

* Updated React build and version number
2023-06-11 11:48:27 -04:00
ianarawjo
000b61200c
0.1.4: Failure Progress on Prompt Nodes, and others (#62)
* Lint Python code with ruff (#60)

* Failure progress on Prompt Nodes

* Change PromptNode preview container color

* Ensure LLM colors are unique and the same across nodes

* Reset LLM colors upon flow load

* Add LLM colors to 3D scatterplot

* Extract inspector internals into separate component. 

* Added inspect modal.

* Lower rate of failure for dummy LLM responses

* Fix useEffect bug in LLMResponseInspector

* Fix export to excel bug

* Remove dependence on browser support for regex negative lookbehind

* Use monospace font in textareas in Safari

* Fix settings modal bug in FireFox

* Change version

* Update README.md
2023-06-07 20:36:55 -04:00
ianarawjo
c6da2314e7
Add Example flows pane and other minor improvements (#59)
* Example flows pane.

* Better Add+ Node UI.

* Added system message comparison example (see .

* Let user switch to gpt4 if initial model is gpt3.5
2023-06-03 14:32:37 -04:00
ianarawjo
7eb5aaa26d
Model settings, more models, and visible temperature (#57)
* Model settings forms

* Editable nicknames and emojis

* Saving and loading model settings

* Temperature indicator on LLM items in PromptNodes

* Ensure LLM nicknames are unique

* Detect when PaLM blocks responses and output standard error msg in response instead

* Fix examples/ to use new cache format

* Add helpful 'could not reach server' text on countQueries fail

* Add Dalai model settings

* Rebuild react and update package version
2023-06-01 15:08:17 -04:00
Ian Arawjo
1e88328598 Version 0.1.2.4 2023-05-28 11:25:14 -04:00
Ian Arawjo
8ccbb9bebd More informative header text on selected vars in inspect nodes 2023-05-28 11:16:13 -04:00
Ian Arawjo
443c3f15c4 Remove auto-capitalization on template var badges 2023-05-28 11:13:48 -04:00
Ian Arawjo
9183fec3e3 Update package to 0.1.2.2 2023-05-24 10:21:30 -04:00