6 Commits

Author SHA1 Message Date
ianarawjo
bdfeb5c26f
Add human ratings to inspectors (#244)
* Add human ratings to inspectors

* Store human labels in cache, not resp objs

* Change rating UI to pull from Zustand store

* Lazy load inspectors

* Update version and rebuild app
2024-03-14 13:02:47 -04:00
ianarawjo
7e1f43688f
"Small" changes (#213)
* Remove notification dots

* Add batch uids to response objects.

* Regroup responses by batch ids in inspectors. Add batch ids to resp objs. Update examples.

* Bug fix: clear RF state first before loading a flow

* Add random sample toggle to Tabular Data node

* Make sample UI loc conditional on num cols and fit nicer into whitespace

* Adds 'settings template vars' to parametrize on model settings.

* Typecast settings vars params

* Rebuild app and update version
2024-01-19 20:23:24 -05:00
ianarawjo
f43861f075
Add Simple Eval Node. Other minor UI improvements. (#121)
* Add red dot in Inspect Responses footer to indicate something changed

* Abstract out inspect footer button to component

* Add tooltips to AddNode menu items.

* Simple eval wip

* Add menu sections to Add Node. Minor tweaks to simple eval.

* Save state of simple eval when editing fields

* Add 'only show scores' toggle to response inspector

* Change 2 example flows to use simple evals. Fix bg of toolbar buttons.

* Update version and rebuild react
2023-08-19 16:50:11 -04:00
ianarawjo
f7fab1d76d
Update 0.2.5: Add Chat Turns, LLM Scorers (#109)
* Add LLM scorer node (#107)

* Modularize the LLM list container, extracting it from prompt node

* Working LLM scorer node

* Bug and minor fixes

* Change modals to use perc left. 

* Add inspect response footer to LLMEvalNode.

* Make Play buttons light green

* Fix React errors w keys in JSX arrays

* Add Chat Turn node and support for chat history (#108)

* Adds chat_history across backend's cache and querying mechanisms.

* Adds Chat Turn nodes, which allow for continuing a conversation.

* Adds automatic conversions of ChatHistory (in OpenAI format) to Anthropic and Google PaLM's chat formats. Converts chat history to appropriate format and passes it as context in the API call.

* Bug fix and error popup when missing past convo in Chat Turn

* Bug squashing to progress in chat turn node

* bug squashing

* Color false scores bright red in eval inspector

* fix tooltip when cont chat present

* Rebuild react

* bug fix llm eval node

* Add HF chat model support.

* Show multiple response objs in table inspector view

* Fix LLM item deletion bug

* Rebuild react and update package version

* Fix obscure bug when LLM outputs have no 'llm' property (due to prior CF version)

* Fix isLooselyEqual bug

* Update examples so that their cached 'fields' include llm nicknames

* rebuild react

* Add Chelse to readme
2023-07-26 12:06:09 -04:00
ianarawjo
b33397930b
TypeScript backend, HuggingFace models, JavaScript evaluators, Comment Nodes, and more (#81)
* Beginning to convert Python backend to Typescript

* Change all fetch() calls to fetch_from_backend switcher

* wip converting query.py to query.ts

* wip started utils.js conversion. Tested that OpenAI API call works

* more progress on converting utils.py to Typescript

* jest tests for query, utils, template.ts. Confirmed PromptPipeline works.

* wip converting queryLLM in flask_app to TS

* Tested queryLLM and StorageCache compressed saving/loading

* wip execute() in backend.ts

* Added execute() and tested w concrete func. Need to test eval()

* Added craco for optional webpack config. Config'd for TypeScript with Node.js packages browserify'd

* Execute JS code on iframe sandbox

* Tested and working JS Evaluator execution.

* wip swapping backends

* Tested TypeScript backendgit status! :) woot

* Added fetchEnvironAPIKeys to Flask server to fetch os.environ keys when running locally

* Route Anthropic calls through Flask when running locally

* Added info button to Eval nodes. Rebuilt react

* Edits to info modal on Eval node

* Remove/error out on Python eval nodes when not running locally.

* Check browser compat and display error if not supported

* Changed all example flows to use JS. Bug fix in query.ts

* Refactored to LLMProvider to streamline model additions

* Added HuggingFace models API

* Added back Dalai call support, routing through Flask

* Remove flask app calls and socketio server that are no longer used

* Added Comment Nodes. Rebuilt react.

* Fix PaLM temp=0 build, update package vers and rebuild react
2023-06-30 15:11:20 -04:00
ianarawjo
c6da2314e7
Add Example flows pane and other minor improvements (#59)
* Example flows pane.

* Better Add+ Node UI.

* Added system message comparison example (see .

* Let user switch to gpt4 if initial model is gpt3.5
2023-06-03 14:32:37 -04:00