Daniel's personal website

LLM frameworks

frameworkstooling

This is an incomplete doc about an assortment of things loosely grouped for now? These are external tools that support interacting with multiple LLMs for querying, evaluating, or developing.

  • ChainForge - chainforge.ai

    • "ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation and experimentation."
    • HT: Ian Arawjo (website, Twitter
    • citation: @arawjo2023chainforge

  • Comet - comet.com/

    • "Comet provides a meta machine learning platform that enables data scientists & teams to track, compare, explain, & optimize their experiment tracking."

    • HT: @omarsar0

      I have been really impressed with the new prompting tools by @Cometml. I already use their tools for tracking and managing my fine-tuned LLMs, so it's cool to see that they also enable LLMOps and prompting tools to easily track and debug prompts at scale.

  • Cursive - github.com/meistrari/cursive-py

  • GodMode - github.com/smol-ai/GodMode

  • LangChain - github.com/langchain-ai/langchain

  • ragas - explodinggradients/ragas

    • "Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines"
    • HT: Aaron Tay (website | Twitter)

@aarontay via Twitter on Sep 8, 2023

My anecdotal experiences trying academic search + generative AI that generate direct answers with citations using RAG (retrieval Augmented Generation) method (1)