Chidori – A reactive runtime for building durable AI agents

Chidori is a reactive runtime for building AI agents. It provides a framework for building AI agents that are reactive, observable, and robust. It supports building agents with Node.js, Python, and Rust.

It is currently in alpha, and is not yet ready for production use. We are continuing to make significant changes in response to feedback.

• Built from the ground up for constructing agents
• Runtime written in Rust supporting Python and Node.js out of the box
• Build agents that actually work :emoji:
• LLM caching to minimize cost during development
• Optimized for long-running AI workflows
• Embedded code interpreter
• Time travel debugging

Examples

In the table below are examples for Node.js, Python and Rust. You’ll need to scroll horizontally to view each.

The following examples show how to build a simple agent that fetches the top stories from Hacker News and call the OpenAI API to filter to AI related launches and then format that data into markdown. Results from the example are pushed into the Chidori database and can be visualized using the prompt-graph-ui project. We’ll update this example with a pattern that makes those results more accessible soon.

const axios = require('axios');
const {Chidori, GraphBuilder} = require("@1kbirds/chidori");

class Story {
    constructor(title, url, score) {
        this.title = title;
        this.url = url;
        this.score = score;
    }
}

const HN_URL_TOP_STORIES = "https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty";

function fetchStory(id) {
    return axios.get(`https://hacker-news.firebaseio.com/v0/item/${id}.json?print=pretty`)
        .then(response => response.data);
}

function fetchHN() {
    return axios.get(HN_URL_TOP_STORIES)
        .then(response => {
            const storyIds = response.data;
            const tasks = storyIds.slice(0, 30).map(id => fetchStory(id));  // Limit to 30 stories
            return Promise.all(tasks)
                .then(stories => {
                    return stories.map(story => {
                        const { title, url, score } = story;
                        return new Story(title, url, score);
                    });
                });
        });
}

class ChidoriWorker {
    constructor() {
        this.c = new Chidori("0", "http://localhost:9800");  // Assuming this is a connection object, replaced with an empty object for now
    }

    async buildGraph() {
        const g = new GraphBuilder();

        const h = g.customNode({
            name: "FetchTopHN",
            nodeTypeName: "FetchTopHN",
            output: "type FetchTopHN { output: String }"
        });

        const hInterpret = g.promptNode({
            name: "InterpretTheGroup",
            template: `
                Based on the following list of HackerNews threads,
                filter this list to only launches of new AI projects: {{FetchTopHN.output}}
            `
        });
        hInterpret.runWhen(g, h);

        const hFormatAndRank = g.promptNode({
            name: "FormatAndRank",
            template: `
                Format this list of new AI projects in markdown, ranking the most 
                interesting projects from most interesting to least. 
                
                {{InterpretTheGroup.promptResult}}
            `
        });
        hFormatAndRank.runWhen(g, hInterpret);

        await g.commit(this.c, 0)
    }

    async run() {
        // Construct the agent graph
        await this.buildGraph();

        // Start graph execution from the root
        // Implement the functionality of the play function
        await this.c.play(0, 0);

        // Run the node execution loop
        // Implement the functionality of the run_custom_node_loop function
        await this.c.runCustomNodeLoop()
    }
}


async function handleFetchHN(nodeWillExec, cb) {
    const stories = await fetchHN();
    // return JSON.stringify(stories);
    return cb({ "output": JSON.stringify(stories) });
    // return ;
}

async function main() {
    let w = new ChidoriWorker();
    await w.c.startServer(":memory:")
    await w.c.registerCustomNodeHandle("FetchTopHN", handleFetchHN);
    await w.run()
}


main();

About Chidori

Reactive Runtime

At its core, Chidori brings a reactive runtime that orchestrates interactions between different agents and their components. The runtime is comprised of “nodes”, which react to system changes they subscribe to, providing dynamic and responsive behavior in your AI systems. Nodes can encompass code, prompts, vector databases, custom code, services, or even complete systems.

Monitoring and Observability

Chidori ensures comprehensive monitoring and observability of your agents. We record all the inputs and outputs emitted by nodes, enabling us to explain precisely what led to what, enhancing your debugging experience and understanding of the system’s production behavior.

Branching and Time-Travel

With Chidori, you can take snapshots of your system and explore different possible outcomes from that point (branching), or rewind the system to a previous state (time-travel). This functionality improves error handling, debugging, and system robustness by offering alternative pathways and do-overs.

Code Interpreter Environments

Chidori comes with first-class support for code interpreter environments like Deno or Starlark. You can execute code directly within your system, providing quick startup, ease of use, and secure execution. We’re continually working on additional safeguards against running untrusted code, with containerized nodes support coming soon.

FAQ

Why Another AI Framework?

Chidori focuses on the specifics of how LLM+code execution operates rather than providing specific compositions of prompts. Other frameworks haven’t focused on this space, and it’s an important one. We reduce accidental complexity in building systems for long-running agents; this helps developers build successful systems.

Why Chidori?

Chidori is the name of the lightning blade technique used by Kakashi in the Naruto anime series. It also happens to mean Thousand Birds in Japanese, which is a nice coincidence.

Well then why Thousand Birds?

Thousand Birds is a reference to flocks of birds (or a murmuration) and the emergent behavior that arises from their interactions. We think this is a good metaphor for the behavior of long running agents, the internal units of LLM execution within them, and the emergent behavior that arises from their interactions.

Why Rust?

Rust is a great language for building systems, we like the type system and the guarantees provided by it. We also like the performance characteristics of Rust, and the ability to build a single binary that can be deployed anywhere. The Rust ecosystem makes it fairly easy to provide bindings to other languages, which is important for us to provide a good developer experience.

Chidori on GitHub: https://github.com/ThousandBirdsInc/chidori
Platform: AI
⭐️: 481