Langroid

Langroid — интуитивно понятный фреймворк на Python для создания надежных многоагентных приложений на базе LLM. Используйте RAG, вызов функций и интегрируйте различные LLM для совместных AI-решений.

Data Science и ML Безопасность и тестирование Инструменты разработчика ★ 3,392 GitHub (3,392 ★)

Langroid — это интуитивно понятный, легковесный, расширяемый и принципиальный Python-фреймворк для простого создания приложений на основе LLM, разработанный исследователями из CMU и UW-Madison. Он позволяет пользователям создавать Агентов, оснащать их дополнительными компонентами, такими как LLM, векторные хранилища и инструменты/функции, назначать им задачи и решать проблемы совместно, обмениваясь сообщениями. Эта мультиагентная парадигма упрощает опыт разработчика, предлагая новый подход к созданию приложений с LLM, разработанный для бесшовной работы практически с любой LLM без зависимости от других крупных LLM-фреймворков.

Ключевые возможности

01Обеспечивает многолетнее сотрудничество с помощью обмена сообщениями.

02Предлагает надежные возможности RAG с DocChatAgent и различными парсерами документов.

03Поддерживает широкий спектр LLM, включая локальные и удаленные модели.

04Интуитивно понятный и расширяемый мультиагентный фреймворк для LLM-приложений.

05Включает поддержку вызова функций, структурированного извлечения информации и интеграции инструментов.

063,392 звезды на GitHub

Варианты использования

01Разработка безопасных программных приложений с использованием ИИ-агентов для управления уязвимостями.

02Извлечение структурированной информации из документов с помощью локальных или удаленных LLM.

03Создание мультиагентных RAG-систем для специализированных областей, таких как фармаконадзор.

Documentation · Examples Repo · Discord · Contributing

Langroid is an intuitive, lightweight, extensible and principled Python framework to easily build LLM-powered applications, from CMU and UW-Madison researchers. You set up Agents, equip them with optional components (LLM, vector-store and tools/functions), assign them tasks, and have them collaboratively solve a problem by exchanging messages. This Multi-Agent paradigm is inspired by the Actor Framework (but you do not need to know anything about this!).

Langroid is a fresh take on LLM app-development, where considerable thought has gone into simplifying the developer experience; it does not use Langchain, or any other LLM framework, and works with practically any LLM.

🔥 Read the (WIP) overview of the langroid architecture, and a quick tour of Langroid.

🔥 MCP Support: Allow any LLM-Agent to leverage MCP Servers via Langroid's simple MCP tool adapter that converts the server's tools into Langroid's ToolMessage instances.

📢 Companies are using/adapting Langroid in production. Here is a quote:

Nullify uses AI Agents for secure software development. It finds, prioritizes and fixes vulnerabilities. We have internally adapted Langroid's multi-agent orchestration framework in production, after evaluating CrewAI, Autogen, LangChain, Langflow, etc. We found Langroid to be far superior to those frameworks in terms of ease of setup and flexibility. Langroid's Agent and Task abstractions are intuitive, well thought out, and provide a great developer experience. We wanted the quickest way to get something in production. With other frameworks it would have taken us weeks, but with Langroid we got to good results in minutes. Highly recommended!
-- Jacky Wong, Head of AI at Nullify.

🔥 See this Intro to Langroid blog post from the LanceDB team

🔥 Just published in ML for Healthcare (2024): a Langroid-based Multi-Agent RAG system for pharmacovigilance, see blog post

We welcome contributions: See the contributions document for ideas on what to contribute.

Are you building LLM Applications, or want help with Langroid for your company, or want to prioritize Langroid features for your company use-cases? Prasad Chalasani is available for consulting (advisory/development): pchalasani at gmail dot com.

Sponsorship is also accepted via GitHub Sponsors

Questions, Feedback, Ideas? Join us on Discord!

Quick glimpse of coding with Langroid

This is just a teaser; there's much more, like function-calling/tools, Multi-Agent Collaboration, Structured Information Extraction, DocChatAgent (RAG), SQLChatAgent, non-OpenAI local/remote LLMs, etc. Scroll down or see docs for more. See the Langroid Quick-Start Colab that builds up to a 2-agent information-extraction example using the OpenAI ChatCompletion API. See also this version that uses the OpenAI Assistants API instead.

🔥 just released! Example script showing how you can use Langroid multi-agents and tools to extract structured information from a document using only a local LLM (Mistral-7b-instruct-v0.2).

import langroid as lr
import langroid.language_models as lm

# set up LLM
llm_cfg = lm.OpenAIGPTConfig( # or OpenAIAssistant to use Assistant API 
  # any model served via an OpenAI-compatible API
  chat_model=lm.OpenAIChatModel.GPT4o, # or, e.g., "ollama/mistral"
)
# use LLM directly
mdl = lm.OpenAIGPT(llm_cfg)
response = mdl.chat("What is the capital of Ontario?", max_tokens=10)

# use LLM in an Agent
agent_cfg = lr.ChatAgentConfig(llm=llm_cfg)
agent = lr.ChatAgent(agent_cfg)
agent.llm_response("What is the capital of China?") 
response = agent.llm_response("And India?") # maintains conversation state 

# wrap Agent in a Task to run interactive loop with user (or other agents)
task = lr.Task(agent, name="Bot", system_message="You are a helpful assistant")
task.run("Hello") # kick off with user saying "Hello"

# 2-Agent chat loop: Teacher Agent asks questions to Student Agent
teacher_agent = lr.ChatAgent(agent_cfg)
teacher_task = lr.Task(
  teacher_agent, name="Teacher",
  system_message="""
    Ask your student concise numbers questions, and give feedback. 
    Start with a question.
    """
)
student_agent = lr.ChatAgent(agent_cfg)
student_task = lr.Task(
  student_agent, name="Student",
  system_message="Concisely answer the teacher's questions.",
  single_round=True,
)

teacher_task.add_sub_task(student_task)
teacher_task.run()

🔥 Updates/Releases

Click to expand

🚀 Demo

Suppose you want to extract structured information about the key terms of a commercial lease document. You can easily do this with Langroid using a two-agent system, as we show in the langroid-examples repo. (See this script for a version with the same functionality using a local Mistral-7b model.) The demo showcases just a few of the many features of Langroid, such as:

Multi-agent collaboration: LeaseExtractor is in charge of the task, and its LLM (GPT4) generates questions to be answered by the DocAgent.
Retrieval augmented question-answering, with source-citation: DocAgent LLM (GPT4) uses retrieval from a vector-store to answer the LeaseExtractor's questions, cites the specific excerpt supporting the answer.
Function-calling (also known as tool/plugin): When it has all the information it needs, the LeaseExtractor LLM presents the information in a structured format using a Function-call.

Here is what it looks like in action (a pausable mp4 video is here).

⚡ Highlights

(For a more up-to-date list see the Updates/Releases section above)

Agents as first-class citizens: The Agent class encapsulates LLM conversation state, and optionally a vector-store and tools. Agents are a core abstraction in Langroid; Agents act as message transformers, and by default provide 3 responder methods, one corresponding to each entity: LLM, Agent, User.
Tasks: A Task class wraps an Agent, and gives the agent instructions (or roles, or goals), manages iteration over an Agent's responder methods, and orchestrates multi-agent interactions via hierarchical, recursive task-delegation. The Task.run() method has the same type-signature as an Agent's responder's methods, and this is key to how a task of an agent can delegate to other sub-tasks: from the point of view of a Task, sub-tasks are simply additional responders, to be used in a round-robin fashion after the agent's own responders.
Modularity, Reusability, Loose coupling: The Agent and Task abstractions allow users to design Agents with specific skills, wrap them in Tasks, and combine tasks in a flexible way.
LLM Support: Langroid supports OpenAI LLMs as well as LLMs from hundreds of providers (local/open or remote/commercial) via proxy libraries and local model servers such as ollama, oobabooga, LiteLLM that in effect mimic the OpenAI API. See the supported LLMs.
Caching of LLM responses: Langroid supports Redis to cache LLM responses.
Vector-stores: Qdrant, Chroma, LanceDB, Pinecone, PostgresDB (PGVector), Weaviate are currently supported. Vector stores allow for Retrieval-Augmented-Generation (RAG).
Grounding and source-citation: Access to external documents via vector-stores allows for grounding and source-citation.
Observability, Logging, Lineage: Langroid generates detailed logs of multi-agent interactions and maintains provenance/lineage of messages, so that you can trace back the origin of a message.
Tools/Plugins/Function-calling: Langroid supports OpenAI's function calling, as well as an equivalent ToolMessage mechanism which works with any LLM, not just OpenAI's. Function calling and tools have the same developer-facing interface, implemented using Pydantic, which makes it very easy to define tools/functions and enable agents to use them. Benefits of using Pydantic are that you never have to write complex JSON specs for function calling, and when the LLM hallucinates malformed JSON, the Pydantic error message is sent back to the LLM so it can fix it.

⚙️ Installation and Setup

Install `langroid`

Langroid requires Python 3.11+. We recommend using a virtual environment. Use pip to install a bare-bones slim version of langroid (from PyPi) to your virtual environment:

pip install langroid

The core Langroid package lets you use OpenAI Embeddings models via their API. If you instead want to use the sentence-transformers embedding models from HuggingFace, install Langroid like this:

pip install "langroid[hf-embeddings]"

For many practical scenarios, you may need additional optional dependencies:

To use various document-parsers, install langroid with the doc-chat extra:
```
pip install "langroid[doc-chat]"
```
For "chat with databases", use the db extra:
```
pip install "langroid[db]"
```
You can specify multiple extras by separating them with commas, e.g.:
```
pip install "langroid[doc-chat,db]"
```
To simply install all optional dependencies, use the all extra (but note that this will result in longer load/startup times and a larger install size):
```
pip install "langroid[all]"
```

Optional Installs for using SQL Chat with a PostgreSQL DB

If you are using SQLChatAgent (e.g. the script examples/data-qa/sql-chat/sql_chat.py), with a postgres db, you will need to:

Install PostgreSQL dev libraries for your platform, e.g.
- sudo apt-get install libpq-dev on Ubuntu,
- brew install postgresql on Mac, etc.
Install langroid with the postgres extra, e.g. pip install langroid[postgres] or poetry add "langroid[postgres]" or poetry install -E postgres, (or the corresponding uv versions, e.g. uv add "langroid[postgres]" or uv pip install langroid[postgres]). If this gives you an error, try pip install psycopg2-binary in your virtualenv.

📝 If you get strange errors involving mysqlclient, try doing pip uninstall mysqlclient followed by pip install mysqlclient.

Set up environment variables (API keys, etc)

To get started, all you need is an OpenAI API Key. If you don't have one, see this OpenAI Page. (Note that while this is the simplest way to get started, Langroid works with practically any LLM, not just those from OpenAI. See the guides to using Open/Local LLMs, and other non-OpenAI proprietary LLMs.)

In the root of the repo, copy the .env-template file to a new file .env:

cp .env-template .env

Then insert your OpenAI API Key. Your .env file should look like this (the organization is optional but may be required in some scenarios).

OPENAI_API_KEY=your-key-here-without-quotes
OPENAI_ORGANIZATION=optionally-your-organization-id

Alternatively, you can set this as an environment variable in your shell (you will need to do this every time you open a new shell):

export OPENAI_API_KEY=your-key-here-without-quotes

Optional Setup Instructions (click to expand)

All of the following environment variable settings are optional, and some are only needed to use specific features (as noted below).

Qdrant Vector Store API Key, URL. This is only required if you want to use Qdrant cloud. Alternatively Chroma or LanceDB are also currently supported. We use the local-storage version of Chroma, so there is no need for an API key.
Redis Password, host, port: This is optional, and only needed to cache LLM API responses using Redis Cloud. Redis offers a free 30MB Redis account which is more than sufficient to try out Langroid and even beyond. If you don't set up these, Langroid will use a pure-python Redis in-memory cache via the Fakeredis library.
Momento Serverless Caching of LLM API responses (as an alternative to Redis). To use Momento instead of Redis:
- enter your Momento Token in the .env file, as the value of MOMENTO_AUTH_TOKEN (see example file below),
- in the .env file set CACHE_TYPE=momento (instead of CACHE_TYPE=redis which is the default).
GitHub Personal Access Token (required for apps that need to analyze git repos; token-based API calls are less rate-limited). See this GitHub page.
Google Custom Search API Credentials: Only needed to enable an Agent to use the GoogleSearchTool. To use Google Search as an LLM Tool/Plugin/function-call, you'll need to set up a Google API key, then setup a Google Custom Search Engine (CSE) and get the CSE ID. (Documentation for these can be challenging, we suggest asking GPT4 for a step-by-step guide.) After obtaining these credentials, store them as values of GOOGLE_API_KEY and GOOGLE_CSE_ID in your .env file. Full documentation on using this (and other such "stateless" tools) is coming soon, but in the meantime take a peek at this chat example, which shows how you can easily equip an Agent with a GoogleSearchtool.

If you add all of these optional variables, your .env file should look like this:

OPENAI_API_KEY=your-key-here-without-quotes
GITHUB_ACCESS_TOKEN=your-personal-access-token-no-quotes
CACHE_TYPE=redis # or momento
REDIS_PASSWORD=your-redis-password-no-quotes
REDIS_HOST=your-redis-hostname-no-quotes
REDIS_PORT=your-redis-port-no-quotes
MOMENTO_AUTH_TOKEN=your-momento-token-no-quotes # instead of REDIS* variables
QDRANT_API_KEY=your-key
QDRANT_API_URL=https://your.url.here:6333 # note port number must be included
GOOGLE_API_KEY=your-key
GOOGLE_CSE_ID=your-cse-id

Optional setup instructions for Microsoft Azure OpenAI(click to expand)

When using Azure OpenAI, additional environment variables are required in the .env file. This page Microsoft Azure OpenAI provides more information, and you can set each environment variable as follows:

AZURE_OPENAI_API_KEY, from the value of API_KEY
AZURE_OPENAI_API_BASE from the value of ENDPOINT, typically looks like https://your.domain.azure.com.
For AZURE_OPENAI_API_VERSION, you can use the default value in .env-template, and latest version can be found here
AZURE_OPENAI_DEPLOYMENT_NAME is the name of the deployed model, which is defined by the user during the model setup
AZURE_OPENAI_MODEL_NAME Azure OpenAI allows specific model names when you select the model for your deployment. You need to put precisly the exact model name that was selected. For example, GPT-4 (should be gpt-4-32k or gpt-4).
AZURE_OPENAI_MODEL_VERSION is required if AZURE_OPENAI_MODEL_NAME = gpt=4, which will assist Langroid to determine the cost of the model

🐳 Docker Instructions

We provide a containerized version of the langroid-examples repository via this Docker Image. All you need to do is set up environment variables in the .env file. Please follow these steps to setup the container:

# get the .env file template from `langroid` repo
wget -O .env https://raw.githubusercontent.com/langroid/langroid/main/.env-template

# Edit the .env file with your favorite editor (here nano), and remove any un-used settings. E.g. there are "dummy" values like "your-redis-port" etc -- if you are not using them, you MUST remove them.
nano .env

# launch the container (the appropriate image for your architecture will be pulled automatically)
docker run -it --rm  -v ./.env:/langroid/.env langroid/langroid:latest

# Use this command to run any of the scripts in the `examples` directory
python examples/<Path/To/Example.py>

🎉 Usage Examples

These are quick teasers to give a glimpse of what you can do with Langroid and how your code would look.

⚠️ The code snippets below are intended to give a flavor of the code and they are not complete runnable examples! For that we encourage you to consult the langroid-examples repository.

ℹ️ The various LLM prompts and instructions in Langroid have been tested to work well with GPT-4 (and to some extent GPT-4o). Switching to other LLMs (local/open and proprietary) is easy (see guides mentioned above), and may suffice for some applications, but in general you may see inferior results unless you adjust the prompts and/or the multi-agent setup.

📖 Also see the Getting Started Guide for a detailed tutorial.

Click to expand any of the code examples below. All of these can be run in a Colab notebook:

Direct interaction with LLM

import langroid.language_models as lm

mdl = lm.OpenAIGPT(
    lm.OpenAIGPTConfig(
        chat_model=lm.OpenAIChatModel.GPT4o, # or, e.g.  "ollama/qwen2.5"
    ),
)

messages = [
  lm.LLMMessage(content="You are a helpful assistant",  role=lm.Role.SYSTEM), 
  lm.LLMMessage(content="What is the capital of Ontario?",  role=lm.Role.USER),
]

response = mdl.chat(messages, max_tokens=200)
print(response.message)

See the guides to use (local/open LLMs or remote/commercial LLMs).

Interaction with non-OpenAI LLM (local or remote)

Local model: if model is served at `http://localhost:8000`:

cfg = lm.OpenAIGPTConfig(
  chat_model="local/localhost:8000", 
  chat_context_length=4096
)
mdl = lm.OpenAIGPT(cfg)
# now interact with it as above, or create an Agent + Task as shown below.

Define an agent, set up a task, and run it

import langroid as lr

agent = lr.ChatAgent()

# get response from agent's LLM, and put this in an interactive loop...
# answer = agent.llm_response("What is the capital of Ontario?")
  # ... OR instead, set up a task (which has a built-in loop) and run it
task = lr.Task(agent, name="Bot") 
task.run() # ... a loop seeking response from LLM or User at each turn

Three communicating agents

A toy numbers game, where when given a number n:

repeater_task's LLM simply returns n,
even_task's LLM returns n/2 if n is even, else says "DO-NOT-KNOW"
odd_task's LLM returns 3*n+1 if n is odd, else says "DO-NOT-KNOW"

Each of these Tasks automatically configures a default ChatAgent.

import langroid as lr
from langroid.utils.constants import NO_ANSWER

repeater_task = lr.Task(
    name = "Repeater",
    system_message="""
    Your job is to repeat whatever number you receive.
    """,
    llm_delegate=True, # LLM takes charge of task
    single_round=False, 
)

even_task = lr.Task(
    name = "EvenHandler",
    system_message=f"""
    You will be given a number. 
    If it is even, divide by 2 and say the result, nothing else.
    If it is odd, say {NO_ANSWER}
    """,
    single_round=True,  # task done after 1 step() with valid response
)

odd_task = lr.Task(
    name = "OddHandler",
    system_message=f"""
    You will be given a number n. 
    If it is odd, return (n*3+1), say nothing else. 
    If it is even, say {NO_ANSWER}
    """,
    single_round=True,  # task done after 1 step() with valid response
)

Then add the even_task and odd_task as sub-tasks of repeater_task, and run the repeater_task, kicking it off with a number as input:

repeater_task.add_sub_task([even_task, odd_task])
repeater_task.run("3")

Simple Tool/Function-calling example

Langroid leverages Pydantic to support OpenAI's Function-calling API as well as its own native tools. The benefits are that you don't have to write any JSON to specify the schema, and also if the LLM hallucinates a malformed tool syntax, Langroid sends the Pydantic validation error (suitably sanitized) to the LLM so it can fix it!

Simple example: Say the agent has a secret list of numbers, and we want the LLM to find the smallest number in the list. We want to give the LLM a probe tool/function which takes a single number n as argument. The tool handler method in the agent returns how many numbers in its list are at most n.

First define the tool using Langroid's ToolMessage class:

import langroid as lr

class ProbeTool(lr.agent.ToolMessage):
  request: str = "probe" # specifies which agent method handles this tool
  purpose: str = """
        To find how many numbers in my list are less than or equal to  
        the <number> you specify.
        """ # description used to instruct the LLM on when/how to use the tool
  number: int  # required argument to the tool

Then define a SpyGameAgent as a subclass of ChatAgent, with a method probe that handles this tool:

class SpyGameAgent(lr.ChatAgent):
  def __init__(self, config: lr.ChatAgentConfig):
    super().__init__(config)
    self.numbers = [3, 4, 8, 11, 15, 25, 40, 80, 90]

  def probe(self, msg: ProbeTool) -> str:
    # return how many numbers in self.numbers are less or equal to msg.number
    return str(len([n for n in self.numbers if n <= msg.number]))

We then instantiate the agent and enable it to use and respond to the tool:

spy_game_agent = SpyGameAgent(
    lr.ChatAgentConfig(
        name="Spy",
        vecdb=None,
        use_tools=False, #  don't use Langroid native tool
        use_functions_api=True, # use OpenAI function-call API
    )
)
spy_game_agent.enable_message(ProbeTool)

For a full working example see the chat-agent-tool.py script in the langroid-examples repo.

Tool/Function-calling to extract structured information from text

Suppose you want an agent to extract the key terms of a lease, from a lease document, as a nested JSON structure. First define the desired structure via Pydantic models:

from pydantic import BaseModel
class LeasePeriod(BaseModel):
    start_date: str
    end_date: str


class LeaseFinancials(BaseModel):
    monthly_rent: str
    deposit: str

class Lease(BaseModel):
    period: LeasePeriod
    financials: LeaseFinancials
    address: str

Then define the LeaseMessage tool as a subclass of Langroid's ToolMessage. Note the tool has a required argument terms of type Lease:

import langroid as lr

class LeaseMessage(lr.agent.ToolMessage):
    request: str = "lease_info"
    purpose: str = """
        Collect information about a Commercial Lease.
        """
    terms: Lease

Then define a LeaseExtractorAgent with a method lease_info that handles this tool, instantiate the agent, and enable it to use and respond to this tool:

class LeaseExtractorAgent(lr.ChatAgent):
    def lease_info(self, message: LeaseMessage) -> str:
        print(
            f"""
        DONE! Successfully extracted Lease Info:
        {message.terms}
        """
        )
        return json.dumps(message.terms.dict())
    
lease_extractor_agent = LeaseExtractorAgent()
lease_extractor_agent.enable_message(LeaseMessage)

See the chat_multi_extract.py script in the langroid-examples repo for a full working example.

Chat with documents (file paths, URLs, etc)

Langroid provides a specialized agent class DocChatAgent for this purpose. It incorporates document sharding, embedding, storage in a vector-DB, and retrieval-augmented query-answer generation. Using this class to chat with a collection of documents is easy. First create a DocChatAgentConfig instance, with a doc_paths field that specifies the documents to chat with.

import langroid as lr
from langroid.agent.special import DocChatAgentConfig, DocChatAgent

config = DocChatAgentConfig(
  doc_paths = [
    "https://en.wikipedia.org/wiki/Language_model",
    "https://en.wikipedia.org/wiki/N-gram_language_model",
    "/path/to/my/notes-on-language-models.txt",
  ],
  vecdb=lr.vector_store.QdrantDBConfig(),
)

Then instantiate the DocChatAgent (this ingests the docs into the vector-store):

agent = DocChatAgent(config)

Then we can either ask the agent one-off questions,

agent.llm_response("What is a language model?")

or wrap it in a Task and run an interactive loop with the user:

task = lr.Task(agent)
task.run()

See full working scripts in the docqa folder of the langroid-examples repo.

🔥 Chat with tabular data (file paths, URLs, dataframes)

Using Langroid you can set up a TableChatAgent with a dataset (file path, URL or dataframe), and query it. The Agent's LLM generates Pandas code to answer the query, via function-calling (or tool/plugin), and the Agent's function-handling method executes the code and returns the answer.

Here is how you can do this:

import langroid as lr
from langroid.agent.special import TableChatAgent, TableChatAgentConfig

Set up a TableChatAgent for a data file, URL or dataframe (Ensure the data table has a header row; the delimiter/separator is auto-detected):

dataset =  "https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv"
# or dataset = "/path/to/my/data.csv"
# or dataset = pd.read_csv("/path/to/my/data.csv")
agent = TableChatAgent(
    config=TableChatAgentConfig(
        data=dataset,
    )
)

Set up a task, and ask one-off questions like this:

task = lr.Task(
  agent, 
  name = "DataAssistant",
  default_human_response="", # to avoid waiting for user input
)
result = task.run(
  "What is the average alcohol content of wines with a quality rating above 7?",
  turns=2 # return after user question, LLM fun-call/tool response, Agent code-exec result
) 
print(result.content)

Or alternatively, set up a task and run it in an interactive loop with the user:

task = lr.Task(agent, name="DataAssistant")
task.run()

For a full working example see the table_chat.py script in the langroid-examples repo.

❤️ Thank you to our supporters

If you like this project, please give it a star ⭐ and 📢 spread the word in your network or social media:

Your support will help build Langroid's momentum and community.

Langroid Co-Founders

Prasad Chalasani (IIT BTech/CS, CMU PhD/ML; Independent ML Consultant)
Somesh Jha (IIT BTech/CS, CMU PhD/CS; Professor of CS, U Wisc at Madison)

Langroid — мультиагентный фреймворк для LLM-приложений на Python

Langroid — лёгкий, интуитивный и расширяемый Python-фреймворк для создания приложений на основе больших языковых моделей (LLM). Разработан исследователями из CMU и UW-Madison. В основе лежит парадигма мультиагентного взаимодействия, вдохновлённая моделью акторов (Actor Model). Вы создаёте агентов, оснащаете их LLM, векторными базами данных, инструментами (tools/functions), назначаете им задачи, и они совместно решают проблему, обмениваясь сообщениями.

Langroid не использует LangChain или другие LLM-фреймворки — это самостоятельная, продуманная альтернатива. Работает с практически любой LLM (OpenAI, локальные модели через Ollama, Gemini, DeepSeek, Cerebras, OpenRouter и др.).

Основные возможности

Агенты и задачи: агент — это сущность с LLM и опциональными инструментами; задача — это интерактивный цикл, в котором агент общается с пользователем или другими агентами.
Мультиагентная оркестрация: задачи могут содержать подзадачи, образуя иерархию. Агенты могут обмениваться сообщениями, вызывать инструменты друг друга.
Инструменты (Tools): поддержка OpenAI tools API, XML-инструментов, MCP-серверов. Инструменты определяются как подклассы ToolMessage.
RAG (Retrieval-Augmented Generation): DocChatAgent для вопросов по документам с чанкингом, эмбеддингами, гибридным поиском.
Структурированное извлечение: извлечение данных в заданном JSON-схемой формате.
SQL-агент: SQLChatAgent для работы с базами данных через естественный язык.
Мультимодальность: поддержка PDF, изображений как входных данных для LLM.
MCP (Model Context Protocol): адаптер для превращения инструментов MCP-сервера в ToolMessage Langroid.
Поддержка различных LLM: OpenAI, Gemini, DeepSeek, Cerebras, OpenRouter, локальные через Ollama, LiteLLM, Portkey AI Gateway и др.
Векторные БД: Qdrant, Pinecone, Weaviate, pgvector, Chroma (через Qdrant).
Парсеры документов: PDF (Docling, PyMuPDF4LLM, GeminiPdfParser, Markitdown), DOCX, PPTX, XLSX, Markdown.

Установка

pip install langroid

Для дополнительных возможностей используйте экстра-зависимости:

pip install langroid[doc-chat] — для RAG (парсеры PDF, DOCX, эмбеддинги).
pip install langroid[db] — для работы с базами данных (SQL).
pip install langroid[all] — всё сразу.

Подробнее — в документации.

Быстрый старт

1. Прямое использование LLM

import langroid.language_models as lm

llm_cfg = lm.OpenAIGPTConfig(
    chat_model=lm.OpenAIChatModel.GPT4o,  # или "ollama/mistral"
)
mdl = lm.OpenAIGPT(llm_cfg)
response = mdl.chat("What is the capital of Ontario?", max_tokens=10)
print(response.content)

2. Агент с LLM

import langroid as lr

agent_cfg = lr.ChatAgentConfig(llm=llm_cfg)
agent = lr.ChatAgent(agent_cfg)
response = agent.llm_response("What is the capital of China?")
print(response.content)

3. Задача (интерактивный диалог с пользователем)

task = lr.Task(agent, name="Bot", system_message="You are a helpful assistant")
task.run("Hello")  # пользователь говорит "Hello", агент отвечает

4. Два агента: учитель и ученик

teacher_agent = lr.ChatAgent(agent_cfg)
teacher_task = lr.Task(
    teacher_agent,
    name="Teacher",
    system_message="""
    Ask your student concise numbers questions, and give feedback.
    Start with a question.
    """,
)
student_agent = lr.ChatAgent(agent_cfg)
student_task = lr.Task(
    student_agent,
    name="Student",
    system_message="Concisely answer the teacher's questions.",
    single_round=True,
)
teacher_task.add_sub_task(student_task)
teacher_task.run()

Конфигурация

Основные компоненты настраиваются через конфигурационные классы:

OpenAIGPTConfig (или OpenAIAssistantConfig) — параметры LLM (модель, температура, max_tokens и т.д.).
ChatAgentConfig — настройки агента (LLM, системное сообщение, инструменты).
Task — обёртка над агентом, управляющая циклом общения (можно задать single_round, interactive, restart и др.).
ToolMessage — базовый класс для определения инструментов (поля, обработчик).

Пример настройки инструмента:

from langroid.agent.tool_message import ToolMessage

class WeatherTool(ToolMessage):
    request: str = "get_weather"
    purpose: str = "Get weather for a location"
    location: str

    def handle(self) -> str:
        # логика вызова API погоды
        return f"Weather in {self.location}: sunny"

MCP (Model Context Protocol)

Langroid поддерживает MCP-серверы через адаптер MCPToolAdapter. Он преобразует инструменты, предоставляемые MCP-сервером, в ToolMessage Langroid, что позволяет любому агенту использовать их. Подробнее — в документации по MCP.

RAG и работа с документами

DocChatAgent — агент для вопросов по документам. Поддерживает:

Чанкинг (в том числе с учётом структуры Markdown).
Эмбеддинги (OpenAI, Gemini, FastEmbed, Llama-cpp).
Гибридный поиск (RRF — Reciprocal Rank Fusion).
Парсинг PDF, DOCX, PPTX, XLSX, Markdown.

Пример:

from langroid.agent.doc_chat_agent import DocChatAgent, DocChatAgentConfig

config = DocChatAgentConfig(llm=llm_cfg)
agent = DocChatAgent(config)
agent.ingest_doc("path/to/document.pdf")
response = agent.llm_response("What is the main topic?")

Поддерживаемые LLM и провайдеры

OpenAI (GPT-4o, o1, o1-mini и др.)
Gemini (через OpenAI-совместимый клиент)
DeepSeek (v3, R1)
Cerebras (очень быстрый инференс)
OpenRouter
Portkey AI Gateway (200+ моделей)
Локальные модели через Ollama (например, ollama/mistral)
LiteLLM Proxy-server
LangDB LLM Gateway
Azure OpenAI
glhf.chat (Qwen2.5-Coder и др.)

Полный список — в документации.

Релизы (последние)

0.54.0 — поддержка Portkey AI Gateway.
0.53.0 — поддержка MCP Tools.
0.52.0 — мультимодальность (PDF, изображения).
0.51.0 — LLMPdfParser для парсинга PDF через LLM.
0.50.0 — структурированный Markdown-чанкинг.
0.49.0 — лёгкое переключение на LiteLLM Proxy.
0.48.0 — Exa Crawler, Markitdown Parser.
0.47.0 — Firecrawl.
0.46.0 — LangDB Gateway.
0.45.0 — парсинг Markdown через Marker.
0.44.0 — отложенные импорты для ускорения запуска.

Полный список изменений — на GitHub Releases.

Ссылки

Вклад и поддержка

Приветствуются любые вклады — идеи, баг-репорты, pull request. См. CONTRIBUTING.md.

Если вы используете Langroid в продакшене или хотите помочь с развитием, свяжитесь с Prasad Chalasani (pchalasani at gmail dot com).

Спонсорство принимается через GitHub Sponsors.

Вопросы и предложения — в Discord.

Источник: https://mcpmarket.com/server/langroid

Langroid

Ключевые возможности

Варианты использования

Documentation · Examples Repo · Discord · Contributing

Quick glimpse of coding with Langroid

🔥 Updates/Releases

🚀 Demo

⚡ Highlights

⚙️ Installation and Setup

Install `langroid`

Set up environment variables (API keys, etc)

🐳 Docker Instructions

🎉 Usage Examples

❤️ Thank you to our supporters

Langroid Co-Founders

Langroid — мультиагентный фреймворк для LLM-приложений на Python

Основные возможности

Установка

Быстрый старт

1. Прямое использование LLM

2. Агент с LLM

3. Задача (интерактивный диалог с пользователем)

4. Два агента: учитель и ученик

Конфигурация

MCP (Model Context Protocol)

RAG и работа с документами

Поддерживаемые LLM и провайдеры

Релизы (последние)

Ссылки

Вклад и поддержка

Какие ключевые функции предлагает Langroid для разработки LLM-приложений?

Подходит ли Langroid для производственной среды?

Какие типы LLM поддерживает Langroid?

Что такое Langroid?

Комментарии

Ключевые возможности

Варианты использования

Documentation · Examples Repo · Discord · Contributing

Quick glimpse of coding with Langroid

🔥 Updates/Releases

🚀 Demo

⚡ Highlights

⚙️ Installation and Setup

Install langroid

Set up environment variables (API keys, etc)

🐳 Docker Instructions

🎉 Usage Examples

❤️ Thank you to our supporters

Langroid Co-Founders

Langroid — мультиагентный фреймворк для LLM-приложений на Python

Основные возможности

Установка

Быстрый старт

1. Прямое использование LLM

2. Агент с LLM

3. Задача (интерактивный диалог с пользователем)

4. Два агента: учитель и ученик

Конфигурация

MCP (Model Context Protocol)

RAG и работа с документами

Поддерживаемые LLM и провайдеры

Релизы (последние)

Ссылки

Вклад и поддержка

Какие ключевые функции предлагает Langroid для разработки LLM-приложений?

Подходит ли Langroid для производственной среды?

Какие типы LLM поддерживает Langroid?

Что такое Langroid?

Комментарии

Install `langroid`