AgentDialogues Framework Beta 1 Released: Open-Source Tool for Multi-Turn Language Model Research

We've open-sourced the research framework powering our toxicity studies, enabling researchers to conduct controlled multi-turn conversations between language models with built-in analytics and dataset generation.

We’re releasing AgentDialogues v0.1.0-beta.1, the open-source framework we developed to conduct our ongoing toxicity research. This Python module enables researchers to simulate multi-turn conversations between language models while capturing detailed interaction data for analysis.

The framework emerged from our need to study toxicity propagation patterns in extended conversations—something difficult to achieve with traditional single-turn evaluation methods. AgentDialogues allows controlled experimentation with different language models, conversation scenarios, and safety mechanisms across hundreds of dialogues.

Key capabilities in Beta 1

The framework provides a modular architecture for simulation development with pluggable configuration and state management. Researchers can define custom conversation scenarios using YAML files and run batch experiments with automated seed injection for reproducible results.

Built-in analytics support includes functions for data exploration in Jupyter notebooks, plotting utilities for visualizing conversation patterns, and automated dataset creation from simulation logs. All interactions are saved as structured JSON with CSV aggregation support for statistical analysis.

CLI support enables large-scale batch runs suitable for systematic research studies. The framework integrates with LangGraph Studio through langgraph.json compatibility, providing visual workflow development capabilities.

Research applications

Our toxicity research used AgentDialogues to simulate 850 multi-turn conversations, enabling us to identify the toxicity echo effect and measure model-specific vulnerability patterns. The framework’s logging capabilities captured the 2-gram repetition data that revealed how language models amplify rather than neutralize toxic input.

The modular design supports various research applications beyond toxicity studies, including conversation quality evaluation, multi-agent coordination research, and safety mechanism testing across different language models and scenarios.

Technical foundation

AgentDialogues provides a stable API for core module functionality while supporting extensible analytics and visualization components. The refactored architecture separates simulation logic from data processing, enabling researchers to focus on experimental design rather than infrastructure development.

The framework supports integration with major language model APIs and local model deployments, allowing researchers to test across different model families and sizes within the same experimental framework.

Future development

This beta release establishes the foundation for community-driven development. We’re planning expanded model support, additional analytics functions, and integration with popular machine learning evaluation frameworks based on research community feedback.

The framework is designed to evolve with the research community’s needs while maintaining API stability for reproducible experiments and longitudinal studies.

AgentDialogues is available on GitHub with comprehensive documentation, setup guides, and example scenarios to help researchers get started with multi-turn language model evaluation.

Repository: github.com/Savalera/agent-dialogues

This release supports Agentic Lab’s mission to advance language model safety through open research tools and reproducible methodologies.