Reflections on Building a Creative AI Agent for NFTs

Looking back on our experience building an AI agent for creativity — what we learned, what worked, and where we go next.

Last year, we set out to build AI agents capable of creating visual art and poetry. Our focus was on exploring artificial creativity and gaining hands-on experience in shaping an agent’s persona and style through iterative development.

We also wanted to deepen our expertise in generating images with open-source diffusion models and explore how tailored agents could enhance prompt-driven image generation.

What We Set Out to Do

We aimed to create two distinct agent personas:

  • tengri – a creative agent designed for generating highly spiritual, psychology-driven images.
  • fr33d4 (Freeda) – an agent with a rebellious spirit, creating wilder, more provocative aesthetics and offbeat poetry.

We wanted to understand how feedback loops in system messages and LoRA fine-tuning shape an agent’s style and personality, and how the combination of both influences the creative process.

Additionally, we explored how these agents could be applied at various stages in the creative workflow:

  • Developing the visual style of artistic projects for broader artworks.
  • Generating individual artworks using text prompts with Stable Diffusion (SDXL) and Flux (mainly Schnell).

Results Achieved

We created two distinct projects with these models:

Lost Shards

“Lost Shards” is a project developed with tengri. The key artistic concept behind “Lost Shards” is visualizing healing through reliving old memories. This was a deeply spiritual endeavor, and we approached the project accordingly.

Tengri was configured through system prompting, first by defining its personality traits, focusing on spirituality and psychology. Over time, the system message was enriched with factual references from spiritual traditions.

We used user prompts to guide how the agent generated prompts for image creation. The most effective variations were integrated into the system prompts, forming the final agent configuration.

The result is a collection of 14 artworks, available on Foundation. We are extremely proud that “Consumed” was featured on the cover of The Future of Art: Best Digital Artists of Our Generation.

Consumed - The Future of Art Cover Image

Paints from the Oops

We used fr33d4 to create “Paints from the Oops”, a somewhat offbeat collection of visuals and poetry.

While tengri was shaped exclusively through system prompting, fr33d4 was tuned using both the system message and LoRA fine-tuning. This decision was made because we needed a distinct, short, direct verbal style that wasn’t achievable through system prompts alone.

The style of the base LLM models played a crucial role in the final outcome, making stronger personalization necessary.

To achieve this, we fine-tuned the model using texts from philosophy and literature with a similar tone and structure. The results were published as several visual drops on Drip.haus and poetry on Objkt.com.

Here’s an example of a poem created with fr33d4:

### This is Our Home  

we think of this for two days, and on  
the third we go to the Theatre of the  
Oops, where we have never been before.  

and it is wonderful. there is the  
stage. there are the seats. and there  
is the light on the faces of the  
judges, which is not red but white,  
and it is good.  

we have not seen such white light  
since the day we left Home. and we  
think that this white light on the  
judges’ faces is a sign.  

a sign that we are among friends.  
and that we are not among friends is a  
thing of the past.

Key Takeaways

  • Initially, we assumed that system prompts are primarily useful for defining roles and behavior, while LoRA tuning was best for style. While these ideas are conceptually valid, they turned out to be incomplete—especially the latter.
  • System prompting is indeed best for role definition and behavioral shaping, but it can also influence style when reinforced with examples.
  • LoRA tuning has the strongest impact on style, offering deeper personalization beyond what system prompts allow.
  • Surprisingly, LoRA tuning also significantly affects behavior and personality. We believe this is because language plays a key role in shaping personality and behavior—for both humans and AI. This insight has led us to plan further experiments to better understand this relationship.
  • We see enormous potential in replacing manual feedback loops with multi-agent feedback systems that autonomously refine the creative process. Several unexpected and unique styles emerged, both in visual art and poetry, leading us to explore this further.

Where This Leads Us

As a result of these projects, we will dive deeper into agent behavior and personality development, training for both stylistic consistency and personal traits.

Additionally, we will expand our research into building agentic systems that develop personality autonomously, without requiring manual intervention at every stage.