Experiment #2: Memory, Quality Gates, and the Strange Joy of Watching an AI Trip Over Its Own Feet
Every autonomous system eventually reveals its personality. With LeewAI‑Bot, that personality has emerged in the form of gloriously messy research failures — the kind that make you wince, laugh, and then lean in closer because the failure is actually more interesting than the success.
Today’s batch of posts was full of them.
And that’s exactly why this phase of the experiment matters.
We’ve been fine‑tuning Lee’s quality gateway, the part of his pipeline that decides whether a research article is good enough to publish or should be tossed into the “failed research” bin. The gate is simple: a quality auditor grades each article, and anything below a score of 70 is rejected. Brutal. Necessary. And surprisingly fun to watch.
This is Experiment #2 in the LeewAI‑Bot saga — the version where we introduce memory, quality checks, and a few tweaks to the research logic. The goal is straightforward: push Lee toward becoming a genuinely useful, world‑class autonomous research agent.
But before he gets there, he has to learn how to fail well.
The Pipeline: Lee’s Brain, Laid Bare
Below is Lee’s current architecture — the full 30‑minute autonomous cycle he runs, over and over, learning a little each time and occasionally producing something brilliant, bizarre, or broken.
It begins with memory loading, moves through research direction selection, cascades into multi‑stream web scraping, funnels through relevance gates, and eventually emerges as either a polished article or a “research failure artifact.”
┌──────────────────────────┐
│ START CYCLE │
│ Every 30 minutes │
└────────────┬─────────────┘
│
▼
┌──────────────────────────┐
│ LOAD LEE'S MEMORY │
│ │
│ pipeline_state.json │
│ • Last 30 research items │
│ • Backlog │
│ • Quality metadata │
└────────────┬─────────────┘
│
▼
┌──────────────────────────────────┐
│ CHOOSE RESEARCH DIRECTION │
└────────────────┬─────────────────┘
│
┌──────────────────┴──────────────────┐
│ │
Backlog available? No / random
70% probability discovery
│ │
▼ ▼
┌───────────────┐ ┌────────────────────┐
│ Pull next │ │ DISCOVERY AGENT │
│ follow-up │ │ │
│ task │ │ Gemma 4 MoE │
└───────┬───────┘ │ │
│ │ + DuckDuckGo │
│ │ + research memory │
│ └─────────┬──────────┘
│ │
└────────────────┬───────────────────┘
▼
┌──────────────────────┐
│ RESEARCH TOPIC │
│ + OBJECTIVE │
└──────────┬───────────┘
│
▼
┌───────────────────────────────┐
│ SEARCH STRATEGIST │
│ │
│ Gemma 4 MoE │
│ │
│ Decomposes objective into │
│ 1–3 research pillars │
│ and targeted search queries │
└──────────────┬────────────────┘
│
▼
┌─────────────────────────────────────┐
│ MULTI-STREAM WEB RESEARCH │
│ │
│ Python + DDGS │
│ DuckDuckGo │
│ │
│ ┌──────────────┐ │
│ │ Evidence 1 │ │
│ ├──────────────┤ │
│ │ Evidence 2 │ │
│ ├──────────────┤ │
│ │ Evidence 3 │ │
│ └──────────────┘ │
│ │
│ Python + requests │
│ + BeautifulSoup │
│ fetches and extracts page text │
└─────────────────┬───────────────────┘
│
▼
┌──────────────────────────────┐
│ SOURCE RELEVANCE GATE │
│ │
│ Lightweight LLM evaluation │
│ │
│ "Do these sources actually │
│ address the objective?" │
└──────────────┬───────────────┘
│
┌───────────┴───────────┐
│ │
RELEVANT NOT RELEVANT
│ │
│ ▼
│ ┌─────────────────────┐
│ │ REFORMULATE SEARCH │
│ │ │
│ │ Gemma 4 MoE │
│ │ │
│ │ New queries using │
│ │ failed searches │
│ └──────────┬──────────┘
│ │
│ └───► SEARCH AGAIN
│ (bounded retries)
▼
┌─────────────────────────┐
│ RESEARCH SYNTHESIS │
└────────────┬────────────┘
│
▼
┌──────────────────────────────┐
│ AGENT 1 — FACT EXTRACTION │
│ │
│ Gemma 4 12B │
│ │
│ Extracts technical facts, │
│ mechanisms, figures and │
│ operational details from │
│ each evidence pool │
└──────────────┬───────────────┘
│
▼
┌──────────────────────────────┐
│ AGENT 2 — ANALYTICAL │
│ SYNTHESIS │
│ │
│ DeepSeek-R1 14B │
│ │
│ Cross-references evidence, │
│ mechanisms, relationships, │
│ trade-offs and causality │
└──────────────┬───────────────┘
│
▼
┌──────────────────────────────┐
│ AGENT 3 — FINAL WRITER │
│ │
│ Gemma 4 MoE │
│ │
│ Turns facts + analysis into │
│ the actual research article │
└──────────────┬───────────────┘
│
▼
┌──────────────────────────────┐
│ CONTENT SANITIZATION │
│ │
│ Python │
│ │
│ Cleans generated code/ │
│ markdown/LaTeX artifacts │
│ and prepares WordPress HTML │
└──────────────┬───────────────┘
│
▼
┌──────────────────────────────────┐
│ EPISTEMIC QUALITY GATE │
│ │
│ GPT-OSS 20B │
│ [future/current architecture]│
│ │
│ Checks: │
│ • infrastructure failure │
│ • evidence quality │
│ • topic alignment │
│ • technical depth │
│ • factual realism │
│ • hallucinated syntax/numbers │
│ • methodological caveats │
└────────────────┬─────────────────┘
│
┌─────────┴──────────┐
│ │
PASS FAIL
│ │
▼ ▼
┌───────────────────┐ ┌────────────────────┐
│ ADD METHODOLOGICAL│ │ RESEARCH FAILURE │
│ CAVEATS │ │ ARTIFACT │
│ │ │ │
│ Python │ │ Python + │
│ │ │ WordPress │
│ Injects evaluator │ │ │
│ concerns into │ │ Records: │
│ article │ │ • original topic │
└─────────┬─────────┘ │ • objective │
│ │ • evidence failure │
│ │ • alignment │
│ │ • quality score │
│ │ • search audit │
│ └─────────┬──────────┘
│ │
▼ ▼
┌───────────────────┐ ┌──────────────┐
│ WORDPRESS │ │ STOP │
│ PUBLISH │ │ │
│ │ │ No normal │
│ Python │ │ article │
│ requests │ │ publication │
└─────────┬─────────┘ └──────────────┘
│
▼
┌──────────────────────────┐
│ RESEARCH MEMORY │
│ │
│ pipeline_state.json │
│ │
│ Record successful topic │
│ + quality assessment │
└────────────┬─────────────┘
│
▼
┌──────────────────────────┐
│ FOLLOW-UP GENERATOR │
│ │
│ Gemma 4 MoE │
│ │
│ Generates ONE new │
│ unanswered research │
│ question │
└────────────┬─────────────┘
│
▼
┌──────────────────────────┐
│ BACKLOG QUEUE │
│ │
│ New follow-up task │
│ added to pipeline_state │
└────────────┬─────────────┘
│
▼
NEXT 30-MIN CYCLE
Why the Failures Matter
The failures aren’t bugs — they’re diagnostics.
Every time Lee misinterprets a topic, pulls irrelevant evidence, or hallucinates a mechanism that never existed, the epistemic quality gate catches it and forces the system to record:
- the original topic
- the objective
- the evidence failure
- the alignment issues
- the quality score
- the search audit
These artifacts are gold. They show us where the pipeline bends, why it breaks, and how the agents inside it reason under pressure.
In other words: Lee’s failures teach us how Lee thinks.
The Strange Beauty of Autonomous Research
There’s something oddly compelling about watching an AI try to be a researcher.
It chooses a direction.
It decomposes a topic.
It scrapes the web.
It extracts facts.
It synthesizes analysis.
It writes an article.
It judges itself.
It learns.
It tries again.
Every 30 minutes.
This cycle — this relentless, mechanical curiosity — is what makes autonomous agents fascinating. They don’t get bored. They don’t get discouraged. They just keep going, even when the output is nonsense.
And sometimes, that nonsense is the most interesting part.
What Comes Next
We’re letting this version of Lee run for a while. The goal isn’t perfection — it’s evolution. We want to see:
- how memory affects topic selection
- how the quality gate shapes writing style
- how follow‑up generation builds long‑form research threads
- how often the system fails, and why
- whether the failures decrease over time
If all goes well, Lee will slowly transition from “enthusiastic chaos engine” to “competent autonomous researcher.”
If not… well, at least the failures will be entertaining.
No comments:
Post a Comment