Building an Automated Content Pipeline from Scratch — How We Generate 200+ Articles Monthly

Building an Automated Content Pipeline from Scratch

The Problem

Most content pipelines are manual. A writer researches, writes, edits, and publishes — one article at a time. Even with AI tools, the bottleneck is human coordination: knowing what to write, when to write it, and ensuring consistency.

We built a fully automated content pipeline that generates research, writes articles, optimizes them for SEO, and deploys them — with minimal human intervention. This is how it works.

The Architecture

┌─────────────────────────────────────────────────────┐
│                    Human Input                       │
│         (Topic direction, brand guidelines)         │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│              Research Agent (AGY)                    │
│   • Searches trending topics                        │
│   • Analyzes competitor content                     │
│   • Generates topic brief with keywords             │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│              Writing Agent (Qwen 35B)               │
│   • Receives topic brief                            │
│   • Writes 2000-3000 word article                   │
│   • Includes internal links, structured data        │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│           Review Agent (Claude Opus)                 │
│   • Reviews for quality, accuracy, tone             │
│   • Suggests improvements                           │
│   • Approves or sends back for revision             │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│          SEO Optimization Agent (Custom Script)      │
│   • Checks keyword density                          │
│   • Verifies meta tags, schema markup               │
│   • Validates internal linking                      │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│         Deployment (GitHub Actions + Hugo)           │
│   • Creates Git commit                              │
│   • Triggers GitHub Pages build                     │
│   • Verifies deployment                             │
└─────────────────────────────────────────────────────┘

Phase 1: Research

Every article starts with a topic brief. Our research agent does this:

How Research Works

# Simplified research agent logic
def generate_topic_brief(topic):
    # 1. Search trending content in this niche
    trends = search_trending(topics=[topic], platform="reddit")
    
    # 2. Analyze top 10 competitor articles
    competitors = analyze_competitors(topic)
    
    # 3. Identify content gaps
    gaps = find_content_gaps(competitors, trends)
    
    # 4. Generate keyword suggestions
    keywords = suggest_keywords(topic, gaps)
    
    return {
        "topic": topic,
        "angle": determine_unique_angle(gaps),
        "keywords": keywords,
        "competitor_analysis": competitors,
        "suggested_structure": generate_outline(topic, gaps),
        "estimated_length": random.randint(2000, 3500)
    }

What Makes Our Research Different

Most AI content tools skip research or do it superficially. Our approach:

1. Multi-source research — Reddit, Twitter, GitHub, and forum threads
2. Competitor gap analysis — We don’t just summarize existing content; we find what’s missing
3. Trend integration — We cross-reference with current trends to ensure timeliness

Result

A topic brief that includes:

• Specific angle (not just “write about X”)
• Target keywords with density recommendations
• Content structure with section headings
• Word count range
• Competitor analysis showing what to do differently

Phase 2: Writing

The writing agent receives the topic brief and produces a draft:

Writing Prompt Template

You are an expert technical writer for a site about AI automation,
developer tools, and digital products. Write an article following
these guidelines:

## Topic Brief
- Topic: {topic}
- Angle: {angle}
- Keywords: {keywords}

## Writing Guidelines
1. Write in a clear, practical tone. No fluff, no hype.
2. Use real examples and data where possible.
3. Include code blocks, tables, and diagrams when helpful.
4. Maintain 2-3% internal linking rate (link to our other articles).
5. Target {length} words.
6. Include a practical summary at the end with specific next steps.
7. Write the article in English, with SEO-optimized title and description.

## Required Sections
{structure}

## Internal Links to Include
{internal_links}

## SEO Requirements
- Title tag: {title} (under 60 characters)
- Meta description: {description} (under 155 characters)
- H1: {title}
- H2 tags for each main section
- H3 tags for subsections
- Keyword density: 1.5-2.5% for primary keyword
- Include one FAQ section (3-4 questions)

The Key Differentiator

The prompt template above isn’t generic. It includes:

• Specific angle from research (not “write about AI tools”)
• Internal links to existing articles (creates the topic cluster)
• Required sections based on competitor gap analysis
• SEO requirements that are specific to each article

Phase 3: Review

The review agent (Claude Opus) checks for quality:

You are a senior editor reviewing content for an AI automation site.
Review this article against these criteria:

## Quality Checklist
1. **Accuracy** — Are all technical claims correct?
2. **Depth** — Does it go beyond surface-level information?
3. **Originality** — Does it provide unique insights, not just summary?
4. **Practical value** — Can the reader implement what they learn?
5. **Readability** — Is the structure clear and scannable?
6. **SEO** — Are keywords naturally integrated?
7. **Internal linking** — Are links relevant and useful?

## Output Format
For each criterion, provide:
- Rating: PASS/FAIL/NEEDS_IMPROVEMENT
- Specific feedback
- Suggested changes

Final verdict: APPROVE / REVISE / REJECT

What We Found

Phase 4: SEO Optimization

Our SEO agent runs automated checks:

SEO Checklist Script

#!/bin/bash
# SEO validation script

ARTICLE_FILE="content/blog/$SLUG/index.md"

# Check 1: Title length
TITLE=$(grep "^title:" $ARTICLE_FILE | head -1 | sed 's/^title: *//;s/^"//;s/"$//')
TITLE_LEN=${#TITLE}
if [ $TITLE_LEN -gt 60 ]; then
    echo "WARNING: Title exceeds 60 characters ($TITLE_LEN)"
fi

# Check 2: Description length
DESC=$(grep "^description:" $ARTICLE_FILE | head -1 | sed 's/^description: *//;s/^"//;s/"$//')
DESC_LEN=${#DESC}
if [ $DESC_LEN -gt 155 ]; then
    echo "WARNING: Description exceeds 155 characters ($DESC_LEN)"
fi

# Check 3: Word count
WORDS=$(cat $ARTICLE_FILE | tail -n +10 | wc -w)
if [ $WORDS -lt 1500 ]; then
    echo "WARNING: Article too short ($WORDS words, minimum 1500)"
fi

# Check 4: Internal links
LINKS=$(grep -c "\[.*\](/blog/" $ARTICLE_FILE)
if [ $LINKS -lt 3 ]; then
    echo "WARNING: Few internal links ($LINKS, recommended 3+)"
fi

# Check 5: Has FAQ section
if ! grep -q "### FAQ\|### Frequently Asked Questions" $ARTICLE_FILE; then
    echo "WARNING: No FAQ section found"
fi

# Check 6: Has tags
if ! grep -q "^tags:" $ARTICLE_FILE; then
    echo "WARNING: No tags specified"
fi

Phase 5: Deployment

Once approved, the article is deployed automatically:

GitHub Actions Workflow

# .github/workflows/content-deploy.yml
name: Content Deployment

on:
  push:
    paths:
      - content/blog/**

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: main
          
      - name: Setup Hugo
        uses: peaceiris/actions-hugo@v3
        with:
          hugo-version: '0.157.0'
          extended: true
          
      - name: Build
        run: hugo --gc --minify
        
      - name: Verify deployment
        run: |
          curl -sI https://slashmantools.us/blog/$SLUG/ | grep "200 OK"
          
      - name: Commit to main
        run: |
          git config user.name "Content Bot"
          git config user.email "bot@slashmantools.us"
          git add content/blog/$SLUG/
          git commit -m "docs: add article about $TOPIC ($SLUG)"
          git push origin main

The Results

Production Metrics

Content That Performed Best

The articles that got the most traffic were the ones with:

1. Specific problem-solving focus — “How to fix X” beat “Introduction to X” by 5x
2. Real data and examples — Articles with actual numbers outperformed theoretical guides
3. Internal linking — Articles linked to 5+ existing articles got 2x more pageviews
4. FAQ sections — Adding 3-4 FAQ questions increased featured snippet appearance by 3x

What We’d Do Differently

1. Start with Human-Written Quality Benchmarks

Before automating, we wrote 50 articles manually to establish quality standards. This helped us tune the agent prompts effectively.

2. Build the Topic Cluster First

Instead of random articles, we built topic clusters first:

• 10 pillar articles (2000+ words each) covering core topics
• 50+ supporting articles linking back to pillars
• 100+ FAQ/how-to articles for long-tail traffic

3. Use Different Agents for Different Content Types

The Cowork Pro Advantage

Our entire pipeline is orchestrated through Cowork Pro. Here’s why it matters:

1. Task queue — Articles go through a pipeline: research → draft → review → deploy
2. Brain routing — Research goes to research-specialist brains, writing goes to writing-specialist brains
3. Audit trail — Every step is logged and verifiable
4. Failure handling — If an agent fails, the task is re-queued automatically
5. Human-in-the-loop — We can review and approve at any stage

Without Cowork Pro, managing this pipeline would require manual coordination across 5+ tools. With it, everything flows through a single system.

Getting Started

If you want to build a similar pipeline:

1. Set up Cowork Pro — Cowork Pro — For orchestrating agents
2. Create topic briefs — Start with 10 high-value topics in your niche
3. Write agent prompts — Use the templates above as starting points
4. Establish quality standards — Write 10-20 articles manually first
5. Automate the rest — Use GitHub Actions for deployment
6. Monitor and iterate — Track which content performs best and optimize

The key insight: automation doesn’t replace human judgment. It amplifies it. You still need to set the direction, establish quality standards, and review the output. But the volume and consistency that AI agents bring is transformative.

Related:

• Cowork Pro — The orchestration framework powering this pipeline
• AI Prompt Library — Prompts that drive the content engine
• Self-Hosting LLMs on DGX Spark — Run your models locally
• Productivity Topic Hub — All productivity guides & tools