Deployment Guide

Local Development

# Terminal 1: Backend
pnpm dev

# Terminal 2: Frontend (Next.js)
cd ui && pnpm dev

# Or use Caddy for reverse proxy
caddy run --config config/Caddyfile

Access: http://localhost:3000 (Backend API) or http://localhost:9002 (Next.js UI)

---

Production Deployment

Architecture

User → https://YOUR_DOMAIN (HTTPS)
  ↓
┌─────────────────────────────────────────┐
│ Vercel (Frontend)                       │
│   └─ Next.js UI (/ui)                   │
└─────────────────────────────────────────┘
  ↓ API calls
┌─────────────────────────────────────────┐
│ Docker Container (Backend)              │
│   ├─ Caddy (ports 80/443)               │
│   │   ├─ Auto SSL via Let's Encrypt     │
│   │   └─ /api/* → Backend (localhost:3000)
│   └─ Node.js Backend (localhost:3000)   │
└─────────────────────────────────────────┘

Separated frontend (Vercel) and backend (Docker) architecture.

---

Step 1: DNS Configuration

# Add A record
your-domain.com → YOUR_VPS_IP

# Verify
dig your-domain.com +short

Step 2: Firewall

# Ubuntu/Debian
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw reload

Step 3: Deploy

# Create release tag
git tag v1.0.0
git push origin v1.0.0

# CI builds Docker image automatically
# Manually trigger deploy with DEPLOY_VERSION=v1.0.0

On the server:

# Create volume for SSL certificates and cache
mkdir -p /opt/firecrawl-lite/caddy-data
mkdir -p /opt/firecrawl-lite/cache
# Ensure correct permissions for the cache directory (UID 1001 is the node user in container)
chown -R 1001:1001 /opt/firecrawl-lite/cache

# Run container
docker run -d \
  -p 80:80 -p 443:443 \
  --name firecrawl-lite \
  --restart=always \
  -v /opt/firecrawl-lite/caddy-data:/data \
  -v /opt/firecrawl-lite/cache:/cache \
  -e CACHE_PATH="/cache" \
  -e VERSION="v1.0.0" \
  docker.cnb.cool/ai-alchemy-factory/firecrawl-lite:v1.0.0

Step 4: Verify

# Check container
docker ps | grep firecrawl-lite

# Test HTTPS
curl https://your-domain.com/api/health

# Verify SSL
curl -vI https://your-domain.com 2>&1 | grep "issuer"

---

Domain & SSL

Caddy handles SSL automatically:

• Detects domain from Caddyfile
• Requests certificate from Let's Encrypt
• Auto-renews before expiration (90-day cycle)
• Redirects HTTP to HTTPS

Troubleshooting SSL:

# Check DNS
dig your-domain.com +short

# Check ports accessible
nc -zv YOUR_VPS_IP 80
nc -zv YOUR_VPS_IP 443

# Check Caddy logs
docker logs firecrawl-lite | grep -i "acme\|certificate"

---

Environment Variables

Variable	Default	Description
PORT	3000	Server port
NODE_ENV	development	Environment
API_KEY	-	API auth key (optional)
MAX_BROWSERS	5	Concurrent browser instances
MAX_CRAWL_DEPTH	3	Max crawl depth
MAX_CRAWL_PAGES	50	Max pages per crawl
CACHE_PATH	./data/cache	External cache directory path
OPENAI_API_KEY	-	For AI extraction

---

Rollback

# Trigger rollback in CI with ROLLBACK_VERSION=v1.0.0
# Or manually:
docker stop firecrawl-lite
docker rm firecrawl-lite
docker run -d ... docker.cnb.cool/ai-alchemy-factory/firecrawl-lite:v1.0.0

SSL certificates are preserved in /opt/firecrawl-lite/caddy-data.

---

Troubleshooting

Container not starting

docker logs firecrawl-lite
# Check: port conflict, permission denied

Backend not responding

docker exec firecrawl-lite curl -sf http://localhost:3000/api/health

Browser pool exhausted

# Increase pool size
-e MAX_BROWSERS=8

Memory issues

# Limit crawl scope
-e MAX_CRAWL_PAGES=30

---

Cost Estimation

Item	Cost
VPS (2GB RAM)	$10-20/month
Domain	$10-15/year
SSL Certificate	Free (Let's Encrypt)
Total	$10-20/month

Puppeteer requires at least 1GB RAM. 2GB recommended.