Initial Burmddit deployment - AI news aggregator in Burmese

2026-02-19 02:52:58 +00:00
commit dddb86ea94
27 changed files with 5039 additions and 0 deletions
--- a/DEPLOYMENT-GUIDE.md
+++ b/DEPLOYMENT-GUIDE.md
@@ -0,0 +1,474 @@
+# Burmddit - Complete Deployment Guide
+## From Zero to Production in 30 Minutes
+
+**Target URL:** burmddit.vercel.app  
+**Cost:** $0-$60/month (mostly Claude API)
+
+---
+
+## 📋 PRE-REQUISITES
+
+### Accounts Needed (All Free to Start):
+1. ✅ **GitHub** - github.com (code hosting)
+2. ✅ **Vercel** - vercel.com (frontend hosting)
+3. ✅ **Railway** - railway.app (backend + database)
+4. ✅ **Anthropic** - console.anthropic.com (Claude API for translation)
+
+**Time to create accounts:** ~10 minutes
+
+---
+
+## 🚀 DEPLOYMENT STEPS
+
+### STEP 1: Push Code to GitHub (5 mins)
+
+```bash
+# On your local machine or server:
+cd /home/ubuntu/.openclaw/workspace/burmddit
+
+# Initialize git
+git init
+git add .
+git commit -m "Initial Burmddit deployment"
+
+# Create repository on GitHub (via website):
+# 1. Go to github.com/new
+# 2. Name: burmddit
+# 3. Public or Private (your choice)
+# 4. Create repository
+
+# Push to GitHub
+git remote add origin https://github.com/YOUR_USERNAME/burmddit.git
+git branch -M main
+git push -u origin main
+```
+
+✅ **Done!** Your code is now on GitHub
+
+---
+
+### STEP 2: Deploy Backend to Railway (10 mins)
+
+#### 2.1: Create Railway Project
+
+1. Go to **railway.app**
+2. Click "New Project"
+3. Select "Deploy from GitHub repo"
+4. Choose your `burmddit` repository
+5. Railway will auto-detect it as a Python project
+
+#### 2.2: Add PostgreSQL Database
+
+1. In your Railway project, click "+ New"
+2. Select "Database" → "PostgreSQL"
+3. Railway creates database instantly
+4. Copy the `DATABASE_URL` (Click database → Connect → Copy connection string)
+
+#### 2.3: Set Environment Variables
+
+In Railway project settings → Variables, add:
+
+```env
+DATABASE_URL=<paste from step 2.2>
+ANTHROPIC_API_KEY=<your Claude API key from console.anthropic.com>
+ADMIN_PASSWORD=<choose a secure password>
+PYTHONPATH=/app/backend
+```
+
+#### 2.4: Configure Build
+
+Railway → Settings → Build:
+- **Root Directory:** `backend`
+- **Build Command:** `pip install -r requirements.txt`
+- **Start Command:** `python run_pipeline.py`
+
+#### 2.5: Initialize Database
+
+In Railway console (click database service):
+```bash
+python init_db.py init
+```
+
+✅ **Done!** Backend is live on Railway
+
+---
+
+### STEP 3: Deploy Frontend to Vercel (5 mins)
+
+#### 3.1: Connect GitHub to Vercel
+
+1. Go to **vercel.com/new**
+2. Click "Import Git Repository"
+3. Select your `burmddit` repo
+4. Vercel auto-detects Next.js
+
+#### 3.2: Configure Settings
+
+**Framework Preset:** Next.js  
+**Root Directory:** `frontend`  
+**Build Command:** (default `next build`)  
+**Output Directory:** (default `.next`)
+
+#### 3.3: Set Environment Variables
+
+In Vercel project settings → Environment Variables:
+
+```env
+DATABASE_URL=<same as Railway PostgreSQL URL>
+NEXT_PUBLIC_SITE_URL=https://burmddit.vercel.app
+```
+
+#### 3.4: Deploy
+
+Click "Deploy"  
+Wait 2-3 minutes...
+
+✅ **Done!** Frontend is live at `burmddit.vercel.app`!
+
+---
+
+### STEP 4: Set Up Automation (5 mins)
+
+#### Option A: GitHub Actions (Recommended)
+
+Create file: `.github/workflows/daily-publish.yml`
+
+```yaml
+name: Daily Content Pipeline
+
+on:
+  schedule:
+    # Run at 6 AM UTC daily
+    - cron: '0 6 * * *'
+  workflow_dispatch: # Allow manual trigger
+
+jobs:
+  run-pipeline:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.11'
+      
+      - name: Install dependencies
+        run: |
+          cd backend
+          pip install -r requirements.txt
+      
+      - name: Run pipeline
+        env:
+          DATABASE_URL: ${{ secrets.DATABASE_URL }}
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+        run: |
+          cd backend
+          python run_pipeline.py
+```
+
+**Add secrets to GitHub:**
+1. Repo → Settings → Secrets and variables → Actions
+2. Add: `DATABASE_URL` and `ANTHROPIC_API_KEY`
+
+#### Option B: Railway Cron (Simpler but less flexible)
+
+In Railway, use built-in cron:
+1. Project settings → Deployments
+2. Add cron trigger: `0 6 * * *`
+3. Command: `python backend/run_pipeline.py`
+
+✅ **Done!** Auto-publishes 10 articles daily at 6 AM UTC!
+
+---
+
+## 🧪 TESTING
+
+### Test the Pipeline Manually:
+
+```bash
+# SSH into Railway or run locally with env vars:
+cd backend
+
+# Test scraper
+python scraper.py
+
+# Test compiler
+python compiler.py
+
+# Test translator
+python translator.py
+
+# Test full pipeline
+python run_pipeline.py
+```
+
+### Check Database:
+
+```bash
+python init_db.py stats
+```
+
+### Test Website:
+
+1. Visit **burmddit.vercel.app**
+2. Should see homepage with categories
+3. Once pipeline runs, articles will appear
+
+---
+
+## 💰 COST BREAKDOWN
+
+### Monthly Costs:
+
+**Free tier (Month 1-3):**
+- Vercel: FREE (Hobby plan)
+- Railway: FREE ($5 credit/month) or $5/month
+- GitHub Actions: FREE (2,000 mins/month)
+- **Total: $0-$5/month**
+
+**With Claude API (Month 1+):**
+- Claude API: ~$30-60/month
+  - 10 articles/day × 30 days = 300 articles
+  - ~1,500 words per article = 2,000 tokens
+  - 300 × 2,000 = 600k tokens/month
+  - Claude pricing: ~$0.015/1k tokens input, $0.075/1k tokens output
+  - Estimated: $30-60/month
+- **Total: $35-65/month**
+
+**Optimization tips:**
+- Use Claude 3 Haiku for translation (cheaper, still good quality)
+- Batch translations to reduce API calls
+- Cache common translations
+
+---
+
+## 📊 MONITORING
+
+### Check Pipeline Status:
+
+**Railway Dashboard:**
+- View logs for each pipeline run
+- Check database size
+- Monitor CPU/memory usage
+
+**Vercel Dashboard:**
+- Page views
+- Load times
+- Error rates
+
+**Database Stats:**
+```bash
+python init_db.py stats
+```
+
+---
+
+## 🐛 TROUBLESHOOTING
+
+### Pipeline Not Running
+
+**Check logs:**
+```bash
+# Railway → Deployments → View logs
+# Or locally:
+tail -f burmddit_pipeline.log
+```
+
+**Common issues:**
+- API key not set → Check environment variables
+- Database connection failed → Verify DATABASE_URL
+- Scraping blocked → Check rate limits, use VPN if needed
+
+### No Articles Showing
+
+**Verify pipeline ran:**
+```bash
+python init_db.py stats
+```
+
+**Check articles table:**
+```sql
+SELECT COUNT(*) FROM articles WHERE status = 'published';
+```
+
+**Manual trigger:**
+```bash
+python backend/run_pipeline.py
+```
+
+### Translation Errors
+
+**Check API key:**
+```bash
+echo $ANTHROPIC_API_KEY
+```
+
+**Test translation:**
+```bash
+python backend/translator.py
+```
+
+**Rate limit hit:**
+- Anthropic free tier: 50 requests/min
+- Paid tier: 1,000 requests/min
+- Add delays if needed
+
+---
+
+## 🔧 CUSTOMIZATION
+
+### Change Number of Articles:
+
+Edit `backend/config.py`:
+```python
+PIPELINE = {
+    'articles_per_day': 20,  # Change from 10 to 20
+    ...
+}
+```
+
+### Add New Content Sources:
+
+Edit `backend/config.py`:
+```python
+SOURCES = {
+    'your_source': {
+        'enabled': True,
+        'url': 'https://example.com/feed/',
+        ...
+    }
+}
+```
+
+Update `backend/scraper.py` to handle new source format.
+
+### Change Translation Quality:
+
+Use Claude 3 Opus for best quality (more expensive):
+```python
+TRANSLATION = {
+    'model': 'claude-3-opus-20240229',
+    ...
+}
+```
+
+Or Claude 3 Haiku for lower cost (still good):
+```python
+TRANSLATION = {
+    'model': 'claude-3-haiku-20240307',
+    ...
+}
+```
+
+---
+
+## 🎨 FRONTEND CUSTOMIZATION
+
+### Change Colors:
+
+Edit `frontend/tailwind.config.ts`:
+```typescript
+colors: {
+  primary: {
+    500: '#YOUR_COLOR',
+    // ... other shades
+  }
+}
+```
+
+### Add Logo:
+
+Replace text logo in `frontend/components/Header.tsx`:
+```tsx
+<Image src="/logo.png" width={40} height={40} alt="Burmddit" />
+```
+
+Add `logo.png` to `frontend/public/`
+
+### Change Fonts:
+
+Update `frontend/app/layout.tsx` with Google Fonts link
+
+---
+
+## 📈 SCALING
+
+### When Traffic Grows:
+
+**Vercel (Frontend):**
+- Free tier: Unlimited bandwidth
+- Pro tier ($20/mo): Analytics, more team members
+
+**Railway (Backend + DB):**
+- Free tier: $5 credit (good for 1-2 months)
+- Hobby tier: $5/mo (500 hours)
+- Pro tier: $20/mo (unlimited)
+
+**Database:**
+- Railway PostgreSQL: 100 MB free → 8 GB on paid
+- For larger: Migrate to Supabase or AWS RDS
+
+**Claude API:**
+- Pay-as-you-go scales automatically
+- Monitor costs in Anthropic console
+
+---
+
+## ✅ POST-DEPLOYMENT CHECKLIST
+
+After deployment, verify:
+
+- [ ] Frontend loads at burmddit.vercel.app
+- [ ] Database tables created (run `init_db.py stats`)
+- [ ] Pipeline runs successfully (trigger manually first)
+- [ ] Articles appear on homepage
+- [ ] All 4 categories work
+- [ ] Mobile responsive (test on phone)
+- [ ] Search works (if implemented)
+- [ ] Burmese fonts display correctly
+- [ ] GitHub Actions/Railway cron scheduled
+- [ ] Environment variables secure (not in code)
+
+---
+
+## 🎉 SUCCESS!
+
+**You now have:**
+✅ Fully automated AI content platform  
+✅ 10 articles published daily  
+✅ Professional Burmese website  
+✅ Zero manual work needed  
+✅ Scalable infrastructure  
+
+**Next steps:**
+1. Monitor first week's content quality
+2. Adjust scraping sources if needed
+3. Promote on social media
+4. Apply for Google AdSense (after 3 months)
+5. Build email list
+6. Scale to 20 articles/day if demand grows
+
+---
+
+## 📞 SUPPORT
+
+**Issues?** Check:
+1. Railway logs
+2. Vercel deployment logs
+3. GitHub Actions run history
+4. Database stats (`init_db.py stats`)
+
+**Still stuck?** Review code comments in:
+- `backend/run_pipeline.py` (main orchestrator)
+- `backend/scraper.py` (if scraping issues)
+- `backend/translator.py` (if translation issues)
+
+---
+
+**Built by Bob (OpenClaw AI) for Zeya Phyo**  
+**Deploy time: ~30 minutes**  
+**Maintenance: ~5 minutes/week**  
+**Passive income potential: $2,000-5,000/month** 🚀
+
+**Let's make AI accessible to all Burmese speakers!** 🇲🇲