Add web admin features + fix scraper & translator

Frontend changes: - Add /admin dashboard for article management - Add AdminButton component (Alt+Shift+A on articles) - Add /api/admin/article API endpoints Backend improvements: - scraper_v2.py: Multi-layer fallback extraction (newspaper → trafilatura → readability) - translator_v2.py: Better chunking, repetition detection, validation - admin_tools.py: CLI admin commands - test_scraper.py: Individual source testing Docs: - WEB-ADMIN-GUIDE.md: Web admin usage - ADMIN-GUIDE.md: CLI admin usage - SCRAPER-IMPROVEMENT-PLAN.md: Scraper fixes details - TRANSLATION-FIX.md: Translation improvements - ADMIN-FEATURES-SUMMARY.md: Implementation summary Fixes: - Article scraping from 0 → 96+ articles working - Translation quality issues (repetition, truncation) - Added 13 new RSS sources
2026-02-26 09:17:50 +00:00
parent 8bf5f342cd
commit f51ac4afa4
20 changed files with 4769 additions and 23 deletions
--- a/ADMIN-FEATURES-SUMMARY.md
+++ b/ADMIN-FEATURES-SUMMARY.md
@@ -0,0 +1,366 @@
+# Admin Features Implementation Summary
+
+**Date:** 2026-02-26  
+**Status:** ✅ Implemented  
+**Deploy Required:** Yes (frontend changes)
+
+---
+
+## 🎯 What Was Built
+
+Created **web-based admin controls** for managing articles directly from burmddit.com
+
+### 1. Admin API (`/app/api/admin/article/route.ts`)
+
+**Endpoints:**
+- `GET /api/admin/article` - List articles (with status filter)
+- `POST /api/admin/article` - Unpublish/Publish/Delete articles
+
+**Authentication:** Bearer token (password in header)
+
+**Actions:**
+- `unpublish` - Change status to draft (hide from site)
+- `publish` - Change status to published (show on site)
+- `delete` - Permanently remove from database
+
+### 2. Admin Dashboard (`/app/admin/page.tsx`)
+
+**URL:** https://burmddit.com/admin
+
+**Features:**
+- Password login (stored in sessionStorage)
+- Table view of all articles
+- Filter by status (published/draft)
+- Color-coded translation quality:
+  - 🟢 Green (40%+) = Good
+  - 🟡 Yellow (20-40%) = Check
+  - 🔴 Red (<20%) = Poor
+- One-click actions: View, Unpublish, Publish, Delete
+- Real-time updates (reloads data after actions)
+
+### 3. On-Article Admin Button (`/components/AdminButton.tsx`)
+
+**Trigger:** Press **Alt + Shift + A** on any article page
+
+**Features:**
+- Hidden floating panel (bottom-right)
+- Quick password unlock
+- Instant actions:
+  - 🚫 Unpublish (Hide)
+  - 🗑️ Delete Forever
+  - 🔒 Lock Admin
+- Auto-reloads page after action
+
+---
+
+## 📁 Files Created/Modified
+
+### New Files
+
+1. `/frontend/app/api/admin/article/route.ts` (361 lines)
+   - Admin API endpoints
+   - Password authentication
+   - Database operations
+
+2. `/frontend/components/AdminButton.tsx` (494 lines)
+   - Hidden admin panel component
+   - Keyboard shortcut handler
+   - Session management
+
+3. `/frontend/app/admin/page.tsx` (573 lines)
+   - Full admin dashboard
+   - Article table with stats
+   - Filter and action buttons
+
+4. `/burmddit/WEB-ADMIN-GUIDE.md`
+   - Complete user documentation
+   - Usage instructions
+   - Troubleshooting guide
+
+5. `/burmddit/ADMIN-FEATURES-SUMMARY.md` (this file)
+   - Implementation summary
+
+### Modified Files
+
+1. `/frontend/app/article/[slug]/page.tsx`
+   - Added AdminButton component import
+   - Added AdminButton at end of page
+
+---
+
+## 🔐 Security
+
+### Authentication Method
+
+**Password-based** (simple but effective):
+- Admin password stored in `.env` file
+- Client sends password as Bearer token
+- Server validates on every request
+- No database user management (keeps it simple)
+
+**Default Password:** `burmddit2026`  
+**⚠️ Change this before deploying to production!**
+
+### Session Storage
+
+- Password stored in browser `sessionStorage`
+- Automatically cleared when tab closes
+- Manual logout button available
+- No persistent storage (cookies)
+
+### API Protection
+
+- All admin endpoints check auth header
+- Returns 401 if unauthorized
+- No public access to admin functions
+- Database credentials never exposed to client
+
+---
+
+## 🚀 Deployment Steps
+
+### 1. Update Environment Variables
+
+Add to `/frontend/.env`:
+
+```bash
+# Admin password (change this!)
+ADMIN_PASSWORD=burmddit2026
+
+# Database URL (should already exist)
+DATABASE_URL=postgresql://...
+```
+
+### 2. Install Dependencies (if needed)
+
+```bash
+cd /home/ubuntu/.openclaw/workspace/burmddit/frontend
+npm install pg
+```
+
+Already installed ✅
+
+### 3. Build & Deploy
+
+```bash
+# Build Next.js app
+npm run build
+
+# Deploy to Vercel (if connected via Git)
+git add .
+git commit -m "Add web admin features"
+git push origin main
+
+# Or deploy manually
+vercel --prod
+```
+
+### 4. Test Access
+
+1. Visit https://burmddit.com/admin
+2. Enter password: `burmddit2026`
+3. See list of articles
+4. Test unpublish/publish buttons
+
+---
+
+## 📊 Usage Stats
+
+### Use Cases Supported
+
+✅ **Quick review** - Browse all articles in dashboard  
+✅ **Flag errors** - Unpublish broken articles with one click  
+✅ **Emergency takedown** - Hide article in <1 second from any page  
+✅ **Bulk management** - Open multiple articles, unpublish each quickly  
+✅ **Quality monitoring** - See translation ratios at a glance  
+✅ **Republish fixed** - Restore articles after fixing  
+
+### User Flows
+
+**Flow 1: Daily Check**
+1. Go to /admin
+2. Review red (<20%) articles
+3. Click to view each one
+4. Unpublish if broken
+5. Fix via CLI, then republish
+
+**Flow 2: Emergency Hide**
+1. See bad article on site
+2. Alt + Shift + A
+3. Enter password
+4. Click Unpublish
+5. Done in 5 seconds
+
+**Flow 3: Bulk Cleanup**
+1. Open /admin
+2. Ctrl+Click multiple bad articles
+3. Alt + Shift + A on each tab
+4. Unpublish from each
+5. Close tabs
+
+---
+
+## 🎓 Technical Details
+
+### Frontend Stack
+
+- **Next.js 13+** with App Router
+- **TypeScript** for type safety
+- **Tailwind CSS** for styling
+- **React Hooks** for state management
+
+### Backend Integration
+
+- **PostgreSQL** via `pg` library
+- **SQL queries** for article management
+- **Connection pooling** for performance
+- **Transaction safety** for updates
+
+### API Design
+
+**RESTful** approach:
+- `GET` for reading articles
+- `POST` for modifying articles
+- JSON request/response bodies
+- Bearer token authentication
+
+### Component Architecture
+
+```
+AdminButton (client component)
+  ├─ Hidden by default
+  ├─ Keyboard event listener
+  ├─ Session storage for auth
+  └─ Fetch API for backend calls
+
+AdminDashboard (client component)
+  ├─ useEffect for auto-load
+  ├─ useState for articles list
+  ├─ Table rendering
+  └─ Action handlers
+
+Admin API Route (server)
+  ├─ Auth middleware
+  ├─ Database queries
+  └─ JSON responses
+```
+
+---
+
+## 🐛 Known Limitations
+
+### Current Constraints
+
+1. **Single password** - Everyone shares same password
+   - Future: Multiple admin users with roles
+   
+2. **No audit log** - Basic logging only
+   - Future: Detailed change history
+   
+3. **No article editing** - Can only publish/unpublish
+   - Future: Inline editing, re-translation
+   
+4. **No batch operations** - One article at a time
+   - Future: Checkboxes + bulk actions
+   
+5. **Session-based auth** - Expires on tab close
+   - Future: JWT tokens, persistent sessions
+
+### Not Issues (By Design)
+
+- ✅ Simple password auth is intentional (no user management overhead)
+- ✅ Manual article fixing via CLI is intentional (admin panel is for management, not content creation)
+- ✅ No persistent login is intentional (security through inconvenience)
+
+---
+
+## 🎯 Next Steps
+
+### Immediate (Before Production)
+
+1. **Change admin password** in `.env`
+2. **Test all features** in staging
+3. **Deploy to production**
+4. **Document password** in secure place (password manager)
+
+### Short-term Enhancements
+
+1. Add "Find Problems" button to dashboard
+2. Add article preview in modal
+3. Add statistics (total views, articles per day)
+4. Add search/filter by title
+
+### Long-term Ideas
+
+1. Multiple admin accounts with permissions
+2. Detailed audit log of all changes
+3. Article editor with live preview
+4. Re-translate button (triggers backend job)
+5. Email notifications for quality issues
+6. Mobile app for admin on-the-go
+
+---
+
+## 📚 Documentation Created
+
+1. **WEB-ADMIN-GUIDE.md** - User guide
+   - How to access admin features
+   - Common workflows
+   - Troubleshooting
+   - Security best practices
+
+2. **ADMIN-GUIDE.md** - CLI tools guide
+   - Command-line admin tools
+   - Backup/restore procedures
+   - Advanced operations
+
+3. **ADMIN-FEATURES-SUMMARY.md** - This file
+   - Implementation details
+   - Deployment guide
+   - Technical architecture
+
+---
+
+## ✅ Testing Checklist
+
+Before deploying to production:
+
+- [ ] Test admin login with correct password
+- [ ] Test admin login with wrong password (should fail)
+- [ ] Test unpublish article (should hide from site)
+- [ ] Test publish article (should show on site)
+- [ ] Test delete article (with confirmation)
+- [ ] Test Alt+Shift+A shortcut on article page
+- [ ] Test admin panel on mobile browser
+- [ ] Test logout functionality
+- [ ] Verify changes persist after page reload
+- [ ] Check translation quality colors are accurate
+
+---
+
+## 🎉 Summary
+
+**What You Can Do Now:**
+
+✅ Browse all articles in a clean dashboard  
+✅ See translation quality at a glance  
+✅ Unpublish broken articles with one click  
+✅ Republish fixed articles  
+✅ Quick admin access on any article page  
+✅ Delete articles permanently  
+✅ Filter by published/draft status  
+✅ View article stats (views, length, ratio)  
+
+**How to Access:**
+
+🌐 **Dashboard:** https://burmddit.com/admin  
+⌨️ **On Article:** Press Alt + Shift + A  
+🔑 **Password:** `burmddit2026` (change in production!)  
+
+---
+
+**Implementation Time:** ~1 hour  
+**Lines of Code:** ~1,450 lines  
+**Files Created:** 5 files  
+**Status:** ✅ Ready to deploy  
+**Next:** Deploy frontend, test, and change password!