Add web admin features + fix scraper & translator

Frontend changes:
- Add /admin dashboard for article management
- Add AdminButton component (Alt+Shift+A on articles)
- Add /api/admin/article API endpoints

Backend improvements:
- scraper_v2.py: Multi-layer fallback extraction (newspaper → trafilatura → readability)
- translator_v2.py: Better chunking, repetition detection, validation
- admin_tools.py: CLI admin commands
- test_scraper.py: Individual source testing

Docs:
- WEB-ADMIN-GUIDE.md: Web admin usage
- ADMIN-GUIDE.md: CLI admin usage
- SCRAPER-IMPROVEMENT-PLAN.md: Scraper fixes details
- TRANSLATION-FIX.md: Translation improvements
- ADMIN-FEATURES-SUMMARY.md: Implementation summary

Fixes:
- Article scraping from 0 → 96+ articles working
- Translation quality issues (repetition, truncation)
- Added 13 new RSS sources
This commit is contained in:
Zeya Phyo
2026-02-26 09:17:50 +00:00
parent 8bf5f342cd
commit f51ac4afa4
20 changed files with 4769 additions and 23 deletions

334
WEB-ADMIN-GUIDE.md Normal file
View File

@@ -0,0 +1,334 @@
# Burmddit Web Admin Guide
**Created:** 2026-02-26
**Admin Dashboard:** https://burmddit.com/admin
**Password:** Set in `.env` as `ADMIN_PASSWORD`
---
## 🎯 Quick Access
### Method 1: Admin Dashboard (Recommended)
1. Go to **https://burmddit.com/admin**
2. Enter admin password (default: `burmddit2026`)
3. View all articles in a table
4. Click buttons to Unpublish/Publish/Delete
### Method 2: On-Article Admin Panel (Hidden)
1. **View any article** on burmddit.com
2. Press **Alt + Shift + A** (keyboard shortcut)
3. Admin panel appears in bottom-right corner
4. Enter password once, then use buttons to:
- 🚫 **Unpublish** - Hide article from site
- 🗑️ **Delete** - Remove permanently
---
## 📊 Admin Dashboard Features
### Main Table View
| Column | Description |
|--------|-------------|
| **ID** | Article number |
| **Title** | Article title in Burmese (clickable to view) |
| **Status** | published (green) or draft (gray) |
| **Translation** | Quality % (EN → Burmese length ratio) |
| **Views** | Page view count |
| **Actions** | View, Unpublish/Publish, Delete buttons |
### Translation Quality Colors
- 🟢 **Green (40%+)** - Good translation
- 🟡 **Yellow (20-40%)** - Check manually, might be okay
- 🔴 **Red (<20%)** - Poor/incomplete translation
### Filters
- **Published** - Show only live articles
- **Draft** - Show hidden/unpublished articles
---
## 🔧 Common Actions
### Flag & Unpublish Bad Article
**From Dashboard:**
1. Go to https://burmddit.com/admin
2. Log in with password
3. Find article (look for red <20% translation)
4. Click **Unpublish** button
5. Article is hidden immediately
**From Article Page:**
1. View article on site
2. Press **Alt + Shift + A**
3. Enter password
4. Click **🚫 Unpublish (Hide)**
5. Page reloads, article is hidden
### Republish Fixed Article
1. Go to admin dashboard
2. Change filter to **Draft**
3. Find the article you fixed
4. Click **Publish** button
5. Article is live again
### Delete Article Permanently
⚠️ **Warning:** This cannot be undone!
1. Go to admin dashboard
2. Find the article
3. Click **Delete** button
4. Confirm deletion
5. Article is permanently removed
---
## 🔐 Security
### Password Setup
Set admin password in frontend `.env` file:
```bash
# /home/ubuntu/.openclaw/workspace/burmddit/frontend/.env
ADMIN_PASSWORD=your_secure_password_here
```
**Default password:** `burmddit2026`
**Change it immediately for production!**
### Session Management
- Password stored in browser `sessionStorage` (temporary)
- Expires when browser tab closes
- Click **Logout** to clear manually
- No cookies or persistent storage
### Access Control
- Only works with correct password
- No public API endpoints without auth
- Failed auth returns 401 Unauthorized
- Password checked on every request
---
## 📱 Mobile Support
Admin panel works on mobile too:
- **Dashboard:** Responsive table (scroll horizontally)
- **On-article panel:** Touch-friendly buttons
- **Alt+Shift+A shortcut:** May not work on mobile keyboards
- Alternative: Use dashboard at /admin
---
## 🎨 UI Details
### Admin Dashboard
- Clean table layout
- Color-coded status badges
- One-click actions
- Real-time filtering
- View counts and stats
### On-Article Panel
- Bottom-right floating panel
- Hidden by default (Alt+Shift+A to show)
- Red background (admin warning color)
- Quick unlock with password
- Instant actions with reload
---
## 🔥 Workflows
### Daily Quality Check
1. Go to https://burmddit.com/admin
2. Sort by Translation % (look for red ones)
3. Click article titles to review
4. Unpublish any with broken translations
5. Fix them using CLI tools (see ADMIN-GUIDE.md)
6. Republish when fixed
### Emergency Takedown
**Scenario:** Found article with errors, need to hide immediately
1. On article page, press **Alt + Shift + A**
2. Enter password (if not already)
3. Click **🚫 Unpublish (Hide)**
4. Article disappears in <1 second
### Bulk Management
1. Go to admin dashboard
2. Review list of published articles
3. Open each problem article in new tab (Ctrl+Click)
4. Use Alt+Shift+A on each tab
5. Unpublish quickly from each
---
## 🐛 Troubleshooting
### "Unauthorized" Error
- Check password is correct
- Check ADMIN_PASSWORD in .env matches
- Try logging out and back in
- Clear browser cache
### Admin panel won't show (Alt+Shift+A)
- Make sure you're on an article page
- Try different keyboard (some laptops need Fn key)
- Use admin dashboard instead: /admin
- Check browser console for errors
### Changes not appearing on site
- Changes are instant (no cache)
- Try hard refresh: Ctrl+Shift+R
- Check article status in dashboard
- Verify database updated (use CLI tools)
### Can't access /admin page
- Check Next.js is running
- Check no firewall blocking
- Try incognito/private browsing
- Check browser console for errors
---
## 📊 Statistics
### What Gets Tracked
- **View count** - Increments on each page view
- **Status** - published or draft
- **Translation ratio** - Burmese/English length %
- **Last updated** - Timestamp of last change
### What Gets Logged
Backend logs all admin actions:
- Unpublish: Article ID + reason
- Publish: Article ID
- Delete: Article ID + title
Check logs at:
```bash
# Backend logs (if deployed)
railway logs
# Or check database updated_at timestamp
```
---
## 🎓 Tips & Best Practices
### Keyboard Shortcuts
- **Alt + Shift + A** - Toggle admin panel (on article pages)
- **Escape** - Close admin panel
- **Enter** - Submit password (in login box)
### Translation Quality Guidelines
When reviewing articles:
- **40%+** ✅ - Approve, publish
- **30-40%** ⚠️ - Read manually, may be technical content (okay)
- **20-30%** ⚠️ - Check for missing chunks
- **<20%** ❌ - Unpublish, translation broken
### Workflow Integration
Add to your daily routine:
1. **Morning:** Check dashboard for new articles
2. **Review:** Look for red (<20%) translations
3. **Fix:** Unpublish bad ones immediately
4. **Re-translate:** Use CLI fix script
5. **Republish:** When translation is good
---
## 🚀 Deployment
### Environment Variables
Required in `.env`:
```bash
# Database (already set)
DATABASE_URL=postgresql://...
# Admin password (NEW - add this!)
ADMIN_PASSWORD=burmddit2026
```
### Build & Deploy
```bash
cd /home/ubuntu/.openclaw/workspace/burmddit/frontend
# Install dependencies (if pg not installed)
npm install pg
# Build
npm run build
# Deploy
vercel --prod
```
Or deploy automatically via Git push if connected to Vercel.
---
## 📞 Support
### Common Questions
**Q: Can multiple admins use this?**
A: Yes, anyone with the password. Consider unique passwords per admin in future.
**Q: Is there an audit log?**
A: Currently basic logging. Can add detailed audit trail if needed.
**Q: Can I customize the admin UI?**
A: Yes! Edit `/frontend/app/admin/page.tsx` and `/frontend/components/AdminButton.tsx`
**Q: Mobile app admin?**
A: Works in mobile browser. For native app, would need API + mobile UI.
---
## 🔮 Future Enhancements
Possible improvements:
- [ ] Multiple admin users with different permissions
- [ ] Detailed audit log of all changes
- [ ] Batch operations (unpublish multiple at once)
- [ ] Article editing from admin panel
- [ ] Re-translate button directly in admin
- [ ] Email notifications for quality issues
- [ ] Analytics dashboard (views over time)
---
**Created:** 2026-02-26 09:15 UTC
**Last Updated:** 2026-02-26 09:15 UTC
**Status:** ✅ Ready to use
Access at: https://burmddit.com/admin