forked from minzeyaphyo/burmddit
Add web admin features + fix scraper & translator
Frontend changes: - Add /admin dashboard for article management - Add AdminButton component (Alt+Shift+A on articles) - Add /api/admin/article API endpoints Backend improvements: - scraper_v2.py: Multi-layer fallback extraction (newspaper → trafilatura → readability) - translator_v2.py: Better chunking, repetition detection, validation - admin_tools.py: CLI admin commands - test_scraper.py: Individual source testing Docs: - WEB-ADMIN-GUIDE.md: Web admin usage - ADMIN-GUIDE.md: CLI admin usage - SCRAPER-IMPROVEMENT-PLAN.md: Scraper fixes details - TRANSLATION-FIX.md: Translation improvements - ADMIN-FEATURES-SUMMARY.md: Implementation summary Fixes: - Article scraping from 0 → 96+ articles working - Translation quality issues (repetition, truncation) - Added 13 new RSS sources
This commit is contained in:
Zeya Phyo
336
ADMIN-GUIDE.md
Normal file
336
ADMIN-GUIDE.md
Normal file
@@ -0,0 +1,336 @@
|
||||
# Burmddit Admin Tools Guide
|
||||
|
||||
**Location:** `/home/ubuntu/.openclaw/workspace/burmddit/backend/admin_tools.py`
|
||||
|
||||
Admin CLI tool for managing articles on burmddit.com
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
```bash
|
||||
cd /home/ubuntu/.openclaw/workspace/burmddit/backend
|
||||
python3 admin_tools.py --help
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Available Commands
|
||||
|
||||
### 1. List Articles
|
||||
|
||||
View all articles with status and stats:
|
||||
|
||||
```bash
|
||||
# List all articles (last 20)
|
||||
python3 admin_tools.py list
|
||||
|
||||
# List only published articles
|
||||
python3 admin_tools.py list --status published
|
||||
|
||||
# List only drafts
|
||||
python3 admin_tools.py list --status draft
|
||||
|
||||
# Show more results
|
||||
python3 admin_tools.py list --limit 50
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```
|
||||
ID Title Status Views Ratio
|
||||
----------------------------------------------------------------------------------------------------
|
||||
87 Co-founders behind Reface and Prisma... published 0 52.3%
|
||||
86 OpenAI, Reliance partner to add AI search... published 0 48.7%
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Find Problem Articles
|
||||
|
||||
Automatically detect articles with issues:
|
||||
|
||||
```bash
|
||||
python3 admin_tools.py find-problems
|
||||
```
|
||||
|
||||
**Detects:**
|
||||
- ❌ Translation too short (< 30% of original)
|
||||
- ❌ Missing Burmese translation
|
||||
- ❌ Very short articles (< 500 chars)
|
||||
|
||||
**Example output:**
|
||||
```
|
||||
Found 3 potential issues:
|
||||
----------------------------------------------------------------------------------------------------
|
||||
ID 50: You ar a top engineer wiht expertise on cutting ed
|
||||
Issue: Translation too short
|
||||
Details: EN: 51244 chars, MM: 3400 chars (6.6%)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Unpublish Article
|
||||
|
||||
Remove article from live site (changes status to "draft"):
|
||||
|
||||
```bash
|
||||
# Unpublish article ID 50
|
||||
python3 admin_tools.py unpublish 50
|
||||
|
||||
# With custom reason
|
||||
python3 admin_tools.py unpublish 50 --reason "Translation incomplete"
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Changes `status` from `published` to `draft`
|
||||
- Article disappears from website immediately
|
||||
- Data preserved in database
|
||||
- Can be republished later
|
||||
|
||||
---
|
||||
|
||||
### 4. Republish Article
|
||||
|
||||
Restore article to live site:
|
||||
|
||||
```bash
|
||||
# Republish article ID 50
|
||||
python3 admin_tools.py republish 50
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Changes `status` from `draft` to `published`
|
||||
- Article appears on website immediately
|
||||
|
||||
---
|
||||
|
||||
### 5. View Article Details
|
||||
|
||||
Get detailed information about an article:
|
||||
|
||||
```bash
|
||||
# Show full details for article 50
|
||||
python3 admin_tools.py details 50
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```
|
||||
================================================================================
|
||||
Article 50 Details
|
||||
================================================================================
|
||||
Title (EN): You ar a top engineer wiht expertise on cutting ed...
|
||||
Title (MM): ကျွန်တော်က AI (အထက်တန်းကွန်ပျူတာဦးနှောက်) နဲ့...
|
||||
Slug: k-n-tteaa-k-ai-athk-ttn...
|
||||
Status: published
|
||||
Author: Compiled from 3 sources
|
||||
Published: 2026-02-19 14:48:52.238217
|
||||
Views: 0
|
||||
|
||||
Content length: 51244 chars
|
||||
Burmese length: 3400 chars
|
||||
Translation ratio: 6.6%
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Delete Article (Permanent)
|
||||
|
||||
**⚠️ WARNING:** This permanently deletes the article from the database!
|
||||
|
||||
```bash
|
||||
# Delete article (requires --confirm flag)
|
||||
python3 admin_tools.py delete 50 --confirm
|
||||
```
|
||||
|
||||
**Use with caution!** Data cannot be recovered after deletion.
|
||||
|
||||
---
|
||||
|
||||
## 🔥 Common Workflows
|
||||
|
||||
### Fix Broken Translation Article
|
||||
|
||||
1. **Find problem articles:**
|
||||
```bash
|
||||
python3 admin_tools.py find-problems
|
||||
```
|
||||
|
||||
2. **Check article details:**
|
||||
```bash
|
||||
python3 admin_tools.py details 50
|
||||
```
|
||||
|
||||
3. **Unpublish if broken:**
|
||||
```bash
|
||||
python3 admin_tools.py unpublish 50 --reason "Incomplete translation"
|
||||
```
|
||||
|
||||
4. **Fix the article** (re-translate, edit, etc.)
|
||||
|
||||
5. **Republish:**
|
||||
```bash
|
||||
python3 admin_tools.py republish 50
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Quick Daily Check
|
||||
|
||||
```bash
|
||||
# 1. Find any problems
|
||||
python3 admin_tools.py find-problems
|
||||
|
||||
# 2. If issues found, unpublish them
|
||||
python3 admin_tools.py unpublish <ID> --reason "Quality check"
|
||||
|
||||
# 3. List current published articles
|
||||
python3 admin_tools.py list --status published
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Article Statuses
|
||||
|
||||
| Status | Meaning | Visible on Site? |
|
||||
|--------|---------|------------------|
|
||||
| `published` | Active article | ✅ Yes |
|
||||
| `draft` | Unpublished/hidden | ❌ No |
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Tips
|
||||
|
||||
### Finding Articles by ID
|
||||
|
||||
Articles have sequential IDs (1, 2, 3...). To find a specific article:
|
||||
|
||||
```bash
|
||||
# Show details
|
||||
python3 admin_tools.py details <ID>
|
||||
|
||||
# Check on website
|
||||
# URL format: https://burmddit.com/article/<SLUG>
|
||||
```
|
||||
|
||||
### Bulk Operations
|
||||
|
||||
To unpublish multiple articles, use a loop:
|
||||
|
||||
```bash
|
||||
# Unpublish articles 50, 83, and 9
|
||||
for id in 50 83 9; do
|
||||
python3 admin_tools.py unpublish $id --reason "Translation issues"
|
||||
done
|
||||
```
|
||||
|
||||
### Checking Translation Quality
|
||||
|
||||
Good translation ratios:
|
||||
- ✅ **40-80%** - Normal (Burmese is slightly shorter than English)
|
||||
- ⚠️ **20-40%** - Check manually (might be okay for technical content)
|
||||
- ❌ **< 20%** - Likely incomplete translation
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Security
|
||||
|
||||
**Access control:**
|
||||
- Only works with direct server access
|
||||
- Requires database credentials (`.env` file)
|
||||
- No public API or web interface
|
||||
|
||||
**Backup before major operations:**
|
||||
```bash
|
||||
# List all published articles first
|
||||
python3 admin_tools.py list --status published > backup_published.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### "Article not found"
|
||||
- Check article ID is correct
|
||||
- Use `list` command to see available articles
|
||||
|
||||
### "Database connection error"
|
||||
- Check `.env` file has correct `DATABASE_URL`
|
||||
- Verify database is running
|
||||
|
||||
### Changes not showing on website
|
||||
- Frontend may cache for a few minutes
|
||||
- Try clearing browser cache or private browsing
|
||||
|
||||
---
|
||||
|
||||
## 📞 Examples
|
||||
|
||||
### Example 1: Hide broken article immediately
|
||||
|
||||
```bash
|
||||
# Quick unpublish
|
||||
cd /home/ubuntu/.openclaw/workspace/burmddit/backend
|
||||
python3 admin_tools.py unpublish 50 --reason "Broken translation"
|
||||
```
|
||||
|
||||
### Example 2: Weekly quality check
|
||||
|
||||
```bash
|
||||
# Find and review all problem articles
|
||||
python3 admin_tools.py find-problems
|
||||
|
||||
# Review each one
|
||||
python3 admin_tools.py details 50
|
||||
python3 admin_tools.py details 83
|
||||
|
||||
# Unpublish bad ones
|
||||
python3 admin_tools.py unpublish 50
|
||||
python3 admin_tools.py unpublish 83
|
||||
```
|
||||
|
||||
### Example 3: Emergency cleanup
|
||||
|
||||
```bash
|
||||
# List all published
|
||||
python3 admin_tools.py list --status published
|
||||
|
||||
# Unpublish several at once
|
||||
for id in 50 83 9; do
|
||||
python3 admin_tools.py unpublish $id
|
||||
done
|
||||
|
||||
# Verify they're hidden
|
||||
python3 admin_tools.py list --status draft
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎓 Integration Ideas
|
||||
|
||||
### Add to cron for automatic checks
|
||||
|
||||
Create `/home/ubuntu/.openclaw/workspace/burmddit/scripts/auto-quality-check.sh`:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
cd /home/ubuntu/.openclaw/workspace/burmddit/backend
|
||||
|
||||
# Find problems and log
|
||||
python3 admin_tools.py find-problems > /tmp/quality_check.log
|
||||
|
||||
# If problems found, send alert
|
||||
if [ $(wc -l < /tmp/quality_check.log) -gt 5 ]; then
|
||||
echo "⚠️ Quality issues found - check /tmp/quality_check.log"
|
||||
fi
|
||||
```
|
||||
|
||||
Run weekly:
|
||||
```bash
|
||||
# Add to crontab
|
||||
0 10 * * 1 /home/ubuntu/.openclaw/workspace/burmddit/scripts/auto-quality-check.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Created:** 2026-02-26
|
||||
**Last updated:** 2026-02-26 09:09 UTC
|
||||
Reference in New Issue
Block a user