MySMSAPio/README.md

# SMS Gateway API

A Ruby on Rails REST API and WebSocket server for managing SMS messaging through Android gateway devices. This system allows you to send and receive SMS messages programmatically, manage OTP codes, and integrate SMS capabilities into your applications.

## Features

- **Gateway Management**: Register and manage multiple Android SMS gateway devices
- **Inbound SMS**: Receive and process incoming SMS messages
- **Outbound SMS**: Send SMS messages through connected gateway devices
- **OTP Management**: Generate, send, and verify one-time passwords
- **WebSocket Communication**: Real-time bidirectional communication with gateway devices
- **Webhook Support**: Trigger webhooks for SMS events
- **Push Notifications (ntfy)**: Self-hosted ntfy integration for real-time alerts on gateway status, SMS delivery, and API key changes
- **Admin Interface**: Web-based dashboard for managing API keys, SMS logs, gateways, and notification settings
- **Rate Limiting**: Protect API endpoints from abuse
- **Load Balancing**: Automatically distribute messages across multiple gateways
- **Auto Failover**: Retry failed messages and handle gateway offline scenarios

## Tech Stack

- **Ruby**: 3.4.7
- **Rails**: 8.0.3
- **Database**: PostgreSQL 14+
- **Cache/Queue**: Redis 7+
- **Background Jobs**: Sidekiq 7
- **WebSocket**: Action Cable (Redis adapter)
- **API Authentication**: JWT + API Keys
- **Push Notifications**: ntfy (self-hosted, via Kamal accessory)
- **Admin UI**: Tailwind CSS v4 + Font Awesome

## Prerequisites

- Ruby 3.4.7
- PostgreSQL 14+
- Redis 7+
- Bundler 2.x

## Installation

### 1. Clone the repository

```bash
git clone <repository-url>
cd MySMSAPio
```

### 2. Install dependencies

```bash
bundle install
```

### 3. Set up environment variables

Create a `.env` file in the root directory:

```env
# Database
DATABASE_URL=postgresql://localhost/my_smsa_pio_development

# Redis
REDIS_URL=redis://localhost:6379/1

# CORS
ALLOWED_ORIGINS=*

# Phone validation
DEFAULT_COUNTRY_CODE=US

# Rails
SECRET_KEY_BASE=your_secret_key_here
RAILS_ENV=development

# ntfy Push Notifications (optional — defaults to https://ntfy.sh)
# NTFY_SERVER_URL=https://ntfy.yourdomain.com
```

### 4. Create and set up the database

```bash
bin/rails db:create
bin/rails db:migrate
bin/rails db:seed
```

**Important**: Save the API keys displayed after seeding! They won't be shown again.

### 5. Start Redis (if not running)

```bash
redis-server
```

### 6. Start Sidekiq (background jobs)

```bash
bundle exec sidekiq
```

### 7. Start the Rails server

```bash
bin/rails server
```

The API will be available at `http://localhost:3000`

## API Documentation

### Base URL

```
Development: http://localhost:3000
Production: https://your-domain.com
```

### Authentication

All API endpoints (except gateway registration) require an API key in the Authorization header:

```
Authorization: Bearer your_api_key_here
```

There are two types of API keys:
- **Gateway Keys**: Start with `gw_live_` - used by Android gateway devices
- **Client Keys**: Start with `api_live_` - used by your applications

---

## Gateway Device APIs

### Register a New Gateway

Register an Android device as an SMS gateway.

**Endpoint**: `POST /api/v1/gateway/register`

**Request**:
```json
{
  "device_id": "unique-device-identifier",
  "name": "My Gateway Phone"
}
```

**Response** (201 Created):
```json
{
  "success": true,
  "api_key": "gw_live_abc123...",
  "device_id": "unique-device-identifier",
  "websocket_url": "ws://localhost:3000/cable"
}
```

⚠️ **Important**: Save the `api_key` immediately. It will only be shown once!

### Send Heartbeat

Keep the gateway connection alive and update status.

**Endpoint**: `POST /api/v1/gateway/heartbeat`

**Headers**: `Authorization: Bearer gw_live_...`

**Request**:
```json
{
  "status": "online",
  "messages_in_queue": 5,
  "battery_level": 85,
  "signal_strength": 4
}
```

**Response** (200 OK):
```json
{
  "success": true,
  "pending_messages": 2
}
```

### Report Received SMS

Submit an SMS message received by the gateway device.

**Endpoint**: `POST /api/v1/gateway/sms/received`

**Headers**: `Authorization: Bearer gw_live_...`

**Request**:
```json
{
  "sender": "+1234567890",
  "message": "Hello, this is a test message",
  "timestamp": "2025-10-19T10:30:00Z"
}
```

**Response** (200 OK):
```json
{
  "success": true,
  "message_id": "msg_abc123..."
}
```

### Report SMS Delivery Status

Update the delivery status of an outbound SMS.

**Endpoint**: `POST /api/v1/gateway/sms/status`

**Headers**: `Authorization: Bearer gw_live_...`

**Request**:
```json
{
  "message_id": "msg_abc123",
  "status": "delivered",
  "error_message": null
}
```

Status values: `delivered`, `failed`, `sent`

**Response** (200 OK):
```json
{
  "success": true
}
```

---

## Client Application APIs

### Send SMS

Send an SMS message through the gateway.

**Endpoint**: `POST /api/v1/sms/send`

**Headers**: `Authorization: Bearer api_live_...`

**Request**:
```json
{
  "to": "+1234567890",
  "message": "Your verification code is: 123456"
}
```

**Response** (202 Accepted):
```json
{
  "success": true,
  "message_id": "msg_xyz789",
  "status": "queued"
}
```

### Check SMS Status

Check the delivery status of a sent message.

**Endpoint**: `GET /api/v1/sms/status/:message_id`

**Headers**: `Authorization: Bearer api_live_...`

**Response** (200 OK):
```json
{
  "message_id": "msg_xyz789",
  "status": "delivered",
  "sent_at": "2025-10-19T10:30:00Z",
  "delivered_at": "2025-10-19T10:30:05Z",
  "failed_at": null,
  "error_message": null
}
```

### Get Received SMS

Retrieve inbound SMS messages.

**Endpoint**: `GET /api/v1/sms/received`

**Headers**: `Authorization: Bearer api_live_...`

**Query Parameters**:
- `phone_number` (optional): Filter by phone number
- `since` (optional): ISO 8601 timestamp to filter messages after this time
- `limit` (optional): Number of messages per page (default: 50, max: 100)

**Response** (200 OK):
```json
{
  "messages": [
    {
      "message_id": "msg_abc",
      "from": "+1234567890",
      "message": "Reply message content",
      "received_at": "2025-10-19T10:30:00Z"
    }
  ],
  "total": 25,
  "page": 1,
  "pages": 1
}
```

---

## OTP APIs

### Send OTP

Generate and send a one-time password.

**Endpoint**: `POST /api/v1/otp/send`

**Headers**: `Authorization: Bearer api_live_...`

**Request**:
```json
{
  "phone_number": "+1234567890",
  "purpose": "authentication",
  "expiry_minutes": 5
}
```

**Response** (200 OK):
```json
{
  "success": true,
  "expires_at": "2025-10-19T10:35:00Z",
  "message_id": "msg_otp123"
}
```

**Rate Limits**: Maximum 3 OTP codes per phone number per hour.

### Verify OTP

Verify an OTP code.

**Endpoint**: `POST /api/v1/otp/verify`

**Headers**: `Authorization: Bearer api_live_...`

**Request**:
```json
{
  "phone_number": "+1234567890",
  "code": "123456"
}
```

**Response** (200 OK) - Success:
```json
{
  "success": true,
  "verified": true
}
```

**Response** (200 OK) - Failed:
```json
{
  "success": false,
  "verified": false,
  "error": "Invalid or expired OTP",
  "attempts_remaining": 2
}
```

---

## Admin APIs

### List Gateways

Get all registered gateway devices.

**Endpoint**: `GET /api/v1/admin/gateways`

**Headers**: `Authorization: Bearer api_live_...`

**Response** (200 OK):
```json
{
  "gateways": [
    {
      "id": 1,
      "device_id": "test-gateway-001",
      "name": "Test Gateway 1",
      "status": "online",
      "last_heartbeat_at": "2025-10-19T10:30:00Z",
      "messages_sent_today": 145,
      "messages_received_today": 23,
      "total_messages_sent": 1543,
      "total_messages_received": 892,
      "active": true,
      "priority": 1,
      "metadata": {},
      "created_at": "2025-10-19T08:00:00Z"
    }
  ]
}
```

### Toggle Gateway Status

Enable or disable a gateway.

**Endpoint**: `POST /api/v1/admin/gateways/:id/toggle`

**Headers**: `Authorization: Bearer api_live_...`

**Response** (200 OK):
```json
{
  "success": true,
  "gateway": {
    "id": 1,
    "device_id": "test-gateway-001",
    "active": false
  }
}
```

### Get System Statistics

Get overall system statistics.

**Endpoint**: `GET /api/v1/admin/stats`

**Headers**: `Authorization: Bearer api_live_...`

**Response** (200 OK):
```json
{
  "gateways": {
    "total": 2,
    "active": 2,
    "online": 1,
    "offline": 1
  },
  "messages": {
    "total_sent": 5432,
    "total_received": 892,
    "sent_today": 168,
    "received_today": 23,
    "total_today": 191,
    "pending": 3,
    "failed_today": 2
  },
  "otp": {
    "sent_today": 45,
    "verified_today": 42,
    "verification_rate": 93.33
  },
  "timestamp": "2025-10-19T10:30:00Z"
}
```

---

## WebSocket Connection (Gateway Devices)

Gateway devices connect via WebSocket for real-time bidirectional communication.

### Connection URL

```
ws://localhost:3000/cable?api_key=gw_live_your_key_here
```

### Subscribe to Gateway Channel

```javascript
{
  "command": "subscribe",
  "identifier": "{\"channel\":\"GatewayChannel\",\"api_key_digest\":\"sha256_hash_of_api_key\"}"
}
```

### Messages from Server

**Send SMS Command**:
```json
{
  "action": "send_sms",
  "message_id": "msg_123",
  "recipient": "+1234567890",
  "message": "Your OTP is: 123456"
}
```

### Messages to Server

**Heartbeat**:
```json
{
  "action": "heartbeat",
  "battery_level": 85,
  "signal_strength": 4,
  "messages_in_queue": 0
}
```

**Delivery Report**:
```json
{
  "action": "delivery_report",
  "message_id": "msg_123",
  "status": "delivered"
}
```

**Message Received**:
```json
{
  "action": "message_received",
  "sender": "+1234567890",
  "message": "Hello",
  "timestamp": "2025-10-19T10:30:00Z"
}
```

---

## Background Jobs

The system uses Sidekiq for background processing:

### Scheduled Jobs

- **CheckGatewayHealthJob**: Runs every minute to mark offline gateways
- **CleanupExpiredOtpsJob**: Runs every 15 minutes to delete expired OTP codes
- **ResetDailyCountersJob**: Runs daily at midnight to reset message counters

### Processing Jobs

- **SendSmsJob**: Handles outbound SMS delivery
- **ProcessInboundSmsJob**: Processes received SMS and triggers webhooks
- **RetryFailedSmsJob**: Retries failed messages with exponential backoff
- **TriggerWebhookJob**: Executes webhook HTTP calls
- **SendNtfyNotificationJob**: Sends push notifications via ntfy (async, retries 3×)

---

## Webhooks

Configure webhooks to receive real-time notifications for SMS events.

### Webhook Events

- `sms_received`: Triggered when an inbound SMS is received
- `sms_sent`: Triggered when an outbound SMS is sent
- `sms_failed`: Triggered when an SMS fails to send

### Webhook Payload Example

```json
{
  "event": "sms_received",
  "message_id": "msg_xyz",
  "from": "+1234567890",
  "message": "Hello",
  "received_at": "2025-10-19T10:30:00Z",
  "gateway_id": "test-gateway-001"
}
```

### Webhook Signature

If a `secret_key` is configured, webhooks include an HMAC-SHA256 signature in the `X-Webhook-Signature` header for verification.

---

## Push Notifications (ntfy)

The system integrates with [ntfy](https://ntfy.sh) for real-time push notifications to your phone or desktop. ntfy is a simple HTTP-based pub/sub notification service — no signup required.

### How It Works

1. A self-hosted ntfy server runs as a Docker container alongside the Rails app (configured as a Kamal accessory in `config/deploy.yml`).
2. Each admin user configures their own ntfy topic + access token in the admin UI at `/admin/notifications`.
3. When a monitored event occurs, `SendNtfyNotificationJob` fires asynchronously, POSTing a JSON payload to the ntfy server with title, message, priority, tags, and optional click URL.
4. The ntfy app on your phone (Android/iOS) or desktop receives the push notification.

### Notification Events

| Event | Priority | Tags | Trigger |
|---|---|---|---|
| `gateway_online` | default (3) | white_check_mark | Gateway transitions from offline → online |
| `gateway_offline` | high (4) | rotating_light | Gateway goes offline (WebSocket disconnect or stale heartbeat) |
| `sms_delivered` | low (2) | white_check_mark | Delivery report confirms SMS reached handset |
| `sms_failed` | urgent (5) | x, rotating_light | SMS fails after retries exhausted |
| `api_key_revoked` | default (3) | key, no_entry | API key revoked by an admin |

### Setup

1. **Install the ntfy app** on your phone ([Android](https://play.google.com/store/apps/details?id=io.heckel.ntfy) / [iOS](https://apps.apple.com/us/app/ntfy/id1625396347)) or use the [web app](https://ntfy.sh/app).
2. **Log in** to the admin interface at `/admin/login`.
3. **Navigate** to **Notifications** in the sidebar.
4. **Enter your ntfy topic** (treat it like a password — pick something hard to guess), access token (if your server has auth), and server URL (leave blank to use the default from `NTFY_SERVER_URL` env var).
5. **Click "Send Test Notification"** to verify connectivity.

### Configuration

- **Admin UI**: `/admin/notifications` — per-admin ntfy settings + test button
- **Environment variable**: `NTFY_SERVER_URL` — default ntfy server URL (used when an admin's `ntfy_server_url` field is blank). Defaults to `https://ntfy.sh`.
- **Kamal accessory**: ntfy runs as an accessory container in production (see `config/deploy.yml`). Requires `NTFY_AUTH_FILE` secret for access control.
- **Model**: `AdminUser` ntfy fields — `ntfy_topic`, `ntfy_token`, `ntfy_enabled`, `ntfy_server_url`

### Architecture

```
Event (gateway offline, SMS failed, etc.)
  ↓
NtfyDispatchable concern → dispatch_ntfy(event, payload)
  ↓
SendNtfyNotificationJob (async, :notifications queue, retries 3×)
  ↓
Ntfy::Publisher → HTTParty.post(ntfy_server_url/topic, JSON + Bearer auth)
  ↓
ntfy server → push to phone/desktop app
```

---

## Error Responses

All errors follow this format:

```json
{
  "error": "Error message here"
}
```

Common HTTP status codes:
- `400 Bad Request`: Invalid request parameters
- `401 Unauthorized`: Missing or invalid API key
- `403 Forbidden`: Insufficient permissions
- `404 Not Found`: Resource not found
- `422 Unprocessable Entity`: Validation errors
- `429 Too Many Requests`: Rate limit exceeded
- `500 Internal Server Error`: Server error

---

## Development

### Running Tests

```bash
bin/rails test
```

### Code Quality

Check code style:
```bash
bin/rubocop
```

Security scan:
```bash
bin/brakeman
```

### Console Access

```bash
bin/rails console
```

### Database Console

```bash
bin/rails dbconsole
```

---

## Deployment

### Using Kamal

This project is configured for deployment with Kamal.

```bash
# Deploy to production
bin/kamal deploy

# View logs
bin/kamal logs

# Access console
bin/kamal console
```

### Environment Variables (Production)

Required environment variables:

```env
DATABASE_URL=postgresql://user:pass@host/database
REDIS_URL=redis://host:6379/1
SECRET_KEY_BASE=your_production_secret
RAILS_ENV=production
ALLOWED_ORIGINS=https://yourdomain.com
DEFAULT_COUNTRY_CODE=US

# ntfy Push Notifications
NTFY_SERVER_URL=https://ntfy.yourdomain.com
NTFY_AUTH_FILE=/var/lib/ntfy/user.db
```

ntfy runs as a Kamal accessory container. It listens on port 80 (mapped to `127.0.0.1:8090` on the host) with a persistent volume at `ntfy_data:/var/lib/ntfy`. The `NTFY_AUTH_FILE` secret points to the ntfy access control database. See `config/deploy.yml` for the full accessory configuration.

---

## Monitoring

### Health Check

```
GET /up
```

Returns 200 if the application is healthy.

### Sidekiq Web UI

Mount Sidekiq web interface in `config/routes.rb` (protect with authentication in production):

```ruby
require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'
```

---

## Architecture

```
┌─────────────────┐         ┌──────────────────┐
│                 │         │                  │
│  Android SMS    │◄───WS──►│  Rails API       │
│  Gateway App    │         │  Action Cable    │
│                 │         │                  │
└─────────────────┘         └────────┬─────────┘
                                     │
                            ┌────────┴─────────┐
                            │                  │
        ┌───────────────────┤  PostgreSQL      │
        │                   │  (Messages, OTP) │
        │                   │                  │
        │                   └──────────────────┘
        │
        │                   ┌──────────────────┐
        │                   │                  │
        └──────────────────►│  Redis           │
                            │  (Cache, Jobs,   │
                            │   WebSockets)    │
                            │                  │
                            └────────┬─────────┘
                                     │
                            ┌────────┴─────────┐
                            │                  │
                            │  Sidekiq         │
                            │  (Background     │
                            │   Jobs)          │
                            │                  │
                            └────────┬─────────┘
                                     │
                            ┌────────┴─────────┐
                            │                  │
                            │  ntfy Server     │
                            │  (Push Notifs)   │
                            │                  │
                            └──────────────────┘
```

---

## License

MIT License

---

## Support

For issues and questions, please create an issue on GitHub.