From b5a32c797e37ffdbe7230ad2987d5b724d713dae Mon Sep 17 00:00:00 2001 From: Russell Ballestrini Date: Mon, 22 Dec 2025 12:43:35 -0500 Subject: [PATCH] Clean up PayPal docs: remove references to non-existent files --- docs/CHANGES_PAYPAL.md | 568 +++-------------------------------------- 1 file changed, 34 insertions(+), 534 deletions(-) diff --git a/docs/CHANGES_PAYPAL.md b/docs/CHANGES_PAYPAL.md index 7c5f142..07d5058 100644 --- a/docs/CHANGES_PAYPAL.md +++ b/docs/CHANGES_PAYPAL.md @@ -1,556 +1,56 @@ -# PayPal Integration - Changes Log +# PayPal Integration -**Date:** November 7, 2025 -**Feature:** Complete PayPal Payment Integration -**Status:** ✅ Fully Tested and Production Ready +Adds PayPal as a payment processor alongside Stripe and crypto (XMR/DOGE). ---- - -## Overview - -Integrated PayPal as a payment processor following the same architectural patterns as Stripe. PayPal now works alongside Stripe, Monero, and Dogecoin as a supported payment method. - ---- - -## Files Created +## Files Added ### Models -- **`make_post_sell/models/paypal_user_shop.py`** - - Tracks PayPal payer IDs and billing agreements per user/shop relationship - - Similar to `StripeUserShop` model - - Columns: id, user_id, shop_id, payer_id, billing_agreement_id, active_payment_token - -- **`make_post_sell/models/paypal_payment.py`** - - Tracks PayPal order transactions and payment status per invoice - - Columns: id, invoice_id, paypal_order_id, paypal_payer_id, paypal_capture_id, status, amount_in_cents, timestamps - - Methods: `is_completed`, `is_pending`, `is_failed`, `update_status()` - - Helper: `get_paypal_payment_by_order_id()` +- `make_post_sell/models/paypal_user_shop.py` - Tracks PayPal payer IDs per user/shop +- `make_post_sell/models/paypal_payment.py` - Tracks PayPal transactions per invoice ### Views -- **`make_post_sell/views/paypal.py`** - - `paypal_create_order` - Creates PayPal order from cart (JSON API) - - `paypal_complete_checkout` - Captures PayPal payment and creates invoice - - Handles payment flow, invoice creation, product unlocking, email notifications +- `make_post_sell/views/paypal.py` - Order creation and checkout capture +- `make_post_sell/views/paypal_webhooks.py` - Webhook handler for payment events -- **`make_post_sell/views/paypal_webhooks.py`** - - `paypal_webhook` - Handles PayPal webhook notifications - - Processes events: PAYMENT.CAPTURE.COMPLETED, PAYMENT.CAPTURE.DENIED, CUSTOMER.DISPUTE.CREATED - - Updates payment status in database - - Note: PAYMENT.CAPTURE.REFUNDED is NOT handled (refunds are external) - -### Migrations -- **`make_post_sell/scripts/alembic/versions/418933067d81_add_paypal_credentials_to_shop.py`** - - Adds PayPal columns to Shop table: paypal_client_id, paypal_secret, paypal_enabled - - Note: New tables (mps_paypal_user_shop, mps_paypal_payment) are auto-created by SQLAlchemy - -### Documentation -- **`MIGRATIONS.md`** - - Comprehensive migration guide - - Pre-migration checklist - - Manual and CI/CD migration procedures - - Rollback instructions - - Verification commands - - Troubleshooting guide - -- **`CLAUDE.md`** (updated) - - Added PayPal Integration section - - Configuration instructions - - How to get PayPal credentials - - Payment flow explanation - - Database schema overview - - Webhook setup - - Testing with PayPal Sandbox - -- **`scripts/run_migrations.sh`** - - Automated migration script with safety features - - Automatic database backup before migration - - Pre-flight checks - - Migration verification - - Color-coded output - - Supports --auto-approve and --dry-run modes - -- **`.gitlab-ci.yml.migration-example`** - - Multiple CI/CD migration options - - Using migration script - - Inline migration commands - - Salt/Ansible integration example - - Two-step migration process (safer) - -- **`test_paypal_integration.py`** - - Comprehensive test suite - - Tests database structure, models, routes, configuration, migrations - - Color-coded output - - All tests passing ✅ - -- **`CHANGES_PAYPAL.md`** (this file) - - Complete changelog of all PayPal integration work - ---- +### Migration +- `make_post_sell/scripts/alembic/versions/418933067d81_add_paypal_credentials_to_shop.py` + - Adds `paypal_client_id`, `paypal_secret`, `paypal_enabled` to mps_shop + - New tables (mps_paypal_user_shop, mps_paypal_payment) are auto-created ## Files Modified -### Models -- **`make_post_sell/models/meta.py`** - - Added `PayPalUserShop` and `PayPalPayment` to `CLASS_TO_TABLE` mapping +- `make_post_sell/models/__init__.py` - Import new models +- `make_post_sell/models/meta.py` - Register models in CLASS_TO_TABLE +- `make_post_sell/models/shop.py` - PayPal columns, `is_paypal_ready`, `paypal_user_shop()` +- `make_post_sell/routes.py` - PayPal routes +- `make_post_sell/request_methods.py` - `request.paypal_enabled` +- `make_post_sell/views/cart.py` - Pass PayPal context to checkout template +- `make_post_sell/templates/cart_checkout.j2` - PayPal button +- `make_post_sell/templates/shop_settings.j2` - PayPal settings form +- `make_post_sell/static/css/common.css` - PayPal input styles +- `development.ini` - PayPal config settings -- **`make_post_sell/models/shop.py`** - - Added columns: paypal_client_id, paypal_secret, paypal_enabled - - Added property: `is_paypal_ready` - checks if PayPal credentials configured - - Added property: `is_paypal_not_ready` - inverse check - - Added property: `paypal` - lazy-loaded PayPal SDK client instance - - Added method: `paypal_user_shop(user)` - retrieves PayPalUserShop for user - - Updated method: `is_ready_for_payment()` - includes PayPal check +## Configuration -- **`make_post_sell/models/invoice.py`** - - Updated property: `payment_status` - maps PayPal status to standard statuses - - Updated property: `payment_method` - returns "paypal" for PayPal payments - -### Views -- **`make_post_sell/views/cart.py`** - - Updated `cart_checkout()` - loads PayPalUserShop, passes to template - - Updated payment method availability check - includes PayPal - - Updated template context - adds paypal_enabled, paypal_user_shop - -### Routes -- **`make_post_sell/routes.py`** - - Added route: `paypal_create_order` - /paypal/create-order/{cart_id} - - Added route: `paypal_complete_checkout` - /paypal/complete-checkout/{cart_id} - - Added route: `paypal_webhook` - /webhooks/paypal - -### Configuration -- **`development.ini`** - - Added setting: `app.paypal.sandbox_mode` (default: True) - - Added setting: `app.paypal.webhook_id` (for webhook verification) - - Added setting: `app.payments.paypal.enabled` (default: True for testing) - -- **`data/development.ini`** (local copy) - - Same changes as above - - PayPal enabled by default for local testing - -### Request Methods -- **`make_post_sell/request_methods.py`** - - Added function: `add_paypal_enabled()` - checks global + shop-level enable - - Added function: `add_paypal_globally_enabled()` - checks global config - - Registered: `request.paypal_enabled` property - - Registered: `request.paypal_globally_enabled` property - -### Templates -- **`make_post_sell/templates/cart_checkout.j2`** - - Added PayPal button section (after Stripe, before free checkout) - - Integrated PayPal JavaScript SDK - - PayPal Buttons widget configuration - - Order creation via fetch to `/paypal/create-order/{cart_id}` - - Form submission with order ID to `/paypal/complete-checkout/{cart_id}` - - Error handling for failed payments - -- **`make_post_sell/templates/shop_settings.j2`** - - Added "PayPal Settings" section (after Stripe settings) - - Client ID and Secret input fields - - Show/hide toggle for credentials - - Enable/Disable PayPal buttons - - Status indicators (configured ✓ / disabled ✗) - - Mirrors Stripe settings UI/UX - -### Shop Settings View -- **`make_post_sell/views/shop.py`** - - Added PayPal sandbox mode detection - - Added PayPal form parameters extraction (client_id, secret) - - Added "paypal-settings" form section handler - - Validates PayPal credentials (length check, sandbox mode check) - - Handles enable/disable PayPal actions - - Provides user feedback messages - -### Styles -- **`make_post_sell/static/css/common.css`** - - Added CSS rule for `#toggle-paypal:checked ~ .hidden-control` (lines 1613-1616) - - Enables show/hide toggle functionality for PayPal credentials in shop settings - - Added `input.mps-paypal-client-id` and `input.mps-paypal-secret` to input width rules (lines 264-265) - - Sets max-width: 600px and width: 100% for proper field display - - Mirrors existing Stripe input field styling - ---- - -## Database Schema Changes - -### New Tables - -**mps_paypal_user_shop:** -```sql -CREATE TABLE mps_paypal_user_shop ( - id CHAR(32) PRIMARY KEY, - user_id CHAR(32) NOT NULL, - shop_id CHAR(32) NOT NULL, - payer_id VARCHAR(64), - billing_agreement_id VARCHAR(64), - active_payment_token VARCHAR(128), - FOREIGN KEY (user_id) REFERENCES mps_user (id), - FOREIGN KEY (shop_id) REFERENCES mps_shop (id) -); +In `development.ini`: +```ini +app.payments.paypal.enabled = True +app.paypal.sandbox_mode = True ``` -**mps_paypal_payment:** -```sql -CREATE TABLE mps_paypal_payment ( - id CHAR(32) PRIMARY KEY, - invoice_id CHAR(32) NOT NULL, - paypal_order_id VARCHAR(64) NOT NULL, - paypal_payer_id VARCHAR(64), - paypal_capture_id VARCHAR(64), - status VARCHAR(32) NOT NULL, - amount_in_cents BIGINT NOT NULL, - created_timestamp BIGINT NOT NULL, - updated_timestamp BIGINT NOT NULL, - FOREIGN KEY (invoice_id) REFERENCES mps_invoice (id) -); -``` - -### Modified Tables - -**mps_shop:** -- Added: `paypal_client_id` VARCHAR(128) NULL -- Added: `paypal_secret` VARCHAR(128) NULL -- Added: `paypal_enabled` BOOLEAN NOT NULL DEFAULT 1 - ---- - -## Dependencies Added - -- **`paypalrestsdk`** (v1.13.3) - - Official PayPal REST SDK for Python - - Required dependencies: pyopenssl, cryptography - ---- - -## Configuration Requirements - -### Environment Variables (Optional) +Or via environment: ```bash -# Enable PayPal globally (default: False in production) export MPS_PAYMENTS_PAYPAL_ENABLED=True - -# Set sandbox mode (default: True for development) export MPS_PAYPAL_SANDBOX_MODE=True - -# PayPal webhook ID for verification (optional) -export MPS_PAYPAL_WEBHOOK_ID=your_webhook_id_here ``` -### Per-Shop Configuration -Each shop configures their own PayPal credentials via admin UI: -- PayPal Client ID (from PayPal Developer Dashboard) -- PayPal Secret (from PayPal Developer Dashboard) -- Enable/Disable PayPal toggle +## Shop Setup ---- +1. Get credentials from https://developer.paypal.com/dashboard/ +2. Go to Shop Settings → PayPal Settings +3. Enter Client ID and Secret +4. Save -## Payment Flow +## Dependencies -### User Checkout Process -1. User adds items to cart -2. User proceeds to checkout at `/u/cart/{cart_id}/checkout` -3. PayPal button renders (if PayPal enabled and configured) -4. User clicks PayPal button -5. JavaScript SDK calls `/paypal/create-order/{cart_id}` (creates PayPal order) -6. PayPal popup opens for user approval -7. User approves payment in PayPal -8. JavaScript posts order ID to `/paypal/complete-checkout/{cart_id}` -9. Server captures PayPal order -10. Invoice and PayPalPayment records created -11. Products unlocked for user -12. Confirmation emails sent -13. User redirected to invoice/product page - -### Server-Side Order Creation -- Endpoint: `POST /paypal/create-order/{cart_id}` -- Returns: `{"order_id": "xxx"}` or `{"error": "message"}` -- Creates PayPal order with cart total - -### Server-Side Order Capture -- Endpoint: `POST /paypal/complete-checkout/{cart_id}` -- Receives: `paypal_order_id` parameter -- Captures PayPal order -- Creates invoice and payment record -- Unlocks products -- Sends emails -- Redirects to success page - -### Webhook Processing -- Endpoint: `POST /webhooks/paypal` -- Processes payment events asynchronously -- Updates payment status in database -- Handles: completion, denial, disputes -- Note: Refunds are NOT handled (managed externally by shop owners) - ---- - -## Testing - -### Automated Tests -Run `python test_paypal_integration.py` to verify: -- ✅ Database structure (tables, columns) -- ✅ Model imports -- ✅ Model registration in meta.py -- ✅ Migrations applied -- ✅ Configuration loaded -- ✅ Routes registered - -**Result:** All 6 tests passing - -### Manual Testing Checklist -- [ ] Access shop settings at `/s/{shop_id}/settings` -- [ ] Verify PayPal Settings section appears -- [ ] Configure PayPal credentials (sandbox) -- [ ] Create test product -- [ ] Add to cart and checkout -- [ ] Verify PayPal button appears -- [ ] Complete payment with sandbox account -- [ ] Verify invoice created -- [ ] Verify PayPalPayment record in database -- [ ] Verify product unlocked -- [ ] Verify emails sent - -### PayPal Sandbox Setup -1. Go to https://developer.paypal.com/dashboard/ -2. Create sandbox application -3. Copy Client ID and Secret -4. Use sandbox test accounts for payment -5. View transactions at https://www.sandbox.paypal.com - ---- - -## Migration Instructions - -### Local/Development -```bash -# 1. Backup database -cp data/make_post_sell.sqlite data/make_post_sell.sqlite.backup-$(date +%Y%m%d-%H%M%S) - -# 2. Run migrations -source env/bin/activate -alembic -c data/development.ini upgrade head - -# 3. Verify -alembic -c data/development.ini current -``` - -### Automated Script -```bash -# Run migration script (interactive) -./scripts/run_migrations.sh - -# Or for CI/CD (no prompts) -./scripts/run_migrations.sh --auto-approve -``` - -### Production (via GitLab CI) -See `.gitlab-ci.yml.migration-example` for multiple deployment options. - ---- - -## Rollback Procedure - -If issues occur after migration: - -```bash -# Option 1: Downgrade migrations -alembic -c data/development.ini downgrade 81d65d8605c2 - -# Option 2: Restore from backup -cp data/make_post_sell.sqlite.backup-YYYYMMDD-HHMMSS data/make_post_sell.sqlite -``` - ---- - -## Critical Improvements (Post-Testing Session) - -### 🔥 Production-Ready Enhancements - -**Date:** January 2025 -**Status:** ✅ All Critical Bugs Fixed - -After comprehensive testing, the following critical improvements were implemented: - -#### 1. **Independent Shop Payment Processing** (CRITICAL BUG FIX) -- **Problem:** All-or-nothing multi-shop checkout caused customers to be charged without invoices created -- **Solution:** Each shop's payment now processes independently -- **Impact:** - - Shop A succeeds → Invoice created, products delivered, removed from cart - - Shop B fails → Stays in cart for retry, no charge - - No refunds needed, better user experience -- **Files:** `make_post_sell/views/paypal.py` - Complete refactor of `paypal_complete_checkout()` - -#### 2. **Webhook Signature Verification** (SECURITY FIX) -- **Problem:** Webhook handler had placeholder code, accepting any webhook -- **Solution:** Full PayPal webhook signature verification using PayPal API -- **Impact:** Prevents fake webhook attacks, validates using shop-specific credentials -- **Files:** `make_post_sell/views/paypal_webhooks.py` - -#### 3. **Amount Validation Before Capture** (FRAUD PREVENTION) -- **Problem:** No validation that captured amount matches expected invoice total -- **Solution:** Validates captured amount with 1¢ tolerance for rounding -- **Impact:** Prevents race conditions, amount manipulation, cart total changes -- **Files:** `make_post_sell/views/paypal.py` - Added in capture flow - -#### 4. **Shop-Specific Coupon Calculation** (CRITICAL BUG FIX) -- **Problem:** Proportional discount bug - coupons for one shop applied to all shops -- **Solution:** Uses Invoice model for correct shop-specific coupon application -- **Impact:** - - Store A with $10 coupon pays $50 (correct) - - Store B with no coupon pays $40 (correct) - - Previously both got proportional discount (wrong) -- **Files:** `make_post_sell/views/paypal.py` - Fixed in `paypal_create_order()` - -#### 5. **Double-Click Protection** (UX IMPROVEMENT) -- **Problem:** Users could accidentally create duplicate orders -- **Solution:** JavaScript flags prevent duplicate order creation and submission -- **Impact:** Prevents duplicate charges, better error recovery -- **Files:** `make_post_sell/templates/cart_checkout.j2` - -#### 6. **Comprehensive Error Logging** (DEBUGGING IMPROVEMENT) -- **Problem:** Minimal logging made debugging difficult -- **Solution:** ISO timestamp logging throughout payment flow with full tracebacks -- **Impact:** Complete audit trail for every payment, easier debugging -- **Files:** `make_post_sell/views/paypal.py`, `make_post_sell/views/paypal_webhooks.py` - -#### 7. **Transaction Rollback Verification** (DATA INTEGRITY) -- **Problem:** Unclear if database transactions rolled back properly on failures -- **Solution:** Verified all error paths call `request.tm.abort()` correctly -- **Impact:** Ensures atomicity - either all succeeds or nothing persists -- **Files:** Code review verified - all paths correct - -#### 8. **Multi-Shop PayPal Support** (FEATURE ENHANCEMENT) -- **Problem:** UI showed warning that multi-shop carts weren't supported -- **Solution:** Removed limitation with independent payment processing -- **Impact:** Users can checkout with products from multiple shops -- **Files:** `make_post_sell/templates/cart_checkout.j2` - -### Documentation Added -- **`PAYPAL_MULTI_SHOP_BEHAVIOR.md`** - Comprehensive guide to multi-shop payment processing - - Payment flow explanation - - Error handling strategies - - User messaging examples - - Testing scenarios - - Security considerations - -#### 9. **Refund Policy Change** (ARCHITECTURE DECISION) -- **Decision:** PayPal refunds are NOT handled by the application -- **Rationale:** Refunds are a business decision between shop owner and customer -- **Implementation:** - - Removed PAYMENT.CAPTURE.REFUNDED webhook handler - - No automatic access revocation - - No refund tracking or logging - - Shop owners manage refunds directly via PayPal dashboard -- **Impact:** Simplifies platform, gives shop owners full control -- **Files Modified:** - - `make_post_sell/views/paypal_webhooks.py` - Removed refund handler - - `REFUND_ABUSE_PREVENTION.md` - Updated policy documentation -- **Note:** Disputes are still logged via CUSTOMER.DISPUTE.CREATED for awareness - ---- - -## Known Limitations (Updated) - -1. ~~**Multi-shop carts:** Currently uses first shop's PayPal credentials for multi-shop carts~~ ✅ **FIXED** -2. ~~**Webhook verification:** Webhook signature verification not fully implemented~~ ✅ **FIXED** -3. **Saved payment methods:** PayPal billing agreements supported but not tested (low priority) -4. ~~**Refunds:** Refund handling in webhooks is stubbed~~ ✅ **DECISION:** Refunds handled externally -5. ~~**Currency:** Currently hardcoded to USD only~~ ✅ **FIXED:** Multi-currency support added -6. **Session expiration:** Long PayPal approval times may cause session timeout (needs testing) - ---- - -## Security Considerations (Updated) - -✅ **Per-shop credentials:** Each shop uses their own PayPal account -✅ **Sandbox mode:** Automatic detection in development -✅ **CSRF protection:** All POST endpoints require CSRF token -✅ **Transaction safety:** Database rollback on payment failure (verified) -✅ **Credential hiding:** Show/hide toggle in admin UI -✅ **Validation:** Client ID and Secret validation before saving -✅ **Webhook signature verification:** Full implementation using PayPal API ✅ **NEW** -✅ **Amount validation:** Captured amount matches expected total ✅ **NEW** -✅ **Double-click protection:** Prevents duplicate order creation ✅ **NEW** -✅ **Independent processing:** Failed payments don't block successful ones ✅ **NEW** - -⚠️ **TODO:** Add rate limiting to PayPal endpoints -⚠️ **TODO:** Add session timeout handling for long PayPal approval flows - ---- - -## Performance Considerations - -- PayPal SDK client lazy-loaded per shop (cached) -- No N+1 queries in checkout flow -- Async webhook processing (doesn't block checkout) -- Database indexes on paypal_order_id for quick lookups - ---- - -## Browser Compatibility - -PayPal JavaScript SDK supports: -- Chrome/Edge (latest 2 versions) -- Firefox (latest 2 versions) -- Safari (latest 2 versions) -- Mobile Safari (iOS 11+) -- Chrome Mobile (Android 5+) - ---- - -## Next Steps (Optional Enhancements) - -1. **Billing Agreements:** Implement saved PayPal payment methods -2. **Refund UI:** Add admin interface for processing refunds -3. **Webhook Verification:** Complete webhook signature validation -4. **Analytics:** Track PayPal vs Stripe conversion rates -5. **Multi-currency:** Support currencies beyond USD -6. **Subscription Support:** Integrate PayPal subscriptions for recurring products - ---- - -## Support & Documentation - -- **PayPal Integration Guide:** See `CLAUDE.md` "Payment Processor Configuration" section -- **Migration Guide:** See `MIGRATIONS.md` -- **PayPal Developer Docs:** https://developer.paypal.com/docs/ -- **Webhook Events:** https://developer.paypal.com/api/rest/webhooks/ - ---- - -## Git Commit Checklist - -Files to commit: -- [ ] All new files in `make_post_sell/models/` -- [ ] All new files in `make_post_sell/views/` -- [ ] All new migration files in `make_post_sell/scripts/alembic/versions/` -- [ ] Modified files (meta.py, shop.py, invoice.py, cart.py, routes.py, request_methods.py) -- [ ] Modified templates (cart_checkout.j2, shop_settings.j2) -- [ ] Modified styles (static/css/common.css - PayPal toggle) -- [ ] Modified configuration (development.ini) -- [ ] Documentation (MIGRATIONS.md, CLAUDE.md updates, CHANGES_PAYPAL.md) -- [ ] Scripts (scripts/run_migrations.sh, test_paypal_integration.py) -- [ ] GitLab CI example (.gitlab-ci.yml.migration-example) - ---- - -## Contributors - -- Integration developed following existing Stripe patterns -- All tests passing -- Production-ready code -- Comprehensive documentation - ---- - -## Version - -- **PayPal Integration Version:** 1.0.0 -- **Compatible with:** make_post_sell 1.1.4+ -- **Tested on:** Python 3.12, SQLite 3.x -- **PayPal SDK:** paypalrestsdk 1.13.3 - ---- - -**Status:** ✅ READY FOR PRODUCTION - -All automated tests passing. Manual testing recommended before deploying to production. +- `paypalrestsdk` (added to requirements)