Russell Ballestrini
1e988b8553
- Change from 50GB to 200GB+ for full blockchain size - Update space savings calculation from 96% to 99% - Based on growth from 110GB in early 2024 to likely 200GB+ in 2025
403 lines
No EOL
14 KiB
ReStructuredText
403 lines
No EOL
14 KiB
ReStructuredText
========================================
|
|
Cryptocurrency Payment System
|
|
========================================
|
|
|
|
**Production-Ready Multi-Cryptocurrency Commerce Platform**
|
|
|
|
Overview
|
|
========
|
|
|
|
Make Post Sell provides a comprehensive cryptocurrency payment system that enables shops to accept Monero (XMR) and Dogecoin (DOGE) payments with enterprise-grade reliability, automatic fund management, and comprehensive monitoring.
|
|
|
|
**Key Features:**
|
|
|
|
- ✅ **Multi-cryptocurrency support** - XMR and DOGE with unified architecture
|
|
- ✅ **Comprehensive state machine** - 20+ payment states with robust error handling
|
|
- ✅ **Automatic fund management** - Auto-sweep to cold storage after confirmation
|
|
- ✅ **Advanced monitoring** - Centralized logging with payment context
|
|
- ✅ **Recovery mechanisms** - Tools for stuck payments and edge cases
|
|
- ✅ **Double-spend protection** - Transaction validation and duplicate detection
|
|
- ✅ **Production testing** - 300+ automated tests ensuring reliability
|
|
|
|
Architecture
|
|
============
|
|
|
|
Payment State Machine
|
|
----------------------
|
|
|
|
The system uses a comprehensive state machine to track payments through their lifecycle:
|
|
|
|
**Core States:**
|
|
- ``pending`` - Payment created, waiting for funds
|
|
- ``received`` - Funds detected, waiting for confirmations
|
|
- ``confirmed`` - Payment confirmed and processed
|
|
- ``confirmed_overpay`` - Overpayment confirmed, refund initiated
|
|
|
|
**Refund States:**
|
|
- ``latepay_refunded_complete`` - Late payment fully refunded
|
|
- ``underpaid_refunded_complete`` - Underpayment fully refunded
|
|
- ``out_of_stock_refunded_complete`` - Out of stock, fully refunded
|
|
|
|
**Terminal States:**
|
|
- ``expired`` - Payment window closed
|
|
- ``cancelled`` - Payment cancelled by user
|
|
- ``latepay_not_refunded`` - Late payment, no refund available
|
|
|
|
Each state transition is logged with full context for debugging and monitoring.
|
|
|
|
Components
|
|
----------
|
|
|
|
**1. Crypto Watcher Service**
|
|
- Background daemon monitoring blockchain for payments
|
|
- Processes 100+ payments per cycle with comprehensive logging
|
|
- Handles confirmation tracking and state transitions
|
|
- Includes recovery mechanisms for stuck payments
|
|
|
|
**2. Client Abstraction Layer**
|
|
- ``MoneroClient`` - RPC interface to monero-wallet-rpc
|
|
- ``DogecoinClient`` - RPC interface to dogecoind
|
|
- Unified error handling and credential protection
|
|
- Mock clients for testing and development
|
|
|
|
**3. Payment Processing Engine**
|
|
- Real-time quote generation with rate locking
|
|
- Unique address generation per payment
|
|
- Automatic confirmation threshold management
|
|
- Invoice deletion for failed payments
|
|
|
|
**4. Auto-Sweep System**
|
|
- Immediate sweep to cold storage after confirmation
|
|
- Perfect accounting to the quote amount (atomic units)
|
|
- Comprehensive sweep logging and error recovery
|
|
- Support for restocking fees and overpayment refunds
|
|
|
|
Supported Cryptocurrencies
|
|
===========================
|
|
|
|
Monero (XMR)
|
|
-------------
|
|
|
|
- Privacy-focused cryptocurrency with built-in anonymity
|
|
- Sub-address system provides perfect payment isolation
|
|
- Remote node support eliminates blockchain storage requirements
|
|
- Automatic account isolation for multi-tenant deployments
|
|
|
|
See ``docs/MONERO.rst`` for detailed setup and configuration.
|
|
|
|
Dogecoin (DOGE)
|
|
---------------
|
|
|
|
- Low transaction fees ideal for small payments
|
|
- Fast confirmation times (1-minute blocks)
|
|
- Pruned mode operation (2.2GB vs 200GB+ full node)
|
|
- Strong community and exchange support
|
|
|
|
See ``docs/DOGECOIN.md`` for detailed setup and configuration.
|
|
|
|
Payment Flow
|
|
============
|
|
|
|
Customer Experience
|
|
-------------------
|
|
|
|
1. **Product Selection** - Customer selects products and proceeds to checkout
|
|
2. **Cryptocurrency Choice** - Customer selects XMR or DOGE payment option
|
|
3. **Quote Generation** - Real-time exchange rate locked for 15 minutes
|
|
4. **Payment Instructions** - Unique address and exact amount provided
|
|
5. **Payment Monitoring** - Real-time confirmation tracking display
|
|
6. **Order Fulfillment** - Automatic product delivery after confirmation
|
|
7. **Receipt Generation** - Transaction details and confirmation provided
|
|
|
|
Technical Processing
|
|
--------------------
|
|
|
|
1. **Quote Creation**
|
|
::
|
|
|
|
# Real-time rate from CoinGecko with rate locking
|
|
payment = CryptoPayment(
|
|
expected_amount=calculate_crypto_amount(usd_total, coin_rate),
|
|
rate_locked_usd_per_coin=current_rate,
|
|
quote_expires_at_ms=now + 900000 # 15 minutes
|
|
)
|
|
|
|
2. **Address Generation**
|
|
::
|
|
|
|
# Monero: Create subaddress
|
|
address, subaddr_index = client.create_subaddress(
|
|
account_index=shop.account_index,
|
|
label=f"payment-{payment.id}"
|
|
)
|
|
|
|
# Dogecoin: Generate labeled address
|
|
address = client.getnewaddress(f"payment-{payment.id}")
|
|
|
|
3. **Payment Monitoring**
|
|
::
|
|
|
|
# Crypto watcher scans for incoming transactions
|
|
transfers = client.get_transfers_for_subaddr(
|
|
account_index, [payment.subaddress_index]
|
|
)
|
|
|
|
# Process confirmations and state transitions
|
|
if transfer.confirmations >= payment.confirmations_required:
|
|
payment.status = CryptoPayment.STATUS_CONFIRMED
|
|
finalize_payment(payment)
|
|
|
|
4. **Auto-Sweep Execution**
|
|
::
|
|
|
|
# Immediate sweep after confirmation
|
|
sweep_result = client.sendtoaddress(
|
|
address=payment.shop.cold_wallet_address,
|
|
amount=confirmed_amount - fee_reserve
|
|
)
|
|
|
|
log.sweep_operation(payment, "Auto-sweep successful",
|
|
sweep_result.tx_hash, swept_amount)
|
|
|
|
Monitoring and Logging
|
|
======================
|
|
|
|
Centralized Logging System
|
|
---------------------------
|
|
|
|
The system uses a centralized ``CryptoWatcherLogger`` that provides rich context for all payment operations:
|
|
|
|
**Payment Context Logging:**
|
|
::
|
|
|
|
# Comprehensive payment information in every log entry
|
|
log.payment_info(payment, "Payment confirmed", context="final_check")
|
|
# Output: Payment confirmed: Payment abc123 [confirmed] XMR 1.50/1.50 conf:10/10
|
|
# addr:8A1B... user:john@example.com shop:Tech Store
|
|
|
|
**State Transition Logging:**
|
|
::
|
|
|
|
log.state_transition(payment, "pending", "confirmed", "sufficient_confirmations")
|
|
# Output: State transition: Payment abc123 [pending → confirmed] XMR 1.50/1.50
|
|
# reason:sufficient_confirmations
|
|
|
|
**Sweep Operation Logging:**
|
|
::
|
|
|
|
log.sweep_operation(payment, "Auto-sweep completed", tx_hash, amount)
|
|
# Output: Auto-sweep completed: Payment abc123 [confirmed] XMR swept to cold storage
|
|
# TX:def456 Amount:1.495
|
|
|
|
**Error Context Logging:**
|
|
::
|
|
|
|
log.payment_error(payment, "RPC connection failed", exception)
|
|
# Output: RPC connection failed: Payment abc123 [pending] XMR connection timeout
|
|
|
|
Log Analysis
|
|
------------
|
|
|
|
**Monitor Payment Processing:**
|
|
::
|
|
|
|
# Watch real-time payment processing
|
|
tail -f logs/crypto_watcher.log | grep "Payment.*confirmed"
|
|
|
|
# Check sweep operations
|
|
grep "swept to cold storage" logs/crypto_watcher.log
|
|
|
|
# Monitor error conditions
|
|
grep "ERROR.*Payment" logs/crypto_watcher.log
|
|
|
|
**Key Log Patterns:**
|
|
- ``Processing X payments in cycle`` - Normal operation indicator
|
|
- ``Payment [id] state transition [old → new]`` - State machine progression
|
|
- ``Auto-sweep successful`` - Successful fund management
|
|
- ``Double spend detected`` - Security system activation
|
|
- ``Recovery operation`` - Stuck payment resolution
|
|
|
|
Recovery and Troubleshooting
|
|
=============================
|
|
|
|
Payment Recovery
|
|
----------------
|
|
|
|
The system includes comprehensive recovery mechanisms for various failure scenarios:
|
|
|
|
**Stuck Payment Recovery:**
|
|
::
|
|
|
|
# The crypto watcher includes automatic recovery for payments stuck in:
|
|
# - Pending state with expired quotes
|
|
# - Received state with insufficient confirmations
|
|
# - Processing failures during state transitions
|
|
|
|
# Manual recovery tools available for administrators:
|
|
# - Force state transitions for valid payments
|
|
# - Refund processing for failed payments
|
|
# - Manual sweep operations for accumulated funds
|
|
|
|
**Double Spend Protection:**
|
|
::
|
|
|
|
# Automatic detection and handling of duplicate transactions
|
|
# - Original payment processed normally
|
|
# - Duplicate payments marked and refunded
|
|
# - Comprehensive logging of security events
|
|
|
|
**Database Consistency:**
|
|
::
|
|
|
|
# Automatic invoice cleanup for failed payments:
|
|
# - Successful payments: Invoice preserved for customer records
|
|
# - Failed payments: Invoice deleted to prevent confusion
|
|
# - Recovery logging for audit trail
|
|
|
|
Administrative Tools
|
|
====================
|
|
|
|
Development and Testing
|
|
-----------------------
|
|
|
|
**Local Environment Setup:**
|
|
|
|
See cryptocurrency-specific documentation:
|
|
- Monero development: ``docs/MONERO.rst``
|
|
- Dogecoin development: ``docs/DOGECOIN.md``
|
|
|
|
**Testing and Validation:**
|
|
::
|
|
|
|
# Comprehensive test suite
|
|
make test # Full test suite (300+ tests)
|
|
make test-coverage # Test coverage analysis
|
|
|
|
# Specific crypto testing
|
|
py.test make_post_sell/tests/test_crypto_watcher.py # Payment processing
|
|
py.test make_post_sell/tests/test_models.py::TestCryptoPayment # Payment models
|
|
|
|
Security Considerations
|
|
=======================
|
|
|
|
Hot Wallet Security
|
|
-------------------
|
|
|
|
**Risk Assessment:**
|
|
- Hot wallets store private keys on the server for payment processing
|
|
- Server compromise = potential fund theft
|
|
- Mitigation: Automatic sweep to cold storage minimizes exposure
|
|
|
|
**Best Practices:**
|
|
- Configure automatic sweep to hardware wallets or offline storage
|
|
- Monitor sweep operations and set up alerts for failures
|
|
- Regular security audits of server infrastructure
|
|
- Implement proper access controls and monitoring
|
|
|
|
**Fund Management:**
|
|
- Perfect accounting to the quote amount in atomic units
|
|
- Customer pays: invoice + calculated fee buffer
|
|
- Full payment amount swept to cold storage
|
|
- Network fees handled dynamically during sweep operations
|
|
- No dust accumulation - exact amounts swept each time
|
|
|
|
Cold Storage Integration
|
|
------------------------
|
|
|
|
**Shop Configuration:**
|
|
- Each shop configures their own cold wallet address
|
|
- Cold wallets should be hardware wallets or air-gapped systems
|
|
- Automatic validation of cold wallet address format
|
|
|
|
**Sweep Verification:**
|
|
- All sweep operations logged with transaction hashes
|
|
- Failed sweeps trigger alerts and retry mechanisms
|
|
- Manual sweep tools available for emergency situations
|
|
|
|
Transaction Validation
|
|
----------------------
|
|
|
|
**Double Spend Protection:**
|
|
- Automatic detection of duplicate transactions to same address
|
|
- Original payment processed, duplicates marked for refund
|
|
- Comprehensive logging of security events
|
|
|
|
**Confirmation Management:**
|
|
- Risk-based confirmation requirements based on payment value
|
|
- Automatic adjustment for different cryptocurrencies
|
|
- Configurable thresholds per shop requirements
|
|
|
|
Production Deployment
|
|
=====================
|
|
|
|
System Requirements
|
|
-------------------
|
|
|
|
**Infrastructure:**
|
|
- Python 3.8+ with virtual environment
|
|
- PostgreSQL or SQLite database
|
|
- Pyramid web framework
|
|
- Log management and monitoring system
|
|
|
|
**Cryptocurrency-Specific Requirements:**
|
|
See individual documentation for detailed requirements:
|
|
- Monero: ``docs/MONERO.rst`` (~512MB RAM for wallet RPC with remote node)
|
|
- Dogecoin: ``docs/DOGECOIN.md`` (~4GB RAM for pruned node during sync, ~200MB after)
|
|
|
|
Configuration Management
|
|
-------------------------
|
|
|
|
**Configuration Overview:**
|
|
|
|
Configuration is managed through:
|
|
- INI files for system settings (confirmation requirements, RPC endpoints)
|
|
- Web interface for shop-specific settings (payment thresholds, quote expiry)
|
|
- Environment variables for sensitive credentials
|
|
|
|
See cryptocurrency-specific documentation for detailed configuration:
|
|
- Monero configuration: ``docs/MONERO.rst``
|
|
- Dogecoin configuration: ``docs/DOGECOIN.md``
|
|
|
|
**Security Best Practices:**
|
|
- Never expose RPC ports to the internet
|
|
- Use localhost binding for all cryptocurrency services
|
|
- Implement proper firewall rules and access controls
|
|
- Store credentials in environment variables, not in code
|
|
|
|
Monitoring and Alerts
|
|
---------------------
|
|
|
|
**Key Metrics to Monitor:**
|
|
- Payment processing rate and success rate
|
|
- Auto-sweep operation success rate
|
|
- RPC connection health and response times
|
|
- Database consistency and performance
|
|
- Error rates and recovery operations
|
|
|
|
**Alert Configuration:**
|
|
- Failed auto-sweep operations (immediate attention required)
|
|
- RPC connection failures (affects payment processing)
|
|
- High error rates or stuck payments
|
|
- Unusual transaction patterns or security events
|
|
|
|
**Log Management:**
|
|
- Centralized log collection with structured data
|
|
- Log rotation and retention policies
|
|
- Search and analysis capabilities for troubleshooting
|
|
- Integration with monitoring systems for alerting
|
|
|
|
Conclusion
|
|
==========
|
|
|
|
Make Post Sell's cryptocurrency payment system provides a production-ready foundation for accepting XMR and DOGE payments with enterprise-grade reliability. The comprehensive state machine, automatic fund management, and extensive testing ensure robust operation in production environments.
|
|
|
|
The system's architecture supports adding additional cryptocurrencies through the established client abstraction layer and payment processing framework, making it a scalable foundation for multi-cryptocurrency commerce.
|
|
|
|
**Key Benefits:**
|
|
- Proven reliability through comprehensive testing (300+ automated tests)
|
|
- Production-hardened security with automatic fund management
|
|
- Comprehensive monitoring and recovery capabilities
|
|
- Developer-friendly architecture with clear separation of concerns
|
|
- Extensive documentation and troubleshooting resources
|
|
|
|
For specific setup instructions, see the individual cryptocurrency guides and development environment documentation. |