russellballestrini
ee2e180793
modified: 2010-11-30-the-barrymores-stole-my-heart-then-crushed-it.rst modified: 2010-12-31-four2go-a-new-spin-on-an-old-classic.rst modified: 2011-01-02-a-homegrown-python-bread-crumb-module.rst modified: 2011-01-05-how-did-stack-overflow-get-initial-traction.rst modified: 2011-01-09-graphic-design-powerhouse-wacom-bamboo-ubuntu-and-mypaint.rst modified: 2011-01-10-i-just-used-google-to-purchase-chinese-food-takeout-for-two.rst modified: 2011-01-11-response-to-l-theanine-a-4000-year-old-mind-hack.rst modified: 2011-03-26-porting-the-chaostheory-wordpress-theme-to-pylowiki.rst modified: 2011-04-03-voice-over-ip-with-teamspeak.rst modified: 2011-04-16-dropbox-encryption-with-truecrypt.rst modified: 2011-06-25-google-bot-attempts-to-crawl-shortest-urls-first.rst modified: 2011-07-02-add-a-breadcrumb-subscriber-to-a-pyramid-project-using-4-simple-steps.rst modified: 2011-07-11-a-hack-to-gain-80-percent-efficiency-when-creating-github-projects.rst modified: 2011-08-08-programming-is-like-alchemy.rst modified: 2011-08-18-security-professionals-yes-we-appear-vulnerable-but-that-attack-vector-will-never-happen.rst modified: 2011-08-22-python-image-grabber-pig-py.rst modified: 2011-09-06-deliberating-the-viewers-vs-doers-concept.rst modified: 2011-09-08-a-system-administrators-guide-to-installing-and-maintaining-multiple-python-environments.rst modified: 2011-09-09-hacker-olive-oil-lamp-crafted-from-home-materials.rst modified: 2011-10-01-how-do-i-calculate-the-m-in-my-mvp.rst modified: 2011-10-05-linkpeek-com-web-address-thumbnail-api-alpha-release.rst modified: 2011-10-12-a-better-way-to-show-website-backlinks.rst modified: 2011-10-15-i-cancelled-my-xbox-live-automatic-renewal.rst modified: 2011-10-15-r8168-driver-issues-after-ubuntu-11-10-upgrade-kernel-linux-3-0.rst modified: 2011-11-30-how-to-incorporate-custom-configuration-in-a-pyramid-application.rst modified: 2011-12-19-linkpeek-com-webpage-to-image-was-a-by-product.rst modified: 2012-01-18-today-i-lost-a-customer.rst modified: 2012-02-16-my-4-month-olds-15-minutes-of-fame.rst modified: 2012-03-03-how-to-save-hundreds-of-dollars-on-groceries-without-clipping-coupons.rst modified: 2012-03-26-nosslsearch-cname-is-a-bad-idea-and-solution.rst modified: 2012-05-19-high-load-on-web-server-after-updating-from-ubuntu-10-04-to-ubuntu-12-04-lts.rst modified: 2012-06-10-always-attempt-to-scale-vertically-first.rst modified: 2012-06-29-miniuri-parser-and-ago-human-timedelta.rst modified: 2013-01-09-ago-py-human-readable-timedelta0-0-4-release.rst modified: 2013-03-02-virt-backs-domfetcher-class-returns-doms-from-libvirt-api.rst modified: 2013-03-22-guido-name-dropped-tornado-python-tulip-and-pep-3156.rst modified: 2013-05-26-honey-i-just-deleted-linkpeek-com.rst modified: 2013-06-20-high-load-and-cpu-usage-craftbukkit-compared-to-vanilla-minecraft.rst modified: 2013-10-19-simplify-deployments-with-upstart-and-uwsgi.rst modified: 2013-10-27-backup-all-virtual-machines-on-a-smartos-hypervisor-with-smart-back-sh.rst modified: 2013-12-13-hackathon-2013-virtualization.rst modified: 2013-12-17-sensu-salt.rst modified: 2014-01-02-test-game-engine-with-python-and-sfml.rst modified: 2014-01-25-how-to-reset-hp-ilo-lights-out-user-and-password-settings-with-ipmitools.rst modified: 2014-02-21-configuration-management-and-the-golden-image.rst modified: 2014-03-08-replace-the-nagios-scheduler-and-nrpe-with-salt-stack.rst modified: 2014-04-08-how-to-patch-heartbleed-openssl-defect-libssl-on-ubuntu.rst modified: 2014-08-23-heka-world2.rst modified: 2014-12-03-custom-rundeck-hipchat-notification-templates.rst modified: 2015-02-22-securely-publish-jenkins-build-artifacts-on-salt-master.rst modified: 2015-05-08-a-python-script-which-searches-for-available-interpreters.rst modified: 2015-11-15-migrating-from-wordpress-to-pelican.rst modified: 2016-06-21-output-all-instance-identifiers-of-an-aws-vpc-to-json.rst modified: 2016-11-21-sharing-a-pyramid-cookie-with-flask-or-tornado.rst
77 lines
3.1 KiB
ReStructuredText
77 lines
3.1 KiB
ReStructuredText
Simplify deployments with Upstart and uWSGI
|
|
###########################################
|
|
:date: 2013-10-19 22:35
|
|
:author: Russell Ballestrini
|
|
:tags: DevOps, Opinion
|
|
:slug: simplify-deployments-with-upstart-and-uwsgi
|
|
:status: published
|
|
|
|
As you know from my previous post, I recently `deleted
|
|
LinkPeek.com <https://russell.ballestrini.net/honey-i-just-deleted-linkpeek-com/>`__
|
|
and after struggling to get it back online, I vowed to start utilizing
|
|
configuration management. During this exercise, I noticed that the
|
|
architecture I use in production seems overly complicated.
|
|
|
|
**The current production deployment stack:**
|
|
|
|
- Nginx listen on 80/443 proxy upstream 9901/9902
|
|
- Upstart => Supervisord => Cherrypy/Paste listen on 9901/9902
|
|
|
|
*Each of these services and processes have their own configuration files
|
|
which must work together. Upstart needs to know the location of
|
|
Supervisord's configuration files. Supervisord needs to know the
|
|
location of Cherrypy/PasteDeploy's configuration files. Supervisord also
|
|
must bring up a specified number of worker processes who listen on a
|
|
pool of ports. Nginx needs to proxy upstream to that pool of worker
|
|
ports.*
|
|
|
|
This architecture seems difficult to automate because of the numerous
|
|
places errors may sneak in. I started researching an alternative
|
|
architecture and stumbled upon Nicholas Piel's `Benchmark of Python Web
|
|
Servers <https://nichol.as/benchmark-of-python-web-servers>`__. The
|
|
graphs Nicholas compiled allowed me to narrow down a list of potential
|
|
replacements.
|
|
|
|
I chose to review uWSGI first because it was a top contender on every
|
|
chart. After briefly testing uWSGI, I halted my explorations and
|
|
selected it as the winner. I know you probably feel that was a premature
|
|
decision but uWSGI fit what I was looking for.
|
|
|
|
**The new simplified stack:**
|
|
|
|
- Nginx listen on 80/443 proxy upstream 5200
|
|
- Upstart => uWSGI listen on 5200
|
|
|
|
The uWSGI server possesses great performance out-of-the-box but also
|
|
presents many options for fine tuning. These options may be specified
|
|
either directly in the Upstart script using flags or in a configuration
|
|
file (like a Pyramid .ini). Either way, the stack uses less files, less
|
|
processes, and less complexity then the original architecture. For
|
|
`LinkPeek <https://linkpeek.com>`__ deployments I decided to place all
|
|
uWSGI related configuration directly in the Upstart script which I
|
|
embedded below:
|
|
|
|
**linkpeek/weblinkpeek.conf \| Upstart for starting uWSGI**
|
|
|
|
::
|
|
|
|
description "Start the uWSGI master for the LinkPeek WEB"
|
|
|
|
start on runlevel [2345]
|
|
stop on runlevel [!2345]
|
|
|
|
respawn
|
|
|
|
# run service as linkpeek user
|
|
setuid linkpeek
|
|
setgid linkpeek
|
|
|
|
# uWSGI command to execute and treat as a daemon
|
|
exec /path/to/linkpeek/web/env/bin/uwsgi --master --processes=4 --http=127.0.0.1:5200 --die-on-term --logto2=/path/to/linkpeek/web/uwsgi.log --virtualenv=/path/to/linkpeek/web/env --ini-paste=/path/to/linkpeek/web/linkpeek/production.ini
|
|
|
|
.. raw:: html
|
|
|
|
</p>
|
|
|
|
The web application is now daemonized and listening on port 5200 with 4
|
|
workers. I leave implementing the Nginx front end to the reader.
|