engineering/unrhodecode
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: updated documentation structure

Browse source
This commit is contained in:
Marcin Kuzminski 2019-05-21 16:33:01 +02:00
parent 434efcd94f
commit eddc784e19
47 changed files with 158 additions and 98 deletions
Download patch file Download diff file Expand all files Collapse all files

264
docs/admin/system_admin/admin-tricks.rst Normal file
View file

@ -0,0 +1,264 @@
.. _admin-tricks:
One-time Admin Tasks
--------------------
* :ref:`web-analytics`
* :ref:`admin-tricks-license`
* :ref:`announcements`
* :ref:`md-rst`
* :ref:`repo-stats`
* :ref:`server-side-merge`
* :ref:`remap-rescan`
* :ref:`custom-hooks`
* :ref:`clear-repo-cache`
* :ref:`set-repo-pub`
* :ref:`ping`
.. _web-analytics:
Adding Web Analytics
^^^^^^^^^^^^^^^^^^^^
If you wish to add a Google Analytics, or any other kind of tracker to your
|RCE| instance you can add the necessary codes to the header or footer
section of each instance using the following steps:
1. From the |RCE| interface, select
:menuselection:`Admin --> Settings --> Global`
2. To add a tracking code to you instance, enter it in the header or footer
section and select **Save**
Use the example templates in the drop-down menu to set up your configuration.
.. _admin-tricks-license:
Licence Key Management
^^^^^^^^^^^^^^^^^^^^^^
To manage your license key, go to
:menuselection:`Admin --> Settings --> License`.
On this page you can see the license key details. If you need a new license,
or have questions about your current one, contact support@rhodecode.com
.. _announcements:
Server-wide Announcements
^^^^^^^^^^^^^^^^^^^^^^^^^
If you need to make a server-wide announcement to all users,
you can add a message to be displayed using the following steps:
1. From the |RCE| interface, select
:menuselection:`Admin --> Settings --> Global`
2. To add a message that will be displayed to all users,
select :guilabel:`Server Announcement` from the drop-down menu and
change the ``var message = "TYPE YOUR MESSAGE HERE";`` example line.
3. Select :guilabel:`Save`, and you will see the message once your page
refreshes.
.. image:: ../../images/server-wide-announcement.png
:alt: Server Wide Announcement
.. _md-rst:
Suppress license warnings or errors
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In case you're running on maximum allowed users, RhodeCode will display a
warning message on pages that you're close to the license limits.
It's often not desired to show that all the time. Here's how you can suppress
the license messages.
1. From the |RCE| interface, select
:menuselection:`Admin --> Settings --> Global`
2. Select :guilabel:`Flash message filtering` from the drop-down menu.
3. Select :guilabel:`Save`, and you will no longer see the license message
once your page refreshes.
.. _admin-tricks-suppress-license-messages:
Markdown or RST Rendering
^^^^^^^^^^^^^^^^^^^^^^^^^
|RCE| can use `Markdown`_ or `reStructured Text`_ in commit message,
code review messages, and inline comments. To set the default to either,
select your preference from the drop-down menu on the
:menuselection:`Admin --> Settings --> Visual` page and select
:guilabel:`Save settings`.
.. _repo-stats:
Enabling Repository Statistics
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To enable |repo| statistics, use the following steps:
1. From the |RCE| interface, open
:menuselection:`Admin --> Repositories` and select
:guilabel:`Edit` beside the |repo| for which you wish to enable statistics.
2. Check the :guilabel:`Enable statistics` box, and select :guilabel:`Save`
.. _server-side-merge:
Enabling Server-side Merging
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To enable server-side merging, use the following steps:
1. From the |RCE| interface, open :menuselection:`Admin --> Settings --> VCS`
2. Check the :guilabel:`Server-side merge` box, and select
:guilabel:`Save Settings`
If you encounter slow performance with server-side merging enabled, check the
speed at which your server is performing actions. When server-side merging is
enabled, the following actions occurs on the server.
* A |pr| is created in the database.
* A shadow |repo| is created as a working environment for the |pr|.
* On display, |RCE| checks if the |pr| can be merged.
To check how fast the shadow |repo| creation is occurring on your server, use
the following steps:
1. Log into your server and create a directory in your |repos| folder.
2. Clone a |repo| that is showing slow performance and time the action.
.. code-block:: bash
# One option is to use the time command
$ time hg clone SOURCE_REPO TARGET
.. _remap-rescan:
Remap and Rescan Repositories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You may want to Remap and rescan the |repos| that |RCE| is managing to ensure
the system is always up-to-date. This is useful after importing, deleting,
or carrying out general cleaning up operations. To do this use the
following steps:
1. From the |RCE|, open
:menuselection:`Admin --> Settings --> Remap and rescan`
2. Click :guilabel:`Rescan Repositories`
Check the additional options if needed:
* :guilabel:`Destroy old data`: Useful for purging deleted repository
information from the database.
* :guilabel:`Invalidate cache for all repositories`: Use this to completely
remap all |repos|. Useful when importing or migrating |repos| to ensure all
new information is picked up.
.. _custom-hooks:
Adding Custom Hooks
^^^^^^^^^^^^^^^^^^^
To add custom hooks to your instance, use the following steps:
1. Open :menuselection:`Admin --> Settings --> Hooks`
2. Add your custom hook details, you can use a file path to specify custom
hook scripts, for example:
``pretxnchangegroup.example`` with value ``python:/path/to/custom_hook.py:my_func_name``
3. Select :guilabel:`Save`
Also, see the RhodeCode Extensions section of the :ref:`rc-tools` guide. RhodeCode
Extensions can be used to add additional hooks to your instance and comes
with a number of pre-built plugins if you chose to install them.
.. _clear-repo-cache:
Clearing |repo| cache
^^^^^^^^^^^^^^^^^^^^^
If you need to clear the cache for a particular |repo|, use the following steps:
1. Open :menuselection:`Admin --> Repositories` and select :guilabel:`Edit`
beside the |repo| whose cache you wish to clear.
2. On the |repo| settings page, go to the :guilabel:`Caches` tab and select
:guilabel:`Invalidate repository cache`.
.. _set-lang:
Changing Default Language
^^^^^^^^^^^^^^^^^^^^^^^^^
To change the default language of a |RCE| instance, change the language code
in the :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file. To
do this, use the following steps.
1. Open the :file:`rhodecode.ini` file and set the required language code.
.. code-block:: ini
## Optional Languages
## en(default), de, fr, it, ja, pl, pt, ru, zh
lang = de
2. Restart the |RCE| instance and check that the language has been updated.
.. code-block:: bash
$ rccontrol restart enterprise-2
Instance "enterprise-2" successfully stopped.
Instance "enterprise-2" successfully started.
.. image:: ../../images/language.png
.. _set-repo-pub:
Setting Repositories to Publish
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To automatically promote your local |repos| to public after pushing to |RCE|,
enable the :guilabel:`Set repositories as publishing` option on the
:menuselection:`Admin --> Settings --> VCS` page.
.. note::
This option is enabled by default on most |RCE| versions, but if upgrading
from a 1.7.x version it could be disabled on upgrade due to inheriting
older default settings.
.. _ping:
Pinging the |RCE| Server
^^^^^^^^^^^^^^^^^^^^^^^^
You can check the IP Address of your |RCE| instance using the
following URL: ``{instance-URL}/_admin/ping``.
.. code-block:: bash
$ curl https://your.rhodecode.url/_admin/ping
pong[rce-7880] => 203.0.113.23
.. _Markdown: http://daringfireball.net/projects/markdown/
.. _reStructured Text: http://docutils.sourceforge.net/docs/index.html
Unarchiving a repository
^^^^^^^^^^^^^^^^^^^^^^^^^
Archive operation for the repository is similar as delete. Archive keeps the data for future references
but makes the repository read-only. After archiving the repository it shouldn't be modified in any way.
This is why repository settings are disabled for an archived repository.
If there's a need for unarchiving a repository for some reasons, the interactive
ishell interface should be used.
.. code-block:: bash
# Open iShell from the terminal
$ rccontrol ishell enterprise-1/community-1
.. code-block:: python
# Set repository as un-archived
In [1]: repo = Repository.get_by_repo_name('SOME_REPO_NAME')
In [2]: repo.archived = False
In [3]: Session().add(repo);Session().commit()

14
docs/admin/system_admin/apache-config.rst Normal file
View file

@ -0,0 +1,14 @@
.. _apache-ws-ref:
Apache HTTP Server Configuration
--------------------------------
To set up your Apache Web Server for optimal performance and security, use
the information in the following sections.
.. toctree::
apache/apache-conf-example
apache/apache-diffie-hellman
apache/apache-subdirectory
apache/apache-wsgi-coding

107
docs/admin/system_admin/apache/apache-conf-example.rst Normal file
View file

@ -0,0 +1,107 @@
.. _apache-conf-eg:
Apache Configuration Example
----------------------------
Use the following example to configure Apache as a your web server.
Below config if for an Apache Reverse Proxy configuration.
.. note::
Apache requires the following modules to be enabled. Below is an example
how to enable them on Ubuntu Server
.. code-block:: bash
$ sudo a2enmod proxy
$ sudo a2enmod proxy_http
$ sudo a2enmod proxy_balancer
$ sudo a2enmod headers
$ sudo a2enmod ssl
$ sudo a2enmod rewrite
# requires Apache 2.4+, required to handle websockets/channelstream
$ sudo a2enmod proxy_wstunnel
.. code-block:: apache
## HTTP to HTTPS rewrite
<VirtualHost *:80>
ServerName rhodecode.myserver.com
DocumentRoot /var/www/html
Redirect permanent / https://rhodecode.myserver.com/
</VirtualHost>
## MAIN SSL enabled server
<VirtualHost *:443>
ServerName rhodecode.myserver.com
ServerAlias rhodecode.myserver.com
## Skip ProxyPass the _static to backend server
#ProxyPass /_static !
## serve static files by Apache, recommended for performance
#Alias /_static/rhodecode /home/ubuntu/.rccontrol/community-1/static
## Allow Apache to access the static files in this directory
#<Directory /home/ubuntu/.rccontrol/community-1/static/>
# AllowOverride none
# Require all granted
#</Directory>
RequestHeader set X-Forwarded-Proto "https"
## channelstream websocket handling
ProxyPass /_channelstream ws://localhost:9800
ProxyPassReverse /_channelstream ws://localhost:9800
<Proxy *>
Order allow,deny
Allow from all
</Proxy>
# Directive to properly generate url (clone url) for RhodeCode
ProxyPreserveHost On
# It allows request bodies to be sent to the backend using chunked transfer encoding.
SetEnv proxy-sendchunked 1
# Increase headers size for large Mercurial headers sent with many branches
LimitRequestLine 16380
# Url to running RhodeCode instance. This is shown as `- URL:` when
# running rccontrol status.
ProxyPass / http://127.0.0.1:10002/ timeout=7200 Keepalive=On
ProxyPassReverse / http://127.0.0.1:10002/
# strict http prevents from https -> http downgrade
Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
# Set x-frame options
Header always append X-Frame-Options SAMEORIGIN
# To enable https use line below
# SetEnvIf X-Url-Scheme https HTTPS=1
# SSL setup
SSLEngine On
SSLCertificateFile /etc/apache2/ssl/rhodecode.myserver.pem
SSLCertificateKeyFile /etc/apache2/ssl/rhodecode.myserver.key
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
SSLHonorCipherOrder on
# Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
#SSLOpenSSLConfCmd DHParameters "/etc/apache2/dhparam.pem"
## custom 502 error page. Will be displayed while RhodeCode server
## is turned off
ErrorDocument 502 /path/to/.rccontrol/enterprise-1/static/502.html
</VirtualHost>

34
docs/admin/system_admin/apache/apache-diffie-hellman.rst Normal file
View file

@ -0,0 +1,34 @@
.. _dh-apache:
Diffie-Hellman Security
-----------------------
To secure your web server, the `Guide to Deploying Diffie-Hellman for TLS`_
contains important information worth reading. This link contains some good
`secure Apache configuration`_ examples.
To secure your deployment of Diffie-Hellman, configure the following:
1. Generate a strong Diffie-hellman group, 2048-bit or stronger.
.. code-block:: bash
# to generate your dhparam.pem file, run in the terminal
openssl dhparam -out /etc/apache/ssl/dhparam.pem 2048
2. Configure your server to only use modern, secure cipher suites in the
virtual hosts configuration file.
.. code-block:: apache
# Set the protocol to only use modern, secure cipher suites.
SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder on
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
# Specify your DH params file as follows
SSLOpenSSLConfCmd DHParameters "{path to dhparams.pem}"
.. _Guide to Deploying Diffie-Hellman for TLS: https://weakdh.org/sysadmin.html
.. _secure Apache configuration: http://www.apache-ssl.org/httpd.conf.example

35
docs/admin/system_admin/apache/apache-subdirectory.rst Normal file
View file

@ -0,0 +1,35 @@
.. _apache-sub-ref:
Apache URL Prefix Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Use the following example to configure Apache to use a URL prefix.
.. code-block:: apache
# Change someprefix into your chosen prefix
<Location /someprefix >
ProxyPreserveHost On
ProxyPass "http://127.0.0.1:5000/"
ProxyPassReverse "http://127.0.0.1:5000/"
Header set X-Url-Scheme https env=HTTPS
</Location>
In addition to the regular Apache setup you will need to add the following
lines into the ``rhodecode.ini`` file.
* Above ``[app:main]`` section of the ``rhodecode.ini`` file add the
following section if it doesn't exist yet.
.. code-block:: ini
[filter:proxy-prefix]
use = egg:PasteDeploy#prefix
prefix = /<someprefix> # Change <someprefix> into your chosen prefix
* In the the ``[app:main]`` section of your ``rhodecode.ini`` file add the
following line.
.. code-block:: ini
filter-with = proxy-prefix

55
docs/admin/system_admin/apache/apache-wsgi-coding.rst Normal file
View file

@ -0,0 +1,55 @@
.. _apache-wsgi-ref:
Apache WSGI Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^
|RCE| can also be set up with Apache under ``mod_wsgi``. To configure this
use the following steps.
1. Install ``mod_wsgi`` using the following command:
``aptitude install libapache2-mod-wsgi``.
2. Enable ``mod_wsgi`` using the following command: ``a2enmod wsgi``
3. Create a ``wsgi`` dispatch script, using the following examples.
.. code-block:: bash
WSGIDaemonProcess pylons \
threads=4 \
# check the python virtual env location
python-path=/home/web/rhodecode/pyenv/lib/python2.6/site-packages
# Check the install location
WSGIScriptAlias / /home/web/rhodecode/dispatch.wsgi
WSGIPassAuthorization On
# user=www-data group=www-data # Enable if running Apache as root
.. note::
Do not set ``processes=num`` in this configuration file. Running |RCE| in
multiprocess mode with Apache is not supported.
The following is an example ``wsgi`` dispatch script.
.. code-block:: python
import os
os.environ["HGENCODING"] = "UTF-8"
os.environ['PYTHON_EGG_CACHE'] = '/home/web/rhodecode/.egg-cache'
# Set the current dir
os.chdir('/home/web/rhodecode/')
import site
site.addsitedir("/home/web/rhodecode/pyenv/lib/python2.6/site-packages")
from paste.deploy import loadapp
from paste.script.util.logging_config import fileConfig
fileConfig('/home/web/rhodecode/production.ini')
application = loadapp('config:/home/web/rhodecode/production.ini')
.. note::
When using `mod_wsgi` the same version of |hg| must be running in your
system's |PY| environment and on |RCE|. To check the |RCE| version,
on the interface go to
:menuselection:`Admin --> Settings --> System Info`

151
docs/admin/system_admin/backup-restore.rst Normal file
View file

@ -0,0 +1,151 @@
.. _backup-ref:
Backup and Restore
==================
*“The condition of any backup is unknown until a restore is attempted.”*
`Schrödinger's Backup`_
To snapshot an instance of |RCE|, and save its settings, you need to backup the
following parts of the system at the same time.
* The |repos| managed by the instance together with the stored Gists.
* The |RCE| database.
* Any configuration files or extensions that you've configured. In most
cases it's only the :file:`rhodecode.ini` file.
* Installer files such as those in `/opt/rhodecode` can be backed-up, however
it's not required since in case of a recovery installer simply
re-creates those.
.. important::
Ideally you should script all of these functions so that it creates a
backup snapshot of your system at a particular timestamp and then run that
script regularly.
Backup Details
--------------
To backup the relevant parts of |RCE| required to restore your system, use
the information in this section to identify what is important to you.
Repository Backup
^^^^^^^^^^^^^^^^^
To back up your |repos|, use the API to get a list of all |repos| managed,
and then clone them to your backup location. This is the most safe backup option.
Backing up the storage directory could potentially result in a backup of
partially committed files or commits. (Backup taking place during a big push)
As an alternative you could use a rsync or simple `cp` commands if you can
ensure your instance is only in read-only mode or stopped at the moment.
Use the ``get_repos`` method to list all your managed |repos|,
and use the ``clone_uri`` information that is returned. See the :ref:`api`
for more information. Be sure to keep the structure or repositories with their
repository groups.
.. important::
This will not work for |svn| |repos|. Currently the only way to back up
your |svn| |repos| is to make a copy of them.
It is also important to note, that you can only restore the |svn| |repos|
using the same version as they were saved with.
Database Backup
^^^^^^^^^^^^^^^
The instance database contains all the |RCE| permissions settings,
and user management information. To backup your database,
export it using the following appropriate example, and then move it to your
backup location:
.. code-block:: bash
# For MySQL DBs
$ mysqldump -u <uname> -p <pass> rhodecode_db_name > mysql-db-backup
# MySQL restore command
$ mysql -u <uname> -p <pass> rhodecode_db_name < mysql-db-backup
# For PostgreSQL DBs
$ PGPASSWORD=<pass> pg_dump --inserts -U <uname> -h localhost rhodecode_db_name > postgresql-db-backup
# PosgreSQL restore
$ PGPASSWORD=<pass> psql -U <uname> -h localhost -d rhodecode_db_name -1 -f postgresql-db-backup
# For SQLite
$ sqlite3 rhodecode.db ‘.dump’ > sqlite-db-backup
# SQLite restore
$ copy sqlite-db-backup rhodecode.db
The default |RCE| SQLite database location is
:file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.db`
If running MySQL or PostgreSQL databases, you will have configured these
separately, for more information see :ref:`rhodecode-database-ref`
Configuration File Backup
^^^^^^^^^^^^^^^^^^^^^^^^^
Depending on your setup, you could have a number of configuration files that
should be backed up. You may have some, or all of the configuration files
listed in the :ref:`config-rce-files` section. Ideally you should back these
up at the same time as the database and |repos|. It really depends on if you need
the configuration file like logs, custom modules. We always recommend backing
those up.
Gist Backup
^^^^^^^^^^^
To backup the gists on your |RCE| instance you usually have to backup the
gist storage path. If this haven't been changed it's located inside
:file:`.rc_gist_store` and the metadata in :file:`.rc_gist_metadata`.
You can use the ``get_users`` and ``get_gists`` API methods to fetch the
gists for each user on the instance.
Extension Backups
^^^^^^^^^^^^^^^^^
You should also backup any extensions added in the
:file:`home/{user}/.rccontrol/{instance-id}/rcextensions` directory.
Full-text Search Backup
^^^^^^^^^^^^^^^^^^^^^^^
You may also have full text search set up, but the index can be rebuild from
re-imported |repos| if necessary. You will most likely want to backup your
:file:`search_mapping.ini` file if you've configured that. For more information, see
the :ref:`indexing-ref` section.
Restoration Steps
-----------------
To restore an instance of |RCE| from its backed up components, to a fresh
system use the following steps.
1. Install a new instance of |RCE| using sqlite option as database.
2. Restore your database.
3. Once installed, replace you backed up the :file:`rhodecode.ini` with your
backup version. Ensure this file points to the restored
database, see the :ref:`config-database` section.
4. Restart |RCE| and remap and rescan your |repos| to verify filesystem access,
see the :ref:`remap-rescan` section.
Post Restoration Steps
^^^^^^^^^^^^^^^^^^^^^^
Once you have restored your |RCE| instance to basic functionality, you can
then work on restoring any specific setup changes you had made.
* To recreate the |RCE| index, use the backed up :file:`search_mapping.ini` file if
you had made changes and rerun the indexer. See the
:ref:`indexing-ref` section for details.
* To reconfigure any extensions, copy the backed up extensions into the
:file:`/home/{user}/.rccontrol/{instance-id}/rcextensions` and also specify
any custom hooks if necessary. See the :ref:`extensions-hooks-ref` section for
details.
.. _Schrödinger's Backup: http://novabackup.novastor.com/blog/schrodingers-backup-good-bad-backup/

23
docs/admin/system_admin/cleanup-cmds.rst Normal file
View file

@ -0,0 +1,23 @@
.. _clean-up-cmds:
|RCT| Clean Up Commands
=======================
|RCT| comes with a number of functions which can be used to administer your
|RCE| instances. Two of these can be used to automate the cleanup of gists
and |repos|
rhodecode-cleanup-gists
-----------------------
Use this command to delete gists within RhodeCode Enterprise. It takes a
number of options for specifying the kind of gists you want deleted, and it
is possible to run these commands from a cron job or cleanup script. For more
information, see the :ref:`tools-cli`
rhodecode-cleanup-repos
-----------------------
Use this command to delete |repos| from your |RCE| instances. It takes
a number of options specifying the kind of |repos| you want deleted. For more
information, see the :ref:`tools-cli`

74
docs/admin/system_admin/config-files-overview.rst Normal file
View file

@ -0,0 +1,74 @@
.. _config-files:
Configuration Files Overview
============================
|RCE| and |RCC| have a number of different configuration files. The following
is a brief explanation of each, and links to their associated configuration
sections.
.. rst-class:: dl-horizontal
\- **rhodecode.ini**
Default location:
:file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
This is the main |RCE| configuration file and controls much of its
default behaviour. It is also used to configure certain customer
settings. Here are some of the most common reasons to make changes to
this file.
* :ref:`config-database`
* :ref:`set-up-mail`
* :ref:`increase-gunicorn`
* :ref:`x-frame`
\- **search_mapping.ini**
Default location:
:file:`/home/{user}/.rccontrol/{instance-id}/search_mapping.ini`
This file is used to control the |RCE| indexer. It comes configured
to index your instance. To change the default configuration, see
:ref:`advanced-indexing`.
\- **vcsserver.ini**
Default location:
:file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini`
The VCS Server handles the connection between your |repos| and |RCE|.
See the :ref:`vcs-server` section for configuration options and more
detailed information.
\- **supervisord.ini**
Default location:
:file:`/home/{user}/.rccontrol/supervisor/supervisord.ini`
|RCC| uses Supervisor to monitor and manage installed instances of
|RCE| and the VCS Server. |RCC| will manage this file completely,
unless you install |RCE| in self-managed mode. For more information,
see the :ref:`Supervisor Setup<control:supervisor-setup>` section.
\- **.rccontrol.ini**
Default location: :file:`/home/{user}/.rccontrol.ini`
This file contains the instances that |RCC| starts at boot, which is all
by default, but for more information, see
the :ref:`Manually Start At Boot <control:set-start-boot>` section.
\- **.rhoderc**
Default location: :file:`/home/{user}/.rhoderc`
This file is used by the |RCE| API when accessing an instance from a
remote machine. The API checks this file for connection and
authentication details. For more details, see the :ref:`config-rhoderc`
section.
\- **MANIFEST**
Default location: :file:`/home/{user}/.rccontrol/cache/MANIFEST`
|RCC| uses this file to source the latest available builds from the
secure RhodeCode download channels. The only reason to mess with this file
is if you need to do an offline installation,
see the :ref:`Offline Installation<control:offline-installer-ref>`
instructions, otherwise |RCC| will completely manage this file.

148
docs/admin/system_admin/enable-debug.rst Normal file
View file

@ -0,0 +1,148 @@
.. _debug-mode:
Enabling Debug Mode
-------------------
Debug Mode will enable debug logging, and request tracking middleware. Debug Mode
enabled DEBUG log-level which allows tracking various information about authentication
failures, LDAP connection, email etc.
The request tracking will add a special
unique ID: `| req_id:00000000-0000-0000-0000-000000000000` at the end of each log line.
The req_id is the same for each individual requests, it means that if you want to
track particular user logs only, and exclude other concurrent ones
simply grep by `req_id` uuid which you'll have to find for the individual request.
To enable debug mode on a |RCE| instance you need to set the debug property
in the :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file. To
do this, use the following steps
1. Open the file and set the ``debug`` line to ``true``
2. Restart you instance using the ``rccontrol restart`` command,
see the following example:
.. code-block:: ini
[DEFAULT]
debug = true
.. code-block:: bash
# Restart your instance
$ rccontrol restart enterprise-1
Instance "enterprise-1" successfully stopped.
Instance "enterprise-1" successfully started.
Debug and Logging Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Further debugging and logging settings can also be set in the
:file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
In the logging section, the various packages that run with |RCE| can have
different debug levels set. If you want to increase the logging level change
``level = DEBUG`` line to one of the valid options.
You also need to change the log level for handlers. See the example
``##handler`` section below. The ``handler`` level takes the same options as
the ``debug`` level.
.. code-block:: ini
################################
### LOGGING CONFIGURATION ####
################################
[loggers]
keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
[handlers]
keys = console, console_sql, file, file_rotating
[formatters]
keys = generic, color_formatter, color_formatter_sql
#############
## LOGGERS ##
#############
[logger_root]
level = NOTSET
handlers = console
[logger_sqlalchemy]
level = INFO
handlers = console_sql
qualname = sqlalchemy.engine
propagate = 0
[logger_beaker]
level = DEBUG
handlers =
qualname = beaker.container
propagate = 1
[logger_rhodecode]
level = DEBUG
handlers =
qualname = rhodecode
propagate = 1
[logger_ssh_wrapper]
level = DEBUG
handlers =
qualname = ssh_wrapper
propagate = 1
[logger_celery]
level = DEBUG
handlers =
qualname = celery
##############
## HANDLERS ##
##############
[handler_console]
class = StreamHandler
args = (sys.stderr, )
level = DEBUG
formatter = generic
[handler_console_sql]
class = StreamHandler
args = (sys.stderr, )
level = INFO
formatter = generic
[handler_file]
class = FileHandler
args = ('rhodecode_debug.log', 'a',)
level = INFO
formatter = generic
[handler_file_rotating]
class = logging.handlers.TimedRotatingFileHandler
# 'D', 5 - rotate every 5days
# you can set 'h', 'midnight'
args = ('rhodecode_debug_rotated.log', 'D', 5, 10,)
level = INFO
formatter = generic
################
## FORMATTERS ##
################
[formatter_generic]
class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %Y-%m-%d %H:%M:%S
[formatter_color_formatter]
class = rhodecode.lib.logging_formatter.ColorFormatter
format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %Y-%m-%d %H:%M:%S
[formatter_color_formatter_sql]
class = rhodecode.lib.logging_formatter.ColorFormatterSql
format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %Y-%m-%d %H:%M:%S

50
docs/admin/system_admin/gunicorn-ssl-support.rst Normal file
View file

@ -0,0 +1,50 @@
.. _gunicorn-ssl-support:
Gunicorn SSL support
--------------------
:term:`Gunicorn` wsgi server allows users to use HTTPS connection directly
without a need to use HTTP server like Nginx or Apache. To Configure
SSL support directly with :term:`Gunicorn` you need to simply add the key
and certificate paths to your configuration file.
1. Open the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
2. In the ``[server:main]`` section, add two new variables
called `certfile` and `keyfile`.
.. code-block:: ini
[server:main]
host = 127.0.0.1
port = 10002
use = egg:gunicorn#main
workers = 1
threads = 1
proc_name = RhodeCodeEnterprise
worker_class = sync
max_requests = 1000
timeout = 3600
# adding ssl support
certfile = /home/ssl/my_server_com.pem
keyfile = /home/ssl/my_server_com.key
4. Save your changes.
5. Restart your |RCE| instance, using the following command:
.. code-block:: bash
$ rccontrol restart enterprise-1
After this is enabled you can *only* access your instances via https://
protocol. Check out more docs here `Gunicorn SSL Docs`_
.. note::
This change only can be applied to |RCE|. VCSServer doesn't support SSL
and should be only used with http protocol. Because only |RCE| is available
externally all communication will still be over SSL even without VCSServer
SSL enabled.
.. _Gunicorn SSL Docs: http://docs.gunicorn.org/en/stable/settings.html#ssl

359
docs/admin/system_admin/indexing.rst Normal file
View file

@ -0,0 +1,359 @@
.. _indexing-ref:
Full-text Search
----------------
RhodeCode provides a full text search capabilities to search inside file content,
commit message, and file paths. Indexing is not enabled by default and to use
full text search building an index is a pre-requisite.
By default RhodeCode is configured to use `Whoosh`_ to index |repos| and
provide full-text search. `Whoosh`_ works well for a small amount of data and
shouldn't be used in case of large code-bases and lots of repositories.
|RCE| also provides support for `ElasticSearch 6`_ as a backend more for advanced
and scalable search. See :ref:`enable-elasticsearch` for details.
Indexing
^^^^^^^^
To run the indexer you need to have an |authtoken| with admin rights to all |repos|.
To index repositories stored in RhodeCode, you have the option to set the indexer up in a
number of ways, for example:
* Call the indexer via a cron job. We recommend running this once at night.
In case you need everything indexed immediately it's possible to index few
times during the day. Indexer has a special locking mechanism that won't allow
two instances of indexer running at once. It's safe to run it even every 1hr.
* Set the indexer to infinitely loop and reindex as soon as it has run its previous cycle.
* Hook the indexer up with your CI server to reindex after each push.
The indexer works by indexing new commits added since the last run, and comparing
file changes to index only new or modified files.
If you wish to build a brand new index from scratch each time, use the ``force``
option in the configuration file, or run it with --force flag.
.. important::
You need to have |RCT| installed, see :ref:`install-tools`. Since |RCE|
3.5.0 they are installed by default and available with community/enterprise installations.
To set up indexing, use the following steps:
1. :ref:`config-rhoderc`, if running tools remotely.
2. :ref:`run-index`
3. :ref:`set-index`
4. :ref:`advanced-indexing`
.. _config-rhoderc:
Configure the ``.rhoderc`` File
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. note::
Optionally it's possible to use indexer without the ``.rhoderc``. Simply instead of
executing with `--instance-name=enterprise-1` execute providing the host and token
directly: `--api-host=http://127.0.0.1:10000 --api-key=<auth-token-goes-here>`
|RCT| uses the :file:`/home/{user}/.rhoderc` file for connection details
to |RCE| instances. If this file is not automatically created,
you can configure it using the following example. You need to configure the
details for each instance you want to index.
.. code-block:: bash
# Check the instance details
# of the instance you want to index
$ rccontrol status
- NAME: enterprise-1
- STATUS: RUNNING
- TYPE: Enterprise
- VERSION: 4.1.0
- URL: http://127.0.0.1:10003
To get your API Token, on the |RCE| interface go to
:menuselection:`username --> My Account --> Auth tokens`
.. code-block:: ini
# Configure .rhoderc with matching details
# This allows the indexer to connect to the instance
[instance:enterprise-1]
api_host = http://127.0.0.1:10000
api_key = <auth token goes here>
.. _run-index:
Run the Indexer
^^^^^^^^^^^^^^^
Run the indexer using the following command, and specify the instance you want to index:
.. code-block:: bash
# Using default simples indexing of all repositories
$ /home/user/.rccontrol/enterprise-1/profile/bin/rhodecode-index \
--instance-name=enterprise-1
# Using a custom mapping file with indexing rules, and using elasticsearch 6 backend
$ /home/user/.rccontrol/enterprise-1/profile/bin/rhodecode-index \
--instance-name=enterprise-1 \
--mapping=/home/user/.rccontrol/enterprise-1/search_mapping.ini \
--es-version=6 --engine-location=http://elasticsearch-host:9200
# Using a custom mapping file and invocation without ``.rhoderc``
$ /home/user/.rccontrol/enterprise-1/profile/bin/rhodecode-index \
--api-host=http://rhodecodecode.myserver.com --api-key=xxxxx \
--mapping=/home/user/.rccontrol/enterprise-1/search_mapping.ini
# From inside a virtualev on your local machine or CI server.
(venv)$ rhodecode-index --instance-name=enterprise-1
.. note::
In case of often indexing the index may become fragmented. Most often a result of that
is error about `too many open files`. To fix this indexer needs to be executed with
--optimize flag. E.g `rhodecode-index --instance-name=enterprise-1 --optimize`
This should be executed regularly, once a week is recommended.
.. _set-index:
Schedule the Indexer
^^^^^^^^^^^^^^^^^^^^
To schedule the indexer, configure the crontab file to run the indexer inside
your |RCT| virtualenv using the following steps.
1. Open the crontab file, using ``crontab -e``.
2. Add the indexer to the crontab, and schedule it to run as regularly as you
wish.
3. Save the file.
.. code-block:: bash
$ crontab -e
# The virtualenv can be called using its full path, so for example you can
# put this example into the crontab
# Run the indexer daily at 4am using the default mapping settings
* 4 * * * /home/ubuntu/.virtualenv/rhodecode-venv/bin/rhodecode-index \
--instance-name=enterprise-1
# Run the indexer every Sunday at 3am using default mapping
* 3 * * 0 /home/ubuntu/.virtualenv/rhodecode-venv/bin/rhodecode-index \
--instance-name=enterprise-1
# Run the indexer every 15 minutes
# using a specially configured mapping file
*/15 * * * * ~/.rccontrol/enterprise-4/profile/bin/rhodecode-index \
--instance-name=enterprise-4 \
--mapping=/home/user/.rccontrol/enterprise-4/search_mapping.ini
.. _advanced-indexing:
Advanced Indexing
^^^^^^^^^^^^^^^^^
Force Re-Indexing single repository
+++++++++++++++++++++++++++++++++++
Often it's required to re-index whole repository because of some repository changes,
or to remove some indexed secrets, or files. There's a special `--repo-name=` flag
for the indexer that limits execution to a single repository. For example to force-reindex
single repository such call can be made::
rhodecode-index --instance-name=enterprise-1 --force --repo-name=rhodecode-vcsserver
Removing repositories from index
++++++++++++++++++++++++++++++++
The indexer automatically removes renamed repositories and builds index for new names.
In the same way if a listed repository in mapping.ini is not reported existing by the
server it's removed from the index.
In case that you wish to remove indexed repository manually such call would allow that::
rhodecode-index --instance-name=enterprise-1 --remove-only --repo-name=rhodecode-vcsserver
Using search_mapping.ini file for advanced index rules
++++++++++++++++++++++++++++++++++++++++++++++++++++++
By default rhodecode-index runs for all repositories, all files with parsing limits
defined by the CLI default arguments. You can change those limits by calling with
different flags such as `--max-filesize=2048kb` or `--repo-limit=10`
For more advanced execution logic it's possible to use a configuration file that
would define detailed rules which repositories and how should be indexed.
|RCT| provides an example index configuration file called :file:`search_mapping.ini`.
This file is created by default during installation and is located at:
* :file:`/home/{user}/.rccontrol/{instance-id}/search_mapping.ini`, using default |RCT|.
* :file:`~/venv/lib/python2.7/site-packages/rhodecode_tools/templates/mapping.ini`,
when using ``virtualenv``.
.. note::
If you need to create the :file:`search_mapping.ini` file manually, use the |RCT|
``rhodecode-index --create-mapping path/to/search_mapping.ini`` API call.
For details, see the :ref:`tools-cli` section.
To Run the indexer with mapping file provide it using `--mapping` flag::
rhodecode-index --instance-name=enterprise-1 --mapping=/my/path/search_mapping.ini
Here's a detailed example of using :file:`search_mapping.ini` file.
.. code-block:: ini
[__DEFAULT__]
; Create index on commits data, and files data in this order. Available options
; are `commits`, `files`
index_types = commits,files
; Commit fetch limit. In what amount of chunks commits should be fetched
; via api and parsed. This allows server to transfer smaller chunks and be less loaded
commit_fetch_limit = 1000
; Commit process limit. Limit the number of commits indexer should fetch, and
; store inside the full text search index. eg. if repo has 2000 commits, and
; limit is 1000, on the first run it will process commits 0-1000 and on the
; second 1000-2000 commits. Help reduce memory usage, default is 50000
; (set -1 for unlimited)
commit_process_limit = 20000
; Limit of how many repositories each run can process, default is -1 (unlimited)
; in case of 1000s of repositories it's better to execute in chunks to not overload
; the server.
repo_limit = -1
; Default patterns for indexing files and content of files. Binary files
; are skipped by default.
; Add to index those comma separated files; globs syntax
; e.g index_files = *.py, *.c, *.h, *.js
index_files = *,
; Do not add to index those comma separated files, this excludes
; both search by name and content; globs syntax
; e.g index_files = *.key, *.sql, *.xml, *.pem, *.crt
skip_files = ,
; Add to index content of those comma separated files; globs syntax
; e.g index_files = *.h, *.obj
index_files_content = *,
; Do not add to index content of those comma separated files; globs syntax
; Binary files are not indexed by default.
; e.g index_files = *.min.js, *.xml, *.dump, *.log, *.dump
skip_files_content = ,
; Force rebuilding an index from scratch. Each repository will be rebuild from
; scratch with a global flag. Use --repo-name=NAME --force to rebuild single repo
force = false
; maximum file size that indexer will use, files above that limit are not going
; to have they content indexed.
; Possible options are KB (kilobytes), MB (megabytes), eg 1MB or 1024KB
max_filesize = 10MB
[__INDEX_RULES__]
; Ordered match rules for repositories. A list of all repositories will be fetched
; using API and this list will be filtered using those rules.
; Syntax for entry: `glob_pattern_OR_full_repo_name = 0 OR 1` where 0=exclude, 1=include
; When this ordered list is traversed first match will return the include/exclude marker
; For example:
; upstream/binary_repo = 0
; upstream/subrepo/xml_files = 0
; upstream/* = 1
; special-repo = 1
; * = 0
; This will index all repositories under upstream/*, but skip upstream/binary_repo
; and upstream/sub_repo/xml_files, last * = 0 means skip all other matches
; == EXPLICIT REPOSITORY INDEXING ==
; If defined this will skip using __INDEX_RULES__, and will not use API to fetch
; list of repositories, it will explicitly take names defined with [NAME] format and
; try to build the index, to build index just for repo_name_1 and special-repo use:
; [repo_name_1]
; [special-repo]
; == PER REPOSITORY CONFIGURATION ==
; This allows overriding the global configuration per repository.
; example to set specific file limit, and skip certain files for repository special-repo
; the CLI flags doesn't override the conf settings.
; [conf:special-repo]
; max_filesize = 5mb
; skip_files = *.xml, *.sql
In case of 1000s of repositories it can be tricky to write the include/exclude rules at first.
There's a special flag to test the mapping file rules and list repositories that would
be indexed. Run the indexer with `--show-matched-repos` to list only the
match repositories defined in .ini file rules::
rhodecode-index --instance-name=enterprise-1 --show-matched-repos --mapping=/my/path/search_mapping.ini
.. _enable-elasticsearch:
Enabling ElasticSearch
^^^^^^^^^^^^^^^^^^^^^^
ElasticSearch is available in EE edition only. It provides much scalable and more advanced
search capabilities. While Whoosh is fine for upto 1-2GB of data, beyond that amount it
starts slowing down, and can cause other problems.
New ElasticSearch 6 also provides much more advanced query language.
It allows advanced filtering by file paths, extensions, use OR statements, ranges etc.
Please check query language examples in the search field for some advanced query language usage.
1. Open the :file:`rhodecode.ini` file for the instance you wish to edit. The
default location is
:file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
2. Find the search configuration section:
.. code-block:: ini
###################################
## SEARCH INDEXING CONFIGURATION ##
###################################
search.module = rhodecode.lib.index.whoosh
search.location = %(here)s/data/index
and change it to:
.. code-block:: ini
search.module = rc_elasticsearch
search.location = http://localhost:9200
## specify Elastic Search version, 6 for latest or 2 for legacy
search.es_version = 6
where ``search.location`` points to the ElasticSearch server
by default running on port 9200.
Index invocation also needs change. Please provide --es-version= and
--engine-location= parameters to define ElasticSearch server location and it's version.
For example::
rhodecode-index --instace-name=enterprise-1 --es-version=6 --engine-location=http://localhost:9200
.. _Whoosh: https://pypi.python.org/pypi/Whoosh/
.. _ElasticSearch 6: https://www.elastic.co/

14
docs/admin/system_admin/nginx-config.rst Normal file
View file

@ -0,0 +1,14 @@
.. _nginx-ws-ref:
Nginx HTTP Server Configuration
-------------------------------
To set up your Nginx Web Server for optimal performance and security, use
the information in the following sections.
.. toctree::
nginx/nginx-config-example
nginx/nginx-diffie-hellman
nginx/nginx-proxy-conf
nginx/nginx-url-prefix

192
docs/admin/system_admin/nginx/nginx-config-example.rst Normal file
View file

@ -0,0 +1,192 @@
Nginx Configuration Example
---------------------------
Use the following example to configure Nginx as a your web server.
.. code-block:: nginx
## Rate limiter for certain pages to prevent brute force attacks
limit_req_zone $binary_remote_addr zone=req_limit:10m rate=1r/s;
## cache zone
proxy_cache_path /etc/nginx/nginx_cache levels=1:2 use_temp_path=off keys_zone=cache_zone:10m inactive=720h max_size=10g;
## Custom log format
log_format log_custom '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'$request_time $upstream_response_time $pipe';
## Define one or more upstreams (local RhodeCode instance) to connect to
upstream rc {
# Url to running RhodeCode instance.
# This is shown as `- URL: <host>` in output from rccontrol status.
server 127.0.0.1:10002;
# add more instances for load balancing
# server 127.0.0.1:10003;
# server 127.0.0.1:10004;
}
## HTTP to HTTPS rewrite
server {
listen 80;
server_name rhodecode.myserver.com;
if ($http_host = rhodecode.myserver.com) {
rewrite (.*) https://rhodecode.myserver.com$1 permanent;
}
}
## Optional gist alias server, for serving nicer GIST urls.
server {
listen 443;
server_name gist.myserver.com;
access_log /var/log/nginx/gist.access.log log_custom;
error_log /var/log/nginx/gist.error.log;
ssl on;
ssl_certificate gist.rhodecode.myserver.com.crt;
ssl_certificate_key gist.rhodecode.myserver.com.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
## Strict http prevents from https -> http downgrade
add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
## Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
#ssl_dhparam /etc/nginx/ssl/dhparam.pem;
rewrite ^/(.+)$ https://rhodecode.myserver.com/_admin/gists/$1;
rewrite (.*) https://rhodecode.myserver.com/_admin/gists;
}
## MAIN SSL enabled server
server {
listen 443 ssl http2;
server_name rhodecode.myserver.com;
access_log /var/log/nginx/rhodecode.access.log log_custom;
error_log /var/log/nginx/rhodecode.error.log;
ssl_certificate rhodecode.myserver.com.crt;
ssl_certificate_key rhodecode.myserver.com.key;
# enable session resumption to improve https performance
# http://vincent.bernat.im/en/blog/2011-ssl-session-reuse-rfc5077.html
ssl_session_cache shared:SSL:50m;
ssl_session_timeout 5m;
## Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
#ssl_dhparam /etc/nginx/ssl/dhparam.pem;
# enables server-side protection from BEAST attacks
# http://blog.ivanristic.com/2013/09/is-beast-still-a-threat.html
ssl_prefer_server_ciphers on;
# disable SSLv3(enabled by default since nginx 0.8.19) since it's less secure then TLS http://en.wikipedia.org/wiki/Secure_Sockets_Layer#SSL_3.0
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
# ciphers chosen for forward secrecy and compatibility
# http://blog.ivanristic.com/2013/08/configuring-apache-nginx-and-openssl-for-forward-secrecy.html
ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
client_body_buffer_size 128k;
# maximum number and size of buffers for large headers to read from client request
large_client_header_buffers 16 256k;
## uncomment to serve static files by Nginx, recommended for performance
# location /_static/rhodecode {
# gzip on;
# gzip_min_length 500;
# gzip_proxied any;
# gzip_comp_level 4;
# gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
# gzip_vary on;
# gzip_disable "msie6";
# alias /path/to/.rccontrol/community-1/static;
# alias /path/to/.rccontrol/enterprise-1/static;
# }
## channelstream location handler, if channelstream live chat and notifications
## are enable this will proxy the requests to channelstream websocket server
location /_channelstream {
rewrite /_channelstream/(.*) /$1 break;
gzip off;
tcp_nodelay off;
proxy_connect_timeout 10;
proxy_send_timeout 10m;
proxy_read_timeout 10m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://127.0.0.1:9800;
}
## rate limit this endpoint to prevent login page brute-force attacks
location /_admin/login {
limit_req zone=req_limit burst=10 nodelay;
try_files $uri @rhodecode_http;
}
## Special Cache for file store, make sure you enable this intentionally as
## it could bypass upload files permissions
# location /_file_store/download {
#
# proxy_cache cache_zone;
# # ignore Set-Cookie
# proxy_ignore_headers Set-Cookie;
# proxy_ignore_headers Cookie;
#
# proxy_cache_key $host$uri$is_args$args;
# proxy_cache_methods GET;
#
# proxy_cache_bypass $http_cache_control;
# proxy_cache_valid 200 302 720h;
#
# proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
#
# # returns cache status in headers
# add_header X-Proxy-Cache $upstream_cache_status;
# add_header Cache-Control "public";
#
# proxy_cache_lock on;
# proxy_cache_lock_age 5m;
#
# proxy_pass http://rc;
#
# }
location / {
try_files $uri @rhodecode_http;
}
location @rhodecode_http {
# example of proxy.conf can be found in our docs.
include /etc/nginx/proxy.conf;
proxy_pass http://rc;
}
## Custom 502 error page.
## Will be displayed while RhodeCode server is turned off
error_page 502 /502.html;
location = /502.html {
#root /path/to/.rccontrol/community-1/static;
root /path/to/.rccontrol/enterprise-1/static;
}
}

34
docs/admin/system_admin/nginx/nginx-diffie-hellman.rst Normal file
View file

@ -0,0 +1,34 @@
.. _dh-nginx:
Diffie-Hellman Security
-----------------------
To secure your web server, the `Guide to Deploying Diffie-Hellman for TLS`_
contains important information worth reading. This link contains a good
`nginx secure configuration`_ example. The documentation below also contains
good security settings with some additional |RCE| specific examples.
To secure your deployment of Diffie-Hellman, configure the following:
* Generate a strong Diffie-hellman group, 2048-bit or stronger.
.. code-block:: bash
# to generate your dhparam.pem file, run in the terminal
openssl dhparam -out /etc/nginx/ssl/dhparam.pem 2048
* Configure your server to only use modern, secure cipher suites in the
virtual hosts configuration file.
.. code-block:: nginx
# Set the TLS protocols and to only use modern, secure cipher suites.
ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
# Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
ssl_dhparam /etc/nginx/ssl/dhparam.pem;
.. _Guide to Deploying Diffie-Hellman for TLS: https://weakdh.org/sysadmin.html
.. _nginx secure configuration: https://gist.github.com/plentz/6737338

47
docs/admin/system_admin/nginx/nginx-proxy-conf.rst Normal file
View file

@ -0,0 +1,47 @@
.. _nginx-proxy-conf:
Nginx Proxy Config
------------------
Set the following properties in your ``/etc/nginx/proxy.conf`` so it does not
timeout during large pushes.
.. code-block:: nginx
proxy_redirect off;
proxy_set_header Host $http_host;
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
gzip off;
# Don't buffer requests in NGINX stream them using chunked-encoding
proxy_buffering off;
## This is also required for later GIT to use streaming.
## Works only for Nginx 1.7.11 and newer
proxy_request_buffering off;
proxy_http_version 1.1;
## Set this to a larger number if you experience timeouts
## or 413 Request Entity Too Large, 10GB is enough for most cases
client_max_body_size 10240m;
## needed for container auth
# proxy_set_header REMOTE_USER $remote_user;
# proxy_set_header X-Forwarded-User $remote_user;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Proxy-host $proxy_host;
proxy_connect_timeout 7200;
proxy_send_timeout 7200;
proxy_read_timeout 7200;
proxy_buffers 8 32k;
add_header X-Frame-Options SAMEORIGIN;
add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";

36
docs/admin/system_admin/nginx/nginx-url-prefix.rst Normal file
View file

@ -0,0 +1,36 @@
.. _nginx_url-pre:
Nginx URL Prefix Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Use the following example to configure Nginx to use a URL prefix.
.. code-block:: nginx
location /foo {
rewrite /foo(.*) /$1 break;
proxy_pass http://localhost:3200;
proxy_redirect off;
proxy_set_header Host $host;
}
In addition to the Nginx configuration you will need to add the following
lines (if they not exist) into the ``rhodecode.ini`` file.
* Above ``[app:main]`` section of the ``rhodecode.ini`` file add the
following section if it doesn't exist yet.
.. code-block:: ini
[filter:proxy-prefix]
use = egg:PasteDeploy#prefix
prefix = /<someprefix> # Change <someprefix> into your chosen prefix
* In the the ``[app:main]`` section of your ``rhodecode.ini`` file add the
following line.
.. code-block:: ini
filter-with = proxy-prefix

151
docs/admin/system_admin/reset-information.rst Normal file
View file

@ -0,0 +1,151 @@
.. _rhodecode-reset-ref:
Settings Management
-------------------
All |RCE| settings can be set from the user interface, but in the event that
it somehow becomes unavailable you can use ``ishell`` inside your |RCE|
``virtualenv`` to carry out emergency measures.
.. warning::
Logging into the |RCE| database with ``iShell`` should only be done by an
experienced and knowledgeable database administrator.
Reset Admin Account Privileges
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you accidentally remove your admin privileges from the admin account you
can restore them using ``ishell``. Use the following example to reset your
account permissions.
.. code-block:: bash
# Open iShell from the terminal
$ rccontrol ishell enterprise-1
.. code-block:: mysql
# Use this example to change user permissions
In [1]: adminuser = User.get_by_username('username')
In [2]: adminuser.admin = True
In [3]: Session().add(adminuser);Session().commit()
In [4]: exit()
Set to read global ``.hgrc`` file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By default, |RCE| does not read global ``hgrc`` files in
``/etc/mercurial/hgrc`` or ``/etc/mercurial/hgrc.d`` because it
can lead to issues. This is set in the ``rhodecode_ui`` table for which
there is no UI. If you need to edit this you can
manually change the settings using SQL statements with ``ishell``. Use the
following example to make changes to this table.
.. code-block:: bash
# Open iShell from the terminal
$ rccontrol ishell enterprise-1
.. code-block:: mysql
# Use this example to enable global .hgrc access
In [1]: new_option = RhodeCodeUi()
In [2]: new_option.ui_section='web'
In [3]: new_option.ui_key='allow_push'
In [4]: new_option.ui_value='*'
In [5]: Session().add(new_option);Session().commit()
In [6]: exit()
Manually Reset Password
^^^^^^^^^^^^^^^^^^^^^^^
If you need to manually reset a user password, use the following steps.
1. Navigate to your |RCE| install location.
2. Run the interactive ``ishell`` prompt.
3. Set a new password.
Use the following code example to carry out these steps.
.. code-block:: bash
# starts the ishell interactive prompt
$ rccontrol ishell enterprise-1
.. code-block:: mysql
In [1]: from rhodecode.lib.auth import generate_auth_token
In [2]: from rhodecode.lib.auth import get_crypt_password
# Enter the user name whose password you wish to change
In [3]: my_user = 'USERNAME'
In [4]: u = User.get_by_username(my_user)
# If this fails then the user does not exist
In [5]: u.auth_token = generate_auth_token(my_user)
# Set the new password
In [6]: u.password = get_crypt_password('PASSWORD')
In [7]: Session().add(u);Session().commit()
In [8]: exit()
Change user details
^^^^^^^^^^^^^^^^^^^
If you need to manually change some of users details, use the following steps.
1. Navigate to your |RCE| install location.
2. Run the interactive ``ishell`` prompt.
3. Set a new arguments for users.
Use the following code example to carry out these steps.
.. code-block:: bash
# starts the ishell interactive prompt
$ rccontrol ishell enterprise-1
.. code-block:: mysql
# Use this example to change email and username of LDAP user
In [1]: my_user = User.get_by_username('some_username')
In [2]: my_user.email = 'new_email@foobar.com'
In [3]: my_user.username = 'SomeUser'
In [4]: Session().add(my_user);Session().commit()
In [5]: exit()
Change user login type
^^^^^^^^^^^^^^^^^^^^^^
Sometimes it's required to change account type from RhodeCode to LDAP or
other external authentication type.
If you need to manually change the method of login, use the following steps.
1. Navigate to your |RCE| install location.
2. Run the interactive ``ishell`` prompt.
3. Set a new arguments for users.
Use the following code example to carry out these steps.
Available values for new_extern_type can be found when browsing available
authentication types in RhodeCode admin interface for authentication.
Use the text which is shown after '#' sign, eg.
` LDAP (egg:rhodecode-enterprise-ce#ldap)` it's type is 'ldap'
.. code-block:: bash
# starts the ishell interactive prompt
$ rccontrol ishell enterprise-1
.. code-block:: mysql
# Use this example to change users from authentication
# using rhodecode internal to ldap
In [1]: new_extern_type = 'ldap'
In [2]: my_user = User.get_by_username('some_username')
In [3]: my_user.extern_type = new_extern_type
In [4]: my_user.extern_name = new_extern_type
In [5]: Session().add(my_user);Session().commit()
In [6]: exit()

207
docs/admin/system_admin/svn-http.rst Normal file
View file

@ -0,0 +1,207 @@
.. _svn-http:
|svn| With Write Over HTTP
^^^^^^^^^^^^^^^^^^^^^^^^^^
To use |svn| with read/write support over the |svn| HTTP protocol, you have to
configure the HTTP |svn| backend.
Prerequisites
=============
- Enable HTTP support inside the admin VCS settings on your |RCE| instance
- You need to install the following tools on the machine that is running an
instance of |RCE|:
``Apache HTTP Server`` and ``mod_dav_svn``.
.. tip::
We recommend using Wandisco repositories which provide latest SVN versions
for most platforms. If you skip this version you'll have to ensure the Client version
is compatible with installed SVN version which might differ depending on the operating system.
Here is an example how to add the Wandisco repositories for Ubuntu.
.. code-block:: bash
$ sudo sh -c 'echo "deb http://opensource.wandisco.com/ubuntu `lsb_release -cs` svn110" >> /etc/apt/sources.list.d/subversion110.list'
$ sudo wget -q http://opensource.wandisco.com/wandisco-debian-new.gpg -O- | sudo apt-key add -
$ sudo apt-get update
Here is an example how to add the Wandisco repositories for Centos/Redhat. Using
a yum config
.. code-block:: bash
[wandisco-Git]
name=CentOS-6 - Wandisco Git
baseurl=http://opensource.wandisco.com/centos/6/git/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://opensource.wandisco.com/RPM-GPG-KEY-WANdisco
Example installation of required components for Ubuntu platform:
.. code-block:: bash
$ sudo apt-get install apache2
$ sudo apt-get install libapache2-svn
Once installed you need to enable ``dav_svn`` on Ubuntu:
.. code-block:: bash
$ sudo a2enmod dav_svn
$ sudo a2enmod headers
$ sudo a2enmod authn_anon
Example installation of required components for RedHat/CentOS platform:
.. code-block:: bash
$ sudo yum install httpd
$ sudo yum install subversion mod_dav_svn
Once installed you need to enable ``dav_svn`` on RedHat/CentOS:
.. code-block:: bash
sudo vi /etc/httpd/conf.modules.d/10-subversion.conf
## The file should read:
LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule headers_module modules/mod_headers.so
LoadModule authn_anon_module modules/mod_authn_anon.so
.. tip::
To check the installed mod_dav_svn module version, you can use such command.
`strings /usr/lib/apache2/modules/mod_dav_svn.so | grep 'Powered by'`
Configuring Apache Setup
========================
.. tip::
It is recommended to run Apache on a port other than 80, due to possible
conflicts with other HTTP servers like nginx. To do this, set the
``Listen`` parameter in the ``/etc/apache2/ports.conf`` file, for example
``Listen 8090``.
.. warning::
Make sure your Apache instance which runs the mod_dav_svn module is
only accessible by |RCE|. Otherwise everyone is able to browse
the repositories or run subversion operations (checkout/commit/etc.).
It is also recommended to run apache as the same user as |RCE|, otherwise
permission issues could occur. To do this edit the ``/etc/apache2/envvars``
.. code-block:: apache
export APACHE_RUN_USER=rhodecode
export APACHE_RUN_GROUP=rhodecode
1. To configure Apache, create and edit a virtual hosts file, for example
:file:`/etc/apache2/sites-enabled/default.conf`. Below is an example
how to use one with auto-generated config ```mod_dav_svn.conf```
from configured |RCE| instance.
.. code-block:: apache
<VirtualHost *:8090>
ServerAdmin rhodecode-admin@localhost
DocumentRoot /var/www/html
ErrorLog ${'${APACHE_LOG_DIR}'}/error.log
CustomLog ${'${APACHE_LOG_DIR}'}/access.log combined
LogLevel info
# allows custom host names, prevents 400 errors on checkout
HttpProtocolOptions Unsafe
Include /home/user/.rccontrol/enterprise-1/mod_dav_svn.conf
</VirtualHost>
2. Go to the :menuselection:`Admin --> Settings --> VCS` page, and
enable :guilabel:`Proxy Subversion HTTP requests`, and specify the
:guilabel:`Subversion HTTP Server URL`.
3. Open the |RCE| configuration file,
:file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
4. Add the following configuration option in the ``[app:main]``
section if you don't have it yet.
This enables mapping of the created |RCE| repo groups into special
|svn| paths. Each time a new repository group is created, the system will
update the template file and create new mapping. Apache web server needs to
be reloaded to pick up the changes on this file.
To do this, simply configure `svn.proxy.reload_cmd` inside the .ini file.
Example configuration:
.. code-block:: ini
############################################################
### Subversion proxy support (mod_dav_svn) ###
### Maps RhodeCode repo groups into SVN paths for Apache ###
############################################################
## Enable or disable the config file generation.
svn.proxy.generate_config = true
## Generate config file with `SVNListParentPath` set to `On`.
svn.proxy.list_parent_path = true
## Set location and file name of generated config file.
svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
## Used as a prefix to the <Location> block in the generated config file.
## In most cases it should be set to `/`.
svn.proxy.location_root = /
## Command to reload the mod dav svn configuration on change.
## Example: `/etc/init.d/apache2 reload`
svn.proxy.reload_cmd = /etc/init.d/apache2 reload
## If the timeout expires before the reload command finishes, the command will
## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
#svn.proxy.reload_timeout = 10
This would create a special template file called ```mod_dav_svn.conf```. We
used that file path in the apache config above inside the Include statement.
It's also possible to manually generate the config from the
:menuselection:`Admin --> Settings --> VCS` page by clicking a
`Generate Apache Config` button.
5. Now only things left is to enable svn support, and generate the initial
configuration.
- Select `Proxy subversion HTTP requests` checkbox
- Enter http://localhost:8090 into `Subversion HTTP Server URL`
- Click the `Generate Apache Config` button.
This config will be automatically re-generated once an user-groups is added
to properly map the additional paths generated.
Using |svn|
===========
Once |svn| has been enabled on your instance, you can use it with the
following examples. For more |svn| information, see the `Subversion Red Book`_
.. code-block:: bash
# To clone a repository
svn checkout http://my-svn-server.example.com/my-svn-repo
# svn commit
svn commit
.. _Subversion Red Book: http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.ref.svn
.. _Ask Ubuntu: http://askubuntu.com/questions/162391/how-do-i-fix-my-locale-issue

80
docs/admin/system_admin/svn-path-permissions.rst Normal file
View file

@ -0,0 +1,80 @@
.. _svn-path-permissions:
|svn| Enabling Path Permissions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Because |RCEE| uses standard svn apache mod_svn we can take advantage of the
authz configuration to protect paths and branches.
Configuring RhodeCode
=====================
1. To configure path based permissions first we need to use a customized
mod_dav_svn.conf.
Open :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
And find `svn.proxy.config_template` setting. Now set a new path to read
the template from. For example:
.. code-block:: ini
svn.proxy.config_template = /home/ubuntu/rhodecode/custom_mod_dav_svn.conf.mako
2. Create the file as in example: `/home/ubuntu/rhodecode/custom_mod_dav_svn.conf.mako`
You can download one from:
`<https://code.rhodecode.com/rhodecode-enterprise-ce/files/default/rhodecode/apps/svn_support/templates/mod-dav-svn.conf.mako/>`_
3. Add (if not yet exists) a section `AuthzSVNReposRelativeAccessFile` in order
to read the path auth file.
Example modified config section enabling reading the authz file relative
to repository path. Means located in `/storage_dir/repo_name/conf/authz`
.. code-block:: text
# snip ...
# use specific SVN conf/authz file for each repository
AuthzSVNReposRelativeAccessFile authz
Allow from all
# snip ...
.. note::
The `AuthzSVNReposRelativeAccessFile` should go above the `Allow from all`
directive.
4. Restart RhodeCode, Go to
the :menuselection:`Admin --> Settings --> VCS` page, and
click :guilabel:`Generate Apache Config`.
This will now generate a new configuration with enabled changes to read
the authz file. You can verify if changes were made by checking the generated
mod_dav_svn.conf file which is included in your apache configuration.
5. Specify new rules in the repository authz configuration.
edit a file in :file:`repo_name/conf/authz`. For example, we specify that
only admin is allowed to push to develop branch
.. code-block:: ini
[/branches/develop]
* = r
admin = rw
For more example see:
`<https://svn.apache.org/repos/asf/subversion/trunk/subversion/mod_authz_svn/INSTALL/>`_
Those rules also work for paths, so not only branches but all different
paths inside the repository can be specified.
6. Reload Apache. If all is configured correctly it should not be allowed to
commit according to specified rules.

21
docs/admin/system_admin/tuning-rhodecode.rst Normal file
View file

@ -0,0 +1,21 @@
.. _rhodecode-tuning-ref:
Tuning |RCE|
============
To customize your |RCE| |version| installation for maximum performance you
may find some of the following methods useful.
.. toctree::
tuning/tuning-gunicorn
tuning/tuning-vcs-memory-cache
tuning/tuning-user-sessions-performance
tuning/tuning-increase-db-performance
tuning/tuning-scale-horizontally-cluster
tuning/tuning-mount-cache-memory
tuning/tuning-change-encoding
tuning/tuning-change-large-file-dir
tuning/tuning-change-lfs-dir
tuning/tuning-hg-auth-loop

22
docs/admin/system_admin/tuning/tuning-change-encoding.rst Normal file
View file

@ -0,0 +1,22 @@
.. _change-encoding:
Change Default Encoding
-----------------------
|RCE| uses ``utf8`` encoding by default. You can change the default encoding
in the :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file. To
change the default encoding used by |RCE|, set a new value for the
``default_encoding``.
.. code-block:: ini
# default encoding used to convert from and to unicode
# can be also a comma separated list of encoding in case of mixed
# encodings
default_encoding = utf8
.. note::
Changing the default encoding will affect many parts of your |RCE|
installation, including committers names,
file names, and the encoding of commit messages.

14
docs/admin/system_admin/tuning/tuning-change-large-file-dir.rst Normal file
View file

@ -0,0 +1,14 @@
.. _hg-lrg-loc:
|hg| Large Files Location
-------------------------
|RCE| manages |hg| larges files from the following default location
:file:`/home/{user}/repos/.cache/largefiles`. If you wish to change this, use
the following steps:
1. Open :menuselection:`Admin --> Settings --> VCS` as super-admin.
In section called `Mercurial Settings` you can change where the largefiles
objects should be stored.

14
docs/admin/system_admin/tuning/tuning-change-lfs-dir.rst Normal file
View file

@ -0,0 +1,14 @@
.. _git-lfs-loc:
|git| LFS storage Location
--------------------------
|RCE| manages |git| LFS files from the following default location
:file:`/home/{user}/repos/.cache/lfs_store`. If you wish to change this, use
the following steps:
1. Open :menuselection:`Admin --> Settings --> VCS` as super-admin.
In section called `Git Settings` you can change where the LFS
objects should be stored.

126
docs/admin/system_admin/tuning/tuning-gunicorn.rst Normal file
View file

@ -0,0 +1,126 @@
.. _increase-gunicorn:
Configure Gunicorn Workers
--------------------------
|RCE| comes with `Gunicorn`_ which is a Python WSGI HTTP Server for UNIX.
To improve |RCE| performance you can increase the number of `Gunicorn`_ workers.
This allows to handle more connections concurrently, and provide better
responsiveness and performance.
By default during installation |RCC| tries to detect how many CPUs are
available in the system, and set the number workers based on that information.
However sometimes it's better to manually set the number of workers.
To do this, use the following steps:
1. Open the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
2. In the ``[server:main]`` section, change the number of Gunicorn
``workers`` using the following default formula :math:`(2 * Cores) + 1`.
We however not recommend using more than 8-12 workers per server. It's better
to start using the :ref:`scale-horizontal-cluster` in case that performance
with 8-12 workers is not enough.
.. code-block:: ini
use = egg:gunicorn#main
## Sets the number of process workers. You must set `instance_id = *`
## when this option is set to more than one worker, recommended
## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
## The `instance_id = *` must be set in the [app:main] section below
workers = 4
## process name
proc_name = rhodecode
## type of worker class, one of sync, gevent
## recommended for bigger setup is using of of other than sync one
worker_class = sync
## The maximum number of simultaneous clients. Valid only for Gevent
#worker_connections = 10
## max number of requests that worker will handle before being gracefully
## restarted, could prevent memory leaks
max_requests = 1000
max_requests_jitter = 30
## amount of time a worker can spend with handling a request tuning-change-lfs-dir.before it
## gets killed and restarted. Set to 6hrs
timeout = 21600
3. In the ``[app:main]`` section, set the ``instance_id`` property to ``*``.
.. code-block:: ini
# In the [app:main] section
[app:main]
# You must set `instance_id = *`
instance_id = *
4. Change the VCSServer workers too. Open the
:file:`home/{user}/.rccontrol/{instance-id}/vcsserver.ini` file.
5. In the ``[server:main]`` section, increase the number of Gunicorn
``workers`` using the following formula :math:`(2 * Cores) + 1`.
.. code-block:: ini
## run with gunicorn --log-config vcsserver.ini --paste vcsserver.ini
use = egg:gunicorn#main
## Sets the number of process workers. Recommended
## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
workers = 4
## process name
proc_name = rhodecode_vcsserver
## type of worker class, currently `sync` is the only option allowed.
worker_class = sync
## The maximum number of simultaneous clients. Valid only for Gevent
#worker_connections = 10
## max number of requests that worker will handle before being gracefully
## restarted, could prevent memory leaks
max_requests = 1000
max_requests_jitter = 30
## amount of time a worker can spend with handling a request before it
## gets killed and restarted. Set to 6hrs
timeout = 21600
6. Save your changes.
7. Restart your |RCE| instances, using the following command:
.. code-block:: bash
$ rccontrol restart '*'
Gunicorn Gevent Backend
-----------------------
Gevent is an asynchronous worker type for Gunicorn. It allows accepting multiple
connections on a single `Gunicorn`_ worker. This means you can handle 100s
of concurrent clones, or API calls using just few workers. A setting called
`worker_connections` defines on how many connections each worker can
handle using `Gevent`.
To enable `Gevent` on |RCE| do the following:
1. Open the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
2. In the ``[server:main]`` section, change `worker_class` for Gunicorn.
.. code-block:: ini
## type of worker class, one of sync, gevent
## recommended for bigger setup is using of of other than sync one
worker_class = gevent
## The maximum number of simultaneous clients. Valid only for Gevent
worker_connections = 30
.. note::
`Gevent` is currently only supported for Enterprise/Community instances.
VCSServer doesn't yet support gevent.
.. _Gunicorn: http://gunicorn.org/

17
docs/admin/system_admin/tuning/tuning-hg-auth-loop.rst Normal file
View file

@ -0,0 +1,17 @@
.. _hg-auth-loop:
|hg| Authentication Tuning
--------------------------
When using external authentication tools such as LDAP with |hg|, a
password retry loop in |hg| can result in users being locked out due to too
many failed password attempts. To prevent this from happening, add the
following setting to your
:file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file, in the
``[app:main]`` section.
.. code-block:: ini
[app:main]
auth_ret_code_detection = true

8
docs/admin/system_admin/tuning/tuning-increase-db-performance.rst Normal file
View file

@ -0,0 +1,8 @@
.. _db-session-ref:
Database Performance
--------------------
For tuning PostgreSQL we recommend reading: http://www.revsys.com/writings/postgresql-performance.html
For tuning MySQL we recommend reading: http://www.tecmint.com/mysql-mariadb-performance-tuning-and-optimization/

51
docs/admin/system_admin/tuning/tuning-mount-cache-memory.rst Normal file
View file

@ -0,0 +1,51 @@
.. _data-mem:
Mount Cache Folders To Memory
-----------------------------
To increase the performance of folders containing cache data, you can mount
them to memory. The following folders specified in the :file:`rhodecode.ini`
file would benefit from this.
.. code-block:: ini
cache_dir = %(here)s/data
search.location = %(here)s/data/index
Use the following Ubuntu example to mount these to memory, or see your
particular |os| instructions. The expected performance benefit is
approximately 5%. You should ensure you allocate an adequate amount of memory
depending on your available resources.
.. code-block:: bash
# mount to memory with 2GB limit and 755 write permissions
mount -t tmpfs -o size=2G,mode=0755 tmpfs /home/user/.rccontrol/enterprise-1/data
mount -t tmpfs -o size=2G,mode=0755 tmpfs /home/user/.rccontrol/enterprise-1/data/index
.. _move-tmp:
In order to make this change permanent it's recommend to set it as /etc/fstab
entry.
.. code-block:: bash
tmpfs /home/user/.rccontrol/enterprise-1/data tmpfs nodev,nosuid,noexec,nodiratime,size=2G 0 0
Move ``tmp`` to TMPFS
---------------------
|RCE| components heavily use the :file:`/tmp` folder, so moving your
:file:`/tmp` folder into to a RAM-based TMPS can lead to a noticeable
performance boost.
.. code-block:: bash
# mount tmp to memory with 2GB limit and 1777 write permissions
mount -t tmpfs -o size=2G,mode=1777 tmpfs /tmp
For more information about TMPFS, see the documentation `here`_.
.. _here: https://wiki.archlinux.org/index.php/Tmpfs

395
docs/admin/system_admin/tuning/tuning-scale-horizontally-cluster.rst Normal file
View file

@ -0,0 +1,395 @@
.. _scale-horizontal-cluster:
Scale Horizontally / RhodeCode Cluster
--------------------------------------
|RCE| is built in a way it support horizontal scaling across multiple machines.
There are three main pre-requisites for that:
- Shared storage that each machine can access. Using NFS or other shared storage system.
- Shared DB connection across machines. Using `MySQL`/`PostgreSQL` that each node can access.
- |RCE| user sessions and caches need to use a shared storage (e.g `Redis`_/`Memcached`)
Horizontal scaling means adding more machines or workers into your pool of
resources. Horizontally scaling |RCE| gives a huge performance increase,
especially under large traffic scenarios with a high number of requests.
This is very beneficial when |RCE| is serving many users simultaneously,
or if continuous integration servers are automatically pulling and pushing code.
It also adds High-Availability to your running system.
Cluster Overview
^^^^^^^^^^^^^^^^
Below we'll present a configuration example that will use two separate nodes to serve
|RCE| in a load-balanced environment. The 3rd node will act as a shared storage/cache
and handle load-balancing. In addition 3rd node will be used as shared database instance.
This setup can be used both in Docker based configuration or with individual
physical/virtual machines. Using the 3rd node for Storage/Redis/PostgreSQL/Nginx is
optional. All those components can be installed on one of the two nodes used for |RCE|.
We'll use following naming for our nodes:
- `rc-node-1` (NFS, DB, Cache node)
- `rc-node-2` (Worker node1)
- `rc-node-3` (Worker node2)
Our shares NFS storage in the example is located on `/home/rcdev/storage` and
it's RW accessible on **each** node.
In this example we used certain recommended components, however many
of those can be replaced by other, in case your organization already uses them, for example:
- `MySQL`/`PostgreSQL`: Aren't replaceable and are the two only supported databases.
- `Nginx`_ on `rc-node-1` can be replaced by: `Hardware Load Balancer (F5)`, `Apache`_, `HA-Proxy` etc.
- `Nginx`_ on rc-node-2/3 acts as a reverse proxy and can be replaced by other HTTP server
acting as reverse proxy such as `Apache`_.
- `Redis`_ on `rc-node-1` can be replaced by: `Memcached`
Here's an overview what components should be installed/setup on each server in our example:
- **rc-node-1**:
- main storage acting as NFS host.
- `nginx` acting as a load-balancer.
- `postgresql-server` used for database and sessions.
- `redis-server` used for storing shared caches.
- optionally `rabbitmq-server` for `Celery` if used.
- optionally if `Celery` is used Enterprise/Community instance + VCSServer.
- optionally mailserver that can be shared by other instances.
- optionally channelstream server to handle live communication for all instances.
- **rc-node-2/3**:
- `nginx` acting as a reverse proxy to handle requests to |RCE|.
- 1x RhodeCode Enterprise/Community instance.
- 1x VCSServer instance.
- optionally for testing connection: postgresql-client, redis-client (redis-tools).
Before we start here are few assumptions that should be fulfilled:
- make sure each node can access each other.
- make sure `Redis`_/`MySQL`/`PostgreSQL`/`RabbitMQ`_ are running on `rc-node-1`
- make sure both `rc-node-2`/`3` can access NFS storage with RW access
- make sure rc-node-2/3 can access `Redis`_/`PostgreSQL`, `MySQL` database on `rc-node-1`.
- make sure `Redis`_/Database/`RabbitMQ`_ are password protected and accessible only from rc-node-2/3.
Setup rc-node-2/3
^^^^^^^^^^^^^^^^^
Initially before `rc-node-1` we'll configure both nodes 2 and 3 to operate as standalone
nodes with their own hostnames. Use a default installation settings, and use
the default local addresses (127.0.0.1) to configure VCSServer and Community/Enterprise instances.
All external connectivity will be handled by the reverse proxy (`Nginx`_ in our example).
This way we can ensure each individual host works,
accepts connections, or do some operations explicitly on chosen node.
In addition this would allow use to explicitly direct certain traffic to a node, e.g
CI server will only call directly `rc-node-3`. This should be done similar to normal
installation so check out `Nginx`_/`Apache`_ configuration example to configure each host.
Each one should already connect to shared database during installation.
1) Assuming our final url will be http://rc-node-1, Configure `instances_id`, `app.base_url`
a) On **rc-node-2** find the following settings and edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
.. code-block:: ini
## required format is: *NAME-
instance_id = *rc-node-2-
app.base_url = http://rc-node-1
b) On **rc-node-3** find the following settings and edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
.. code-block:: ini
## required format is: *NAME-
instance_id = *rc-node-3-
app.base_url = http://rc-node-1
2) Configure `User Session` to use a shared database. Example config that should be
changed on both **rc-node-2** and **rc-node-3** .
Edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
.. code-block:: ini
####################################
### BEAKER SESSION ####
####################################
## Disable the default `file` sessions
#beaker.session.type = file
#beaker.session.data_dir = %(here)s/data/sessions
## use shared db based session, fast, and allows easy management over logged in users
beaker.session.type = ext:database
beaker.session.table_name = db_session
# use our rc-node-1 here
beaker.session.sa.url = postgresql://postgres:qweqwe@rc-node-1/rhodecode
beaker.session.sa.pool_recycle = 3600
beaker.session.sa.echo = false
In addition make sure both instances use the same `session.secret` so users have
persistent sessions across nodes. Please generate other one then in this example.
.. code-block:: ini
# use a unique generated long string
beaker.session.secret = 70e116cae2274656ba7265fd860aebbd
3) Configure stored cached/archive cache to our shared NFS `rc-node-1`
.. code-block:: ini
# note the `_` prefix that allows using a directory without
# remap and rescan checking for vcs inside it.
cache_dir = /home/rcdev/storage/_cache_dir/data
# note archive cache dir is disabled by default, however if you enable
# it also needs to be shared
#archive_cache_dir = /home/rcdev/storage/_tarball_cache_dir
4) Use shared exception store. Example config that should be
changed on both **rc-node-2** and **rc-node-3**, and also for VCSServer.
Edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` and
:file:`/home/{user}/.rccontrol/{vcsserver-instance-id}/vcsserver.ini`
and add/change following setting.
.. code-block:: ini
exception_tracker.store_path = /home/rcdev/storage/_exception_store_data
5) Change cache backends to use `Redis`_ based caches. Below full example config
that replaces default file-based cache to shared `Redis`_ with Distributed Lock.
.. code-block:: ini
#####################################
### DOGPILE CACHE ####
#####################################
## `cache_perms` cache settings for permission tree, auth TTL.
#rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
#rc_cache.cache_perms.expiration_time = 300
## alternative `cache_perms` redis backend with distributed lock
rc_cache.cache_perms.backend = dogpile.cache.rc.redis
rc_cache.cache_perms.expiration_time = 300
## redis_expiration_time needs to be greater then expiration_time
rc_cache.cache_perms.arguments.redis_expiration_time = 7200
rc_cache.cache_perms.arguments.socket_timeout = 30
rc_cache.cache_perms.arguments.host = rc-node-1
rc_cache.cache_perms.arguments.password = qweqwe
rc_cache.cache_perms.arguments.port = 6379
rc_cache.cache_perms.arguments.db = 0
rc_cache.cache_perms.arguments.distributed_lock = true
## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
#rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
#rc_cache.cache_repo.expiration_time = 2592000
## alternative `cache_repo` redis backend with distributed lock
rc_cache.cache_repo.backend = dogpile.cache.rc.redis
rc_cache.cache_repo.expiration_time = 2592000
## redis_expiration_time needs to be greater then expiration_time
rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
rc_cache.cache_repo.arguments.socket_timeout = 30
rc_cache.cache_repo.arguments.host = rc-node-1
rc_cache.cache_repo.arguments.password = qweqwe
rc_cache.cache_repo.arguments.port = 6379
rc_cache.cache_repo.arguments.db = 1
rc_cache.cache_repo.arguments.distributed_lock = true
## cache settings for SQL queries, this needs to use memory type backend
rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
rc_cache.sql_cache_short.expiration_time = 30
## `cache_repo_longterm` cache for repo object instances, this needs to use memory
## type backend as the objects kept are not pickle serializable
rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
## by default we use 96H, this is using invalidation on push anyway
rc_cache.cache_repo_longterm.expiration_time = 345600
## max items in LRU cache, reduce this number to save memory, and expire last used
## cached objects
rc_cache.cache_repo_longterm.max_size = 10000
6) Configure `Nginx`_ as reverse proxy on `rc-node-2/3`:
Minimal `Nginx`_ config used:
.. code-block:: nginx
## rate limiter for certain pages to prevent brute force attacks
limit_req_zone $binary_remote_addr zone=req_limit:10m rate=1r/s;
## custom log format
log_format log_custom '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'$request_time $upstream_response_time $pipe';
server {
listen 80;
server_name rc-node-2;
#server_name rc-node-3;
access_log /var/log/nginx/rhodecode.access.log log_custom;
error_log /var/log/nginx/rhodecode.error.log;
# example of proxy.conf can be found in our docs.
include /etc/nginx/proxy.conf;
## serve static files by Nginx, recommended for performance
location /_static/rhodecode {
gzip on;
gzip_min_length 500;
gzip_proxied any;
gzip_comp_level 4;
gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
gzip_vary on;
gzip_disable "msie6";
#alias /home/rcdev/.rccontrol/community-1/static;
alias /home/rcdev/.rccontrol/enterprise-1/static;
}
location /_admin/login {
limit_req zone=req_limit burst=10 nodelay;
try_files $uri @rhode;
}
location / {
try_files $uri @rhode;
}
location @rhode {
# Url to running RhodeCode instance.
# This is shown as `- URL: <host>` in output from rccontrol status.
proxy_pass http://127.0.0.1:10020;
}
## custom 502 error page. Will be displayed while RhodeCode server
## is turned off
error_page 502 /502.html;
location = /502.html {
#root /home/rcdev/.rccontrol/community-1/static;
root /home/rcdev/.rccontrol/enterprise-1/static;
}
}
7) Optional: Full text search, in case you use `Whoosh` full text search we also need a
shared storage for the index. In our example our NFS is mounted at `/home/rcdev/storage`
which represents out storage so we can use the following:
.. code-block:: ini
# note the `_` prefix that allows using a directory without
# remap and rescan checking for vcs inside it.
search.location = /home/rcdev/storage/_index_data/index
.. note::
If you use ElasticSearch it's by default shared, and simply running ES node is
by default cluster compatible.
8) Optional: If you intend to use mailing all instances need to use either a shared
mailing node, or each will use individual local mail agent. Simply put node-1/2/3
needs to use same mailing configuration.
Setup rc-node-1
^^^^^^^^^^^^^^^
Configure `Nginx`_ as Load Balancer to rc-node-2/3.
Minimal `Nginx`_ example below:
.. code-block:: nginx
## define rc-cluster which contains a pool of our instances to connect to
upstream rc-cluster {
# rc-node-2/3 are stored in /etc/hosts with correct IP addresses
server rc-node-2:80;
server rc-node-3:80;
}
server {
listen 80;
server_name rc-node-1;
location / {
proxy_pass http://rc-cluster;
}
}
.. note::
You should configure your load balancing accordingly. We recommend writing
load balancing rules that will separate regular user traffic from
automated process traffic like continuous servers or build bots. Sticky sessions
are not required.
Show which instance handles a request
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can easily check if load-balancing is working as expected. Visit our main node
`rc-node-1` URL which at that point should already handle incoming requests and balance
it across node-2/3.
Add a special GET param `?showrcid=1` to show current instance handling your request.
For example: visiting url `http://rc-node-1/?showrcid=1` will show, in the bottom
of the screen` cluster instance info.
e.g: `RhodeCode instance id: rc-node-3-rc-node-3-3246`
which is generated from::
<NODE_HOSTNAME>-<INSTANCE_ID>-<WORKER_PID>
Using Celery with cluster
^^^^^^^^^^^^^^^^^^^^^^^^^
If `Celery` is used we recommend setting also an instance of Enterprise/Community+VCSserver
on the node that is running `RabbitMQ`_. Those instances will be used to executed async
tasks on the `rc-node-1`. This is the most efficient setup. `Celery` usually
handles tasks such as sending emails, forking repositories, importing
repositories from external location etc. Using workers on instance that has
the direct access to disks used by NFS as well as email server gives noticeable
performance boost. Running local workers to the NFS storage results in faster
execution of forking large repositories or sending lots of emails.
Those instances need to be configured in the same way as for other nodes.
The instance in rc-node-1 can be added to the cluser, but we don't recommend doing it.
For best results let it be isolated to only executing `Celery` tasks in the cluster setup.
.. _Gunicorn: http://gunicorn.org/
.. _Whoosh: https://pypi.python.org/pypi/Whoosh/
.. _Elasticsearch: https://www.elastic.co/..
.. _RabbitMQ: http://www.rabbitmq.com/
.. _Nginx: http://nginx.io
.. _Apache: http://nginx.io
.. _Redis: http://redis.io

67
docs/admin/system_admin/tuning/tuning-user-sessions-performance.rst Normal file
View file

@ -0,0 +1,67 @@
.. _user-session-ref:
User Session Performance
------------------------
The default file-based sessions are only suitable for smaller setups, or
instances that doesn't have a lot of users or traffic.
They are set as default option because it's setup-free solution.
The most common issue of file based sessions are file limit errors which occur
if there are lots of session files.
Therefore, in a large scale deployment, to give better performance,
scalability, and maintainability we recommend switching from file-based
sessions to database-based user sessions or Redis based sessions.
To switch to database-based user sessions uncomment the following section in
your :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
.. code-block:: ini
## db based session, fast, and allows easy management over logged in users
beaker.session.type = ext:database
beaker.session.table_name = db_session
# use just one of the following according to the type of database
beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
# or
beaker.session.sa.url = mysql://root:secret@127.0.0.1/rhodecode
beaker.session.sa.pool_recycle = 3600
beaker.session.sa.echo = false
and make sure you comment out the file based sessions.
.. code-block:: ini
## types are file, ext:memcached, ext:database, and memory (default).
#beaker.session.type = file
#beaker.session.data_dir = %(here)s/data/sessions/data
The `table_name` will be automatically created on specified database if it isn't yet existing.
Database specified in the `beaker.session.sa.url` can be the same that RhodeCode
uses, or if required it can be a different one. We recommend to use the same database.
To switch to reds-based user sessions uncomment the following section in
your :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
.. code-block:: ini
## redis sessions
beaker.session.type = ext:redis
beaker.session.url = localhost:6379
and make sure you comment out the file based sessions.
.. code-block:: ini
## types are file, ext:memcached, ext:database, and memory (default).
#beaker.session.type = file
#beaker.session.data_dir = %(here)s/data/sessions/data

8
docs/admin/system_admin/tuning/tuning-vcs-memory-cache.rst Normal file
View file

@ -0,0 +1,8 @@
.. _adjust-vcs-mem-cache:
VCSServer Memory Cache
----------------------
The VCS Server mamory cache can be adjusted to work best with the resources
available to your |RCE| instance. If you find that memory resources are under
pressure, see the :ref:`vcs-server-maintain` section for details.

324
docs/admin/system_admin/vcs-server.rst Normal file
View file

@ -0,0 +1,324 @@
.. _vcs-server:
VCS Server Management
---------------------
The VCS Server handles |RCE| backend functionality. You need to configure
a VCS Server to run with a |RCE| instance. If you do not, you will be missing
the connection between |RCE| and its |repos|. This will cause error messages
on the web interface. You can run your setup in the following configurations,
currently the best performance is one of following:
* One VCS Server per |RCE| instance.
* One VCS Server handling multiple instances.
.. important::
If your server locale settings are not correctly configured,
|RCE| and the VCS Server can run into issues. See this `Ask Ubuntu`_ post
which explains the problem and gives a solution.
For more information, see the following sections:
* :ref:`install-vcs`
* :ref:`config-vcs`
* :ref:`vcs-server-options`
* :ref:`vcs-server-versions`
* :ref:`vcs-server-maintain`
* :ref:`vcs-server-config-file`
* :ref:`svn-http`
.. _install-vcs:
VCS Server Installation
^^^^^^^^^^^^^^^^^^^^^^^
To install a VCS Server, see
:ref:`Installing a VCS server <control:install-vcsserver>`.
.. _config-vcs:
Hooking |RCE| to its VCS Server
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To configure a |RCE| instance to use a VCS server, see
:ref:`Configuring the VCS Server connection <control:manually-vcsserver-ini>`.
.. _vcs-server-options:
|RCE| VCS Server Options
^^^^^^^^^^^^^^^^^^^^^^^^
The following list shows the available options on the |RCE| side of the
connection to the VCS Server. The settings are configured per
instance in the
:file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
.. rst-class:: dl-horizontal
\vcs.backends <available-vcs-systems>
Set a comma-separated list of the |repo| options available from the
web interface. The default is ``hg, git, svn``,
which is all |repo| types available. The order of backends is also the
order backend will try to detect requests type.
\vcs.connection_timeout <seconds>
Set the length of time in seconds that the VCS Server waits for
requests to process. After the timeout expires,
the request is closed. The default is ``3600``. Set to a higher
number if you experience network latency, or timeout issues with very
large push/pull requests.
\vcs.server.enable <boolean>
Enable or disable the VCS Server. The available options are ``true`` or
``false``. The default is ``true``.
\vcs.server <host:port>
Set the host, either hostname or IP Address, and port of the VCS server
you wish to run with your |RCE| instance.
.. code-block:: ini
##################
### VCS CONFIG ###
##################
# set this line to match your VCS Server
vcs.server = 127.0.0.1:10004
# Set to False to disable the VCS Server
vcs.server.enable = True
vcs.backends = hg, git, svn
vcs.connection_timeout = 3600
.. _vcs-server-versions:
VCS Server Versions
^^^^^^^^^^^^^^^^^^^
An updated version of the VCS Server is released with each |RCE| version. Use
the VCS Server number that matches with the |RCE| version to pair the
appropriate ones together. For |RCE| versions pre 3.3.0,
VCS Server 1.X.Y works with |RCE| 3.X.Y, for example:
* VCS Server 1.0.0 works with |RCE| 3.0.0
* VCS Server 1.2.2 works with |RCE| 3.2.2
For |RCE| versions post 3.3.0, the VCS Server and |RCE| version numbers
match, for example:
* VCS Server |release| works with |RCE| |release|
.. _vcs-server-maintain:
VCS Server Memory Optimization
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To optimize the VCS server to manage the cache and memory usage efficiently, you need to
configure the following options in the
:file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini` file. Once
configured, restart the VCS Server. By default we use an optimal settings, but in certain
conditions tunning expiration_time and max_size can affect memory usage and performance
.. code-block:: ini
## cache region for storing repo_objects cache
rc_cache.repo_object.backend = dogpile.cache.rc.memory_lru
## cache auto-expires after N seconds, setting this to 0 disabled cache
rc_cache.repo_object.expiration_time = 300
## max size of LRU, old values will be discarded if the size of cache reaches max_size
## Sets the maximum number of items stored in the cache, before the cache
## starts to be cleared.
## As a general rule of thumb, running this value at 120 resulted in a
## 5GB cache. Running it at 240 resulted in a 9GB cache. Your results
## will differ based on usage patterns and |repo| sizes.
## Tweaking this value to run at a fairly constant memory load on your
## server will help performance.
rc_cache.repo_object.max_size = 120
To clear the cache completely, you can restart the VCS Server.
.. important::
While the VCS Server handles a restart gracefully on the web interface,
it will drop connections during push/pull requests. So it is recommended
you only perform this when there is very little traffic on the instance.
Use the following example to restart your VCS Server,
for full details see the :ref:`RhodeCode Control CLI <control:rcc-cli>`.
.. code-block:: bash
$ rccontrol status
.. code-block:: vim
- NAME: vcsserver-1
- STATUS: RUNNING
logs:/home/ubuntu/.rccontrol/vcsserver-1/vcsserver.log
- VERSION: 4.7.2 VCSServer
- URL: http://127.0.0.1:10008
- CONFIG: /home/ubuntu/.rccontrol/vcsserver-1/vcsserver.ini
$ rccontrol restart vcsserver-1
Instance "vcsserver-1" successfully stopped.
Instance "vcsserver-1" successfully started.
.. _vcs-server-config-file:
VCS Server Configuration
^^^^^^^^^^^^^^^^^^^^^^^^
You can configure settings for multiple VCS Servers on your
system using their individual configuration files. Use the following
properties inside the configuration file to set up your system. The default
location is :file:`home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini`.
For a more detailed explanation of the logger levers, see :ref:`debug-mode`.
.. rst-class:: dl-horizontal
\host <ip-address>
Set the host on which the VCS Server will run. VCSServer is not
protected by any authentication, so we *highly* recommend running it
under localhost ip that is `127.0.0.1`
\port <int>
Set the port number on which the VCS Server will be available.
\locale <locale_utf>
Set the locale the VCS Server expects.
\workers <int>
Set the number of process workers.Recommended
value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
\max_requests <int>
The maximum number of requests a worker will process before restarting.
Any value greater than zero will limit the number of requests a work
will process before automatically restarting. This is a simple method
to help limit the damage of memory leaks.
\max_requests_jitter <int>
The maximum jitter to add to the max_requests setting.
The jitter causes the restart per worker to be randomized by
randint(0, max_requests_jitter). This is intended to stagger worker
restarts to avoid all workers restarting at the same time.
.. note::
After making changes, you need to restart your VCS Server to pick them up.
.. code-block:: ini
################################################################################
# RhodeCode VCSServer with HTTP Backend - configuration #
# #
################################################################################
[server:main]
## COMMON ##
host = 127.0.0.1
port = 10002
##########################
## GUNICORN WSGI SERVER ##
##########################
## run with gunicorn --log-config vcsserver.ini --paste vcsserver.ini
use = egg:gunicorn#main
## Sets the number of process workers. Recommended
## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
workers = 3
## process name
proc_name = rhodecode_vcsserver
## type of worker class, one of sync, gevent
## recommended for bigger setup is using of of other than sync one
worker_class = sync
## The maximum number of simultaneous clients. Valid only for Gevent
#worker_connections = 10
## max number of requests that worker will handle before being gracefully
## restarted, could prevent memory leaks
max_requests = 1000
max_requests_jitter = 30
## amount of time a worker can spend with handling a request before it
## gets killed and restarted. Set to 6hrs
timeout = 21600
[app:main]
use = egg:rhodecode-vcsserver
pyramid.default_locale_name = en
pyramid.includes =
## default locale used by VCS systems
locale = en_US.UTF-8
# cache regions, please don't change
beaker.cache.regions = repo_object
beaker.cache.repo_object.type = memorylru
beaker.cache.repo_object.max_items = 100
# cache auto-expires after N seconds
beaker.cache.repo_object.expire = 300
beaker.cache.repo_object.enabled = true
################################
### LOGGING CONFIGURATION ####
################################
[loggers]
keys = root, vcsserver, beaker
[handlers]
keys = console
[formatters]
keys = generic
#############
## LOGGERS ##
#############
[logger_root]
level = NOTSET
handlers = console
[logger_vcsserver]
level = DEBUG
handlers =
qualname = vcsserver
propagate = 1
[logger_beaker]
level = DEBUG
handlers =
qualname = beaker
propagate = 1
##############
## HANDLERS ##
##############
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = DEBUG
formatter = generic
################
## FORMATTERS ##
################
[formatter_generic]
format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %Y-%m-%d %H:%M:%S
.. _Subversion Red Book: http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.ref.svn
.. _Ask Ubuntu: http://askubuntu.com/questions/162391/how-do-i-fix-my-locale-issue