engineering/unrhodecode
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

release: merge back stable branch into default

Browse source
This commit is contained in:
Marcin Kuzminski 2016-08-23 10:22:30 +00:00
commit 9ae8eec560
8 changed files with 162 additions and 65 deletions
Download patch file Download diff file Expand all files Collapse all files

96
docs/admin/svn-http.rst
View file

@ -3,16 +3,21 @@
|svn| With Write Over HTTP
--------------------------
To use |svn| with write access, the currently supported method is over HTTP.
This requires you to configure your local machine so that it can access your
|RCE| instance.
To use |svn| with read/write support over the |svn| protocol, you have to
configure HTTP |svn| backend.
Prerequisites
^^^^^^^^^^^^^
- Enable lab setting on your |RCE| instance, see :ref:`lab-settings`.
- You need to install the following tools on your local machine: ``Apache`` and
``mod_dav_svn``. Use the following Ubuntu as an example.
- Enable HTTP support inside labs setting on your |RCE| instance,
see :ref:`lab-settings`.
- You need to install the following tools on the machine that is running an
instance of |RCE|:
``Apache HTTP Server`` and
``mod_dav_svn``.
Using Ubuntu Distribution as an example you can run:
.. code-block:: bash
@ -32,41 +37,80 @@ Configuring Apache Setup
It is recommended to run Apache on a port other than 80, due to possible
conflicts with other HTTP servers like nginx. To do this, set the
``Listen`` parameter in the ``/etc/apache2/ports.conf`` file, for example
``Listen 8090``
``Listen 8090``.
It is also recommended to run apache as the same user as |RCE|, otherwise
permission issues could occur. To do this edit the ``/etc/apache2/envvars``
.. warning::
Make sure your Apache instance which runs the mod_dav_svn module is
only accessible by RhodeCode. Otherwise everyone is able to browse
the repositories or run subversion operations (checkout/commit/etc.).
It is also recommended to run apache as the same user as |RCE|, otherwise
permission issues could occur. To do this edit the ``/etc/apache2/envvars``
.. code-block:: apache
export APACHE_RUN_USER=ubuntu
export APACHE_RUN_GROUP=ubuntu
export APACHE_RUN_USER=rhodecode
export APACHE_RUN_GROUP=rhodecode
1. To configure Apache, create and edit a virtual hosts file, for example
:file:`/etc/apache2/sites-available/default.conf`, or create another
virtual hosts file and add a location section inside the
``<VirtualHost>`` section.
:file:`/etc/apache2/sites-available/default.conf`. Below is an example
how to use one with auto-generated config ```mod_dav_svn.conf```
from configured |RCE| instance.
.. code-block:: apache
<Location />
DAV svn
# Must be explicit path, relative not supported
SVNParentPath /PATH/TO/REPOSITORIES
SVNListParentPath On
Allow from all
Order allow,deny
</Location>
<VirtualHost *:8080>
ServerAdmin rhodecode-admin@localhost
DocumentRoot /var/www/html
ErrorLog ${'${APACHE_LOG_DIR}'}/error.log
CustomLog ${'${APACHE_LOG_DIR}'}/access.log combined
Include /home/user/.rccontrol/enterprise-1/mod_dav_svn.conf
</VirtualHost>
.. note::
Once configured, check that you can see the list of repositories on your
|RCE| instance.
2. Go to the :menuselection:`Admin --> Settings --> Labs` page, and
enable :guilabel:`Proxy Subversion HTTP requests`, and specify the
:guilabel:`Subversion HTTP Server URL`.
3. Open the |RCE| configuration file,
:file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
4. Add the following configuration option in the ``[app:main]``
section if you don't have it yet.
This enable mapping of created |RCE| repo groups into special |svn| paths.
Each time a new repository group will be created the system will update
the template file, and create new mapping. Apache web server needs to be
reloaded to pick up the changes on this file.
It's recommended to add reload into a crontab so the changes can be picked
automatically once someone creates an repository group inside RhodeCode.
.. code-block:: ini
##############################################
### Subversion proxy support (mod_dav_svn) ###
##############################################
## Enable or disable the config file generation.
svn.proxy.generate_config = true
## Generate config file with `SVNListParentPath` set to `On`.
svn.proxy.list_parent_path = true
## Set location and file name of generated config file.
svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
## File system path to the directory containing the repositories served by
## RhodeCode.
svn.proxy.parent_path_root = /path/to/repo_store
## Used as a prefix to the <Location> block in the generated config file. In
## most cases it should be set to `/`.
svn.proxy.location_root = /
This would create a special template file called ```mod_dav_svn.conf```. We
used that file path in the apache config above inside the Include statement.
Using |svn|
^^^^^^^^^^^

15
docs/release-notes/release-notes-4.3.1.rst Normal file
View file

@ -0,0 +1,15 @@
|RCE| 4.3.1 |RNS|
-----------------
Release Date
^^^^^^^^^^^^
- 2016-08-23
Fixes
^^^^^
- Core: fixed database session cleanups. This will make sure RhodeCode can
function correctly after database server problems. Fixes #4173, refs #4166
- Diffs: limit the file context to ~1mln lines. Fixes #4184, also make sure
this doesn't trigger Integer overflow for msgpack.

1
docs/release-notes/release-notes.rst
View file

@ -9,6 +9,7 @@ Release Notes
.. toctree::
:maxdepth: 1
release-notes-4.3.1.rst
release-notes-4.3.0.rst
release-notes-4.2.1.rst
release-notes-4.2.0.rst

25
rhodecode/config/middleware.py
View file

@ -39,6 +39,7 @@ from routes.middleware import RoutesMiddleware
import routes.util
import rhodecode
from rhodecode.model import meta
from rhodecode.config import patches
from rhodecode.config.routing import STATIC_FILE_PREFIX
from rhodecode.config.environment import (
@ -159,6 +160,10 @@ def make_pyramid_app(global_config, **settings):
pyramid_app = config.make_wsgi_app()
pyramid_app = wrap_app_in_wsgi_middlewares(pyramid_app, config)
pyramid_app.config = config
# creating the app uses a connection - return it after we are done
meta.Session.remove()
return pyramid_app
@ -381,7 +386,25 @@ def wrap_app_in_wsgi_middlewares(pyramid_app, config):
pyramid_app = make_gzip_middleware(
pyramid_app, settings, compress_level=1)
return pyramid_app
# this should be the outer most middleware in the wsgi stack since
# middleware like Routes make database calls
def pyramid_app_with_cleanup(environ, start_response):
try:
return pyramid_app(environ, start_response)
finally:
# Dispose current database session and rollback uncommitted
# transactions.
meta.Session.remove()
# In a single threaded mode server, on non sqlite db we should have
# '0 Current Checked out connections' at the end of a request,
# if not, then something, somewhere is leaving a connection open
pool = meta.Base.metadata.bind.engine.pool
log.debug('sa pool status: %s', pool.status())
return pyramid_app_with_cleanup
def sanitize_settings_and_apply_defaults(settings):

21
rhodecode/lib/diffs.py
View file

@ -40,12 +40,17 @@ from rhodecode.lib.utils2 import safe_unicode
log = logging.getLogger(__name__)
# define max context, a file with more than this numbers of lines is unusable
# in browser anyway
MAX_CONTEXT = 1024 * 1014
class OPS(object):
ADD = 'A'
MOD = 'M'
DEL = 'D'
def wrap_to_table(str_):
return '''<table class="code-difftable">
<tr class="line no-comment">
@ -57,8 +62,8 @@ def wrap_to_table(str_):
def wrapped_diff(filenode_old, filenode_new, diff_limit=None, file_limit=None,
show_full_diff=False, ignore_whitespace=True, line_context=3,
enable_comments=False):
show_full_diff=False, ignore_whitespace=True, line_context=3,
enable_comments=False):
"""
returns a wrapped diff into a table, checks for cut_off_limit for file and
whole diff and presents proper message
@ -79,8 +84,9 @@ def wrapped_diff(filenode_old, filenode_new, diff_limit=None, file_limit=None,
f_gitdiff = get_gitdiff(filenode_old, filenode_new,
ignore_whitespace=ignore_whitespace,
context=line_context)
diff_processor = DiffProcessor(f_gitdiff, format='gitdiff', diff_limit=diff_limit,
file_limit=file_limit, show_full_diff=show_full_diff)
diff_processor = DiffProcessor(
f_gitdiff, format='gitdiff', diff_limit=diff_limit,
file_limit=file_limit, show_full_diff=show_full_diff)
_parsed = diff_processor.prepare()
diff = diff_processor.as_html(enable_comments=enable_comments)
@ -115,6 +121,10 @@ def get_gitdiff(filenode_old, filenode_new, ignore_whitespace=True, context=3):
"""
# make sure we pass in default context
context = context or 3
# protect against IntOverflow when passing HUGE context
if context > MAX_CONTEXT:
context = MAX_CONTEXT
submodules = filter(lambda o: isinstance(o, SubModuleNode),
[filenode_new, filenode_old])
if submodules:
@ -190,7 +200,8 @@ class DiffProcessor(object):
# used for inline highlighter word split
_token_re = re.compile(r'()(&gt;|&lt;|&amp;|\W+?)')
def __init__(self, diff, format='gitdiff', diff_limit=None, file_limit=None, show_full_diff=True):
def __init__(self, diff, format='gitdiff', diff_limit=None,
file_limit=None, show_full_diff=True):
"""
:param diff: A `Diff` object representing a diff from a vcs backend
:param format: format of diff passed, `udiff` or `gitdiff`

7
rhodecode/lib/hooks_daemon.py
View file

@ -30,6 +30,7 @@ import Pyro4
import pylons
import rhodecode
from rhodecode.model import meta
from rhodecode.lib import hooks_base
from rhodecode.lib.utils2 import (
AttributeDict, safe_str, get_routes_generator_for_server_url)
@ -64,7 +65,10 @@ class HooksHttpHandler(BaseHTTPRequestHandler):
def _call_hook(self, method, extras):
hooks = Hooks()
result = getattr(hooks, method)(extras)
try:
result = getattr(hooks, method)(extras)
finally:
meta.Session.remove()
return result
def log_message(self, format, *args):
@ -261,6 +265,7 @@ class Hooks(object):
}
finally:
pylons.url._pop_object()
meta.Session.remove()
return {
'status': result.status,

7
rhodecode/lib/middleware/simplevcs.py
View file

@ -406,8 +406,11 @@ class SimpleVCS(object):
yield chunk
finally:
# invalidate cache on push
if action == 'push':
self._invalidate_cache(repo_name)
try:
if action == 'push':
self._invalidate_cache(repo_name)
finally:
meta.Session.remove()
def _get_repository_name(self, environ):
"""Get repository name out of the environmnent

55
rhodecode/tweens.py
View file

@ -40,40 +40,35 @@ def pylons_compatibility_tween_factory(handler, registry):
from pyramid. For example while rendering an old template that uses the
'c' or 'h' objects. This tween sets up the needed pylons globals.
"""
try:
config = rhodecode.CONFIG
environ = request.environ
session = request.session
session_key = (config['pylons.environ_config']
.get('session', 'beaker.session'))
config = rhodecode.CONFIG
environ = request.environ
session = request.session
session_key = (config['pylons.environ_config']
.get('session', 'beaker.session'))
# Setup pylons globals.
pylons.config._push_object(config)
pylons.request._push_object(request)
pylons.session._push_object(session)
environ[session_key] = session
pylons.url._push_object(URLGenerator(config['routes.map'],
environ))
# Setup pylons globals.
pylons.config._push_object(config)
pylons.request._push_object(request)
pylons.session._push_object(session)
environ[session_key] = session
pylons.url._push_object(URLGenerator(config['routes.map'],
environ))
# TODO: Maybe we should use the language from pyramid.
translator = _get_translator(config.get('lang'))
pylons.translator._push_object(translator)
# TODO: Maybe we should use the language from pyramid.
translator = _get_translator(config.get('lang'))
pylons.translator._push_object(translator)
# Get the rhodecode auth user object and make it available.
auth_user = get_auth_user(environ)
request.user = auth_user
environ['rc_auth_user'] = auth_user
# Get the rhodecode auth user object and make it available.
auth_user = get_auth_user(environ)
request.user = auth_user
environ['rc_auth_user'] = auth_user
# Setup the pylons context object ('c')
context = ContextObj()
context.rhodecode_user = auth_user
attach_context_attributes(context, request)
pylons.tmpl_context._push_object(context)
return handler(request)
finally:
# Dispose current database session and rollback uncommitted
# transactions.
meta.Session.remove()
# Setup the pylons context object ('c')
context = ContextObj()
context.rhodecode_user = auth_user
attach_context_attributes(context, request)
pylons.tmpl_context._push_object(context)
return handler(request)
return pylons_compatibility_tween