engineering/unrhodecode
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

rcextensions: new builtin rcextensions.

Browse source 
- ability to expose output to users from rcextensions
- new docs
- linking to the code makes the tests always check the integrity of rcextensions
- added new hook variables and described them to end users
This commit is contained in:
Marcin Kuzminski 2018-10-23 11:45:45 +02:00
parent 2f4516d1ef
commit 175a9b344e
38 changed files with 1120 additions and 2133 deletions
Download patch file Download diff file Expand all files Collapse all files

150
docs/extensions/config-ext.rst
View file

@ -1,150 +0,0 @@
.. _config-ext:
Configure |RCX|
---------------
To get the the built in plugins and extensions working the way you want them
to, you have to configure them to work with your services. An overview of
what needs to be done is:
* :ref:`config-rcx-plugin` to carry out your desired actions once its hook is
triggered. There are default actions built in, but you may wish to alter
those.
* :ref:`config-rcx-hook` to execute actions for the plugin, when certain
actions are carried out with |RCE|.
.. _config-rcx-plugin:
Tweak a Default Plugin
^^^^^^^^^^^^^^^^^^^^^^
Each of the default plugins comes with a standard configuration, but you may
wish to change those settings. In this example, the Redmine plugin watches
for the words defined in the ``HASH_REGEX`` variable and takes actions if one
of those words is used in conjunction with a ``#{number}``, which matches a
ticket number in Redmine. You can configure this to work differently based on
the `Redmine documentation`_.
.. code-block:: python
:emphasize-lines: 3-5, 37
import re
HASH_REGEX = re.compile(
r"(?:fix|fixes|fixing|close|closes|closing)\s*#([0-9]+)\b",
re.IGNORECASE)
def link_to_commit(repo_url, commit_id):
rev_url = '%s/changeset/%s' % (repo_url, commit_id)
return '"%s":%s' % (commit_id[:6], rev_url)
def run(*args, **kwargs):
issues = kwargs['RM_ISSUES']
if not issues:
return 0
# repo extra fields can control this, they should be propagated with
# extract repo fields
tracker_url = kwargs.get('redmine_tracker_url') or kwargs['RM_URL']
project_id = kwargs.get('redmine_project_id') or kwargs['RM_PROJECT']
api_key = kwargs.get('redmine_api_key') or kwargs['RM_APIKEY']
if project_id:
from redmine import Redmine
remote_redmine = Redmine(tracker_url, key=api_key)
project = remote_redmine.project.get(project_id)
repo_url = '%(server_url)s/%(repository)s' % kwargs
# for each fetched issue id make a redmine api call
for _id, details in issues.items():
commits = ', '.join([link_to_commit(repo_url,
x['raw_id'],)
for x in details])
issue = project.issues.get(int(_id))
if issue:
issue.notes = 'Issue resolved by %s' % (commits,)
issue.status_id = 3 # Resolved
issue.save()
.. _config-rcx-hook:
Configure a Hook
^^^^^^^^^^^^^^^^
To configure the default hooks in the
:file:`/home/{user}/.rccontrol/{instance-id}/rcextensions/__init.py__` file,
use the following steps.
1. Configure the connection details, either in the file or import from a
dictionary. For these connection scenarios the following details need to
be configured.
* **REDMINE_URL** = '<redmine-url>'
* **REDMINE_API_KEY** = '<secret>'
* **SLACK_API_URL** = '<slack-url>?token=<secret>'
* **SLACK_API_KEY** = '<secret>'
2. You will also need to configure other variables, such as the
**SLACK_ROOM** or **RM_PROJECT** (Redmine Project). These set where the
commit message is posted. Various hooks can take different variables and
they are documented in the file.
3. Inside each hook you can then configure it to carry out actions
per service. In this example, the push hook is pushing to the Redmine and
Slack plugins on each push if the hook criteria are matched.
.. code-block:: python
:emphasize-lines: 21-29, 37-44
def _push_hook(*args, **kwargs):
kwargs['commit_ids'] = kwargs['pushed_revs']
call = load_extension('extra_fields.py')
if call:
repo_extra_fields = call(**kwargs)
# now update if we have extra fields, they have precedence
# this way users can store any configuration inside
# the database per repo
for key, data in repo_extra_fields.items():
kwargs[key] = data['field_value']
# fetch pushed commits
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# slack !
call = load_extension('slack.py')
if call:
kwargs['INCOMING_WEBHOOK_URL'] = SLACK_API_URL
kwargs['SLACK_TOKEN'] = SLACK_API_KEY
kwargs['SLACK_ROOM'] = '#slack-channel'
kwargs['SLACK_FROM'] = 'Slack-Message-Poster'
kwargs['SLACK_FROM_ICON_EMOJI'] = ':slack-emoji:'
call(**kwargs)
# fetch issues from given commits
call = load_extension('extract_issues.py')
issues = {}
if call:
issues = call(**kwargs)
# redmine smart commits
call = load_extension('redmine_smart_commits.py')
if call:
kwargs['RM_URL'] = REDMINE_URL
kwargs['RM_APIKEY'] = REDMINE_API_KEY
kwargs['RM_PROJECT'] = None # uses extra_fields from repo
kwargs['RM_ISSUES'] = issues
call(**kwargs)
return 0
PUSH_HOOK = _push_hook
.. _Redmine documentation: http://www.redmine.org/projects/redmine/wiki/Rest_api

823
docs/extensions/example-ext.py
View file

@ -1,823 +0,0 @@
"""
rcextensions module.
"""
import os
import imp
import string
import functools
here = os.path.dirname(os.path.abspath(__file__))
registered_extensions = dict()
class DotDict(dict):
def __contains__(self, k):
try:
return dict.__contains__(self, k) or hasattr(self, k)
except:
return False
# only called if k not found in normal places
def __getattr__(self, k):
try:
return object.__getattribute__(self, k)
except AttributeError:
try:
return self[k]
except KeyError:
raise AttributeError(k)
def __setattr__(self, k, v):
try:
object.__getattribute__(self, k)
except AttributeError:
try:
self[k] = v
except:
raise AttributeError(k)
else:
object.__setattr__(self, k, v)
def __delattr__(self, k):
try:
object.__getattribute__(self, k)
except AttributeError:
try:
del self[k]
except KeyError:
raise AttributeError(k)
else:
object.__delattr__(self, k)
def toDict(self):
return unserialize(self)
def __repr__(self):
keys = list(self.iterkeys())
keys.sort()
args = ', '.join(['%s=%r' % (key, self[key]) for key in keys])
return '%s(%s)' % (self.__class__.__name__, args)
@staticmethod
def fromDict(d):
return serialize(d)
def serialize(x):
if isinstance(x, dict):
return DotDict((k, serialize(v)) for k, v in x.iteritems())
elif isinstance(x, (list, tuple)):
return type(x)(serialize(v) for v in x)
else:
return x
def unserialize(x):
if isinstance(x, dict):
return dict((k, unserialize(v)) for k, v in x.iteritems())
elif isinstance(x, (list, tuple)):
return type(x)(unserialize(v) for v in x)
else:
return x
def load_extension(filename, async=False):
"""
use to load extensions inside rcextension folder.
for example::
callback = load_extension('email.py', async=False)
if callback:
callback('foobar')
put file named email.py inside rcextensions folder to load it. Changing
async=True will make the call of the plugin async, it's useful for
blocking calls like sending an email or notification with APIs.
"""
mod = ''.join(filename.split('.')[:-1])
loaded = imp.load_source(mod, os.path.join(here, filename))
callback = getattr(loaded, 'run', None)
if not callback:
raise Exception('Plugin missing `run` method')
if async:
# modify callback so it's actually an async call
def _async_callback(*args, **kwargs):
import threading
thr = threading.Thread(target=callback, args=args, kwargs=kwargs)
thr.start()
if kwargs.get('_async_block'):
del kwargs['_async_block']
thr.join()
return _async_callback
return callback
def _verify_kwargs(expected_parameters, kwargs):
"""
Verify that exactly `expected_parameters` are passed in as `kwargs`.
"""
expected_parameters = set(expected_parameters)
kwargs_keys = set(kwargs.keys())
if kwargs_keys != expected_parameters:
missing_kwargs = expected_parameters - kwargs_keys
unexpected_kwargs = kwargs_keys - expected_parameters
raise AssertionError(
"Missing parameters: %r, unexpected parameters: %s" %
(missing_kwargs, unexpected_kwargs))
def verify_kwargs(required_args):
"""
decorator to verify extension calls arguments.
:param required_args:
"""
def wrap(func):
def wrapper(*args, **kwargs):
_verify_kwargs(required_args, kwargs)
return func(*args, **kwargs)
return wrapper
return wrap
def register(name=None):
def wrap(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
# register load_extensions in kwargs, so we can chain plugins
kwargs['_load_extension'] = load_extension
# append this path for us to use added plugins or modules
import sys
_cur_path = os.path.dirname(os.path.abspath(__file__))
if _cur_path not in sys.path:
sys.path.append(_cur_path)
registered_extensions[func.__name__] = func
return func(*args, **kwargs)
return wrapper
return wrap
# =============================================================================
# END OF UTILITY FUNCTIONS HERE
# =============================================================================
# Additional mappings that are not present in the pygments lexers
# used for building stats
# format is {'ext':['Names']} eg. {'py':['Python']} note: there can be
# more than one name for extension
# NOTE: that this will override any mappings in LANGUAGES_EXTENSIONS_MAP
# build by pygments
EXTRA_MAPPINGS = {}
# additional lexer definitions for custom files it's overrides pygments lexers,
# and uses defined name of lexer to colorize the files. Format is {'ext':
# 'lexer_name'} List of lexers can be printed running:
# >> python -c "import pprint;from pygments import lexers;
# pprint.pprint([(x[0], x[1]) for x in lexers.get_all_lexers()]);"
EXTRA_LEXERS = {}
CONFIG = DotDict(
slack=DotDict(
api_key='api-key',
api_url='slack-incoming-hook-url',
default_room='#slack-channel',
default_plugin_config={},
),
redmine=DotDict(
api_key='api-key',
default_tracker_url='https://redmine.tracker.url',
default_project_id=None,
default_status_resolved_id=3
),
)
# slack conf
CONFIG.slack.default_plugin_config = {
'INCOMING_WEBHOOK_URL': CONFIG.slack.api_url,
'SLACK_TOKEN': CONFIG.slack.api_key,
'SLACK_ROOM': CONFIG.slack.default_room,
'SLACK_FROM': 'RhodeCode',
'SLACK_FROM_ICON_EMOJI': ':rhodecode:',
}
# redmine smart_pr configuration
def configure_redmine_smart_pr(issues, kwargs):
kwargs['REDMINE_ISSUES'] = issues
kwargs['redmine_tracker_url'] = kwargs.pop(
'redmine_tracker_url', '') or CONFIG.redmine.default_tracker_url
kwargs['redmine_api_key'] = kwargs.pop(
'redmine_api_key', '') or CONFIG.redmine.api_key
kwargs['redmine_project_id'] = kwargs.pop(
'redmine_project_id', '') or CONFIG.redmine.default_project_id
@register('CREATE_REPO_HOOK')
@verify_kwargs(
['_load_extension', 'repo_name', 'repo_type', 'description', 'private',
'created_on', 'enable_downloads', 'repo_id', 'user_id', 'enable_statistics',
'clone_uri', 'fork_id', 'group_id', 'created_by'])
def _create_repo_hook(*args, **kwargs):
"""
POST CREATE REPOSITORY HOOK. This function will be executed after
each repository is created. kwargs available:
:param repo_name:
:param repo_type:
:param description:
:param private:
:param created_on:
:param enable_downloads:
:param repo_id:
:param user_id:
:param enable_statistics:
:param clone_uri:
:param fork_id:
:param group_id:
:param created_by:
"""
return 0
CREATE_REPO_HOOK = _create_repo_hook
@register('CREATE_REPO_GROUP_HOOK')
@verify_kwargs(
['_load_extension', 'group_name', 'group_parent_id', 'group_description',
'group_id', 'user_id', 'created_by', 'created_on', 'enable_locking'])
def _create_repo_group_hook(*args, **kwargs):
"""
POST CREATE REPOSITORY GROUP HOOK, this function will be
executed after each repository group is created. kwargs available:
:param group_name:
:param group_parent_id:
:param group_description:
:param group_id:
:param user_id:
:param created_by:
:param created_on:
:param enable_locking:
"""
return 0
CREATE_REPO_GROUP_HOOK = _create_repo_group_hook
@register('PRE_CREATE_USER_HOOK')
@verify_kwargs(
['_load_extension', 'username', 'password', 'email', 'firstname',
'lastname', 'active', 'admin', 'created_by'])
def _pre_create_user_hook(*args, **kwargs):
"""
PRE CREATE USER HOOK, this function will be executed before each
user is created, it returns a tuple of bool, reason.
If bool is False the user creation will be stopped and reason
will be displayed to the user. kwargs available:
:param username:
:param password:
:param email:
:param firstname:
:param lastname:
:param active:
:param admin:
:param created_by:
"""
reason = 'allowed'
return True, reason
PRE_CREATE_USER_HOOK = _pre_create_user_hook
@register('CREATE_USER_HOOK')
@verify_kwargs(
['_load_extension', 'username', 'full_name_or_username', 'full_contact',
'user_id', 'name', 'firstname', 'short_contact', 'admin', 'lastname',
'ip_addresses', 'extern_type', 'extern_name', 'email', 'api_key',
'api_keys', 'last_login', 'full_name', 'active', 'password', 'emails',
'inherit_default_permissions', 'created_by', 'created_on'])
def _create_user_hook(*args, **kwargs):
"""
POST CREATE USER HOOK, this function will be executed after each user is created
kwargs available:
:param username:
:param full_name_or_username:
:param full_contact:
:param user_id:
:param name:
:param firstname:
:param short_contact:
:param admin:
:param lastname:
:param ip_addresses:
:param extern_type:
:param extern_name:
:param email:
:param api_key:
:param api_keys:
:param last_login:
:param full_name:
:param active:
:param password:
:param emails:
:param inherit_default_permissions:
:param created_by:
:param created_on:
"""
return 0
CREATE_USER_HOOK = _create_user_hook
@register('DELETE_REPO_HOOK')
@verify_kwargs(
['_load_extension', 'repo_name', 'repo_type', 'description', 'private',
'created_on', 'enable_downloads', 'repo_id', 'user_id', 'enable_statistics',
'clone_uri', 'fork_id', 'group_id', 'deleted_by', 'deleted_on'])
def _delete_repo_hook(*args, **kwargs):
"""
POST DELETE REPOSITORY HOOK, this function will be executed after
each repository deletion kwargs available:
:param repo_name:
:param repo_type:
:param description:
:param private:
:param created_on:
:param enable_downloads:
:param repo_id:
:param user_id:
:param enable_statistics:
:param clone_uri:
:param fork_id:
:param group_id:
:param deleted_by:
:param deleted_on:
"""
return 0
DELETE_REPO_HOOK = _delete_repo_hook
@register('DELETE_USER_HOOK')
@verify_kwargs(
['_load_extension', 'username', 'full_name_or_username', 'full_contact',
'user_id', 'name', 'firstname', 'short_contact', 'admin', 'lastname',
'ip_addresses', 'email', 'api_key', 'last_login', 'full_name', 'active',
'password', 'emails', 'inherit_default_permissions', 'deleted_by'
])
def _delete_user_hook(*args, **kwargs):
"""
POST DELETE USER HOOK, this function will be executed after each
user is deleted kwargs available:
:param username:
:param full_name_or_username:
:param full_contact:
:param user_id:
:param name:
:param firstname:
:param short_contact:
:param admin:
:param lastname:
:param ip_addresses:
:param ldap_dn:
:param email:
:param api_key:
:param last_login:
:param full_name:
:param active:
:param password:
:param emails:
:param inherit_default_permissions:
:param deleted_by:
"""
return 0
DELETE_USER_HOOK = _delete_user_hook
@register('PRE_PUSH_HOOK')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username',
'ip', 'action', 'repository', 'repo_store_path'])
def _pre_push_hook(*args, **kwargs):
"""
Post push hook
kwargs available:
:param server_url: url of instance that triggered this hook
:param config: path to .ini config used
:param scm: type of VS 'git' or 'hg'
:param username: name of user who pushed
:param ip: ip of who pushed
:param action: push
:param repository: repository name
:param repo_store_path: full path to where repositories are stored
"""
return 0
PRE_PUSH_HOOK = _pre_push_hook
@register('PUSH_HOOK')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username',
'ip', 'action', 'repository', 'repo_store_path', 'pushed_revs'])
def _push_hook(*args, **kwargs):
"""
POST PUSH HOOK, this function will be executed after each push it's
executed after the build-in hook that RhodeCode uses for logging pushes
kwargs available:
:param server_url: url of instance that triggered this hook
:param config: path to .ini config used
:param scm: type of VS 'git' or 'hg'
:param username: name of user who pushed
:param ip: ip of who pushed
:param action: push
:param repository: repository name
:param repo_store_path: full path to where repositories are stored
:param pushed_revs: list of pushed commit ids
"""
# backward compat
kwargs['commit_ids'] = kwargs['pushed_revs']
# fetch extra fields from repository
call = load_extension('extra_fields.py')
_extra_fields = {}
if call:
repo_extra_fields = call(**kwargs)
# now update if we have extra fields, they have precedence
# this way users can store any configuration inside the database per
# repo
for key, data in repo_extra_fields.items():
kwargs[key] = data['field_value']
_extra_fields[key] = data['field_value']
# fetch pushed commits, from commit_ids list
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# slack !
call = load_extension('slack_push_notify.py')
if call:
kwargs.update(CONFIG.slack.default_plugin_config)
call(**kwargs)
# fetch redmine issues from given commits
call = load_extension('extract_redmine_issues.py')
issues = {}
if call:
issues = call(**kwargs)
# redmine smart commits
call = load_extension('redmine_smart_commits.py')
if call:
kwargs['REDMINE_ISSUES'] = issues
kwargs['redmine_tracker_url'] = kwargs.pop(
'redmine_tracker_url', '') or CONFIG.redmine.default_tracker_url
kwargs['redmine_api_key'] = kwargs.pop(
'redmine_api_key', '') or CONFIG.redmine.api_key
kwargs['redmine_status_resolved_id'] = kwargs.pop(
'redmine_status_resolved_id', '') or CONFIG.redmine.default_status_resolved_id
kwargs['redmine_project_id'] = kwargs.pop(
'redmine_project_id', '') or CONFIG.redmine.default_project_id
call(**kwargs)
return 0
PUSH_HOOK = _push_hook
@register('PRE_PULL_HOOK')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username', 'ip',
'action', 'repository'])
def _pre_pull_hook(*args, **kwargs):
"""
Post pull hook
kwargs available::
:param server_url: url of instance that triggered this hook
:param config: path to .ini config used
:param scm: type of VS 'git' or 'hg'
:param username: name of user who pulled
:param ip: ip of who pulled
:param action: pull
:param repository: repository name
"""
return 0
PRE_PULL_HOOK = _pre_pull_hook
@register('PULL_HOOK')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username', 'ip',
'action', 'repository'])
def _pull_hook(*args, **kwargs):
"""
POST PULL HOOK, this function will be executed after each push it's
executed after the build-in hook that RhodeCode uses for logging pulls
kwargs available:
:param server_url: url of instance that triggered this hook
:param config: path to .ini config used
:param scm: type of VS 'git' or 'hg'
:param username: name of user who pulled
:param ip: ip of who pulled
:param action: pull
:param repository: repository name
"""
return 0
PULL_HOOK = _pull_hook
# =============================================================================
# PULL REQUEST RELATED HOOKS
# =============================================================================
@register('CREATE_PULL_REQUEST')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username', 'ip',
'action', 'repository', 'pull_request_id', 'url', 'title', 'description',
'status', 'created_on', 'updated_on', 'commit_ids', 'review_status',
'mergeable', 'source', 'target', 'author', 'reviewers'])
def _create_pull_request_hook(*args, **kwargs):
"""
"""
# extract extra fields and default reviewers from target
kwargs['REPOSITORY'] = kwargs['target']['repository']
call = load_extension('extra_fields.py')
if call:
repo_extra_fields = call(**kwargs)
# now update if we have extra fields, they have precedence
# this way users can store any configuration inside the database per
# repo
for key, data in repo_extra_fields.items():
kwargs[key] = data['field_value']
call = load_extension('default_reviewers.py')
if call:
# read default_reviewers key propagated from extra fields
kwargs['default_reviewers'] = map(string.strip, kwargs.pop(
'default_reviewers', '').split(','))
call(**kwargs)
# extract below from source repo as commits are there
kwargs['REPOSITORY'] = kwargs['source']['repository']
# # fetch pushed commits, from commit_ids list
# call = load_extension('extract_commits.py')
# extracted_commits = {}
# if call:
# extracted_commits = call(**kwargs)
# # store the commits for the next call chain
# kwargs['COMMITS'] = extracted_commits
#
# # fetch issues from given commits
# call = load_extension('extract_redmine_issues.py')
# issues = {}
# if call:
# issues = call(**kwargs)
#
# # redmine smart pr update
# call = load_extension('redmine_pr_flow.py')
# if call:
# # updates kwargs on the fly
# configure_redmine_smart_pr(issues=issues, kwargs=kwargs)
# call(**kwargs)
#
# # slack notification on merging PR
# call = load_extension('slack_message.py')
# if call:
# kwargs.update(CONFIG.slack.default_plugin_config)
# kwargs['SLACK_ROOM'] = '#develop'
# kwargs['SLACK_MESSAGE'] = 'Pull request <%s|#%s> (%s) was created.' % (
# kwargs.get('url'), kwargs.get('pull_request_id'), kwargs.get('title'))
#
# call(**kwargs)
return 0
CREATE_PULL_REQUEST = _create_pull_request_hook
@register('REVIEW_PULL_REQUEST')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username', 'ip',
'action', 'repository', 'pull_request_id', 'url', 'title', 'description',
'status', 'created_on', 'updated_on', 'commit_ids', 'review_status',
'mergeable', 'source', 'target', 'author', 'reviewers'])
def _review_pull_request_hook(*args, **kwargs):
"""
"""
# extract extra fields and default reviewers from target
kwargs['REPOSITORY'] = kwargs['target']['repository']
# fetch extra fields
call = load_extension('extra_fields.py')
if call:
repo_extra_fields = call(**kwargs)
# now update if we have extra fields, they have precedence
# this way users can store any configuration inside the database per
# repo
for key, data in repo_extra_fields.items():
kwargs[key] = data['field_value']
# extract below from source repo as commits are there
kwargs['REPOSITORY'] = kwargs['source']['repository']
# fetch pushed commits, from commit_ids list
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# fetch issues from given commits
call = load_extension('extract_redmine_issues.py')
issues = {}
if call:
issues = call(**kwargs)
# redmine smart pr update
call = load_extension('redmine_pr_flow.py')
if call:
# updates kwargs on the fly
configure_redmine_smart_pr(issues=issues, kwargs=kwargs)
call(**kwargs)
return 0
REVIEW_PULL_REQUEST = _review_pull_request_hook
@register('UPDATE_PULL_REQUEST')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username', 'ip',
'action', 'repository', 'pull_request_id', 'url', 'title', 'description',
'status', 'created_on', 'updated_on', 'commit_ids', 'review_status',
'mergeable', 'source', 'target', 'author', 'reviewers'])
def _update_pull_request_hook(*args, **kwargs):
"""
"""
# extract extra fields and default reviewers from target
kwargs['REPOSITORY'] = kwargs['target']['repository']
# fetch extra fields
call = load_extension('extra_fields.py')
if call:
repo_extra_fields = call(**kwargs)
# now update if we have extra fields, they have precedence
# this way users can store any configuration inside the database per
# repo
for key, data in repo_extra_fields.items():
kwargs[key] = data['field_value']
# extract below from source repo as commits are there
kwargs['REPOSITORY'] = kwargs['source']['repository']
# fetch pushed commits, from commit_ids list
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# fetch issues from given commits
call = load_extension('extract_redmine_issues.py')
issues = {}
if call:
issues = call(**kwargs)
# redmine smart pr updated
call = load_extension('redmine_pr_flow.py')
if call:
# updates kwargs on the fly
configure_redmine_smart_pr(issues=issues, kwargs=kwargs)
call(**kwargs)
return 0
UPDATE_PULL_REQUEST = _update_pull_request_hook
@register('MERGE_PULL_REQUEST')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username', 'ip',
'action', 'repository', 'pull_request_id', 'url', 'title', 'description',
'status', 'created_on', 'updated_on', 'commit_ids', 'review_status',
'mergeable', 'source', 'target', 'author', 'reviewers'])
def _merge_pull_request_hook(*args, **kwargs):
"""
"""
# extract extra fields and default reviewers from target
kwargs['REPOSITORY'] = kwargs['target']['repository']
# fetch extra fields
call = load_extension('extra_fields.py')
if call:
repo_extra_fields = call(**kwargs)
# now update if we have extra fields, they have precedence
# this way users can store any configuration inside the database per
# repo
for key, data in repo_extra_fields.items():
kwargs[key] = data['field_value']
# extract below from source repo as commits are there
kwargs['REPOSITORY'] = kwargs['source']['repository']
# fetch pushed commits, from commit_ids list
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# fetch issues from given commits
call = load_extension('extract_redmine_issues.py')
issues = {}
if call:
issues = call(**kwargs)
# redmine smart pr update
call = load_extension('redmine_pr_flow.py')
if call:
# updates kwargs on the fly
configure_redmine_smart_pr(issues=issues, kwargs=kwargs)
call(**kwargs)
# slack notification on merging PR
call = load_extension('slack_message.py')
if call:
kwargs.update(CONFIG.slack.default_plugin_config)
kwargs['SLACK_ROOM'] = '#develop'
kwargs['SLACK_MESSAGE'] = 'Pull request <%s|#%s> (%s) was merged.' % (
kwargs.get('url'), kwargs.get('pull_request_id'), kwargs.get('title'))
call(**kwargs)
return 0
MERGE_PULL_REQUEST = _merge_pull_request_hook
@register('CLOSE_PULL_REQUEST')
@verify_kwargs(
['_load_extension', 'server_url', 'config', 'scm', 'username', 'ip',
'action', 'repository', 'pull_request_id', 'url', 'title', 'description',
'status', 'created_on', 'updated_on', 'commit_ids', 'review_status',
'mergeable', 'source', 'target', 'author', 'reviewers'])
def _close_pull_request_hook(*args, **kwargs):
"""
"""
# extract extra fields and default reviewers from target
kwargs['REPOSITORY'] = kwargs['target']['repository']
# fetch extra fields
call = load_extension('extra_fields.py')
if call:
repo_extra_fields = call(**kwargs)
# now update if we have extra fields, they have precedence
# this way users can store any configuration inside the database per
# repo
for key, data in repo_extra_fields.items():
kwargs[key] = data['field_value']
# extract below from source repo as commits are there
kwargs['REPOSITORY'] = kwargs['source']['repository']
# fetch pushed commits, from commit_ids list
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# fetch issues from given commits
call = load_extension('extract_redmine_issues.py')
issues = {}
if call:
issues = call(**kwargs)
# redmine smart pr update
call = load_extension('redmine_pr_flow.py')
if call:
# updates kwargs on the fly
configure_redmine_smart_pr(issues=issues, kwargs=kwargs)
call(**kwargs)
return 0
CLOSE_PULL_REQUEST = _close_pull_request_hook

27
docs/extensions/extensions-hooks.rst
View file

@ -14,12 +14,23 @@ so to clarify what is meant each time, read the following definitions:
between software components and can be used to trigger plugins, or their
extensions.
.. toctree::
rcx
install-ext
config-ext
extensions
hooks
full-blown-example
int-slack
Hooks
-----
Within |RCM| there are two types of supported hooks.
* **Internal built-in hooks**: The internal |hg|, |git| or |svn| hooks are
triggered by different VCS operations, like push, pull,
or clone and are non-configurable, but you can add your own VCS hooks,
see :ref:`custom-hooks`.
* **Custom rcextensions hooks**: User defined hooks centre around the lifecycle of
certain actions such are |repo| creation, user creation etc. The actions
these hooks trigger can be rejected based on the API permissions of the
user calling them.
On instructions how to use the custom `rcextensions`
see :ref:`integrations-rcextensions` section.

59
docs/extensions/extensions.rst
View file

@ -1,59 +0,0 @@
.. _dev-plug:
Developing Plugins/Extensions
-----------------------------
An Extension or a Plugin is simply a |PY| module with a ``run`` method that
expects a number of parameters, depending on which event it is listening
for. To get an extension working, use the following steps:
1. Create an extension or plugin using the below example.
2. Save the plugin inside the
:file:`/home/{user}/.rccontrol/{instance-id}/rcextensions` folder.
3. Add a hook to the
:file:`/home/{user}/.rccontrol/{instance-id}/rcextensions/__init__.py` file.
For more information, see :ref:`event-listener`.
4. Restart your |RCM| instance.
Extension example
^^^^^^^^^^^^^^^^^
In the following example, the ``run`` method listens for a push to a |repo|
and parses the commit.
.. code-block:: python
def run(*args, **kwargs):
revs = kwargs.get('pushed_revs')
if not revs:
return 0
from rhodecode.lib.utils2 import extract_mentioned_users
from rhodecode.model.db import Repository
repo = Repository.get_by_repo_name(kwargs['repository'])
changesets = []
reviewers = []
# reviewer fields from extra_fields, users can store their custom
# reviewers inside the extra fields to pre-define a set of people who
# will get notifications about changesets
field_key = kwargs.get('reviewers_extra_field')
if field_key:
for xfield in repo.extra_fields:
if xfield.field_key == field_key:
reviewers.extend(xfield.field_value.split())
vcs_repo = repo.scm_instance_no_cache()
for rev in kwargs['pushed_revs']:
cs = vcs_repo.get_changeset(rev) # or get_commit. See API doc
cs_data = cs.__json__()
cs_data['mentions'] = extract_mentioned_users(cs_data['message'])
cs_data['reviewers'] = reviewers
# optionally add more logic to parse the commits, like reading extra
# fields of repository to read managers of reviewers
changesets.append(cs_data)
return changesets

19
docs/extensions/full-blown-example.rst
View file

@ -1,19 +0,0 @@
.. _int-full-blown:
Extensions Extended Example
---------------------------
This example
:file:`/home/{user}/.rccontrol/{instance-id}/rcextensions/__init.py__` file
has been highlighted to show a Redmine integration in full. To extend your
|RCE| instances, use the below example to integrate with other
applications.
This example file also contains a Slack integration, but it is not
highlighted.
.. literalinclude:: example-ext.py
:language: python
:emphasize-lines: 186,193-198,210-218,474-496,648-660,749-760,810-822
:linenos:

86
docs/extensions/hooks.rst
View file

@ -1,86 +0,0 @@
.. _hooks:
Hooks
-----
Within |RCM| there are two types of supported hooks.
* **Internal built-in hooks**: The internal |hg| or |git| hooks are
triggered by different VCS operations, like push, pull,
or clone and are non-configurable, but you can add your own VCS hooks,
see :ref:`custom-hooks`.
* **User defined hooks**: User defined hooks centre around the lifecycle of
certain actions such are |repo| creation, user creation etc. The actions
these hooks trigger can be rejected based on the API permissions of the
user calling them.
Those custom hooks can be called using |RCT|, see :ref:`rc-tools`. To create
a custom hook, see the :ref:`event-listener` section.
.. _event-listener:
Making your Extension listen for Events
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To create a hook to work with a plugin or extension,
you need configure a listener in the
:file:`/home/{user}/{instance-id}/rcextensions/__init__.py` file,
and use the ``load_extension`` method.
Use the following example to create your extensions.
In this example:
* The hook is calling the ``('my_post_push_extension.py')`` extension.
* The hook is listening to |RCM| for pushes to |repos|.
* This highlighted code is the hook, and configured in the ``__init__.py`` file.
* It is inserted into the ``def _pushhook(*args, **kwargs)`` section,
if it is not in the default ``__ini__.py`` file, use the below
non-highlighted section to create it.
.. code-block:: python
:emphasize-lines: 23-38
# ==========================================================================
# POST PUSH HOOK
# ==========================================================================
# this function will be executed after each push is executed after the
# build-in hook that RhodeCode uses for logging pushes
def _pushhook(*args, **kwargs):
"""
Post push hook
kwargs available:
:param server_url: url of instance that triggered this hook
:param config: path to .ini config used
:param scm: type of VS 'git' or 'hg'
:param username: name of user who pushed
:param ip: ip of who pushed
:param action: push
:param repository: repository name
:param repo_store_path: full path to where repositories are stored
:param pushed_revs: list of pushed revisions
"""
# Your hook code goes in here
call = load_extension('my_post_push_extension.py')
if call:
# extra arguments in kwargs
call_kwargs = dict()
call_kwargs.update(kwargs)
my_kw = {
'reviewers_extra_field': 'reviewers',
# defines if we have a comma
# separated list of reviewers
# in this repo stored in extra_fields
}
call_kwargs.update(my_kw) # pass in hook args
parsed_revs = call(**call_kwargs)
# returns list of dicts with changesets data
# Default code
return 0
PUSH_HOOK = _pushhook
Once your plugin and hook are configured, restart your instance of |RCM| and
your event listener will triggered as soon as a user pushes to a |repo|.

55
docs/extensions/install-ext.rst
View file

@ -1,55 +0,0 @@
.. _install-rcx:
Install |RCX|
-------------
To install |RCX|, you need to have |RCT| installed. See the :ref:`install-tools`
and :ref:`tools-cli` sections. Use the ``--plugins`` option with
the ``rhodecode-extensions`` argument.
Upgrading |RCX|
^^^^^^^^^^^^^^^
.. important::
You should back up any plugins or extensions that you have created before
continuing.
To upgrade your |RCX|, use the following example:
1. From inside the |RCT| virtualenv, upgrade to the latest version:
.. code-block:: bash
(venv)$ pip install -U https://my.rhodecode.com/dl/rhodecode-tools/latest
Downloading/unpacking https://my.rhodecode.com/dl/rhodecode-tools/latest
Downloading latest (143kB): 143kB downloaded
Running setup.py (path:/tmp/pip-9qYsxf-build/setup.py) egg_info
for package from https://my.rhodecode.com/dl/rhodecode-tools/latest
2. Once |RCT| are upgraded to the latest version, you can install the latest
extensions using the following example:
.. code-block:: bash
(venv)$ rhodecode-extensions --instance-name=enterprise-1 \
--ini-file=rhodecode.ini --plugins
Extension file already exists, do you want to overwrite it? [y/N]: y
Writen new extensions file to rcextensions
Copied hipchat_push_notify.py plugin to rcextensions
Copied jira_pr_flow.py plugin to rcextensions
Copied default_reviewers.py plugin to rcextensions
Copied extract_commits.py plugin to rcextensions
Copied extract_issues.py plugin to rcextensions
Copied redmine_pr_flow.py plugin to rcextensions
Copied extra_fields.py plugin to rcextensions
Copied jira_smart_commits.py plugin to rcextensions
Copied http_notify.py plugin to rcextensions
Copied slack_push_notify.py plugin to rcextensions
Copied slack_message.py plugin to rcextensions
Copied extract_jira_issues.py plugin to rcextensions
Copied extract_redmine_issues.py plugin to rcextensions
Copied redmine_smart_commits.py plugin to rcextensions
Copied send_mail.py plugin to rcextensions

151
docs/extensions/int-slack.rst
View file

@ -1,151 +0,0 @@
.. _slack-int:
Integrate Slack
===============
To integrate |RCE| and Slack, you need to configure some things on the Slack
side of the integration, and some things on the |RCE| side.
On the Slack side you need to allow incoming webhooks, see their
documentation on this, `Slack Webhooks`_. You will also need to get an
Authorization Token from Slack that will allow |RCE| to post to your account.
On the |RCE| side, this is an overview of what you need to do:
1. Configure the built-in Slack extensions to post to the correct Slack URL.
2. Set your Slack authentication details in the |RCX| :file:`__init.py__` file.
3. Configure the different hooks in the :file:`__init.py__` file to extract
whatever information you want from |RCE|, and then using the Slack extensions
post that information to your Slack channel.
.. hint::
The below examples should help you to get started. Once you have your
integration up and running, there is a more detailed Slack integration in
the :ref:`int-full-blown` section.
Configure Built-in Extensions
-----------------------------
|RCE| comes with 3 Slack extensions: ``slack_message.py``,
``slack_push_notify.py``, and ``slack.py``. The default
location is :file:`/home/{user}/.rccontrol/{instance-id}/rcextensions`.
To enable these to post to your Slack account, configure each of
these files with the following Slack details.
.. code-block:: python
BASE_URL = 'https://your.slack.com/api/link'
INCOMING_WEBHOOK_URL = 'https://hooks.slack.com/services/your/hook/link'
API_VERSION = 1
Configure |RCE| to Post to Slack
--------------------------------
In the |RCX| :file:`__init.py__` file, configure your Slack authentication
details. The default location is
:file:`/home/{user}/.rccontrol/{instance-id}/rcextensions`
.. code-block:: python
CONFIG = DotDict(
slack=DotDict(
api_key='api-key',
api_url='slack-incoming-hook-url',
default_room='#slack-channel',
default_plugin_config={},
),
)
# slack conf
CONFIG.slack.default_plugin_config = {
'INCOMING_WEBHOOK_URL': CONFIG.slack.api_url,
'SLACK_TOKEN': CONFIG.slack.api_key,
'SLACK_ROOM': CONFIG.slack.default_room,
'SLACK_FROM': 'RhodeCode',
'SLACK_FROM_ICON_EMOJI': ':rhodecode:',
}
Add Push Notifications to Slack
-------------------------------
To add notification to Slack when someone pushes to |RCE|, configure the push
hook to extract the commits pushed, and then call the built-in
``slack_push_notify.py`` extension to post them into your chosen Slack
channel. To do this, add the following code to the push hook section of the
:file:`__init.py__` file
.. code-block:: python
:emphasize-lines: 10-16,18-22
def _push_hook(*args, **kwargs):
"""
POST PUSH HOOK, this function will be executed after each push, it's
executed after the build-in hook that RhodeCode uses for logging pushes
kwargs available:
"""
# backward compat
kwargs['commit_ids'] = kwargs['pushed_revs']
# fetch pushed commits, from commit_ids list
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# slack !
call = load_extension('slack_push_notify.py')
if call:
kwargs.update(CONFIG.slack.default_plugin_config)
call(**kwargs)
return 0
PUSH_HOOK = _push_hook
Add Pull Request Notifications to Slack
---------------------------------------
To add |pr| notifications to Slack, use the following example. This example
shows a merged |pr| notification. You can add similar notifications to the
following hooks in the :file:`__init.py__` file, and for those examples see
the :ref:`int-full-blown` section:
* ``_create_pull_request_hook``
* ``_review_pull_request_hook``
* ``_update_pull_request_hook``
* ``_close_pull_request_hook``
.. code-block:: python
:emphasize-lines: 5-23
def _merge_pull_request_hook(*args, **kwargs):
"""
"""
# extract below from source repo as commits are there
kwargs['REPOSITORY'] = kwargs['source']['repository']
# fetch pushed commits, from commit_ids list
call = load_extension('extract_commits.py')
extracted_commits = {}
if call:
extracted_commits = call(**kwargs)
# store the commits for the next call chain
kwargs['COMMITS'] = extracted_commits
# slack notification on merging PR
call = load_extension('slack_message.py')
if call:
kwargs.update(CONFIG.slack.default_plugin_config)
kwargs['SLACK_ROOM'] = '#develop'
kwargs['SLACK_MESSAGE'] = 'Pull request <%s|#%s> (%s) was merged.' % (
kwargs.get('url'), kwargs.get('pull_request_id'), kwargs.get('title'))
call(**kwargs)
return 0
MERGE_PULL_REQUEST = _merge_pull_request_hook
.. _Slack Webhooks: https://api.slack.com/incoming-webhooks

53
docs/extensions/rcx.rst
View file

@ -1,53 +0,0 @@
.. _rc-ext:
|RCX|
-----
|RCX| add additional functionality for push/pull/create/delete |repo| hooks.
These hooks can be used to send signals to build-bots such as `Jenkins`_. It
also adds built in plugin and extension support. Once installed, you will see
a :file:`rcextensions` folder in the instance directory, for example:
:file:`home/{user}/.rccontrol/{instance-id}/rcextensions`
Built-in Plugins
^^^^^^^^^^^^^^^^
* A number of `Jira`_ plugins, enabling you to integrate with that issue
tracker: ``extract_jira_issues.py``, ``jira_pr_flow.py``,
``jira_smart_commits.py``
* A number of `Redmine`_ plugins, enabling you to integrate with that issue
tracker: ``extract_redmine_issues.py``, ``redmine_pr_flow.py``,
``redmine_smart_commits.py``.
* ``hipchat.py`` and ``hipchat_push.py`` enable you to integrate with
`HipChat`_ and set channel or user notifications.
* ``slack.py``, ``slack_message.py``, and ``slack_push_notify.py`` enable
you to integrate with `Slack`_ and set channel or user notifications.
Built-in Extensions
^^^^^^^^^^^^^^^^^^^
* ``commit_parser.py``: Enables you to parse commit messages,
and set a list of users to get notifications about change sets.
* ``default_reviewers.py``: Enables you to add default reviewers to a |pr|.
* ``extra_fields.py``: Returns a list of extra fields added to a |repo|.
* ``http_notify``: Enables you to send data over a web hook.
* ``mail.py`` : This extension uses the |RCE| mail configuration from the
instance :file:`rhodecode.ini` file to send email.
* ``push_post.py``: Enables you to set up push based actions such as
automated Jenkins builds.
Event Listeners
^^^^^^^^^^^^^^^
To enable the extensions to listen to the different events that they are
configured for, you need to also set up an event listener (hook). Event
listeners are configured in the
:file:`/home/{user}/.rccontrol/{instance-id}/rcextensions/__init.__.py` file.
For more details, see the example hook in :ref:`event-listener`.
.. _Jenkins: http://jenkins-ci.org/
.. _HipChat: https://www.hipchat.com/
.. _Slack: https://slack.com/
.. _Redmine: http://www.redmine.org/
.. _Jira: https://www.atlassian.com/software/jira