release: merge back stable branch into default

docs/integrations/ci.rst

@@ -16,7 +16,7 @@ General Webhook
 +++++++++++++++
 
 :ref:`integrations-webhook` allows sending a JSON payload information to specified
-url with GET or POST methods. There are several variables that could be used
+custom URL with GET or POST methods. There are several variables that could be used
 in the URL greatly extending the flexibility of this type of integration.
 
 Most of the modern CI systems such as Jenkins, TeamCity, Bamboo or CircleCi
@@ -50,7 +50,7 @@ the url would look like that::
 
     Please note that some variables will result in multiple calls.
     e.g. for |HG| specifying `${branch}` will trigger as many builds as how
-    many branches the suer actually pushed. Same applies to `${commit_id}`
+    many branches the user actually pushed. Same applies to `${commit_id}`
     This will trigger many builds if many commits are pushed. This allows
     triggering individual builds for each pushed commit.
 
@@ -66,6 +66,9 @@ calling such example url provided in :ref:`integrations-webhook` integration::
 
     http://server/job/${project_id}/build-branch-${branch}/buildWithParameters?token=TOKEN&PARAMETER=value&PARAMETER2=value2
 
+The |RCEE| includes a custom :ref:`integrations-jenkins` integration. That allows using some
+of Jenkins added security features. Please check also the above link for more examples.
+
 
 Team City
 +++++++++

docs/integrations/integrations.rst

@@ -19,7 +19,8 @@ Type/Name                       |RC| Edition    Description
 :ref:`integrations-webhook`     |RCCEshort|     POST events as `json` to a custom url
 :ref:`integrations-ci`          |RCCEshort|     Trigger Builds for Common CI Systems
 :ref:`integrations-email`       |RCCEshort|     Send repo push commits by email
-:ref:`integrations-redmine`     |RCEEshort|     Close/Resolve/Reference redmine issues
+:ref:`integrations-jenkins`     |RCEEshort|     Trigger Builds for Jenkins CI System
+:ref:`integrations-redmine`     |RCEEshort|     Close/Resolve/Reference Redmine issues
 :ref:`integrations-jira`        |RCEEshort|     Close/Resolve/Reference JIRA issues
 ============================    ============    =====================================
 
@@ -53,3 +54,4 @@ See pages specific to each type of integration for more instructions:
    webhook
    email
    ci
+   jenkins

docs/integrations/jenkins.rst

@@ -0,0 +1,113 @@
+.. _integrations-jenkins:
+
+Jenkins integration
+===================
+
+.. important::
+
+    Jenkins integration is only available in |RCEE|.
+
+
+
+Below are few examples how to use dedicated Jenkins integrations with |RCEE|.
+
+
+
+Configure Jenkins to check each commit after push
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In order to configure Jenkins to trigger a build on a project after each push we
+need to create a project that will allow triggering a build from external sources.
+
+In Jenkins, check `Trigger builds remotely (e.g., from scripts)` option. This gives
+now ability to trigger builds using an URL.
+
+.. image:: ../images/jenkins-build-with-parameters.png
+
+We also need to specify the parameters we're going to build the project with.
+In this example we'll use build with 3 parameters `COMMIT_ID`, `BRANCH_HEAD` and `BRANCH`.
+In Jenkins we need to add those 3 parameters, and check an option
+`This project is parameterised`
+
+.. image:: ../images/jenkins-adding-build-parameters.png
+
+
+Our final URL that |RCEE| Jenkins integration would use could look like this:
+
+This URL will be triggered for each pushed branch, and will include the branch head commit id.
+
+https://jenkins.server.com/job/test-build-commits/buildWithParameters?token=secret&BRANCH=${branch}&BRANCH_HEAD=${branch_head}
+
+Or we can also use commit_id in this case URL will be triggered for each pushed commit in the branch, and will include the commit id.
+
+https://jenkins.server.com/job/test-build-commits/buildWithParameters?token=secret&BRANCH=${branch}&COMMIT_ID=${commit_id}
+
+
+Now we have all the required information to fetch/checkout and test our project.
+We can write an example build common in Jenkins.
+
+
+.. image:: ../images/jenkins-build-command.png
+
+
+Below is an example of the integration page we used on |RCEE| side.
+
+
+.. image:: ../images/rhodecode-jenkins-integration-settings.png
+
+
+
+Configure Jenkins to check each pull-request on creation and updates
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Jenkins integration also allows checking created pull-requests inside |RCCE|. This is
+somehow a special case compared to testing commits. The pull-requests cannot be cloned
+from the repository. Instead |RCCE| exposes different links back to CI systems which
+store information on how to obtain the pull-request code.
+
+On |RCCE| side we need to create a new integration that would use only two events:
+
+ - pullrequest created
+ - pullrequest commits updated
+
+This will trigger the builds when pull-request is firstly created, and then each time
+pull-requests is updated and commits that it stores change.
+
+
+In Jenkins, we should create at least two new build string parameters.
+We could use `PR_ID` and `SHADOW_URL`.
+Using those parameters, we can then create a customized build command to checkout a
+merged state of pull-request. And later on using rhodecode-api notify about the checks
+using the `PR_ID`.
+
+
+.. image:: ../images/jenkins-pr-build-command.png
+
+
+There's even a helper script we created to send notifications.
+Here's how the build on success could look like in Jenkins.
+
+.. code-block:: bash
+
+    # clone the ci-notify script, and save it into our $BUILD_TAG/notify directory
+    hg clone https://code.rhodecode.com/ci-notify $BUILD_TAG/notify
+
+    # execute the script. $RC_TOKEN is auth-token for API
+    # url is your rhodecode instance, repoid is repository project which PR was opened
+    # status could be "approved" or "rejected"
+    python $BUILD_TAG/notify/notify.py \
+        --token $RC_TOKEN \
+        --url https://code.server.com \
+        --repoid my-example-project \
+        --prid $PR_ID \
+        --status "approved" \
+        --message "Build Succeeded: $BUILD_URL"
+
+
+Below is also an example that we could use on |RCCE| side in the Jenkins integration settings.
+A Jenkins Trigger url that exposes all required information back to Jenkins CI server:
+
+.. code-block:: bash
+
+    http://jenkins.server.com/job/my-project-pull-request-tests/buildWithParameters?token=secret&PR_ID=${pull_request_id}&PR_TITLE=${pull_request_title}&SHADOW_URL=${pull_request_shadow_url}&PR_UID=${pull_request_commits_uid}
+

docs/release-notes/release-notes-4.12.4.rst

@@ -0,0 +1,47 @@
+|RCE| 4.12.4 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2018-07-13
+
+
+New Features
+^^^^^^^^^^^^
+
+
+
+General
+^^^^^^^
+
+
+
+Security
+^^^^^^^^
+
+
+
+Performance
+^^^^^^^^^^^
+
+
+
+Fixes
+^^^^^
+
+- Repository: fixed problems with mercurial/git url validators.
+- User Groups: fixed audit log data on user group permissions view.
+  This caused an 500 error when editing user groups.
+- Authentication: fixed problem with displaying social auth providers in the login page.
+- Google Authentication: updated google user info api to latest version. The previous
+  endpoint was deprecated and it caused problems with Google authentication.
+- API: fixed problem with setting recursive permissions changes to repo group.
+- API: fixed problem with creation of pull request with custom reviewers rules.
+- API: fixed problem with proper diff calculation with using pull-request API.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- Scheduled release addressing reported problems, and improving stability.

docs/release-notes/release-notes.rst

@@ -9,6 +9,7 @@ Release Notes
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.12.4.rst
    release-notes-4.12.3.rst
    release-notes-4.12.2.rst
    release-notes-4.12.1.rst

rhodecode/integrations/types/base.py

@@ -124,6 +124,7 @@ class EEIntegration(IntegrationTypeBase):
 
 
 # Helpers #
+# updating this required to update the `common_vars` as well.
 WEBHOOK_URL_VARS = [
     ('event_name', 'Unique name of the event type, e.g pullrequest-update'),
     ('repo_name', 'Full name of the repository'),
@@ -134,8 +135,9 @@ WEBHOOK_URL_VARS = [
     ('extra:<extra_key_name>', 'Extra repo variables, read from its settings.'),
 
     # special attrs below that we handle, using multi-call
-    ('branch', 'Name of each brach submitted, if any.'),
-    ('commit_id', 'Id of each commit submitted, if any.'),
+    ('branch', 'Name of each branch submitted, if any.'),
+    ('branch_head', 'Head ID of pushed branch (full sha of last commit), if any.'),
+    ('commit_id', 'ID (full sha) of each commit submitted, if any.'),
 
     # pr events vars
     ('pull_request_id', 'Unique ID of the pull request.'),
@@ -173,11 +175,13 @@ class CommitParsingDataHandler(object):
                 _branch = branch_data[commit_branch] \
                     if commit_branch else commit_branch
                 branch_commits = {'branch': _branch,
+                                  'branch_head': '',
                                   'commits': []}
                 branches_commits[commit_branch] = branch_commits
 
             branch_commits = branches_commits[commit_branch]
             branch_commits['commits'].append(commit)
+            branch_commits['branch_head'] = commit['raw_id']
         return branches_commits
 
 
@@ -218,10 +222,17 @@ class WebhookDataHandler(CommitParsingDataHandler):
 
         branches_commits = self.aggregate_branch_data(
             data['push']['branches'], data['push']['commits'])
-        if '${branch}' in url:
+        if '${branch}' in url or '${branch_head}' in url:
             # call it multiple times, for each branch if used in variables
             for branch, commit_ids in branches_commits.items():
                 branch_url = string.Template(url).safe_substitute(branch=branch)
+
+                if '${branch_head}' in branch_url:
+                    # last commit in the aggregate is the head of the branch
+                    branch_head = commit_ids['branch_head']
+                    branch_url = string.Template(branch_url).safe_substitute(
+                        branch_head=branch_head)
+
                 # call further down for each commit if used
                 if '${commit_id}' in branch_url:
                     for commit_data in commit_ids['commits']: