Managing Git branch level permissions with TeamForge and Gerrit

Since version 6.2,  TeamForge supports Git, using Gerrit as the backend. Gerrit is not only our Git server, it is also a great code review system.

In this blog post we will talk about Git branch level permissions and how to control them from TeamForge. Why would you like to use it in the first place? Well, with branch level permissions you can specify that certain groups of users can only read or push certain “refs” (branches or tags) but not others. There are two options for doing that:

  1. Use the custom repository category, which is turning off TeamForge autopilot and lets you, based on synched TeamForge project roles, control branch level access directly from Gerrit.
  2. Define your own repository category with different permissions per branch (or pattern to be more precise) and use it for multiple repositories.

You could ask: Why two options? Each option has its own use case. The first one is more suitable if your branch based permissions vary a lot from repository to repository. On the other hand, if you already managed (or consider) a common branch naming schema across repositories, the second option might be more appealing to you, since you don’t have to manually apply the same access rights fine tuning again and again.

Now let’s have an example which will show us how to use both options in real life.

But before we start, let’s address one question: what is a repository category? A short answer is that it is a mapping between TeamForge SCM permissions and Gerrit access rights. In other words it defines how Gerrit access rights are derived from Source Code Admin, Delete/View, Commit/View and View Only permissions from TeamForge. Those mappings are defined per repository –  that is, all branches of a given repository are affected.

In TeamForge you typically can choose from three pre-defined repository categories: default (no review), optional_review and mandatory_review. Those categories apply to all branches and cover most of the use cases we have seen at our customers. They do not require you to look into Gerrit’s Web UI at all, in other words, they are using TeamForge auto pilot.

There is a fourth category called custom. In this one only Source Code Admin is mapped to Gerrit OWN access right. All the other permissions are fine tuned by users belonging to administrator roles from Gerrit. Role definition and membership is still controlled from TeamForge.

Back to the example. We would like to fulfill the following:
- Some users (let’s call them observers) should only have read access to the master branch.
- Developers can read all branches but only push to development branches following a certain naming pattern. We will use refs/heads/devel/* for our example.
- Release managers can read and push to all branches

How to achieve that?

Regardless of which option we choose some preparation on TeamForge side is necessary.

We need to decide what SCM permissions from TeamForge to use. In our case it is quite obvious: we will use View only permission for observers. Commit/View for developers and Source Code Admin for release managers. But SCM permissions alone are not enough.  Just in case you didn’t know, in TeamForge users are not assigned directly to permissions but are assigned to roles, and roles have SCM permissions. That means we will have to create three project roles (with corresponding permissions). Let’s name them according to our requirements.

To create a role we go to TeamForge in Project Admin Menu and select Permissions.

Here how it should look after all three roles have been created.

 Managing Git branch level permissions with TeamForge and Gerrit

Do not forget to assign the permissions to the roles. Let’s edit developer role and add Commit/View in Source Code Permissions:
 Managing Git branch level permissions with TeamForge and Gerrit

Of course we need to add permissions to all three roles.

Eventually we will have: release manager (with Source Code Admin), developer (with Commit/View) and (observer with View only).

Last but not least, we will also need some users associated with those roles. Here it comes:
 Managing Git branch level permissions with TeamForge and Gerrit

Now, we are done with the preparation part and ready to talk about our two options.

Let’s consider the first one – using custom category and fine tuning from Gerrit.

First, we need to login into TeamForge as admin and create repository with [RepoCategory:custom] in our demo project. Of course you could also use an existing repository and add [RepoCategory:custom] to its description.
 Managing Git branch level permissions with TeamForge and Gerrit

After the new repository got created in TeamForge we go to Gerrit (embedded into TeamForge)   and choose the project in question. Please note that it is a Gerrit project so it has the name of its corresponding TeamForge repository. In our case it is option1. We can see that Lucy, our admin, who belongs to TeamForge’s project role release manager, has access to option1 project and that she is able to add access rights to it:
 Managing Git branch level permissions with TeamForge and Gerrit

[Please note that I unchecked the “Show Inherited Rights” checkbox to remove unrelated rights.]

Now it is time to add the necessary rights for the other TeamForge project roles one by one. As long as you are member of a TeamForge project role, Gerrit will even provide auto completion. Let’s start with the TeamForge observer role:

 Managing Git branch level permissions with TeamForge and Gerrit

We continue to add required rights until our access rights table looks like this:
 Managing Git branch level permissions with TeamForge and Gerrit

As we can see here release manager has admin rights, developer has read only access to everything besides refs/heads/devel/* where he can also push and observer has read only access to refs/heads/master.

As stated before, TeamForge will not override any of your access rights if you selected the custom category, so you can setup different configurations for every single project role. Let’s say, you had two different developer roles, both with Commit/View in TeamForge but one group should push to refs/heads/devel1/* and the other to refs/heads/devel2/*, you can just change this ad-hoc by using different ref sepcs for the two project roles synched to Gerrit. Of course it is also possible to have different settings for other Gerrit categories like review, submit and verify. While this allows you the most flexibility possible, it is also the most demanding approach since Gerrit access right are quite hard to understand and beginners might be confused by their complexity. This is why most projects typically start with predefined categories like default, optional_review or mandatory_review which do not require you to enter Gerrit Web UI at all. To match the basic requirements of all our customers those pre-shipped categories apply to all branches.  If you like to see how you can define your own category with Git branch level permissions, read on!

Now we will focus on the second option – using a new repository category, which can be used across multiple repositories.
Let’s see how to implement it in this case:

First we define our new category. We will call it branch_based.
Let’s edit the gerritforge.mappings file located in the etc directory of the ctf-git-integration installation.
Typically it will be located under /opt/collabnet/gerrit/etc/gerritforge.mappings
By default, it contains the definitions for the out of the box categories default, mandatory_review, optional_review and custom. We will copy the default category definition and modify it according to our needs. In our case we only have to modify the properties corresponding to View Only (scm_view) and Commit/View (scm_commit) to add restrictions on certain refs. For more details on the exact format of this file, please refer to our README typically located under /opt/collabnet/gerrit/doc/README.pdf or at http://help.collab.net.

Let’s have a look at our new properties that should be added to the existing ones:

# branch based:
branch_based.scm_admin.1=READ,1,3
branch_based.scm_admin.2=pHD,1,3
branch_based.scm_admin.3=pTAG,1,2,refs/tags/*
branch_based.scm_admin.4=FORG,1,3
branch_based.scm_admin.5=OWN,1,1

branch_based.scm_delete.1=READ,1,3
branch_based.scm_delete.2=pHD,1,3
branch_based.scm_delete.3=pTAG,1,2,refs/tags/*
branch_based.scm_delete.4=FORG,1,2

branch_based.scm_commit.1=READ,1,1
branch_based.scm_commit.2=READ,1,3,refs/heads/devel/*
branch_based.scm_commit.3=pHD,1,3,refs/heads/devel/*

branch_based.scm_view.1=READ,1,1,refs/heads/master

branch_based.none.1=READ,-1,-1

branch_based.keep_rights_added_in_gerrit=false

And after restarting Gerrit integration with the following command:

$ service gerrit restart

we are ready to use our newly defined mappings.
Let’s go to our demo project in TeamForge and create a git repository which will use our mappings:

 Managing Git branch level permissions with TeamForge and Gerrit

What is important to note here is the part: [RepoCategory:branch_based] in the description field. This tells Gerrit to use our new mappings for this repository. The category name has to exactly match the one we used in our properties definition in gerritforge.mappings file (it is branch_based in our case).

We now go into Gerrit again and have a look at the access rights for option2:

 Managing Git branch level permissions with TeamForge and Gerrit

Looks familiar? Yes, it is exactly the same configuration as in the first option, just realized differently with the potential to use it across multiple TeamForge Git repositories. You can also switch between categories if you change the repository description.

Now it is time to verify that our settings worked. We will do that for the first option, leaving the second one as an exercise to the reader.

First, let’s try with Joe, our developer. After cloning and preparing one commit the git status should give us something like this:

$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
nothing to commit (working directory clean)

So it is time to check if Joe is able to push to master:

$ git push origin HEAD:master
Total 0 (delta 0), reused 0 (delta 0)
To ssh://joe@tf62personal:29418/option1
! [remote rejected] HEAD -> master (prohibited by Gerrit)
error: failed to push some refs to 'ssh://joe@tf62personal:29418/option1'

So far so good, as this attempts have failed as expected.

Now, let’s try to create a new branch under refs/heads/devel/:

$ git push origin HEAD:devel/branchOne
Counting objects: 4, done.
Writing objects: 100% (3/3), 237 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To ssh://joe@tf62personal:29418/option1
* [new branch]      HEAD -> devel/branchOne

Great, it worked. Now let’s check if Robin, the observer has read-only access to master.
A good way of checking this would be cloning the repository.
Check if git clone works:

$  git clone ssh://robin@tf62personal:29418/option1
Cloning into 'option1'...
remote: Counting objects: 5, done
remote: Finding sources: 100% (5/5)
remote: Total 5 (delta 0), reused 3 (delta 0)
Receiving objects: 100% (5/5), done.

Yes it worked, so the read permission works as expected.
So let’s see which branches are visible:

$ git ls-remote origin
1d3b415c5c1ea3c2c90318ef7410096d39b8add0    HEAD
1d3b415c5c1ea3c2c90318ef7410096d39b8add0    refs/heads/master

We cannot see any other branches here, so it seems Robin can access only master.
The same command run by Joe (our developer) gives us the following:

$ git ls-remote origin
1d3b415c5c1ea3c2c90318ef7410096d39b8add0    HEAD
3ac13ac746a6d11a74843d2d4fd33fec0551a600    refs/heads/devel/branchOne
1d3b415c5c1ea3c2c90318ef7410096d39b8add0    refs/heads/master

We can see here the development branch branchOne which was not available to Robin.

Now let’s try to prepare a commit and push as Robin (observer):

$ git push origin HEAD:master

fatal: Upload denied for project 'option1'
fatal: The remote end hung up unexpectedly

And for another branch:

$ git push origin HEAD:devel/branchTwo

fatal: Upload denied for project 'option1'
fatal: The remote end hung up unexpectedly

As you can see it didn’t work and that’s good because it was expected like that.

Settings for release manager are equivalent to default admin settings so it is safe to assume that it just works icon smile Managing Git branch level permissions with TeamForge and Gerrit But for the sake of completeness we will push to master as Lucy:

$ git push origin HEAD:master
Counting objects: 4, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 286 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To ssh://lucy@tf62personal:29418/option1
a56343f..0f299c2  HEAD -> master

It worked as expected.

Last thing to consider is what will happen if we work with more than one repository. If we use the first option (category custom)  you will have to configure access rights in Gerrit for every repository and TeamForge project role separately. With the second option (defining your own category), all repositories using it will automatically follow your templatized access right configuration. On the other hand, they have to share the same branch naming conventions and you will not be able to make differentiations based on individual TeamForge roles (but only on TeamForge SCM access rights).

So, let’s summarize: branch based permissions are working out of the box on Gerrit and there are two different ways of configuring it. First one is to use the predefined repository category custom which is turning off TeamForge autopilot and lets you, based on synched TeamForge project roles, control branch level access directly from Gerrit.
The second, requires you to define your own repository category, which might sound more work intense initially. However, once defined, repository category can be used across multiple Git repositories managed in TeamForge. It is also possible to use more than one repository category definitions in one TeamForge project (one per repository) if you cannot narrow down to one branching naming pattern or require different Gerrit access rights per TeamForge SCM permission.

Do you already have your own repository categories defined? Which naming patterns do you use for your branches? Or maybe you have your own way how to deal with branch based permissions? Please share that knowledge with us by commenting here. Follow up questions appreciated.

 

Tagged with: , , , , , , , , , , , ,
Posted in Git, TeamForge
21 comments on “Managing Git branch level permissions with TeamForge and Gerrit
  1. Great blog post, just wanted to point out that webinar series is starting today where you will learn how to use those categories to marry Jenkins with Gerrit properly: http://visit.collab.net/WebinarPromoLPs_PromoLPRegAll.html

  2. Jonas Christensen says:

    Hi,

    A few comments and questions:

    1) If I create a new category in gerritforge.mappings, and mention it in the Description, which Repository Category to choose then? Default? Other? Or doesn’t it matter?

    2) When I create a new repo the parent is default set to “– All Projects –”. In a native Gerrit installation I can choose, at creation time, which project to set as parent. This is not possible in the CTF integration. As I find the category handling in gerritforge.mappings a bit cumbersome, have you any plan to implement the ability to manually choose parent at creation time? Or, the ability to change parent to other than “– All Projects –”? This will allow for a very common Gerrit setup where you create a “access control” repo which only serve as parent for all your code repos.

    Br,
    Jonas Christensen
    Schneider Electric

    • Hi Jonas,

      regarding 1): Suppose your new category is called foo, you would have to mentioned it in the repo description like [RepoCategory:foo] if you are using TeamForge 6.2 If you are using a newer version, you would select “Other” as category in the repository settings and enter foo in the text box.

      regarding 2): At the moment, all Gerrit projects are direct children of –All projects–. We have an item in our roadmap to rather use the TeamForge project hierarchy instead. It is not our goal to allow arbitrary parent/child project relationships as in our opinion this undermines one of the main value props of our integration: Most end users are overwhlemed with Gerrit’s access rights (in our newest supported version, there are 24 differenct access rights categories). We have often seen that people feel unsure on how to properly configure those access rights and not accidentally expose their source code to a broader audience than anticipated. Think about it like an electro oven with 24 different knobs and your job is to just warm up your pizza. You started turning those knobs until your Pizza is warmed up but do not realize that by turning on the second knob, you also activated something different until your apartment is on fire (== you managed to get your team access to your Gerrit project but did not realize you granted everybody permissions because one obscure detail about rule inheritance was misunderstood). Our goal is to shield most users from Gerrit’s complexities by using predefined repo categories and rather generic TeamForge SCM permissions without hiding advanced features for power users. More experienced end users can still configure things themselves by switching off the TeamForge auto pilot “custom category” and fine tune things themselves in the Gerrit Web UI. Allowing arbitrary parent child relationships will introduce another layer of complexity that due to its right inheritance nature will also affect repositories of not so experienced end users. To strike the balance between those two user groups, we will probably only allow a hierarchy as defined in TeamForge, so that our support and casual Gerrit users still have a chance to find out what is actually going on and how to fix things if broken.

  3. Jonas Christensen says:

    Hi Johannes,

    Thanks for your explanation. I have a few comments.

    Is it possible to use a CTF role in a repository category, e.g. instead of “branch_based.scm_commit.3=pHD,1,3,refs/heads/devel/*” then use “branch_based.role2547.3=pHD,1,3,refs/heads/devel/*” ??

    If not, then I can’t see how your repository category implementation is usable in a standard CI/CD setup where you have developers which are allowed to commit to some branches (e.g. develop), integrators which are able to commit to other branches (e.g. release), and ci robots which are able to commit to pristine branches (e.g. master). Because they all have “scm_commit” permission, but need different branch access rights.

    I know that in this case the “custom category” can handle such a setup, but with 20+ or 100+ or more repositories it gets really tiresome to setup custom access controls for each repo, especially when aware of the fact that a simple change of parent would solve it.

    I also know that, at least in Gerrit 2.6 coming with CTF 7.1, it is possible to push a “groups” and “project.config” file to each repo instead of manually changing access controls in the web gui, but still you would have to push those files to all your repos each time you change access control settings. Instead of changing it once in the parent project.

    So, please think about how you can support medium and large scale companies (in terms of number of repositories) to configure access controls for a standard CI/CD setup.

    Br,
    Jonas Christensen
    Schneider Electric

    • Hi Jonas,

      our mappings file does not support any references to roles as we wanted it to be applicable across TeamForge projects. I completely get your CI use case. Your proposed solution of pushing that config into refs/meta/config would make sense and I also agree with you that parent/child project hierarchies would help here (see my response above, it is in the road map).

      Independent of both options, here are some thoughts what you can do (independent of Gerrit 2.1/2.6):
      * you can create internal Gerrit groups and assign those to Gerrit projects/access rights
      * you can assign access rights to TeamForge user groups (like CI users)
      * when access rights change in TeamForge, we will not touch configurations for internal Gerrit groups or TeamForge user groups (independent of repo category selected)
      * internal Gerrit groups can include both TeamForge project roles as well as TeamForge project groups
      * internal Gerrit groups and TeamForg user groups can be assigned on the – All projects — level as well
      * both Gerrit 2.1 and 2.6 support exclusive ref spects (in Gerrit 2.6 there is a check box, in Gerrit 2.1 there is a – in front of the refspec).

      With that model, you could put your CI users in a TF user group or internal Gerrit group and grant them PHD+3 access to -refs/heads/master (exclusive push permissions only for CI)

      If you wanted to extend certain TF project roles with access to a secret branch, create an internal Gerrit group (secret), include the project roles in question and give them access to refs/heads/secret

      Last but not least, Eryk is going to publish a new blog post series very soon how to handle a variant of gitflow with very similar access restrictions as the one you just described. The idea here is that developers get View&Commit rights in TeamForge, integrators would get View&Commit+Delete/View and CI would get SCM Admin. With that approach, you can actually work with a user defined repo category.

  4. Jonas Christensen says:

    Hi Johannes,

    Currently we are using TF user roles for Developer, Integrator, CI robot, CI admin with custom access controls. Using internal Gerrit groups would make access granting a bit less straight forward, I think.

    Anyway, the setup is manual with custom access controls, no matter if you use TF roles or TF groups, correct, or have I misunderstood some feature when using TF groups?

    The way to ease the access control maintenance is some usage of a parent. As you propose, a solution could be to use the 3 different source code permission categories, and create a user defined repo category. The restriction would be only 3 different roles though.

    I’m unsure exactly how it would work if you implement project hierarchy to determine repository parent in terms of access rights inheritance. The project I’m working in has a number of repositories, but there are no parent/child relationship, they are all on the same level. Or is it the “top level” TF project which needs to have a parent, and then the repositories in the child TF project(s) can inherit the access controls?

    Br,
    Jonas

    • >Anyway, the setup is manual with custom access controls, no matter if >you use TF roles or TF groups, correct, or have I misunderstood some feature when using TF groups?

      Manual in the sense of not completely controlled by TeamForge unless you push your additonal access rights in refs/meta/config during repo creation with a custom event handler: http://blogs.collab.net/teamforge/custom-workflow-in-teamforge-a-guide-to-iaf-event-handlers

      >The restriction would be only 3 different roles though.
      Well 5 “role types” are possible: no access, view access, commit&view, delete&view, scm admin

      >I’m unsure exactly how it would work if you implement project hierarchy to determine repository parent in terms of access rights
      Step one will be that all Git repositories within the same TeamForge project would get a new Gerrit parent project which is named after the TeamForge project they are in. You would not be able to commit anything but config changes into that parent project but can use it to manually set access rights (for certain internal and TF managed Gerrit groups) which will apply to all git repos of the same TF project.

      Step two would be, to have this parent Gerrit project also have a parent other than –All projects — if the TeamForge project in question had a TF parent project. With that approach you can define access patterns that inherit to all Git repos of any TF project with the same parent.

      We are currently spiking on step one (highest priorities for Q1 are to finally release our advanced Gerrit notification plugin (ability to get same messages as git-multi-mail sends out into TF forums) and the upgrade to Gerrit 2.7 though), it does not seem unlikely that we can have this one in early Q2.

  5. Jonas Christensen says:

    Ok Johannes, seems your “step one” solution will do the trick, looking forward to giving it a spin ;o)

    Br,
    Jonas

    • If you already wanted to give it a spin now:

      * once our integration created a project, it will not try to change its parent project again
      * there is a Gerrit command to change parent projects yourself: http://gerrit.googlecode.com/svn/documentation/2.6/cmd-set-project-parent.html and https://gerrit-documentation.googlecode.com/svn/Documentation/2.6/rest-api-projects.html#set-project-parent

      It will require Gerrit Administrator permissions though as selecting a different parent than –All Projects — (even at project creation time) is an operation which in our opinion requires more privileges than just having SCM Admin (as you could reparent a repo into an entire different part of the hierarchy and by that strip access rights away from other SCM Admins).

      Knowing that, you could already achieve what you want today, if you
      * create a Git repo in TeamForge with custom category and use this as an “umbrella Gerrit project”
      * reparent the Gerrit projects of your choice so that they can inherit permissions from there

      We tested that with our Gerrit 2.6.1 based integration and it worked as expected, don’t see any reason why it would not work with Gerrit 2.1.10

  6. Jonas Christensen says:

    Hi Johannes,

    Well, now I’m confused …

    In my first blog post I ask if it is possible to “change parent to other than “– All Projects –””. And from your answer it seems that that is not possible.

    I have also asked our CTF contact here at Schneider Electric about if it was possible to change parent using “cmd set project parent”, but was told that that was not possible (from a CTF employee).

    So, a bit confused about you now stating that it is indeed possible to change the parent using “cmd set project parent” … but, great if it works, this has been my preferred solution all along ;o)

    Br,
    Jonas

    • Hi Jonas,

      my development team has tested what happens if you use that command with Gerrit 2.6.1 and could not spot any problems. We also think it will work with Gerrit 2.1.10 but have not tested this.
      From official support, you will only get statements regarding scenarios we officially qualified. This prevents unexperienced customers getting into situations where we cannot help them recover from it any more.

      That said, I think your knowledge of Gerrit is strong enough to test this approach in a sandbox project first and it works, apply to mission critical projects. If you want an official statement that we support parent/child hierarchies, you will have to wait at least until April this year (and this statemengt will only apply to the hierarchies we set up programaticcaly with Gerrit 2.7).

      Best, Johannes

  7. Jonas Christensen says:

    Hi Johannes,

    Thanks for clarifying :)

    I will test “cmd set project parent” with our Gerrit 2.1.10, but as you also state, it should work. I just need to find out who is Gerrit Administrator ;o)

    And then I will look forward to getting the Gerrit 2.7 CTF integration and test how the parent handling is implemented in details.

    Br,
    Jonas

  8. veer says:

    Hi,

    Wanted to check if it is possible to set a role by name lets say as tester, and set reference name as /refs/heads/test*. Basically intent is to create a role who can create branches and push to TF repo, if the branch name is test1 or test2 etc – without having to create role per branch.

    So, what I understand is we can create role and allow user to push to specific branch, can we generalize that by using regular expressions. If not, any other way to give non admin users permissions to create and push new branches to TF repo, without involvement of the admin?

    Thanks,
    Veer

    • This is definitely possible, see the reference to the custom repo category in this post or have a look at Eryk’s more recent Gerrit 2.6 post.

    • Jonas Christensen says:

      Hi Veer,

      Yes, by using “Custom Repository Category” you can setup a reference as refs/heads/test/* (this I know works, I haven’t tested refs/heads/test*).

      And then the role Tester just needs Create Reference, Push and Push Merge (if you allow merge commits) on that reference. This will allow them to push directly without review.

      And if you would like to enable your Tester’s to use review for their commits, add a reference refs/for/refs/heads/test/* with Push and Push Merge for them.

      Br,
      Jonas Christensen (Not CollabNet employee ;o)

  9. Eryk Szymanski says:

    Hi,

    that’s correct. The way to go is to create refs/heads/test/*. Actually it will not work with refs/heads/test* (so you need a ‘/’ before the ‘*’).

    You may also consider to add a personal namespace for each user. This is done by using the special ${username} keyword: refs/heads/test/${username}/*. If you want to do that, consider to give push force permission on that refs so the users can clean-up their own branches.

  10. Satish says:

    Thanks for these detailed analysis…
    would it be possible to change the access level for a specific branch in a project from the command line ?

  11. We already support Gerrit 2.7 with TeamForge 7.1, have a look at the more recent blogs on http://blogs.collab.net/ and our online help on http://help.collab.net

    Best, Johannes

Leave a Reply

Your email address will not be published. Required fields are marked *

*

CAPTCHA Image

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

connect with CollabNet
   Contact Us
sign up for emails
looking for something
conversations

CollabNet: .@rsabbagh: #CST will be teaching a Certified #ScrumMaster course in Toronto, ON Sept 25-26! Don't forget to register! http://t.co/xCuQ0wP4dr
Date: 19 September 2014 | 10:06 pm

CollabNet: Executive #DevOps workshop in NJ - Engineering & management practices using the most popular #OpenSource dev tools http://t.co/hCE4CSz0IK
Date: 19 September 2014 | 2:00 pm

CollabNet: Any Exec or Mgr interested in learning how #DevOps is working for other large organizations, should join this webinar http://t.co/WNtsLBX5F3
Date: 18 September 2014 | 10:30 pm