{"id":23,"date":"2010-02-11T08:38:04","date_gmt":"2010-02-11T08:38:04","guid":{"rendered":"http:\/\/clayb.net\/blog\/?p=23"},"modified":"2012-05-04T19:37:01","modified_gmt":"2012-05-04T19:37:01","slug":"mercurial-learning","status":"publish","type":"post","link":"https:\/\/clayb.net\/blog\/mercurial-learning\/","title":{"rendered":"Mercurial Learning"},"content":{"rendered":"<h1>Why make folks do new things?<\/h1>\n<p>I realized that despite the importance for software development, many folks are largely unaware of the features and power their S(ource) C(ode) M(anagement) tool can provide them. In my group, we use <a href=\"http:\/\/mercurial.selenic.com\/\">Mercurial<\/a>. It is an awesome SCM and perfect for our team (which is largely Python writing) as it is written in Python.<br \/>\n<!--more--><\/p>\n<p>However, as with any tool under development, there are features which were not mature when we started using Mercurial, or otherwise, we have just never taken advantage of some features despite their obvious use.<\/p>\n<p>For us, we have a release of OpenSolaris every six months roughly and to prevent stopping all non-release development, we will fork our source base into a release and development (non-release targeting) code base. The first time I did this, as the gatekeeper (what we call the team&#8217;s build maintainer and release engineer) I simply created a second Mercurial repository &#8212; this was for the OpenSolaris 0906 release. This repository mirrored our <a href=\"http:\/\/src.opensolaris.org\/source\/xref\/caiman\/slim_source\/\">slim_source<\/a> Mercurial repository. All was well.<br \/>\nThis approach illustrated three areas of uncertainty with developer understanding, however:<\/p>\n<ul>\n<li>Exporting changes from a repo<\/li>\n<li>Importing changes to a repo<\/li>\n<li>Viewing and understanding a file&#8217;s SCM history<\/li>\n<\/ul>\n<p>Further, this required the team to hold a dead code base around (the now dead <a href=\"http:\/\/src.opensolaris.org\/source\/xref\/caiman\/slim_0906\/\">slim_0906<\/a> repository for the similarly named release). If we want to know what changes made that release we must <a href=\"http:\/\/mercurial.selenic.com\/wiki\/QuickStart#line-19\">clone<\/a> in and go through it trying to compare it to our main-line code base <a href=\"http:\/\/src.opensolaris.org\/source\/xref\/caiman\/slim_source\/\">slim_source<\/a>.<\/p>\n<p>This was awesome to know, for as a gatekeeper one becomes far more intimate with the SCM than most developers need to be. But it also questioned the power lurking underneath the SCM. For Mercurial can address all these issues easily:<\/p>\n<ul>\n<li>Exporting changes from a repo &#8212; use <tt>hg export<\/tt><\/li>\n<li>Importing changes to a repo &#8212; use <tt>hg import<\/tt><\/li>\n<li>Viewing and understanding a file&#8217;s SCM history &#8212; use <tt>hg glog<\/tt> or <tt>hg view<\/tt><\/li>\n<li>Forking a source code base for a project &#8212; use <tt>hg branches<\/tt><\/li>\n<\/ul>\n<h1>The idea<\/h1>\n<p>Given these issues and the potential fixes, I figured I should try and help the developer community using our code base learn these new tools. Further, trying to automate as much as I can, using Mercurial <a href=\"http:\/\/mercurial.selenic.com\/wiki\/NamedBranches\">named branches<\/a> was a key thing to keeping all of our source code in one repository for later reference and analysis. Though, it meant going beyond established OpenSolaris territory of Mercurial use as to my knowledge no other team is using named branches on hg.opensolaris.org yet.<\/p>\n<h1>Issues<\/h1>\n<p>Named branches caused some issues!<\/p>\n<p>First, the OpenSolaris developer environment uses community-developed extensions called Cadmium. These are to mimic some tools which Sun&#8217;s previous TeamWare software provided as the Solaris SCM of old (such as our recommit extension). Other features are simply to establish standard operating procedure for development (like webrev&#8217;s as can be seen at <a href=\"http:\/\/cr.opensolaris.org\">cr.opensolaris.org<\/a>). These Cadmium extensions were developed assuming no-one would be working on branched code bases.<\/p>\n<p>Luckily, since we rely on Cadmium&#8217;s recommit the <a href=\"http:\/\/hub.opensolaris.org\/bin\/view\/Project+pkg\/\">IPS<\/a> folks had talked about the issues of using branches before. <a href=\"https:\/\/auth.opensolaris.org\/info\/userProfile.action?userId=5678\">Rich Lowe<\/a> in particular already even <strike>had a potential fix for <tt>hg recommit<\/tt> ready for <a href=\"http:\/\/cr.opensolaris.org\/~richlowe\/onnv_6922488\/\">codereview<\/a>! Putting this in place until it is accepted into the OpenSolaris SUNWonbld package simply required sending a <a href=\"http:\/\/opensolaris.org\/jive\/thread.jspa?messageID=457427\">heads-up<\/a> to the slim_source developing community to install the updated Cadmium files.<\/strike> has integrated into build 134 and later (see the changeset log for <a href=\"http:\/\/dlc.sun.com\/osol\/on\/downloads\/b134\/on-changelog-b134.html\">build 134<\/a>) a fix for bug <a href=\"http:\/\/bugs.opensolaris.org\/bugdatabase\/view_bug.do?bug_id=6922488\"><br \/>\n6922488 &#8211; cadmium recommit should be more relaxed about in-repository branching<\/a>. To use the new code do nothing more than <tt>pkg image-update<\/tt> to build 134 or newer and your <tt>pkg:\/developer\/build\/onbld<\/tt> package will be updated!<\/p>\n<p>However, then as development continued, we found that there was an issue on our gate machine. The gate machine for the <a href=\"http:\/\/hub.opensolaris.org\/bin\/view\/Project+onnv\/\">OS\/Networking<\/a> consolidation is different than hg.opensolaris.org so that the ON gatekeepers can configure their own Mercurial <a href=\"http:\/\/hgbook.red-bean.com\/read\/handling-repository-events-with-hooks.html\">gate-hooks<\/a>. On our gate machine hg.opensolaris.org we can not configure our own gate-hooks, and this caused us issue as the gate machine, of course, will accept one making or merging a branch in the source code base.<\/p>\n<p>Mercurial, will of course, let you happily close a branch (e.g. <tt>hg commit --close-branch slim_1003<\/tt>). However, we found folks could do interesting things like:<\/p>\n<h3>Case 1<\/h3>\n<pre>\r\nhg clone ssh:\/\/anon&#064;hg.opensolaris.org\/hg\/caiman\/slim_source\r\nhg update -C <branch 1>\r\n[do changes]\r\nhg branch -f <branch 2 -- which already exists>\r\nhg commit\r\nhg push\r\n<\/pre>\n<h3>Case 2 (simply merge the branches)<\/h3>\n<pre>\r\nhg clone ssh:\/\/anon&#064;hg.opensolaris.org\/hg\/caiman\/slim_source\r\nhg update -C <branch 1>\r\n[do changes]\r\nhg commit\r\n[do more changes]\r\nhg commit\r\nhg recommit (with an un-patched hg recommit)\r\n[hg recommit would say to merge all heads -- each branch has its own head!]\r\nhg merge -r <other branch revision>\r\nhg commit\r\nhg push\r\n<\/pre>\n<p>These two cases, though undesirable to our community, would happily be accepted by hg.opensolaris.org into our slim_source repository and require lots of coordination to revert their affects. So the answer was to write a Mercurial hook. But this hook would be a client-side hook.<\/p>\n<p>As such, with the help of <a href=\"https:\/\/auth.opensolaris.org\/info\/userProfile.action?userId=15781\">Dave Marker<\/a> we were able to craft a user-side hook to check if all pushes were continuing on the same branch or if someone is jumping branches. This hook needs to be added to an hgrc file in you path. For a build machine, this can be <tt>\/etc\/hgrc<\/tt> or for a user you can use your <tt>~\/.hgrc<\/tt>.<\/p>\n<p>First, download the <a href=\"https:\/\/clayb.net\/blog\/wp-content\/uploads\/2012\/01\/slim_hooks.tar\">tar ball<\/a> with the necessary hook and Python file. Next, modify your hgrc as below and you might have other lines in your <tt>[extensions]<\/tt> and <tt>[hooks]<\/tt> sections simply append these lines to their correct sections:<\/p>\n<pre>\r\n[extensions]\r\nhook = ~\/slim_hooks\r\n[hooks]\r\npretxncommit.0 = python:hook.branch_change_chk.branch_change_chk\r\n<\/pre>\n<h1>What about export and import?<\/h1>\n<p>Well, unintentionally, developers were exposed to <tt>hg export<\/tt> and <tt>hg import<\/tt>. As some pushes before the user-hook was available were unfortunately toxic, developers ended up having to learn to export their changes and import them into clean source pulls. Luckily this went very smoothly but a copy of the <a href=\"http:\/\/opensolaris.org\/jive\/thread.jspa?messageID=457544&#038;tstart=0\">instructions<\/a> are reproduced below for interest&#8217;s sake:<\/p>\n<h2>Instructions to move your changes to a non-toxic workspace:<\/h2>\n<hr \/>\n<ol>\n<li>\n<h2>First, determine which changeset you wish to preserve:<\/h2>\n<p>Use <tt>hg log<\/tt> to see what changes you have made to the workspace.<br \/>\nFor example, here I would want changeset 716:7c6e6a587d85:<\/p>\n<pre>\r\n[0] clayb&#064;xsplat:hg log|less\r\nchangeset: 716:7c6e6a587d85\r\nbranch: slim_1003\r\ntag: tip\r\nuser: Clay Baenziger <ClayB at OpenSolaris dot ORG>\r\ndate: Fri Feb 05 15:32:44 2010 -0700\r\ndescription:\r\nTest\r\nchangeset: 715:a836473e08d3\r\nbranch: slim_1003\r\nuser: Matt Keenan <matt dot keenan at sun dot com>\r\ndate: Fri Feb 05 14:52:13 2010 +0000\r\ndescription:\r\n14358 Installer graphic needs changes\r\n<\/pre>\n<li>\n<h2>Next, export your change:<\/h2>\n<pre>\r\n[0] clayb@xsplat:hg export 7c6e6a587d85 > \/tmp\/mychange\r\n<\/pre>\n<li>\n<h2>Now, pull a clean workspace via:<\/h2>\n<pre>\r\n[0] clayb@xsplat:hg clone \\\r\nssh:\/\/anon at hg dot opensolaris dot org\/hg\/caiman\/slim_source\r\n<\/pre>\n<li>\n<h2>Next, select the branch you want to push to:<\/h2>\n<h4>Confirm it&#8217;s clean:<\/h4>\n<hr \/>\n<pre>\r\n[0] clayb@xsplat:cd slim_source\r\n[0] clayb@xsplat:hg log --template=\"True\\n\" -r 5f70c8ce60b9|| \\\r\necho \"False\"\r\nabort: unknown revision '5f70c8ce60b9'!\r\nFalse\r\n<\/pre>\n<h4>For slim_1003 (blocker bugs):<\/h4>\n<hr \/>\n<pre>\r\n[0] clayb@xsplat:hg update -C slim_1003\r\n6 files updated, 0 files merged, 0 files removed, 0 files unresolved\r\n<\/pre>\n<h4>For default (future development):<\/h4>\n<hr \/>\n<pre>\r\n[0] clayb@xsplat:hg update -C default\r\n0 files updated, 0 files merged, 0 files removed, 0 files unresolved\r\n<\/pre>\n<h3>Add your change to the clean repo:<\/h3>\n<pre>\r\n[0] clayb@xsplat:hg import \/tmp\/mychange\r\napplying \/tmp\/mychange\r\n<\/pre>\n<li>\n<h2>Confirm only your change is outgoing (and in the right branch):<\/h2>\n<pre>\r\n[0] clayb@xsplat:hg outgoing\r\ncomparing with ssh:\/\/anon at hg dot opensolaris dot org\/hg\/caiman\/slim_source\r\nsearching for changes\r\nchangeset: 715:b3867e9369ed\r\nbranch: slim_1003\r\ntag: tip\r\nparent: 712:ec8ad8a8fb9c\r\nuser: Clay Baenziger <ClayB at OpenSolaris dot ORG>\r\ndate: Fri Feb 05 15:32:44 2010 -0700\r\ndescription:\r\nTest\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Interesting Mercurial learnings for the slim_source gatelings<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/posts\/23"}],"collection":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/comments?post=23"}],"version-history":[{"count":0,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/posts\/23\/revisions"}],"wp:attachment":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/media?parent=23"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/categories?post=23"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/tags?post=23"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}