groovy scripts for Jenkins

At some point, if you are working on managing Jenkins for your team, you will find yourself looking for a set of useful groovy scripts to control jenkins configuration for a large no of builds/jobs.

Following are some mostly frequently needed ones

Get all jobs that belong to a view 

 
hudson.model.Hudson.instance.getView('my-view')?.items.each() { 
    println it.fullDisplayName 
} 

Clear the build queue  

import hudson.model.*
def queue = Hudson.instance.queue
println "Queue contains ${queue.items.length} items"
queue.clear()
println "Queue cleared"

Disable jobs in a view 

hudson.model.Hudson.instance.getView('1.1.x-tests')?.items.each() { 
   println("JOB : "+it.name)
   it.disabled=true
   it.save()
} 

Get all jobs that that define a TimerTrigger with the trigger spec 

 
import hudson.model.* 
import hudson.triggers.*
  
for(item in Hudson.instance.items) {
  
  for(trigger in item.triggers.values()) { 
    
    if(trigger instanceof TimerTrigger) {    
       println("--- Timer trigger for " + item.name + " ---")    
        println(trigger.spec + '\n')   
    }  
  } 
}

Disconnect and re-connect to slaves

import hudson.model.* 

Hudson.instance.computers.each { c ->
  c.disconnect()
  c.connect(true)
}

specs 2, Scala and maven with IntelliJ 12.1 and getting things right

Recently, to learn a test framework for Scala ( even for Java) I chose specs2 .  Its still difficult to move to SBT, largely due to the fact that I am yet to be comfortable using it. So, for the time being I will stick to maven.
To configure the scala/java project edit the pom.xml to include

    
     <dependencies>
        <!-- specs2 version I used : 1.12.3-->
        <dependency>
            <groupid>org.specs2</groupid>
            <artifactid>specs2_${scala.version}</artifactid>
            <version>${specs2.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>${scala.version}</version>
        </dependency>
    </dependencies>

    <build>
        <sourceDirectory>src/main/scala</sourceDirectory>
        <testSourceDirectory>src/test/scala</testSourceDirectory>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <version>2.15.2</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                     <jvmArgs>
                        <jvmArg>-Xms128m</jvmArg>
                        <jvmArg>-Xmx1024m</jvmArg>
                    </jvmArgs>
                    <scalaVersion>${scala.version}</scalaVersion>
                </configuration>
            </plugin>
     </build>

Include tests under  src/scala/test . An ubiquitous example

  
import org.specs2.mutable._
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner


@RunWith(classOf[JUnitRunner])
class MyTest extends Specification{



  "The 'Hello world' string" should {
    "contain 11 characters" in {
      "Hello world" must have size (11)
    }
    "start with 'Hello'" in {
      "Hello world" must startWith("Hello")
    }
    "end with 'world'" in {
      "Hello world" must endWith("world")
    }
  }
  // Update : by default tests are run in parallel, to execute them sequentially
  // add `sequential` before the spec e.g.

  sequential
  "My sequential spec" should {
     var a = 5
    "add 1 to a" in {
      a = a+1
      a mustEqual(6)
    }

    "multiply current a by 2" in {
      a = a* 2
      a mustEqual(12)
    }
  }

 // try removing `sequential` from the spec, it would fail randomly.
}

Thats it!. For running tests from IntelliJ simply right-click and run, and from cmdline execute mvn test

me, akka and scala

I have been learning Scala recently, also took one Coursera course. I am overly impressed with Scala to a level that I only want to use Scala, every where. I want to Scalanize everything that is complex. To add more, I am inoculating myself with a Scala doze every now and then.

So, after doing some coding, writing some Java-like Scala, I am gradually moving towards functional programming way of thinking. I am converting all my Java projects into Scala. A very striking thing that appeared to me was that I was writing very few lines of code, not thinking about language details but converting my ideas into programs which would be as elegant as their Java/C++ counterparts but very easy to maintain and simple, performing same complex logic. Ok, enough said. Lets get to back to business.

Recently, I came across akka, which is a middle-ware for building highly concurrent, distributed and fault tolerant application over JVM. I was really impressed with akka, and especially because it supported Scala out of box. In fact, Scala code for writing distributed systems is way more succinct than Java, without caring about all the synchronization mess with threads. The Actor model, which akka employs is just too awesome.  Its not that only a few contrived use-cases allow actor model to work, but there are myriad of them. Go visit akka site for a complete listing of it. With akka and scala its just becomes a lot more fun.

I tried writing a word count program using akka and scala. You can find its source with some comments here on github.

How to make most out of Jira?

It has been quite a while (still < 1 year), I am using Atlassian Jira for bug/issue tracking withing my team of roughly 15. Previously, we were using a version of Redmine, doped to suit company environment.

I had read, heard a lot about Jira's abilities regarding how it can be useful to ease the project management and accurately measure the project's progress. But, recently under peer pressure of my group, where I was required to file some appropriate bugs/improvements/tasks, along with doing myriad of other things, I learned  importance of using Jira.

How we can make most out it?

Well, Jira is no magic, but just a tool; rather, it provides a platform for task management. So, all task management is left to you. Here's how I did and worked well for me

• create a high-level abstract issue of big task you will be working on and put in the initial estimates if you already know or leave it to the sub-tasks
• divide the task into concrete sub-tasks with concrete estimates
• and work !

That's it! Its that really simple. Instead of adding a bunch of many issues, this way of filing issues into Jira really helped me to work efficiently and convey accurate project progress to my mangers. At times, I also had to update the estimates for the tasks but earlier you do it the better it is. That was the overall gist of it.

getting android emulator to work normally on 64 bit linux

Recently I started working on a nfc android app but I was stuck with very basic thing of getting emulator to work normally on my home laptop running Linux Mint. I tried numerous ways to configure eclipse, java paths, AVDs (android virtual devices), checking adb logcat etc but nothing really helped. The emulator could start but always hanged on the boot screen. (believe me, I even waited a complete day for it to boot up).

Thanks to StackOverflow again! As described in the comments here and also here, the Android team still has some more work to do until they make the emulator ( and other stuff) work with 64-bit version of java. But, I could happily run my android project with emulator ( which works quite fast now :) ) by simply configuring eclipse's default jre to be a 32-bit version of java 7. While 64 bit jdk being default for other things. 

unique id generation at high scale

Moving away from relational db allows us to work with huge data, which would otherwise be not feasible. But, it also mandates us to look into the issues that we didn't have to consider much about.

One of such thing was generating unique ids. With mysql you get auto-incremented ids which work absolutely fine for most of the use cases. But, for other systems, in context of nosql/distributed systems, that job is on system designer's shoulder. Inherently one-size fits all solution is difficult to come with.

There are many solutions proposed for this ubiquitous problem.

Twitter has developed snowflake which solves it.  They guarantee id generation with the following constraints :

• minimum 10k ids per second per process
• response rate 2ms (plus network latency

The other solution I found is  flake  by boundary. It achieves the goal of generating ids with good performance and adequate bit size 128 which snowflake lacked.

I will update more information on it later, as I confront with more of these.

Migrating svn repository to git

After googling I found this StackOverflow link. First, I needed to install git-svn module.

$ sudo apt-get install git-svn

Now we also need to map svn users to git users. we can get all svn users  ( skip if you already know them) with this command ( execute under svn project root )

$ svn log | grep '^r[0-9]' | awk '{print $3}' | sort | uniq 

Once you get all users, create a users.txt file mapping svn users to git

user1 = first_name last_lame <email@address.com>
user2 = first_name last_lame <email@address.com>

Once you make sure all of your svn users are mapped to git, then execute

$ git svn clone --stdlayout --no-metadata -A users.txt svn://hostname/path dest_dir

It will start fetching svn repository into git. If you run into other problems, like

Author: (no author) not defined in users.txt file 

then simply add following line to users.txt

 (no author) = first_name last_lame <email@address.com>

Note : If, due to this error, git svn command stops then simply update users.txt file and resume the migration

$ git svn fetch 

Now you can transfer your existing git repository to remote git url e.g.

$ git push --mirror git@github.com:username/project.git