This is likely an obvious process for those who use IDE’s and develop in Maven daily but for those who do operations or otherwise need to work on the JUnit tests in HBase only infrequently, here’s how I worked when submitting a patch for HBASE-16700.

First, create your code:

Here I was adding a MasterObserver coprocessor to HBase, so I could work relatively easily writing my code as it was simply one class. I was able to do the following — very crude — workflow:

1. Add the following to my HBase master’s hbase-site.xml:
   <property>
<name>hbase.coprocessor.master.classes</name>
<value>org.apache.hadoop.hbase.security.access.AccessController</value>
</property>

2. export CLASSPATH=$(hbase classpath)
3. vi <my code>.java
4. javac <my code>.java
5. Copy my class file into my HBase master’s lib directory
6. Restart my HBase master

Next, create a test:

This is the novel part to operators, you simply need to create a file under the relevant directory for the feature you are committing but in traditional Java fashion it will be under src/test while your feature will go under src/main. HBase has some guidelines on writing a test. Similarly, a useful class for writing HBase-server tests which need a minicluster is HBaseTestingUtility. Remember to write positive and negative tests (prove that your code does what you expect and handles unexpected operations gracefully).

Testing your test

To test your test you can ask Maven to run a build and test just your test class via the following: mvn -X test '-Dtest=org.apache.hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver'.

Now, the -x is not necessary, it runs Maven in debug mode which is useful here. As to see the log output in your test you will want to run it standalone and Maven in debug mode will give you the proper incantation with the classes it built. You will see a line akin to the following while your test is forked off: Forking command line: /bin/sh -c cd hbase/hbase-server && /usr/lib/jvm/jdk1.8.0_101/jre/bin/java -enableassertions -Dhbase.build.id=2016-11-13T22:53:41Z -Xmx2800m -Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -jar hbase/hbase-server/target/surefire/surefirebooter5454815236698078750.jar hbase/hbase-server/target/surefire/surefire4890497615179486565tmp hbase/hbase-server/target/surefire/surefire_09143864480388952525tmp
Running org.apache.hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver

This line is useful as you can copy-and-paste it to run your test manually. A particularly useful feature is seeing log output. But also with this line you can attach a debugger too!

Attaching a debugger

To attach a debugger, one needs to launch Java with some options for it to wait until the debugger attaches. I was using the particular incantation: export JAVA_DEBUG='-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000'. I would simply add $JAVA_DEBUG just after the /usr/bin/java. Here we ask the process to listen on port 8000 for the debugger to connect (and as one could guess, suspend=n will not wait for a debugger to connect).

Java ships a command-line debugger (jdb) but it has no command line history or class tab completion which is a pain. I used Andrew Pimlott’s rlwrap-jdb to provide these features. I could spin up a debugger with: CLASSPATH=hbase/hbase-server/target/test-classes/org/apache/hadoop/hbase/security/access/:hbase/hbase-server/target/ ./list-java-breakpoints 2>/dev/null > breakpoints_file && ./rlwrap-jdb --breakpoints-file breakpoints_file jdb -attach 8000.

Running a debugger on an already running process

As a side-note, getting familiar with the debugger is quite useful, as one can use this on production systems to inspect an already running Java daemon. From the JPDA Connection and Invocation documentation one can track down a number of Java debugger connector processes. The useful one for an already running process is the SA PID Attaching Connector run via jdb -connect sun.jvm.hotspot.jdi.SAPIDAttachingConnector:pid=<pid>.

Similarly, today I often take jmap -dump:format=b,file=<filename> dumps of misbehaving Java processes for later analysis with jhat but figure in the future I should perhaps investigate using sun.jvm.hotspot.jdi.SACoreAttachingConnector on core files of the misbehaving process to get a different view of the world.