{"id":90,"date":"2016-11-13T23:48:57","date_gmt":"2016-11-13T23:48:57","guid":{"rendered":"http:\/\/clayb.net\/blog\/?p=90"},"modified":"2021-02-16T18:05:33","modified_gmt":"2021-02-16T18:05:33","slug":"debugging-hbase-unit-tests","status":"publish","type":"post","link":"https:\/\/clayb.net\/blog\/debugging-hbase-unit-tests\/","title":{"rendered":"Debugging HBase Unit Tests"},"content":{"rendered":"

This is likely an obvious process for those who use IDE’s and develop in Maven daily but for those who do operations or otherwise need to work on the JUnit tests in HBase only infrequently, here’s how I worked when submitting a patch for HBASE-16700<\/a>.<\/p>\n

First, create your code:<\/h1>\n

Here I was adding a MasterObserver coprocessor<\/a> to HBase, so I could work relatively easily writing my code as it was simply one class. I was able to do the following — very crude — workflow:<\/p>\n

    \n
  1. Add the following to my HBase master’s hbase-site.xml<\/tt>:
    \n<property>
    \n<name>hbase.coprocessor.master.classes<\/name>
    \n<value>org.apache.hadoop.hbase.security.access.AccessController<\/value>
    \n<\/property>
    \n<\/code><\/li>\n
  2. export CLASSPATH=$(hbase classpath)<\/tt><\/li>\n
  3. vi <my code>.java<\/tt><\/li>\n
  4. javac <my code>.java<\/tt><\/li>\n
  5. Copy my class file into my HBase master’s lib directory<\/li>\n
  6. Restart my HBase master<\/li>\n<\/ol>\n

    Next, create a test:<\/h1>\n

    This is the novel part to operators, you simply need to create a file under the relevant directory for the feature you are committing but in traditional Java fashion it will be under src\/test<\/tt> while your feature will go under src\/main<\/tt>. HBase has some guidelines<\/a> on writing a test. Similarly, a useful class for writing HBase-server tests which need a minicluster is HBaseTestingUtility<\/a>. Remember to write positive and negative tests (prove that your code does what you expect and handles unexpected operations gracefully).<\/p>\n

    Testing your test<\/h2>\n

    To test your test you can ask Maven to run a build and test just your test class via the following: mvn -X test '-Dtest=org.apache.hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver'<\/tt>.<\/p>\n

    Now, the -x<\/tt> is not necessary, it runs Maven in debug mode which is useful here. As to see the log output in your test you will want to run it standalone and Maven in debug mode will give you the proper incantation with the classes it built. You will see a line akin to the following while your test is forked off: Forking command line: \/bin\/sh -c cd hbase\/hbase-server && \/usr\/lib\/jvm\/jdk1.8.0_101\/jre\/bin\/java -enableassertions -Dhbase.build.id=2016-11-13T22:53:41Z -Xmx2800m -Djava.security.egd=file:\/dev\/.\/urandom -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -jar hbase\/hbase-server\/target\/surefire\/surefirebooter5454815236698078750.jar hbase\/hbase-server\/target\/surefire\/surefire4890497615179486565tmp hbase\/hbase-server\/target\/surefire\/surefire_09143864480388952525tmp
    \nRunning org.apache.hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver<\/tt><\/p>\n

    This line is useful as you can copy-and-paste it to run your test manually. A particularly useful feature is seeing log output. But also with this line you can attach a debugger too!<\/p>\n

    Attaching a debugger<\/h2>\n

    To attach a debugger, one needs to launch Java with some options for it to wait until the debugger attaches. I was using the particular incantation: export JAVA_DEBUG='-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000'<\/tt>. I would simply add $JAVA_DEBUG<\/tt> just after the \/usr\/bin\/java<\/tt>. Here we ask the process to listen on port 8000 for the debugger to connect (and as one could guess, suspend=n<\/tt> will not wait for a debugger to connect).<\/p>\n

    Java ships a command-line debugger (jdb<\/a>) but it has no command line history or class tab completion which is a pain. I used Andrew Pimlott’s rlwrap-jdb<\/a> to provide these features. I could spin up a debugger with: CLASSPATH=hbase\/hbase-server\/target\/test-classes\/org\/apache\/hadoop\/hbase\/security\/access\/:hbase\/hbase-server\/target\/ .\/list-java-breakpoints 2>\/dev\/null > breakpoints_file && .\/rlwrap-jdb --breakpoints-file breakpoints_file jdb -attach 8000<\/tt>.<\/p>\n

    Running a debugger on an already running process<\/h1>\n

    As a side-note, getting familiar with the debugger is quite useful, as one can use this on production systems to inspect an already running Java daemon. From the JPDA Connection and Invocation<\/a> documentation one can track down a number of Java debugger connector processes. The useful one for an already running process is the SA PID Attaching Connector<\/i> run via jdb -connect sun.jvm.hotspot.jdi.SAPIDAttachingConnector:pid=<pid><\/tt>.<\/p>\n

    Similarly, today I often take jmap<\/a> -dump:format=b,file=<filename><\/tt> dumps of misbehaving Java processes for later analysis with jhat<\/a><\/tt> but figure in the future I should perhaps investigate using sun.jvm.hotspot.jdi.SACoreAttachingConnector<\/tt> on core files of the misbehaving process to get a different view of the world.<\/p>\n","protected":false},"excerpt":{"rendered":"

    This is likely an obvious process for those who use IDE’s and develop in Maven daily but for those who do operations or otherwise need to work on the JUnit tests in HBase only infrequently, here’s how I worked when submitting a patch for HBASE-16700. First, create your code: Here I was adding a MasterObserver […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[19,24],"tags":[],"_links":{"self":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/posts\/90"}],"collection":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/comments?post=90"}],"version-history":[{"count":0,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/posts\/90\/revisions"}],"wp:attachment":[{"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/media?parent=90"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/categories?post=90"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/clayb.net\/blog\/wp-json\/wp\/v2\/tags?post=90"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}