Google App Engine Development Environment on Mavericks w/IntelliJ and maven

Google App Engine/Java is an interesting platform which does not seem to get the same attention as AWS EC2.  I work on contract and it is a rare day that someone asks me to work on a GAE project while AWS EC2 is extremely common.  I understand there are many reasons for this, and (IMO) one reason is the vast quantity of stale/incomplete blogs posts which make GAE/J difficult to learn and adopt.  I am certain this post will also become stale in the future, so ensure the products I am using match your goals before investing.  Today is 1 June, 2014 and the tools I want to employ are:

1. OSX 10.9
2. Google App Engine 1.9.5
3. Java 1.7
4. Maven 3.1
5. IntelliJ Idea 13
6. Objectify 5

Note: example application is located on github.

The first issue you may encounter as a OSX user is that GAE requires Java 1.7 and you might still be on Java 1.6 - DO NOT get the latest Java 1.7 (which is currently 1.7.0_60) because of bug JDK-8025876.   Instead select Java 1.7.0_25 - there has been much discussion about this issue, here is an example from Stack Overflow.

You should be able to obtain results similar to this:

gsc@duckman:72>java -version

java version "1.7.0_25"

Java(TM) SE Runtime Environment (build 1.7.0_25-b15)

Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

Now that your system is working on Java 1.7, you should ensure that maven is aware of the update.  Simply updating $JAVA_HOME was not sufficient and I also needed to tweak .mavenrc - once again Stack Overflow to the rescue.  

You should be able to obtain results similar to this:

gsc@duckman:73>mvn -v

Apache Maven 3.1.1 (NON-CANONICAL_2013-11-17_20-36_gsc; 2013-11-17 20:36:09-0800)

Maven home: /Users/gsc/local/apache-maven-3.1.1

Java version: 1.7.0_25, vendor: Oracle Corporation

Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre

Default locale: en_US, platform encoding: UTF-8

OS name: "mac os x", version: "10.9.3", arch: "x86_64", family: "mac"

At this point, java and maven are ready.  If you have not already done so, download the latest Google App Engine SDK (currently 1.9.5) and install it on your system.  Point to the GAE installation with $APPENGINE_HOME

Google provides the guestbook tutorial which has been updated recently (note the EAR directory in addition to the original WAR).  The application skeleton is generated from maven with features incrementally added through the tutorial.  Instructions are reliable until until runtime when Google shows the old (now wrong) command to start/deploy on a local development server.  Assuming you have this directory structure:

gsc@duckman:89>ll

total 32

drwxr-xr-x  8 gsc  staff   272 Jun  1 10:21 ./

drwxr-xr-x  7 gsc  staff   238 Jun  1 10:38 ../

-rw-r--r--  1 gsc  staff   106 Jun  1 10:21 .gitignore

-rw-r--r--  1 gsc  staff   121 Jun  1 10:21 README.md

drwxr-xr-x  6 gsc  staff   204 Jun  1 11:47 guestbook-ear/

drwxr-xr-x  6 gsc  staff   204 Jun  1 11:47 guestbook-war/

-rw-r--r--  1 gsc  staff   583 Jun  1 10:21 guestbook.iml

-rw-r--r--  1 gsc  staff  1009 Jun  1 10:21 pom.xml

The correct command to start/deploy on a local development server is:

gsc@duckman:90>mvn -pl guestbook-ear appengine:devserver                                 

When you see "[INFO] INFO: Dev App Server is now running" then you should be able to visit the guestbook application at http://localhost:8080 and the GAE console is on http://localhost:8080/_ah/admin (guestbook screenshot below)

At this point, you have a simple, working application which you can compile and deploy using command line arguments.  Now I bring IntelliJ 13 into the mix.

First you must configure IntelliJ for your freshly installed Java 1.7 - start IntelliJ and from the splash panel select "Configure"->"Project Defaults"->"Project Structure" and the goal is a display like below.

Now we need a project.  You can continue to use the original guestbook application or consider my derivative version from github.  If you are interested in a quick introduction to objectify then use my version.

IntelliJ can be challenging to configure, YMMV but in my experience the tutorials and help are rarely up to date even for the "ultimate" version.  Once again, Stack Overflow is probably your best resource for assistance.

Ensure you have the "GAE Integration" plugin enabled.  The following screenshots might help in comparing your environment from mine.

The proof is your success in being able to compile and deploy to your local development server from within IntelliJ.  Attempt it now using the "make" triangle in the top right toolbar.

At this point you should now be able to work on a GAE/J application using IntelliJ (including the debugger).  Only objectify remains to be explored.

The original guestbook sample application uses the DatastoreService for persistence in two spots: reading guestbook entries within guestbook.jsp and writing guestbook entries within SignGuestbookServlet.java

To use objectify I have added Greeting.java as an Entity and GreetingDao.java as a DAO.  I also modified guestbook.jsp and SignGuestbookServlet.java to use the DAO (and objectify) rather than the datastore directly.

This concludes my short tour of making a small GAE project and I hope it saved you some time.

Hybrid Android Applications w/WebView

Many Android applications look entirely native but have at least a portion of the UI implemented using a WebView to display HTML and JavaScript.  (In this context, "native" means a UI created in the traditional Java/XML approach and not the NDK).

This post illustrates the use of WebView along w/WebViewClient and WebChromeClient.  I have created a small demonstration application ("AndroidWebView") which is available on github.

AndroidWebView has four tabs to demonstrate different use cases (screenshots below).

The first tab ("About") illustrates the simple use case of reading a locally stored HTML file into a WebView.  Using HTML is a frequent solution for non-interactive, wordy content such as T&C, EULA or perhaps application help pages.  As you can see AboutFragment.java was extremely simple to implement.

"About" tab selection

"Client" tab selection

The second tab ("Client") reads a remote web page and manages interaction w/the remote server using a WebViewClient.  Again ClientFragment,java is small and simple.

DemoWebViewClient.java reacts to events such as "page started", "page finished", "authorization requests", "errors", etc.  I have implemented the interesting methods to write log messages so we can view the progress using "adb logcat".

The received web page (above, on the right) and there are four navigation buttons (i.e. "Contact", "News", "Products", "Services") which are implemented as HTTP HREF(s).  Within DemoWebViewClient.shouldOverrideUrlLoading() I have arranged for the "News" button to invoke NewsDialogFragment.java rather than perform the usual HTTP GET.  This illustrates the use case of HTML content invoking an Android View.

It is useful to view the log output ("adb logcat") when viewing the "Client" tab.  When reviewing the log, you will see the various requests necessary to render the page.  This illustrates that you can detect and perhaps alter the content, or react to certain events.  The log output should look similar to this: DemoWebViewClient(20221): interceptRequest:http://www.digiburo.com/mobi/db_index.html DemoWebViewClient(20221): pageStarted:http://www.digiburo.com/mobi/db_index.html DemoWebViewClient(20221): load:http://www.digiburo.com/mobi/db_index.html DemoWebViewClient(20221): interceptRequest:http://www.digiburo.com/css/stylesheet2.css DemoWebViewClient(20221): load:http://www.digiburo.com/css/stylesheet2.css DemoWebViewClient(20221): interceptRequest:http://www.google-analytics.com/ga.js DemoWebViewClient(20221): load:http://www.google-analytics.com/ga.js DemoWebViewClient(20221): interceptRequest:http://www.digiburo.com/grafix/home_logo.png DemoWebViewClient(20221): load:http://www.digiburo.com/grafix/home_logo.png DemoWebViewClient(20221): interceptRequest:http://www.digiburo.com/grafix/canvas1.png DemoWebViewClient(20221): load:http://www.digiburo.com/grafix/canvas1.png DemoWebViewClient(20221): interceptRequest:http://www.google-analytics.com/__utm.gif? //deleted DemoWebViewClient(20221): load:http://www.google-analytics.com/__utm.gif? //deleted DemoWebViewClient(20221): pageFinished:http://www.digiburo.com/mobi/db_index.html

The third tab ("JS") illustrates interaction w/JavaScript.

JavaScriptFragment.java creates a WebChromeClient  DemoWebChromeClient.java which enables support like connecting the JavaScript console to write via the Android log or report loading progress.  If you wish to use JavaScript Alert() rather than an Android View, you will need to use WebChromeClient.onJsAlert() (which I have implemented for this purpose).  The "generateAlert" button produces a JavaScript alert.

JavaScriptFragment.java relies upon DemoJavaScript.java to expose Java methods which can be invoked from JavaScript.  I have implemented simple examples to illustrate using Android logging or return a String to JavaScript.  The "@JavascriptInterface" is limited in many ways regarding parameters and return results: primitives and Strings usually work but you will want to experiment before promising complicated arguments.  Note that generatePrimes() returns a String which contains a JSON formatted array, and this is a successful approach.  Pressing "generateLog" will cause an Android log message to be written (visible via "adb logcat").  "generatePrime" invokes a Java based prime number generator and returns the results as a JSON formatted String.

What about the use case of Java invoking JavaScript?  This is also possible and pressing "Invoke JavaScript" shows how.  "Invoke JavaScript" is a standard Android button which invokes WebView.loadUrl() for bridge.html which writes a log entry back to Java.

The fourth tab ("Simple") SimpleFragment.java demonstrates the trivial use case of reading a remote file without WebViewClient or WebChromeClient.  

Updated BackProp1 Now Available

I have recently updated the back propagation neural network "backprop1."

BackProp1 is a derivative of an image classification project I delivered in 2001.  It has been available on  SourceForge since 2009 (and my own web site prior to that).  BackProp1 consists of a library which can be embedded in your own projects and three demonstration applications:

1. demo1 is a point classifier which can determine if a point is above or below the line y = -5x + 2
2. demo2 is the XOR classifier
3. demo3 features an interactive UI and trains to recognize the digits 0-9.  You can select a pattern and then submit it for classification.  You can also flip the state of individual pixels and discover if the classifier will still recognize the pattern.

Get the latest version of backprop1 on github.

This new offering includes:

1. Demo3 now should look OK on any platform (was effectively broken on OS X).
2. Now builds w/gradle.