Interview with Talip Ozturk – founder & CTO from Hazelcast

Bildschirmfoto 2015-11-11 um 15.31.48Recently we had a very special guest in our Lightweight Java User Group. Talip Ozturk is the founder & CEO of Hazelcast. He is a pioneering startup founder having created the first Silicon Valley startup from Turkey. He has been working with enterprise Java since 1999. Talip gave a nice talk on Distributed Caching with JCache and beyond. After his talk we took the opportunity to ask a couple questions about him and his work.

Check out the interview with Talip Ozturk below:

You gave the talk “Distributed Caching with JCache and Beyond” on Tuesday, October 3rd in our Lightweight Java User Group.

What’s your impression of the Lightweight Java User Group & comSysto?

Talip Ozturk: Very smart and interactive group of developers. I love it.

What is so special about JCache?

Talip Ozturk: It standardizes the caching in Java. No need to learn a new caching API. We are now free to switch to any caching provider. Life is much easier this way.

How did you come up with the idea to establish Hazelcast?

Talip Ozturk: I thought it would be really cool if we had distributed implementations of all data structures and services available in java concurrency package. This way we can build distributed applications easily.

Did you always dream of creating your own company?

Talip Ozturk: Yes, since I was in college. I was curious to see how far the rabbit hole goes.

What makes Hazelcast different from its competitors?

Talip Ozturk: Its elegant design from its API to implementation to packaging and also the fact that there is great community around the product. It is built to give “feels right” feeling.

In which direction is your company headed for 2016?

Talip Ozturk: Out new hot-restart persistence is a big deal for us. Also we are working on to make Hazelcast more cloud friendly through better integration with Openshift, Docker, Cloud Foundry etc.

If you could describe yourself in only 5 words, what would they be?

Talip Ozturk: passionate puzzled curious mind

Where do you see yourself in 5 years?

Talip Ozturk: building new technology products

What would you say is so special about your job?

Talip Ozturk: understanding customer/user experience.

What event did you most recently visit and why did you visit this event?

Talip Ozturk: JavaOne, it is great place to demo Hazelcast and meet developers.

How did you come across our meetup user group?

Talip Ozturk: Through Java community. We try to attend all active Java user groups.

What do you do in your free time if you don’t write codes?

Talip Ozturk: Playing soccer. Spending time with my kids.

Imagine you’d be the king of the java world for one day, what would you try to change?

Talip Ozturk: I would ask Java Unsafe to be made ‘safe’.

When and where can we expect to see you again? :)

Talip Ozturk: Another Java User Group or Conference

If you missed out on Talip’s talk please check out the video below:

The slides to Talip’s talk are here available.

If you have a great topic and would like to share it with the Lightweight Java User Group, please contact us here.




Anatomy of a large Angular application

Do I really need a strategy?


A fresh application always starts out as that one application that’s going to be designed for easy maintenance and development.
Unfortunately, it’s just a matter of time until that application becomes non-trivial and needs reorganisation and/or a rewrite. In those moments, it helps if you’ve designed your application in a way that’s easy to refactor and, with some forethought (and luck), a reorganisation might not even be necessary. A bigger application usually also means a bigger team consisting of people with varying degree of front-end and Angular knowledge. Having a clear set of guidelines regarding the architecture and coding style pays off very fast.

The aforementioned problems are exactly the problems we faced while building an application that gets more than 10 million visitors each month. After a while, developing a feature becomes a chore. The same questions always pop up:

Where do I put this piece of code?

How do I modify data?

How come this event changed my data and state?

Why does modifying a piece of code suddenly break more than half of my unit tests?

It was clear — we needed a new direction.

Setting a direction

Our goal at that point was to have something that’s easy to develop, maintain and test. If we accomplish that, there’s a good chance that our application is going to be future-proof as well.

This article aims to tell the story of a better architecture but also to provide a working example of all the principles discussed here. That’s why you’ll find an accompanying repository with an interactive demo application. Details of the repository and how it relates to this article will be discussed later.

Separation of concerns

Looking at the problem from a different angle, we’ve noticed that the biggest problem was writing tests that are not too brittle. Easy testing means that mocking various parts of an application is easy which lead us to the conclusion that we need better separation of concerns.

sketchThat also meant we needed a better data flow; one where it’s completely clear who provides and modifies data and who (and how) triggers data changes. After a few initial sketches, we’ve come to a rough sketch of a data flow that resembled React’s Flux. It’s pretty clear how data flows in a flux(-like) application. In a nutshell — an event (e.g. user or callback) requests a data change from a service which modifies the data and propagates the changes to components that need that data. This in turn makes it easy to see who triggered a data change and there’s always one data source.

Better tooling

One thing that made our life easier was using a language that transpiles to JavaScript. That’s something I would seriously recommend. The top two contenders right now are TypeScript and Babel. We chose TypeScript because the tooling made it easier to notice errors at compile time and refactor bigger pieces of code.

Future proofing

Future proofing means having an application that’s easy to maintain but also reasonably easy to upgrade. It won’t be long until Angular 2 becomes production ready and a sane architecture with TypeScript goes a long way in making the gradual upgrade easier.

The bare necessities

What follows is a list of advices I expect developers of a sane Angular application are going to follow:

  • separate your concerns,
  • keep the flow of data unidirectional,
  • manage your UI state using data,
  • use a transpiled language,
  • have a build process in place,
  • test.

Let’s dive into each one of them.

Separating concerns

When each layer of an application can run as a separate entity, doesn’t know too much about the system (layers that aren’t in direct contact) and is easily testable, you’ll have an application that’s a joy to work with. Angular offers building blocks that lend itself to such a separation of concerns. If you want a deep insight into the subject, check out this blog post.

Vertical separation

Concerns can be separated horizontally and vertically. Vertical separation happens when you split an application into verticals. Each vertical has a life of its own and internally should have horizontal separation. What worked best for us, was completely separating parts of the application (e.g. separate home page, details page, configuration page, etc.) into standalone web pages that each initialise an Angular application. Communication between these modules is easy and achievable by using standard techniques like sessions, URL parameters, etc.


Horizontal separation

Where it gets interesting is horizontal separation. That’s where you actually build up your Angular application and place all its building blocks. It’s important to note that each layer (and block inside a layer) only knows about the layer above itself and doesn’t care about layers underneath that are going to consume its exposed functionalities.

Each vertical features a similar structure:

  • services layer,
  • facade layer,
  • components layer.


Components layer

The components layer is the layer that the users can interact with.
It contains directives with accompanying HTML templates and controllers. When testing (and conceptually designing), directives and HTML templates build one block and controllers build the other block of this layer.

The reason is simple — testing controllers is easy because they can be tested without a dependency on Angular. This exact feature of controllers makes them also the perfect place to put any functionality your directive requires. The preferred way then, would be to use controllerAs and bindToController in directives to build up components.

Blocks in this layer get parts of the facade layer injected and, through these, can pull data and request data modification.


A question often pops up in this layer — are we going to reach data to a component through isolated scope or get a service injected and request it? 

The answer to that question is not always clear and involves using common sense.
Smaller, reusable components without child components are a clear candidate for getting data through isolated scope and directly using that data.
Components featuring child components or more logic often benefit much more from getting their data through an injected service because they don’t get coupled to their parent.

Facade layer

The facade layer is an abstraction layer. A facade is defined as follows:

A facade can (…) reduce dependencies of outside code on the inner workings of a library, since most code uses the facade, thus allowing more flexibility in developing the system.

In our architecture, its only job is abstracting the back facing part (services layer) from the front facing part of your application (components layer). The blocks in this layer are services whose methods get called from the components layer and are then redirected to corresponding services in the services layer.

It’s that simple.

But also powerful, because such an abstraction is easy to split up and changes done to the services layer never affect your components layer.

Services layer

The services layer features all the smart things your application is supposed to do. Be it data modification, async fetching, UI state modification, etc. This layer is also the layer where your data lives and gets handed to the components layer through the facade layer.


This layer is typically going to feature:

  • services that handle your data or UI state (e.g. DataService and UIStateService),
  • services that assist them in doing so (e.g. DataFetchService or LocalStorageService) and
  • other services that you may need like a service that’s going to tell you at which breakpoint in a responsive layout you are.

Keeping the flow of data unidirectional

Now is the time to explain how all the layers and blocks fit together in a unidirectional flow of data.


Getting data

The services layer features services that know how to get data. The initial set of data is either already present as part of the HTML, asynchronously fetched or hardcoded. This data gets transformed into objects (your models) and is available through methods present on the services in your services layer.

The blocks in the components layer can now make a request for the data through the facade layer, get the already parsed data and display it. Easy.

Modifying data

If an event happens that should modify data, the blocks in the components layer make a request to the facade layer (e.g. “refresh list of users” or “update the contents of this article with this data”).

The facade layer passes the request to the correct service.

In the services layer, the request gets processed, the needed data gets modified and all the directives get the new data (because it was already bound to the directives). This works thanks to the digest cycle. Most events that happen are going to trigger a digest cycle which will then update the views. If you’ve got an event that doesn’t trigger the digest cycle (like a slider’s slide event), you can trigger a digest cycle manually.

Keep it flowing

As you can see, there’s only one place in your application that modifies your data (or a part of it). That same place provides that data and is the only part where something wrong with the data can happen which makes it much easier to debug.

Managing UI state using data

A larger Angular application is probably going to feature various states in which it can find itself. Clicking on a toggle can lead to the change of a tab, selection of a product and highlighting of a row in a table, all at the same time. Doing that on the DOM level (like jQuery manipulation) would be a bad idea because you lose the connection between your data and view.

Since we’ve already established a nice architecture, let’s use it to manage our UI state. You’d create a UIStateService in the services layer. That service would hold all relevant UI data and modify it if needed. Like already explained, that service would provide that data but also be in charge of modifying it. The facade layer would then delegate all needed changes to the correct service(s).

It’s important to note that a UIStateService might not be needed. Since views depend on data, most of the time it’s possible to just use that data and control the state of the views. A separate state service makes sense when you have to manage UI state that’s completely separated from your model.

Transpiling code

There are many benefits transpiling from a language to JavaScript. A few obvious ones are:

  • using features that are coming in newer versions of ECMAScript,
  • abstraction of JavaScript quirks,
  • compile time errors,
  • better tooling…

You can transpile from future versions of ECMAScript with Babel or even add typing support with TypeScript or Flow. You can’t go wrong with either of these choices because, at the end of the day, you get usable JavaScript. If any of the tools no longer exist, you can continue working with the generated JavaScript.


Seeing as how the Angular Team teamed up with Microsoft and are basing Angular 2 on TypeScript, it is safe to assume that the support for that stack is going to be really good. In that sense, it makes sense to get acquainted with TypeScript.

Aside from offering type safety, TypeScript has really good tooling support with editors like Sublime, Visual Studio Code or WebStorm which all offer autocompletion, inline documentation, refactoring, etc. Most of them also have a built-in TypeScript compiler so you can find compile-time errors while coding. The great autocompletion and inline documentation is possible because of type definition files. You would typically get a type definition file, put it in your project and reference it — the mentioned features work then out of the box. Visit DefinitelyTyped to see which libraries and frameworks are supported (hint: odds are, you’re going to find every library or framework you use there) and then use tsd to easily install them from the CLI.

The team at Angular is proposing a concept where libraries directly include the type definition files. The benefits of that approach are two-fold: there’s no need to search for type definition files and the type definition file you get with a version of a library always corresponds to the API of that version.

To get a quick look at all the benefits of developing with TypeScript, you can watch this video from Angular Connect.

A switch to TypeScript is mostly painless because valid JavaScript code is valid TypeScript code. Just change the file extensions to .ts, put a TypeScript compiler in your build process and you’re good to go.

Speaking of build process…

Having a build process in place

You do have a build process in place, don’t you?

If not, pick Grunt, Gulp, Webpack or whichever build/packaging tool you’d like to work with and get going. The repository accompanying this article uses Gulp, so you can get an idea how the code gets transpiled, packed for the web and tested. I won’t go into details on build tools because there are many articles out there detailing them.



You should test all parts of your application.

I see quite often that people leave out testing HTML templates because they’ve got integration tests. Unfortunately, Angular won’t let you know if you’ve got a typo somewhere in your template and integration tests can get big and slow very fast while still not covering enough ground (not to mention the time needed to maintain them).

The point is — with a good architecture in place, testing is easy because you only test code you’ve written and mock away all dependencies. Angular’s dependency injection plays a big role as well and testing with Angular is straightforward.

A combination of Karma as test runner and Jasmine as testing framework is probably going to be enough for all of your test cases. Testing in your build process (between transpiling and packaging) is also going to make sure you’re not introducing regression bugs.

Testing directives means separately testing the directive definition with its accompanying template and controllers.
Controllers are easy to test because they just get instantiated with all of their dependencies mocked away and you can get straight to testing its insides. Most of the time, you’ll just be testing if your controllers delegated to the correct service in the facade layer.
Instantiating directives and mocking away their controller is also easy because the controller is present at the compiled element after Angular’s compilation. To test what’s happening in a template, change the controller or scope mock and run a digest cycle. The new values should be present.

Testing services in the facade or services layer is just as easy because you can mock away every dependency and really test only the code that’s present.

That’s also the main take-away here — test code that’s present in the component you’re testing. Tests should fail if you modify the public methods of a component, but only tests that are associated with that component and not half of all your tests. If writing tests is hard, you’re either testing too much (and not mocking away enough) or having a problem with the architecture of your application.

Real world example


Heroes of Warcraft is a trademark and Hearthstone is a trademark or registered trademark of Blizzard Entertainment, Inc., in the U.S. and/or other countries.

As part of this article, you can check out and play with a demo application here.

It’s a deck management application for card games. Games like Hearthstone, Magic the Gathering and similar have players building decks from an ever-growing collection of cards and battle against each other. You can create and manage decks with a pre-built array of custom made cards taken from HearthCards.

Source repository

What we’ll discuss here is the repository from which the demo application was built and you can find that repository here. The idea behind this repository is to give you a working application that explores the ideas discussed in this article and a nice cheat sheet when you’re not sure how to implement a feature in Angular using TypeScript.

To get started, clone the repository and follow the README. That’s going to start up your server and serve the compiled Angular modules.

For easier work later, I recommend starting a watcher in each vertical by running gulp watch. Now, each time you modify a file inside of a vertical, Gulp is going to compile and test your changes.

Vertical separation

The application is divided into three verticals: common, deckmanager and deckbuilder. Each of these verticals is an Angular module. The common module is a utility module and gets injected into other modules.

Horizontal separation

All verticals feature a similar structure which follows what we’ve already discussed here in the article. You’ll find the directories components and services where the components directory contains directives, controllers and templates making it the components layer and the services directory where you’ll find the facade and services layer.

Let’s explore the layers.

Services layer

The deckmanager vertical is a good candidate because it features a data managing service and a UI state managing service. Each of these services has its own model consisting of objects that they’ll manage and provide.

DataService, further more, gets LocalStorageService from the common module. This is where separation of concerns pays off — the data (decks and cards in the decks) are going to be stored into local storage. Because our layers are decoupled, it’s easy to replace that storage service with something completely different.

If you take a look at the DataService in the deckbuilder vertical, you’ll see that we’re also injecting a PageValueExtractorService. That service allows us to have pre-populated data in HTML that gets parsed and used right away. This is a powerful technique that can make application startup much faster. Once again, it’s easy to see how trivial it is to combine data storage strategies and, if we decide to change the concept completely, our components won’t notice it. They just care about getting the right data, not how it got there.

Facade layer

Let’s look at the facade layer and see how it works in practice.

// ... imports

export default class FacadeService implements IFacadeService {
    private dataService:IDataService;
    private uiStateService:IUIStateService;

    constructor(dataService:IDataService, uiStateService:IUIStateService) {
        this.dataService = dataService;
        this.uiStateService = uiStateService;

    public getDecks():IDeck[] {
    return this.dataService.getDecks();

public createNewDeck(name:string):void {

// ... rest of service

FacadeService.$inject = ['DataService', 'UIStateService'];

The FacadeService gets the DataService and UIStateService by injection and can then further delegate logic between the other two layers.

If you look at the createNewDeck() method, you can see that the FacadeService isn’t necessarily just a delegation class. It can also decide simple things. The main idea is that we want a layer between components and services so that they don’t know anything about each other’s implementation.

Components layer

The structure of components includes the directive definition, a template and a controller. The template and controller are optional but, more often than not, they’re going to be present.

You can notice that the components are, for a lack of better words, dumb. They get their data and request modifications from the facade layer. Such a structure yields two big wins: less complexity and easier testing.

Take a look at a controller:

// ... imports

export default class DeckController {
    private facadeService:IFacadeService;

    constructor(facadeService:IFacadeService) {
        this.facadeService = facadeService;

    public getDecks():IDeck[] {
        return this.facadeService.getDecks();
    public addDeck():void {
    public editDeck(deck:IDeck):void {
    public deleteDeck(deck:IDeck):void {

DeckController.$inject = ['FacadeService'];

A quick glance makes it obvious that this component provides CRUD functionalities for our game decks and that it’s going to be really easy to test this class.

Data flow

As discussed in the article, the data flow is going to feature components using the facade layer which is going to delegate those requests to the correct services and deliver results.

Because of the digest cycle, every modification is going to also update the values in the components.

To clarify, consider the following image:


This image shows the data flow when a user clicks on a card in the Deck Builder. Even before the user interacts with the card gallery, the application has to read the contents of the current deck and all cards supported in the application. So, the first step is the initial pull of data that happens from the components through the facade to the services.

After a user clicks on a card the facade layer gets notified that a user action needs to be delegated. The services layer gets notified and does the needed actions (updating the model, persisting the changes, etc.).

Because a user click using ngClick triggers a digest cycle, the views are going to get updated with fresh data just like it happened in the first step.

Under consideration

The application is tested and features a simple build process. I’m not going to dive deep into these topics because the article is big enough as is, but they are self-explanatory.

The build process consists of a main Gulp configuration file and little configuration files for each vertical. The main Gulp file uses the vertical files to build each vertical. The files are also heavily annotated and shouldn’t be a problem to follow.

The tests try to be limited just to files that they’re concerned with and mock everything else away.

What now?

The application has lots of places where it could be improved upon:

  • additional filtering of cards by cost, hit points, attack points or card rarity
  • sorting by all possible criteria,
  • adding Bootstrap’s Affix to the chosen cards in the deck builder
  • developing a better Local Storage service which has much better object checking and casting
  • further improving the Page Value Extractor service to allow for metadata being included in the JSON for better type association
  • etc.

If you check the source code of the application, you’ll notice that there are comments marked with TODO. It’s possible to track these comments in IDEs and text editors (WebStorm and Visual Studio Code do it out of the box, Sublime has several plugins that support it). I’ve included several TODOs that range from new features to improvements and you’re very welcome to fix them and learn a few things along the way.

The devil is in the detail

The points discussed in this article mostly deal with big picture stuff.

If you want to find out about implementation details that can creep up while developing an Angular application, watch this entertaining video from Angular Connect about the usual errors in Angular applications.

Another great resource is this blog post by a developer who re-built the checkout flow at PayPal with Angular.

Back to the drawing board

We have a working application and an idea on how to structure our applications. It’s time to go back to the drawing board now and see if this can really be considered a win.

Consider the demo (tutorial) application that’s featured at the official Angular 2 page — John Papa’s Tour of Heroes. I’ve linked directly to the sources so you can click through the various parts of the application source code. What you’ll notice right away is how similar it feels to the application that’s part of this article. Also, you’ll notice that the take-aways from this article can easily be applied to this application as well — just take the logic out of the components and add layers for a better data flow.

The biggest advantage of developing a well-structured Angular application with TypeScript is the future-proofing that you get. Angular 2 is shaping up to be a great framework and easier to use than Angular 1 with lots of sugar (like annotating components).

Why not, then, upgrade our knowledge for things to come?

Machine Learning with Spark: Kaggle’s Driver Telematics Competition

Do you want to learn how to apply high-performance distributed computing to real-world machine learning problems? Then this article on how we used Apache Spark to participate in an exciting Kaggle competition might be of interest.

The Lab

At comSysto we regularly engage in labs, where we assess emerging technologies and share our experiences afterwards. While planning our next lab, came out with an interesting data science challenge:

AXA has provided a dataset of over 50,000 anonymized driver trips. The intent of this competition is to develop an algorithmic signature of driving type. Does a driver drive long trips? Short trips? Highway trips? Back roads? Do they accelerate hard from stops? Do they take turns at high speed? The answers to these questions combine to form an aggregate profile that potentially makes each driver unique.1

We signed up for the competition to take our chances and to get more hands on experience with Spark. For more information on how Kaggle works check out their data science competitions.

This first post describes our approach to explore the data set, the feature extraction process we used and how we identified drivers given the features. We were mostly using APIs and Libraries provided by Spark. Spark is a “fast and general computation engine for large scale data processing” that provides APIs for Python, Scala, Java and most recently R, as well as an interactive REPL (spark-shell). What makes Spark attractive is the proposition of a “unified stack” that covers multiple processing models on local machine or a cluster: Batch processing, streaming data, machine learning, graph processing, SQL queries and interactive ad-hoc analysis.

For computations on the entire data set we used a comSysto cluster with 3 nodes at 8 cores (i7) and 16GB RAM each, providing us with 24 cores and 48GB RAM in total. The cluster is running the MapR Hadoop distribution with MapR provided Spark libraries. The main advantage of this setup is a high-performance file system (mapr-fs) which also offers regular NFS access. For more details on the technical insights and challenges stay tuned for the second part of this post.

Telematic Data

Let’s look at the data provided for the competition. We first expected the data to contain different features regarding drivers and their trips but the raw data only contained pairs of anonymized coordinates (x, y) of a trip: e.g. (1.3, 4.4), (2.1, 4.8), (2.9, 5.2), … The trips were  re-centered to the same origin (0, 0) and randomly rotated around the origin (see Figure 1).

Figure 1: Anonymized driver data from Kaggle’s Driver Telematic competition1

At this point our enthusiasm got a little setback: How should we identify a driver simply by looking at anonymized trip coordinates?

Defining a Telelematic Fingerprint

It seemed that if we wanted useful and significant machine learning data, we would have to derive it ourselves using the provided raw data. Our first approach was to establish a “telematic fingerprint” for each driver. This fingerprint was composed of a list of features that we found meaningful and distinguishing. In order to get the driver’s fingerprint we used the following features:

Distance: The summation of all the euclidean distances between every two consecutive coordinates.

Absolute Distance: The euclidean distance between the first and last point.

Trip’s total time stopped: The total time that the driver has stopped.

Trip’s total time: The total number of entries for a certain trip (if we assume that every trip’s records are recorded every second, the number of entries in a trip would equal the duration of that trip in seconds)

Speed: For calculating the speed at a certain point, we calculated the euclidean distance between one coordinate and the previous one. Assuming that the coordinates units were meters and that the entries are distributed with a frequency of 1 second. This result would be given in m/s. But this is totally irrelevant since we are not doing any semantic analysis on it and we only compare it with other drivers/trips. For the speed we stored the percentiles 10, 25, 50, 80, 98. We did the same also for acceleration, deceleration and centripetal acceleration.

Acceleration: We set the acceleration to the difference between the speed at one coordinate and the speed at the previous one (when we are increasing speed).

Deceleration: We set the deceleration to the difference between the speed at one coordinate and the speed at the previous one (when we are decreasing speed).

Centripetal acceleration: We used the formulae:

centripetal acceleration

where v is the speed and r is the radius of the circle that the turning curve path would form. We already have the speed at every point so the only thing that is missing is the radius. For calculating the radius we take the current, previous and subsequent points (coordinate). This feature is an indicator of “aggressiveness” in driving style: high average of centripetal acceleration indicates turning at higher speeds.

From all derived features we computed a driver profile (“telematic fingerprint”) over all trips of that driver. From experience we know that the average speed varies between driving in the city compared to driving on the highway. Therefore the average speed over all trips for a driver is maybe not revealing too much. For better results we would need to map trip features such as average speed or maximum speed to different trip types like inner city trips, long distance highway trips, rural road trips, etc. 

Data Statistics: Around 2700 drivers with 200 trips each, resulting in about 540,000 trips. All trips together contain 360 million X/Y coordinates, which means – as they are tracked per second – we have 100,000 hours of trip data.

Machine Learning

After the inital data preparation and feature extraction we could turn towards selecting and testing machine learning models for driver prediction.


The first task was to categorize the trips: we decided to use an automated clustering algorithm (k-means) to build categories which should reflect the different trip types. The categories were derived from all trips of all drivers, which means they are not specific to a certain driver. A first look at the extracted features and computed categories revealed that some of the categories are indeed dependent on the trip length, which is an indicator for the trip type. From the cross validation results we decided to use 8 categories for our final computations. The computed cluster IDs were added to the features of every trip and used for further analysis.


For the driver prediction we used a Random Forest algorithm to train a model for each driver, which can predict the probability of a given trip (identified by its features) belonging to a specific driver. The first task was to build a training set. This was done by taking all (around 200) trips of a driver and label them with “1” (match) and then randomly choosing (also about 200) trips of other drivers and label them with “0” (no match). This training set is then fed into the Random Forest training algorithm which results in a Random Forest model for each driver. Afterwards the model was used for cross validation (i.e. evaluating the error rate on an unseen test data set) and to compute the submission for the Kaggle competition. From the cross validation results we decided to use 10 trees and a maximum tree depth of 12 for the Random Forest model (having 23 features).

An interesting comparison between the different ensemble learning algorithms for prediction (Random Forest and Gradient-BoostedTrees (GBT) from Spark’s Machine Learning Library (MLib)) can be found on the Databricks Blog.


Our workflow is splitted into several self-contained steps implemented as small Java applications that can be directly submitted to Spark via the “spark-submit” command. We used Hadoop Sequence files and CSV files for input and output. The steps are as follows:


Figure 2: ML pipeline for predicting drivers

Converting the raw input files: We are faced with about 550,000 small CSV files each containing a single trip of one driver. Loading all the files for each run of our model can be a major performance issue, therefore we converted all input files into a single Hadoop Sequence file which is served from the mapr-fs file system.

Extracting the features and computing statistics: We load the trip data from the sequence file, compute all the features described above as well as statistics such as variance and mean of features using the Spark RDD transformation API and write the results to a CSV file.

Computing the clusters: We load the trip features and statistics and use the Spark MLlib API to compute the clusters that categorize the trips using k-means. The features CSV is enriched with the clusterID for each trip.

Random Forest Training: For the actual model training we load the features for each trip together with some configuration values for the model parameters (e.g. maxDepth, crossValidation) and start a Random Forest model training for each driver with labeled training data and optional testdata for crossvalidation analysis. We serialize each Random Forest model to disk using Java serialization. In its current version Spark provides native saving and loading of model result instances, as well as configuring alternative serialization strategies.

For the actual Kaggle submission we simply load the serialized models and predict the likelihood of each trip belonging to that driver and save the result it in the required CSV format.

Results and Conclusions

This blog post describes our approach and methodology to solve the Kaggle Driver Competition using Apache Spark. Our prediction model based on Random Forest decision trees was able to predict the driver with an accuracy of around 74 percent which placed us at position 670 at the Kaggle leaderboard at the time of submission. Not bad for 2 days of work, however there are many possible improvements we identified during the lab.

To learn more about the implementation details, technical challenges and lessons learned regarding Spark stay tuned for the second part of this post.

You want to shape a fundamental change in dealing with data in Germany? Then join our Big Data Community Alliance!


Introduction To The E-Commerce Backend commercetools platform

This blog post is an introduction to the e-commerce backend commercetools platform, which is a Platform as a Service (PaaS) of commercetools GmbH from Munich, and gives impulses on how to use it.

First the facts to commercetools and commercetools platform:

commercetools GmbH is a Munich based company situated in the north near Olympia Park and has further offices in Berlin and New York. The commercetools platform is a backend for all kinds of e-commerce use cases including online pure players, mobile and point-of-sales application, couch-commerce and marketplaces. commercetools began developing its platform in 2006 and has never stopped since.
I will at first give an overview of the UI of the platform with examples as to how to use it and then talk about the Rest API they provide in order to access data for an imaginary online shop.

User interface of commercetools platform

The sign up process is fairly easy and completed in about 5 minutes. You create an account and associate a project with it. One account can hold several projects and you can invite several accounts to one project. You will be asked whether you want to include test data in the project which is advisable for your first project.

Sphere Dashboard

Dashboard commercetools platform

The self-explanatory UI allows access to all needed functionalities from Products to Orders to Settings and content for developers. The first thing you will see is the dashboard which gives you revenue statistics for any given time.

I will guide you through the account as the workflow of creating a project should be:

  • Creating Product Types:
    At first you have to understand the difference between product types and categories. Product types are used to describe common characteristics and most importantly, common custom attributes, whereas categories are used to organize products in a hierarchical structure.

    creating a product type

    Creating a product type

    Look at the product type drink I created. I added two attributes, alcohol as a boolean and volume as a number. Now every product which is created using this product type has to have these two attributes additionally to all the other attributes I will show you later.

  • Creating Categories:
    As mentioned the categories are used to organize products in you project. This should be nothing spectacularly new.

    Creating categories

    Creating categories

    I decided to use a root category containing all other categories as subcategories to make my life easier later when retrieving the categories for the online shop. A category has just name, description, parents and children.

  • Creating Products:
    Now to the important part of the setup, the products itself. When creating a product you will have to choose one of the previously created product types. Note that a product can only be of one product type.

    Creating a product

    Creating a product

    After inserting name, description, the custom attributes and a few others the product is created. You can now upload pictures, add categories, create product variants (for example for different colors), add prices and even define SEO attributes.

  • Everything else via API:
    Creating Customers and Orders is possible in the UI but is, in my opinion, more practicable by API calls. This will be explained in the next part of this post.

REST API of commercetools platform

There are a lot of SDKs in different languages like Java, PHP and Node.JS for access to the API (check out the git-repository) but I decided to code directly against the API via the REST service. The API is fully documented here. I wrote an OnePage App with AngularJS and used the Angular $http service for my API calls, which I will show you in this part of my post. Transported data in both directions is in JSON format which allows fast and reliable handling.


A client has to obtain an access token via an OAuth2 service. There are several access scopes, such as view_products, manage_orders and view_customers, which allow different kind of interaction with the platform. Normally you would have to implement a small server which handles the authentication and authorization. Otherwise the token would have to be stored on client side, which is not save, for with the manage_orders token a client can not only manage his own orders but all orders of the project. I ignored that for my test application and concentrated on the Rest API.

Getting Products

To obtain the products from the platform I used Angular’s http service:

function loadProducts(){
        .success(function(data){$scope.loadProductsResponse = data;

As response to this request you will receive a list of products with all parameters you can possibly need. Notable is the fast response time of the server which was never over 200 ms.

Carts, Customers and Orders

The most important task for an online shop is the handling of customers and their carts and orders. My test implementation creates an anonymous cart for every new user that enters the website:

if(localStorage['cartId'] === null){
    $'', {'currency':'EUR'/*,'customerId':localStorage['customerId']*/})
          .success(function(data){localStorage['cartId'] =;})

As you can see I use the localStorage feature to store data. That way the customer can come back later or refresh the website without loosing previously obtained data. Once a customer logs in, the cart will be merged into the existing cart of the customer.

Registration for a customer is as simple as this:

function signUp(emailAddress, password, lastName, firstName, streetName, streetNumber, routingCode, city){
    $scope.registerCustomer = {
      email: emailAddress,
      firstName: firstName,
      lastName: lastName,
      password: password,
      anonymousCartId: localStorage['cartId'],
      addresses :[{
        email: emailAddress,
        firstName: firstName,
        lastName: lastName,
        streetName: streetName,
        streetNumber: streetNumber,
        postalCode: routingCode,
        city: city,
        country: 'DE'
  $'', $scope.registerCustomer)
    .success(function(data){$scope.signUpResponse = data;
    .error(function(data){$scope.signUpResponse = data;

The customer can add several addresses including shipping and billing addresses which allows him to select one of them for checkout.

An order is created from a cart or an anonymous cart via POST:

function cartToOrder(updateCartResponse){
      $scope.makeOrder = {
        id :,
        version : updateCartResponse.version
      $'', $scope.makeOrder)
        .success(function(data){$scope.cartToOrderResponse = data;

The process a customer goes through until a product is ordered is fairly simple and only uses a few API calls.


commercetools platform gives you built in fast search and filtering capabilities. Using NoSQL technology, the API allows you to create comprehensive product searches, after-search navigation and configuration. In addition, every change made to the product catalog is automatically indexed.
With the built-in facet technology you can enhance customer experience and usability with extended search and navigation capabilities. Therefore customers can find products faster – especially if you have a comprehensive and complex catalog.

The operators point of view

As the company which operates the online shop you have a pretty easy job, too. All products can be uploaded and updated via CSV files which allows you to manipulate all products at once and not one after the other. There are a few different payment statuses which can be given to orders with the payment state.

plug in integrations

plug in integrations

Orders can be downloaded in CSV or XML to feed them to your inventory control system and logistics provider.

Unfortunately as of yet there are no plug in payment methods which is sad but there is a silver lining. commercetools is working on that right now. The same with the direct integration of Hippo CMS which would allow you to manage all content via Hippo.
Other than that there are several ways to integrate the commercetools platform to your existing IT landscape (see graphic).

For more information on the commercetools platform, here are a few links which might be useful:

All in all I enjoyed working with commercetools because of the complete API documentation, the fast and very helpful support and the very fast and easy accessible API. Just sign up for a free trial and see for your self.

If you want to learn more about AngularJS, register now for our Training and get Early Bird Tickets.

Building a desktop application with Electron

The how and what of JavaScript desktop applications

Desktop applications always had a special place in my heart. Ever since browsers and mobile devices got powerful, there’s been a steady decline of desktop applications which are getting replaced by mobile and web applications. Still, there’s are a lot of upsides to writing desktop applications — they are always present once they’re in your start menu or dock, they’re alt(cmd)-tabbable (I hope that’s a word) and mostly connect better with the underlying operating system (with its shortcuts, notifications, etc) than web applications.

Github ElectronIn this article, I’ll try to guide you through the process of building a simple desktop application and touch on important concepts for building desktop application with JavaScript.

The main idea behind developing desktop applications with JavaScript is that you build one codebase and package it for each operating system separately. This abstracts away the knowledge needed to build native desktop applications and makes maintenance easier. Nowadays, developing a desktop application with JavaScript relies on either Electron or NW.js. Although both tools offer more or less the same features, I went with Electron because it has some advantages I found important. At the end of the day, you can’t go wrong with either.

Basic assumptions

I assume that you’ve got your basic text editor (or IDE) and Node.js/npm installed. I’ll also assume you’ve got HTML/CSS/JavaScript knowledge (Node.js knowledge with CommonJS modules would be great, but isn’t crucial) so we can focus on learning Electron concepts without worrying about building the user interface (which, as it turns out, are just common web pages). If not, you’ll probably feel somewhat lost and I recommend visiting my previous blog post to brush up on your basics.

A 10,000 foot view of Electron

In a nutshell, Electron provides a runtime to build desktop applications with pure JavaScript. The way it works is — Electron takes a main file defined in your package.json file and executes it. This main file (usually named main.js) then creates application windows which contain rendered web pages with the added power of interacting with the native GUI (graphical user interface) of your operating system.

In detail, once you start up an application using Electron, a main process is created. This main process is responsible for interacting with the native GUI of your operating system and creates the GUI of your application (your application windows).

Purely starting the main process doesn’t give the users of your application any application windows. Those are created by the main process in the main file by using something called a BrowserWindow module. Each browser window then runs its own renderer process. This renderer process takes a web page (an HTML file which references the usual CSS files, JavaScript files, images, etc.) and renders it in the window.

For example, if you only had a calculator application, your main process would instantiate a window with a web page where your actual web page (calculator) is.

Although it is said that only the main process interacts with the native GUI of your operating system, there are techniques to offload some of that work to renderer processes (we’ll look into building a feature leveraging such a technique).

The main process can access the native GUI through a series of modules available directly in Electron. Your desktop application can access all Node modules like the excellent node-notifier to show system notifications, request to make HTTP calls, etc.

Hello, world!

Let’s get started with a traditional greeting and install all the necessary prerequisites.

Accompanying repository

This guide is accompanied by the sound-machine-tutorial repository.
Use the repository to follow along or continue at certain points. Clone the repository to get started:

git clone

and then you can jump to a git tag in the sound-machine-tutorial folder with:

git checkout <tag-name>

I’ll let you know when a tag is available with a code block:

Follow along:
git checkout 00-blank-repository

Once you clone/checkout your desired tag, run:

npm install

so that you aren’t missing any Node modules.

If you can’t switch to another tag, it would be easiest to just reset your repository state and then do the checkout:

git add -A
git reset --hard

Set up shop

Follow along with the tag 00-blank-repository:
git checkout 00-blank-repository

In the project folder create a new package.json file in it with the following contents:

    "name": "sound_machine",
    "version": "0.1.0",
    "main": "./main.js",
    "scripts": {
        "start": "electron ."

This barebones package.json:

  • sets up the name and version of the application,
  • lets Electron know which script the main process is going to run (main.js) and
  • sets up a useful shortcut — an npm script to run the application easily by running “npm start” in your CLI (terminal or command prompt).

It’s time to get Electron. The easiest way of accomplishing that is by installing a prebuilt binary for your operating system through npm and save it as a development dependency in your package.json (that happens automatically with –save-dev). Run the following in your CLI (in the project folder):

npm install --save-dev electron-prebuilt

The prebuilt binary is tailored to the operating system it’s being installed on and allows the running of “npm start”. We’re installing it as a development dependency because we will only need it during development.

That’s, more or less, everything you need to start developing with Electron.

Greeting the world

Create an app folder and an index.html file in that folder with the following contents:

<h1>Hello, world!</h1>

In the root of the project create a main.js file. That’s the file that Electron’s main process is going to spin up and allow the creation of our “Hello, world!” web page. Create the file with the following contents:

'use strict';

var app = require('app');
var BrowserWindow = require('browser-window');

var mainWindow = null;

app.on('ready', function() {
    mainWindow = new BrowserWindow({
        height: 600,
        width: 800

    mainWindow.loadUrl('file://' + __dirname + '/app/index.html');

Nothing scary, right?
The app module controls your application life (for example — watching when your application is ready for creating windows).
The BrowserWindow module allows window creation.
The mainWindow object is going to be your main application window and is declared as null because the window would otherwise be closed once JavaScripts garbage collection kicks in.

Once app gets the ready event, we create a new 800 pixels wide and 600 pixels high window using BrowserWindow.
That window’s renderer process is going to render our index.html file.

Run our “Hello, World!” application by running the following in your CLI:

npm start

and bask in the glory that is your app.

Hello, world!

Hello indeed.

Developing a real application

A glorious sound machine

First things first — what’s a sound machine?
A sound machine is a little device that makes sounds when you press various buttons, mostly cartoon or reaction sounds. It’s a funny little tool to lighten up the mood in an office and a great use case to develop a desktop application as we’re going to explore quite a few concepts while developing it (and get a nifty sound machine to boot).


The features we’re going to build and concepts we’re going to explore are:

  • basic sound machine (basic browser window instantiation),
  • closing the sound machine (remote messages between main and renderer process),
  • playing sounds without having the application in focus (global keyboard shortcuts),
  • creating a settings screen for shortcut modifier keys (Shift, Ctrl and Alt) (storing user settings in home folder),
  • adding a tray icon (remotely creating native GUI elements and getting to know menus and tray icon) and
  • packaging your application (packaging your application for Mac, Windows and Linux).

Building the basic feature of a sound machine

Starting point and application organization

With a working “Hello, world!” application under your belt, it’s high time to start building a sound machine.files@2x

A typical sound machine features several rows of buttons which respond to presses by making sounds. The sounds are mostly cartoonish and/or reaction based (laughter, clapping, glass breaking, etc.).

That’s also the very first feature we’ll build — a basic sound machine that responds to clicks.

Our application structure is going to be very straightforward.

In the root of the application we’ll keep the package.json file, the main.js file and any other application-wide files we need.

The app folder will house our HTML files of various types within folders like css, js, wav and img.

To make things easier, all the files needed for web page design have already been included in the initial state of the repository. Please check the tag 01-start-project out. If you followed along and created the “Hello, world!” application, you’ll have to reset your repository and then do the checkout:

If you followed along with the "Hello, world!" example:
git add -A
git reset --hard
Follow along with the tag 01-start-project:
git checkout 01-start-project

To keep things simple, we’re going to have only two sounds but extending it to the full 16 sounds is simply a matter of finding extra sounds, extra icons and modifying index.html.

Defining the rest of the main process

Let’s revisit main.js to define the look of the sound machine. Replace the contents of the file with:

'use strict';

var app = require('app');
var BrowserWindow = require('browser-window');

var mainWindow = null;

app.on('ready', function() {
    mainWindow = new BrowserWindow({
        frame: false,
        height: 700,
        resizable: false,
        width: 368

    mainWindow.loadUrl('file://' + __dirname + '/app/index.html');

We’re customizing the window we’re creating by giving it a dimension, making it non-resizable and frameless. It’s going to look like a real sound machine hovering on your desktop.

The question now is — how to move a frameless window (with no title bar) and close it?
I’ll talk about custom window (and application) closing very soon (and introduce a way of communicating between the main process and a renderer process), but the dragging part is easy. If you look at the index.css file (in app/css), you’ll see the following:

body {
    -webkit-app-region: drag;

-webkit-app-region: drag; allows the whole html to be a draggable object. There is a problem now, though — you can’t click buttons on a draggable object. The other piece of the puzzle is -webkit-app-region: no-drag; which allows you to define undraggable (and thus clickable elements). Consider the following excerpt from index.css:

.button-sound {
    -webkit-app-region: no-drag;

Displaying the sound machine in its own window

The main.js file can now make a new window and display the sound machine. And really, if you start your application with npm start, you’ll see the sound machine come alive. Of course, there’s nothing happening right now because we just have a static web page.

Put the following in the index.js file (located in app/js) to get the interactivity going:

'use strict';

var soundButtons = document.querySelectorAll('.button-sound');

for (var i = 0; i < soundButtons.length; i++) {
    var soundButton = soundButtons[i];
    var soundName = soundButton.attributes['data-sound'].value;

    prepareButton(soundButton, soundName);

function prepareButton(buttonEl, soundName) {
    buttonEl.querySelector('span').style.backgroundImage = 'url("img/icons/' + soundName + '.png")';

    var audio = new Audio(__dirname + '/wav/' + soundName + '.wav');
    buttonEl.addEventListener('click', function () {
        audio.currentTime = 0;;

This code is pretty simple. We:

  • query for the sound buttons,
  • iterate through the buttons reading out the data-sound attribute,
  • add a background image to each button
  • and add a click event to each button that plays audio (using the HTMLAudioElement interface)

Test out your application by running the following in your CLI:

npm start

Sound machine

A working sound machine!

Closing the application from a browser window via remote events

Follow along with the tag 02-basic-sound-machine:
git checkout 02-basic-sound-machine

To reiterate — application windows (more exactly their renderer process) shouldn’t be interacting with the GUI (and that’s what closing a window is). The official Electron quick start guide says:

In web pages, it is not allowed to call native GUI related APIs because managing native GUI resources in web pages is very dangerous and it is easy to leak resources. If you want to perform GUI operations in a web page, the renderer process of the web page must communicate with the main process to request the main process perform those operations.

Electron provides the ipc (inter-process communication) module for that type of communication. ipc allows subscribing to messages on a channel and sending messages to subscribers of a channel. A channel is used to differentiate between receivers of messages and is represented by a string (for example “channel-1”, “channel-2”…). The message can also contain data. Upon receiving a message, the subscriber can react by doing some work and can even answer. The biggest benefit of messaging is separation of concerns — the main process doesn’t have to know which renderer processes there are or which one sent a message.


That’s exactly what we’ll do here — subscribe the main process (main.js) to the “close-main-window” channel and send a message on that channel from the renderer process (index.js) when someone clicks the close button.

Add the following to main.js to subscribe to a channel:

var ipc = require('ipc');

ipc.on('close-main-window', function () {

After requiring the module, subscribing to messages on a channel is very easy and involves using the on() method with the channel name and a callback function.

To send a message on that channel, add the following to index.js:

var ipc = require('ipc');

var closeEl = document.querySelector('.close');
closeEl.addEventListener('click', function () {

Again, we require the ipc module and bind a click event to the element with the close button. On clicking the close button we send a message via the “close-main-window” channel with the send() method.

There’s one more detail that could bite you and we’ve talked about it already — the clickability of draggable areas. index.css has to define the close button as non-draggable.

.settings {
    -webkit-app-region: no-drag;

That’s all, our application can now be closed via the close button. Communicating via ipc can get complicated by examining the event or passing arguments and we’ll see an example of passing arguments later.

Playing sounds via global keyboard shortcuts

Follow along with the tag 03-closable-sound-machine:
git checkout 03-closable-sound-machine

Our basic sound machine is working great. But we do have a usability issue — what use is a sound machine that has to sit in front of all your windows the whole time and be clicked repeatedly?

This is where global keyboard shortcuts come in. Electron provides a global shortcut module which allows you to listen to custom keyboard combinations and react. The keyboard combinations are known as Accelerators and are string representations of a combination of keypresses (for example “Ctrl+Shift+1”).

Keyboard shortcuts Since we want to catch a native GUI event (global keyboard shortcut) and do an application window event (play a sound), we’ll use our trusted ipc module to send a message from the main process to the renderer process.

Before diving into the code, there are two things to consider:

  1. global shortcuts have to be registered after the app “ready” event (the code should be in that block) and
  2. when sending messages via ipc from the main process to a renderer process you have to use the reference to that window (something like “createdWindow.webContents.send(‘channel’)”)

With that in mind, let’s alter our main.js and add the following code:

var globalShortcut = require('global-shortcut');

app.on('ready', function() {
    ... // existing code from earlier

    globalShortcut.register('ctrl+shift+1', function () {
        mainWindow.webContents.send('global-shortcut', 0);
    globalShortcut.register('ctrl+shift+2', function () {
        mainWindow.webContents.send('global-shortcut', 1);

First, we require the global-shortcut module. Then, once our application is ready, we register two shortcuts — one that will respond to pressing Ctrl, Shift and 1 together and the other that will respond to pressing Ctrl, Shift and 2 together. Each of those will send a message on the “global-shortcut” channel with an argument. We’ll use that argument to play the correct sound. Add the following to index.js:

ipc.on('global-shortcut', function (arg) {
    var event = new MouseEvent('click');

To keep things simple, we’re going to simulate a button click and use the soundButtons selector that we’ve created while binding buttons to playing sounds. Once a message comes with an argument of 1, we’ll take the soundButtons[1] element and trigger a mouse click on it (note: in a production application, you’d want to encapsulate the sound playing code and execute that).

Configuring modifier keys via user settings in a new window

Follow along with the tag 04-global-shortcuts-bound:
git checkout 04-global-shortcuts-bound

With so many applications running at the same time, it could very well be that the shortcuts we’ve envisioned are already taken. That’s why we’re going to introduce a settings screen and store which modifiers (Ctrl, Alt and/or Shift) we’re going to use.

To accomplish all of that, we’ll need the following:

  • a settings button in our main window,
  • a settings window (with accompanying HTML, CSS and JavaScript files),
  • ipc messages to open and close the settings window and update our global shortcuts and
  • storing/reading of a settings JSON file from the user system.

Phew, that’s quite a list.

Settings button and settings window

Similar to closing the main window, we’re going to send messages on a channel from index.js when the settings button gets clicked. Add the following to index.js:

var settingsEl = document.querySelector('.settings');
settingsEl.addEventListener('click', function () {

After clicking the settings button, a message on the channel “open-settings-window” gets sent. main.js can now react to that event and open up the new window. Add the following to main.js:

var settingsWindow = null;

ipc.on('open-settings-window', function () {
    if (settingsWindow) {

    settingsWindow = new BrowserWindow({
        frame: false,
        height: 200,
        resizable: false,
        width: 200

    settingsWindow.loadUrl('file://' + __dirname + '/app/settings.html');

    settingsWindow.on('closed', function () {
        settingsWindow = null;

Nothing new to see here, we’re opening up a new window just like we did with the main window. The only difference is that we’re checking if the settings window is already open so that we don’t open up two instances.

Once that works, we need a way of closing that settings window. Again, we’ll send a message on a channel, but this time from settings.js (as that is where the settings close button is located). Create (or replace the contents of) settings.js with the following:

'use strict';

var ipc = require('ipc');

var closeEl = document.querySelector('.close');
closeEl.addEventListener('click', function (e) {

And listen on that channel in main.js. Add the following:

ipc.on('close-settings-window', function () {
    if (settingsWindow) {

Our settings window is now ready to implement its own logic.

Storing and reading user settings

Follow along with the tag 05-settings-window-working:
git checkout 05-settings-window-working

The process of interacting with the setting windows, storing the settings and promoting them to our application will look like this:

  • create a way of storing and reading user settings in a JSON file,
  • use these settings to display the initial state of the settings window,
  • update the settings upon user interaction and
  • let the main process know of the changes.

We could just implement the storing and reading of settings in our main.js file but it sounds like a great use case for writing a little module that we can then include in various places.

Working with a JSON configuration 

That’s why we’re going to create configuration.js file and require it whenever we need it. Node.js uses the CommonJS module pattern which means that you export only your API and other files require/use the functions available on that API.


To make storing and reading easier, we’ll use the nconf module which abstracts the reading and writing of a JSON file for us. It’s a great fit. But first, we have to include it in the project with the following command executed in the CLI:

npm install --save nconf

This tells npm to install the nconf module as an application dependency and it will be included and used when we package our application for an end user (in contrast to installing with the save-dev argument which will only include modules for development purposes).

The configuration.js file is pretty simple, so let’s examine it fully. Create a configuration.js file in the root of the project with the following contents:

'use strict';

var nconf = require('nconf').file({file: getUserHome() + '/sound-machine-config.json'});

function saveSettings(settingKey, settingValue) {
    nconf.set(settingKey, settingValue);;

function readSettings(settingKey) {
    return nconf.get(settingKey);

function getUserHome() {
    return process.env[(process.platform == 'win32') ? 'USERPROFILE' : 'HOME'];

module.exports = {
    saveSettings: saveSettings,
    readSettings: readSettings

nconf only wants to know where to store your settings and we’re giving it the location of the user home folder and a file name. Getting the user home folder is simply a matter of asking Node.js (process.env) and differentiating between various platforms (as observed in the getUserHome() function).

Storing or reading settings is then accomplished with the built-in methods of nconf (set() for storing, get() for reading with save() and load() for file operations) and exporting the API by using the standard CommonJS module.exports syntax.

Initializing default shortcut key modifiers

Before moving on with settings interaction, let’s initialize the settings in case we’re starting the application for the first time. We’ll store the modifier keys as an array with the key “shortcutKeys” and initialise it in main.js. For all of that to work, we must first require our configuration module:

'use strict';

var configuration = require('./configuration');

app.on('ready', function () {
    if (!configuration.readSettings('shortcutKeys')) {
        configuration.saveSettings('shortcutKeys', ['ctrl', 'shift']);

We try reading if there’s anything stored under the setting key “shortcutKeys”. If not, we set an initial value.

As an additional thing in main.js, we’ll rewrite the registering of global shortcut keys as a function that we can call later when we update our settings. Remove the registering of shortcut keys from main.js and alter the file this way:

app.on('ready', function () {

function setGlobalShortcuts() {

    var shortcutKeysSetting = configuration.readSettings('shortcutKeys');
    var shortcutPrefix = shortcutKeysSetting.length === 0 ? '' : shortcutKeysSetting.join('+') + '+';

    globalShortcut.register(shortcutPrefix + '1', function () {
        mainWindow.webContents.send('global-shortcut', 0);
    globalShortcut.register(shortcutPrefix + '2', function () {
        mainWindow.webContents.send('global-shortcut', 1);

The function resets the global shortcuts so that we can set new ones, reads the modifier keys array from settings, transforms it to a Accelerator-compatible string and does the usual global shortcut key registration.

Interaction in the settings window

Back in the settings.js file, we need to bind click events which are going to change our global shortcuts. First, we’ll iterate through the checkboxes and mark the active ones (reading the values from the configuration module):

var configuration = require('../configuration.js');

var modifierCheckboxes = document.querySelectorAll('.global-shortcut');

for (var i = 0; i < modifierCheckboxes.length; i++) {
    var shortcutKeys = configuration.readSettings('shortcutKeys');
    var modifierKey = modifierCheckboxes[i].attributes['data-modifier-key'].value;
    modifierCheckboxes[i].checked = shortcutKeys.indexOf(modifierKey) !== -1;

... // Binding of clicks comes here

And now we’ll bind the checkbox behavior. Take into consideration that the settings window (and its renderer process) are not allowed to change GUI binding. That means that we’ll need to send an ipc message from settings.js (and handle that message later):

for (var i = 0; i < modifierCheckboxes.length; i++) {

    modifierCheckboxes[i].addEventListener('click', function (e) {

function bindModifierCheckboxes(e) {
    var shortcutKeys = configuration.readSettings('shortcutKeys');
    var modifierKey =['data-modifier-key'].value;

    if (shortcutKeys.indexOf(modifierKey) !== -1) {
        var shortcutKeyIndex = shortcutKeys.indexOf(modifierKey);
        shortcutKeys.splice(shortcutKeyIndex, 1);
    else {

    configuration.saveSettings('shortcutKeys', shortcutKeys);

It’s a bigger piece of code but still pretty simple.
We iterate through all the checkboxes, bind a click event and on each click check if the settings array contains the modifier key or not — and according to that result, modify the array, save the result to settings and send a message to the main process which should update our global shortcuts.

All that’s left to do is subscribe to the ipc channel “set-global-shortcuts” in main.js and update our global shortcuts:

ipc.on('set-global-shortcuts', function () {

Easy. And with that, our global shortcut keys are configurable!

What’s on the menu?

Follow along with the tag 06-shortcuts-configurable:
git checkout 06-shortcuts-configurable

Another important concept in desktop applications are menus. There’s the always useful context menu (AKA right-click menu), tray menu (bound to a tray icon), application menu (on OS X), etc.

Menu In this guide we’ll add a tray icon with a menu. We’ll also use this opportunity to explore an other way of inter-process communication — the remote module.

The remote module makes RPC calls from the renderer process to the main process. In practice, it means that you remotely request native GUI modules from main.js and call methods on them. In that way, you could require the BrowserWindow object from the main process and instantiate a new browser window in a window (renderer process). Behind the scenes, that’s still a synchronous ipc message but it provides a very good tool to promote organization in your code.


Let’s see how we’d create a menu and bind it to a tray icon while doing it in a renderer process. Add the following to index.js:

var remote = require('remote');
var Tray = remote.require('tray');
var Menu = remote.require('menu');
var path = require('path');

var trayIcon = null;

if (process.platform === 'darwin') {
    trayIcon = new Tray(path.join(__dirname, 'img/tray-iconTemplate.png'));
else {
    trayIcon = new Tray(path.join(__dirname, 'img/tray-icon-alt.png'));

var trayMenuTemplate = [
        label: 'Sound machine',
        enabled: false
        label: 'Settings',
        click: function () {
        label: 'Quit',
        click: function () {
var trayMenu = Menu.buildFromTemplate(trayMenuTemplate);

The native GUI modules (menu and tray) were required remotely and that way it’s safe to use them here.

A tray icon is defined through its icon. OS X supports image templates (by convention, an image is considered a template image if its filename ends with “Template”) which makes it easy to work with the dark and light themes. Other OSes get a regular icon.

There are multiple ways of building a menu in Electron. This way creates a menu template (a simple array with menu items) and builds a menu from that template. At the end, the new menu is attached to the tray icon.

Packaging your application

Follow along with the tag 07-ready-for-packaging:
git checkout 07-ready-for-packaging

What’s the use of an application which you can’t let people download and use?Packaging

Packaging your application for all platforms is easy using electron-packager. In a nutshell, electron-packager abstracts away all work going into wrapping your app with Electron and generates all platforms for which you’re going to publish.

It can be used as a CLI application or as part of a build process. Building a more complicated build scenario is not in the scope of this article, but we’ll leverage the power of npm scripts to make packaging easier. Using electron-packager is trivial, the general form when packaging an application is:

electron-packager <location of project> <name of project> <platform> <architecture> <electron version> <optional options>


  • location of project points to the folder where your project is,
  • name of project defines the name of your project,
  • platform decides for which platforms to build (all to build for Windows, Mac and Linux),
  • architecture decides for which architectures to build (x86 or x64, all for both) and
  • electron version lets you choose which Electron version to use.

The first package is going to take a while because all the binaries for all platforms have to be downloaded. Subsequent packages are much faster.

I package the sound machine typically like this (on a Mac):

electron-packager ~/Projects/sound-machine SoundMachine --all --version=0.30.2 --out=~/Desktop --overwrite --icon=~/Projects/sound-machine/app/img/app-icon.icns

The new options included in the command are self-explanatory. To get a nice icon, you’ll first have to convert it to .icns (for Mac) and/or .ico (for Windows). Just search for a tool to convert your PNG file to these formats like this one (be sure to download the file with the .icns extension and not .hqx). If packaging for Windows from a non-Windows OS, you’ll need wine on your path (Mac users can use brew, while Linux users can use apt-get).

It doesn’t make sense to run that big command every time. We can add another script to our package.json. First of all, install electron-packager as a development dependency:

npm install --save-dev electron-packager

Now we can add a new script to our package.json file:

"scripts": {
    "start": "electron .",
    "package": "electron-packager ./ SoundMachine --all --out ~/Desktop/SoundMachine --version 0.30.2 --overwrite --icon=./app/img/app-icon.icns"

And then run the following in CLI:

npm run-script package

The package command starts the electron-packager, looks in the current directory and build to Desktop. The script should be changed if you are using Windows, but that is trivial.

The sound machine in its current state ends up weighing a whopping 100 MB. Don’t worry, once you archive it (zip or an archive type of your choice), it’ll lose more than half its size.

If you really want to go to town, take a look at electron-builder which takes the packages produced by electron-packager and creates automated installers.

Additional features to add

With the application packaged and ready to go, you can now start developing your own features.

Here are some ideas:

  • a help screen with info about the app, its shortcuts and author,
  • adding an icon and a menu entry to open the info screen,
  • build a nice packaging script for faster builds and distribution,
  • add notifications using node-notifier to let users know which sound they’re playing,
  • use lodash to a greater extent for a cleaner code base (like iterating through arrays),
  • minify all your CSS and JavaScript with a build tool before packaging,
  • combine the aforementioned node-notifier with a server call to check for new versions of your app and notify users…

For a nice challenge — try extracting your Sound machine browser windows logic and using something like browserify to create a web page with the same sound machine you’ve just created. One code base — two products (desktop application and web application). Nifty!

Diving deeper into Electron

We’ve only scratched the surface of what Electron brings to the table. It’s pretty easy to do things like watching for power events on the host machine or getting various information on the screen (like cursor position).

For all of those built-in utilities (and generally while developing applications with Electron), check out the Electron API docs.

These Electron API docs are a part of the docs folder at the Electron GitHub repository and that folder is well worth checking out.

Sindre Sorhus maintans an awesome list of Electron resources on which you can find really cool projects and information like an excellent overview of a typical Electron application architecture which can serve as a refresher on the code we’ve been developing up until now.

In the end, Electron is based on io.js (which is going to be merged back into Node.js) and most of Node.js modules are compatible and can be used to extend your application. Just browse and grab what you need.

Is that all?

Not by a long shot.

Now, it’s time to build a bigger application. I’ve mostly skipped on using extra libraries or build tools in this guide to concentrate on important issues but you can easily write your app in ES6 or Typescript, use Angular or React and simplify your build with gulp or Grunt.

With your favorite language, framework and build tool, why not build a Flickr sync desktop application using the Flickr API and node-flickrapi or a GMail client using Google’s official Node.js client library?

Pick an idea that’s going to motivate you, init a repository and just do it.

Agiles Projektmanagement auf dem PMCamp

Was ist das PMCamp?

Das PMCamp ist die wichtigste Unkonferenz im Projektmanagement, offen und vielfältig, die Ende Juli im München stattgefunden hat. Das PMCamp bringt Menschen auf Augenhöhe zusammen, um von- und miteinander zu lernen und gemeinsam die Zukunft im Projektmanagement zu gestalten. Erklärtes Ziel dieses Barcamps ist: Wissen teilen, Wissen vermehren – es schlägt eine Brücke zwischen (scheinbar) widerstrebenden Aspekten des gelebten Projektmanagements. Wir finden, dass das PMCamp ein in jeglicher Hinsicht unterstützenswerter Event ist. Weswegen wir mit großer Freude dieses Jahr als Teilnehmer, Programmgestalter und Sponsor dabei waren.


Warum war comSysto beim PMCamp?

comSysto hilft vielen, Expertise für moderne Technologien aufzubauen und mittels fortschrittlicher Methoden selbst agiler zu werden. Das PMCamp war für uns eine exzellente Gelegenheit und genau die richtige Plattform, um sich mit Gleichgesinnten auszutauschen und neue Impulse mitzugeben und mitzunehmen.

Wir – Manuela, Florian, Tobias, Christian und ich – sind dort angetreten, eine in punkto Skills und gemeinsamen Interessesgebieten bunte Truppe aus Lean Java Experts, Profis mit Scrum Master Skills und Rollen im Projekt, agil arbeitenden Projektmanagern, Agile Coaches und Lean Change Managern. Jeder von uns hatte mehrere Topics im Kopf, ganz konkret, aus unseren Projekten, die uns im Moment gerade beschäftigen oder die wir in naher Zukunft angehen wollen. Unser Ziel für das PMCamp war innovativen Input in Bezug auf agiles Projektmanagement für unsere tägliche Arbeit zu sammeln, unsererseits aus dem agilen Nähkästchen zu plaudern, zu networken und so einen Mehrwert für alle Anwesenden schaffen.

Das comSysto-Team auf dem PMCamp









Die Agenda und unsere Sessions im Einzelnen

Agenda PMCampMuc








Die Agenda war kunterbunt zusammegestellt aus Diskussionsthemen, Workshops, vorbereiteten Talks und Funsessions. Was uns bewegt hat:


“Das PMCamp ist als Gegenstück zu klassischen Konferenzen eine echte Bereicherung. Statt polierten Folien und Frontal-Unterricht steht hier der Erfahrungsaustausch aus unterschiedlichsten Perspektiven auf Augenhöhe im Mittelpunkt. Dank der offenen Barcamp-Kultur benötigt man nicht viel Vorbereitung um spontan eine Session abzuhalten. Es genügt Interesse an einem Thema und mindestens eine Hand voll Teilnehmer die sich ebenfalls darüber austauschen wollen. Ich selbst habe eine auf 2 Slots an 2 Tagen verteilte Diskussions-Session zu “Projektmanagement mit Agilen Teams – Planung, Metriken und Reporting” vorgeschlagen und dann auch erfolgreich durchgeführt. Das PMCamp war dafür die perfekte Bühne da Teilnehmer aus verschiedenen Branchen, Abteilungen und Unternehmenskulturen ihre Erfahrungen teilen und voneinander lernen konnten. So können wir gemeinsamen einen wertvollen Beitrag leisten um moderne Management-Methodik und agiles Gedankengut auch in größeren Unternehmen zu etablieren. Ich bin nächstes Jahr definitiv wieder am Start!”














“In der Session zum Thema “Verteilte Teams” wurde diskutiert, welche Probleme entstehen, wenn Teams an mehreren Standorten arbeiten, die eventuell sogar über die ganze Welt verteilt sind. Die Kommunikation erfolgt verstärkt über E-Mails und online (Video-)Konferenz Tools. Je größer die Kultur- und Sprachunterschiede, desto schwieriger wird die Kommunikation und der Aufbau einer Vertrauensbasis. Umso wichtiger ist es, persönliche Treffen zu organisieren – auch wenn man Teammitglieder um die halbe Welt schicken muss. Die Teilnehmer brachten sehr anschauliche Beispiele ein, zum Beispiel von Projekten jahrelang schlecht liefen, bis man sich schließlich doch zu durchgerungen hat, alle Projektmitlieder für ein paar Wochen zu Workshops an den selben Ort brachte. Interessant war auch die Frage, warum in großen Open Source Projekten wie Wikipedia oder Linux diese Probleme nicht so ausgeprägt zu sein scheinen. Spannende Themen, die uns in den nächsten Jahren definitiv weiter begleiten werden.

Das tolle am PMCamp war für mich einerseits die Diversität der Teilnehmer – Projektleiter aus großen Konzernen, Manager aus mittelständischen Betrieben, Software Entwickler, Freelancer, Consultants und Studenten waren gleichermaßen vertreten. Dadurch ergaben sich sehr spannende, vielseitige Sessions und Diskussionen. Eine echte Bereicherung!”

IMG_6660 IMG_6665













“Das Teilnehmerfeld war sehr heterogen aufgestellt (klassische Wasserfall Projektleiter, Bauingenieure, Agenturhengste, Prozessberater auf Company Ebene, viele Selbständige). Dies ist einerseits sehr positiv, weil man sehr viele Einblicke bekommt in einem unbekannte Arbeitsumfelder; ferner erfährt man wie dort Probleme und Herausforderungen angegangen und gelöst werden. Etwas negativ hierbei ist natürlich gewesen, dass man selbst für seine eigene Tätigkeit nicht immer etwas mitnehmen konnte. Allgemein fand ich den Socializing Faktor höher als bei klassischen Konferenzen.

Process Change- Fluch oder Segen“: Meine Session war als Diskussionsrunde ausgelegt. In dieser wollte ich über die Erfahrungen, Probleme und Lösungen der anderen Teilnehmer in Bezug auf die Adaptierfähigkeit von Prozessrichtlinien im eigenen Team / Abteilung sprechen. Wann zu welchem Zeitpunkt in welcher Iteration sind Anpassungen am eigenen Prozess notwendig? Welche Probleme interner und externer Art entstehen hierbei? Sehr schnell stellte sich heraus, dass die Teilnehmer wie im kompletten Barcamp sehr heterogen besetzt waren. Es entstand schnell eine rege Diskussion, die zum Teil auch durchaus kontrovers geführt wurde. Während bei uns Anpassungen am Prozess monatlich stattfinden, berichtete ein anderer Diskutant, dass sie 8 Monate brauchen für Veränderungen. Insgesamt eine lustige Erfahrung, bei der ein interessanter Erfahrungsaustausch stattgefunden hat.

Fazit: Generell finde ich das Konzept der Barcamps sehr gelungen, es findet ein weitaus höherer Erfahrungs- und Meinungsaustausch zwischen den Teilnehmern statt, von dem zumindest ich oftmals mehr profitiere als von klassischen Konferenzen mit Front-Beschallung.”

IMG_6658 IMG_6662














“Das PMCamp: Eine tolle Veranstaltung, um mit Projektverantwortlichen anderer Unternehmen Erfahrungen auszutauschen. Klasssiches Projektmanagement spielt für uns keine große Rolle – um so mehr war ich überrascht zu sehen, wie viele Projekte noch immer klassisch “gemanaged” werden, obwohl Vor- und Nachteile zwischen klassischen- und agilen Methoden hinlänglich bekannt sind. Trauen sich viele Unternehmen noch immer nicht zu mehr Agilität?

In einer eigenen Session “Wie schaffe ich den Change?” haben wir dann mit weiteren Teilnehmern über die Herausforderungen, die sich im Rahmen der Agilen Transition innerhalb eines Unternehmens ergeben, gesprochen. Viele der Teilnehmer konnten unsere Erfahrungen bestätigen: Teams arbeiten sehr oft agil – wohingegen die Organisation um die Teams herum oft den klassischen Ansätzen folgt. So haben Teams oft damit zu leben, dass Entscheidungen von oben nach unten weitergegeben werden, ohne die, die mit den Entscheidungen im Alltag leben werden, bei der Entscheidungsfindung mit einzubeziehen. Dieses Vorgehen kann dazu führen, dass der Mehrwert echter Agilität nicht genutzt wird. Die durch die Teams unterstützte Produktentwicklung kann nur bedingt auf Veränderungen reagieren und das Produkt nicht den gewünschten Wert liefern.”

IMG_6657 IMG_6670













Was folgt nach dem PMCamp?

Jeder von uns nimmt aus den selbst durchgeführten Sessions und aus denen, die wir besucht haben, Insights mit für unsere tägliche Arbeit. Öffentlich werden sich die Sessions von Christian in einer ähnlichen Form auf einem der nächsten Management 3.0 Stammtische wiederfinden bzw. fortgeführt. Hier geht es zur offiziellen Gruppe bei Xing.

Der kommende Management 3.0 Stammtisch findet wieder bei comSysto statt. Besonderheit: wir bekommen die Gelegenheit Jeff Sutherland, den “Erfinder”  des Scrum, persönlich kennenzulernen und seinem Vortrag über “Scrum@Scale and Leadership Development” zu lauschen.

Mehr zum PMCamp München

Mehr zu Management 3.0 und Trainingstermine

Connecting your secured OAuth2 webapp with Android

In my last post I showed how to send custom notifications for Android Wear devices to an Android Wear watch.

In order to demonstrate this better I used the events from our Teamgeist App as a case example and how notifications could look like.

I mocked the server side data as the focus was on the notification part. Today, I wanted to connect to our server side data via an Android app. Therefore I needed to authenticate the Android user at the server.

How the Google OAuth2 flow works for our web app

Our Teamgeist App uses OAuth2 and Google to authenticate a user and gain some information about the users profile picture or email address. The OAuth2 flow can be quite tricky the first time. The flow between our JavaScript client and the server part can be simplified to following the steps:

1. The first request from our JavaScript application to the server will be sent without token in the HTTP header. Therefore, we redirect the user to the login page where we show the Google+ login button:




2. Depending on the current user login status the next pages could be a login screen or selection page (e.g. if the user is using multiple Google accounts). Let’s assume the user logs in. The next page will be our consent screen:

Web3 Consent


As you can see, our App wants access to the users’ profile information. If the user grants the permission and accepts this screen, Google will generate an authorization code. During the OAuth2 setup Google asked for a callback URL. The generated authorization code will be sent to this URL.

In exchange of the authorization code the server retrieves an access token from Google. With this token the server can obtain information about the users’ profile for a certain amount of time. Furthermore, the server gets a refresh token which lasts longer and can be used to get a new access token.

The access and refresh tokens should be stored on the server. These tokens should never be given to the client app (Neither Android nor JavaScript). On the client side we store an application bearer token. We associate this token to the user and give it to the client. That’s the only token the client needs to communicate to our server.

Connect Android to the existing flow

Let’s assume that the user using an Android app already registered over the web. In order to get any information from the server e.g. events or kudos, we must initiate a request for a bearer token. We followed a blog post from Google, step-by-step about cross-client-identity which lead us to two working solutions. Both have their restrictions.

For both of them you need to register an Android app INSIDE of your application project in the Google Developer Console. Don’t create a new one, as these will be linked together and must be part of the same project. Please verify twice the SHA1 key for the Android app you enter AND the package name of your Android application. In the beginning we started with refactoring the current notification app by changing the package name from to This lead to frustrating INVALID_CLIENT_ID and INVALID_AUDIENCE errors. As we changed back to .app and recreated the Android application in the developer console everything started to work. We haven’t tried to rename it back to .android so we can’t tell if this is a forbidden keyword in the package name or maybe we were too confident about our IDE renaming android package names. If you struggle with any of the errors check your keystore SHA1 key and your packagename. Also look into this blog post, which came quite handy.

If you did everything right you can obtain an authorization code or a GoogleIdToken from GoogleAuthUtil. For this, you will need the client id of the server or web application registered in your project.

Select the Google Account

Before you start you need to let the user select an Google Account. This is done by invoking a Choose Account Intent from the AccountPicker class:

Intent intent = AccountPicker.newChooseAccountIntent(
        null, null,
        false, null, null, null, null);
startActivityForResult(intent, PICK_ACCOUNT_CODE);

When the user picks one Account the onActivityResult method of the initiating activity will be triggered. In order to get the authorization code or GoogleIdToken, you need the e-mail address of the user from the intent extras.

protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (resultCode == RESULT_OK) {
            String email = data.getStringExtra(AccountManager.KEY_ACCOUNT_NAME);

Authorization Code

The idea using the same flow with the authorization code was tempting. Get the code from your Android application and send it to the server where the server can exchange it for a pair of refresh/access tokens. You can request an authorization code by calling:

        getToken(yourActivity, email,

This call is blocking and therefore has to be executed e.g. in an AsyncTask. Besides, the server_client_id of the scope parameter must be the client id of the server.

When you call this, you will not get an authorization code but an exception of type UserRecoverableAuthException, because you need to authorize your Android app for offline access. The exception itself contains already an intent to be triggered. It will launch a consent screen where the user has to grant the requested permissions of the app.

catch (UserRecoverableAuthException userRecoverableException) {



When you add more scopes to the scope string (see for available permissions) the consent will contain more permission requests.

After the user accepts the consent, the onActivityResult of the initiating Activity will be called. From the extras you get the authorization code:

protected void onActivityResult(int requestCode,int resultCode,Intent data){
            Bundle extras=data.getExtras();
            String authtoken=extras.getString("authtoken");

The code has a very short time-to-live (TTL) and can only be used once. Once you send the token to the server you can get a refresh and access token in exchange. After that create a bearer token and return it to the Android app as you would do it with your JavaScript app. Add the bearer token to the header of all your REST calls.

The authorization code grants offline access to the app through the refresh token. That’s one of our reasons why we don’t like this solution:

1. If the user logs out from the Android app (remove the Bearer Token), you will need to go through all the steps again, including the consent screen.

2. In our example we also don’t need the offline access to user data (meaning the server can interact with Google without any user interaction). As we suppose the user is already registered via web and has granted the permission for offline access.

In our Android app we only want to fetch data contained in our server. Next, let’s take a look into the GoogleIdToken approach.


The GoogleIdToken is a JSON Web Token (JWT). The JWT contains three parts: Header, payload and signature. The signature is encrypted and contains the header and the payload. With the public keys from Google ( everybody can decrypt the signature and check if it matches the header and the payload.

The GoogleIdToken payload contains several information regarding the user and the application. If you send the token to[jwt] a payload could look like this:

    "issuer": "",
    "issued_to": "[Client id of your android application from the developer console]",
    "audience": "[Client id of your web application from the developer console]",
    "user_id": "10186*************",
    "expires_in": 3581,
    "issued_at": 1420230999,
    "email": "",
    "email_verified": true

On the server you would verify the signature and then look into the payload.

1. If the signature check is ok, you know the token has been created by Google.

2. You know Google has already verified your Android app (it checks the SHA-1 key and the package name of your Android app and compares them with the Android client registered in the same project as the web/server application) and thus provided your app with the JWT for the user with the e-mail address in the payload.

This is why you have to check the “audience” field. It must contain your web/server application client id. You could also check the “issued_to” field (also called “azp”). It contains the client id of your Android application. But this is not really needed as long as you have only one client communicating this way with your server. Google says this field could be faked from rooted devices although we don’t know how we would accomplish this.

So lets come back to our app. We want to get the GoogleIdToken. You can obtain it from Android with the same method call with which you obtained the authorization code:

Change the scope parameter in the call from




Unlike the Authorization Code request,  we directly get the response back this time. There is no need for consent screen, as the user already granted permission to the server app. On the server side Google provides the verification of the token signature with a GoogleIdTokenVerifier. You should also provide your client id of the web/server application through the Builder of the GoogleIdTokenVerifier :

JsonFactory jsonFactory = new JacksonFactory();
NetHttpTransport transport = new NetHttpTransport();
String jwt = "thestringofthejwt";

GoogleIdTokenVerifier verifier = new GoogleIdTokenVerifier.Builder(transport, jsonFactory)
    .setAudience(Arrays.asList("YOUR SERVER CLIENT ID "))

GoogleIdToken googleIdToken = verifier.verify(jwt);
if (googleIdToken != null) {
    //The token is valid. You can add a check for the issued_to/azp field
    if (!"YOUR ANDROID CLIENT ID".equals(googleIdToken.getPayload().getAuthorizedParty())) {
        throw new OAuthException("Wrong authorized party returned");

You verified the user and the application and could send the Bearer Token to the Android app as explained before with the authorization code. The benefits you gain additionally are:

1. You need no extra call to Google to verify the token.

2. You could even change your server application to accept not only the bearer token but also the GoogleIdToken. Therefore, you could spare the creation of the bearer Token and store it to the database.

The only thing you need to do, is to check if the user has already logged in from the web and search for the users’ data in your user database by using the social id or e-mail address from the JWT.


1. The user must have logged in into the app via the authorization code flow from the webapp. There the user has to accept the consent screen.

2. The user is never logged out. If the JWT expires (60 minutes) the Android app can get a new JWT without interaction with your server. Even if you invalidate all Bearer Tokens from the user the JWT would still be valid. Blocking the user is only possible by adding a flag to the user in your database.

3. You can’t access additional data on the server side from Google with the JWT.

4. Checking for the JWT in addition to the Bearer Token needs change on our server side.

Besides all the drawbacks we prefer the JWT approach. One suggestion is to create a mixture of these two possibilities. Use an authorization code for user registration and to get the access/refresh token. For User Identification use the GoogleIdToken only.

In our next episode we will use the login to gather periodically events from our server and push them as notifications to an Android Wear Smartwatch.

Feel free to share your thoughts and opinions in the comments section below.