Wednesday, September 11, 2013

Google Cloud EndPoints - HTML Client Configuration v1

Goals:

  • Access from a HTML Client to the services available in an either external or internal server using Google Cloud EndPoints.
  • Use Google Cloud EndPoints (Client JS) for the communication between Clients to the Server.
  • v1 with open connection without authentication (OAuth).
  • Study Case: Simple app to create notes.

Idea:

  • The access from a WEBAPP Client can be done from the same server that runs the Cloud EndPoints or from an external server.
  • The development in both cases should be similar.
  • We will implement both cases.

Development Steps:

  • External Web Server (GAE) creation:

Obviously, only in case we will place our WEBAPP in a different server than the one running the Cloud EndPoints.

The process shouldn't be different to the standard one, but as we will be developing in local while we have other server already running (with the Cloud Endpoints) we will need extra configuration to make them work together.

  • Create a new Google "Web Application Project":

  • Configuration to run with other server:
If you try to run this server with the default configuration while we are running the other server with the EndPoints, you will get an error like:

      ***********************************************************
      Could not open the requested socket: Address already in use
      Try overriding --address and/or --port.


So we need to change the configuration of the process to use a different port this time. We need to go to its properties through "Run > Run configurations..."





In the "Server" tab, we only would have to mark the "Automatically select an unused port" checkbox to ensure that each time the server starts it will take a port already free.

In this case, to know which is the port used every time we run the server, we could see it in the starting logs in the Console tab of eclipse:

      INFO: Module instance default is running at http://localhost:62601/
      Sep 8, 2013 6:51:25 PM com.google.appengine.tools.development.AbstractModule startup
      INFO: The admin console is running at http://localhost:62601/_ah/admin
      Sep 8, 2013 6:51:25 PM com.google.appengine.tools.development.DevAppServerImpl start
      INFO: Dev App Server is now running


We need this information to execute our pages and access to our administrator console, so that we should check this logs every time we re start the server. This could mean a waste of time when you are developing in local, so that you may prefer to work with a fixed port and access every time to the same URL. To do that, in the same configuration we only would have to change the port to use.


As we already tried to run our server before, we found this configuration "CloudEndPoints_Client_HTML_v1" among the others. But if we do not find our configuration we will need to create it manually with the tools available in the same window.

  • Cleaning of the Server:
As we are talking about a WEB that will keep all the business logic (Model and most part of the Controller) in an external server, we could clean all the Servlet configuration.
    • Delete auto created class "CloudEndPoints_Client_HTML_v1Servlet".
    • Delete Servlet configuration in web.xml file:

      <servlet>  
           <servlet-name>CloudEndPoints_Client_HTML_v1</servlet-name>  
           <servlet-class>com.dtorralbo.CloudEndPoints_Client_HTML_v1Servlet</servlet-class>  
      </servlet>  
      <servlet-mapping>  
           <servlet-name>CloudEndPoints_Client_HTML_v1</servlet-name>  
           <url-pattern>/cloudendpoints_client_html_v1</url-pattern>  
      </servlet-mapping>  


    • And we delete the configuration in the build path of the project:

    • Just remove the "src" folder.



  • Adding Client JS:

It is based in just 4 simple steps:
  1. Add the script to include the JavaScript client library:
  2. 
          <script src="https://apis.google.com/js/client.js?onload=init">
          </script>
    
    

  3. You can  change the function to be executed in the previous configuration after the js file is loaded. You only have to change the "onload" property to point to your custom function.

  4. Modify the function we just configured to load our EndPoint:
  5. 
          var ROOT = 'https://your_app_id.appspot.com/_ah/api';  
          gapi.client.load('your_api_name', 'v1', function() {  
                doSomethingAfterLoading();  
          }, ROOT);  
    
    

  6. This configuration should be modified to use the values of our API:
    • ROOT: is the domain where we can reach our API.
      • If we use the same server for both Client and Server, we can use the value 
        • "//'+ window.location.host +'/_ah/api"
      • If the server is not our current one, we will need to specify here the domain of the API. For local development
        • "//localhost:8888/_ah/api".
    • "your_api_name" is the name of our API, in this case "noteendpoint".


In our case:
  • We create the file "/war/js/base.js" where we will place all our js functions.

  • We configure our page to load our js functions once the page is loaded. This is in our index.html, within the <HEAD> tag:

      <script type="text/javascript" src="/js/base.js"></script> 


  • We add the script to load the js file of Google within the <BODY> of out index.html page:

      <script src="https://apis.google.com/js/client.js?onload=init"></script> 


  • With this configuration, we expect that the script to executed after the "client.js" file is loaded from Google is the function "init", so that we need to create this function in our page, but we have to pay attention to do it before the previous configuration:

      <script type="text/javascript">
            function init() {
                  google.endpoints.notesApi.init('//' + window.location.host + '/_ah/api');
            }
      </script>


  • So that the basic configuration of our "index.html" would be:

      <html>
            <head>
                  <meta http-equiv="content-type" content="text/html; charset=UTF-8">
                  <title>Google Cloud EndPoints - Cloud Notes</title>
                  <script type="text/javascript" src="/js/base.js"></script>
            </head>
            <body>
                  <script type="text/javascript">
                        function init() {
                              google.endpoints.notesApi.init('//localhost:8888/_ah/api');
                        }
                  </script>
                  <script src="https://apis.google.com/js/client.js?onload=init"></script>
            </body>
      </html>


  • In out "init()" we configured to execute the function "google.endpoints.notesApi.init" whose logic will be created in the js file "base.js":

      google.endpoints.notesApi.init = function(apiRoot) {
            var callback = function() {
                  alert("noteendpoint API uploaded");
            }
            gapi.client.load('noteendpoint', 'v1', callback, apiRoot);
      };


  • To make the initialization of the js file properly we need to initialize the namespacing at the beginning of our js file:

      var google = google || {};
      google.endpoints = google.endpoints || {}; 
      google.endpoints.notesApi = google.endpoints.notesApi || {};


  • Without this last configuration, we will get an error like these ones in the firebug initializing the API:

  • But if all the configuration was OK, we will see a message like:

  • Thanks to the descriptors, we could explorer the available services of the API using the console of the firebug:



  • Accessing to our business logic:

Once we have loaded our API, we can access to our business logic following the documentation here

The standard way to execute our services is:

      gapi.client.noteendpoint.note.insert({'id': id, 'emailAddress': emailAddress, 'description': description}).execute(function(resp) {      
            // Code after call is executed 
      }); 


The "resp" object wraps the response from our backend. We can access to its properties to take or modify the information:

      gapi.client.noteendpoint.note.list().execute(function(resp) {
            if (resp.items) {
                  for (var i = 0; i < resp.items.length; i++) {
                        var id = resp.items[i].id;
                        var emailAddress = resp.items[i].emailAddress;
                        var description = resp.items[i].description;
                  }
            }
      }); 


So that, we could have function that execute our endpoints like:
  • Adding a new note:

      google.endpoints.notesApi.add = function() {
            var idInput   = document.querySelector(".newId");
            var emailAddressInput  = document.querySelector(".newEmailAddress");
            var descriptionInput  = document.querySelector(".newDescription");
  
            var id    = idInput.value;
            var emailAddress   = emailAddressInput.value;
            var description   = descriptionInput.value;
 
            gapi.client.noteendpoint.note.insert({'id': id, 'emailAddress': emailAddress, 'description': description}).execute(function(resp) {      
                  idInput.value   = "";
                  emailAddressInput.value  = "";
                  descriptionInput.value  = "";
            });
      }; 


  • List all the notes:

      google.endpoints.notesApi.list = function() {
            gapi.client.noteendpoint.note.list().execute(function(resp) {
                  var listTable = document.getElementById('listTable');
                  listTable.innerHTML = '';
  
                  var header = document.createElement('tr');
                  header.innerHTML =  "<th>Id</th><th>Mail</th><th>Note</th>";
                  listTable.appendChild(header);
  
                  if (resp.items) {
                        for (var i = 0; i < resp.items.length; i++) {
                              var note = document.createElement('tr');
                              note.innerHTML = "<td><a href=\"javascript:google.endpoints.notesApi.showNote('"+ resp.items[i].id +"', '"+ resp.items[i].emailAddress +"', '"+ resp.items[i].description +"')\">"+ resp.items[i].id +"</a></td><td>"+ resp.items[i].emailAddress +"</td><td>"+ resp.items[i].description +"</td>";      
                              listTable.appendChild(note);
                        }
                  }
            });
      }; 


  • Modify a note:

      google.endpoints.notesApi.update = function() {
            var idInput   = document.querySelector(".id");
            var emailAddressInput  = document.querySelector(".emailAddress");
            var descriptionInput  = document.querySelector(".description");
 
            var id    = idInput.value;
            var emailAddress   = emailAddressInput.value;
            var description   = descriptionInput.value;

            gapi.client.noteendpoint.note.update({'id': id, 'emailAddress': emailAddress, 'description': description}).execute(function(resp) {});      
      };
 

  • Delete a note:

      google.endpoints.notesApi.remove = function() {
            var idInput = document.querySelector(".id");
            var id = idInput.value;
 
            gapi.client.noteendpoint.note.remove({'id': id}).execute(function(resp) {});
      };
 


  • Publish the endpoints to GAE.

As we did to publish the server with the endpoints, we need to create a new project in the App Engine console and we will configure the new identifier in our "appengine-web.xml".

In this case we need to remind that we need to access to our endpoints using the protocol HTTPS. If we try to access through HTTP we will a forbidden error while loading our endpoints.

To make this access totally transparent for the user, better to configure our GAE to use always HTTPS protocol by default. Following the documentation here we can make that any call from the user is transformed into HTTPS calls:


      <security-constraint>
            <web-resource-collection>
                  <url-pattern>/</url-pattern>
            </web-resource-collection>
            <user-data-constraint>
                  <transport-guarantee>CONFIDENTIAL</transport-guarantee>
            </user-data-constraint>
      </security-constraint>
 

With this configuration and specifying this "url-pattern" we will make that automatically any http call is transformed into a https one.

In our cases, this configuration should be applied to both GAE servers, where we have our endpoints and where we have the external client, as far as we have a service to access our endpoints from both scenarios.

With this configuration we are ready to deploy to App Engine, but we have to pay attention to the configuration of the endpoint location when we deploy our external client because for local we used to have:

      google.endpoints.notesApi.init('//localhost:8888/_ah/api');
 

Now that we will use as endpoint a service already deployed in GAE, we need to specify this parameter:

      google.endpoints.notesApi.init('//cloudendpointsservergae.appspot.com/_ah/api');
 

So that now we can use our clients to access to our services:



  • List the notes:



  • Modify or delete a note:



  • Create a new note:





Tuesday, September 3, 2013

Google Cloud EndPoints - GAE Server Configuration v1


Goals:

  • Create a service in Google App engine (GAE) to serve different external clients (WEB and Android) placing the Model and Controller in one single point (GAE) making simpler the integration from different Views (WEB, Android and iOS).
  • Use Google Cloud EndPoints (JAVA) for the communication between Server and Clients.
  • v1 with open connection without authentication (OAuth).
  • Study Case: Simple app to create notes.


Environment:

Idea:

  • Google Cloud EndPoints: Placing all the business logic (Model and Controller) in one single server (GAE) makes easier and simpler the development of the different clients (Views) that can access to it. 
  • These EndPoints wrap the logic needed to access to a remote service in a simple API. It is not necessary to manage the communication via HTTP calls, like opening and closing the connection, configuring the end point to access, creating the query string, etc.
  • Cloud EndPoints will create and provide the API, access specifications and infrastructure to access a GAE from clients like WEBAPP, Android or iOS.  


Testing App:

  • Note entity with:
    • Id
    • Email Address
    • Description
  • Server on Google Appengine.
  • Clients:
    • WEBAPP.
    • Android.

Development Steps:

  • Back-End Server creation:

  • Using GPE we create a "New Web Application Project":



  • GPE creates a project with the following structure by default:



  • As we are not using Servlets in this project we can remove all classes and configuration related to it:
    • Java class "CloudEndPoints_Server_GAP_v1Servlet.java"
    • web.xml configuration:

      <servlet>  
           <servlet-name>CloudEndPoints_Client_HTML_v1</servlet-name>  
           <servlet-class>com.dtorralbo.CloudEndPoints_Client_HTML_v1Servlet</servlet-class>  
      </servlet>  
      <servlet-mapping>  
           <servlet-name>CloudEndPoints_Client_HTML_v1</servlet-name>  
           <url-pattern>/cloudendpoints_client_html_v1</url-pattern>  
      </servlet-mapping>  


  • POJO Model creation:

  • Create a POJO that represents the entity we will use (Note).
  • The Model interface we will use is JDO.
  • We will place this entity in the "com.dtorralbo.db" package.




  • We annotate the POJO attributes as corresponds.



  • EndPoints Generation:

  • GPE will help us to generate the EndPoints from the POJO created, so that we will be able to manage this data structure from any client.

  • We execute "Google > Generate Cloud Endpoint Class" over each POJO in our project:


  • During this process, GPE generates 3 new files:
    • PMF.java: Factory in charge of providing object type "PersistenceManagerFactory" that will be used to store and retrieve from DB the POJOs we are working with.
    • NoteEndpoint.java: Endpoint with the Model logic to manage the Note objects, like:
      • List entities: CollectionResponse<Note> listNote(String, Integer)
      • Get an entity from its id: Note getNote(String id)
      • Add a new entity: Note insertNote(Note note)
      • Update a already present entity: Note updateNote(Note note)
      • Remove: Note removeNote(String id)
    • noteendpoint-v1.api: in the path "/war/WEB-INF/", it is the descriptor of the API generated in JSON format. It is used in the API console (<your-service>.appspot.com/_ah/api/explorer) to show all the description and info that this API has.

  • Refactoring of the generated code:
    • Refactor of the package names for a more proper classification:



    • API annotation modification for a more legible nomenclature:
      • API configuration: name, version, etc.

      @API(
            name = "noteendpoint",
            version = "v1"
      )


  • Services configuration:


      @ApiMethod(name = "note.list")
      @ApiMethod(name = "note.get")
      @ApiMethod(name = "note.insert")
      @ApiMethod(name = "note.update")
      @ApiMethod(name = "note.remove")


  • GPE generates automatically the basic services to manage the entities in the DB. But Google Cloud EndPoints allows you to add any further custom service that your API may require.


  • These basic services are indeed RESTful services that can be reached with the following HTTP methods:
    • listNote > GET
    • getNote > GET
    • insertNote > POST
    • updateNote > PUT
    • removeNote > DELETE


  • Once we created the Endpoints, we will create the clients API. These are the libraries that will be used by the clients to reach and work with our new API and its services. Over the project, we execute "Google > Generate Cloud Endpoints Client Library"



  • This process will generate a set of new files and directories used by the clients that will access to the services:
    • noteendpoint-v1-rest.discovery y noteendpoint-v1-rpc.discovery: In the path "/war/WEB-INF/". Descriptors with value information of the API services for REST and RPC access. Basically they contain util information in case we prefer to access to the service via REST or RPC interfaces, offering info like name of the available services, parameters needed by each service, etc.
    • endpoint-libs: This new directory will contain all the code useful for the clients.


  • The library in this case called "myapp_appspot_com-noteendpoint-v1-20130902175821-java-1.15.0-rc-sources.jar" is the client library that in our example the Android app will use to access to our API. With this library, the Android developer won't need to care nor manage the communication issues to an external service.

  • This library contains exactly what we can find in the path "/endpoint-libs/libnoteendpoint-v1/noteendpoint/noteendpoint-v1-generated-source/". So that we can check o review the code we will provide to the clients without unzipping the jar file.



      davidtorralbo$ jar -tvf myapp_appspot_com-noteendpoint-v1-20130902175821-java-1.15.0-rc-sources.jar 
           0 Tue Jan 01 00:00:00 EST 1980 com/
           0 Tue Jan 01 00:00:00 EST 1980 com/appspot/
           0 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/
           0 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/noteendpoint/
        6557 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/noteendpoint/NoteendpointRequest.java
       26054 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/noteendpoint/Noteendpoint.java
           0 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/noteendpoint/model/
        2747 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/noteendpoint/model/CollectionResponseNote.java
        2754 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/noteendpoint/model/Note.java
        3549 Tue Jan 01 00:00:00 EST 1980 com/appspot/myapp/noteendpoint/NoteendpointRequestInitializer.java
        2658 Tue Jan 01 00:00:00 EST 1980 pom.xml



  • Testing the endpoints in local:

  • We can test our new endpoints before deploying them to GAE.
  • To do that, we firstly launch the GAE server in local as a "Web application"

  • There are several ways to test our endpoints in local:
    • RESTful access:

      davidtorralbo$ curl --header "Content-Type: application/json" -X GET http://localhost:8888/_ah/api/noteendpoint/v1/note/id1      
      {
        "id" : "id1",
        "emailAddress" : "davidtorralbo@gmail.com",
        "description" : "note #1"
      }









      davidtorralbo$ curl --header "Content-Type: application/json" -X POST -d '{"id":"id2", "emailAddress":"davidtorralbo@gmail.com", "description":"note #2"}' http://localhost:8888/_ah/api/noteendpoint/v1/note      
      {
        "id" : "id2",
        "emailAddress" : "davidtorralbo@gmail.com",
        "description" : "note #2"
      }







      davidtorralbo$ curl --header "Content-Type: application/json" -X GET http://localhost:8888/_ah/api/noteendpoint/v1/note      
      {
        "items" : [ {
          "id" : "id1",
          "emailAddress" : "davidtorralbo@gmail.com",
          "description" : "note #1"
        }, {
          "id" : "id2",
          "emailAddress" : "davidtorralbo@gmail.com",
          "description" : "note #2"
        } ]





      • If we click on our API (noteendpoint API), we will explorer all the services.
 

      • We can choose and test any of these services, e.g. "note.insert" for creating a new note.

      • Thanks to the descriptors we talked before, the API console can assist you to complete the parameters needed in each service.


      • Once we execute it, we can see in the API console both the RESTfull call done and the response we got from the service.

      • To complete the tests, we can check if the new Note was properly created in our local db. The GAE console (http://localhost:8888/_ah/admin) helps you to do that in the "Datastore Viewer" section.





  • Deploying to App Engine:
  • Create a new Application in Google App engine:

  • Configure our application id in the "appengine-web.xml" file:

      <appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
            <application>cloudendpointsservergae</application>
            <version>1</version>
            [...]


  • Deploy to App Engine:



Once we published our application we can test our services and the behavior is the expected one:

  • Create a new note with the API console:





  • List all the notes:



We have to take into account that once we place the application in Google App Engine we ill have to access to our EndPoints via SSL communications, using for that the HTTPS protocol.