Sunday, 22 September 2013

GSoC - Push Notify test examples.

To show a simple usage of push notifications I developed 2 test examples.
I really want to thank FPComplete for giving me the possibility of hosting the Yesod app.
The tests are available online on [1] and the code is available on GitHub [3].

* BackAndForth Messaging:
In this example, I show how to use the push-general library to handle the registration/reception and sending of messages. From a Yesod app, you can send notifications to devices registered and receive messages from them.
When the server needs to send a message, uses the general api for sending a notification and handling the result in order to correctly actualize the DB. This means removing the devices that have been unregistered and changing the regId of the devices that have changed.
To try this example:
+ On an Android device, you follow these steps: [2].
+ On a WindowsPhone device, you can download the app from Microsoft Store: [4]
+ Once you registered on Server, you can start sending/receiving notifications through the website: [1].



* Connect 4:
I have been thinking of an app which lets me show how useful push notifications are. The advantage of this notifications is that you can receive information from server when it is available without needing to continually poll. So, I started to think about a multiplayers game. Every time a player does something, you let the rest know about the new movement. So, I ran into the board game: "Connect 4".
Both, authenticated web players and users of the android app, can play one against each other.
For actualizing the website, I use long polling, while for the Android app, push notifications.

Main modules description:
 - CommDevices : defines the main callback functions to communicate with mobile devices using the general library.
 - CommWebUsers : defines the main functions to handle the set of Web users.
 - Connect4 : defines the main functions to save the state of the Connect4 game.
 - DataBase : defines the DB structure.
 - Handlers :  defines the main function to handle the messages that comes from users (web or Android users).
 - Main : defines the Yesod app.

To try the example:
+ On an Android device, you follow these steps: [2].
+ Once you registered on Server, you can start playing against web or Android users. [1]


Every feedback is welcome!

[1] http://gsoc.marcospividori.com.ar/
[2] http://gsoc.marcospividori.com.ar/apps
[3] https://github.com/MarcosPividori/GSoC-Communicating-with-mobile-devices/tree/master/test
[4] http://www.windowsphone.com/es-ar/store/app/gsoc_example/1e4813bf-c72d-4c18-87f8-65a418433b50

3 Libraries for communicating with Mobile devices through Push Notifications.

I am happy to announce the result of 3 months working on the GSoC project: "Communicating with mobile devices". After reading many documentation, thinking about a good abstraction, learning about mobile apps and networks connections, and really valuable recommendations from my mentor, (I really want to thank Michael Snoyman for his availability and kindness) I succeeded in developing 3 libraries to make it easy to send push notifications to mobile devices. They are available on Hackage:

* push-notify-0.1.0.0 : This library offers a simple API for sending notifications through APNS, GCM and MPNS. [1]

* push-notify-ccs-0.1.0.0 : This library offers an API for sending/receiving notifications through CCS (XMPP - Google Cloud Messaging). [2]

* push-notify-general-0.1.0.0 : This library offers a general API for sending/receiving notifications, and handling the registration of devices on the server. It provides a general abstraction which can be used to communicate through different services as APNS, GCM, MPNS, hiding as much as possible, the differences between them. [3]

Now, it is easy to send/receive information with mobile devices, so you can easily develop server applications which interact with them.
So, to clarify this, as part of this project, I developed many test examples, including Android/WPhone and Yesod apps. The code is available on GitHub: [4] and they are running online on: [5] , so can be downloaded and tested on mobile devices. (BackAndForth Messaging and Connect4 apps)
Every feedback is welcome!

[1] http://hackage.haskell.org/package/push-notify-0.1.0.0
[2] http://hackage.haskell.org/package/push-notify-ccs-0.1.0.0
[3] http://hackage.haskell.org/package/push-notify-general
[4] https://github.com/MarcosPividori/GSoC-Communicating-with-mobile-devices
[5] http://gsoc.marcospividori.com.ar

Saturday, 21 September 2013

push-notify-general: A general library for sending Push Notifications.

The purpose of this library

This library defines a general API for communicating with iOS, Android and WPhone powered devices, sending/receiving Push Notifications. It is available on Hackage: link
The main idea is to hide as much as possible the differences between the different services.
Push Notification services in general, are very similar. They all are asynchronous, best-effort services that offer third-party developers a channel to send data to apps from a cloud service in a power-efficient manner. The main differences between them, refer to details as: the maxim payload length, the quality of service, queueing the messages or not, and the time limit for this, the way the messages are handled on devices, etc.
The registration of devices is very similar too. The device/app gets a unique ID provided by the different services, and sends this identifier to your 3rd party server. So then, when you want to send messages to these devices, you have to send them to the Push Servers, and they will deliver the messages to the proper devices.
So, Device will be our general identifier.

How to use this library

All the communication with devices will be abstracted behind the PushManager data type.
You will reference to it when intending to send notifications. Also, if you properly set the callback functions, you will be able to receive messages from devices.
So, the mains steps:
  • First you establish the main configuration for the push service: PushServiceConfig . This means, set the configurations for the different Push services you want to use, and also set the callbacks functions. These functions could be used to actualize a DB, do some processing in background, etc.
  • Then you start the service with the startPushService function and you get the PushManager. So, you can add this subsite to your Yesod app or just ignore it if you don't want to receive messages from devices.
  • When you want to send notifications:
  1. You have to specify the PushNotification , setting the parameters. Here you have to do it for each kind of notification, because they contain different structures.
  2. Then, you can send these with the sendPush function, passing the PushManager as an argument. Also, you can get useful information from the PushResult value after communicating with servers.
When your Apps on devices need to send an upstream message they have 2 options:
  1. Messages through CCS (only Android devices). For this you have to set the proper CCS settings when starting the Push Service.
  2. Messages as HTTP requests to PushManager Yesod subsite (all devices). (using the route of the Yesod subsite: something like ".../messages" )
You can see many test examples on the GitHub repository: https://github.com/MarcosPividori/GSoC-Communicating-with-mobile-devices

PushManager Subsite

Everytime a device sends a message to our system as HTTP POST request it should provide the pair {"regId" : identifier , "system" : system } specifing the ID and the system running on the device, so we can build the Device identifier.
Now we support APNS, MPNS and GCM. So the possible combinations are:
{ "regId" : RegId , "system" : "ANDROID" }
{ "regId" : DeviceURI , "system" : "WPHONE" }
{ "regId" : DeviceToken , "system" : "IOS" } (token stored in hexadecimal representation)
This PushManager Yesod subsite consists of 2 routes for receiving JSON data:
  • /register -> this route will receive the registration from devices. So devices have to send the pairs {"regId" : identifier , "system" : system } and any extra information. When a new intent for a registration arrives, the newDeviceCallback function will be called with the identifier and JSON data as arguments so you can ask for more information as username and password, etc. Depending on the callback function result, the response to the POST request will be set, so device can know if it has successfully registered on the server.
  • /messages -> this route will receive POST messages from devices. Again, devices have to send the pairs {"regId" : identifier , "system" : system } and any other extra information. Everytime a new messages arrives to the Yesod subsite, the newMessageCallback function will be called with the identifier and JSON data as arguments. This abstraction lets us use the same callback function to handle the messages that arrive from CCS too (XMPP connection for GCM).

Sunday, 25 August 2013

API for communicating through CCS (Google Cloud Messaging)

About CCS:

The GCM Cloud Connection Server (CCS) is a connection server based on XMPP, offering a new option to the old HTTP request mechanism to send messages to GCM servers. CCS allows 3rd-party app servers to communicate with Android devices by establishing a persistent TCP connection with Google servers using the XMPP protocol. This communication is asynchronous and bidirectional.
CCS offers some important benefits:
- The asynchronous nature of XMPP allows you to send more messages with fewer resources.
- Communication is bidirectional, not only can the server send messages to the device, but the device can send messages back to the server.
- You can send messages back using the same connection used for receiving, thereby improving battery life.

The format of CCS messages continue with the same general structure than with the GCM HTTP request mecanism.

(Information taken from: [4])

Api for communicating through CCS:

I developed an API for communicating through CCS ([1]). For this, I designed a way of building the messages, sending them and handling the result.
For efficiently sending/receiving messages without blocking a connection, I decided to communicate with CCS server through a worker thread which maintains a XMPP connection and handles the ack/nack protocol of messages. So you have to start this worker thread before sending notifications, and then communicate with it through a channel. Also, it is possible to define a callback function to be called each time a message arrives from devices.
For dealing with XMPP connections, I opted to use the Pontarius XMPP library, which has a very friendly interface. I want to specially thank to Pontarius mantainers for their quick response to my issues using this library.
So, the API looks like this:
startCCS :: GCMAppConfig              -- Main configuration for the GCM service.
         -> (RegId -> Value -> IO ()) -- Callback function to be called each time
                                      -- a message arrives from a device.
         -> IO CCSManager

closeCCS :: CCSManager -> IO ()

sendCCS  :: CCSManager -> GCMmessage -> IO GCMresult

The main idea is: -  First of all, the sevice is started with: 'startCCS'.
    This function:
     *  Creates a channel to put the notifications to be sent.
     *  Starts a worker thread, which maintains a XMPP connection with CCS servers, sends the data it receives through the cannel, handle the ack/nack protocol of messages and everytime it receives a message from devices, it will call the callback function.
     *  Returns a CCSManager.

 - Then, I can use 'sendCCS' to send messages.
    Every time I call it:
     *  It puts the message on the channel.
     *  Wait until the time the message is taken by the worker thread.
     *  When the message is taken from the channel, the worker thread will send me a duplicate copy of the error channel. So, I will wait for the end of the sending, or an error msg through the error channel.
     *  I return the appropiate result depending on a successful or not operation.

 - I stop the service with 'closeCCS'. It kills the worker thread.

The worker identifies each message it sends with a number, which is sent to CCS. This number is returned in nack/ack messages by the CCS servers, so it can know if it was successfully sent and returns a proper result.
The code is available on GitHub:
 The CCS Api : [1]
 Test example for CCS: [2]
 - Android App for CCS: [3]

For more information about:
 The CCS Service: [4]

Monday, 29 July 2013

API for communicating through MPNS

I developed an API for communicating through MPNS with Windows Phone mobile devices ([1]). For this, I designed a way of building the messages, sending them and handling the result.
To communicate with MPNS servers, I need to send POST requests, so I decided to use Conduit, as I have done with GCM. The MPNS Api looks like this:

MPNS let you send notification in an authenticated or unauthenticated mode, so you can configure this with  MPNSAppConfig:
 data MPNSAppConfig = MPNSAppConfig{
        numRet       :: Int      -- def = 5
    ,   useSecure    :: Bool     -- def = False
    ,   certificate  :: String   -- def = ""
    ,   privateKey   :: String   -- def = ""
    }
To build messages, I developed a data type "MPNSmessage"
 data MPNSmessage = MPNSmessage{
        deviceURIs          :: [DeviceURI]  -- destinations
    ,   batching_interval   :: MPNSInterval -- Immediate | Sec450 | Sec900
    ,   target              :: MPNSType     -- Toast     | Raw    | Tile
    ,   restXML             :: Document     -- the XML data to be sent
    }
For sending notifications:
sendMPNS :: Manager -> MPNSAppConfig -> MPNSmessage -> IO MPNSresult
Last, for handling the result:
 data MPNSresult = MPNSresult{
        sucessfullResults :: [(DeviceURI,MPNSinfo)]
    ,   errorException    :: [(DeviceURI,CE.SomeException)]
    }

The code is available on GitHub:
 The MPNS Api : [1]
 Test example for MPNS: [2]

For more information about:
 The MPNS Service: [3]
 The communication with MPNS servers:  [4]

Microsoft Push Notification Service

The Microsoft Push Notification Service in Windows Phone is an asynchronous, best-effort service that offers third-party developers a channel to send data to a Windows Phone app from a cloud service in a power-efficient manner.

The process of registration is very similar to the rest of push services. The app requests a push notification URI from the Push client service, and then it sends it to your cloud service.
When you have info to send, you use the notification URI to send a push notification to MPNS servers, and MPNS routes the push notification to your app.

Depending on the format of the push notification and the payload attached to it, the info is delivered as raw data to the app, the app's Tile is visually updated, or a toast notification is displayed.
MPNS returns a response code to your cloud service after a push notification is sent indicating that the notification has been received and will be delivered to the device at the next possible opportunity.
Also, in a response code, you can get information about the state of the device (Connected / Temp Disconnected / Disconnected). For more information: [1]

When communicating with MPNS servers, you have to send POST messages for each Windows Phone device to which you want to send a notification. You have to specify the appropriate notification type (Raw / Tile / Toast), the destination Uri, and the rest of information in a XML format.
If you want to send multiple notifications, you must create separate POST messages for each notification. This is one of the differences with GCM, where you can send multicast messages in a single POST request.


Tuesday, 16 July 2013

API for communicating through APNS

I developed an API for communicating through APNS with iOS mobile devices ([1]).
For this, I designed a way of building the messages, sending them and handling the result.
One important thing that I had to take into account, was how to maintain the connection. For APNS, I needed to communicate through a streaming SSL connection, so I couldn't use conduit as I 've done with GCM. So, to face this problem, I developed an APNSManager.
The API looks like this:

 startAPNS     :: APNSAppConfig -> IO APNSManager  
 closeAPNS     :: APNSManager   -> IO ()  
 sendAPNS      :: APNSManager   -> APNSmessage -> IO APNSresult  
 feedBackAPNS  :: APNSAppConfig -> IO APNSFeedBackresult  

The main idea is:

- First of all, the sevice is started with: 'startAPNS'.
  This function:
    * Creates a channel to put the notifications to be sent.
    * Starts a worker thread, which maintains the connection with APNS servers, and sends the data it receives through the cannel.
    * Returns an APNSManager.

- Then, I can use 'sendAPNS' to send messages. Every time I call it:
    * It puts the message on the channel.
    * Wait until the time the message is taken by the worker thread.
    * When the message is taken from the channel, the worker thread will send me a duplicate copy of the error channel. So, I will wait for the end of the sending, or an error msg through the error channel.
    * I return the appropiate result depending on a successful or not operation.

- I stop the service with 'closeAPNS'. It kills the worker thread.

The worker identifies each message it sends with a number, which is sent to APNS. If an error ocurred, this number is returned in an error messages by the APNS servers, representing the number of the last message that was successfully sent.
Then, the worker continually sends messages until it receives an error message. In this case, it resends the error message to the error channel for all the threads waiting for a response, and then it restarts itself, because after an error, APNS servers close the connection.
The time we wait for an error response, after having sent all the messages, can be stablished with the timeoutLimit parameter, when creating a Manager.

As to use this service, Apple requires you to be an enrolled iOS Developer or something similar, I decided to test the API with a local server. This server was taken from: [3] . It simulates the APNS servers and I modified it to test different common scenes.

The code is available on GitHub:
 - The APNS Api : [1]
 - Test example for APNS: [2]

For more information about:
 - The APNS Service: [4]
 The communication between providers and APNS servers:  [5]


Monday, 15 July 2013

Apple Push Notification Service

Apple Push Notification service is a robust and highly efficient service for propagating information to devices iOS. Each device establishes an accredited and encrypted IP connection with the service and receives notifications over this persistent connection.

To send notifications, the provider connects to APNS through a permanent and secure connection. When there is new information to be sent, the provider prepares and sends a notification through the APNS channel, which pushes the notification to the target device.


(https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Art/remote_notif_simple_2x.png)

A notification is a short message consisting of two main parts of data: the device token and the payload:
  * The device token identifies the device and application on the client side.
  * The payload is a JSON-defined property list that specifies how the user of an application on a device is to be alerted.

Each provider requires a unique provider certificate and private cryptographic key for validation of its connection to the APN. This certificate, provisioned by Apple, must identify the particular topic published by the provider; the topic is the bundle ID of the client application.

For more information: Apple Developers - Apple Push Notification Service

Sunday, 30 June 2013

API for communicating through GCM

At beginning, I started with an API for communicating through GCM with mobile devices running Android. I developed a way of building the messages, sending them and handling the result.
To build the messages, I developed a data type: GCMmessage, where you can customize some parameters for the servers providing the GCM service.
To send the messages you need to establish some server configurations through the data type GCMAppConfig (as the correct Api key provided by Google). Passing this configuration, a GCMmessage and a number of retries as arguments to the function sendGCM, the notifications will be sent to the GCM Servers.
The result is shown in a new data type: GCMresult, which provides some useful information, as selecting the devices that have changed their regIds, or have been unregistered, and so on. The programmer can take his own desitions on how to handle this result in his server.

To show a simple usage of this API for GCM (Network.PushNotify.Gcm), I developed a simple test example of a Yesod server, where devices can register to receive GCM messages and users can send messages through the web service. This is composed of:

+ a Yesod app for registering the devices and sending push notifications.

+ an Android app for registering on GCM and receiving the notifications.

The picture at the bottom shows the main structure of the Google Cloud Message service. We have 3 main points:

The server, which I developed as a Yesod app.
The devices, running Android.
+ The GCM Servers which provide the service.


About the Yesod App:

In this example, I show how to handle with the registration of devices. Also, I provide a web service, so users can send notifications to the devices registered. When the server receives a post request to send a notification to a device, uses the GCM apis for sending the notifications and handling the result in order to correctly actualize the DB. This means removing the devices that have been unregistered, changing the regId of the devices that have changed, and resending the notifications that weren't sent because of an internal error in the GCM Servers.

To try the example:
+ On an Android device, you follow these steps: 
http://gsoc.marcospividori.com.ar/apps
+ If the registration succeded, when you go to: http://gsoc.marcospividori.com.ar/ , you will see your username. And you can start sending notifications through the website.






Tuesday, 18 June 2013

General Overview of Push Notifications Services

Nowadays, every company is developing Push Notification services, and these are very similar.
When I say they are very similar, I refer to the fact that they all are asynchronous, best-effort services that offers third-party developers a channel to send data to apps from a cloud service in a power-efficient manner. The most popular are:
    - Google Cloud Messaging (Android)
    - Apple Push Notification Service (iPhone / iPad)
    - Microsoft Push Notification Service (Windows Phone)
    - BlackBerry Push Service (BlackBerry)
    - Windows Push Notification Services (Windows 8)
    - etc.
Pushing data to smartphones provides users with instant access to desired updates as they happen, such as news and weather, sports scores, stock prices and other time-sensitive content. The push services provide an efficient way to quickly push timely information updates to many smartphones at once, in a centrally managed and controlled manner.
Generally, you can also be very selective in who you send information to, including individual customers or many customers (multicast).
This services minimizes the impact on the smartphones battery life. Instead of actively checking for new data, the applications can remain closed.
This processes offer an alternative to other less efficient methods, such as polling, where a device regularly polls an application server to see if new content is available.
The main differences between the services, refer to details as: the maxim payload length, the quality of service, queueing the messages or not, and the time limit for this, the way the messages are handled in the devices, etc.

GCM:




APNS:


MPNS:

General Information

Abstract
The aim of this project is to develop a server-side library in Haskell for sending push notifications to devices running different OS, such as Android, iOS, Windows Phone, BlackBerry, and so on.
The fact is that every company is developing Push Notification services, and these are very similar. Then, I want to find the fundamental concepts to construct a library which enable to configure the options for the different services and send messages easily.
When I say they are very similar, I refer to the fact that they all are asynchronous, best-effort services that offers third-party developers a channel to send data to apps from a cloud service in a power-efficient manner. The most popular are:
- Google Cloud Messaging (Android)
- Apple Push Notification Service (iPhone / iPad)
- Microsoft Push Notification Service (Windows Phone)
- BlackBerry Push Service (BlackBerry)
- Windows Push Notification Services (Windows 8)
- etc.
Once we have this libraries, I will investigate the possibility of mainting a "back and forth" communication between a server and mobile devices and I will develop a library to handle this.

Motivation and expected benefits
I think this idea would be very useful because it will allow all Haskell developers to open to a new world of mobile devices and to build useful programs/services that interact with them.
Pushing data to smartphones provides users with instant access to desired updates as they happen, such as news and weather, sports scores, stock prices and other time-sensitive content. The push services provide an efficient way to quickly push timely information updates to many smartphones at once, in a centrally managed and controlled manner.
Generally, you can also be very selective in who you send information to, including individual customers or many customers (multicast).
This services minimizes the impact on the smartphones battery life. Instead of actively checking for new data, the applications can remain closed. Once the data is delivered, the application can be launched in the background to process it as needed.
This processes offer an alternative to other less efficient methods, such as polling, where a device regularly polls an application server to see if new content is available.
The main differences between the services, refer to details as: the maxim payload length, the quality of service, queueing the messages or not, and the time limit for this, the way the messages are handled in the devices, etc.
As all the libraries to access to these services are developed in Java, Python and so on, I thought that it would be a good idea to offer an option to Haskell programmers. Taking advantage of the similarity of these services, I could develop a very adaptable library which fits the necessities for each one and at the same time offer an abstraction to the user.

Technical Considerations
In the developing of the APIs for the communication through Push Notifications, I will aim to develop a good abstraction and find the properties in common between the differents services in order to develope an customizable tool but at the same time with a common structure.
I want to let the user build messages and send these in a simple way following each protocol. Also, I will abstract the process of registering the devices in the server and let the user manage the different registrations behind a similar abstraccion.

To develop a “back and forth” comunication between a server and mobile devices, I will investigate the different possibilities of maintaining a state of the connection. It could be through the use of cookies stored by the clients or maintaining some extra information in the server which would enable it to identify the different connections and provide the appropiate services.